2023-07-14T11:24:51.513 INFO:root:teuthology version: 0.0.1.dev145+gaa6a80e 2023-07-14T11:24:51.514 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} --name rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435/orig.config.yaml 2023-07-14T11:24:51.557 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T11:24:51.647 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435 branch: wip-rishabh-2023Jul13 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} email: null first_in_suite: false job_id: '7337435' kernel: flavor: default kdb: true sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-rishabh-2023Jul13 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul13 sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul13 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi104.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0= smithi121.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_pool_perm teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: aa6a80e6cc655fff48452a7eca02070e61d074fc timestamp: 2023-07-14_10:26:42 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-14T11:24:51.648 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa; will attempt to use it 2023-07-14T11:24:51.648 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks 2023-07-14T11:24:51.649 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-14T11:24:51.650 INFO:teuthology.task.internal:Checking packages... 2023-07-14T11:24:51.678 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '3fc0089cc1082313a98853d48b892acdc1385b8b' 2023-07-14T11:24:51.678 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-14T11:24:51.678 INFO:teuthology.packaging:ref: None 2023-07-14T11:24:51.679 INFO:teuthology.packaging:tag: None 2023-07-14T11:24:51.679 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul13 2023-07-14T11:24:51.679 INFO:teuthology.packaging:sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:24:51.679 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-rishabh-2023Jul13 2023-07-14T11:24:51.995 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4757-g3fc0089c-1focal 2023-07-14T11:24:51.995 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-14T11:24:52.002 INFO:teuthology.task.internal:no buildpackages task found 2023-07-14T11:24:52.002 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-14T11:24:52.024 INFO:teuthology.task.internal:Saving configuration 2023-07-14T11:24:52.039 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-14T11:24:52.046 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T11:24:52.076 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 11:18:43.961934', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-07-14T11:24:52.103 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 11:18:43.964802', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-07-14T11:24:52.103 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-14T11:24:52.109 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-07-14T11:24:52.110 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-07-14T11:24:52.110 INFO:teuthology.run_tasks:Running task console_log... 2023-07-14T11:24:52.199 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f40403da430>, signals=[15]) 2023-07-14T11:24:52.201 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-14T11:24:52.207 INFO:teuthology.task.internal:Opening connections... 2023-07-14T11:24:52.208 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:24:52.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:24:52.261 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:24:52.263 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:24:52.313 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-14T11:24:52.320 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2023-07-14T11:24:52.345 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2023-07-14T11:24:52.346 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2023-07-14T11:24:52.393 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="Ubuntu" 2023-07-14T11:24:52.393 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-07-14T11:24:52.393 INFO:teuthology.orchestra.run.smithi104.stdout:ID=ubuntu 2023-07-14T11:24:52.394 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE=debian 2023-07-14T11:24:52.394 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-07-14T11:24:52.394 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="20.04" 2023-07-14T11:24:52.394 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://www.ubuntu.com/" 2023-07-14T11:24:52.394 INFO:teuthology.orchestra.run.smithi104.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-07-14T11:24:52.394 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-07-14T11:24:52.394 INFO:teuthology.orchestra.run.smithi104.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-07-14T11:24:52.395 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_CODENAME=focal 2023-07-14T11:24:52.395 INFO:teuthology.orchestra.run.smithi104.stdout:UBUNTU_CODENAME=focal 2023-07-14T11:24:52.396 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2023-07-14T11:24:52.420 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2023-07-14T11:24:52.427 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2023-07-14T11:24:52.427 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2023-07-14T11:24:52.475 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="Ubuntu" 2023-07-14T11:24:52.475 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-07-14T11:24:52.475 INFO:teuthology.orchestra.run.smithi121.stdout:ID=ubuntu 2023-07-14T11:24:52.475 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE=debian 2023-07-14T11:24:52.476 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-07-14T11:24:52.476 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="20.04" 2023-07-14T11:24:52.476 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://www.ubuntu.com/" 2023-07-14T11:24:52.476 INFO:teuthology.orchestra.run.smithi121.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-07-14T11:24:52.476 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-07-14T11:24:52.476 INFO:teuthology.orchestra.run.smithi121.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-07-14T11:24:52.476 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_CODENAME=focal 2023-07-14T11:24:52.477 INFO:teuthology.orchestra.run.smithi121.stdout:UBUNTU_CODENAME=focal 2023-07-14T11:24:52.477 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2023-07-14T11:24:52.504 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-14T11:24:52.514 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-14T11:24:52.543 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-14T11:24:52.544 DEBUG:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2023-07-14T11:24:52.546 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2023-07-14T11:24:52.553 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-14T11:24:52.566 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-14T11:24:52.566 DEBUG:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2023-07-14T11:24:52.593 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2023-07-14T11:24:52.612 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-14T11:24:52.677 INFO:teuthology.run_tasks:Running task kernel... 2023-07-14T11:24:52.692 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'} 2023-07-14T11:24:52.692 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'}}, timeout 300 2023-07-14T11:24:52.693 DEBUG:teuthology.orchestra.run.smithi104:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T11:24:52.694 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T11:24:52.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:24:52.699 DEBUG:teuthology.task.kernel:sha1 for mon.a is a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T11:24:52.699 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "a440c79eee0258ae07e4988d8690fe6187b8b7a3"... 2023-07-14T11:24:52.700 DEBUG:teuthology.orchestra.run.smithi104:> uname -r 2023-07-14T11:24:52.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:24:52.701 DEBUG:teuthology.task.kernel:sha1 for mon.b is a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T11:24:52.701 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "a440c79eee0258ae07e4988d8690fe6187b8b7a3"... 2023-07-14T11:24:52.701 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2023-07-14T11:24:52.707 INFO:teuthology.orchestra.run.smithi104.stdout:5.4.0-124-generic 2023-07-14T11:24:52.707 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-124-generic vs a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T11:24:52.708 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-14T11:24:52.708 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T11:24:52.711 INFO:teuthology.orchestra.run.smithi121.stdout:5.4.0-124-generic 2023-07-14T11:24:52.711 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-124-generic vs a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T11:24:52.711 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-14T11:24:52.711 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T11:24:53.027 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2023-07-14T11:24:53.028 DEBUG:teuthology.orchestra.run.smithi104:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-07-14T11:24:53.038 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2023-07-14T11:24:53.038 DEBUG:teuthology.orchestra.run.smithi121:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-07-14T11:24:53.046 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2023-07-14T11:24:53.052 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2023-07-14T11:24:53.163 INFO:teuthology.orchestra.run.smithi121.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-07-14T11:24:53.165 DEBUG:teuthology.orchestra.run.smithi121:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-07-14T11:24:53.166 INFO:teuthology.orchestra.run.smithi104.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-07-14T11:24:53.169 DEBUG:teuthology.orchestra.run.smithi104:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-07-14T11:24:53.198 DEBUG:teuthology.orchestra.run.smithi104:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-07-14T11:24:53.213 DEBUG:teuthology.orchestra.run.smithi104:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-07-14T11:24:53.262 DEBUG:teuthology.orchestra.run.smithi121:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-07-14T11:24:53.278 DEBUG:teuthology.orchestra.run.smithi121:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-07-14T11:37:19.916 INFO:teuthology.orchestra.run.smithi121.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-07-14T11:37:19.917 INFO:teuthology.orchestra.run.smithi121.stderr: * [new branch] main -> origin/main 2023-07-14T11:37:19.970 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20190312 -> 20190312 2023-07-14T11:37:19.970 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20190416 -> 20190416 2023-07-14T11:37:19.970 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20190514 -> 20190514 2023-07-14T11:37:19.971 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20190618 -> 20190618 2023-07-14T11:37:19.971 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20190717 -> 20190717 2023-07-14T11:37:19.971 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20190815 -> 20190815 2023-07-14T11:37:19.971 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20190923 -> 20190923 2023-07-14T11:37:19.971 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20191022 -> 20191022 2023-07-14T11:37:19.971 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20191215 -> 20191215 2023-07-14T11:37:19.971 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200122 -> 20200122 2023-07-14T11:37:19.972 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200316 -> 20200316 2023-07-14T11:37:19.972 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200421 -> 20200421 2023-07-14T11:37:19.975 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200519 -> 20200519 2023-07-14T11:37:19.975 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200619 -> 20200619 2023-07-14T11:37:19.975 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200721 -> 20200721 2023-07-14T11:37:19.975 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200817 -> 20200817 2023-07-14T11:37:19.975 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200918 -> 20200918 2023-07-14T11:37:19.976 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20201022 -> 20201022 2023-07-14T11:37:19.976 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20201118 -> 20201118 2023-07-14T11:37:19.976 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20201218 -> 20201218 2023-07-14T11:37:19.976 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20210208 -> 20210208 2023-07-14T11:37:19.976 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20210315 -> 20210315 2023-07-14T11:37:19.976 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20210511 -> 20210511 2023-07-14T11:37:19.977 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20210716 -> 20210716 2023-07-14T11:37:19.977 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20210818 -> 20210818 2023-07-14T11:37:19.977 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20210919 -> 20210919 2023-07-14T11:37:19.977 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20211027 -> 20211027 2023-07-14T11:37:19.977 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20211216 -> 20211216 2023-07-14T11:37:19.977 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220209 -> 20220209 2023-07-14T11:37:19.977 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220310 -> 20220310 2023-07-14T11:37:19.978 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220411 -> 20220411 2023-07-14T11:37:19.978 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220509 -> 20220509 2023-07-14T11:37:19.978 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220610 -> 20220610 2023-07-14T11:37:19.978 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220708 -> 20220708 2023-07-14T11:37:19.978 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220815 -> 20220815 2023-07-14T11:37:19.978 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220913 -> 20220913 2023-07-14T11:37:19.978 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20221012 -> 20221012 2023-07-14T11:37:19.979 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20221109 -> 20221109 2023-07-14T11:37:19.979 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20221214 -> 20221214 2023-07-14T11:37:19.979 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20230117 -> 20230117 2023-07-14T11:37:19.979 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20230210 -> 20230210 2023-07-14T11:37:19.979 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20230310 -> 20230310 2023-07-14T11:37:19.979 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20230404 -> 20230404 2023-07-14T11:37:19.979 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20230515 -> 20230515 2023-07-14T11:37:19.979 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20230625 -> 20230625 2023-07-14T11:37:23.187 INFO:teuthology.orchestra.run.smithi121.stderr:Updating files: 50% (1513/2977) Updating files: 51% (1519/2977) Updating files: 52% (1549/2977) Updating files: 53% (1578/2977) Updating files: 54% (1608/2977) Updating files: 55% (1638/2977) Updating files: 56% (1668/2977) Updating files: 57% (1697/2977) Updating files: 58% (1727/2977) Updating files: 59% (1757/2977) Updating files: 60% (1787/2977) Updating files: 61% (1816/2977) Updating files: 62% (1846/2977) Updating files: 63% (1876/2977) Updating files: 64% (1906/2977) Updating files: 65% (1936/2977) Updating files: 65% (1957/2977) Updating files: 66% (1965/2977) Updating files: 67% (1995/2977) Updating files: 68% (2025/2977) Updating files: 69% (2055/2977) Updating files: 70% (2084/2977) Updating files: 71% (2114/2977) Updating files: 72% (2144/2977) Updating files: 73% (2174/2977) Updating files: 74% (2203/2977) Updating files: 75% (2233/2977) Updating files: 76% (2263/2977) Updating files: 77% (2293/2977) Updating files: 78% (2323/2977) Updating files: 79% (2352/2977) Updating files: 80% (2382/2977) Updating files: 81% (2412/2977) Updating files: 82% (2442/2977) Updating files: 82% (2457/2977) Updating files: 83% (2471/2977) Updating files: 84% (2501/2977) Updating files: 85% (2531/2977) Updating files: 86% (2561/2977) Updating files: 87% (2590/2977) Updating files: 88% (2620/2977) Updating files: 89% (2650/2977) Updating files: 90% (2680/2977) Updating files: 91% (2710/2977) Updating files: 92% (2739/2977) Updating files: 93% (2769/2977) Updating files: 94% (2799/2977) Updating files: 95% (2829/2977) Updating files: 96% (2858/2977) Updating files: 97% (2888/2977) Updating files: 98% (2918/2977) Updating files: 99% (2948/2977) Updating files: 100% (2977/2977) Updating files: 100% (2977/2977), done. 2023-07-14T11:37:23.191 INFO:teuthology.orchestra.run.smithi121.stdout:HEAD is now at d3f66064 Partially revert "amdgpu: DMCUB updates for DCN 3.1.4 and 3.1.5" 2023-07-14T11:37:23.206 INFO:teuthology.task.kernel:Downloading kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on mon.b... 2023-07-14T11:37:23.208 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T11:37:23.525 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc1-ga440c79eee02/ 2023-07-14T11:37:23.525 DEBUG:teuthology.orchestra.run.smithi121:> rm -f /tmp/linux-image.deb && echo linux-image-6.5.0-rc1-ga440c79eee02_6.5.0-rc1-ga440c79eee02-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc1-ga440c79eee02/ --input-file=- 2023-07-14T11:37:23.528 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi121.front.sepia.ceph.com to complete... 2023-07-14T11:37:27.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14 11:37:27 URL:https://4.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc1-ga440c79eee02/linux-image-6.5.0-rc1-ga440c79eee02_6.5.0-rc1-ga440c79eee02-1_amd64.deb [38851590/38851590] -> "/tmp/linux-image.deb" [1] 2023-07-14T11:37:27.404 INFO:teuthology.orchestra.run.smithi121.stderr:FINISHED --2023-07-14 11:37:27-- 2023-07-14T11:37:27.404 INFO:teuthology.orchestra.run.smithi121.stderr:Total wall clock time: 3.8s 2023-07-14T11:37:27.404 INFO:teuthology.orchestra.run.smithi121.stderr:Downloaded: 1 files, 37M in 3.5s (10.7 MB/s) 2023-07-14T11:37:27.405 INFO:teuthology.task.kernel:Installing kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on mon.b... 2023-07-14T11:37:27.405 DEBUG:teuthology.orchestra.run.smithi121:> sudo dpkg -i /tmp/linux-image.deb 2023-07-14T11:37:27.713 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package linux-image-6.5.0-rc1-ga440c79eee02. 2023-07-14T11:37:29.508 INFO:teuthology.orchestra.run.smithi121.stdout:(Reading database ... 104373 files and directories currently installed.) 2023-07-14T11:37:29.514 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-07-14T11:37:29.667 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking linux-image-6.5.0-rc1-ga440c79eee02 (6.5.0-rc1-ga440c79eee02-1) ... 2023-07-14T11:37:31.934 INFO:teuthology.orchestra.run.smithi104.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-07-14T11:37:31.934 INFO:teuthology.orchestra.run.smithi104.stderr: * [new branch] main -> origin/main 2023-07-14T11:37:31.985 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20190312 -> 20190312 2023-07-14T11:37:31.985 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20190416 -> 20190416 2023-07-14T11:37:31.985 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20190514 -> 20190514 2023-07-14T11:37:31.985 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20190618 -> 20190618 2023-07-14T11:37:31.986 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20190717 -> 20190717 2023-07-14T11:37:31.986 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20190815 -> 20190815 2023-07-14T11:37:31.987 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20190923 -> 20190923 2023-07-14T11:37:31.988 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20191022 -> 20191022 2023-07-14T11:37:31.988 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20191215 -> 20191215 2023-07-14T11:37:31.988 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200122 -> 20200122 2023-07-14T11:37:31.988 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200316 -> 20200316 2023-07-14T11:37:31.988 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200421 -> 20200421 2023-07-14T11:37:31.988 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200519 -> 20200519 2023-07-14T11:37:31.988 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200619 -> 20200619 2023-07-14T11:37:31.988 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200721 -> 20200721 2023-07-14T11:37:31.989 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200817 -> 20200817 2023-07-14T11:37:31.989 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200918 -> 20200918 2023-07-14T11:37:31.989 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20201022 -> 20201022 2023-07-14T11:37:31.989 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20201118 -> 20201118 2023-07-14T11:37:31.989 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20201218 -> 20201218 2023-07-14T11:37:31.989 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20210208 -> 20210208 2023-07-14T11:37:31.989 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20210315 -> 20210315 2023-07-14T11:37:31.990 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20210511 -> 20210511 2023-07-14T11:37:31.990 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20210716 -> 20210716 2023-07-14T11:37:31.990 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20210818 -> 20210818 2023-07-14T11:37:31.990 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20210919 -> 20210919 2023-07-14T11:37:31.990 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20211027 -> 20211027 2023-07-14T11:37:31.991 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20211216 -> 20211216 2023-07-14T11:37:31.992 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220209 -> 20220209 2023-07-14T11:37:31.992 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220310 -> 20220310 2023-07-14T11:37:31.992 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220411 -> 20220411 2023-07-14T11:37:31.992 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220509 -> 20220509 2023-07-14T11:37:31.992 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220610 -> 20220610 2023-07-14T11:37:31.992 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220708 -> 20220708 2023-07-14T11:37:31.992 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220815 -> 20220815 2023-07-14T11:37:31.993 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220913 -> 20220913 2023-07-14T11:37:31.993 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20221012 -> 20221012 2023-07-14T11:37:31.993 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20221109 -> 20221109 2023-07-14T11:37:31.993 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20221214 -> 20221214 2023-07-14T11:37:31.993 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20230117 -> 20230117 2023-07-14T11:37:31.993 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20230210 -> 20230210 2023-07-14T11:37:31.993 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20230310 -> 20230310 2023-07-14T11:37:31.993 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20230404 -> 20230404 2023-07-14T11:37:31.994 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20230515 -> 20230515 2023-07-14T11:37:31.994 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20230625 -> 20230625 2023-07-14T11:37:32.555 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up linux-image-6.5.0-rc1-ga440c79eee02 (6.5.0-rc1-ga440c79eee02-1) ... 2023-07-14T11:37:32.705 INFO:teuthology.orchestra.run.smithi121.stderr:update-initramfs: Generating /boot/initrd.img-6.5.0-rc1-ga440c79eee02 2023-07-14T11:37:35.200 INFO:teuthology.orchestra.run.smithi104.stderr:Updating files: 51% (1521/2977) Updating files: 52% (1549/2977) Updating files: 53% (1578/2977) Updating files: 54% (1608/2977) Updating files: 55% (1638/2977) Updating files: 56% (1668/2977) Updating files: 57% (1697/2977) Updating files: 58% (1727/2977) Updating files: 59% (1757/2977) Updating files: 60% (1787/2977) Updating files: 61% (1816/2977) Updating files: 62% (1846/2977) Updating files: 63% (1876/2977) Updating files: 64% (1906/2977) Updating files: 65% (1936/2977) Updating files: 65% (1957/2977) Updating files: 66% (1965/2977) Updating files: 67% (1995/2977) Updating files: 68% (2025/2977) Updating files: 69% (2055/2977) Updating files: 70% (2084/2977) Updating files: 71% (2114/2977) Updating files: 72% (2144/2977) Updating files: 73% (2174/2977) Updating files: 74% (2203/2977) Updating files: 75% (2233/2977) Updating files: 76% (2263/2977) Updating files: 77% (2293/2977) Updating files: 78% (2323/2977) Updating files: 79% (2352/2977) Updating files: 80% (2382/2977) Updating files: 81% (2412/2977) Updating files: 82% (2442/2977) Updating files: 82% (2459/2977) Updating files: 83% (2471/2977) Updating files: 84% (2501/2977) Updating files: 85% (2531/2977) Updating files: 86% (2561/2977) Updating files: 87% (2590/2977) Updating files: 88% (2620/2977) Updating files: 89% (2650/2977) Updating files: 90% (2680/2977) Updating files: 91% (2710/2977) Updating files: 92% (2739/2977) Updating files: 93% (2769/2977) Updating files: 94% (2799/2977) Updating files: 95% (2829/2977) Updating files: 96% (2858/2977) Updating files: 97% (2888/2977) Updating files: 98% (2918/2977) Updating files: 99% (2948/2977) Updating files: 100% (2977/2977) Updating files: 100% (2977/2977), done. 2023-07-14T11:37:35.204 INFO:teuthology.orchestra.run.smithi104.stdout:HEAD is now at d3f66064 Partially revert "amdgpu: DMCUB updates for DCN 3.1.4 and 3.1.5" 2023-07-14T11:37:35.219 INFO:teuthology.task.kernel:Downloading kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on mon.a... 2023-07-14T11:37:35.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T11:37:35.495 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc1-ga440c79eee02/ 2023-07-14T11:37:35.496 DEBUG:teuthology.orchestra.run.smithi104:> rm -f /tmp/linux-image.deb && echo linux-image-6.5.0-rc1-ga440c79eee02_6.5.0-rc1-ga440c79eee02-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc1-ga440c79eee02/ --input-file=- 2023-07-14T11:37:35.498 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi104.front.sepia.ceph.com to complete... 2023-07-14T11:37:38.733 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14 11:37:38 URL:https://4.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc1-ga440c79eee02/linux-image-6.5.0-rc1-ga440c79eee02_6.5.0-rc1-ga440c79eee02-1_amd64.deb [38851590/38851590] -> "/tmp/linux-image.deb" [1] 2023-07-14T11:37:38.733 INFO:teuthology.orchestra.run.smithi104.stderr:FINISHED --2023-07-14 11:37:38-- 2023-07-14T11:37:38.733 INFO:teuthology.orchestra.run.smithi104.stderr:Total wall clock time: 3.2s 2023-07-14T11:37:38.733 INFO:teuthology.orchestra.run.smithi104.stderr:Downloaded: 1 files, 37M in 2.9s (12.6 MB/s) 2023-07-14T11:37:38.735 INFO:teuthology.task.kernel:Installing kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on mon.a... 2023-07-14T11:37:38.735 DEBUG:teuthology.orchestra.run.smithi104:> sudo dpkg -i /tmp/linux-image.deb 2023-07-14T11:37:39.061 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package linux-image-6.5.0-rc1-ga440c79eee02. 2023-07-14T11:37:40.811 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... 104373 files and directories currently installed.) 2023-07-14T11:37:40.815 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-07-14T11:37:40.945 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-image-6.5.0-rc1-ga440c79eee02 (6.5.0-rc1-ga440c79eee02-1) ... 2023-07-14T11:37:43.852 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-image-6.5.0-rc1-ga440c79eee02 (6.5.0-rc1-ga440c79eee02-1) ... 2023-07-14T11:37:43.977 INFO:teuthology.orchestra.run.smithi104.stderr:update-initramfs: Generating /boot/initrd.img-6.5.0-rc1-ga440c79eee02 2023-07-14T11:37:57.210 INFO:teuthology.orchestra.run.smithi121.stderr:Sourcing file `/etc/default/grub' 2023-07-14T11:37:57.222 INFO:teuthology.orchestra.run.smithi121.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-07-14T11:37:57.249 INFO:teuthology.orchestra.run.smithi121.stderr:Generating grub configuration file ... 2023-07-14T11:37:57.606 INFO:teuthology.orchestra.run.smithi121.stderr:Found linux image: /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-14T11:37:57.635 INFO:teuthology.orchestra.run.smithi121.stderr:Found initrd image: /boot/initrd.img-6.5.0-rc1-ga440c79eee02 2023-07-14T11:37:58.006 INFO:teuthology.orchestra.run.smithi121.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-07-14T11:37:58.020 INFO:teuthology.orchestra.run.smithi121.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-07-14T11:38:00.721 INFO:teuthology.orchestra.run.smithi121.stderr:done 2023-07-14T11:38:00.896 DEBUG:teuthology.orchestra.run.smithi121:> dpkg-deb -c /tmp/linux-image.deb 2023-07-14T11:38:00.905 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./ 2023-07-14T11:38:00.905 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./boot/ 2023-07-14T11:38:00.906 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 4380308 2023-07-13 12:25 ./boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:00.923 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 168433 2023-07-13 12:25 ./boot/config-6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:00.923 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11280128 2023-07-13 12:25 ./boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:00.948 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./etc/ 2023-07-14T11:38:00.949 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./etc/kernel/ 2023-07-14T11:38:00.949 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./etc/kernel/postinst.d/ 2023-07-14T11:38:00.949 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./etc/kernel/postrm.d/ 2023-07-14T11:38:00.949 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./etc/kernel/preinst.d/ 2023-07-14T11:38:00.950 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./etc/kernel/prerm.d/ 2023-07-14T11:38:00.950 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/ 2023-07-14T11:38:00.950 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/ 2023-07-14T11:38:00.950 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/ 2023-07-14T11:38:00.951 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/ 2023-07-14T11:38:00.951 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/ 2023-07-14T11:38:00.951 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/ 2023-07-14T11:38:00.951 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ 2023-07-14T11:38:00.951 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 490385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/aesni-intel.ko 2023-07-14T11:38:00.951 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-14T11:38:00.952 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-14T11:38:00.952 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-14T11:38:00.952 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 93113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-14T11:38:00.952 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 106729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-14T11:38:00.952 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 152241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-14T11:38:00.952 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-14T11:38:00.953 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-14T11:38:00.953 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 83889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-14T11:38:00.953 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-14T11:38:00.953 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-14T11:38:00.953 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-14T11:38:00.953 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-14T11:38:00.954 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-14T11:38:00.954 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-14T11:38:00.954 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 58585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-14T11:38:00.954 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-14T11:38:00.954 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-14T11:38:00.954 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-14T11:38:00.954 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/ 2023-07-14T11:38:00.955 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/ 2023-07-14T11:38:00.955 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/ 2023-07-14T11:38:00.955 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-14T11:38:00.955 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpuid.ko 2023-07-14T11:38:00.955 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/msr.ko 2023-07-14T11:38:00.955 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/ 2023-07-14T11:38:00.956 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 348417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-14T11:38:00.956 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 972641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-14T11:38:00.992 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 2273145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm.ko 2023-07-14T11:38:00.993 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ 2023-07-14T11:38:00.993 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/af_alg.ko 2023-07-14T11:38:00.993 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/algif_hash.ko 2023-07-14T11:38:00.993 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/algif_skcipher.ko 2023-07-14T11:38:00.993 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ansi_cprng.ko 2023-07-14T11:38:00.994 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/anubis.ko 2023-07-14T11:38:00.994 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/arc4.ko 2023-07-14T11:38:00.994 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/asymmetric_keys/ 2023-07-14T11:38:00.994 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-07-14T11:38:00.994 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/ 2023-07-14T11:38:00.994 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_memcpy.ko 2023-07-14T11:38:00.995 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_pq.ko 2023-07-14T11:38:00.995 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-14T11:38:00.995 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_tx.ko 2023-07-14T11:38:00.995 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_xor.ko 2023-07-14T11:38:00.995 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/raid6test.ko 2023-07-14T11:38:00.995 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authenc.ko 2023-07-14T11:38:00.995 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authencesn.ko 2023-07-14T11:38:00.996 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blake2b_generic.ko 2023-07-14T11:38:00.996 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_common.ko 2023-07-14T11:38:00.996 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_generic.ko 2023-07-14T11:38:00.996 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/camellia_generic.ko 2023-07-14T11:38:00.996 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast5_generic.ko 2023-07-14T11:38:00.996 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast6_generic.ko 2023-07-14T11:38:00.996 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast_common.ko 2023-07-14T11:38:00.997 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ccm.ko 2023-07-14T11:38:00.997 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cmac.ko 2023-07-14T11:38:00.997 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crc32_generic.ko 2023-07-14T11:38:00.997 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cryptd.ko 2023-07-14T11:38:00.997 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_engine.ko 2023-07-14T11:38:00.997 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_simd.ko 2023-07-14T11:38:00.998 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_user.ko 2023-07-14T11:38:00.998 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cts.ko 2023-07-14T11:38:00.998 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/des_generic.ko 2023-07-14T11:38:00.998 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/dh_generic.ko 2023-07-14T11:38:00.998 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/echainiv.ko 2023-07-14T11:38:00.998 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/essiv.ko 2023-07-14T11:38:00.998 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/fcrypt.ko 2023-07-14T11:38:00.999 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/khazad.ko 2023-07-14T11:38:00.999 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lrw.ko 2023-07-14T11:38:00.999 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4.ko 2023-07-14T11:38:00.999 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4hc.ko 2023-07-14T11:38:00.999 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/md4.ko 2023-07-14T11:38:00.999 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/michael_mic.ko 2023-07-14T11:38:00.999 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcbc.ko 2023-07-14T11:38:01.000 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcrypt.ko 2023-07-14T11:38:01.000 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/rmd160.ko 2023-07-14T11:38:01.000 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/seed.ko 2023-07-14T11:38:01.000 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/serpent_generic.ko 2023-07-14T11:38:01.001 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 149193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tcrypt.ko 2023-07-14T11:38:01.001 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tea.ko 2023-07-14T11:38:01.001 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_common.ko 2023-07-14T11:38:01.002 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_generic.ko 2023-07-14T11:38:01.002 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/vmac.ko 2023-07-14T11:38:01.002 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/wp512.ko 2023-07-14T11:38:01.003 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xcbc.ko 2023-07-14T11:38:01.003 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xor.ko 2023-07-14T11:38:01.003 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xts.ko 2023-07-14T11:38:01.004 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xxhash_generic.ko 2023-07-14T11:38:01.004 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ 2023-07-14T11:38:01.004 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ 2023-07-14T11:38:01.005 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_extlog.ko 2023-07-14T11:38:01.005 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-14T11:38:01.005 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_pad.ko 2023-07-14T11:38:01.006 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/apei/ 2023-07-14T11:38:01.006 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/apei/einj.ko 2023-07-14T11:38:01.006 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ec_sys.ko 2023-07-14T11:38:01.007 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/platform_profile.ko 2023-07-14T11:38:01.007 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbs.ko 2023-07-14T11:38:01.007 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbshc.ko 2023-07-14T11:38:01.007 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 100401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/video.ko 2023-07-14T11:38:01.008 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ 2023-07-14T11:38:01.008 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/acard-ahci.ko 2023-07-14T11:38:01.008 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 88625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci.ko 2023-07-14T11:38:01.009 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci_platform.ko 2023-07-14T11:38:01.009 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 87841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci.ko 2023-07-14T11:38:01.009 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci_platform.ko 2023-07-14T11:38:01.009 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_acpi.ko 2023-07-14T11:38:01.010 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ali.ko 2023-07-14T11:38:01.010 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_amd.ko 2023-07-14T11:38:01.010 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_artop.ko 2023-07-14T11:38:01.010 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atiixp.ko 2023-07-14T11:38:01.011 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atp867x.ko 2023-07-14T11:38:01.011 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd640.ko 2023-07-14T11:38:01.012 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd64x.ko 2023-07-14T11:38:01.012 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cypress.ko 2023-07-14T11:38:01.012 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_efar.ko 2023-07-14T11:38:01.012 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt366.ko 2023-07-14T11:38:01.012 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt37x.ko 2023-07-14T11:38:01.013 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-14T11:38:01.013 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-14T11:38:01.013 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it8213.ko 2023-07-14T11:38:01.013 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it821x.ko 2023-07-14T11:38:01.013 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_jmicron.ko 2023-07-14T11:38:01.013 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_legacy.ko 2023-07-14T11:38:01.014 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_marvell.ko 2023-07-14T11:38:01.014 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_mpiix.ko 2023-07-14T11:38:01.014 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_netcell.ko 2023-07-14T11:38:01.014 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ninja32.ko 2023-07-14T11:38:01.014 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87410.ko 2023-07-14T11:38:01.014 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87415.ko 2023-07-14T11:38:01.014 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_oldpiix.ko 2023-07-14T11:38:01.015 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_opti.ko 2023-07-14T11:38:01.015 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_optidma.ko 2023-07-14T11:38:01.015 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pcmcia.ko 2023-07-14T11:38:01.015 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-14T11:38:01.015 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-14T11:38:01.015 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_piccolo.ko 2023-07-14T11:38:01.015 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_radisys.ko 2023-07-14T11:38:01.016 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rdc.ko 2023-07-14T11:38:01.016 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rz1000.ko 2023-07-14T11:38:01.016 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sch.ko 2023-07-14T11:38:01.016 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_serverworks.ko 2023-07-14T11:38:01.016 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sil680.ko 2023-07-14T11:38:01.017 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sl82c105.ko 2023-07-14T11:38:01.017 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_triflex.ko 2023-07-14T11:38:01.017 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_via.ko 2023-07-14T11:38:01.017 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pdc_adma.ko 2023-07-14T11:38:01.017 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_inic162x.ko 2023-07-14T11:38:01.017 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 83225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_mv.ko 2023-07-14T11:38:01.017 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 70537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_nv.ko 2023-07-14T11:38:01.018 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37369 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_promise.ko 2023-07-14T11:38:01.018 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_qstor.ko 2023-07-14T11:38:01.018 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil.ko 2023-07-14T11:38:01.018 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil24.ko 2023-07-14T11:38:01.018 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sis.ko 2023-07-14T11:38:01.018 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_svw.ko 2023-07-14T11:38:01.018 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sx4.ko 2023-07-14T11:38:01.019 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_uli.ko 2023-07-14T11:38:01.019 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_via.ko 2023-07-14T11:38:01.019 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_vsc.ko 2023-07-14T11:38:01.019 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ 2023-07-14T11:38:01.019 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ks0108.ko 2023-07-14T11:38:01.019 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/ 2023-07-14T11:38:01.019 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 124457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/bcma.ko 2023-07-14T11:38:01.020 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/ 2023-07-14T11:38:01.020 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/ 2023-07-14T11:38:01.020 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 113705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/aoe.ko 2023-07-14T11:38:01.020 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/ 2023-07-14T11:38:01.020 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 968433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/drbd.ko 2023-07-14T11:38:01.021 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 173201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/floppy.ko 2023-07-14T11:38:01.021 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/ 2023-07-14T11:38:01.021 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 94297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-14T11:38:01.021 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 115345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/nbd.ko 2023-07-14T11:38:01.021 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/ 2023-07-14T11:38:01.021 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 110361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/null_blk.ko 2023-07-14T11:38:01.021 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 98225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/pktcdvd.ko 2023-07-14T11:38:01.022 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 257529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/rbd.ko 2023-07-14T11:38:01.022 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/ 2023-07-14T11:38:01.022 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 98129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-14T11:38:01.022 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/ 2023-07-14T11:38:01.022 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/zram.ko 2023-07-14T11:38:01.022 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ 2023-07-14T11:38:01.022 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/applicom.ko 2023-07-14T11:38:01.023 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hangcheck-timer.ko 2023-07-14T11:38:01.023 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/ 2023-07-14T11:38:01.023 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-14T11:38:01.023 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-14T11:38:01.023 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-14T11:38:01.023 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/via-rng.ko 2023-07-14T11:38:01.023 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15633 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-14T11:38:01.024 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ 2023-07-14T11:38:01.024 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-14T11:38:01.024 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 150665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-14T11:38:01.024 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-14T11:38:01.024 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 139057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-14T11:38:01.024 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 70833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-14T11:38:01.024 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-14T11:38:01.025 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/lp.ko 2023-07-14T11:38:01.025 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/ 2023-07-14T11:38:01.025 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 84121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/mwave.ko 2023-07-14T11:38:01.025 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/nvram.ko 2023-07-14T11:38:01.025 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ppdev.ko 2023-07-14T11:38:01.025 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tlclk.ko 2023-07-14T11:38:01.025 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/ 2023-07-14T11:38:01.026 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-14T11:38:01.026 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-07-14T11:38:01.026 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-07-14T11:38:01.026 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-07-14T11:38:01.026 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-14T11:38:01.026 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-14T11:38:01.026 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-07-14T11:38:01.027 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/xillybus/ 2023-07-14T11:38:01.027 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/xillybus/xillybus_class.ko 2023-07-14T11:38:01.027 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/xillybus/xillybus_core.ko 2023-07-14T11:38:01.027 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-07-14T11:38:01.027 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/clk/ 2023-07-14T11:38:01.027 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/clk/clk-palmas.ko 2023-07-14T11:38:01.028 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/clk/clk-si5351.ko 2023-07-14T11:38:01.028 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/clk/clk-twl6040.ko 2023-07-14T11:38:01.028 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/clk/clk-wm831x.ko 2023-07-14T11:38:01.028 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/ 2023-07-14T11:38:01.028 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-14T11:38:01.028 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-14T11:38:01.028 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-14T11:38:01.029 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ 2023-07-14T11:38:01.029 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ 2023-07-14T11:38:01.029 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 70785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-14T11:38:01.029 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 234473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp.ko 2023-07-14T11:38:01.029 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/ 2023-07-14T11:38:01.029 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/ 2023-07-14T11:38:01.030 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/ 2023-07-14T11:38:01.030 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 408489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-14T11:38:01.030 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-07-14T11:38:01.030 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-14T11:38:01.030 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-aes.ko 2023-07-14T11:38:01.030 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-sha.ko 2023-07-14T11:38:01.030 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/ 2023-07-14T11:38:01.031 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 68937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-14T11:38:01.031 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/ 2023-07-14T11:38:01.031 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/dca.ko 2023-07-14T11:38:01.031 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ 2023-07-14T11:38:01.031 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/ 2023-07-14T11:38:01.031 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-14T11:38:01.032 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ 2023-07-14T11:38:01.032 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 122641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-14T11:38:01.032 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ 2023-07-14T11:38:01.032 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 80273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/amd64_edac.ko 2023-07-14T11:38:01.032 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/e752x_edac.ko 2023-07-14T11:38:01.032 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 63617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/edac_mce_amd.ko 2023-07-14T11:38:01.032 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3000_edac.ko 2023-07-14T11:38:01.033 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3200_edac.ko 2023-07-14T11:38:01.033 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5100_edac.ko 2023-07-14T11:38:01.033 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5400_edac.ko 2023-07-14T11:38:01.033 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7300_edac.ko 2023-07-14T11:38:01.033 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 52025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7core_edac.ko 2023-07-14T11:38:01.033 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i82975x_edac.ko 2023-07-14T11:38:01.033 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ie31200_edac.ko 2023-07-14T11:38:01.034 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 56809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/sb_edac.ko 2023-07-14T11:38:01.034 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/x38_edac.ko 2023-07-14T11:38:01.034 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/ 2023-07-14T11:38:01.034 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/extcon-gpio.ko 2023-07-14T11:38:01.034 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/extcon-max14577.ko 2023-07-14T11:38:01.034 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/extcon-max77693.ko 2023-07-14T11:38:01.034 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/extcon-max8997.ko 2023-07-14T11:38:01.035 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/extcon-palmas.ko 2023-07-14T11:38:01.035 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/extcon-rt8973a.ko 2023-07-14T11:38:01.035 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/extcon-sm5502.ko 2023-07-14T11:38:01.035 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/ 2023-07-14T11:38:01.035 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/dmi-sysfs.ko 2023-07-14T11:38:01.035 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/efi/ 2023-07-14T11:38:01.035 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/efi/efi-pstore.ko 2023-07-14T11:38:01.036 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-14T11:38:01.036 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/ 2023-07-14T11:38:01.036 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-adp5520.ko 2023-07-14T11:38:01.036 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-amd8111.ko 2023-07-14T11:38:01.036 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-arizona.ko 2023-07-14T11:38:01.036 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-da9052.ko 2023-07-14T11:38:01.036 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-da9055.ko 2023-07-14T11:38:01.037 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-dln2.ko 2023-07-14T11:38:01.037 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 77849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-f7188x.ko 2023-07-14T11:38:01.037 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-generic.ko 2023-07-14T11:38:01.037 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ich.ko 2023-07-14T11:38:01.037 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-07-14T11:38:01.037 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-kempld.ko 2023-07-14T11:38:01.037 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-lp3943.ko 2023-07-14T11:38:01.038 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-max7300.ko 2023-07-14T11:38:01.038 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-max730x.ko 2023-07-14T11:38:01.038 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-max732x.ko 2023-07-14T11:38:01.038 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-07-14T11:38:01.038 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-pca953x.ko 2023-07-14T11:38:01.038 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-pcf857x.ko 2023-07-14T11:38:01.038 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-rdc321x.ko 2023-07-14T11:38:01.039 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-sch.ko 2023-07-14T11:38:01.039 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-sch311x.ko 2023-07-14T11:38:01.039 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-tps65912.ko 2023-07-14T11:38:01.039 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-twl4030.ko 2023-07-14T11:38:01.039 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-twl6040.ko 2023-07-14T11:38:01.039 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-14T11:38:01.039 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-vx855.ko 2023-07-14T11:38:01.040 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-wm831x.ko 2023-07-14T11:38:01.040 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-wm8350.ko 2023-07-14T11:38:01.040 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-wm8994.ko 2023-07-14T11:38:01.040 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/ 2023-07-14T11:38:01.040 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-a4tech.ko 2023-07-14T11:38:01.040 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-apple.ko 2023-07-14T11:38:01.041 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-appleir.ko 2023-07-14T11:38:01.041 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-aureal.ko 2023-07-14T11:38:01.041 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-axff.ko 2023-07-14T11:38:01.041 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-belkin.ko 2023-07-14T11:38:01.041 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-cherry.ko 2023-07-14T11:38:01.041 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-chicony.ko 2023-07-14T11:38:01.041 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-cp2112.ko 2023-07-14T11:38:01.041 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-cypress.ko 2023-07-14T11:38:01.042 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-dr.ko 2023-07-14T11:38:01.042 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elecom.ko 2023-07-14T11:38:01.042 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elo.ko 2023-07-14T11:38:01.042 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-emsff.ko 2023-07-14T11:38:01.042 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ezkey.ko 2023-07-14T11:38:01.042 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gaff.ko 2023-07-14T11:38:01.043 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-generic.ko 2023-07-14T11:38:01.043 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gt683r.ko 2023-07-14T11:38:01.043 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gyration.ko 2023-07-14T11:38:01.043 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-14T11:38:01.043 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-14T11:38:01.043 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtekff.ko 2023-07-14T11:38:01.043 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-hyperv.ko 2023-07-14T11:38:01.044 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-icade.ko 2023-07-14T11:38:01.044 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kensington.ko 2023-07-14T11:38:01.044 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-keytouch.ko 2023-07-14T11:38:01.044 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kye.ko 2023-07-14T11:38:01.044 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lcpower.ko 2023-07-14T11:38:01.044 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-led.ko 2023-07-14T11:38:01.044 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lenovo.ko 2023-07-14T11:38:01.045 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lg-g15.ko 2023-07-14T11:38:01.045 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-14T11:38:01.045 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 100897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-14T11:38:01.045 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 80993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech.ko 2023-07-14T11:38:01.045 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-magicmouse.ko 2023-07-14T11:38:01.045 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-microsoft.ko 2023-07-14T11:38:01.045 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-monterey.ko 2023-07-14T11:38:01.046 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-multitouch.ko 2023-07-14T11:38:01.046 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ntrig.ko 2023-07-14T11:38:01.046 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ortek.ko 2023-07-14T11:38:01.046 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-penmount.ko 2023-07-14T11:38:01.047 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-petalynx.ko 2023-07-14T11:38:01.047 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 72937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-picolcd.ko 2023-07-14T11:38:01.047 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-pl.ko 2023-07-14T11:38:01.047 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-plantronics.ko 2023-07-14T11:38:01.047 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-primax.ko 2023-07-14T11:38:01.047 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-rmi.ko 2023-07-14T11:38:01.047 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-14T11:38:01.048 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-common.ko 2023-07-14T11:38:01.048 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-14T11:38:01.048 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-14T11:38:01.048 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-14T11:38:01.048 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-14T11:38:01.048 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-14T11:38:01.049 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-14T11:38:01.049 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-14T11:38:01.049 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-14T11:38:01.049 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-14T11:38:01.050 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat.ko 2023-07-14T11:38:01.050 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-saitek.ko 2023-07-14T11:38:01.050 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-samsung.ko 2023-07-14T11:38:01.050 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-14T11:38:01.050 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sjoy.ko 2023-07-14T11:38:01.050 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sony.ko 2023-07-14T11:38:01.050 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-speedlink.ko 2023-07-14T11:38:01.051 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-steelseries.ko 2023-07-14T11:38:01.051 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sunplus.ko 2023-07-14T11:38:01.051 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-14T11:38:01.051 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tivo.ko 2023-07-14T11:38:01.051 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tmff.ko 2023-07-14T11:38:01.051 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-topseed.ko 2023-07-14T11:38:01.051 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-twinhan.ko 2023-07-14T11:38:01.052 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 71457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-uclogic.ko 2023-07-14T11:38:01.052 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-waltop.ko 2023-07-14T11:38:01.052 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 143585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-wiimote.ko 2023-07-14T11:38:01.052 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-xinmo.ko 2023-07-14T11:38:01.052 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zpff.ko 2023-07-14T11:38:01.052 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zydacron.ko 2023-07-14T11:38:01.052 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 267713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid.ko 2023-07-14T11:38:01.053 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/uhid.ko 2023-07-14T11:38:01.053 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/usbhid/ 2023-07-14T11:38:01.053 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 125721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/usbhid/usbhid.ko 2023-07-14T11:38:01.053 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/usbhid/usbkbd.ko 2023-07-14T11:38:01.053 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/usbhid/usbmouse.ko 2023-07-14T11:38:01.053 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 217401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/wacom.ko 2023-07-14T11:38:01.053 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/ 2023-07-14T11:38:01.054 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 63497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_balloon.ko 2023-07-14T11:38:01.054 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 89609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_utils.ko 2023-07-14T11:38:01.054 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 301449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_vmbus.ko 2023-07-14T11:38:01.054 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ 2023-07-14T11:38:01.054 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru.ko 2023-07-14T11:38:01.054 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 61393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru3.ko 2023-07-14T11:38:01.054 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-14T11:38:01.054 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7414.ko 2023-07-14T11:38:01.055 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7418.ko 2023-07-14T11:38:01.055 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adc128d818.ko 2023-07-14T11:38:01.055 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1025.ko 2023-07-14T11:38:01.055 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 97977 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1026.ko 2023-07-14T11:38:01.055 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1029.ko 2023-07-14T11:38:01.055 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 45553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1031.ko 2023-07-14T11:38:01.055 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm9240.ko 2023-07-14T11:38:01.056 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ads7828.ko 2023-07-14T11:38:01.056 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7410.ko 2023-07-14T11:38:01.056 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7411.ko 2023-07-14T11:38:01.056 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 74433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7462.ko 2023-07-14T11:38:01.056 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7470.ko 2023-07-14T11:38:01.056 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7475.ko 2023-07-14T11:38:01.056 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7x10.ko 2023-07-14T11:38:01.056 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/amc6821.ko 2023-07-14T11:38:01.057 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 55265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/applesmc.ko 2023-07-14T11:38:01.057 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asb100.ko 2023-07-14T11:38:01.057 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 58617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asc7621.ko 2023-07-14T11:38:01.057 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-14T11:38:01.057 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/atxp1.ko 2023-07-14T11:38:01.057 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/coretemp.ko 2023-07-14T11:38:01.057 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/da9052-hwmon.ko 2023-07-14T11:38:01.058 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/da9055-hwmon.ko 2023-07-14T11:38:01.058 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-14T11:38:01.058 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 99601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dme1737.ko 2023-07-14T11:38:01.058 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds1621.ko 2023-07-14T11:38:01.058 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds620.ko 2023-07-14T11:38:01.058 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc1403.ko 2023-07-14T11:38:01.058 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc2103.ko 2023-07-14T11:38:01.058 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc6w201.ko 2023-07-14T11:38:01.059 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71805f.ko 2023-07-14T11:38:01.059 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 94457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71882fg.ko 2023-07-14T11:38:01.059 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f75375s.ko 2023-07-14T11:38:01.059 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fam15h_power.ko 2023-07-14T11:38:01.059 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 55433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fschmd.ko 2023-07-14T11:38:01.059 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g760a.ko 2023-07-14T11:38:01.059 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g762.ko 2023-07-14T11:38:01.059 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl518sm.ko 2023-07-14T11:38:01.060 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 45529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl520sm.ko 2023-07-14T11:38:01.060 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hih6130.ko 2023-07-14T11:38:01.060 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-14T11:38:01.060 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5500_temp.ko 2023-07-14T11:38:01.060 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5k_amb.ko 2023-07-14T11:38:01.060 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmaem.ko 2023-07-14T11:38:01.060 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmpex.ko 2023-07-14T11:38:01.061 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina209.ko 2023-07-14T11:38:01.061 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina2xx.ko 2023-07-14T11:38:01.061 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 129217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/it87.ko 2023-07-14T11:38:01.061 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/jc42.ko 2023-07-14T11:38:01.061 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k10temp.ko 2023-07-14T11:38:01.061 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k8temp.ko 2023-07-14T11:38:01.061 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lineage-pem.ko 2023-07-14T11:38:01.061 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm63.ko 2023-07-14T11:38:01.062 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm73.ko 2023-07-14T11:38:01.062 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm75.ko 2023-07-14T11:38:01.062 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm77.ko 2023-07-14T11:38:01.062 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm78.ko 2023-07-14T11:38:01.062 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm80.ko 2023-07-14T11:38:01.062 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm83.ko 2023-07-14T11:38:01.062 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 72833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm85.ko 2023-07-14T11:38:01.063 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm87.ko 2023-07-14T11:38:01.063 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm90.ko 2023-07-14T11:38:01.063 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm92.ko 2023-07-14T11:38:01.063 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 108841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm93.ko 2023-07-14T11:38:01.063 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95234.ko 2023-07-14T11:38:01.063 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95241.ko 2023-07-14T11:38:01.063 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95245.ko 2023-07-14T11:38:01.063 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc2945.ko 2023-07-14T11:38:01.064 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4151.ko 2023-07-14T11:38:01.064 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4215.ko 2023-07-14T11:38:01.064 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4222.ko 2023-07-14T11:38:01.064 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4245.ko 2023-07-14T11:38:01.064 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4260.ko 2023-07-14T11:38:01.064 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4261.ko 2023-07-14T11:38:01.064 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max16065.ko 2023-07-14T11:38:01.065 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1619.ko 2023-07-14T11:38:01.065 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1668.ko 2023-07-14T11:38:01.065 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max197.ko 2023-07-14T11:38:01.065 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6639.ko 2023-07-14T11:38:01.065 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6650.ko 2023-07-14T11:38:01.065 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6697.ko 2023-07-14T11:38:01.065 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mc13783-adc.ko 2023-07-14T11:38:01.065 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mcp3021.ko 2023-07-14T11:38:01.066 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-07-14T11:38:01.066 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6683.ko 2023-07-14T11:38:01.066 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 116737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775-core.ko 2023-07-14T11:38:01.066 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775.ko 2023-07-14T11:38:01.066 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct7802.ko 2023-07-14T11:38:01.066 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 82337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87360.ko 2023-07-14T11:38:01.066 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87427.ko 2023-07-14T11:38:01.067 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pcf8591.ko 2023-07-14T11:38:01.067 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ 2023-07-14T11:38:01.067 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-14T11:38:01.067 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-14T11:38:01.067 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-14T11:38:01.067 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-14T11:38:01.067 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-14T11:38:01.067 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-14T11:38:01.068 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-14T11:38:01.068 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 81841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-14T11:38:01.068 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-14T11:38:01.068 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-14T11:38:01.068 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-14T11:38:01.068 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-14T11:38:01.068 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/powr1220.ko 2023-07-14T11:38:01.068 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5627.ko 2023-07-14T11:38:01.069 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5636.ko 2023-07-14T11:38:01.069 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-14T11:38:01.069 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht15.ko 2023-07-14T11:38:01.069 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht21.ko 2023-07-14T11:38:01.069 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/shtc1.ko 2023-07-14T11:38:01.069 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sis5595.ko 2023-07-14T11:38:01.069 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smm665.ko 2023-07-14T11:38:01.070 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47b397.ko 2023-07-14T11:38:01.070 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m1.ko 2023-07-14T11:38:01.070 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m192.ko 2023-07-14T11:38:01.070 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/thmc50.ko 2023-07-14T11:38:01.070 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp102.ko 2023-07-14T11:38:01.070 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp103.ko 2023-07-14T11:38:01.070 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp401.ko 2023-07-14T11:38:01.070 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp421.ko 2023-07-14T11:38:01.071 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via-cputemp.ko 2023-07-14T11:38:01.071 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via686a.ko 2023-07-14T11:38:01.071 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 52457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt1211.ko 2023-07-14T11:38:01.071 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt8231.ko 2023-07-14T11:38:01.071 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627ehf.ko 2023-07-14T11:38:01.071 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627hf.ko 2023-07-14T11:38:01.071 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 71345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83781d.ko 2023-07-14T11:38:01.071 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 63289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83791d.ko 2023-07-14T11:38:01.072 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 71321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83792d.ko 2023-07-14T11:38:01.072 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 101457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83793.ko 2023-07-14T11:38:01.072 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 119505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83795.ko 2023-07-14T11:38:01.072 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l785ts.ko 2023-07-14T11:38:01.072 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l786ng.ko 2023-07-14T11:38:01.072 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-07-14T11:38:01.072 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-07-14T11:38:01.073 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/ 2023-07-14T11:38:01.073 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/ 2023-07-14T11:38:01.073 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-14T11:38:01.073 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-14T11:38:01.073 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/ 2023-07-14T11:38:01.073 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-07-14T11:38:01.073 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-07-14T11:38:01.073 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-07-14T11:38:01.074 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-14T11:38:01.074 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-14T11:38:01.074 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-14T11:38:01.074 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-07-14T11:38:01.074 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 6929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-14T11:38:01.074 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-07-14T11:38:01.074 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-14T11:38:01.074 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-14T11:38:01.075 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-14T11:38:01.075 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-14T11:38:01.075 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-07-14T11:38:01.075 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-07-14T11:38:01.075 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 62729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-14T11:38:01.075 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-14T11:38:01.075 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-14T11:38:01.075 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17369 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-07-14T11:38:01.076 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-14T11:38:01.076 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-14T11:38:01.076 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-07-14T11:38:01.076 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-14T11:38:01.076 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-14T11:38:01.076 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-14T11:38:01.077 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-07-14T11:38:01.077 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-14T11:38:01.077 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-14T11:38:01.077 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-07-14T11:38:01.077 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-07-14T11:38:01.077 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-14T11:38:01.077 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-07-14T11:38:01.077 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-14T11:38:01.078 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-14T11:38:01.078 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-14T11:38:01.078 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-14T11:38:01.078 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-07-14T11:38:01.078 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-mux.ko 2023-07-14T11:38:01.078 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-smbus.ko 2023-07-14T11:38:01.078 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-stub.ko 2023-07-14T11:38:01.078 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/muxes/ 2023-07-14T11:38:01.079 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-07-14T11:38:01.079 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-07-14T11:38:01.079 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-07-14T11:38:01.079 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ 2023-07-14T11:38:01.079 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ 2023-07-14T11:38:01.079 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 273641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-14T11:38:01.079 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 876913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_core.ko 2023-07-14T11:38:01.080 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 69321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-14T11:38:01.080 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 324145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-14T11:38:01.080 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 97961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-14T11:38:01.080 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 279321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-14T11:38:01.080 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-14T11:38:01.080 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/ 2023-07-14T11:38:01.080 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/ 2023-07-14T11:38:01.081 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 492561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-14T11:38:01.081 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/ 2023-07-14T11:38:01.081 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 419217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-14T11:38:01.081 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/ 2023-07-14T11:38:01.081 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 739705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-14T11:38:01.138 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ 2023-07-14T11:38:01.139 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 296649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-14T11:38:01.139 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/ocrdma/ 2023-07-14T11:38:01.139 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 153681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-07-14T11:38:01.139 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/usnic/ 2023-07-14T11:38:01.139 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 169017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-07-14T11:38:01.139 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ 2023-07-14T11:38:01.140 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ 2023-07-14T11:38:01.140 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 263177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-14T11:38:01.140 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ 2023-07-14T11:38:01.140 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 100225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-14T11:38:01.140 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ 2023-07-14T11:38:01.140 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 111993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-14T11:38:01.141 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ 2023-07-14T11:38:01.141 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 129081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-14T11:38:01.141 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ 2023-07-14T11:38:01.141 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 129553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-14T11:38:01.141 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/ 2023-07-14T11:38:01.141 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/evbug.ko 2023-07-14T11:38:01.141 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/ff-memless.ko 2023-07-14T11:38:01.142 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ 2023-07-14T11:38:01.142 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-14T11:38:01.142 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-14T11:38:01.142 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/gameport.ko 2023-07-14T11:38:01.142 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/lightning.ko 2023-07-14T11:38:01.142 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ns558.ko 2023-07-14T11:38:01.142 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/joydev.ko 2023-07-14T11:38:01.142 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/ 2023-07-14T11:38:01.143 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-07-14T11:38:01.143 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-07-14T11:38:01.143 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-07-14T11:38:01.143 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-07-14T11:38:01.143 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-14T11:38:01.143 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-07-14T11:38:01.143 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/lkkbd.ko 2023-07-14T11:38:01.143 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/lm8323.ko 2023-07-14T11:38:01.144 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/lm8333.ko 2023-07-14T11:38:01.144 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-07-14T11:38:01.144 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-07-14T11:38:01.144 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-07-14T11:38:01.144 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-07-14T11:38:01.145 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/newtonkbd.ko 2023-07-14T11:38:01.145 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-07-14T11:38:01.145 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/qt1070.ko 2023-07-14T11:38:01.145 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/qt2160.ko 2023-07-14T11:38:01.145 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-07-14T11:38:01.145 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/stowaway.ko 2023-07-14T11:38:01.145 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/sunkbd.ko 2023-07-14T11:38:01.145 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-07-14T11:38:01.146 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-07-14T11:38:01.146 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-07-14T11:38:01.146 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/xtkbd.ko 2023-07-14T11:38:01.146 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/matrix-keymap.ko 2023-07-14T11:38:01.146 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/ 2023-07-14T11:38:01.146 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/appletouch.ko 2023-07-14T11:38:01.146 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/bcm5974.ko 2023-07-14T11:38:01.146 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 123681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/cyapatp.ko 2023-07-14T11:38:01.147 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 89449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-14T11:38:01.147 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/gpio_mouse.ko 2023-07-14T11:38:01.147 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 335849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/psmouse.ko 2023-07-14T11:38:01.147 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/sermouse.ko 2023-07-14T11:38:01.147 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-14T11:38:01.147 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-14T11:38:01.148 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-14T11:38:01.148 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/ 2023-07-14T11:38:01.148 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 109849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-14T11:38:01.148 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/ 2023-07-14T11:38:01.148 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/altera_ps2.ko 2023-07-14T11:38:01.148 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/arc_ps2.ko 2023-07-14T11:38:01.148 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/ct82c710.ko 2023-07-14T11:38:01.148 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-14T11:38:01.149 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/parkbd.ko 2023-07-14T11:38:01.149 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/pcips2.ko 2023-07-14T11:38:01.149 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/ps2mult.ko 2023-07-14T11:38:01.149 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serio_raw.ko 2023-07-14T11:38:01.149 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17977 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serport.ko 2023-07-14T11:38:01.149 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/sparse-keymap.ko 2023-07-14T11:38:01.149 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/ 2023-07-14T11:38:01.149 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/ 2023-07-14T11:38:01.150 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-14T11:38:01.150 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/ 2023-07-14T11:38:01.150 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-88pm860x.ko 2023-07-14T11:38:01.150 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-adp5520.ko 2023-07-14T11:38:01.150 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-bd2802.ko 2023-07-14T11:38:01.150 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-blinkm.ko 2023-07-14T11:38:01.150 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-da903x.ko 2023-07-14T11:38:01.151 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-da9052.ko 2023-07-14T11:38:01.151 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-gpio.ko 2023-07-14T11:38:01.151 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3530.ko 2023-07-14T11:38:01.151 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3533.ko 2023-07-14T11:38:01.151 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm355x.ko 2023-07-14T11:38:01.151 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3642.ko 2023-07-14T11:38:01.151 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp3944.ko 2023-07-14T11:38:01.152 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp8788.ko 2023-07-14T11:38:01.152 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-max8997.ko 2023-07-14T11:38:01.152 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-mc13783.ko 2023-07-14T11:38:01.152 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-menf21bmc.ko 2023-07-14T11:38:01.152 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-pca9532.ko 2023-07-14T11:38:01.152 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-pca955x.ko 2023-07-14T11:38:01.152 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-pca963x.ko 2023-07-14T11:38:01.153 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-ss4200.ko 2023-07-14T11:38:01.153 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-tca6507.ko 2023-07-14T11:38:01.153 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-wm831x-status.ko 2023-07-14T11:38:01.153 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-wm8350.ko 2023-07-14T11:38:01.153 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ 2023-07-14T11:38:01.153 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-14T11:38:01.153 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-14T11:38:01.153 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-14T11:38:01.154 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-14T11:38:01.154 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-14T11:38:01.154 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-14T11:38:01.154 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-14T11:38:01.154 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-14T11:38:01.154 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/ 2023-07-14T11:38:01.155 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/ 2023-07-14T11:38:01.155 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 655473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/bcache.ko 2023-07-14T11:38:01.155 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bio-prison.ko 2023-07-14T11:38:01.155 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 89601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bufio.ko 2023-07-14T11:38:01.155 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache-smq.ko 2023-07-14T11:38:01.155 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 155993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache.ko 2023-07-14T11:38:01.155 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 107049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-crypt.ko 2023-07-14T11:38:01.155 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-delay.ko 2023-07-14T11:38:01.156 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-era.ko 2023-07-14T11:38:01.156 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-flakey.ko 2023-07-14T11:38:01.156 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log-userspace.ko 2023-07-14T11:38:01.156 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log.ko 2023-07-14T11:38:01.156 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-mirror.ko 2023-07-14T11:38:01.156 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-multipath.ko 2023-07-14T11:38:01.156 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-queue-length.ko 2023-07-14T11:38:01.157 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 70769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-raid.ko 2023-07-14T11:38:01.157 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-region-hash.ko 2023-07-14T11:38:01.157 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-round-robin.ko 2023-07-14T11:38:01.157 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-service-time.ko 2023-07-14T11:38:01.157 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 109593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-snapshot.ko 2023-07-14T11:38:01.157 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-switch.ko 2023-07-14T11:38:01.157 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 181585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-thin-pool.ko 2023-07-14T11:38:01.158 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-verity.ko 2023-07-14T11:38:01.158 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-zero.ko 2023-07-14T11:38:01.158 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/faulty.ko 2023-07-14T11:38:01.158 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/linear.ko 2023-07-14T11:38:01.158 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/multipath.ko 2023-07-14T11:38:01.158 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/ 2023-07-14T11:38:01.158 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 199937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-14T11:38:01.159 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid0.ko 2023-07-14T11:38:01.159 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 112609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid1.ko 2023-07-14T11:38:01.159 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 160553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid10.ko 2023-07-14T11:38:01.159 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 396153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid456.ko 2023-07-14T11:38:01.159 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/ 2023-07-14T11:38:01.159 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/ 2023-07-14T11:38:01.159 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 178113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptbase.ko 2023-07-14T11:38:01.159 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 63385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptctl.ko 2023-07-14T11:38:01.160 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptfc.ko 2023-07-14T11:38:01.160 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptlan.ko 2023-07-14T11:38:01.160 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 123305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptsas.ko 2023-07-14T11:38:01.160 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 79017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptscsih.ko 2023-07-14T11:38:01.160 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptspi.ko 2023-07-14T11:38:01.160 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/ 2023-07-14T11:38:01.160 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/88pm800.ko 2023-07-14T11:38:01.161 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/88pm805.ko 2023-07-14T11:38:01.161 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/88pm80x.ko 2023-07-14T11:38:01.161 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/arizona-i2c.ko 2023-07-14T11:38:01.161 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 123721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/arizona.ko 2023-07-14T11:38:01.161 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/bcm590xx.ko 2023-07-14T11:38:01.161 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/cros_ec_dev.ko 2023-07-14T11:38:01.161 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/dln2.ko 2023-07-14T11:38:01.162 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/janz-cmodio.ko 2023-07-14T11:38:01.162 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/kempld-core.ko 2023-07-14T11:38:01.162 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lm3533-core.ko 2023-07-14T11:38:01.162 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-07-14T11:38:01.162 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lp3943.ko 2023-07-14T11:38:01.162 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_ich.ko 2023-07-14T11:38:01.162 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_sch.ko 2023-07-14T11:38:01.162 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/max8907.ko 2023-07-14T11:38:01.163 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mc13xxx-core.ko 2023-07-14T11:38:01.163 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-07-14T11:38:01.163 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/menf21bmc.ko 2023-07-14T11:38:01.163 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/pcf50633-adc.ko 2023-07-14T11:38:01.163 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/pcf50633-gpio.ko 2023-07-14T11:38:01.163 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/pcf50633.ko 2023-07-14T11:38:01.163 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-07-14T11:38:01.164 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/retu-mfd.ko 2023-07-14T11:38:01.164 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 80425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/si476x-core.ko 2023-07-14T11:38:01.164 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/sm501.ko 2023-07-14T11:38:01.164 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-07-14T11:38:01.164 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/tps65010.ko 2023-07-14T11:38:01.164 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/tps6507x.ko 2023-07-14T11:38:01.164 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/viperboard.ko 2023-07-14T11:38:01.165 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/vx855.ko 2023-07-14T11:38:01.165 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wl1273-core.ko 2023-07-14T11:38:01.165 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wm8994.ko 2023-07-14T11:38:01.165 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ 2023-07-14T11:38:01.165 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-07-14T11:38:01.165 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ad525x_dpot.ko 2023-07-14T11:38:01.165 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/ 2023-07-14T11:38:01.165 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-14T11:38:01.166 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds9802als.ko 2023-07-14T11:38:01.166 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds990x.ko 2023-07-14T11:38:01.166 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 45593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/bh1770glc.ko 2023-07-14T11:38:01.166 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/c2port/ 2023-07-14T11:38:01.166 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-07-14T11:38:01.166 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/c2port/core.ko 2023-07-14T11:38:01.166 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/ 2023-07-14T11:38:01.166 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/cb710.ko 2023-07-14T11:38:01.167 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ds1682.ko 2023-07-14T11:38:01.167 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/dummy-irq.ko 2023-07-14T11:38:01.167 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/echo/ 2023-07-14T11:38:01.167 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/echo/echo.ko 2023-07-14T11:38:01.167 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/ 2023-07-14T11:38:01.167 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/at24.ko 2023-07-14T11:38:01.167 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-14T11:38:01.168 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-14T11:38:01.168 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/max6875.ko 2023-07-14T11:38:01.168 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/enclosure.ko 2023-07-14T11:38:01.168 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/genwqe/ 2023-07-14T11:38:01.168 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 152137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-07-14T11:38:01.168 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hmc6352.ko 2023-07-14T11:38:01.168 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hpilo.ko 2023-07-14T11:38:01.169 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ibmasm/ 2023-07-14T11:38:01.169 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 72601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-07-14T11:38:01.169 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ics932s401.ko 2023-07-14T11:38:01.169 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29003.ko 2023-07-14T11:38:01.169 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29020.ko 2023-07-14T11:38:01.169 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/ 2023-07-14T11:38:01.169 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-14T11:38:01.169 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-14T11:38:01.170 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/ 2023-07-14T11:38:01.170 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 125209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-me.ko 2023-07-14T11:38:01.170 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 56521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-txe.ko 2023-07-14T11:38:01.170 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 309857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei.ko 2023-07-14T11:38:01.170 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/phantom.ko 2023-07-14T11:38:01.170 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ti-st/ 2023-07-14T11:38:01.170 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 74905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ti-st/st_drv.ko 2023-07-14T11:38:01.170 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_7xx1.ko 2023-07-14T11:38:01.171 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_core.ko 2023-07-14T11:38:01.171 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tsl2550.ko 2023-07-14T11:38:01.171 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_balloon.ko 2023-07-14T11:38:01.171 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/ 2023-07-14T11:38:01.171 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 183857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-14T11:38:01.171 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ 2023-07-14T11:38:01.171 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/ 2023-07-14T11:38:01.172 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 487801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/bonding.ko 2023-07-14T11:38:01.172 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/ 2023-07-14T11:38:01.172 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/ 2023-07-14T11:38:01.172 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 106009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-14T11:38:01.172 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-14T11:38:01.172 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-14T11:38:01.172 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/ 2023-07-14T11:38:01.172 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 452481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-14T11:38:01.173 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dummy.ko 2023-07-14T11:38:01.173 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/eql.ko 2023-07-14T11:38:01.173 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ 2023-07-14T11:38:01.173 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/ 2023-07-14T11:38:01.173 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 45785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-14T11:38:01.173 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-14T11:38:01.173 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 130489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-14T11:38:01.174 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 68785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-14T11:38:01.174 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ 2023-07-14T11:38:01.174 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-14T11:38:01.174 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-14T11:38:01.174 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-14T11:38:01.174 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 99113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-14T11:38:01.175 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/ 2023-07-14T11:38:01.175 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-14T11:38:01.175 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/ 2023-07-14T11:38:01.175 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 69145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-14T11:38:01.175 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/ 2023-07-14T11:38:01.175 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-14T11:38:01.175 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/ 2023-07-14T11:38:01.175 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-14T11:38:01.176 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/ 2023-07-14T11:38:01.176 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-14T11:38:01.176 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-14T11:38:01.176 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 87345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-14T11:38:01.176 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/ 2023-07-14T11:38:01.176 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/ 2023-07-14T11:38:01.176 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 97321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-14T11:38:01.176 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-07-14T11:38:01.177 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 104585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-14T11:38:01.177 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-07-14T11:38:01.177 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 90257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-14T11:38:01.177 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/ 2023-07-14T11:38:01.177 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 81505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-14T11:38:01.177 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 69713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-14T11:38:01.177 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/ 2023-07-14T11:38:01.178 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-14T11:38:01.178 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 176881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-14T11:38:01.178 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-07-14T11:38:01.178 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1422209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-14T11:38:01.178 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 128377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-14T11:38:01.178 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/ 2023-07-14T11:38:01.178 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 132289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-14T11:38:01.179 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 312145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-14T11:38:01.179 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/ 2023-07-14T11:38:01.179 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/ 2023-07-14T11:38:01.179 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 323601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-14T11:38:01.179 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/ 2023-07-14T11:38:01.179 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-07-14T11:38:01.179 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 193433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-14T11:38:01.179 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-07-14T11:38:01.180 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 347641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-14T11:38:01.180 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-07-14T11:38:01.180 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1044145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-14T11:38:01.180 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-07-14T11:38:01.180 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 166113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-14T11:38:01.180 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-07-14T11:38:01.180 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-14T11:38:01.180 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/ 2023-07-14T11:38:01.181 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/ 2023-07-14T11:38:01.181 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 204329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-14T11:38:01.181 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/ 2023-07-14T11:38:01.181 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/ 2023-07-14T11:38:01.181 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 69449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-14T11:38:01.181 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 61489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-14T11:38:01.181 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 141161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-14T11:38:01.182 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-14T11:38:01.182 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 62865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-14T11:38:01.182 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-14T11:38:01.182 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/ 2023-07-14T11:38:01.182 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-14T11:38:01.182 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 62833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-14T11:38:01.182 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 45721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dnet.ko 2023-07-14T11:38:01.182 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ec_bhf.ko 2023-07-14T11:38:01.183 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/ 2023-07-14T11:38:01.183 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/ 2023-07-14T11:38:01.183 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 281329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-14T11:38:01.183 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ethoc.ko 2023-07-14T11:38:01.183 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fealnx.ko 2023-07-14T11:38:01.183 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fujitsu/ 2023-07-14T11:38:01.183 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 45281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-07-14T11:38:01.183 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ 2023-07-14T11:38:01.184 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 90465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-14T11:38:01.184 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/ 2023-07-14T11:38:01.184 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 295713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-14T11:38:01.184 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/ 2023-07-14T11:38:01.184 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 584249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-14T11:38:01.184 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/ 2023-07-14T11:38:01.185 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 246321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-14T11:38:01.185 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/ 2023-07-14T11:38:01.185 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 923969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-14T11:38:01.185 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/ 2023-07-14T11:38:01.185 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 298097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-14T11:38:01.185 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/ 2023-07-14T11:38:01.185 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 514889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-14T11:38:01.185 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/ 2023-07-14T11:38:01.186 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 93593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-14T11:38:01.186 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-07-14T11:38:01.186 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 734297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-14T11:38:01.186 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-07-14T11:38:01.186 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 151841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-14T11:38:01.186 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 86921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/jme.ko 2023-07-14T11:38:01.186 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/ 2023-07-14T11:38:01.186 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20977 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-14T11:38:01.187 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 102385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-14T11:38:01.187 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 122681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-14T11:38:01.187 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/ 2023-07-14T11:38:01.187 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-07-14T11:38:01.187 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 727433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-14T11:38:01.187 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 280513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-14T11:38:01.187 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-07-14T11:38:01.187 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-07-14T11:38:01.188 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1095817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-14T11:38:01.188 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ 2023-07-14T11:38:01.188 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-07-14T11:38:01.188 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-07-14T11:38:01.188 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-07-14T11:38:01.188 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 95665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-14T11:38:01.188 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/ 2023-07-14T11:38:01.189 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-07-14T11:38:01.189 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 108809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-14T11:38:01.189 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ 2023-07-14T11:38:01.189 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 73377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-14T11:38:01.189 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-14T11:38:01.189 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/ 2023-07-14T11:38:01.189 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 150905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-14T11:38:01.189 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/ 2023-07-14T11:38:01.190 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 136785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-14T11:38:01.190 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/ 2023-07-14T11:38:01.190 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-14T11:38:01.190 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-14T11:38:01.190 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/ 2023-07-14T11:38:01.190 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-07-14T11:38:01.190 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 227113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-14T11:38:01.190 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 87081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-14T11:38:01.191 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-07-14T11:38:01.191 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 671393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-14T11:38:01.191 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/ 2023-07-14T11:38:01.191 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-14T11:38:01.191 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/ 2023-07-14T11:38:01.191 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-14T11:38:01.191 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 83921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-14T11:38:01.191 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-14T11:38:01.192 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 190089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-14T11:38:01.192 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/samsung/ 2023-07-14T11:38:01.192 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-07-14T11:38:01.192 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 104241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-07-14T11:38:01.192 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/ 2023-07-14T11:38:01.192 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 952881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-14T11:38:01.192 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/ 2023-07-14T11:38:01.193 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-14T11:38:01.193 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/ 2023-07-14T11:38:01.193 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-14T11:38:01.193 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 63273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-14T11:38:01.193 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/ 2023-07-14T11:38:01.193 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 61537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-14T11:38:01.194 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-14T11:38:01.194 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 73345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-14T11:38:01.194 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 68209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-14T11:38:01.194 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/ 2023-07-14T11:38:01.194 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-07-14T11:38:01.194 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-07-14T11:38:01.194 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-14T11:38:01.195 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-07-14T11:38:01.195 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 481681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-14T11:38:01.195 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/ 2023-07-14T11:38:01.195 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 103321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-14T11:38:01.195 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 166121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-14T11:38:01.195 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-14T11:38:01.195 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-14T11:38:01.195 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/ 2023-07-14T11:38:01.196 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-14T11:38:01.196 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/ 2023-07-14T11:38:01.196 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 86465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-14T11:38:01.196 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/ 2023-07-14T11:38:01.196 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 80257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-14T11:38:01.196 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 83681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-14T11:38:01.197 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/ 2023-07-14T11:38:01.197 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-14T11:38:01.197 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-14T11:38:01.197 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/ 2023-07-14T11:38:01.197 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-14T11:38:01.197 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/fddi/ 2023-07-14T11:38:01.197 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/fddi/defxx.ko 2023-07-14T11:38:01.198 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/fddi/skfp/ 2023-07-14T11:38:01.198 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 216977 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/fddi/skfp/skfp.ko 2023-07-14T11:38:01.198 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 77401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/geneve.ko 2023-07-14T11:38:01.198 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/ 2023-07-14T11:38:01.198 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 235593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-14T11:38:01.198 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/ 2023-07-14T11:38:01.198 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-14T11:38:01.198 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ifb.ko 2023-07-14T11:38:01.199 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ 2023-07-14T11:38:01.199 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 83737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-14T11:38:01.199 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 62065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvlan.ko 2023-07-14T11:38:01.199 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvtap.ko 2023-07-14T11:38:01.199 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/ 2023-07-14T11:38:01.199 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-14T11:38:01.199 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-14T11:38:01.199 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-gpio.ko 2023-07-14T11:38:01.200 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-14T11:38:01.200 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio.ko 2023-07-14T11:38:01.200 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mii.ko 2023-07-14T11:38:01.200 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/netconsole.ko 2023-07-14T11:38:01.200 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/nlmon.ko 2023-07-14T11:38:01.200 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/ 2023-07-14T11:38:01.200 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-14T11:38:01.201 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-14T11:38:01.201 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ 2023-07-14T11:38:01.201 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/amd.ko 2023-07-14T11:38:01.201 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-14T11:38:01.201 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-14T11:38:01.201 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm87xx.ko 2023-07-14T11:38:01.201 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/broadcom.ko 2023-07-14T11:38:01.201 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/cicada.ko 2023-07-14T11:38:01.202 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/davicom.ko 2023-07-14T11:38:01.202 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/et1011c.ko 2023-07-14T11:38:01.202 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/icplus.ko 2023-07-14T11:38:01.202 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/lxt.ko 2023-07-14T11:38:01.202 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/marvell.ko 2023-07-14T11:38:01.202 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 75713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/micrel.ko 2023-07-14T11:38:01.202 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/national.ko 2023-07-14T11:38:01.202 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 97265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/phylink.ko 2023-07-14T11:38:01.203 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/qsemi.ko 2023-07-14T11:38:01.203 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/realtek.ko 2023-07-14T11:38:01.203 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/smsc.ko 2023-07-14T11:38:01.203 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ste10Xp.ko 2023-07-14T11:38:01.203 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/vitesse.ko 2023-07-14T11:38:01.203 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/plip/ 2023-07-14T11:38:01.203 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/plip/plip.ko 2023-07-14T11:38:01.204 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ 2023-07-14T11:38:01.204 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-14T11:38:01.204 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_async.ko 2023-07-14T11:38:01.204 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-14T11:38:01.204 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-14T11:38:01.204 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-14T11:38:01.204 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppoe.ko 2023-07-14T11:38:01.204 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppox.ko 2023-07-14T11:38:01.205 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pptp.ko 2023-07-14T11:38:01.205 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sb1000.ko 2023-07-14T11:38:01.205 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/ 2023-07-14T11:38:01.205 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slip.ko 2023-07-14T11:38:01.205 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sungem_phy.ko 2023-07-14T11:38:01.205 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 64889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tap.ko 2023-07-14T11:38:01.205 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/ 2023-07-14T11:38:01.205 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 105961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team.ko 2023-07-14T11:38:01.206 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-14T11:38:01.206 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-14T11:38:01.206 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-14T11:38:01.206 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_random.ko 2023-07-14T11:38:01.206 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-14T11:38:01.206 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 88193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/veth.ko 2023-07-14T11:38:01.206 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/ 2023-07-14T11:38:01.206 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 120497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-14T11:38:01.207 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/ 2023-07-14T11:38:01.207 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 290665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/vxlan.ko 2023-07-14T11:38:01.207 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/ 2023-07-14T11:38:01.207 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 143801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-14T11:38:01.207 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ 2023-07-14T11:38:01.207 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ntb.ko 2023-07-14T11:38:01.207 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/ 2023-07-14T11:38:01.207 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/ 2023-07-14T11:38:01.208 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 266561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme-core.ko 2023-07-14T11:38:01.208 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 101937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme.ko 2023-07-14T11:38:01.208 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/ 2023-07-14T11:38:01.208 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 133817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport.ko 2023-07-14T11:38:01.208 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_cs.ko 2023-07-14T11:38:01.208 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 84337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_pc.ko 2023-07-14T11:38:01.208 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_serial.ko 2023-07-14T11:38:01.209 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/ 2023-07-14T11:38:01.209 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/ 2023-07-14T11:38:01.209 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-14T11:38:01.209 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-07-14T11:38:01.209 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-07-14T11:38:01.209 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pci-stub.ko 2023-07-14T11:38:01.209 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/xen-pcifront.ko 2023-07-14T11:38:01.209 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/ 2023-07-14T11:38:01.210 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/i82092.ko 2023-07-14T11:38:01.210 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 145785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pcmcia.ko 2023-07-14T11:38:01.210 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 62081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pcmcia_core.ko 2023-07-14T11:38:01.210 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-07-14T11:38:01.210 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pd6729.ko 2023-07-14T11:38:01.210 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 112913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-14T11:38:01.210 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/phy/ 2023-07-14T11:38:01.210 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/phy/broadcom/ 2023-07-14T11:38:01.211 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-07-14T11:38:01.211 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/phy/samsung/ 2023-07-14T11:38:01.211 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-07-14T11:38:01.211 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/ 2023-07-14T11:38:01.211 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/ 2023-07-14T11:38:01.211 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 71521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-14T11:38:01.211 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/ 2023-07-14T11:38:01.212 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/ 2023-07-14T11:38:01.212 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-07-14T11:38:01.212 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/cros_ec.ko 2023-07-14T11:38:01.212 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-07-14T11:38:01.212 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-07-14T11:38:01.212 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-07-14T11:38:01.212 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-07-14T11:38:01.212 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-07-14T11:38:01.213 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/ 2023-07-14T11:38:01.213 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-14T11:38:01.213 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acerhdf.ko 2023-07-14T11:38:01.213 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-14T11:38:01.213 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-14T11:38:01.213 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-14T11:38:01.213 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 106873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-14T11:38:01.214 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-14T11:38:01.214 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-14T11:38:01.214 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-14T11:38:01.214 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-14T11:38:01.214 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-14T11:38:01.214 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/hdaps.ko 2023-07-14T11:38:01.214 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/ibm_rtl.ko 2023-07-14T11:38:01.215 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/ 2023-07-14T11:38:01.215 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-14T11:38:01.215 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-07-14T11:38:01.215 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel_ips.ko 2023-07-14T11:38:01.215 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-14T11:38:01.215 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-14T11:38:01.215 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-14T11:38:01.215 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-14T11:38:01.216 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-14T11:38:01.216 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-14T11:38:01.216 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-14T11:38:01.216 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-14T11:38:01.216 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-14T11:38:01.216 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi.ko 2023-07-14T11:38:01.216 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/ 2023-07-14T11:38:01.216 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/ 2023-07-14T11:38:01.217 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/88pm860x_battery.ko 2023-07-14T11:38:01.217 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/88pm860x_charger.ko 2023-07-14T11:38:01.217 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 55057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/bq2415x_charger.ko 2023-07-14T11:38:01.217 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/bq24190_charger.ko 2023-07-14T11:38:01.217 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/bq24735-charger.ko 2023-07-14T11:38:01.217 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-07-14T11:38:01.217 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/da9030_battery.ko 2023-07-14T11:38:01.218 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/da9052-battery.ko 2023-07-14T11:38:01.218 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/ds2760_battery.ko 2023-07-14T11:38:01.218 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/ds2780_battery.ko 2023-07-14T11:38:01.218 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/ds2781_battery.ko 2023-07-14T11:38:01.218 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/ds2782_battery.ko 2023-07-14T11:38:01.218 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/gpio-charger.ko 2023-07-14T11:38:01.218 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/isp1704_charger.ko 2023-07-14T11:38:01.218 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/lp8727_charger.ko 2023-07-14T11:38:01.219 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/max14577_charger.ko 2023-07-14T11:38:01.219 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/max17040_battery.ko 2023-07-14T11:38:01.219 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/max17042_battery.ko 2023-07-14T11:38:01.219 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/max8903_charger.ko 2023-07-14T11:38:01.219 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/max8925_power.ko 2023-07-14T11:38:01.219 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/pcf50633-charger.ko 2023-07-14T11:38:01.219 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/sbs-battery.ko 2023-07-14T11:38:01.220 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/test_power.ko 2023-07-14T11:38:01.220 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/tps65090-charger.ko 2023-07-14T11:38:01.220 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/wm831x_backup.ko 2023-07-14T11:38:01.220 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/wm831x_power.ko 2023-07-14T11:38:01.220 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/wm8350_power.ko 2023-07-14T11:38:01.220 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/ 2023-07-14T11:38:01.220 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-14T11:38:01.220 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-14T11:38:01.221 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/ 2023-07-14T11:38:01.221 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/ 2023-07-14T11:38:01.221 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-14T11:38:01.221 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-14T11:38:01.221 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps_parport.ko 2023-07-14T11:38:01.221 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/pps_core.ko 2023-07-14T11:38:01.221 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ 2023-07-14T11:38:01.222 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp.ko 2023-07-14T11:38:01.222 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp_kvm.ko 2023-07-14T11:38:01.222 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/ 2023-07-14T11:38:01.222 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-88pm80x.ko 2023-07-14T11:38:01.222 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-88pm860x.ko 2023-07-14T11:38:01.222 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-14T11:38:01.223 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-14T11:38:01.223 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-da9052.ko 2023-07-14T11:38:01.223 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-da9055.ko 2023-07-14T11:38:01.223 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-da9063.ko 2023-07-14T11:38:01.223 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-14T11:38:01.223 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 68577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-14T11:38:01.223 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-14T11:38:01.224 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-14T11:38:01.224 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-14T11:38:01.224 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-14T11:38:01.224 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13369 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-14T11:38:01.224 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-14T11:38:01.224 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-14T11:38:01.224 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-em3027.ko 2023-07-14T11:38:01.224 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-14T11:38:01.225 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-14T11:38:01.225 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-14T11:38:01.225 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-lp8788.ko 2023-07-14T11:38:01.225 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-14T11:38:01.225 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-14T11:38:01.225 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-14T11:38:01.225 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t86.ko 2023-07-14T11:38:01.225 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11633 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max6900.ko 2023-07-14T11:38:01.226 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max8907.ko 2023-07-14T11:38:01.226 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max8925.ko 2023-07-14T11:38:01.226 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max8997.ko 2023-07-14T11:38:01.226 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max8998.ko 2023-07-14T11:38:01.226 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-07-14T11:38:01.226 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-14T11:38:01.227 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-palmas.ko 2023-07-14T11:38:01.227 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-14T11:38:01.227 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17977 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf50633.ko 2023-07-14T11:38:01.227 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-14T11:38:01.227 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-14T11:38:01.227 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-14T11:38:01.227 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-14T11:38:01.227 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rc5t583.ko 2023-07-14T11:38:01.228 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-14T11:38:01.228 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-14T11:38:01.228 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-14T11:38:01.228 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-14T11:38:01.228 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-14T11:38:01.228 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-s35390a.ko 2023-07-14T11:38:01.228 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-14T11:38:01.228 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-tps6586x.ko 2023-07-14T11:38:01.229 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-tps65910.ko 2023-07-14T11:38:01.229 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-wm831x.ko 2023-07-14T11:38:01.229 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-wm8350.ko 2023-07-14T11:38:01.229 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-x1205.ko 2023-07-14T11:38:01.229 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ 2023-07-14T11:38:01.229 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 77617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-9xxx.ko 2023-07-14T11:38:01.229 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-sas.ko 2023-07-14T11:38:01.229 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 55169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-xxxx.ko 2023-07-14T11:38:01.230 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 135169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/BusLogic.ko 2023-07-14T11:38:01.230 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/a100u2w.ko 2023-07-14T11:38:01.230 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/ 2023-07-14T11:38:01.230 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 272353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-14T11:38:01.230 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 100617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/advansys.ko 2023-07-14T11:38:01.230 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/ 2023-07-14T11:38:01.230 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 356265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-14T11:38:01.230 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 295801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-14T11:38:01.231 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic94xx/ 2023-07-14T11:38:01.231 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 189321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-07-14T11:38:01.231 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/am53c974.ko 2023-07-14T11:38:01.231 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/ 2023-07-14T11:38:01.231 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 106569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-14T11:38:01.231 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/atp870u.ko 2023-07-14T11:38:01.231 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/ 2023-07-14T11:38:01.231 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 246897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-14T11:38:01.232 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/ 2023-07-14T11:38:01.232 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1172185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/bfa.ko 2023-07-14T11:38:01.232 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/ 2023-07-14T11:38:01.232 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 237169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-14T11:38:01.232 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/ 2023-07-14T11:38:01.232 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 115393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-14T11:38:01.232 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ch.ko 2023-07-14T11:38:01.233 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/ 2023-07-14T11:38:01.233 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 308889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-14T11:38:01.233 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/ 2023-07-14T11:38:01.233 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-07-14T11:38:01.233 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 76193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-14T11:38:01.233 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-07-14T11:38:01.233 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 101449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-14T11:38:01.233 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 146785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-14T11:38:01.234 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 75529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dc395x.ko 2023-07-14T11:38:01.234 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dmx3191d.ko 2023-07-14T11:38:01.234 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/ 2023-07-14T11:38:01.234 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 212401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-14T11:38:01.234 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esp_scsi.ko 2023-07-14T11:38:01.234 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/ 2023-07-14T11:38:01.234 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-14T11:38:01.235 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 158193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-14T11:38:01.235 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fdomain.ko 2023-07-14T11:38:01.235 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/ 2023-07-14T11:38:01.235 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 261913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/fnic.ko 2023-07-14T11:38:01.235 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 201793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hpsa.ko 2023-07-14T11:38:01.235 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hptiop.ko 2023-07-14T11:38:01.235 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hv_storvsc.ko 2023-07-14T11:38:01.235 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/imm.ko 2023-07-14T11:38:01.236 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/initio.ko 2023-07-14T11:38:01.236 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 266521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ipr.ko 2023-07-14T11:38:01.236 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 101881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ips.ko 2023-07-14T11:38:01.236 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/ 2023-07-14T11:38:01.236 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 317249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/isci.ko 2023-07-14T11:38:01.236 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-14T11:38:01.236 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 52897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-14T11:38:01.236 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/ 2023-07-14T11:38:01.237 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 314961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/libfc.ko 2023-07-14T11:38:01.237 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 145569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi.ko 2023-07-14T11:38:01.237 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-14T11:38:01.237 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/ 2023-07-14T11:38:01.237 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 217209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/libsas.ko 2023-07-14T11:38:01.237 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/ 2023-07-14T11:38:01.237 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 2192449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-14T11:38:01.238 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/ 2023-07-14T11:38:01.238 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 73545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-14T11:38:01.238 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-14T11:38:01.238 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 310729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-14T11:38:01.238 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 84697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid.ko 2023-07-14T11:38:01.238 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/ 2023-07-14T11:38:01.238 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 659577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-14T11:38:01.239 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/ 2023-07-14T11:38:01.239 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 119929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-14T11:38:01.239 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvumi.ko 2023-07-14T11:38:01.239 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pcmcia/ 2023-07-14T11:38:01.239 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 81601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-07-14T11:38:01.239 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-07-14T11:38:01.239 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-07-14T11:38:01.239 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-07-14T11:38:01.240 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/ 2023-07-14T11:38:01.240 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 364257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-14T11:38:01.240 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 123489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pmcraid.ko 2023-07-14T11:38:01.240 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ppa.ko 2023-07-14T11:38:01.240 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 55345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla1280.ko 2023-07-14T11:38:01.240 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/ 2023-07-14T11:38:01.240 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1860025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-14T11:38:01.241 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 85313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-14T11:38:01.241 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/ 2023-07-14T11:38:01.241 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 558865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-14T11:38:01.241 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qlogicfas408.ko 2023-07-14T11:38:01.241 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/raid_class.ko 2023-07-14T11:38:01.241 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 238801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_debug.ko 2023-07-14T11:38:01.241 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 173921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-14T11:38:01.242 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 278865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-14T11:38:01.242 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 97705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-14T11:38:01.242 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 73881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-14T11:38:01.242 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-14T11:38:01.242 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ses.ko 2023-07-14T11:38:01.242 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 115849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/st.ko 2023-07-14T11:38:01.242 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/stex.ko 2023-07-14T11:38:01.242 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/ 2023-07-14T11:38:01.243 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 162121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-14T11:38:01.243 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/virtio_scsi.ko 2023-07-14T11:38:01.243 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-14T11:38:01.243 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/wd719x.ko 2023-07-14T11:38:01.243 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/xen-scsifront.ko 2023-07-14T11:38:01.243 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ 2023-07-14T11:38:01.243 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 129417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ssb.ko 2023-07-14T11:38:01.243 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/ 2023-07-14T11:38:01.244 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/ 2023-07-14T11:38:01.244 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 739089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-14T11:38:01.244 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/ 2023-07-14T11:38:01.244 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-14T11:38:01.244 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_file.ko 2023-07-14T11:38:01.244 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_iblock.ko 2023-07-14T11:38:01.244 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 931721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_mod.ko 2023-07-14T11:38:01.245 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_pscsi.ko 2023-07-14T11:38:01.245 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 122545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_user.ko 2023-07-14T11:38:01.245 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/ 2023-07-14T11:38:01.245 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 75641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-14T11:38:01.245 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/ 2023-07-14T11:38:01.245 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/ 2023-07-14T11:38:01.245 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/ 2023-07-14T11:38:01.246 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-14T11:38:01.246 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-14T11:38:01.246 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-14T11:38:01.246 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-14T11:38:01.246 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-14T11:38:01.246 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-14T11:38:01.246 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-14T11:38:01.247 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-14T11:38:01.247 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-14T11:38:01.247 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-14T11:38:01.247 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-14T11:38:01.247 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-14T11:38:01.247 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-07-14T11:38:01.247 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-14T11:38:01.248 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-14T11:38:01.248 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-14T11:38:01.248 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ 2023-07-14T11:38:01.248 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ 2023-07-14T11:38:01.248 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-14T11:38:01.248 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/moxa.ko 2023-07-14T11:38:01.248 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/mxser.ko 2023-07-14T11:38:01.249 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 107545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_gsm.ko 2023-07-14T11:38:01.249 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_hdlc.ko 2023-07-14T11:38:01.249 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 68137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/nozomi.ko 2023-07-14T11:38:01.249 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/ 2023-07-14T11:38:01.249 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/ 2023-07-14T11:38:01.249 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-07-14T11:38:01.249 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-14T11:38:01.249 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-07-14T11:38:01.250 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/altera_uart.ko 2023-07-14T11:38:01.250 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/arc_uart.ko 2023-07-14T11:38:01.250 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 72193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-07-14T11:38:01.250 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/ 2023-07-14T11:38:01.250 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 103017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-14T11:38:01.250 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/rp2.ko 2023-07-14T11:38:01.250 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 52073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/sc16is7xx.ko 2023-07-14T11:38:01.250 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 100185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/synclink_gt.ko 2023-07-14T11:38:01.251 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/ 2023-07-14T11:38:01.251 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ 2023-07-14T11:38:01.251 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 536617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-14T11:38:01.251 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ 2023-07-14T11:38:01.251 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-14T11:38:01.251 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-07-14T11:38:01.251 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/ 2023-07-14T11:38:01.252 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio.ko 2023-07-14T11:38:01.252 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_aec.ko 2023-07-14T11:38:01.252 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_cif.ko 2023-07-14T11:38:01.252 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_dmem_genirq.ko 2023-07-14T11:38:01.252 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_mf624.ko 2023-07-14T11:38:01.252 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_netx.ko 2023-07-14T11:38:01.252 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pci_generic.ko 2023-07-14T11:38:01.252 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-07-14T11:38:01.253 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13633 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_sercos3.ko 2023-07-14T11:38:01.253 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/ 2023-07-14T11:38:01.253 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/c67x00/ 2023-07-14T11:38:01.253 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 73273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/c67x00/c67x00.ko 2023-07-14T11:38:01.253 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/chipidea/ 2023-07-14T11:38:01.253 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 143785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-07-14T11:38:01.253 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-07-14T11:38:01.254 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-07-14T11:38:01.254 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-07-14T11:38:01.254 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/ 2023-07-14T11:38:01.254 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 85457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-acm.ko 2023-07-14T11:38:01.254 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-14T11:38:01.254 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usblp.ko 2023-07-14T11:38:01.255 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 89833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usbtmc.ko 2023-07-14T11:38:01.255 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/common/ 2023-07-14T11:38:01.255 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/common/ulpi.ko 2023-07-14T11:38:01.255 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/dwc2/ 2023-07-14T11:38:01.255 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-07-14T11:38:01.255 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/dwc3/ 2023-07-14T11:38:01.255 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-07-14T11:38:01.255 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-07-14T11:38:01.256 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 445801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/dwc3/dwc3.ko 2023-07-14T11:38:01.256 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/ 2023-07-14T11:38:01.256 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/ 2023-07-14T11:38:01.256 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/u_ether.ko 2023-07-14T11:38:01.256 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/u_serial.ko 2023-07-14T11:38:01.256 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-07-14T11:38:01.256 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-07-14T11:38:01.257 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-07-14T11:38:01.257 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-07-14T11:38:01.257 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 92377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-07-14T11:38:01.257 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-07-14T11:38:01.257 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 122073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-07-14T11:38:01.257 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-07-14T11:38:01.257 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-07-14T11:38:01.257 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-07-14T11:38:01.258 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-07-14T11:38:01.258 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 85841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-07-14T11:38:01.258 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-07-14T11:38:01.258 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 58433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-07-14T11:38:01.258 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-07-14T11:38:01.258 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/ 2023-07-14T11:38:01.258 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-07-14T11:38:01.258 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-07-14T11:38:01.259 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-07-14T11:38:01.259 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-07-14T11:38:01.259 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-07-14T11:38:01.259 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-07-14T11:38:01.259 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-07-14T11:38:01.259 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16977 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-07-14T11:38:01.259 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-07-14T11:38:01.259 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-07-14T11:38:01.260 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-07-14T11:38:01.260 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-07-14T11:38:01.260 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 62153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-07-14T11:38:01.260 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-07-14T11:38:01.260 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 161233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/libcomposite.ko 2023-07-14T11:38:01.260 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/ 2023-07-14T11:38:01.260 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-07-14T11:38:01.261 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/bdc/ 2023-07-14T11:38:01.261 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 157665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-07-14T11:38:01.261 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-07-14T11:38:01.261 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 52993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-07-14T11:38:01.261 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 69689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-07-14T11:38:01.261 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 56009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-07-14T11:38:01.261 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 76033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/net2272.ko 2023-07-14T11:38:01.261 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 75057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/net2280.ko 2023-07-14T11:38:01.262 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 73153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-07-14T11:38:01.262 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 65793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-07-14T11:38:01.262 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-07-14T11:38:01.262 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 81489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-07-14T11:38:01.262 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 156033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-07-14T11:38:01.262 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/ 2023-07-14T11:38:01.262 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/bcma-hcd.ko 2023-07-14T11:38:01.262 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 80097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/isp116x-hcd.ko 2023-07-14T11:38:01.263 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 87337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-07-14T11:38:01.263 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-07-14T11:38:01.263 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-14T11:38:01.263 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811_cs.ko 2023-07-14T11:38:01.263 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/ssb-hcd.ko 2023-07-14T11:38:01.263 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-07-14T11:38:01.263 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/ 2023-07-14T11:38:01.264 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/mdc800.ko 2023-07-14T11:38:01.264 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/microtek.ko 2023-07-14T11:38:01.264 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ 2023-07-14T11:38:01.264 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/adutux.ko 2023-07-14T11:38:01.264 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/appledisplay.ko 2023-07-14T11:38:01.264 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-07-14T11:38:01.264 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/cytherm.ko 2023-07-14T11:38:01.264 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ehset.ko 2023-07-14T11:38:01.265 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi26.ko 2023-07-14T11:38:01.265 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi62.ko 2023-07-14T11:38:01.265 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ezusb.ko 2023-07-14T11:38:01.265 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/idmouse.ko 2023-07-14T11:38:01.265 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/iowarrior.ko 2023-07-14T11:38:01.265 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-14T11:38:01.265 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ldusb.ko 2023-07-14T11:38:01.266 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/legousbtower.ko 2023-07-14T11:38:01.266 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/lvstest.ko 2023-07-14T11:38:01.266 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/ 2023-07-14T11:38:01.266 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 64841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-14T11:38:01.266 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-14T11:38:01.266 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usb3503.ko 2023-07-14T11:38:01.266 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usblcd.ko 2023-07-14T11:38:01.266 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-14T11:38:01.267 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 70873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbtest.ko 2023-07-14T11:38:01.267 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/uss720.ko 2023-07-14T11:38:01.267 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/yurex.ko 2023-07-14T11:38:01.267 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/mon/ 2023-07-14T11:38:01.267 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 73185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/mon/usbmon.ko 2023-07-14T11:38:01.267 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/musb/ 2023-07-14T11:38:01.267 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 301521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/musb/musb_hdrc.ko 2023-07-14T11:38:01.267 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/ 2023-07-14T11:38:01.268 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-generic.ko 2023-07-14T11:38:01.268 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-07-14T11:38:01.268 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-isp1301.ko 2023-07-14T11:38:01.268 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-tahvo.ko 2023-07-14T11:38:01.268 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ 2023-07-14T11:38:01.268 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/aircable.ko 2023-07-14T11:38:01.268 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ark3116.ko 2023-07-14T11:38:01.268 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-14T11:38:01.269 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ch341.ko 2023-07-14T11:38:01.269 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 70209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cp210x.ko 2023-07-14T11:38:01.269 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cyberjack.ko 2023-07-14T11:38:01.269 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-14T11:38:01.269 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-14T11:38:01.269 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/empeg.ko 2023-07-14T11:38:01.269 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/f81232.ko 2023-07-14T11:38:01.270 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 175145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-14T11:38:01.270 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 52713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-14T11:38:01.270 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 137105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-14T11:38:01.270 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 124673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_ti.ko 2023-07-14T11:38:01.270 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipaq.ko 2023-07-14T11:38:01.270 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipw.ko 2023-07-14T11:38:01.270 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ir-usb.ko 2023-07-14T11:38:01.270 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-14T11:38:01.271 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 107417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan.ko 2023-07-14T11:38:01.271 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-14T11:38:01.271 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-14T11:38:01.271 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-14T11:38:01.271 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mct_u232.ko 2023-07-14T11:38:01.271 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/metro-usb.ko 2023-07-14T11:38:01.271 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7720.ko 2023-07-14T11:38:01.271 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7840.ko 2023-07-14T11:38:01.272 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 56593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mxuport.ko 2023-07-14T11:38:01.272 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/navman.ko 2023-07-14T11:38:01.272 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/omninet.ko 2023-07-14T11:38:01.272 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/opticon.ko 2023-07-14T11:38:01.272 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 189729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/option.ko 2023-07-14T11:38:01.272 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/oti6858.ko 2023-07-14T11:38:01.272 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/pl2303.ko 2023-07-14T11:38:01.273 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcaux.ko 2023-07-14T11:38:01.273 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcserial.ko 2023-07-14T11:38:01.273 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/quatech2.ko 2023-07-14T11:38:01.273 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/safe_serial.ko 2023-07-14T11:38:01.273 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/sierra.ko 2023-07-14T11:38:01.273 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-14T11:38:01.273 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ssu100.ko 2023-07-14T11:38:01.273 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/symbolserial.ko 2023-07-14T11:38:01.274 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 61337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-14T11:38:01.274 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-14T11:38:01.274 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_debug.ko 2023-07-14T11:38:01.274 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-14T11:38:01.274 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 125505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usbserial.ko 2023-07-14T11:38:01.274 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/visor.ko 2023-07-14T11:38:01.274 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/whiteheat.ko 2023-07-14T11:38:01.274 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/wishbone-serial.ko 2023-07-14T11:38:01.275 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-14T11:38:01.275 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ 2023-07-14T11:38:01.275 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 52913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/uas.ko 2023-07-14T11:38:01.275 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-14T11:38:01.275 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-14T11:38:01.275 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-14T11:38:01.275 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-14T11:38:01.275 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-14T11:38:01.276 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-14T11:38:01.276 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-14T11:38:01.276 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-karma.ko 2023-07-14T11:38:01.276 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-14T11:38:01.276 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-14T11:38:01.276 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-14T11:38:01.276 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-14T11:38:01.276 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-14T11:38:01.277 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 180017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/usb-storage.ko 2023-07-14T11:38:01.277 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/ 2023-07-14T11:38:01.277 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-14T11:38:01.277 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 71345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-14T11:38:01.277 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 105401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-14T11:38:01.277 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/ 2023-07-14T11:38:01.277 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/ 2023-07-14T11:38:01.277 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 153697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-14T11:38:01.278 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-14T11:38:01.278 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 100569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio.ko 2023-07-14T11:38:01.278 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 75137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-14T11:38:01.278 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/ 2023-07-14T11:38:01.278 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 113889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost.ko 2023-07-14T11:38:01.278 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-14T11:38:01.278 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 58137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_net.ko 2023-07-14T11:38:01.278 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 86521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_scsi.ko 2023-07-14T11:38:01.279 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/ 2023-07-14T11:38:01.279 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/ 2023-07-14T11:38:01.279 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-07-14T11:38:01.279 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/aat2870_bl.ko 2023-07-14T11:38:01.279 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/adp5520_bl.ko 2023-07-14T11:38:01.279 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/adp8860_bl.ko 2023-07-14T11:38:01.279 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/adp8870_bl.ko 2023-07-14T11:38:01.279 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/apple_bl.ko 2023-07-14T11:38:01.280 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/as3711_bl.ko 2023-07-14T11:38:01.280 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/bd6107.ko 2023-07-14T11:38:01.280 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/da903x_bl.ko 2023-07-14T11:38:01.280 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/da9052_bl.ko 2023-07-14T11:38:01.280 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/gpio_backlight.ko 2023-07-14T11:38:01.281 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/kb3886_bl.ko 2023-07-14T11:38:01.281 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lcd.ko 2023-07-14T11:38:01.281 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lm3533_bl.ko 2023-07-14T11:38:01.281 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lm3639_bl.ko 2023-07-14T11:38:01.281 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lv5207lp.ko 2023-07-14T11:38:01.281 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/max8925_bl.ko 2023-07-14T11:38:01.281 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/pandora_bl.ko 2023-07-14T11:38:01.281 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-07-14T11:38:01.282 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-14T11:38:01.282 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/wm831x_bl.ko 2023-07-14T11:38:01.282 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/ 2023-07-14T11:38:01.282 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ 2023-07-14T11:38:01.282 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2482.ko 2023-07-14T11:38:01.282 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2490.ko 2023-07-14T11:38:01.282 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/matrox_w1.ko 2023-07-14T11:38:01.283 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/w1-gpio.ko 2023-07-14T11:38:01.283 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/ 2023-07-14T11:38:01.283 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-14T11:38:01.283 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-14T11:38:01.283 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-14T11:38:01.283 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-14T11:38:01.283 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-14T11:38:01.284 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-14T11:38:01.284 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-14T11:38:01.284 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-14T11:38:01.284 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-14T11:38:01.284 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-14T11:38:01.284 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 56425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-14T11:38:01.284 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 102657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/wire.ko 2023-07-14T11:38:01.284 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ 2023-07-14T11:38:01.285 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/acquirewdt.ko 2023-07-14T11:38:01.285 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19977 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/advantechwdt.ko 2023-07-14T11:38:01.285 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-14T11:38:01.285 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-14T11:38:01.285 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/cpu5wdt.ko 2023-07-14T11:38:01.285 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/da9052_wdt.ko 2023-07-14T11:38:01.285 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/da9055_wdt.ko 2023-07-14T11:38:01.285 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/da9063_wdt.ko 2023-07-14T11:38:01.286 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/dw_wdt.ko 2023-07-14T11:38:01.286 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/eurotechwdt.ko 2023-07-14T11:38:01.286 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-14T11:38:01.286 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/hpwdt.ko 2023-07-14T11:38:01.286 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/i6300esb.ko 2023-07-14T11:38:01.286 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-14T11:38:01.286 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-14T11:38:01.286 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ib700wdt.ko 2023-07-14T11:38:01.287 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ibmasr.ko 2023-07-14T11:38:01.287 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-14T11:38:01.287 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-14T11:38:01.287 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it87_wdt.ko 2023-07-14T11:38:01.287 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/kempld_wdt.ko 2023-07-14T11:38:01.287 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/machzwd.ko 2023-07-14T11:38:01.287 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/mena21_wdt.ko 2023-07-14T11:38:01.287 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-07-14T11:38:01.288 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/nv_tco.ko 2023-07-14T11:38:01.288 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-07-14T11:38:01.288 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pc87413_wdt.ko 2023-07-14T11:38:01.288 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-14T11:38:01.288 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-14T11:38:01.288 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/retu_wdt.ko 2023-07-14T11:38:01.288 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-07-14T11:38:01.288 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-07-14T11:38:01.289 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-14T11:38:01.289 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sc1200wdt.ko 2023-07-14T11:38:01.289 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-14T11:38:01.289 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-07-14T11:38:01.289 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/softdog.ko 2023-07-14T11:38:01.289 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-14T11:38:01.289 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/twl4030_wdt.ko 2023-07-14T11:38:01.290 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/via_wdt.ko 2023-07-14T11:38:01.290 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-14T11:38:01.290 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-14T11:38:01.290 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-14T11:38:01.290 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wafer5823wdt.ko 2023-07-14T11:38:01.290 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wdt_pci.ko 2023-07-14T11:38:01.290 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wm831x_wdt.ko 2023-07-14T11:38:01.290 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wm8350_wdt.ko 2023-07-14T11:38:01.291 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/xen_wdt.ko 2023-07-14T11:38:01.291 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/ 2023-07-14T11:38:01.291 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-evtchn.ko 2023-07-14T11:38:01.291 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntalloc.ko 2023-07-14T11:38:01.291 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntdev.ko 2023-07-14T11:38:01.291 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/ 2023-07-14T11:38:01.291 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 176417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-14T11:38:01.291 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-privcmd.ko 2023-07-14T11:38:01.292 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 71321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-scsiback.ko 2023-07-14T11:38:01.292 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/ 2023-07-14T11:38:01.292 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-14T11:38:01.292 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ 2023-07-14T11:38:01.292 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/ 2023-07-14T11:38:01.292 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 165697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/9p.ko 2023-07-14T11:38:01.292 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/adfs/ 2023-07-14T11:38:01.293 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/adfs/adfs.ko 2023-07-14T11:38:01.293 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/ 2023-07-14T11:38:01.293 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 162617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/affs.ko 2023-07-14T11:38:01.293 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/afs/ 2023-07-14T11:38:01.293 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 931937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/afs/kafs.ko 2023-07-14T11:38:01.293 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/autofs/ 2023-07-14T11:38:01.293 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 100937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/autofs/autofs4.ko 2023-07-14T11:38:01.293 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/ 2023-07-14T11:38:01.294 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 64481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/befs.ko 2023-07-14T11:38:01.294 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/bfs/ 2023-07-14T11:38:01.294 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/bfs/bfs.ko 2023-07-14T11:38:01.294 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/binfmt_misc.ko 2023-07-14T11:38:01.294 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/ 2023-07-14T11:38:01.294 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 3468681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/btrfs.ko 2023-07-14T11:38:01.294 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/ 2023-07-14T11:38:01.294 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 305745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/cachefiles.ko 2023-07-14T11:38:01.295 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ 2023-07-14T11:38:01.295 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1312521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ceph.ko 2023-07-14T11:38:01.295 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/ 2023-07-14T11:38:01.295 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 100321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/coda.ko 2023-07-14T11:38:01.295 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/configfs/ 2023-07-14T11:38:01.295 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 108073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/configfs/configfs.ko 2023-07-14T11:38:01.295 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/ 2023-07-14T11:38:01.295 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/cramfs.ko 2023-07-14T11:38:01.296 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/ 2023-07-14T11:38:01.296 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 633825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/dlm.ko 2023-07-14T11:38:01.296 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/efs/ 2023-07-14T11:38:01.296 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/efs/efs.ko 2023-07-14T11:38:01.296 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/ 2023-07-14T11:38:01.296 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1500513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/f2fs.ko 2023-07-14T11:38:01.325 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/ 2023-07-14T11:38:01.325 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/msdos.ko 2023-07-14T11:38:01.325 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/freevxfs/ 2023-07-14T11:38:01.326 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/freevxfs/freevxfs.ko 2023-07-14T11:38:01.326 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/ 2023-07-14T11:38:01.326 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 226505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/fscache.ko 2023-07-14T11:38:01.326 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/ 2023-07-14T11:38:01.326 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/cuse.ko 2023-07-14T11:38:01.326 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/ 2023-07-14T11:38:01.326 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1226249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/gfs2.ko 2023-07-14T11:38:01.327 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/ 2023-07-14T11:38:01.327 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 141241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/hfs.ko 2023-07-14T11:38:01.327 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/ 2023-07-14T11:38:01.327 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 221553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/hfsplus.ko 2023-07-14T11:38:01.327 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hpfs/ 2023-07-14T11:38:01.327 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 193577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hpfs/hpfs.ko 2023-07-14T11:38:01.327 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/ 2023-07-14T11:38:01.327 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 74393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/isofs.ko 2023-07-14T11:38:01.328 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/ 2023-07-14T11:38:01.328 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 420745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/jfs.ko 2023-07-14T11:38:01.328 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/ 2023-07-14T11:38:01.328 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 279041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/lockd.ko 2023-07-14T11:38:01.328 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/ 2023-07-14T11:38:01.328 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 87897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/minix.ko 2023-07-14T11:38:01.328 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/ 2023-07-14T11:38:01.329 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 124697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/netfs.ko 2023-07-14T11:38:01.329 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/ 2023-07-14T11:38:01.329 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/ 2023-07-14T11:38:01.329 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 81521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-14T11:38:01.329 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/ 2023-07-14T11:38:01.329 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-14T11:38:01.329 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/ 2023-07-14T11:38:01.329 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 117705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-14T11:38:01.330 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1053889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfs.ko 2023-07-14T11:38:01.330 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv2.ko 2023-07-14T11:38:01.330 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 119801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv3.ko 2023-07-14T11:38:01.330 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 2041809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv4.ko 2023-07-14T11:38:01.330 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/ 2023-07-14T11:38:01.330 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/grace.ko 2023-07-14T11:38:01.330 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/nfs_acl.ko 2023-07-14T11:38:01.330 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/ 2023-07-14T11:38:01.331 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1504993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/nfsd.ko 2023-07-14T11:38:01.331 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/ 2023-07-14T11:38:01.331 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 591417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/nilfs2.ko 2023-07-14T11:38:01.331 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/ 2023-07-14T11:38:01.331 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-celtic.ko 2023-07-14T11:38:01.331 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-centeuro.ko 2023-07-14T11:38:01.331 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-croatian.ko 2023-07-14T11:38:01.331 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-cyrillic.ko 2023-07-14T11:38:01.332 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-gaelic.ko 2023-07-14T11:38:01.332 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-greek.ko 2023-07-14T11:38:01.332 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-iceland.ko 2023-07-14T11:38:01.332 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-inuit.ko 2023-07-14T11:38:01.332 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-roman.ko 2023-07-14T11:38:01.332 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-romanian.ko 2023-07-14T11:38:01.332 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-turkish.ko 2023-07-14T11:38:01.333 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_ascii.ko 2023-07-14T11:38:01.333 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1250.ko 2023-07-14T11:38:01.333 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1251.ko 2023-07-14T11:38:01.333 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1255.ko 2023-07-14T11:38:01.333 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp737.ko 2023-07-14T11:38:01.333 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp775.ko 2023-07-14T11:38:01.333 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp850.ko 2023-07-14T11:38:01.333 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp852.ko 2023-07-14T11:38:01.334 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp855.ko 2023-07-14T11:38:01.334 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp857.ko 2023-07-14T11:38:01.334 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp860.ko 2023-07-14T11:38:01.334 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp861.ko 2023-07-14T11:38:01.334 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp862.ko 2023-07-14T11:38:01.334 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp863.ko 2023-07-14T11:38:01.334 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp864.ko 2023-07-14T11:38:01.335 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp865.ko 2023-07-14T11:38:01.335 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp866.ko 2023-07-14T11:38:01.335 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp869.ko 2023-07-14T11:38:01.335 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp874.ko 2023-07-14T11:38:01.335 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 94137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp932.ko 2023-07-14T11:38:01.335 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 142673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp936.ko 2023-07-14T11:38:01.336 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 165745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp949.ko 2023-07-14T11:38:01.336 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 118121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp950.ko 2023-07-14T11:38:01.336 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_euc-jp.ko 2023-07-14T11:38:01.336 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-1.ko 2023-07-14T11:38:01.336 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-13.ko 2023-07-14T11:38:01.336 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-14.ko 2023-07-14T11:38:01.336 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-15.ko 2023-07-14T11:38:01.337 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-2.ko 2023-07-14T11:38:01.337 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-3.ko 2023-07-14T11:38:01.337 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-4.ko 2023-07-14T11:38:01.337 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-5.ko 2023-07-14T11:38:01.337 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-6.ko 2023-07-14T11:38:01.337 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-7.ko 2023-07-14T11:38:01.337 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-9.ko 2023-07-14T11:38:01.338 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-r.ko 2023-07-14T11:38:01.338 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-ru.ko 2023-07-14T11:38:01.338 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-u.ko 2023-07-14T11:38:01.338 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_utf8.ko 2023-07-14T11:38:01.338 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ntfs/ 2023-07-14T11:38:01.338 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 219073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ntfs/ntfs.ko 2023-07-14T11:38:01.338 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ 2023-07-14T11:38:01.338 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ 2023-07-14T11:38:01.339 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 244177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-14T11:38:01.339 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ 2023-07-14T11:38:01.339 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 440785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-14T11:38:01.339 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ 2023-07-14T11:38:01.339 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-14T11:38:01.339 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 2589569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2.ko 2023-07-14T11:38:01.339 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-14T11:38:01.340 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-14T11:38:01.340 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-14T11:38:01.340 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/omfs/ 2023-07-14T11:38:01.340 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 52569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/omfs/omfs.ko 2023-07-14T11:38:01.340 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/ 2023-07-14T11:38:01.340 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 336473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/overlay.ko 2023-07-14T11:38:01.340 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ 2023-07-14T11:38:01.341 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ramoops.ko 2023-07-14T11:38:01.341 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/qnx4/ 2023-07-14T11:38:01.341 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/qnx4/qnx4.ko 2023-07-14T11:38:01.341 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/qnx6/ 2023-07-14T11:38:01.341 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/qnx6/qnx6.ko 2023-07-14T11:38:01.341 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/quota/ 2023-07-14T11:38:01.341 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/quota/quota_tree.ko 2023-07-14T11:38:01.341 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/quota/quota_v1.ko 2023-07-14T11:38:01.342 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/quota/quota_v2.ko 2023-07-14T11:38:01.342 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/ 2023-07-14T11:38:01.342 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 530697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/reiserfs.ko 2023-07-14T11:38:01.342 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/ 2023-07-14T11:38:01.342 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/romfs.ko 2023-07-14T11:38:01.342 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/ 2023-07-14T11:38:01.342 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/ 2023-07-14T11:38:01.343 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 2658777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/cifs.ko 2023-07-14T11:38:01.343 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/ 2023-07-14T11:38:01.343 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_arc4.ko 2023-07-14T11:38:01.343 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_md4.ko 2023-07-14T11:38:01.343 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/ 2023-07-14T11:38:01.343 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 134313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/squashfs.ko 2023-07-14T11:38:01.343 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/ 2023-07-14T11:38:01.343 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 89737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/sysv.ko 2023-07-14T11:38:01.344 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/ 2023-07-14T11:38:01.344 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 248049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/udf.ko 2023-07-14T11:38:01.344 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ 2023-07-14T11:38:01.344 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 160857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ufs.ko 2023-07-14T11:38:01.344 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/ 2023-07-14T11:38:01.344 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 3998129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/xfs.ko 2023-07-14T11:38:01.344 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/ 2023-07-14T11:38:01.344 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/locking/ 2023-07-14T11:38:01.345 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 74865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/locking/locktorture.ko 2023-07-14T11:38:01.345 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/time/ 2023-07-14T11:38:01.345 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/time/test_udelay.ko 2023-07-14T11:38:01.345 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 58473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/torture.ko 2023-07-14T11:38:01.345 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ 2023-07-14T11:38:01.345 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 6833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc-itu-t.ko 2023-07-14T11:38:01.345 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 6529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc7.ko 2023-07-14T11:38:01.346 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc8.ko 2023-07-14T11:38:01.346 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/ 2023-07-14T11:38:01.346 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libarc4.ko 2023-07-14T11:38:01.346 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libdes.ko 2023-07-14T11:38:01.346 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/interval_tree_test.ko 2023-07-14T11:38:01.346 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/libcrc32c.ko 2023-07-14T11:38:01.346 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lru_cache.ko 2023-07-14T11:38:01.346 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/ 2023-07-14T11:38:01.347 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4_compress.ko 2023-07-14T11:38:01.347 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22633 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4hc_compress.ko 2023-07-14T11:38:01.347 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/ 2023-07-14T11:38:01.347 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 6617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/cordic.ko 2023-07-14T11:38:01.347 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/memory-notifier-error-inject.ko 2023-07-14T11:38:01.347 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/notifier-error-inject.ko 2023-07-14T11:38:01.347 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/percpu_test.ko 2023-07-14T11:38:01.348 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/pm-notifier-error-inject.ko 2023-07-14T11:38:01.348 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/ 2023-07-14T11:38:01.348 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 143433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/raid6_pq.ko 2023-07-14T11:38:01.348 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/rbtree_test.ko 2023-07-14T11:38:01.348 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/ 2023-07-14T11:38:01.348 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-14T11:38:01.348 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/test-kstrtox.ko 2023-07-14T11:38:01.348 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/test-string_helpers.ko 2023-07-14T11:38:01.349 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 4760825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/test_bpf.ko 2023-07-14T11:38:01.349 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/test_firmware.ko 2023-07-14T11:38:01.349 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 6369 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/test_module.ko 2023-07-14T11:38:01.349 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/test_user_copy.ko 2023-07-14T11:38:01.349 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_bm.ko 2023-07-14T11:38:01.349 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_fsm.ko 2023-07-14T11:38:01.349 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_kmp.ko 2023-07-14T11:38:01.349 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/xz/ 2023-07-14T11:38:01.350 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/xz/xz_dec_test.ko 2023-07-14T11:38:01.350 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/ 2023-07-14T11:38:01.350 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/hwpoison-inject.ko 2023-07-14T11:38:01.350 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ 2023-07-14T11:38:01.350 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/ 2023-07-14T11:38:01.350 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 55737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/6lowpan.ko 2023-07-14T11:38:01.350 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_dest.ko 2023-07-14T11:38:01.351 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_fragment.ko 2023-07-14T11:38:01.351 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_hop.ko 2023-07-14T11:38:01.351 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 6985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-14T11:38:01.351 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_mobility.ko 2023-07-14T11:38:01.351 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_routing.ko 2023-07-14T11:38:01.351 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_udp.ko 2023-07-14T11:38:01.351 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/ 2023-07-14T11:38:01.351 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/garp.ko 2023-07-14T11:38:01.352 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/mrp.ko 2023-07-14T11:38:01.352 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/p8022.ko 2023-07-14T11:38:01.352 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/psnap.ko 2023-07-14T11:38:01.352 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/stp.ko 2023-07-14T11:38:01.352 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/ 2023-07-14T11:38:01.352 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 81017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/8021q.ko 2023-07-14T11:38:01.352 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/ 2023-07-14T11:38:01.353 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 198417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet.ko 2023-07-14T11:38:01.353 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_fd.ko 2023-07-14T11:38:01.353 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_rdma.ko 2023-07-14T11:38:01.353 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_virtio.ko 2023-07-14T11:38:01.353 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/ 2023-07-14T11:38:01.353 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 652553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/batman-adv.ko 2023-07-14T11:38:01.353 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/ 2023-07-14T11:38:01.353 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 63825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/br_netfilter.ko 2023-07-14T11:38:01.354 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 772505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/bridge.ko 2023-07-14T11:38:01.354 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ 2023-07-14T11:38:01.354 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-14T11:38:01.354 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-14T11:38:01.354 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-14T11:38:01.354 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-14T11:38:01.354 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-14T11:38:01.355 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-14T11:38:01.355 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-14T11:38:01.355 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-14T11:38:01.355 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-14T11:38:01.355 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-14T11:38:01.355 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-14T11:38:01.355 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-14T11:38:01.356 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-14T11:38:01.356 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-14T11:38:01.356 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-14T11:38:01.356 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-14T11:38:01.356 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-14T11:38:01.356 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-14T11:38:01.356 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-14T11:38:01.357 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-14T11:38:01.357 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 55937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtables.ko 2023-07-14T11:38:01.357 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-14T11:38:01.357 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-14T11:38:01.357 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/ 2023-07-14T11:38:01.357 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1046417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/libceph.ko 2023-07-14T11:38:01.357 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/ 2023-07-14T11:38:01.357 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 124793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/pktgen.ko 2023-07-14T11:38:01.358 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/ 2023-07-14T11:38:01.358 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 194321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp.ko 2023-07-14T11:38:01.358 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_diag.ko 2023-07-14T11:38:01.358 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 56945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv4.ko 2023-07-14T11:38:01.358 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv6.ko 2023-07-14T11:38:01.358 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/ 2023-07-14T11:38:01.358 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 390297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/dsa_core.ko 2023-07-14T11:38:01.358 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_brcm.ko 2023-07-14T11:38:01.359 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_dsa.ko 2023-07-14T11:38:01.359 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_none.ko 2023-07-14T11:38:01.359 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_trailer.ko 2023-07-14T11:38:01.359 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/hsr/ 2023-07-14T11:38:01.359 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 100257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/hsr/hsr.ko 2023-07-14T11:38:01.359 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ 2023-07-14T11:38:01.359 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ 2023-07-14T11:38:01.359 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 45169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-14T11:38:01.360 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 254601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154.ko 2023-07-14T11:38:01.360 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-14T11:38:01.360 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ 2023-07-14T11:38:01.360 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ah4.ko 2023-07-14T11:38:01.360 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/esp4.ko 2023-07-14T11:38:01.360 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/fou.ko 2023-07-14T11:38:01.360 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/gre.ko 2023-07-14T11:38:01.361 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/inet_diag.ko 2023-07-14T11:38:01.361 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_gre.ko 2023-07-14T11:38:01.361 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 61281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_tunnel.ko 2023-07-14T11:38:01.361 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34369 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_vti.ko 2023-07-14T11:38:01.361 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipcomp.ko 2023-07-14T11:38:01.361 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipip.ko 2023-07-14T11:38:01.361 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ 2023-07-14T11:38:01.361 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-14T11:38:01.362 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-14T11:38:01.362 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-14T11:38:01.362 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ip_tables.ko 2023-07-14T11:38:01.362 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-14T11:38:01.362 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-07-14T11:38:01.362 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-14T11:38:01.362 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-14T11:38:01.362 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-14T11:38:01.363 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-07-14T11:38:01.363 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-14T11:38:01.363 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-14T11:38:01.363 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-14T11:38:01.363 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-14T11:38:01.363 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-14T11:38:01.363 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-14T11:38:01.364 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-14T11:38:01.364 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-14T11:38:01.364 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-14T11:38:01.364 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-07-14T11:38:01.364 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-14T11:38:01.364 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-14T11:38:01.364 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-14T11:38:01.364 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-14T11:38:01.365 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-14T11:38:01.365 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_bic.ko 2023-07-14T11:38:01.365 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_dctcp.ko 2023-07-14T11:38:01.365 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_diag.ko 2023-07-14T11:38:01.365 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_highspeed.ko 2023-07-14T11:38:01.365 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_htcp.ko 2023-07-14T11:38:01.365 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_hybla.ko 2023-07-14T11:38:01.365 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_illinois.ko 2023-07-14T11:38:01.366 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_lp.ko 2023-07-14T11:38:01.366 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_scalable.ko 2023-07-14T11:38:01.366 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_vegas.ko 2023-07-14T11:38:01.366 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_veno.ko 2023-07-14T11:38:01.366 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_westwood.ko 2023-07-14T11:38:01.366 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_yeah.ko 2023-07-14T11:38:01.366 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tunnel4.ko 2023-07-14T11:38:01.366 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_diag.ko 2023-07-14T11:38:01.367 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_tunnel.ko 2023-07-14T11:38:01.367 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-14T11:38:01.367 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ 2023-07-14T11:38:01.367 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ah6.ko 2023-07-14T11:38:01.367 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/esp6.ko 2023-07-14T11:38:01.367 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/fou6.ko 2023-07-14T11:38:01.367 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 80793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_gre.ko 2023-07-14T11:38:01.367 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 92817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_tunnel.ko 2023-07-14T11:38:01.368 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-14T11:38:01.368 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_vti.ko 2023-07-14T11:38:01.368 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ipcomp6.ko 2023-07-14T11:38:01.368 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/mip6.ko 2023-07-14T11:38:01.368 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ 2023-07-14T11:38:01.368 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-14T11:38:01.368 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-07-14T11:38:01.369 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-14T11:38:01.369 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-14T11:38:01.369 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-14T11:38:01.369 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-14T11:38:01.369 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-14T11:38:01.369 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-14T11:38:01.369 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-14T11:38:01.369 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-14T11:38:01.370 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-14T11:38:01.370 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-14T11:38:01.370 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-14T11:38:01.370 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-14T11:38:01.370 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-14T11:38:01.370 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-14T11:38:01.370 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-14T11:38:01.370 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-14T11:38:01.371 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-14T11:38:01.371 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-14T11:38:01.371 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-14T11:38:01.371 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-14T11:38:01.371 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9633 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-14T11:38:01.371 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-14T11:38:01.371 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-14T11:38:01.371 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 73161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/sit.ko 2023-07-14T11:38:01.372 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/tunnel6.ko 2023-07-14T11:38:01.372 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-14T11:38:01.372 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/ 2023-07-14T11:38:01.372 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 82929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/af_key.ko 2023-07-14T11:38:01.372 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/ 2023-07-14T11:38:01.372 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 125873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_core.ko 2023-07-14T11:38:01.372 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-14T11:38:01.373 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_eth.ko 2023-07-14T11:38:01.373 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip.ko 2023-07-14T11:38:01.373 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip6.ko 2023-07-14T11:38:01.373 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_netlink.ko 2023-07-14T11:38:01.373 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ppp.ko 2023-07-14T11:38:01.373 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/lapb/ 2023-07-14T11:38:01.373 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/lapb/lapb.ko 2023-07-14T11:38:01.374 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/ 2023-07-14T11:38:01.374 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc.ko 2023-07-14T11:38:01.374 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 267017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc2.ko 2023-07-14T11:38:01.374 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/ 2023-07-14T11:38:01.374 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 291041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/mac802154.ko 2023-07-14T11:38:01.374 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/ 2023-07-14T11:38:01.374 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/mpls_gso.ko 2023-07-14T11:38:01.374 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ 2023-07-14T11:38:01.375 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ 2023-07-14T11:38:01.375 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 119649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set.ko 2023-07-14T11:38:01.375 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-14T11:38:01.375 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-14T11:38:01.375 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-14T11:38:01.375 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 102033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-14T11:38:01.375 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 103121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-14T11:38:01.375 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 104593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-14T11:38:01.376 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 105129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-14T11:38:01.376 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 114393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-14T11:38:01.376 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 61161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-14T11:38:01.376 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 111161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-14T11:38:01.376 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 114673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-14T11:38:01.376 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 114329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-14T11:38:01.376 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 113217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-14T11:38:01.377 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 116217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-14T11:38:01.377 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-14T11:38:01.377 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ 2023-07-14T11:38:01.377 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 412481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-14T11:38:01.377 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-14T11:38:01.377 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-14T11:38:01.377 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-14T11:38:01.377 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-14T11:38:01.378 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-14T11:38:01.378 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-14T11:38:01.378 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-14T11:38:01.378 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-14T11:38:01.378 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-14T11:38:01.378 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-14T11:38:01.378 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-14T11:38:01.378 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-14T11:38:01.379 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-14T11:38:01.379 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conncount.ko 2023-07-14T11:38:01.379 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 408689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack.ko 2023-07-14T11:38:01.379 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-14T11:38:01.379 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-14T11:38:01.379 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-14T11:38:01.379 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 118145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-14T11:38:01.380 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-14T11:38:01.380 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-14T11:38:01.380 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 108393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-14T11:38:01.380 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-14T11:38:01.380 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-14T11:38:01.381 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-14T11:38:01.381 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-14T11:38:01.381 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-14T11:38:01.381 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-14T11:38:01.381 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 88513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table.ko 2023-07-14T11:38:01.381 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-14T11:38:01.381 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_log_syslog.ko 2023-07-14T11:38:01.382 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 95353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat.ko 2023-07-14T11:38:01.382 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-14T11:38:01.382 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-14T11:38:01.382 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_irc.ko 2023-07-14T11:38:01.382 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_sip.ko 2023-07-14T11:38:01.382 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-14T11:38:01.382 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-14T11:38:01.382 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 592185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_tables.ko 2023-07-14T11:38:01.383 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink.ko 2023-07-14T11:38:01.383 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-14T11:38:01.383 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-07-14T11:38:01.383 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_log.ko 2023-07-14T11:38:01.383 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-14T11:38:01.383 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-14T11:38:01.383 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_chain_nat.ko 2023-07-14T11:38:01.383 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_compat.ko 2023-07-14T11:38:01.384 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_ct.ko 2023-07-14T11:38:01.384 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-14T11:38:01.384 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fib.ko 2023-07-14T11:38:01.384 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-14T11:38:01.384 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_hash.ko 2023-07-14T11:38:01.384 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_limit.ko 2023-07-14T11:38:01.384 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_log.ko 2023-07-14T11:38:01.385 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_masq.ko 2023-07-14T11:38:01.385 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_nat.ko 2023-07-14T11:38:01.385 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_numgen.ko 2023-07-14T11:38:01.385 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_osf.ko 2023-07-14T11:38:01.385 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_queue.ko 2023-07-14T11:38:01.385 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_redir.ko 2023-07-14T11:38:01.385 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject.ko 2023-07-14T11:38:01.385 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject_inet.ko 2023-07-14T11:38:01.386 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_socket.ko 2023-07-14T11:38:01.386 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_synproxy.ko 2023-07-14T11:38:01.386 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_tproxy.ko 2023-07-14T11:38:01.386 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 89329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/x_tables.ko 2023-07-14T11:38:01.386 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_AUDIT.ko 2023-07-14T11:38:01.386 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9633 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-14T11:38:01.386 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-14T11:38:01.387 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-14T11:38:01.387 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CT.ko 2023-07-14T11:38:01.387 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_DSCP.ko 2023-07-14T11:38:01.387 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HL.ko 2023-07-14T11:38:01.387 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HMARK.ko 2023-07-14T11:38:01.387 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-14T11:38:01.387 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LED.ko 2023-07-14T11:38:01.387 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LOG.ko 2023-07-14T11:38:01.388 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-14T11:38:01.388 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NETMAP.ko 2023-07-14T11:38:01.388 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFLOG.ko 2023-07-14T11:38:01.388 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-14T11:38:01.388 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_RATEEST.ko 2023-07-14T11:38:01.388 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-14T11:38:01.388 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_SECMARK.ko 2023-07-14T11:38:01.388 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-14T11:38:01.389 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-14T11:38:01.389 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17369 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TEE.ko 2023-07-14T11:38:01.389 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TPROXY.ko 2023-07-14T11:38:01.389 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TRACE.ko 2023-07-14T11:38:01.389 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_addrtype.ko 2023-07-14T11:38:01.389 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_bpf.ko 2023-07-14T11:38:01.390 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cgroup.ko 2023-07-14T11:38:01.390 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cluster.ko 2023-07-14T11:38:01.390 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_comment.ko 2023-07-14T11:38:01.390 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connbytes.ko 2023-07-14T11:38:01.390 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlabel.ko 2023-07-14T11:38:01.390 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlimit.ko 2023-07-14T11:38:01.390 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connmark.ko 2023-07-14T11:38:01.391 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_conntrack.ko 2023-07-14T11:38:01.391 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cpu.ko 2023-07-14T11:38:01.391 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dccp.ko 2023-07-14T11:38:01.391 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_devgroup.ko 2023-07-14T11:38:01.391 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dscp.ko 2023-07-14T11:38:01.391 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ecn.ko 2023-07-14T11:38:01.391 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_esp.ko 2023-07-14T11:38:01.392 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hashlimit.ko 2023-07-14T11:38:01.392 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_helper.ko 2023-07-14T11:38:01.392 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hl.ko 2023-07-14T11:38:01.392 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipcomp.ko 2023-07-14T11:38:01.392 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_iprange.ko 2023-07-14T11:38:01.392 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipvs.ko 2023-07-14T11:38:01.392 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_l2tp.ko 2023-07-14T11:38:01.392 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_length.ko 2023-07-14T11:38:01.393 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_limit.ko 2023-07-14T11:38:01.393 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mac.ko 2023-07-14T11:38:01.393 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mark.ko 2023-07-14T11:38:01.393 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_multiport.ko 2023-07-14T11:38:01.393 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nat.ko 2023-07-14T11:38:01.393 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nfacct.ko 2023-07-14T11:38:01.394 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_osf.ko 2023-07-14T11:38:01.394 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_owner.ko 2023-07-14T11:38:01.394 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_physdev.ko 2023-07-14T11:38:01.394 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7633 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_pkttype.ko 2023-07-14T11:38:01.394 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_policy.ko 2023-07-14T11:38:01.394 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_quota.ko 2023-07-14T11:38:01.394 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_rateest.ko 2023-07-14T11:38:01.394 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_realm.ko 2023-07-14T11:38:01.395 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_recent.ko 2023-07-14T11:38:01.395 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_sctp.ko 2023-07-14T11:38:01.395 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_set.ko 2023-07-14T11:38:01.395 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_socket.ko 2023-07-14T11:38:01.395 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_state.ko 2023-07-14T11:38:01.395 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_statistic.ko 2023-07-14T11:38:01.395 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_string.ko 2023-07-14T11:38:01.395 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpmss.ko 2023-07-14T11:38:01.396 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpudp.ko 2023-07-14T11:38:01.396 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_time.ko 2023-07-14T11:38:01.396 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_u32.ko 2023-07-14T11:38:01.396 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/ 2023-07-14T11:38:01.396 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/netlink_diag.ko 2023-07-14T11:38:01.396 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/ 2023-07-14T11:38:01.396 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/nsh.ko 2023-07-14T11:38:01.396 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/ 2023-07-14T11:38:01.397 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 379545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/openvswitch.ko 2023-07-14T11:38:01.397 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-geneve.ko 2023-07-14T11:38:01.397 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-gre.ko 2023-07-14T11:38:01.397 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-vxlan.ko 2023-07-14T11:38:01.397 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/ 2023-07-14T11:38:01.397 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/af_packet_diag.ko 2023-07-14T11:38:01.398 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/phonet/ 2023-07-14T11:38:01.398 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 109961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/phonet/phonet.ko 2023-07-14T11:38:01.398 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/phonet/pn_pep.ko 2023-07-14T11:38:01.398 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/ 2023-07-14T11:38:01.398 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 252825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds.ko 2023-07-14T11:38:01.398 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 165081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_rdma.ko 2023-07-14T11:38:01.398 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_tcp.ko 2023-07-14T11:38:01.398 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rxrpc/ 2023-07-14T11:38:01.399 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 820137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rxrpc/rxrpc.ko 2023-07-14T11:38:01.399 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/ 2023-07-14T11:38:01.399 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_csum.ko 2023-07-14T11:38:01.399 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_gact.ko 2023-07-14T11:38:01.399 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_ipt.ko 2023-07-14T11:38:01.399 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_mirred.ko 2023-07-14T11:38:01.399 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_nat.ko 2023-07-14T11:38:01.399 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_pedit.ko 2023-07-14T11:38:01.400 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_police.ko 2023-07-14T11:38:01.400 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_simple.ko 2023-07-14T11:38:01.400 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_skbedit.ko 2023-07-14T11:38:01.400 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_vlan.ko 2023-07-14T11:38:01.400 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_basic.ko 2023-07-14T11:38:01.400 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_bpf.ko 2023-07-14T11:38:01.401 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_cgroup.ko 2023-07-14T11:38:01.401 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_flow.ko 2023-07-14T11:38:01.401 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_fw.ko 2023-07-14T11:38:01.401 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_route.ko 2023-07-14T11:38:01.401 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_u32.ko 2023-07-14T11:38:01.401 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_cmp.ko 2023-07-14T11:38:01.401 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_ipset.ko 2023-07-14T11:38:01.401 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_meta.ko 2023-07-14T11:38:01.402 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_nbyte.ko 2023-07-14T11:38:01.402 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_text.ko 2023-07-14T11:38:01.402 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_u32.ko 2023-07-14T11:38:01.402 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_choke.ko 2023-07-14T11:38:01.402 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_codel.ko 2023-07-14T11:38:01.402 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_drr.ko 2023-07-14T11:38:01.402 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq.ko 2023-07-14T11:38:01.403 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq_codel.ko 2023-07-14T11:38:01.403 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_gred.ko 2023-07-14T11:38:01.403 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hfsc.ko 2023-07-14T11:38:01.403 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hhf.ko 2023-07-14T11:38:01.403 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 65793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_htb.ko 2023-07-14T11:38:01.403 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_ingress.ko 2023-07-14T11:38:01.403 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio.ko 2023-07-14T11:38:01.404 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio_lib.ko 2023-07-14T11:38:01.404 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_multiq.ko 2023-07-14T11:38:01.404 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_netem.ko 2023-07-14T11:38:01.404 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_pie.ko 2023-07-14T11:38:01.404 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_plug.ko 2023-07-14T11:38:01.404 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_prio.ko 2023-07-14T11:38:01.404 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_qfq.ko 2023-07-14T11:38:01.404 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_red.ko 2023-07-14T11:38:01.405 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfb.ko 2023-07-14T11:38:01.405 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfq.ko 2023-07-14T11:38:01.405 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_tbf.ko 2023-07-14T11:38:01.405 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_teql.ko 2023-07-14T11:38:01.405 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/ 2023-07-14T11:38:01.405 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 930369 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp.ko 2023-07-14T11:38:01.405 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp_diag.ko 2023-07-14T11:38:01.406 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/ 2023-07-14T11:38:01.406 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/ 2023-07-14T11:38:01.406 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 335569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-14T11:38:01.406 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 61833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-14T11:38:01.406 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1514649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/sunrpc.ko 2023-07-14T11:38:01.406 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/ 2023-07-14T11:38:01.406 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 707401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-14T11:38:01.406 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/ 2023-07-14T11:38:01.407 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/diag.ko 2023-07-14T11:38:01.407 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 862233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/tipc.ko 2023-07-14T11:38:01.407 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/ 2023-07-14T11:38:01.407 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/unix_diag.ko 2023-07-14T11:38:01.407 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/ 2023-07-14T11:38:01.407 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 84841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-14T11:38:01.407 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 68033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-14T11:38:01.407 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 95985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock.ko 2023-07-14T11:38:01.408 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-14T11:38:01.408 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-14T11:38:01.408 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/x25/ 2023-07-14T11:38:01.408 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 125097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/x25/x25.ko 2023-07-14T11:38:01.408 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/ 2023-07-14T11:38:01.408 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_algo.ko 2023-07-14T11:38:01.408 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-14T11:38:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 102265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_user.ko 2023-07-14T11:38:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/ 2023-07-14T11:38:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/ 2023-07-14T11:38:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/irqbypass.ko 2023-07-14T11:38:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 638200 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias 2023-07-14T11:38:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 623896 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias.bin 2023-07-14T11:38:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7656 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin 2023-07-14T11:38:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.alias.bin 2023-07-14T11:38:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9821 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.bin 2023-07-14T11:38:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 69903 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.modinfo 2023-07-14T11:38:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 146449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep 2023-07-14T11:38:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 232695 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep.bin 2023-07-14T11:38:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 210 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.devname 2023-07-14T11:38:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 72754 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.order 2023-07-14T11:38:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 879 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.softdep 2023-07-14T11:38:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 252883 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols 2023-07-14T11:38:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 307884 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols.bin 2023-07-14T11:38:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./usr/ 2023-07-14T11:38:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./usr/share/ 2023-07-14T11:38:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./usr/share/doc/ 2023-07-14T11:38:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./usr/share/doc/linux-image-6.5.0-rc1-ga440c79eee02/ 2023-07-14T11:38:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 174 2023-07-13 12:25 ./usr/share/doc/linux-image-6.5.0-rc1-ga440c79eee02/changelog.Debian.gz 2023-07-14T11:38:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 691 2023-07-13 12:25 ./usr/share/doc/linux-image-6.5.0-rc1-ga440c79eee02/copyright 2023-07-14T11:38:01.413 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:01.413 INFO:teuthology.task.kernel:searching for kernel 6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:01.414 DEBUG:teuthology.orchestra.run.smithi121:> egrep '(submenu|menuentry.*6.5.0-rc1-ga440c79eee02).*{' /boot/grub/grub.cfg 2023-07-14T11:38:01.422 INFO:teuthology.orchestra.run.smithi121.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-07-14T11:38:01.423 INFO:teuthology.orchestra.run.smithi121.stdout: menuentry 'Ubuntu, with Linux 6.5.0-rc1-ga440c79eee02' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.5.0-rc1-ga440c79eee02-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-07-14T11:38:01.423 INFO:teuthology.orchestra.run.smithi121.stdout: menuentry 'Ubuntu, with Linux 6.5.0-rc1-ga440c79eee02 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.5.0-rc1-ga440c79eee02-recovery-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-07-14T11:38:01.424 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-07-14T11:38:01.424 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:01.424 DEBUG:teuthology.orchestra.run.smithi121:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-07-14T11:38:01.468 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi121.front.sepia.ceph.com to complete... 2023-07-14T11:38:02.213 INFO:teuthology.orchestra.run.smithi121.stderr:Sourcing file `/etc/default/grub' 2023-07-14T11:38:02.220 INFO:teuthology.orchestra.run.smithi121.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-07-14T11:38:02.231 INFO:teuthology.orchestra.run.smithi121.stderr:Generating grub configuration file ... 2023-07-14T11:38:02.465 INFO:teuthology.orchestra.run.smithi121.stderr:Found linux image: /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:02.476 INFO:teuthology.orchestra.run.smithi121.stderr:Found initrd image: /boot/initrd.img-6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:02.840 INFO:teuthology.orchestra.run.smithi121.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-07-14T11:38:02.852 INFO:teuthology.orchestra.run.smithi121.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-07-14T11:38:04.612 INFO:teuthology.orchestra.run.smithi121.stderr:done 2023-07-14T11:38:08.281 INFO:teuthology.orchestra.run.smithi104.stderr:Sourcing file `/etc/default/grub' 2023-07-14T11:38:08.287 INFO:teuthology.orchestra.run.smithi104.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-07-14T11:38:08.314 INFO:teuthology.orchestra.run.smithi104.stderr:Generating grub configuration file ... 2023-07-14T11:38:08.649 INFO:teuthology.orchestra.run.smithi104.stderr:Found linux image: /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:08.671 INFO:teuthology.orchestra.run.smithi104.stderr:Found initrd image: /boot/initrd.img-6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:09.051 INFO:teuthology.orchestra.run.smithi104.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-07-14T11:38:09.065 INFO:teuthology.orchestra.run.smithi104.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-07-14T11:38:11.717 INFO:teuthology.orchestra.run.smithi104.stderr:done 2023-07-14T11:38:11.919 DEBUG:teuthology.orchestra.run.smithi104:> dpkg-deb -c /tmp/linux-image.deb 2023-07-14T11:38:11.928 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./ 2023-07-14T11:38:11.929 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./boot/ 2023-07-14T11:38:11.929 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 4380308 2023-07-13 12:25 ./boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:11.943 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 168433 2023-07-13 12:25 ./boot/config-6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:11.944 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11280128 2023-07-13 12:25 ./boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:11.966 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./etc/ 2023-07-14T11:38:11.967 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./etc/kernel/ 2023-07-14T11:38:11.967 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./etc/kernel/postinst.d/ 2023-07-14T11:38:11.967 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./etc/kernel/postrm.d/ 2023-07-14T11:38:11.967 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./etc/kernel/preinst.d/ 2023-07-14T11:38:11.967 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./etc/kernel/prerm.d/ 2023-07-14T11:38:11.967 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/ 2023-07-14T11:38:11.967 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/ 2023-07-14T11:38:11.968 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/ 2023-07-14T11:38:11.968 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/ 2023-07-14T11:38:11.968 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/ 2023-07-14T11:38:11.968 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/ 2023-07-14T11:38:11.968 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ 2023-07-14T11:38:11.968 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 490385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/aesni-intel.ko 2023-07-14T11:38:11.969 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-14T11:38:11.969 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-14T11:38:11.969 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-14T11:38:11.969 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 93113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-14T11:38:11.969 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 106729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-14T11:38:11.969 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 152241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-14T11:38:11.969 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-14T11:38:11.970 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-14T11:38:11.970 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 83889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-14T11:38:11.970 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-14T11:38:11.970 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-14T11:38:11.971 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-14T11:38:11.972 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-14T11:38:11.972 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-14T11:38:11.972 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-14T11:38:11.972 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 58585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-14T11:38:11.973 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-14T11:38:11.973 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-14T11:38:11.973 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-14T11:38:11.973 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/ 2023-07-14T11:38:11.973 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/ 2023-07-14T11:38:11.973 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/ 2023-07-14T11:38:11.973 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-14T11:38:11.974 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpuid.ko 2023-07-14T11:38:11.974 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/msr.ko 2023-07-14T11:38:11.974 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/ 2023-07-14T11:38:11.974 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 348417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-14T11:38:11.974 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 972641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-14T11:38:11.974 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 2273145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm.ko 2023-07-14T11:38:11.982 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ 2023-07-14T11:38:11.982 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/af_alg.ko 2023-07-14T11:38:11.982 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/algif_hash.ko 2023-07-14T11:38:11.982 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/algif_skcipher.ko 2023-07-14T11:38:11.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ansi_cprng.ko 2023-07-14T11:38:11.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/anubis.ko 2023-07-14T11:38:11.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/arc4.ko 2023-07-14T11:38:11.984 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/asymmetric_keys/ 2023-07-14T11:38:11.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-07-14T11:38:11.984 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/ 2023-07-14T11:38:11.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_memcpy.ko 2023-07-14T11:38:11.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_pq.ko 2023-07-14T11:38:11.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-14T11:38:11.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_tx.ko 2023-07-14T11:38:11.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_xor.ko 2023-07-14T11:38:11.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/raid6test.ko 2023-07-14T11:38:11.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authenc.ko 2023-07-14T11:38:11.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authencesn.ko 2023-07-14T11:38:11.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blake2b_generic.ko 2023-07-14T11:38:11.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_common.ko 2023-07-14T11:38:11.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_generic.ko 2023-07-14T11:38:11.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/camellia_generic.ko 2023-07-14T11:38:11.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast5_generic.ko 2023-07-14T11:38:11.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast6_generic.ko 2023-07-14T11:38:11.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast_common.ko 2023-07-14T11:38:11.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ccm.ko 2023-07-14T11:38:11.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cmac.ko 2023-07-14T11:38:11.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crc32_generic.ko 2023-07-14T11:38:11.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cryptd.ko 2023-07-14T11:38:11.987 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_engine.ko 2023-07-14T11:38:11.987 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_simd.ko 2023-07-14T11:38:11.987 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_user.ko 2023-07-14T11:38:11.987 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cts.ko 2023-07-14T11:38:11.987 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/des_generic.ko 2023-07-14T11:38:11.988 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/dh_generic.ko 2023-07-14T11:38:11.988 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/echainiv.ko 2023-07-14T11:38:11.988 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/essiv.ko 2023-07-14T11:38:11.988 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/fcrypt.ko 2023-07-14T11:38:11.989 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/khazad.ko 2023-07-14T11:38:11.989 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lrw.ko 2023-07-14T11:38:11.989 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4.ko 2023-07-14T11:38:11.989 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4hc.ko 2023-07-14T11:38:11.989 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/md4.ko 2023-07-14T11:38:11.989 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/michael_mic.ko 2023-07-14T11:38:11.990 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcbc.ko 2023-07-14T11:38:11.990 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcrypt.ko 2023-07-14T11:38:11.990 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/rmd160.ko 2023-07-14T11:38:11.990 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/seed.ko 2023-07-14T11:38:11.990 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/serpent_generic.ko 2023-07-14T11:38:11.991 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 149193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tcrypt.ko 2023-07-14T11:38:11.991 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tea.ko 2023-07-14T11:38:11.991 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_common.ko 2023-07-14T11:38:11.991 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_generic.ko 2023-07-14T11:38:11.991 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/vmac.ko 2023-07-14T11:38:11.991 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/wp512.ko 2023-07-14T11:38:11.992 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xcbc.ko 2023-07-14T11:38:11.992 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xor.ko 2023-07-14T11:38:11.992 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xts.ko 2023-07-14T11:38:11.992 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xxhash_generic.ko 2023-07-14T11:38:11.992 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ 2023-07-14T11:38:11.993 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ 2023-07-14T11:38:11.993 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_extlog.ko 2023-07-14T11:38:11.993 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-14T11:38:11.993 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_pad.ko 2023-07-14T11:38:11.994 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/apei/ 2023-07-14T11:38:11.994 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/apei/einj.ko 2023-07-14T11:38:11.994 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ec_sys.ko 2023-07-14T11:38:11.994 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/platform_profile.ko 2023-07-14T11:38:11.994 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbs.ko 2023-07-14T11:38:11.994 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbshc.ko 2023-07-14T11:38:11.995 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 100401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/video.ko 2023-07-14T11:38:11.995 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ 2023-07-14T11:38:11.995 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/acard-ahci.ko 2023-07-14T11:38:11.995 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 88625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci.ko 2023-07-14T11:38:11.995 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci_platform.ko 2023-07-14T11:38:11.995 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 87841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci.ko 2023-07-14T11:38:11.995 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci_platform.ko 2023-07-14T11:38:11.995 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_acpi.ko 2023-07-14T11:38:11.996 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ali.ko 2023-07-14T11:38:11.996 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_amd.ko 2023-07-14T11:38:11.996 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_artop.ko 2023-07-14T11:38:11.996 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atiixp.ko 2023-07-14T11:38:11.996 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atp867x.ko 2023-07-14T11:38:11.996 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd640.ko 2023-07-14T11:38:11.996 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd64x.ko 2023-07-14T11:38:11.997 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cypress.ko 2023-07-14T11:38:11.997 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_efar.ko 2023-07-14T11:38:11.997 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt366.ko 2023-07-14T11:38:11.997 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt37x.ko 2023-07-14T11:38:11.997 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-14T11:38:11.997 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-14T11:38:11.997 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it8213.ko 2023-07-14T11:38:11.997 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it821x.ko 2023-07-14T11:38:11.998 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_jmicron.ko 2023-07-14T11:38:11.998 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_legacy.ko 2023-07-14T11:38:11.998 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_marvell.ko 2023-07-14T11:38:11.998 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_mpiix.ko 2023-07-14T11:38:11.998 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_netcell.ko 2023-07-14T11:38:11.998 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ninja32.ko 2023-07-14T11:38:11.998 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87410.ko 2023-07-14T11:38:11.999 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87415.ko 2023-07-14T11:38:11.999 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_oldpiix.ko 2023-07-14T11:38:11.999 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_opti.ko 2023-07-14T11:38:11.999 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_optidma.ko 2023-07-14T11:38:11.999 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pcmcia.ko 2023-07-14T11:38:11.999 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-14T11:38:12.000 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-14T11:38:12.000 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_piccolo.ko 2023-07-14T11:38:12.000 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_radisys.ko 2023-07-14T11:38:12.000 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rdc.ko 2023-07-14T11:38:12.000 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rz1000.ko 2023-07-14T11:38:12.001 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sch.ko 2023-07-14T11:38:12.001 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_serverworks.ko 2023-07-14T11:38:12.001 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sil680.ko 2023-07-14T11:38:12.001 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sl82c105.ko 2023-07-14T11:38:12.001 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_triflex.ko 2023-07-14T11:38:12.002 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_via.ko 2023-07-14T11:38:12.002 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pdc_adma.ko 2023-07-14T11:38:12.002 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_inic162x.ko 2023-07-14T11:38:12.002 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 83225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_mv.ko 2023-07-14T11:38:12.002 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_nv.ko 2023-07-14T11:38:12.002 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37369 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_promise.ko 2023-07-14T11:38:12.003 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_qstor.ko 2023-07-14T11:38:12.003 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil.ko 2023-07-14T11:38:12.003 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil24.ko 2023-07-14T11:38:12.003 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sis.ko 2023-07-14T11:38:12.003 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_svw.ko 2023-07-14T11:38:12.003 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sx4.ko 2023-07-14T11:38:12.004 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_uli.ko 2023-07-14T11:38:12.004 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_via.ko 2023-07-14T11:38:12.004 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_vsc.ko 2023-07-14T11:38:12.004 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ 2023-07-14T11:38:12.004 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ks0108.ko 2023-07-14T11:38:12.004 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/ 2023-07-14T11:38:12.005 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 124457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/bcma.ko 2023-07-14T11:38:12.005 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/ 2023-07-14T11:38:12.005 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/ 2023-07-14T11:38:12.005 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 113705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/aoe.ko 2023-07-14T11:38:12.005 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/ 2023-07-14T11:38:12.005 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 968433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/drbd.ko 2023-07-14T11:38:12.052 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 173201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/floppy.ko 2023-07-14T11:38:12.053 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/ 2023-07-14T11:38:12.053 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 94297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-14T11:38:12.053 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 115345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/nbd.ko 2023-07-14T11:38:12.053 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/ 2023-07-14T11:38:12.053 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 110361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/null_blk.ko 2023-07-14T11:38:12.053 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 98225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/pktcdvd.ko 2023-07-14T11:38:12.054 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 257529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/rbd.ko 2023-07-14T11:38:12.054 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/ 2023-07-14T11:38:12.054 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 98129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-14T11:38:12.054 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/ 2023-07-14T11:38:12.054 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/zram.ko 2023-07-14T11:38:12.054 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ 2023-07-14T11:38:12.054 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/applicom.ko 2023-07-14T11:38:12.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hangcheck-timer.ko 2023-07-14T11:38:12.055 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/ 2023-07-14T11:38:12.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-14T11:38:12.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-14T11:38:12.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-14T11:38:12.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/via-rng.ko 2023-07-14T11:38:12.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15633 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-14T11:38:12.056 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ 2023-07-14T11:38:12.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-14T11:38:12.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 150665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-14T11:38:12.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-14T11:38:12.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 139057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-14T11:38:12.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-14T11:38:12.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-14T11:38:12.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/lp.ko 2023-07-14T11:38:12.057 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/ 2023-07-14T11:38:12.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 84121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/mwave.ko 2023-07-14T11:38:12.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/nvram.ko 2023-07-14T11:38:12.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ppdev.ko 2023-07-14T11:38:12.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tlclk.ko 2023-07-14T11:38:12.057 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/ 2023-07-14T11:38:12.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-14T11:38:12.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-07-14T11:38:12.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-07-14T11:38:12.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-07-14T11:38:12.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-14T11:38:12.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-14T11:38:12.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-07-14T11:38:12.058 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/xillybus/ 2023-07-14T11:38:12.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/xillybus/xillybus_class.ko 2023-07-14T11:38:12.059 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/xillybus/xillybus_core.ko 2023-07-14T11:38:12.059 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-07-14T11:38:12.059 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/clk/ 2023-07-14T11:38:12.059 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/clk/clk-palmas.ko 2023-07-14T11:38:12.059 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/clk/clk-si5351.ko 2023-07-14T11:38:12.059 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/clk/clk-twl6040.ko 2023-07-14T11:38:12.059 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/clk/clk-wm831x.ko 2023-07-14T11:38:12.060 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/ 2023-07-14T11:38:12.060 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-14T11:38:12.060 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-14T11:38:12.060 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-14T11:38:12.060 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ 2023-07-14T11:38:12.060 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ 2023-07-14T11:38:12.060 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-14T11:38:12.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 234473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp.ko 2023-07-14T11:38:12.061 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/ 2023-07-14T11:38:12.061 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/ 2023-07-14T11:38:12.061 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/ 2023-07-14T11:38:12.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 408489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-14T11:38:12.061 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-07-14T11:38:12.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-14T11:38:12.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-aes.ko 2023-07-14T11:38:12.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-sha.ko 2023-07-14T11:38:12.062 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/ 2023-07-14T11:38:12.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 68937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-14T11:38:12.062 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/ 2023-07-14T11:38:12.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/dca.ko 2023-07-14T11:38:12.062 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ 2023-07-14T11:38:12.062 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/ 2023-07-14T11:38:12.063 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-14T11:38:12.063 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ 2023-07-14T11:38:12.063 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 122641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-14T11:38:12.063 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ 2023-07-14T11:38:12.063 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 80273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/amd64_edac.ko 2023-07-14T11:38:12.063 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/e752x_edac.ko 2023-07-14T11:38:12.063 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 63617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/edac_mce_amd.ko 2023-07-14T11:38:12.064 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3000_edac.ko 2023-07-14T11:38:12.064 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3200_edac.ko 2023-07-14T11:38:12.064 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5100_edac.ko 2023-07-14T11:38:12.064 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5400_edac.ko 2023-07-14T11:38:12.064 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7300_edac.ko 2023-07-14T11:38:12.064 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7core_edac.ko 2023-07-14T11:38:12.064 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i82975x_edac.ko 2023-07-14T11:38:12.065 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ie31200_edac.ko 2023-07-14T11:38:12.065 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 56809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/sb_edac.ko 2023-07-14T11:38:12.065 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/x38_edac.ko 2023-07-14T11:38:12.065 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/ 2023-07-14T11:38:12.065 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/extcon-gpio.ko 2023-07-14T11:38:12.065 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/extcon-max14577.ko 2023-07-14T11:38:12.065 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/extcon-max77693.ko 2023-07-14T11:38:12.065 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/extcon-max8997.ko 2023-07-14T11:38:12.066 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/extcon-palmas.ko 2023-07-14T11:38:12.066 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/extcon-rt8973a.ko 2023-07-14T11:38:12.066 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/extcon/extcon-sm5502.ko 2023-07-14T11:38:12.066 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/ 2023-07-14T11:38:12.066 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/dmi-sysfs.ko 2023-07-14T11:38:12.066 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/efi/ 2023-07-14T11:38:12.066 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/efi/efi-pstore.ko 2023-07-14T11:38:12.066 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-14T11:38:12.067 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/ 2023-07-14T11:38:12.067 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-adp5520.ko 2023-07-14T11:38:12.067 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-amd8111.ko 2023-07-14T11:38:12.067 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-arizona.ko 2023-07-14T11:38:12.067 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-da9052.ko 2023-07-14T11:38:12.067 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-da9055.ko 2023-07-14T11:38:12.067 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-dln2.ko 2023-07-14T11:38:12.067 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 77849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-f7188x.ko 2023-07-14T11:38:12.068 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-generic.ko 2023-07-14T11:38:12.068 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ich.ko 2023-07-14T11:38:12.068 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-07-14T11:38:12.068 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-kempld.ko 2023-07-14T11:38:12.068 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-lp3943.ko 2023-07-14T11:38:12.068 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-max7300.ko 2023-07-14T11:38:12.068 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-max730x.ko 2023-07-14T11:38:12.069 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-max732x.ko 2023-07-14T11:38:12.069 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-07-14T11:38:12.069 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-pca953x.ko 2023-07-14T11:38:12.069 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-pcf857x.ko 2023-07-14T11:38:12.069 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-rdc321x.ko 2023-07-14T11:38:12.069 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-sch.ko 2023-07-14T11:38:12.069 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-sch311x.ko 2023-07-14T11:38:12.069 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-tps65912.ko 2023-07-14T11:38:12.070 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-twl4030.ko 2023-07-14T11:38:12.070 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-twl6040.ko 2023-07-14T11:38:12.070 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-14T11:38:12.070 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-vx855.ko 2023-07-14T11:38:12.070 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-wm831x.ko 2023-07-14T11:38:12.070 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-wm8350.ko 2023-07-14T11:38:12.070 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-wm8994.ko 2023-07-14T11:38:12.070 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/ 2023-07-14T11:38:12.071 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-a4tech.ko 2023-07-14T11:38:12.071 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-apple.ko 2023-07-14T11:38:12.071 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-appleir.ko 2023-07-14T11:38:12.071 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-aureal.ko 2023-07-14T11:38:12.071 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-axff.ko 2023-07-14T11:38:12.071 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-belkin.ko 2023-07-14T11:38:12.071 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-cherry.ko 2023-07-14T11:38:12.071 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-chicony.ko 2023-07-14T11:38:12.072 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-cp2112.ko 2023-07-14T11:38:12.072 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-cypress.ko 2023-07-14T11:38:12.072 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-dr.ko 2023-07-14T11:38:12.072 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elecom.ko 2023-07-14T11:38:12.072 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elo.ko 2023-07-14T11:38:12.072 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-emsff.ko 2023-07-14T11:38:12.072 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ezkey.ko 2023-07-14T11:38:12.073 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gaff.ko 2023-07-14T11:38:12.073 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-generic.ko 2023-07-14T11:38:12.073 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gt683r.ko 2023-07-14T11:38:12.073 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gyration.ko 2023-07-14T11:38:12.073 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-14T11:38:12.073 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-14T11:38:12.073 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtekff.ko 2023-07-14T11:38:12.073 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-hyperv.ko 2023-07-14T11:38:12.074 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-icade.ko 2023-07-14T11:38:12.074 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kensington.ko 2023-07-14T11:38:12.074 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-keytouch.ko 2023-07-14T11:38:12.074 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kye.ko 2023-07-14T11:38:12.074 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lcpower.ko 2023-07-14T11:38:12.074 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-led.ko 2023-07-14T11:38:12.074 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lenovo.ko 2023-07-14T11:38:12.074 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lg-g15.ko 2023-07-14T11:38:12.075 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-14T11:38:12.075 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 100897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-14T11:38:12.075 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 80993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech.ko 2023-07-14T11:38:12.075 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-magicmouse.ko 2023-07-14T11:38:12.075 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-microsoft.ko 2023-07-14T11:38:12.075 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-monterey.ko 2023-07-14T11:38:12.075 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-multitouch.ko 2023-07-14T11:38:12.075 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ntrig.ko 2023-07-14T11:38:12.076 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ortek.ko 2023-07-14T11:38:12.076 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-penmount.ko 2023-07-14T11:38:12.076 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-petalynx.ko 2023-07-14T11:38:12.076 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 72937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-picolcd.ko 2023-07-14T11:38:12.076 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-pl.ko 2023-07-14T11:38:12.076 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-plantronics.ko 2023-07-14T11:38:12.076 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-primax.ko 2023-07-14T11:38:12.077 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-rmi.ko 2023-07-14T11:38:12.077 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-14T11:38:12.077 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-common.ko 2023-07-14T11:38:12.077 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-14T11:38:12.077 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-14T11:38:12.077 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-14T11:38:12.077 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-14T11:38:12.078 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-14T11:38:12.078 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-14T11:38:12.078 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-14T11:38:12.078 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-14T11:38:12.078 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-14T11:38:12.078 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat.ko 2023-07-14T11:38:12.078 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-saitek.ko 2023-07-14T11:38:12.078 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-samsung.ko 2023-07-14T11:38:12.079 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-14T11:38:12.079 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sjoy.ko 2023-07-14T11:38:12.079 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sony.ko 2023-07-14T11:38:12.079 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-speedlink.ko 2023-07-14T11:38:12.079 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-steelseries.ko 2023-07-14T11:38:12.079 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sunplus.ko 2023-07-14T11:38:12.079 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-14T11:38:12.079 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tivo.ko 2023-07-14T11:38:12.080 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tmff.ko 2023-07-14T11:38:12.080 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-topseed.ko 2023-07-14T11:38:12.080 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-twinhan.ko 2023-07-14T11:38:12.080 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 71457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-uclogic.ko 2023-07-14T11:38:12.080 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-waltop.ko 2023-07-14T11:38:12.080 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 143585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-wiimote.ko 2023-07-14T11:38:12.080 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-xinmo.ko 2023-07-14T11:38:12.081 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zpff.ko 2023-07-14T11:38:12.081 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zydacron.ko 2023-07-14T11:38:12.081 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 267713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid.ko 2023-07-14T11:38:12.081 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/uhid.ko 2023-07-14T11:38:12.081 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/usbhid/ 2023-07-14T11:38:12.081 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 125721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/usbhid/usbhid.ko 2023-07-14T11:38:12.081 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/usbhid/usbkbd.ko 2023-07-14T11:38:12.081 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/usbhid/usbmouse.ko 2023-07-14T11:38:12.082 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 217401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/wacom.ko 2023-07-14T11:38:12.082 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/ 2023-07-14T11:38:12.082 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 63497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_balloon.ko 2023-07-14T11:38:12.082 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 89609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_utils.ko 2023-07-14T11:38:12.082 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 301449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_vmbus.ko 2023-07-14T11:38:12.082 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ 2023-07-14T11:38:12.082 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru.ko 2023-07-14T11:38:12.082 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru3.ko 2023-07-14T11:38:12.083 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-14T11:38:12.083 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7414.ko 2023-07-14T11:38:12.083 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7418.ko 2023-07-14T11:38:12.083 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adc128d818.ko 2023-07-14T11:38:12.083 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1025.ko 2023-07-14T11:38:12.083 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 97977 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1026.ko 2023-07-14T11:38:12.083 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1029.ko 2023-07-14T11:38:12.084 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1031.ko 2023-07-14T11:38:12.084 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm9240.ko 2023-07-14T11:38:12.084 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ads7828.ko 2023-07-14T11:38:12.084 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7410.ko 2023-07-14T11:38:12.084 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7411.ko 2023-07-14T11:38:12.084 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 74433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7462.ko 2023-07-14T11:38:12.084 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7470.ko 2023-07-14T11:38:12.084 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 67929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7475.ko 2023-07-14T11:38:12.085 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7x10.ko 2023-07-14T11:38:12.085 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/amc6821.ko 2023-07-14T11:38:12.085 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/applesmc.ko 2023-07-14T11:38:12.085 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asb100.ko 2023-07-14T11:38:12.085 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 58617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asc7621.ko 2023-07-14T11:38:12.085 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-14T11:38:12.085 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/atxp1.ko 2023-07-14T11:38:12.086 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/coretemp.ko 2023-07-14T11:38:12.086 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/da9052-hwmon.ko 2023-07-14T11:38:12.086 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/da9055-hwmon.ko 2023-07-14T11:38:12.086 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-14T11:38:12.086 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 99601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dme1737.ko 2023-07-14T11:38:12.086 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds1621.ko 2023-07-14T11:38:12.086 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds620.ko 2023-07-14T11:38:12.087 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc1403.ko 2023-07-14T11:38:12.087 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc2103.ko 2023-07-14T11:38:12.087 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc6w201.ko 2023-07-14T11:38:12.087 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71805f.ko 2023-07-14T11:38:12.087 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 94457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71882fg.ko 2023-07-14T11:38:12.087 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f75375s.ko 2023-07-14T11:38:12.087 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fam15h_power.ko 2023-07-14T11:38:12.087 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fschmd.ko 2023-07-14T11:38:12.088 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g760a.ko 2023-07-14T11:38:12.088 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g762.ko 2023-07-14T11:38:12.088 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl518sm.ko 2023-07-14T11:38:12.088 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl520sm.ko 2023-07-14T11:38:12.088 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hih6130.ko 2023-07-14T11:38:12.088 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-14T11:38:12.088 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5500_temp.ko 2023-07-14T11:38:12.089 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5k_amb.ko 2023-07-14T11:38:12.089 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmaem.ko 2023-07-14T11:38:12.089 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmpex.ko 2023-07-14T11:38:12.089 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina209.ko 2023-07-14T11:38:12.089 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina2xx.ko 2023-07-14T11:38:12.089 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 129217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/it87.ko 2023-07-14T11:38:12.089 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/jc42.ko 2023-07-14T11:38:12.089 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k10temp.ko 2023-07-14T11:38:12.090 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k8temp.ko 2023-07-14T11:38:12.090 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lineage-pem.ko 2023-07-14T11:38:12.090 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm63.ko 2023-07-14T11:38:12.090 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm73.ko 2023-07-14T11:38:12.090 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm75.ko 2023-07-14T11:38:12.090 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm77.ko 2023-07-14T11:38:12.090 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm78.ko 2023-07-14T11:38:12.090 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm80.ko 2023-07-14T11:38:12.091 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm83.ko 2023-07-14T11:38:12.091 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 72833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm85.ko 2023-07-14T11:38:12.091 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm87.ko 2023-07-14T11:38:12.091 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm90.ko 2023-07-14T11:38:12.091 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm92.ko 2023-07-14T11:38:12.091 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 108841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm93.ko 2023-07-14T11:38:12.091 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95234.ko 2023-07-14T11:38:12.091 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95241.ko 2023-07-14T11:38:12.092 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95245.ko 2023-07-14T11:38:12.092 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc2945.ko 2023-07-14T11:38:12.092 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4151.ko 2023-07-14T11:38:12.092 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4215.ko 2023-07-14T11:38:12.092 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4222.ko 2023-07-14T11:38:12.092 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4245.ko 2023-07-14T11:38:12.092 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4260.ko 2023-07-14T11:38:12.093 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4261.ko 2023-07-14T11:38:12.093 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max16065.ko 2023-07-14T11:38:12.093 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1619.ko 2023-07-14T11:38:12.093 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1668.ko 2023-07-14T11:38:12.093 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max197.ko 2023-07-14T11:38:12.093 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6639.ko 2023-07-14T11:38:12.093 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6650.ko 2023-07-14T11:38:12.093 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6697.ko 2023-07-14T11:38:12.094 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mc13783-adc.ko 2023-07-14T11:38:12.094 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mcp3021.ko 2023-07-14T11:38:12.094 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-07-14T11:38:12.094 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6683.ko 2023-07-14T11:38:12.094 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 116737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775-core.ko 2023-07-14T11:38:12.094 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775.ko 2023-07-14T11:38:12.094 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct7802.ko 2023-07-14T11:38:12.094 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 82337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87360.ko 2023-07-14T11:38:12.095 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 54433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87427.ko 2023-07-14T11:38:12.095 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pcf8591.ko 2023-07-14T11:38:12.095 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ 2023-07-14T11:38:12.095 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-14T11:38:12.095 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-14T11:38:12.095 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-14T11:38:12.095 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-14T11:38:12.095 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-14T11:38:12.096 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-14T11:38:12.096 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-14T11:38:12.096 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 81841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-14T11:38:12.096 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-14T11:38:12.096 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-14T11:38:12.096 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-14T11:38:12.096 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-14T11:38:12.097 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/powr1220.ko 2023-07-14T11:38:12.097 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5627.ko 2023-07-14T11:38:12.097 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5636.ko 2023-07-14T11:38:12.097 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-14T11:38:12.097 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht15.ko 2023-07-14T11:38:12.097 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht21.ko 2023-07-14T11:38:12.097 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/shtc1.ko 2023-07-14T11:38:12.097 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sis5595.ko 2023-07-14T11:38:12.098 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smm665.ko 2023-07-14T11:38:12.098 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47b397.ko 2023-07-14T11:38:12.098 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m1.ko 2023-07-14T11:38:12.098 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m192.ko 2023-07-14T11:38:12.098 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/thmc50.ko 2023-07-14T11:38:12.098 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp102.ko 2023-07-14T11:38:12.098 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp103.ko 2023-07-14T11:38:12.098 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp401.ko 2023-07-14T11:38:12.099 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp421.ko 2023-07-14T11:38:12.099 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via-cputemp.ko 2023-07-14T11:38:12.099 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via686a.ko 2023-07-14T11:38:12.099 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt1211.ko 2023-07-14T11:38:12.099 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt8231.ko 2023-07-14T11:38:12.099 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627ehf.ko 2023-07-14T11:38:12.099 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627hf.ko 2023-07-14T11:38:12.100 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 71345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83781d.ko 2023-07-14T11:38:12.100 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 63289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83791d.ko 2023-07-14T11:38:12.100 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 71321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83792d.ko 2023-07-14T11:38:12.100 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 101457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83793.ko 2023-07-14T11:38:12.100 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 119505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83795.ko 2023-07-14T11:38:12.100 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l785ts.ko 2023-07-14T11:38:12.100 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l786ng.ko 2023-07-14T11:38:12.100 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-07-14T11:38:12.101 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-07-14T11:38:12.101 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/ 2023-07-14T11:38:12.101 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/ 2023-07-14T11:38:12.101 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-14T11:38:12.101 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-14T11:38:12.101 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/ 2023-07-14T11:38:12.101 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-07-14T11:38:12.101 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-07-14T11:38:12.102 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-07-14T11:38:12.102 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-14T11:38:12.102 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-14T11:38:12.102 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-14T11:38:12.102 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-07-14T11:38:12.102 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 6929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-14T11:38:12.102 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-07-14T11:38:12.102 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-14T11:38:12.103 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-14T11:38:12.103 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-14T11:38:12.103 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-14T11:38:12.103 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-07-14T11:38:12.103 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-07-14T11:38:12.103 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-14T11:38:12.103 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-14T11:38:12.104 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-14T11:38:12.104 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17369 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-07-14T11:38:12.104 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-14T11:38:12.104 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-14T11:38:12.104 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-07-14T11:38:12.104 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-14T11:38:12.104 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-14T11:38:12.104 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-14T11:38:12.105 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-07-14T11:38:12.105 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-14T11:38:12.105 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-14T11:38:12.105 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-07-14T11:38:12.105 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-07-14T11:38:12.105 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-14T11:38:12.105 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-07-14T11:38:12.106 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-14T11:38:12.106 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-14T11:38:12.106 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-14T11:38:12.106 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-14T11:38:12.106 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-07-14T11:38:12.106 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-mux.ko 2023-07-14T11:38:12.106 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-smbus.ko 2023-07-14T11:38:12.106 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-stub.ko 2023-07-14T11:38:12.107 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/muxes/ 2023-07-14T11:38:12.107 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-07-14T11:38:12.107 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-07-14T11:38:12.107 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-07-14T11:38:12.107 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ 2023-07-14T11:38:12.107 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ 2023-07-14T11:38:12.107 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 273641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-14T11:38:12.107 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 876913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_core.ko 2023-07-14T11:38:12.108 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-14T11:38:12.108 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 324145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-14T11:38:12.108 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 97961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-14T11:38:12.108 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 279321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-14T11:38:12.108 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-14T11:38:12.108 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/ 2023-07-14T11:38:12.108 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/ 2023-07-14T11:38:12.109 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 492561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-14T11:38:12.109 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/ 2023-07-14T11:38:12.109 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 419217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-14T11:38:12.109 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/ 2023-07-14T11:38:12.109 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 739705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-14T11:38:12.109 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ 2023-07-14T11:38:12.109 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 296649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-14T11:38:12.109 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/ocrdma/ 2023-07-14T11:38:12.110 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 153681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-07-14T11:38:12.110 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/usnic/ 2023-07-14T11:38:12.110 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 169017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-07-14T11:38:12.110 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ 2023-07-14T11:38:12.110 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ 2023-07-14T11:38:12.110 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 263177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-14T11:38:12.110 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ 2023-07-14T11:38:12.110 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 100225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-14T11:38:12.111 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ 2023-07-14T11:38:12.111 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 111993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-14T11:38:12.111 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ 2023-07-14T11:38:12.111 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 129081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-14T11:38:12.111 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ 2023-07-14T11:38:12.111 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 129553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-14T11:38:12.111 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/ 2023-07-14T11:38:12.111 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/evbug.ko 2023-07-14T11:38:12.112 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/ff-memless.ko 2023-07-14T11:38:12.112 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ 2023-07-14T11:38:12.112 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-14T11:38:12.112 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-14T11:38:12.112 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/gameport.ko 2023-07-14T11:38:12.112 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/lightning.ko 2023-07-14T11:38:12.112 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ns558.ko 2023-07-14T11:38:12.112 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/joydev.ko 2023-07-14T11:38:12.113 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/ 2023-07-14T11:38:12.113 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-07-14T11:38:12.113 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-07-14T11:38:12.113 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-07-14T11:38:12.113 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-07-14T11:38:12.113 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-14T11:38:12.113 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-07-14T11:38:12.113 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/lkkbd.ko 2023-07-14T11:38:12.114 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/lm8323.ko 2023-07-14T11:38:12.114 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/lm8333.ko 2023-07-14T11:38:12.114 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-07-14T11:38:12.114 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-07-14T11:38:12.114 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-07-14T11:38:12.114 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-07-14T11:38:12.114 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/newtonkbd.ko 2023-07-14T11:38:12.115 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-07-14T11:38:12.115 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/qt1070.ko 2023-07-14T11:38:12.115 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/qt2160.ko 2023-07-14T11:38:12.115 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-07-14T11:38:12.115 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/stowaway.ko 2023-07-14T11:38:12.115 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/sunkbd.ko 2023-07-14T11:38:12.115 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-07-14T11:38:12.115 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-07-14T11:38:12.116 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-07-14T11:38:12.116 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/xtkbd.ko 2023-07-14T11:38:12.116 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/matrix-keymap.ko 2023-07-14T11:38:12.116 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/ 2023-07-14T11:38:12.116 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/appletouch.ko 2023-07-14T11:38:12.116 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/bcm5974.ko 2023-07-14T11:38:12.116 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 123681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/cyapatp.ko 2023-07-14T11:38:12.117 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 89449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-14T11:38:12.117 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/gpio_mouse.ko 2023-07-14T11:38:12.117 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 335849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/psmouse.ko 2023-07-14T11:38:12.117 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/sermouse.ko 2023-07-14T11:38:12.117 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-14T11:38:12.117 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-14T11:38:12.117 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-14T11:38:12.117 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/ 2023-07-14T11:38:12.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 109849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-14T11:38:12.118 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/ 2023-07-14T11:38:12.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/altera_ps2.ko 2023-07-14T11:38:12.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/arc_ps2.ko 2023-07-14T11:38:12.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/ct82c710.ko 2023-07-14T11:38:12.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-14T11:38:12.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/parkbd.ko 2023-07-14T11:38:12.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/pcips2.ko 2023-07-14T11:38:12.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/ps2mult.ko 2023-07-14T11:38:12.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serio_raw.ko 2023-07-14T11:38:12.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17977 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serport.ko 2023-07-14T11:38:12.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/sparse-keymap.ko 2023-07-14T11:38:12.119 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/ 2023-07-14T11:38:12.119 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/ 2023-07-14T11:38:12.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-14T11:38:12.120 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/ 2023-07-14T11:38:12.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-88pm860x.ko 2023-07-14T11:38:12.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-adp5520.ko 2023-07-14T11:38:12.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-bd2802.ko 2023-07-14T11:38:12.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-blinkm.ko 2023-07-14T11:38:12.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-da903x.ko 2023-07-14T11:38:12.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-da9052.ko 2023-07-14T11:38:12.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-gpio.ko 2023-07-14T11:38:12.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3530.ko 2023-07-14T11:38:12.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3533.ko 2023-07-14T11:38:12.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm355x.ko 2023-07-14T11:38:12.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3642.ko 2023-07-14T11:38:12.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp3944.ko 2023-07-14T11:38:12.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp8788.ko 2023-07-14T11:38:12.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-max8997.ko 2023-07-14T11:38:12.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-mc13783.ko 2023-07-14T11:38:12.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-menf21bmc.ko 2023-07-14T11:38:12.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-pca9532.ko 2023-07-14T11:38:12.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-pca955x.ko 2023-07-14T11:38:12.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-pca963x.ko 2023-07-14T11:38:12.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-ss4200.ko 2023-07-14T11:38:12.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-tca6507.ko 2023-07-14T11:38:12.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-wm831x-status.ko 2023-07-14T11:38:12.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-wm8350.ko 2023-07-14T11:38:12.123 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ 2023-07-14T11:38:12.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-14T11:38:12.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-14T11:38:12.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-14T11:38:12.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-14T11:38:12.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-14T11:38:12.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-14T11:38:12.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-14T11:38:12.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-14T11:38:12.124 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/ 2023-07-14T11:38:12.124 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/ 2023-07-14T11:38:12.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 655473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/bcache.ko 2023-07-14T11:38:12.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bio-prison.ko 2023-07-14T11:38:12.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 89601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bufio.ko 2023-07-14T11:38:12.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache-smq.ko 2023-07-14T11:38:12.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 155993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache.ko 2023-07-14T11:38:12.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 107049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-crypt.ko 2023-07-14T11:38:12.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-delay.ko 2023-07-14T11:38:12.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-era.ko 2023-07-14T11:38:12.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-flakey.ko 2023-07-14T11:38:12.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log-userspace.ko 2023-07-14T11:38:12.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log.ko 2023-07-14T11:38:12.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-mirror.ko 2023-07-14T11:38:12.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-multipath.ko 2023-07-14T11:38:12.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-queue-length.ko 2023-07-14T11:38:12.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-raid.ko 2023-07-14T11:38:12.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-region-hash.ko 2023-07-14T11:38:12.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-round-robin.ko 2023-07-14T11:38:12.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-service-time.ko 2023-07-14T11:38:12.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 109593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-snapshot.ko 2023-07-14T11:38:12.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-switch.ko 2023-07-14T11:38:12.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 181585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-thin-pool.ko 2023-07-14T11:38:12.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-verity.ko 2023-07-14T11:38:12.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-zero.ko 2023-07-14T11:38:12.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/faulty.ko 2023-07-14T11:38:12.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/linear.ko 2023-07-14T11:38:12.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/multipath.ko 2023-07-14T11:38:12.127 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/ 2023-07-14T11:38:12.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 199937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-14T11:38:12.128 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid0.ko 2023-07-14T11:38:12.128 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 112609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid1.ko 2023-07-14T11:38:12.128 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 160553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid10.ko 2023-07-14T11:38:12.128 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 396153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid456.ko 2023-07-14T11:38:12.128 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/ 2023-07-14T11:38:12.128 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/ 2023-07-14T11:38:12.128 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 178113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptbase.ko 2023-07-14T11:38:12.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 63385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptctl.ko 2023-07-14T11:38:12.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptfc.ko 2023-07-14T11:38:12.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptlan.ko 2023-07-14T11:38:12.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 123305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptsas.ko 2023-07-14T11:38:12.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 79017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptscsih.ko 2023-07-14T11:38:12.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptspi.ko 2023-07-14T11:38:12.129 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/ 2023-07-14T11:38:12.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/88pm800.ko 2023-07-14T11:38:12.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/88pm805.ko 2023-07-14T11:38:12.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/88pm80x.ko 2023-07-14T11:38:12.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/arizona-i2c.ko 2023-07-14T11:38:12.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 123721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/arizona.ko 2023-07-14T11:38:12.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/bcm590xx.ko 2023-07-14T11:38:12.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/cros_ec_dev.ko 2023-07-14T11:38:12.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/dln2.ko 2023-07-14T11:38:12.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/janz-cmodio.ko 2023-07-14T11:38:12.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/kempld-core.ko 2023-07-14T11:38:12.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lm3533-core.ko 2023-07-14T11:38:12.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-07-14T11:38:12.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lp3943.ko 2023-07-14T11:38:12.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 54569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_ich.ko 2023-07-14T11:38:12.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_sch.ko 2023-07-14T11:38:12.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/max8907.ko 2023-07-14T11:38:12.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mc13xxx-core.ko 2023-07-14T11:38:12.132 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-07-14T11:38:12.132 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/menf21bmc.ko 2023-07-14T11:38:12.132 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/pcf50633-adc.ko 2023-07-14T11:38:12.132 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/pcf50633-gpio.ko 2023-07-14T11:38:12.132 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/pcf50633.ko 2023-07-14T11:38:12.132 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-07-14T11:38:12.132 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/retu-mfd.ko 2023-07-14T11:38:12.133 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 80425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/si476x-core.ko 2023-07-14T11:38:12.133 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/sm501.ko 2023-07-14T11:38:12.133 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-07-14T11:38:12.133 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/tps65010.ko 2023-07-14T11:38:12.133 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/tps6507x.ko 2023-07-14T11:38:12.133 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/viperboard.ko 2023-07-14T11:38:12.133 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/vx855.ko 2023-07-14T11:38:12.133 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wl1273-core.ko 2023-07-14T11:38:12.134 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wm8994.ko 2023-07-14T11:38:12.134 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ 2023-07-14T11:38:12.134 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-07-14T11:38:12.134 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ad525x_dpot.ko 2023-07-14T11:38:12.134 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/ 2023-07-14T11:38:12.134 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-14T11:38:12.134 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds9802als.ko 2023-07-14T11:38:12.134 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds990x.ko 2023-07-14T11:38:12.135 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/bh1770glc.ko 2023-07-14T11:38:12.135 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/c2port/ 2023-07-14T11:38:12.135 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-07-14T11:38:12.135 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/c2port/core.ko 2023-07-14T11:38:12.135 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/ 2023-07-14T11:38:12.135 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/cb710.ko 2023-07-14T11:38:12.135 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ds1682.ko 2023-07-14T11:38:12.135 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/dummy-irq.ko 2023-07-14T11:38:12.136 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/echo/ 2023-07-14T11:38:12.136 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/echo/echo.ko 2023-07-14T11:38:12.136 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/ 2023-07-14T11:38:12.136 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/at24.ko 2023-07-14T11:38:12.136 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-14T11:38:12.136 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-14T11:38:12.136 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/max6875.ko 2023-07-14T11:38:12.136 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/enclosure.ko 2023-07-14T11:38:12.137 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/genwqe/ 2023-07-14T11:38:12.137 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 152137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-07-14T11:38:12.137 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hmc6352.ko 2023-07-14T11:38:12.137 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hpilo.ko 2023-07-14T11:38:12.137 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ibmasm/ 2023-07-14T11:38:12.137 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 72601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-07-14T11:38:12.137 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ics932s401.ko 2023-07-14T11:38:12.137 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29003.ko 2023-07-14T11:38:12.138 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29020.ko 2023-07-14T11:38:12.138 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/ 2023-07-14T11:38:12.138 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-14T11:38:12.138 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-14T11:38:12.138 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/ 2023-07-14T11:38:12.138 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 125209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-me.ko 2023-07-14T11:38:12.138 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 56521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-txe.ko 2023-07-14T11:38:12.139 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 309857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei.ko 2023-07-14T11:38:12.139 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/phantom.ko 2023-07-14T11:38:12.139 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ti-st/ 2023-07-14T11:38:12.139 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 74905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/ti-st/st_drv.ko 2023-07-14T11:38:12.139 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_7xx1.ko 2023-07-14T11:38:12.139 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_core.ko 2023-07-14T11:38:12.139 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tsl2550.ko 2023-07-14T11:38:12.140 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_balloon.ko 2023-07-14T11:38:12.140 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/ 2023-07-14T11:38:12.140 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 183857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-14T11:38:12.140 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ 2023-07-14T11:38:12.140 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/ 2023-07-14T11:38:12.140 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 487801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/bonding.ko 2023-07-14T11:38:12.140 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/ 2023-07-14T11:38:12.141 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/ 2023-07-14T11:38:12.141 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 106009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-14T11:38:12.141 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-14T11:38:12.141 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-14T11:38:12.141 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/ 2023-07-14T11:38:12.141 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 452481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-14T11:38:12.141 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dummy.ko 2023-07-14T11:38:12.141 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/eql.ko 2023-07-14T11:38:12.142 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ 2023-07-14T11:38:12.142 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/ 2023-07-14T11:38:12.142 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-14T11:38:12.142 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-14T11:38:12.142 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 130489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-14T11:38:12.142 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 68785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-14T11:38:12.142 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ 2023-07-14T11:38:12.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-14T11:38:12.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-14T11:38:12.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-14T11:38:12.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 99113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-14T11:38:12.143 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/ 2023-07-14T11:38:12.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 54521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-14T11:38:12.143 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/ 2023-07-14T11:38:12.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-14T11:38:12.144 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/ 2023-07-14T11:38:12.144 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-14T11:38:12.144 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/ 2023-07-14T11:38:12.144 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-14T11:38:12.144 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/ 2023-07-14T11:38:12.144 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 54545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-14T11:38:12.144 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-14T11:38:12.144 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 87345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-14T11:38:12.145 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/ 2023-07-14T11:38:12.145 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/ 2023-07-14T11:38:12.145 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 97321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-14T11:38:12.145 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-07-14T11:38:12.145 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 104585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-14T11:38:12.145 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-07-14T11:38:12.145 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 90257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-14T11:38:12.146 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/ 2023-07-14T11:38:12.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 81505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-14T11:38:12.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-14T11:38:12.146 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/ 2023-07-14T11:38:12.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-14T11:38:12.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 176881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-14T11:38:12.146 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-07-14T11:38:12.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1422209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-14T11:38:12.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 128377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-14T11:38:12.147 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/ 2023-07-14T11:38:12.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 132289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-14T11:38:12.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 312145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-14T11:38:12.147 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/ 2023-07-14T11:38:12.147 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/ 2023-07-14T11:38:12.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 323601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-14T11:38:12.147 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/ 2023-07-14T11:38:12.148 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-07-14T11:38:12.148 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 193433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-14T11:38:12.148 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-07-14T11:38:12.148 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 347641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-14T11:38:12.148 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-07-14T11:38:12.148 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1044145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-14T11:38:12.148 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-07-14T11:38:12.148 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 166113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-14T11:38:12.149 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-07-14T11:38:12.149 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-14T11:38:12.149 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/ 2023-07-14T11:38:12.149 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/ 2023-07-14T11:38:12.149 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 204329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-14T11:38:12.149 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/ 2023-07-14T11:38:12.149 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/ 2023-07-14T11:38:12.149 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-14T11:38:12.150 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-14T11:38:12.150 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 141161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-14T11:38:12.150 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-14T11:38:12.150 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-14T11:38:12.150 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-14T11:38:12.150 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/ 2023-07-14T11:38:12.150 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-14T11:38:12.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-14T11:38:12.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dnet.ko 2023-07-14T11:38:12.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ec_bhf.ko 2023-07-14T11:38:12.151 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/ 2023-07-14T11:38:12.151 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/ 2023-07-14T11:38:12.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 281329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-14T11:38:12.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ethoc.ko 2023-07-14T11:38:12.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fealnx.ko 2023-07-14T11:38:12.152 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fujitsu/ 2023-07-14T11:38:12.152 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-07-14T11:38:12.152 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ 2023-07-14T11:38:12.152 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 90465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-14T11:38:12.152 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/ 2023-07-14T11:38:12.152 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 295713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-14T11:38:12.152 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/ 2023-07-14T11:38:12.152 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 584249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-14T11:38:12.200 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/ 2023-07-14T11:38:12.200 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 246321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-14T11:38:12.200 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/ 2023-07-14T11:38:12.200 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 923969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-14T11:38:12.200 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/ 2023-07-14T11:38:12.201 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 298097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-14T11:38:12.201 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/ 2023-07-14T11:38:12.201 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 514889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-14T11:38:12.201 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/ 2023-07-14T11:38:12.201 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 93593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-14T11:38:12.201 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-07-14T11:38:12.201 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 734297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-14T11:38:12.201 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-07-14T11:38:12.202 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 151841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-14T11:38:12.202 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 86921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/jme.ko 2023-07-14T11:38:12.202 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/ 2023-07-14T11:38:12.202 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20977 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-14T11:38:12.202 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 102385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-14T11:38:12.202 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 122681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-14T11:38:12.202 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/ 2023-07-14T11:38:12.203 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-07-14T11:38:12.203 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 727433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-14T11:38:12.203 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 280513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-14T11:38:12.203 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-07-14T11:38:12.203 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-07-14T11:38:12.203 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1095817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-14T11:38:12.203 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ 2023-07-14T11:38:12.203 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-07-14T11:38:12.204 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-07-14T11:38:12.204 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-07-14T11:38:12.204 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 95665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-14T11:38:12.204 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/ 2023-07-14T11:38:12.204 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-07-14T11:38:12.204 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 108809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-14T11:38:12.204 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ 2023-07-14T11:38:12.205 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 73377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-14T11:38:12.205 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-14T11:38:12.205 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/ 2023-07-14T11:38:12.205 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 150905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-14T11:38:12.205 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/ 2023-07-14T11:38:12.205 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 136785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-14T11:38:12.205 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/ 2023-07-14T11:38:12.205 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-14T11:38:12.206 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-14T11:38:12.206 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/ 2023-07-14T11:38:12.206 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-07-14T11:38:12.206 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 227113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-14T11:38:12.206 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 87081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-14T11:38:12.206 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-07-14T11:38:12.206 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 671393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-14T11:38:12.206 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/ 2023-07-14T11:38:12.207 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-14T11:38:12.207 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/ 2023-07-14T11:38:12.207 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 67385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-14T11:38:12.207 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 83921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-14T11:38:12.207 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-14T11:38:12.207 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 190089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-14T11:38:12.207 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/samsung/ 2023-07-14T11:38:12.208 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-07-14T11:38:12.208 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 104241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-07-14T11:38:12.208 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/ 2023-07-14T11:38:12.208 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 952881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-14T11:38:12.208 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/ 2023-07-14T11:38:12.208 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-14T11:38:12.208 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/ 2023-07-14T11:38:12.208 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 54809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-14T11:38:12.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 63273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-14T11:38:12.209 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/ 2023-07-14T11:38:12.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-14T11:38:12.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 67129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-14T11:38:12.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 73345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-14T11:38:12.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 68209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-14T11:38:12.209 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/ 2023-07-14T11:38:12.209 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-07-14T11:38:12.210 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-07-14T11:38:12.210 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-14T11:38:12.210 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-07-14T11:38:12.210 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 481681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-14T11:38:12.210 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/ 2023-07-14T11:38:12.210 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 103321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-14T11:38:12.210 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 166121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-14T11:38:12.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-14T11:38:12.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-14T11:38:12.211 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/ 2023-07-14T11:38:12.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-14T11:38:12.211 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/ 2023-07-14T11:38:12.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 86465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-14T11:38:12.211 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/ 2023-07-14T11:38:12.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 80257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-14T11:38:12.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 83681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-14T11:38:12.212 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/ 2023-07-14T11:38:12.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-14T11:38:12.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-14T11:38:12.212 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/ 2023-07-14T11:38:12.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-14T11:38:12.212 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/fddi/ 2023-07-14T11:38:12.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/fddi/defxx.ko 2023-07-14T11:38:12.213 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/fddi/skfp/ 2023-07-14T11:38:12.213 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 216977 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/fddi/skfp/skfp.ko 2023-07-14T11:38:12.213 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 77401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/geneve.ko 2023-07-14T11:38:12.213 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/ 2023-07-14T11:38:12.213 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 235593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-14T11:38:12.213 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/ 2023-07-14T11:38:12.213 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-14T11:38:12.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ifb.ko 2023-07-14T11:38:12.214 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ 2023-07-14T11:38:12.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 83737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-14T11:38:12.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvlan.ko 2023-07-14T11:38:12.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvtap.ko 2023-07-14T11:38:12.214 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/ 2023-07-14T11:38:12.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-14T11:38:12.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-14T11:38:12.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-gpio.ko 2023-07-14T11:38:12.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-14T11:38:12.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio.ko 2023-07-14T11:38:12.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mii.ko 2023-07-14T11:38:12.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/netconsole.ko 2023-07-14T11:38:12.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/nlmon.ko 2023-07-14T11:38:12.215 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/ 2023-07-14T11:38:12.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-14T11:38:12.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-14T11:38:12.216 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ 2023-07-14T11:38:12.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/amd.ko 2023-07-14T11:38:12.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-14T11:38:12.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-14T11:38:12.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm87xx.ko 2023-07-14T11:38:12.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/broadcom.ko 2023-07-14T11:38:12.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/cicada.ko 2023-07-14T11:38:12.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/davicom.ko 2023-07-14T11:38:12.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/et1011c.ko 2023-07-14T11:38:12.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/icplus.ko 2023-07-14T11:38:12.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/lxt.ko 2023-07-14T11:38:12.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/marvell.ko 2023-07-14T11:38:12.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 75713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/micrel.ko 2023-07-14T11:38:12.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/national.ko 2023-07-14T11:38:12.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 97265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/phylink.ko 2023-07-14T11:38:12.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/qsemi.ko 2023-07-14T11:38:12.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/realtek.ko 2023-07-14T11:38:12.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/smsc.ko 2023-07-14T11:38:12.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ste10Xp.ko 2023-07-14T11:38:12.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/vitesse.ko 2023-07-14T11:38:12.218 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/plip/ 2023-07-14T11:38:12.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/plip/plip.ko 2023-07-14T11:38:12.219 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ 2023-07-14T11:38:12.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-14T11:38:12.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_async.ko 2023-07-14T11:38:12.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-14T11:38:12.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-14T11:38:12.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-14T11:38:12.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppoe.ko 2023-07-14T11:38:12.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppox.ko 2023-07-14T11:38:12.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pptp.ko 2023-07-14T11:38:12.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sb1000.ko 2023-07-14T11:38:12.220 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/ 2023-07-14T11:38:12.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slip.ko 2023-07-14T11:38:12.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sungem_phy.ko 2023-07-14T11:38:12.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 64889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tap.ko 2023-07-14T11:38:12.220 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/ 2023-07-14T11:38:12.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 105961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team.ko 2023-07-14T11:38:12.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-14T11:38:12.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-14T11:38:12.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-14T11:38:12.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_random.ko 2023-07-14T11:38:12.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-14T11:38:12.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 88193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/veth.ko 2023-07-14T11:38:12.221 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/ 2023-07-14T11:38:12.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 120497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-14T11:38:12.222 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/ 2023-07-14T11:38:12.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 290665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/vxlan.ko 2023-07-14T11:38:12.222 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/ 2023-07-14T11:38:12.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 143801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-14T11:38:12.222 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ 2023-07-14T11:38:12.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ntb.ko 2023-07-14T11:38:12.222 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/ 2023-07-14T11:38:12.223 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/ 2023-07-14T11:38:12.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 266561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme-core.ko 2023-07-14T11:38:12.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 101937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme.ko 2023-07-14T11:38:12.223 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/ 2023-07-14T11:38:12.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 133817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport.ko 2023-07-14T11:38:12.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_cs.ko 2023-07-14T11:38:12.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 84337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_pc.ko 2023-07-14T11:38:12.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_serial.ko 2023-07-14T11:38:12.224 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/ 2023-07-14T11:38:12.224 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/ 2023-07-14T11:38:12.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-14T11:38:12.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-07-14T11:38:12.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-07-14T11:38:12.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pci-stub.ko 2023-07-14T11:38:12.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/xen-pcifront.ko 2023-07-14T11:38:12.225 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/ 2023-07-14T11:38:12.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/i82092.ko 2023-07-14T11:38:12.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 145785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pcmcia.ko 2023-07-14T11:38:12.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pcmcia_core.ko 2023-07-14T11:38:12.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-07-14T11:38:12.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pd6729.ko 2023-07-14T11:38:12.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 112913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-14T11:38:12.225 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/phy/ 2023-07-14T11:38:12.226 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/phy/broadcom/ 2023-07-14T11:38:12.226 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-07-14T11:38:12.226 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/phy/samsung/ 2023-07-14T11:38:12.226 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-07-14T11:38:12.226 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/ 2023-07-14T11:38:12.226 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/ 2023-07-14T11:38:12.226 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 71521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-14T11:38:12.226 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/ 2023-07-14T11:38:12.227 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/ 2023-07-14T11:38:12.227 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-07-14T11:38:12.227 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/cros_ec.ko 2023-07-14T11:38:12.227 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-07-14T11:38:12.227 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-07-14T11:38:12.227 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-07-14T11:38:12.227 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-07-14T11:38:12.227 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-07-14T11:38:12.228 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/ 2023-07-14T11:38:12.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 67097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-14T11:38:12.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acerhdf.ko 2023-07-14T11:38:12.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-14T11:38:12.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-14T11:38:12.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-14T11:38:12.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 106873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-14T11:38:12.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-14T11:38:12.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-14T11:38:12.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-14T11:38:12.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-14T11:38:12.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-14T11:38:12.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/hdaps.ko 2023-07-14T11:38:12.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/ibm_rtl.ko 2023-07-14T11:38:12.229 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/ 2023-07-14T11:38:12.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-14T11:38:12.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-07-14T11:38:12.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel_ips.ko 2023-07-14T11:38:12.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-14T11:38:12.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-14T11:38:12.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-14T11:38:12.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-14T11:38:12.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-14T11:38:12.231 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-14T11:38:12.231 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-14T11:38:12.231 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-14T11:38:12.231 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-14T11:38:12.231 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi.ko 2023-07-14T11:38:12.231 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/ 2023-07-14T11:38:12.231 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/ 2023-07-14T11:38:12.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/88pm860x_battery.ko 2023-07-14T11:38:12.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/88pm860x_charger.ko 2023-07-14T11:38:12.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/bq2415x_charger.ko 2023-07-14T11:38:12.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/bq24190_charger.ko 2023-07-14T11:38:12.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/bq24735-charger.ko 2023-07-14T11:38:12.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-07-14T11:38:12.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/da9030_battery.ko 2023-07-14T11:38:12.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/da9052-battery.ko 2023-07-14T11:38:12.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/ds2760_battery.ko 2023-07-14T11:38:12.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/ds2780_battery.ko 2023-07-14T11:38:12.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/ds2781_battery.ko 2023-07-14T11:38:12.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/ds2782_battery.ko 2023-07-14T11:38:12.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/gpio-charger.ko 2023-07-14T11:38:12.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/isp1704_charger.ko 2023-07-14T11:38:12.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/lp8727_charger.ko 2023-07-14T11:38:12.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/max14577_charger.ko 2023-07-14T11:38:12.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/max17040_battery.ko 2023-07-14T11:38:12.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/max17042_battery.ko 2023-07-14T11:38:12.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/max8903_charger.ko 2023-07-14T11:38:12.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/max8925_power.ko 2023-07-14T11:38:12.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/pcf50633-charger.ko 2023-07-14T11:38:12.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/sbs-battery.ko 2023-07-14T11:38:12.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/test_power.ko 2023-07-14T11:38:12.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/tps65090-charger.ko 2023-07-14T11:38:12.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/wm831x_backup.ko 2023-07-14T11:38:12.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/wm831x_power.ko 2023-07-14T11:38:12.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/power/supply/wm8350_power.ko 2023-07-14T11:38:12.235 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/ 2023-07-14T11:38:12.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-14T11:38:12.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-14T11:38:12.235 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/ 2023-07-14T11:38:12.236 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/ 2023-07-14T11:38:12.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-14T11:38:12.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-14T11:38:12.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps_parport.ko 2023-07-14T11:38:12.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/pps_core.ko 2023-07-14T11:38:12.236 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ 2023-07-14T11:38:12.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp.ko 2023-07-14T11:38:12.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp_kvm.ko 2023-07-14T11:38:12.237 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/ 2023-07-14T11:38:12.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-88pm80x.ko 2023-07-14T11:38:12.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-88pm860x.ko 2023-07-14T11:38:12.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-14T11:38:12.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-14T11:38:12.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-da9052.ko 2023-07-14T11:38:12.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-da9055.ko 2023-07-14T11:38:12.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-da9063.ko 2023-07-14T11:38:12.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-14T11:38:12.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 68577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-14T11:38:12.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-14T11:38:12.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-14T11:38:12.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-14T11:38:12.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-14T11:38:12.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13369 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-14T11:38:12.239 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-14T11:38:12.239 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-14T11:38:12.239 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-em3027.ko 2023-07-14T11:38:12.239 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-14T11:38:12.239 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-14T11:38:12.239 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-14T11:38:12.239 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-lp8788.ko 2023-07-14T11:38:12.239 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-14T11:38:12.240 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-14T11:38:12.240 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-14T11:38:12.240 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t86.ko 2023-07-14T11:38:12.240 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11633 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max6900.ko 2023-07-14T11:38:12.240 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max8907.ko 2023-07-14T11:38:12.240 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max8925.ko 2023-07-14T11:38:12.240 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max8997.ko 2023-07-14T11:38:12.240 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max8998.ko 2023-07-14T11:38:12.241 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-07-14T11:38:12.241 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-14T11:38:12.241 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-palmas.ko 2023-07-14T11:38:12.241 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-14T11:38:12.241 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17977 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf50633.ko 2023-07-14T11:38:12.241 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-14T11:38:12.241 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-14T11:38:12.241 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-14T11:38:12.242 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-14T11:38:12.242 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rc5t583.ko 2023-07-14T11:38:12.242 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-14T11:38:12.242 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-14T11:38:12.242 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-14T11:38:12.242 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-14T11:38:12.242 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-14T11:38:12.242 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-s35390a.ko 2023-07-14T11:38:12.243 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-14T11:38:12.243 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-tps6586x.ko 2023-07-14T11:38:12.243 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-tps65910.ko 2023-07-14T11:38:12.243 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-wm831x.ko 2023-07-14T11:38:12.243 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-wm8350.ko 2023-07-14T11:38:12.243 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-x1205.ko 2023-07-14T11:38:12.243 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ 2023-07-14T11:38:12.244 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 77617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-9xxx.ko 2023-07-14T11:38:12.244 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-sas.ko 2023-07-14T11:38:12.244 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-xxxx.ko 2023-07-14T11:38:12.244 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 135169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/BusLogic.ko 2023-07-14T11:38:12.244 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/a100u2w.ko 2023-07-14T11:38:12.244 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/ 2023-07-14T11:38:12.244 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 272353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-14T11:38:12.244 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 100617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/advansys.ko 2023-07-14T11:38:12.245 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/ 2023-07-14T11:38:12.245 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 356265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-14T11:38:12.245 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 295801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-14T11:38:12.245 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic94xx/ 2023-07-14T11:38:12.245 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 189321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-07-14T11:38:12.245 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/am53c974.ko 2023-07-14T11:38:12.245 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/ 2023-07-14T11:38:12.246 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 106569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-14T11:38:12.246 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/atp870u.ko 2023-07-14T11:38:12.246 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/ 2023-07-14T11:38:12.246 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 246897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-14T11:38:12.246 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/ 2023-07-14T11:38:12.246 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1172185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/bfa.ko 2023-07-14T11:38:12.246 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/ 2023-07-14T11:38:12.246 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 237169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-14T11:38:12.247 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/ 2023-07-14T11:38:12.247 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 115393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-14T11:38:12.247 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ch.ko 2023-07-14T11:38:12.247 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/ 2023-07-14T11:38:12.247 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 308889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-14T11:38:12.247 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/ 2023-07-14T11:38:12.247 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-07-14T11:38:12.247 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 76193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-14T11:38:12.248 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-07-14T11:38:12.248 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 101449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-14T11:38:12.248 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 146785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-14T11:38:12.248 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 75529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dc395x.ko 2023-07-14T11:38:12.248 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dmx3191d.ko 2023-07-14T11:38:12.248 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/ 2023-07-14T11:38:12.248 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 212401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-14T11:38:12.249 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esp_scsi.ko 2023-07-14T11:38:12.249 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/ 2023-07-14T11:38:12.249 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-14T11:38:12.249 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 158193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-14T11:38:12.249 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fdomain.ko 2023-07-14T11:38:12.249 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/ 2023-07-14T11:38:12.249 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 261913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/fnic.ko 2023-07-14T11:38:12.249 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 201793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hpsa.ko 2023-07-14T11:38:12.250 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hptiop.ko 2023-07-14T11:38:12.250 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hv_storvsc.ko 2023-07-14T11:38:12.250 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/imm.ko 2023-07-14T11:38:12.250 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/initio.ko 2023-07-14T11:38:12.250 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 266521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ipr.ko 2023-07-14T11:38:12.250 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 101881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ips.ko 2023-07-14T11:38:12.250 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/ 2023-07-14T11:38:12.250 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 317249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/isci.ko 2023-07-14T11:38:12.251 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-14T11:38:12.251 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-14T11:38:12.251 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/ 2023-07-14T11:38:12.251 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 314961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/libfc.ko 2023-07-14T11:38:12.251 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 145569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi.ko 2023-07-14T11:38:12.251 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-14T11:38:12.251 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/ 2023-07-14T11:38:12.251 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 217209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/libsas.ko 2023-07-14T11:38:12.252 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/ 2023-07-14T11:38:12.252 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 2192449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-14T11:38:12.252 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/ 2023-07-14T11:38:12.252 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 73545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-14T11:38:12.252 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-14T11:38:12.252 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 310729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-14T11:38:12.252 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 84697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid.ko 2023-07-14T11:38:12.253 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/ 2023-07-14T11:38:12.253 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 659577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-14T11:38:12.253 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/ 2023-07-14T11:38:12.253 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 119929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-14T11:38:12.253 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvumi.ko 2023-07-14T11:38:12.253 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pcmcia/ 2023-07-14T11:38:12.253 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 81601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-07-14T11:38:12.253 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-07-14T11:38:12.254 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-07-14T11:38:12.254 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-07-14T11:38:12.254 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/ 2023-07-14T11:38:12.254 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 364257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-14T11:38:12.254 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 123489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pmcraid.ko 2023-07-14T11:38:12.254 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ppa.ko 2023-07-14T11:38:12.254 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla1280.ko 2023-07-14T11:38:12.254 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/ 2023-07-14T11:38:12.255 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1860025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-14T11:38:12.255 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 85313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-14T11:38:12.255 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/ 2023-07-14T11:38:12.255 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 558865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-14T11:38:12.255 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qlogicfas408.ko 2023-07-14T11:38:12.255 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/raid_class.ko 2023-07-14T11:38:12.255 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 238801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_debug.ko 2023-07-14T11:38:12.255 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 173921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-14T11:38:12.256 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 278865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-14T11:38:12.256 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 97705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-14T11:38:12.256 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 73881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-14T11:38:12.256 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-14T11:38:12.256 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ses.ko 2023-07-14T11:38:12.256 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 115849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/st.ko 2023-07-14T11:38:12.256 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/stex.ko 2023-07-14T11:38:12.256 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/ 2023-07-14T11:38:12.257 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 162121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-14T11:38:12.257 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/virtio_scsi.ko 2023-07-14T11:38:12.257 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-14T11:38:12.257 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/wd719x.ko 2023-07-14T11:38:12.257 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/xen-scsifront.ko 2023-07-14T11:38:12.257 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ 2023-07-14T11:38:12.257 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 129417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ssb.ko 2023-07-14T11:38:12.257 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/ 2023-07-14T11:38:12.258 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/ 2023-07-14T11:38:12.258 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 739089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-14T11:38:12.258 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/ 2023-07-14T11:38:12.258 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-14T11:38:12.258 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_file.ko 2023-07-14T11:38:12.258 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_iblock.ko 2023-07-14T11:38:12.258 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 931721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_mod.ko 2023-07-14T11:38:12.258 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_pscsi.ko 2023-07-14T11:38:12.259 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 122545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_user.ko 2023-07-14T11:38:12.259 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/ 2023-07-14T11:38:12.259 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 75641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-14T11:38:12.259 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/ 2023-07-14T11:38:12.259 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/ 2023-07-14T11:38:12.259 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/ 2023-07-14T11:38:12.259 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-14T11:38:12.260 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-14T11:38:12.260 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-14T11:38:12.260 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-14T11:38:12.260 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-14T11:38:12.260 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-14T11:38:12.260 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-14T11:38:12.260 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-14T11:38:12.260 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-14T11:38:12.261 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-14T11:38:12.261 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-14T11:38:12.261 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-14T11:38:12.261 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-07-14T11:38:12.261 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-14T11:38:12.261 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-14T11:38:12.261 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-14T11:38:12.262 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ 2023-07-14T11:38:12.262 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ 2023-07-14T11:38:12.262 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-14T11:38:12.262 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/moxa.ko 2023-07-14T11:38:12.262 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/mxser.ko 2023-07-14T11:38:12.262 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 107545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_gsm.ko 2023-07-14T11:38:12.262 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_hdlc.ko 2023-07-14T11:38:12.263 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 68137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/nozomi.ko 2023-07-14T11:38:12.263 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/ 2023-07-14T11:38:12.263 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/ 2023-07-14T11:38:12.263 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-07-14T11:38:12.263 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-14T11:38:12.263 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-07-14T11:38:12.263 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/altera_uart.ko 2023-07-14T11:38:12.263 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/arc_uart.ko 2023-07-14T11:38:12.264 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 72193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-07-14T11:38:12.264 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/ 2023-07-14T11:38:12.264 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 103017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-14T11:38:12.264 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/rp2.ko 2023-07-14T11:38:12.264 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/sc16is7xx.ko 2023-07-14T11:38:12.264 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 100185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/synclink_gt.ko 2023-07-14T11:38:12.264 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/ 2023-07-14T11:38:12.265 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ 2023-07-14T11:38:12.265 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 536617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-14T11:38:12.265 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ 2023-07-14T11:38:12.265 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-14T11:38:12.265 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-07-14T11:38:12.265 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/ 2023-07-14T11:38:12.265 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio.ko 2023-07-14T11:38:12.266 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_aec.ko 2023-07-14T11:38:12.266 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_cif.ko 2023-07-14T11:38:12.266 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_dmem_genirq.ko 2023-07-14T11:38:12.266 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_mf624.ko 2023-07-14T11:38:12.266 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_netx.ko 2023-07-14T11:38:12.266 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pci_generic.ko 2023-07-14T11:38:12.266 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-07-14T11:38:12.266 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13633 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_sercos3.ko 2023-07-14T11:38:12.267 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/ 2023-07-14T11:38:12.267 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/c67x00/ 2023-07-14T11:38:12.267 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 73273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/c67x00/c67x00.ko 2023-07-14T11:38:12.267 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/chipidea/ 2023-07-14T11:38:12.267 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 143785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-07-14T11:38:12.267 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-07-14T11:38:12.267 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-07-14T11:38:12.267 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-07-14T11:38:12.268 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/ 2023-07-14T11:38:12.268 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 85457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-acm.ko 2023-07-14T11:38:12.268 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-14T11:38:12.268 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usblp.ko 2023-07-14T11:38:12.268 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 89833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usbtmc.ko 2023-07-14T11:38:12.268 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/common/ 2023-07-14T11:38:12.268 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/common/ulpi.ko 2023-07-14T11:38:12.269 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/dwc2/ 2023-07-14T11:38:12.269 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-07-14T11:38:12.269 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/dwc3/ 2023-07-14T11:38:12.269 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-07-14T11:38:12.269 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-07-14T11:38:12.269 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 445801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/dwc3/dwc3.ko 2023-07-14T11:38:12.269 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/ 2023-07-14T11:38:12.269 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/ 2023-07-14T11:38:12.270 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/u_ether.ko 2023-07-14T11:38:12.270 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/u_serial.ko 2023-07-14T11:38:12.270 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-07-14T11:38:12.270 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-07-14T11:38:12.270 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-07-14T11:38:12.270 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-07-14T11:38:12.270 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 92377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-07-14T11:38:12.270 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-07-14T11:38:12.271 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 122073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-07-14T11:38:12.271 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-07-14T11:38:12.271 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-07-14T11:38:12.271 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-07-14T11:38:12.271 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-07-14T11:38:12.271 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 85841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-07-14T11:38:12.271 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-07-14T11:38:12.271 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 58433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-07-14T11:38:12.272 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 67337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-07-14T11:38:12.272 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/ 2023-07-14T11:38:12.272 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-07-14T11:38:12.272 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-07-14T11:38:12.272 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-07-14T11:38:12.272 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-07-14T11:38:12.272 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-07-14T11:38:12.272 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-07-14T11:38:12.273 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-07-14T11:38:12.273 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16977 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-07-14T11:38:12.273 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-07-14T11:38:12.273 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-07-14T11:38:12.273 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-07-14T11:38:12.273 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-07-14T11:38:12.273 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-07-14T11:38:12.273 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15201 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-07-14T11:38:12.274 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 161233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/libcomposite.ko 2023-07-14T11:38:12.274 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/ 2023-07-14T11:38:12.274 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-07-14T11:38:12.274 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/bdc/ 2023-07-14T11:38:12.274 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 157665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-07-14T11:38:12.274 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-07-14T11:38:12.274 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-07-14T11:38:12.275 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-07-14T11:38:12.275 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 56009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-07-14T11:38:12.275 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 76033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/net2272.ko 2023-07-14T11:38:12.275 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 75057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/net2280.ko 2023-07-14T11:38:12.275 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 73153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-07-14T11:38:12.275 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 65793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-07-14T11:38:12.275 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-07-14T11:38:12.275 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 81489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-07-14T11:38:12.276 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 156033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-07-14T11:38:12.276 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/ 2023-07-14T11:38:12.276 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/bcma-hcd.ko 2023-07-14T11:38:12.276 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 80097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/isp116x-hcd.ko 2023-07-14T11:38:12.276 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 87337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-07-14T11:38:12.276 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-07-14T11:38:12.276 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-14T11:38:12.276 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811_cs.ko 2023-07-14T11:38:12.277 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/ssb-hcd.ko 2023-07-14T11:38:12.277 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-07-14T11:38:12.277 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/ 2023-07-14T11:38:12.277 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/mdc800.ko 2023-07-14T11:38:12.277 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/microtek.ko 2023-07-14T11:38:12.277 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ 2023-07-14T11:38:12.277 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/adutux.ko 2023-07-14T11:38:12.277 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/appledisplay.ko 2023-07-14T11:38:12.278 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-07-14T11:38:12.278 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/cytherm.ko 2023-07-14T11:38:12.278 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ehset.ko 2023-07-14T11:38:12.278 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi26.ko 2023-07-14T11:38:12.278 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi62.ko 2023-07-14T11:38:12.278 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ezusb.ko 2023-07-14T11:38:12.278 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/idmouse.ko 2023-07-14T11:38:12.279 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/iowarrior.ko 2023-07-14T11:38:12.279 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-14T11:38:12.279 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ldusb.ko 2023-07-14T11:38:12.279 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/legousbtower.ko 2023-07-14T11:38:12.279 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/lvstest.ko 2023-07-14T11:38:12.279 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/ 2023-07-14T11:38:12.279 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 64841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-14T11:38:12.279 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-14T11:38:12.280 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usb3503.ko 2023-07-14T11:38:12.280 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usblcd.ko 2023-07-14T11:38:12.280 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-14T11:38:12.280 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbtest.ko 2023-07-14T11:38:12.280 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/uss720.ko 2023-07-14T11:38:12.280 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/yurex.ko 2023-07-14T11:38:12.280 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/mon/ 2023-07-14T11:38:12.281 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 73185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/mon/usbmon.ko 2023-07-14T11:38:12.281 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/musb/ 2023-07-14T11:38:12.281 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 301521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/musb/musb_hdrc.ko 2023-07-14T11:38:12.281 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/ 2023-07-14T11:38:12.281 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-generic.ko 2023-07-14T11:38:12.281 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-07-14T11:38:12.281 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-isp1301.ko 2023-07-14T11:38:12.281 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-tahvo.ko 2023-07-14T11:38:12.282 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ 2023-07-14T11:38:12.282 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/aircable.ko 2023-07-14T11:38:12.282 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ark3116.ko 2023-07-14T11:38:12.282 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-14T11:38:12.282 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ch341.ko 2023-07-14T11:38:12.282 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cp210x.ko 2023-07-14T11:38:12.282 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cyberjack.ko 2023-07-14T11:38:12.282 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-14T11:38:12.283 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-14T11:38:12.283 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/empeg.ko 2023-07-14T11:38:12.283 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/f81232.ko 2023-07-14T11:38:12.283 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 175145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-14T11:38:12.283 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-14T11:38:12.283 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 137105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-14T11:38:12.283 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 124673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_ti.ko 2023-07-14T11:38:12.283 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 67769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipaq.ko 2023-07-14T11:38:12.284 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipw.ko 2023-07-14T11:38:12.284 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ir-usb.ko 2023-07-14T11:38:12.284 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-14T11:38:12.284 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 107417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan.ko 2023-07-14T11:38:12.284 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-14T11:38:12.284 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-14T11:38:12.284 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-14T11:38:12.285 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mct_u232.ko 2023-07-14T11:38:12.285 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/metro-usb.ko 2023-07-14T11:38:12.285 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7720.ko 2023-07-14T11:38:12.285 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7840.ko 2023-07-14T11:38:12.285 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 56593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mxuport.ko 2023-07-14T11:38:12.285 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/navman.ko 2023-07-14T11:38:12.285 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/omninet.ko 2023-07-14T11:38:12.285 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/opticon.ko 2023-07-14T11:38:12.286 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 189729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/option.ko 2023-07-14T11:38:12.286 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/oti6858.ko 2023-07-14T11:38:12.286 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 54313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/pl2303.ko 2023-07-14T11:38:12.286 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcaux.ko 2023-07-14T11:38:12.286 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcserial.ko 2023-07-14T11:38:12.286 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/quatech2.ko 2023-07-14T11:38:12.286 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/safe_serial.ko 2023-07-14T11:38:12.286 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/sierra.ko 2023-07-14T11:38:12.287 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-14T11:38:12.287 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ssu100.ko 2023-07-14T11:38:12.287 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/symbolserial.ko 2023-07-14T11:38:12.287 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-14T11:38:12.287 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-14T11:38:12.287 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_debug.ko 2023-07-14T11:38:12.287 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-14T11:38:12.287 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 125505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usbserial.ko 2023-07-14T11:38:12.288 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/visor.ko 2023-07-14T11:38:12.288 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/whiteheat.ko 2023-07-14T11:38:12.288 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/wishbone-serial.ko 2023-07-14T11:38:12.288 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-14T11:38:12.288 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ 2023-07-14T11:38:12.288 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/uas.ko 2023-07-14T11:38:12.288 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-14T11:38:12.289 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-14T11:38:12.289 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-14T11:38:12.289 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-14T11:38:12.289 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-14T11:38:12.289 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-14T11:38:12.289 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-14T11:38:12.289 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-karma.ko 2023-07-14T11:38:12.289 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-14T11:38:12.290 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-14T11:38:12.290 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-14T11:38:12.290 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-14T11:38:12.290 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-14T11:38:12.290 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 180017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/usb-storage.ko 2023-07-14T11:38:12.290 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/ 2023-07-14T11:38:12.290 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 67457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-14T11:38:12.290 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 71345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-14T11:38:12.291 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 105401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-14T11:38:12.291 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/ 2023-07-14T11:38:12.291 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/ 2023-07-14T11:38:12.291 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 153697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-14T11:38:12.291 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-14T11:38:12.291 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 100569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio.ko 2023-07-14T11:38:12.291 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 75137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-14T11:38:12.291 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/ 2023-07-14T11:38:12.292 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 113889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost.ko 2023-07-14T11:38:12.292 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-14T11:38:12.292 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 58137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_net.ko 2023-07-14T11:38:12.292 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 86521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_scsi.ko 2023-07-14T11:38:12.292 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/ 2023-07-14T11:38:12.292 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/ 2023-07-14T11:38:12.292 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-07-14T11:38:12.293 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/aat2870_bl.ko 2023-07-14T11:38:12.293 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/adp5520_bl.ko 2023-07-14T11:38:12.293 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/adp8860_bl.ko 2023-07-14T11:38:12.293 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/adp8870_bl.ko 2023-07-14T11:38:12.293 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/apple_bl.ko 2023-07-14T11:38:12.293 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/as3711_bl.ko 2023-07-14T11:38:12.293 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/bd6107.ko 2023-07-14T11:38:12.293 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/da903x_bl.ko 2023-07-14T11:38:12.294 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/da9052_bl.ko 2023-07-14T11:38:12.294 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/gpio_backlight.ko 2023-07-14T11:38:12.294 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14769 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/kb3886_bl.ko 2023-07-14T11:38:12.294 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lcd.ko 2023-07-14T11:38:12.294 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lm3533_bl.ko 2023-07-14T11:38:12.294 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lm3639_bl.ko 2023-07-14T11:38:12.295 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lv5207lp.ko 2023-07-14T11:38:12.295 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/max8925_bl.ko 2023-07-14T11:38:12.295 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/pandora_bl.ko 2023-07-14T11:38:12.295 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-07-14T11:38:12.295 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-14T11:38:12.295 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/wm831x_bl.ko 2023-07-14T11:38:12.295 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/ 2023-07-14T11:38:12.295 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ 2023-07-14T11:38:12.296 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2482.ko 2023-07-14T11:38:12.296 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2490.ko 2023-07-14T11:38:12.296 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/matrox_w1.ko 2023-07-14T11:38:12.296 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/w1-gpio.ko 2023-07-14T11:38:12.296 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/ 2023-07-14T11:38:12.296 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-14T11:38:12.296 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-14T11:38:12.297 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-14T11:38:12.297 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-14T11:38:12.297 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-14T11:38:12.297 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-14T11:38:12.297 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-14T11:38:12.297 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-14T11:38:12.297 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-14T11:38:12.297 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-14T11:38:12.298 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 56425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-14T11:38:12.298 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 102657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/wire.ko 2023-07-14T11:38:12.298 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ 2023-07-14T11:38:12.298 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/acquirewdt.ko 2023-07-14T11:38:12.298 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19977 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/advantechwdt.ko 2023-07-14T11:38:12.298 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-14T11:38:12.298 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-14T11:38:12.298 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/cpu5wdt.ko 2023-07-14T11:38:12.299 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/da9052_wdt.ko 2023-07-14T11:38:12.299 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/da9055_wdt.ko 2023-07-14T11:38:12.299 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/da9063_wdt.ko 2023-07-14T11:38:12.299 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/dw_wdt.ko 2023-07-14T11:38:12.299 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/eurotechwdt.ko 2023-07-14T11:38:12.299 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-14T11:38:12.299 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/hpwdt.ko 2023-07-14T11:38:12.299 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/i6300esb.ko 2023-07-14T11:38:12.300 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-14T11:38:12.300 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-14T11:38:12.300 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ib700wdt.ko 2023-07-14T11:38:12.300 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ibmasr.ko 2023-07-14T11:38:12.300 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-14T11:38:12.300 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-14T11:38:12.300 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it87_wdt.ko 2023-07-14T11:38:12.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/kempld_wdt.ko 2023-07-14T11:38:12.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/machzwd.ko 2023-07-14T11:38:12.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/mena21_wdt.ko 2023-07-14T11:38:12.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-07-14T11:38:12.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/nv_tco.ko 2023-07-14T11:38:12.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-07-14T11:38:12.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pc87413_wdt.ko 2023-07-14T11:38:12.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-14T11:38:12.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-14T11:38:12.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/retu_wdt.ko 2023-07-14T11:38:12.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-07-14T11:38:12.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-07-14T11:38:12.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-14T11:38:12.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sc1200wdt.ko 2023-07-14T11:38:12.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-14T11:38:12.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-07-14T11:38:12.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/softdog.ko 2023-07-14T11:38:12.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-14T11:38:12.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/twl4030_wdt.ko 2023-07-14T11:38:12.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/via_wdt.ko 2023-07-14T11:38:12.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-14T11:38:12.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-14T11:38:12.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-14T11:38:12.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wafer5823wdt.ko 2023-07-14T11:38:12.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wdt_pci.ko 2023-07-14T11:38:12.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wm831x_wdt.ko 2023-07-14T11:38:12.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wm8350_wdt.ko 2023-07-14T11:38:12.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/xen_wdt.ko 2023-07-14T11:38:12.304 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/ 2023-07-14T11:38:12.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-evtchn.ko 2023-07-14T11:38:12.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntalloc.ko 2023-07-14T11:38:12.305 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntdev.ko 2023-07-14T11:38:12.305 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/ 2023-07-14T11:38:12.305 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 176417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-14T11:38:12.305 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-privcmd.ko 2023-07-14T11:38:12.305 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 71321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-scsiback.ko 2023-07-14T11:38:12.305 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/ 2023-07-14T11:38:12.305 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-14T11:38:12.305 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ 2023-07-14T11:38:12.306 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/ 2023-07-14T11:38:12.306 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 165697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/9p.ko 2023-07-14T11:38:12.306 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/adfs/ 2023-07-14T11:38:12.306 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/adfs/adfs.ko 2023-07-14T11:38:12.306 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/ 2023-07-14T11:38:12.306 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 162617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/affs.ko 2023-07-14T11:38:12.306 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/afs/ 2023-07-14T11:38:12.307 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 931937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/afs/kafs.ko 2023-07-14T11:38:12.307 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/autofs/ 2023-07-14T11:38:12.307 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 100937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/autofs/autofs4.ko 2023-07-14T11:38:12.307 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/ 2023-07-14T11:38:12.307 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 64481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/befs.ko 2023-07-14T11:38:12.307 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/bfs/ 2023-07-14T11:38:12.307 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/bfs/bfs.ko 2023-07-14T11:38:12.307 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/binfmt_misc.ko 2023-07-14T11:38:12.308 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/ 2023-07-14T11:38:12.308 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 3468681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/btrfs.ko 2023-07-14T11:38:12.308 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/ 2023-07-14T11:38:12.308 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 305745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/cachefiles.ko 2023-07-14T11:38:12.308 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ 2023-07-14T11:38:12.308 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1312521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ceph.ko 2023-07-14T11:38:12.308 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/ 2023-07-14T11:38:12.308 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 100321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/coda.ko 2023-07-14T11:38:12.309 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/configfs/ 2023-07-14T11:38:12.309 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 108073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/configfs/configfs.ko 2023-07-14T11:38:12.309 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/ 2023-07-14T11:38:12.309 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/cramfs.ko 2023-07-14T11:38:12.309 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/ 2023-07-14T11:38:12.309 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 633825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/dlm.ko 2023-07-14T11:38:12.309 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/efs/ 2023-07-14T11:38:12.309 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/efs/efs.ko 2023-07-14T11:38:12.310 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/ 2023-07-14T11:38:12.310 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1500513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/f2fs.ko 2023-07-14T11:38:12.310 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/ 2023-07-14T11:38:12.310 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/msdos.ko 2023-07-14T11:38:12.310 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/freevxfs/ 2023-07-14T11:38:12.310 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/freevxfs/freevxfs.ko 2023-07-14T11:38:12.310 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/ 2023-07-14T11:38:12.311 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 226505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/fscache.ko 2023-07-14T11:38:12.311 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/ 2023-07-14T11:38:12.311 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/cuse.ko 2023-07-14T11:38:12.311 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/ 2023-07-14T11:38:12.311 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1226249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/gfs2.ko 2023-07-14T11:38:12.311 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/ 2023-07-14T11:38:12.311 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 141241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/hfs.ko 2023-07-14T11:38:12.311 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/ 2023-07-14T11:38:12.312 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 221553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/hfsplus.ko 2023-07-14T11:38:12.312 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hpfs/ 2023-07-14T11:38:12.312 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 193577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hpfs/hpfs.ko 2023-07-14T11:38:12.312 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/ 2023-07-14T11:38:12.312 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 74393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/isofs.ko 2023-07-14T11:38:12.312 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/ 2023-07-14T11:38:12.312 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 420745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/jfs.ko 2023-07-14T11:38:12.312 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/ 2023-07-14T11:38:12.313 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 279041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/lockd.ko 2023-07-14T11:38:12.313 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/ 2023-07-14T11:38:12.313 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 87897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/minix.ko 2023-07-14T11:38:12.313 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/ 2023-07-14T11:38:12.313 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 124697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/netfs.ko 2023-07-14T11:38:12.313 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/ 2023-07-14T11:38:12.313 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/ 2023-07-14T11:38:12.313 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 81521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-14T11:38:12.314 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/ 2023-07-14T11:38:12.314 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-14T11:38:12.314 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/ 2023-07-14T11:38:12.314 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 117705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-14T11:38:12.314 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1053889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfs.ko 2023-07-14T11:38:12.314 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv2.ko 2023-07-14T11:38:12.314 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 119801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv3.ko 2023-07-14T11:38:12.314 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 2041809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv4.ko 2023-07-14T11:38:12.315 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/ 2023-07-14T11:38:12.315 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/grace.ko 2023-07-14T11:38:12.315 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/nfs_acl.ko 2023-07-14T11:38:12.315 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/ 2023-07-14T11:38:12.315 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1504993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/nfsd.ko 2023-07-14T11:38:12.315 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/ 2023-07-14T11:38:12.315 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 591417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/nilfs2.ko 2023-07-14T11:38:12.315 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/ 2023-07-14T11:38:12.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-celtic.ko 2023-07-14T11:38:12.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-centeuro.ko 2023-07-14T11:38:12.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-croatian.ko 2023-07-14T11:38:12.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-cyrillic.ko 2023-07-14T11:38:12.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-gaelic.ko 2023-07-14T11:38:12.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-greek.ko 2023-07-14T11:38:12.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-iceland.ko 2023-07-14T11:38:12.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-inuit.ko 2023-07-14T11:38:12.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-roman.ko 2023-07-14T11:38:12.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-romanian.ko 2023-07-14T11:38:12.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-turkish.ko 2023-07-14T11:38:12.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_ascii.ko 2023-07-14T11:38:12.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1250.ko 2023-07-14T11:38:12.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1251.ko 2023-07-14T11:38:12.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1255.ko 2023-07-14T11:38:12.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp737.ko 2023-07-14T11:38:12.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp775.ko 2023-07-14T11:38:12.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp850.ko 2023-07-14T11:38:12.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp852.ko 2023-07-14T11:38:12.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp855.ko 2023-07-14T11:38:12.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp857.ko 2023-07-14T11:38:12.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp860.ko 2023-07-14T11:38:12.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp861.ko 2023-07-14T11:38:12.319 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp862.ko 2023-07-14T11:38:12.319 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp863.ko 2023-07-14T11:38:12.319 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp864.ko 2023-07-14T11:38:12.319 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp865.ko 2023-07-14T11:38:12.319 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp866.ko 2023-07-14T11:38:12.319 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp869.ko 2023-07-14T11:38:12.319 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp874.ko 2023-07-14T11:38:12.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 94137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp932.ko 2023-07-14T11:38:12.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 142673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp936.ko 2023-07-14T11:38:12.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 165745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp949.ko 2023-07-14T11:38:12.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 118121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp950.ko 2023-07-14T11:38:12.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_euc-jp.ko 2023-07-14T11:38:12.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-1.ko 2023-07-14T11:38:12.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-13.ko 2023-07-14T11:38:12.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-14.ko 2023-07-14T11:38:12.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-15.ko 2023-07-14T11:38:12.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-2.ko 2023-07-14T11:38:12.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-3.ko 2023-07-14T11:38:12.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-4.ko 2023-07-14T11:38:12.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-5.ko 2023-07-14T11:38:12.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-6.ko 2023-07-14T11:38:12.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-7.ko 2023-07-14T11:38:12.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-9.ko 2023-07-14T11:38:12.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-r.ko 2023-07-14T11:38:12.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-ru.ko 2023-07-14T11:38:12.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-u.ko 2023-07-14T11:38:12.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_utf8.ko 2023-07-14T11:38:12.322 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ntfs/ 2023-07-14T11:38:12.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 219073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ntfs/ntfs.ko 2023-07-14T11:38:12.322 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ 2023-07-14T11:38:12.323 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ 2023-07-14T11:38:12.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 244177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-14T11:38:12.323 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ 2023-07-14T11:38:12.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 440785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-14T11:38:12.323 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ 2023-07-14T11:38:12.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-14T11:38:12.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 2589569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2.ko 2023-07-14T11:38:12.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-14T11:38:12.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-14T11:38:12.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-14T11:38:12.324 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/omfs/ 2023-07-14T11:38:12.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/omfs/omfs.ko 2023-07-14T11:38:12.324 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/ 2023-07-14T11:38:12.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 336473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/overlay.ko 2023-07-14T11:38:12.324 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ 2023-07-14T11:38:12.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 54609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ramoops.ko 2023-07-14T11:38:12.325 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/qnx4/ 2023-07-14T11:38:12.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/qnx4/qnx4.ko 2023-07-14T11:38:12.325 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/qnx6/ 2023-07-14T11:38:12.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/qnx6/qnx6.ko 2023-07-14T11:38:12.325 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/quota/ 2023-07-14T11:38:12.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/quota/quota_tree.ko 2023-07-14T11:38:12.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/quota/quota_v1.ko 2023-07-14T11:38:12.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/quota/quota_v2.ko 2023-07-14T11:38:12.326 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/ 2023-07-14T11:38:12.326 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 530697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/reiserfs.ko 2023-07-14T11:38:12.326 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/ 2023-07-14T11:38:12.326 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/romfs.ko 2023-07-14T11:38:12.326 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/ 2023-07-14T11:38:12.326 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/ 2023-07-14T11:38:12.326 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 2658777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/cifs.ko 2023-07-14T11:38:12.327 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/ 2023-07-14T11:38:12.327 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_arc4.ko 2023-07-14T11:38:12.327 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_md4.ko 2023-07-14T11:38:12.327 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/ 2023-07-14T11:38:12.327 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 134313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/squashfs.ko 2023-07-14T11:38:12.327 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/ 2023-07-14T11:38:12.327 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 89737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/sysv.ko 2023-07-14T11:38:12.327 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/ 2023-07-14T11:38:12.328 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 248049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/udf.ko 2023-07-14T11:38:12.328 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ 2023-07-14T11:38:12.328 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 160857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ufs.ko 2023-07-14T11:38:12.328 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/ 2023-07-14T11:38:12.328 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 3998129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/xfs.ko 2023-07-14T11:38:12.350 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/ 2023-07-14T11:38:12.351 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/locking/ 2023-07-14T11:38:12.351 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 74865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/locking/locktorture.ko 2023-07-14T11:38:12.351 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/time/ 2023-07-14T11:38:12.351 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/time/test_udelay.ko 2023-07-14T11:38:12.351 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 58473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/torture.ko 2023-07-14T11:38:12.352 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ 2023-07-14T11:38:12.352 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 6833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc-itu-t.ko 2023-07-14T11:38:12.352 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 6529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc7.ko 2023-07-14T11:38:12.352 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc8.ko 2023-07-14T11:38:12.352 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/ 2023-07-14T11:38:12.352 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libarc4.ko 2023-07-14T11:38:12.352 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libdes.ko 2023-07-14T11:38:12.352 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/interval_tree_test.ko 2023-07-14T11:38:12.353 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/libcrc32c.ko 2023-07-14T11:38:12.353 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lru_cache.ko 2023-07-14T11:38:12.353 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/ 2023-07-14T11:38:12.353 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4_compress.ko 2023-07-14T11:38:12.353 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22633 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4hc_compress.ko 2023-07-14T11:38:12.353 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/ 2023-07-14T11:38:12.353 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 6617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/cordic.ko 2023-07-14T11:38:12.354 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/memory-notifier-error-inject.ko 2023-07-14T11:38:12.354 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/notifier-error-inject.ko 2023-07-14T11:38:12.354 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/percpu_test.ko 2023-07-14T11:38:12.354 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/pm-notifier-error-inject.ko 2023-07-14T11:38:12.354 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/ 2023-07-14T11:38:12.354 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 143433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/raid6_pq.ko 2023-07-14T11:38:12.354 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/rbtree_test.ko 2023-07-14T11:38:12.354 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/ 2023-07-14T11:38:12.355 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-14T11:38:12.355 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/test-kstrtox.ko 2023-07-14T11:38:12.355 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/test-string_helpers.ko 2023-07-14T11:38:12.355 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 4760825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/test_bpf.ko 2023-07-14T11:38:12.355 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/test_firmware.ko 2023-07-14T11:38:12.355 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 6369 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/test_module.ko 2023-07-14T11:38:12.355 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/test_user_copy.ko 2023-07-14T11:38:12.355 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_bm.ko 2023-07-14T11:38:12.356 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_fsm.ko 2023-07-14T11:38:12.356 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_kmp.ko 2023-07-14T11:38:12.356 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/xz/ 2023-07-14T11:38:12.356 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/xz/xz_dec_test.ko 2023-07-14T11:38:12.356 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/ 2023-07-14T11:38:12.356 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/hwpoison-inject.ko 2023-07-14T11:38:12.356 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ 2023-07-14T11:38:12.357 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/ 2023-07-14T11:38:12.357 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/6lowpan.ko 2023-07-14T11:38:12.357 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_dest.ko 2023-07-14T11:38:12.357 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_fragment.ko 2023-07-14T11:38:12.357 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_hop.ko 2023-07-14T11:38:12.357 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 6985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-14T11:38:12.357 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_mobility.ko 2023-07-14T11:38:12.357 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_routing.ko 2023-07-14T11:38:12.358 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_udp.ko 2023-07-14T11:38:12.358 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/ 2023-07-14T11:38:12.358 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/garp.ko 2023-07-14T11:38:12.358 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/mrp.ko 2023-07-14T11:38:12.358 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/p8022.ko 2023-07-14T11:38:12.358 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/psnap.ko 2023-07-14T11:38:12.358 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/stp.ko 2023-07-14T11:38:12.358 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/ 2023-07-14T11:38:12.359 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 81017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/8021q.ko 2023-07-14T11:38:12.359 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/ 2023-07-14T11:38:12.359 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 198417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet.ko 2023-07-14T11:38:12.359 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_fd.ko 2023-07-14T11:38:12.359 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_rdma.ko 2023-07-14T11:38:12.359 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_virtio.ko 2023-07-14T11:38:12.359 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/ 2023-07-14T11:38:12.359 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 652553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/batman-adv.ko 2023-07-14T11:38:12.360 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/ 2023-07-14T11:38:12.360 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 63825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/br_netfilter.ko 2023-07-14T11:38:12.360 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 772505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/bridge.ko 2023-07-14T11:38:12.360 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ 2023-07-14T11:38:12.360 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-14T11:38:12.360 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-14T11:38:12.360 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-14T11:38:12.361 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-14T11:38:12.361 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-14T11:38:12.361 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-14T11:38:12.361 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-14T11:38:12.361 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-14T11:38:12.361 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-14T11:38:12.361 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7585 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-14T11:38:12.361 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-14T11:38:12.362 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8385 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-14T11:38:12.362 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-14T11:38:12.362 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-14T11:38:12.362 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-14T11:38:12.362 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-14T11:38:12.362 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-14T11:38:12.362 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-14T11:38:12.362 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-14T11:38:12.363 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-14T11:38:12.363 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtables.ko 2023-07-14T11:38:12.363 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-14T11:38:12.363 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-14T11:38:12.363 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/ 2023-07-14T11:38:12.363 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1046417 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/libceph.ko 2023-07-14T11:38:12.363 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/ 2023-07-14T11:38:12.364 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 124793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/pktgen.ko 2023-07-14T11:38:12.364 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/ 2023-07-14T11:38:12.364 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 194321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp.ko 2023-07-14T11:38:12.364 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8665 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_diag.ko 2023-07-14T11:38:12.364 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 56945 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv4.ko 2023-07-14T11:38:12.364 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv6.ko 2023-07-14T11:38:12.364 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/ 2023-07-14T11:38:12.365 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 390297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/dsa_core.ko 2023-07-14T11:38:12.365 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_brcm.ko 2023-07-14T11:38:12.365 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_dsa.ko 2023-07-14T11:38:12.365 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_none.ko 2023-07-14T11:38:12.365 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_trailer.ko 2023-07-14T11:38:12.365 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/hsr/ 2023-07-14T11:38:12.365 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 100257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/hsr/hsr.ko 2023-07-14T11:38:12.366 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ 2023-07-14T11:38:12.366 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ 2023-07-14T11:38:12.366 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-14T11:38:12.366 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 254601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154.ko 2023-07-14T11:38:12.366 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-14T11:38:12.366 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ 2023-07-14T11:38:12.366 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ah4.ko 2023-07-14T11:38:12.366 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/esp4.ko 2023-07-14T11:38:12.367 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/fou.ko 2023-07-14T11:38:12.367 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/gre.ko 2023-07-14T11:38:12.367 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/inet_diag.ko 2023-07-14T11:38:12.367 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60529 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_gre.ko 2023-07-14T11:38:12.367 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61281 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_tunnel.ko 2023-07-14T11:38:12.367 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34369 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_vti.ko 2023-07-14T11:38:12.367 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipcomp.ko 2023-07-14T11:38:12.367 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28889 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipip.ko 2023-07-14T11:38:12.368 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ 2023-07-14T11:38:12.368 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-14T11:38:12.368 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-14T11:38:12.368 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-14T11:38:12.368 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ip_tables.ko 2023-07-14T11:38:12.368 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-14T11:38:12.368 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-07-14T11:38:12.368 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-14T11:38:12.369 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-14T11:38:12.369 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-14T11:38:12.369 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-07-14T11:38:12.369 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-14T11:38:12.369 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-14T11:38:12.369 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-14T11:38:12.369 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-14T11:38:12.370 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-14T11:38:12.370 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-14T11:38:12.370 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-14T11:38:12.370 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-14T11:38:12.370 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-14T11:38:12.370 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20561 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-07-14T11:38:12.370 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-14T11:38:12.370 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15225 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-14T11:38:12.371 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-14T11:38:12.371 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-14T11:38:12.371 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-14T11:38:12.371 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_bic.ko 2023-07-14T11:38:12.371 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_dctcp.ko 2023-07-14T11:38:12.371 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_diag.ko 2023-07-14T11:38:12.371 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_highspeed.ko 2023-07-14T11:38:12.371 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_htcp.ko 2023-07-14T11:38:12.372 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_hybla.ko 2023-07-14T11:38:12.372 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_illinois.ko 2023-07-14T11:38:12.372 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_lp.ko 2023-07-14T11:38:12.372 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_scalable.ko 2023-07-14T11:38:12.372 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_vegas.ko 2023-07-14T11:38:12.372 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_veno.ko 2023-07-14T11:38:12.372 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_westwood.ko 2023-07-14T11:38:12.372 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_yeah.ko 2023-07-14T11:38:12.373 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tunnel4.ko 2023-07-14T11:38:12.373 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_diag.ko 2023-07-14T11:38:12.373 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_tunnel.ko 2023-07-14T11:38:12.373 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-14T11:38:12.373 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ 2023-07-14T11:38:12.373 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ah6.ko 2023-07-14T11:38:12.373 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/esp6.ko 2023-07-14T11:38:12.373 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/fou6.ko 2023-07-14T11:38:12.374 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 80793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_gre.ko 2023-07-14T11:38:12.374 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 92817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_tunnel.ko 2023-07-14T11:38:12.374 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-14T11:38:12.374 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 54001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_vti.ko 2023-07-14T11:38:12.374 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ipcomp6.ko 2023-07-14T11:38:12.374 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/mip6.ko 2023-07-14T11:38:12.374 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ 2023-07-14T11:38:12.374 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-14T11:38:12.375 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-07-14T11:38:12.375 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-14T11:38:12.375 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-14T11:38:12.375 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-14T11:38:12.375 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-14T11:38:12.375 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-14T11:38:12.375 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-14T11:38:12.376 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8913 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-14T11:38:12.376 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-14T11:38:12.376 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10937 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-14T11:38:12.376 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-14T11:38:12.376 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-14T11:38:12.376 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-14T11:38:12.376 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-14T11:38:12.376 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-14T11:38:12.377 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-14T11:38:12.377 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-14T11:38:12.377 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-14T11:38:12.377 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-14T11:38:12.377 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-14T11:38:12.377 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-14T11:38:12.377 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9633 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-14T11:38:12.377 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-14T11:38:12.378 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9009 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-14T11:38:12.378 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 73161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/sit.ko 2023-07-14T11:38:12.378 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24193 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/tunnel6.ko 2023-07-14T11:38:12.378 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23305 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-14T11:38:12.378 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/ 2023-07-14T11:38:12.378 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 82929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/af_key.ko 2023-07-14T11:38:12.378 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/ 2023-07-14T11:38:12.378 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 125873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_core.ko 2023-07-14T11:38:12.379 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-14T11:38:12.379 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_eth.ko 2023-07-14T11:38:12.379 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35049 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip.ko 2023-07-14T11:38:12.379 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip6.ko 2023-07-14T11:38:12.379 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_netlink.ko 2023-07-14T11:38:12.379 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53153 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ppp.ko 2023-07-14T11:38:12.379 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/lapb/ 2023-07-14T11:38:12.379 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/lapb/lapb.ko 2023-07-14T11:38:12.380 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/ 2023-07-14T11:38:12.380 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc.ko 2023-07-14T11:38:12.380 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 267017 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc2.ko 2023-07-14T11:38:12.380 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/ 2023-07-14T11:38:12.380 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 291041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/mac802154.ko 2023-07-14T11:38:12.380 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/ 2023-07-14T11:38:12.380 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/mpls_gso.ko 2023-07-14T11:38:12.381 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ 2023-07-14T11:38:12.381 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ 2023-07-14T11:38:12.381 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 119649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set.ko 2023-07-14T11:38:12.381 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-14T11:38:12.381 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-14T11:38:12.381 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-14T11:38:12.381 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 102033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-14T11:38:12.381 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 103121 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-14T11:38:12.382 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 104593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-14T11:38:12.382 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 105129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-14T11:38:12.382 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 114393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-14T11:38:12.382 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-14T11:38:12.382 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 111161 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-14T11:38:12.382 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 114673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-14T11:38:12.382 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 114329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-14T11:38:12.382 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 113217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-14T11:38:12.383 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 116217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-14T11:38:12.383 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-14T11:38:12.383 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ 2023-07-14T11:38:12.383 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 412481 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-14T11:38:12.383 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-14T11:38:12.383 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-14T11:38:12.383 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-14T11:38:12.384 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-14T11:38:12.384 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-14T11:38:12.384 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-14T11:38:12.384 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-14T11:38:12.384 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10681 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-14T11:38:12.384 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-14T11:38:12.384 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-14T11:38:12.384 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-14T11:38:12.385 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7697 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-14T11:38:12.385 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10993 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-14T11:38:12.385 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conncount.ko 2023-07-14T11:38:12.385 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 408689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack.ko 2023-07-14T11:38:12.385 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-14T11:38:12.386 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10177 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-14T11:38:12.386 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-14T11:38:12.386 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 118145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-14T11:38:12.386 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-14T11:38:12.386 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-14T11:38:12.386 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 108393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-14T11:38:12.386 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-14T11:38:12.386 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-14T11:38:12.387 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-14T11:38:12.387 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-14T11:38:12.387 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-14T11:38:12.387 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10113 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-14T11:38:12.387 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 88513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table.ko 2023-07-14T11:38:12.387 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9593 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-14T11:38:12.387 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_log_syslog.ko 2023-07-14T11:38:12.388 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 95353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat.ko 2023-07-14T11:38:12.388 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-14T11:38:12.388 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-14T11:38:12.388 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_irc.ko 2023-07-14T11:38:12.388 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28929 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_sip.ko 2023-07-14T11:38:12.388 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-14T11:38:12.388 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-14T11:38:12.388 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 592185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_tables.ko 2023-07-14T11:38:12.389 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink.ko 2023-07-14T11:38:12.389 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-14T11:38:12.389 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-07-14T11:38:12.389 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39905 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_log.ko 2023-07-14T11:38:12.389 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22713 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-14T11:38:12.389 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-14T11:38:12.389 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_chain_nat.ko 2023-07-14T11:38:12.390 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_compat.ko 2023-07-14T11:38:12.390 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40721 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_ct.ko 2023-07-14T11:38:12.390 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-14T11:38:12.390 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11345 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fib.ko 2023-07-14T11:38:12.390 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13745 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-14T11:38:12.390 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14185 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_hash.ko 2023-07-14T11:38:12.390 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_limit.ko 2023-07-14T11:38:12.391 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_log.ko 2023-07-14T11:38:12.391 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_masq.ko 2023-07-14T11:38:12.391 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_nat.ko 2023-07-14T11:38:12.391 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_numgen.ko 2023-07-14T11:38:12.391 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_osf.ko 2023-07-14T11:38:12.391 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_queue.ko 2023-07-14T11:38:12.391 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_redir.ko 2023-07-14T11:38:12.391 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject.ko 2023-07-14T11:38:12.392 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject_inet.ko 2023-07-14T11:38:12.392 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12737 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_socket.ko 2023-07-14T11:38:12.392 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20393 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_synproxy.ko 2023-07-14T11:38:12.392 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14849 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_tproxy.ko 2023-07-14T11:38:12.392 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 89329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/x_tables.ko 2023-07-14T11:38:12.392 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_AUDIT.ko 2023-07-14T11:38:12.392 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9633 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-14T11:38:12.392 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-14T11:38:12.393 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-14T11:38:12.393 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18169 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CT.ko 2023-07-14T11:38:12.393 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10497 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_DSCP.ko 2023-07-14T11:38:12.393 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HL.ko 2023-07-14T11:38:12.393 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HMARK.ko 2023-07-14T11:38:12.393 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38857 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-14T11:38:12.393 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13873 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LED.ko 2023-07-14T11:38:12.394 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11489 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LOG.ko 2023-07-14T11:38:12.394 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-14T11:38:12.394 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NETMAP.ko 2023-07-14T11:38:12.394 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFLOG.ko 2023-07-14T11:38:12.394 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13289 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-14T11:38:12.394 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19689 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_RATEEST.ko 2023-07-14T11:38:12.394 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-14T11:38:12.394 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_SECMARK.ko 2023-07-14T11:38:12.395 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-14T11:38:12.395 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-14T11:38:12.395 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17369 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TEE.ko 2023-07-14T11:38:12.395 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TPROXY.ko 2023-07-14T11:38:12.395 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TRACE.ko 2023-07-14T11:38:12.395 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_addrtype.ko 2023-07-14T11:38:12.395 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_bpf.ko 2023-07-14T11:38:12.395 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15577 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cgroup.ko 2023-07-14T11:38:12.396 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11209 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cluster.ko 2023-07-14T11:38:12.396 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_comment.ko 2023-07-14T11:38:12.396 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10465 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connbytes.ko 2023-07-14T11:38:12.396 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlabel.ko 2023-07-14T11:38:12.396 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlimit.ko 2023-07-14T11:38:12.396 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connmark.ko 2023-07-14T11:38:12.396 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_conntrack.ko 2023-07-14T11:38:12.397 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cpu.ko 2023-07-14T11:38:12.397 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dccp.ko 2023-07-14T11:38:12.397 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_devgroup.ko 2023-07-14T11:38:12.397 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8801 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dscp.ko 2023-07-14T11:38:12.397 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ecn.ko 2023-07-14T11:38:12.397 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_esp.ko 2023-07-14T11:38:12.397 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hashlimit.ko 2023-07-14T11:38:12.397 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_helper.ko 2023-07-14T11:38:12.398 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7865 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hl.ko 2023-07-14T11:38:12.398 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11921 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipcomp.ko 2023-07-14T11:38:12.398 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_iprange.ko 2023-07-14T11:38:12.398 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11785 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipvs.ko 2023-07-14T11:38:12.398 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16041 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_l2tp.ko 2023-07-14T11:38:12.398 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8001 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_length.ko 2023-07-14T11:38:12.398 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10273 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_limit.ko 2023-07-14T11:38:12.398 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mac.ko 2023-07-14T11:38:12.399 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8641 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mark.ko 2023-07-14T11:38:12.399 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_multiport.ko 2023-07-14T11:38:12.399 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nat.ko 2023-07-14T11:38:12.399 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nfacct.ko 2023-07-14T11:38:12.399 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8297 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_osf.ko 2023-07-14T11:38:12.399 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_owner.ko 2023-07-14T11:38:12.399 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_physdev.ko 2023-07-14T11:38:12.399 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7633 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_pkttype.ko 2023-07-14T11:38:12.400 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_policy.ko 2023-07-14T11:38:12.400 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_quota.ko 2023-07-14T11:38:12.400 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_rateest.ko 2023-07-14T11:38:12.400 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8257 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_realm.ko 2023-07-14T11:38:12.400 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_recent.ko 2023-07-14T11:38:12.400 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13329 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_sctp.ko 2023-07-14T11:38:12.400 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_set.ko 2023-07-14T11:38:12.401 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16089 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_socket.ko 2023-07-14T11:38:12.401 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9505 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_state.ko 2023-07-14T11:38:12.401 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9473 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_statistic.ko 2023-07-14T11:38:12.401 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_string.ko 2023-07-14T11:38:12.401 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8553 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpmss.ko 2023-07-14T11:38:12.401 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15777 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpudp.ko 2023-07-14T11:38:12.401 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11129 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_time.ko 2023-07-14T11:38:12.401 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9105 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_u32.ko 2023-07-14T11:38:12.402 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/ 2023-07-14T11:38:12.402 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13969 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/netlink_diag.ko 2023-07-14T11:38:12.402 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/ 2023-07-14T11:38:12.402 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/nsh.ko 2023-07-14T11:38:12.402 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/ 2023-07-14T11:38:12.402 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 379545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/openvswitch.ko 2023-07-14T11:38:12.402 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10321 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-geneve.ko 2023-07-14T11:38:12.402 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-gre.ko 2023-07-14T11:38:12.403 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-vxlan.ko 2023-07-14T11:38:12.403 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/ 2023-07-14T11:38:12.403 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/af_packet_diag.ko 2023-07-14T11:38:12.403 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/phonet/ 2023-07-14T11:38:12.403 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 109961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/phonet/phonet.ko 2023-07-14T11:38:12.403 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 54761 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/phonet/pn_pep.ko 2023-07-14T11:38:12.403 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/ 2023-07-14T11:38:12.404 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 252825 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds.ko 2023-07-14T11:38:12.404 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 165081 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_rdma.ko 2023-07-14T11:38:12.404 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 67241 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_tcp.ko 2023-07-14T11:38:12.404 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rxrpc/ 2023-07-14T11:38:12.404 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 820137 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rxrpc/rxrpc.ko 2023-07-14T11:38:12.404 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/ 2023-07-14T11:38:12.404 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_csum.ko 2023-07-14T11:38:12.404 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23537 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_gact.ko 2023-07-14T11:38:12.405 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_ipt.ko 2023-07-14T11:38:12.405 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_mirred.ko 2023-07-14T11:38:12.405 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_nat.ko 2023-07-14T11:38:12.405 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43817 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_pedit.ko 2023-07-14T11:38:12.405 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_police.ko 2023-07-14T11:38:12.405 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20313 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_simple.ko 2023-07-14T11:38:12.405 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36353 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_skbedit.ko 2023-07-14T11:38:12.405 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36057 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_vlan.ko 2023-07-14T11:38:12.406 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26145 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_basic.ko 2023-07-14T11:38:12.406 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33673 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_bpf.ko 2023-07-14T11:38:12.406 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_cgroup.ko 2023-07-14T11:38:12.406 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31377 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_flow.ko 2023-07-14T11:38:12.406 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_fw.ko 2023-07-14T11:38:12.406 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_route.ko 2023-07-14T11:38:12.406 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59217 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_u32.ko 2023-07-14T11:38:12.407 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7441 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_cmp.ko 2023-07-14T11:38:12.407 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13457 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_ipset.ko 2023-07-14T11:38:12.407 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33337 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_meta.ko 2023-07-14T11:38:12.407 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_nbyte.ko 2023-07-14T11:38:12.407 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10625 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_text.ko 2023-07-14T11:38:12.407 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7249 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_u32.ko 2023-07-14T11:38:12.407 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_choke.ko 2023-07-14T11:38:12.407 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20961 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_codel.ko 2023-07-14T11:38:12.408 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33753 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_drr.ko 2023-07-14T11:38:12.408 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33609 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq.ko 2023-07-14T11:38:12.408 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq_codel.ko 2023-07-14T11:38:12.408 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35433 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_gred.ko 2023-07-14T11:38:12.408 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48409 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hfsc.ko 2023-07-14T11:38:12.408 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22073 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hhf.ko 2023-07-14T11:38:12.408 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 65793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_htb.ko 2023-07-14T11:38:12.409 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_ingress.ko 2023-07-14T11:38:12.409 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29809 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio.ko 2023-07-14T11:38:12.409 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10793 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio_lib.ko 2023-07-14T11:38:12.409 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26065 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_multiq.ko 2023-07-14T11:38:12.409 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35601 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_netem.ko 2023-07-14T11:38:12.409 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27545 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_pie.ko 2023-07-14T11:38:12.409 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_plug.ko 2023-07-14T11:38:12.409 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_prio.ko 2023-07-14T11:38:12.410 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47617 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_qfq.ko 2023-07-14T11:38:12.410 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32953 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_red.ko 2023-07-14T11:38:12.410 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26881 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfb.ko 2023-07-14T11:38:12.410 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31425 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfq.ko 2023-07-14T11:38:12.410 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28897 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_tbf.ko 2023-07-14T11:38:12.410 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_teql.ko 2023-07-14T11:38:12.410 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/ 2023-07-14T11:38:12.410 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 930369 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp.ko 2023-07-14T11:38:12.411 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp_diag.ko 2023-07-14T11:38:12.411 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/ 2023-07-14T11:38:12.411 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/ 2023-07-14T11:38:12.411 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 335569 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-14T11:38:12.411 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61833 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-14T11:38:12.411 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1514649 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/sunrpc.ko 2023-07-14T11:38:12.411 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/ 2023-07-14T11:38:12.411 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 707401 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-14T11:38:12.412 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/ 2023-07-14T11:38:12.412 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10025 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/diag.ko 2023-07-14T11:38:12.412 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 862233 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/tipc.ko 2023-07-14T11:38:12.412 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/ 2023-07-14T11:38:12.412 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15729 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/unix_diag.ko 2023-07-14T11:38:12.412 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/ 2023-07-14T11:38:12.412 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 84841 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-14T11:38:12.413 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 68033 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-14T11:38:12.413 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 95985 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock.ko 2023-07-14T11:38:12.413 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-14T11:38:12.413 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17521 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-14T11:38:12.413 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/x25/ 2023-07-14T11:38:12.413 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 125097 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/x25/x25.ko 2023-07-14T11:38:12.413 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/ 2023-07-14T11:38:12.413 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22657 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_algo.ko 2023-07-14T11:38:12.414 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20705 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-14T11:38:12.414 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 102265 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_user.ko 2023-07-14T11:38:12.414 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/ 2023-07-14T11:38:12.414 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/ 2023-07-14T11:38:12.414 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15513 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/irqbypass.ko 2023-07-14T11:38:12.414 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 638200 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias 2023-07-14T11:38:12.414 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 623896 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias.bin 2023-07-14T11:38:12.414 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7656 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin 2023-07-14T11:38:12.415 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23361 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.alias.bin 2023-07-14T11:38:12.415 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9821 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.bin 2023-07-14T11:38:12.415 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69903 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.modinfo 2023-07-14T11:38:12.415 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 146449 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep 2023-07-14T11:38:12.416 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 232695 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep.bin 2023-07-14T11:38:12.416 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 210 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.devname 2023-07-14T11:38:12.416 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 72754 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.order 2023-07-14T11:38:12.416 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 879 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.softdep 2023-07-14T11:38:12.416 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 252883 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols 2023-07-14T11:38:12.416 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 307884 2023-07-13 12:25 ./lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols.bin 2023-07-14T11:38:12.416 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./usr/ 2023-07-14T11:38:12.416 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./usr/share/ 2023-07-14T11:38:12.417 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./usr/share/doc/ 2023-07-14T11:38:12.417 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2023-07-13 12:25 ./usr/share/doc/linux-image-6.5.0-rc1-ga440c79eee02/ 2023-07-14T11:38:12.417 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 174 2023-07-13 12:25 ./usr/share/doc/linux-image-6.5.0-rc1-ga440c79eee02/changelog.Debian.gz 2023-07-14T11:38:12.417 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 691 2023-07-13 12:25 ./usr/share/doc/linux-image-6.5.0-rc1-ga440c79eee02/copyright 2023-07-14T11:38:12.418 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:12.418 INFO:teuthology.task.kernel:searching for kernel 6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:12.418 DEBUG:teuthology.orchestra.run.smithi104:> egrep '(submenu|menuentry.*6.5.0-rc1-ga440c79eee02).*{' /boot/grub/grub.cfg 2023-07-14T11:38:12.427 INFO:teuthology.orchestra.run.smithi104.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-07-14T11:38:12.428 INFO:teuthology.orchestra.run.smithi104.stdout: menuentry 'Ubuntu, with Linux 6.5.0-rc1-ga440c79eee02' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.5.0-rc1-ga440c79eee02-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-07-14T11:38:12.428 INFO:teuthology.orchestra.run.smithi104.stdout: menuentry 'Ubuntu, with Linux 6.5.0-rc1-ga440c79eee02 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.5.0-rc1-ga440c79eee02-recovery-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-07-14T11:38:12.429 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-07-14T11:38:12.429 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:12.429 DEBUG:teuthology.orchestra.run.smithi104:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-07-14T11:38:12.474 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi104.front.sepia.ceph.com to complete... 2023-07-14T11:38:13.092 INFO:teuthology.orchestra.run.smithi104.stderr:Sourcing file `/etc/default/grub' 2023-07-14T11:38:13.103 INFO:teuthology.orchestra.run.smithi104.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-07-14T11:38:13.121 INFO:teuthology.orchestra.run.smithi104.stderr:Generating grub configuration file ... 2023-07-14T11:38:13.365 INFO:teuthology.orchestra.run.smithi104.stderr:Found linux image: /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:13.377 INFO:teuthology.orchestra.run.smithi104.stderr:Found initrd image: /boot/initrd.img-6.5.0-rc1-ga440c79eee02 2023-07-14T11:38:13.757 INFO:teuthology.orchestra.run.smithi104.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-07-14T11:38:13.769 INFO:teuthology.orchestra.run.smithi104.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-07-14T11:38:15.432 INFO:teuthology.orchestra.run.smithi104.stderr:done 2023-07-14T11:38:35.744 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-07-14T11:38:35.745 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-07-14T11:38:35.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:38:46.552 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-07-14T11:38:46.553 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2023-07-14T11:38:46.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:38:54.196 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-07-14T11:39:03.204 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-07-14T11:39:03.205 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:39:04.948 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.104 2023-07-14T11:39:06.260 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-07-14T11:39:13.956 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2023-07-14T11:39:13.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:39:17.008 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.104 2023-07-14T11:39:18.262 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-07-14T11:39:18.263 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:39:29.019 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2023-07-14T11:39:29.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:39:36.688 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-07-14T11:39:47.444 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.104 2023-07-14T11:39:51.692 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-07-14T11:39:51.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:39:52.138 DEBUG:teuthology.orchestra.run.smithi121:> true 2023-07-14T11:39:52.862 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-07-14T11:39:52.862 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.5.0-rc1-ga440c79eee02"... 2023-07-14T11:39:52.862 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2023-07-14T11:39:52.920 INFO:teuthology.orchestra.run.smithi121.stdout:6.5.0-rc1-ga440c79eee02 2023-07-14T11:39:52.920 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-ga440c79eee02 vs 6.5.0-rc1-ga440c79eee02 2023-07-14T11:39:52.921 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-14T11:39:52.921 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-07-14T11:39:53.922 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-07-14T11:39:53.922 DEBUG:teuthology.orchestra.run.smithi121:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-14T11:39:54.023 INFO:teuthology.orchestra.run.smithi121.stdout:ttyS1 2023-07-14T11:39:54.086 DEBUG:teuthology.parallel:result is None 2023-07-14T11:40:02.445 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2023-07-14T11:40:02.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:40:02.694 DEBUG:teuthology.orchestra.run.smithi104:> true 2023-07-14T11:40:03.491 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2023-07-14T11:40:03.491 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.5.0-rc1-ga440c79eee02"... 2023-07-14T11:40:03.491 DEBUG:teuthology.orchestra.run.smithi104:> uname -r 2023-07-14T11:40:03.551 INFO:teuthology.orchestra.run.smithi104.stdout:6.5.0-rc1-ga440c79eee02 2023-07-14T11:40:03.552 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-ga440c79eee02 vs 6.5.0-rc1-ga440c79eee02 2023-07-14T11:40:03.552 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-14T11:40:03.552 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-07-14T11:40:04.553 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-07-14T11:40:04.554 DEBUG:teuthology.orchestra.run.smithi104:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-14T11:40:04.671 INFO:teuthology.orchestra.run.smithi104.stdout:ttyS1 2023-07-14T11:40:04.725 DEBUG:teuthology.parallel:result is None 2023-07-14T11:40:04.725 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-14T11:40:04.736 INFO:teuthology.task.internal:Creating test directory... 2023-07-14T11:40:04.736 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T11:40:04.743 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T11:40:04.762 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-14T11:40:04.774 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-14T11:40:04.803 INFO:teuthology.task.internal:Creating archive directory... 2023-07-14T11:40:04.804 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T11:40:04.811 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T11:40:04.870 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-14T11:40:04.882 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-14T11:40:04.884 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T11:40:04.892 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T11:40:04.931 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T11:40:04.943 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T11:40:04.952 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T11:40:04.968 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T11:40:04.971 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-14T11:40:04.980 INFO:teuthology.task.internal:Configuring sudo... 2023-07-14T11:40:04.981 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T11:40:05.006 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T11:40:05.049 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-14T11:40:05.067 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-14T11:40:05.068 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T11:40:05.085 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T11:40:05.109 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T11:40:05.158 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T11:40:05.216 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:40:05.217 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T11:40:05.323 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T11:40:05.341 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T11:40:05.407 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:40:05.408 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T11:40:05.488 DEBUG:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2023-07-14T11:40:05.497 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2023-07-14T11:40:05.681 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-14T11:40:05.690 INFO:teuthology.task.internal:Starting timer... 2023-07-14T11:40:05.690 INFO:teuthology.run_tasks:Running task pcp... 2023-07-14T11:40:05.723 INFO:teuthology.run_tasks:Running task selinux... 2023-07-14T11:40:05.754 DEBUG:teuthology.task.selinux:Excluding smithi104: OS 'ubuntu' does not support SELinux 2023-07-14T11:40:05.754 DEBUG:teuthology.task.selinux:Excluding smithi121: OS 'ubuntu' does not support SELinux 2023-07-14T11:40:05.755 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-14T11:40:05.755 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-07-14T11:40:05.755 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-14T11:40:05.755 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-14T11:40:05.791 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-07-14T11:40:05.798 INFO:teuthology.repo_utils:Fetching main from origin 2023-07-14T11:40:05.933 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-14T11:40:05.962 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-14T11:40:05.963 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi104.front.sepia.ceph.com,smithi121.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-14T11:48:00.174 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi104.front.sepia.ceph.com'), Remote(name='ubuntu@smithi121.front.sepia.ceph.com')] 2023-07-14T11:48:00.176 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2023-07-14T11:48:00.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:48:00.283 DEBUG:teuthology.orchestra.run.smithi104:> true 2023-07-14T11:48:00.512 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2023-07-14T11:48:00.513 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-07-14T11:48:00.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:48:00.618 DEBUG:teuthology.orchestra.run.smithi121:> true 2023-07-14T11:48:00.844 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-07-14T11:48:00.845 INFO:teuthology.run_tasks:Running task clock... 2023-07-14T11:48:00.856 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-14T11:48:00.856 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T11:48:00.857 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T11:48:00.864 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T11:48:00.865 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T11:48:00.940 INFO:teuthology.orchestra.run.smithi104.stdout:14 Jul 11:48:00 ntpd[22104]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-07-14T11:48:00.941 INFO:teuthology.orchestra.run.smithi104.stdout:14 Jul 11:48:00 ntpd[22104]: Command line: ntpd -gq 2023-07-14T11:48:00.941 INFO:teuthology.orchestra.run.smithi104.stdout:14 Jul 11:48:00 ntpd[22104]: proto: precision = 0.040 usec (-24) 2023-07-14T11:48:00.942 INFO:teuthology.orchestra.run.smithi104.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-07-14T11:48:00.942 INFO:teuthology.orchestra.run.smithi104.stdout:14 Jul 11:48:00 ntpd[22104]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-07-14T11:48:00.942 INFO:teuthology.orchestra.run.smithi104.stderr:restrict ::: KOD does nothing without LIMITED. 2023-07-14T11:48:00.943 INFO:teuthology.orchestra.run.smithi104.stdout:14 Jul 11:48:00 ntpd[22104]: restrict ::: KOD does nothing without LIMITED. 2023-07-14T11:48:00.944 INFO:teuthology.orchestra.run.smithi104.stdout:14 Jul 11:48:00 ntpd[22104]: Listen and drop on 0 v6wildcard [::]:123 2023-07-14T11:48:00.944 INFO:teuthology.orchestra.run.smithi104.stdout:14 Jul 11:48:00 ntpd[22104]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-07-14T11:48:00.944 INFO:teuthology.orchestra.run.smithi104.stdout:14 Jul 11:48:00 ntpd[22104]: Listen normally on 2 lo 127.0.0.1:123 2023-07-14T11:48:00.944 INFO:teuthology.orchestra.run.smithi104.stdout:14 Jul 11:48:00 ntpd[22104]: Listen normally on 3 enp3s0f1 172.21.15.104:123 2023-07-14T11:48:00.944 INFO:teuthology.orchestra.run.smithi104.stdout:14 Jul 11:48:00 ntpd[22104]: Listen normally on 4 lo [::1]:123 2023-07-14T11:48:00.944 INFO:teuthology.orchestra.run.smithi104.stdout:14 Jul 11:48:00 ntpd[22104]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce89%5]:123 2023-07-14T11:48:00.944 INFO:teuthology.orchestra.run.smithi104.stdout:14 Jul 11:48:00 ntpd[22104]: Listening on routing socket on fd #22 for interface updates 2023-07-14T11:48:00.964 INFO:teuthology.orchestra.run.smithi121.stdout:14 Jul 11:48:00 ntpd[22075]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-07-14T11:48:00.964 INFO:teuthology.orchestra.run.smithi121.stdout:14 Jul 11:48:00 ntpd[22075]: Command line: ntpd -gq 2023-07-14T11:48:00.966 INFO:teuthology.orchestra.run.smithi121.stdout:14 Jul 11:48:00 ntpd[22075]: proto: precision = 0.066 usec (-24) 2023-07-14T11:48:00.966 INFO:teuthology.orchestra.run.smithi121.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-07-14T11:48:00.967 INFO:teuthology.orchestra.run.smithi121.stdout:14 Jul 11:48:00 ntpd[22075]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-07-14T11:48:00.967 INFO:teuthology.orchestra.run.smithi121.stderr:restrict ::: KOD does nothing without LIMITED. 2023-07-14T11:48:00.968 INFO:teuthology.orchestra.run.smithi121.stdout:14 Jul 11:48:00 ntpd[22075]: restrict ::: KOD does nothing without LIMITED. 2023-07-14T11:48:00.968 INFO:teuthology.orchestra.run.smithi121.stdout:14 Jul 11:48:00 ntpd[22075]: Listen and drop on 0 v6wildcard [::]:123 2023-07-14T11:48:00.968 INFO:teuthology.orchestra.run.smithi121.stdout:14 Jul 11:48:00 ntpd[22075]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-07-14T11:48:00.969 INFO:teuthology.orchestra.run.smithi121.stdout:14 Jul 11:48:00 ntpd[22075]: Listen normally on 2 lo 127.0.0.1:123 2023-07-14T11:48:00.969 INFO:teuthology.orchestra.run.smithi121.stdout:14 Jul 11:48:00 ntpd[22075]: Listen normally on 3 enp3s0f1 172.21.15.121:123 2023-07-14T11:48:00.969 INFO:teuthology.orchestra.run.smithi121.stdout:14 Jul 11:48:00 ntpd[22075]: Listen normally on 4 lo [::1]:123 2023-07-14T11:48:00.969 INFO:teuthology.orchestra.run.smithi121.stdout:14 Jul 11:48:00 ntpd[22075]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cba3%5]:123 2023-07-14T11:48:00.969 INFO:teuthology.orchestra.run.smithi121.stdout:14 Jul 11:48:00 ntpd[22075]: Listening on routing socket on fd #22 for interface updates 2023-07-14T11:48:01.965 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:01 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:01.965 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:01 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:01.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:01 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:01.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:01 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:01.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:01 ntpd[22075]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:01.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:01 ntpd[22075]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:02.965 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:02 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:02.965 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:02 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:02.965 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:02 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:02.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:02 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:02.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:02 ntpd[22075]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:02.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:02 ntpd[22075]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:03.941 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:03 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:03.941 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:03 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:03.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:03 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:03.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:03 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:03.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:03 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:03.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:03 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:03.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:03 ntpd[22104]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:03.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:03 ntpd[22104]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:03.965 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:03 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:03.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:03 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:03.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:03 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:03.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:03 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:03.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:03 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:03.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:03 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:03.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:03 ntpd[22075]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:03.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:03 ntpd[22075]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:04.941 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:04 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:04.941 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:04 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:04.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:04 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:04.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:04 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:04.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:04 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:04.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:04 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:04.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:04 ntpd[22104]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:04.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:04 ntpd[22104]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:04.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:04 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:04.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:04 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:04.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:04 ntpd[22104]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:04.944 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:04 ntpd[22104]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:04.965 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:04 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:04.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:04 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:04.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:04 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:04.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:04 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:04.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:04 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:04.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:04 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:04.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:04 ntpd[22075]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:04.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:04 ntpd[22075]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:04.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:04 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:04.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:04 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:04.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:04 ntpd[22075]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:04.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:04 ntpd[22075]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:05.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:05 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:05.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:05 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:05.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:05 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:05.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:05 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:05.944 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:05 ntpd[22104]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:05.945 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:05 ntpd[22104]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:05.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:05 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:05.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:05 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:05.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:05 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:05.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:05 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:05.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:05 ntpd[22075]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:05.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:05 ntpd[22075]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:06.941 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:06 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:06.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:06 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:06.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:06 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:06.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:06 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:06.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:06 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:06.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:06 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:06.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:06 ntpd[22104]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:06.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:06 ntpd[22104]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:06.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:06 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:06.944 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:06 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:06.944 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:06 ntpd[22104]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:06.944 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:06 ntpd[22104]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:06.965 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:06 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:06.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:06 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:06.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:06 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:06.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:06 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:06.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:06 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:06.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:06 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:06.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:06 ntpd[22075]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:06.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:06 ntpd[22075]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:06.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:06 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:06.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:06 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:06.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:06 ntpd[22075]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:06.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:06 ntpd[22075]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:07.941 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:07 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:07.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:07 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:07.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:07 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:07.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:07 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:07.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:07 ntpd[22104]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:07.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:07 ntpd[22104]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:07.965 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:07 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:07.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:07 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:07.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:07 ntpd[22075]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:07.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:07 ntpd[22075]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:07.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:07 ntpd[22075]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:07.966 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:07 ntpd[22075]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:07.967 INFO:teuthology.orchestra.run.smithi121.stdout:14 Jul 11:48:07 ntpd[22075]: ntpd: time slew -0.001225 s 2023-07-14T11:48:07.967 INFO:teuthology.orchestra.run.smithi121.stdout:ntpd: time slew -0.001225s 2023-07-14T11:48:07.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:07 ntpd[22075]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-07-14T11:48:07.967 INFO:teuthology.orchestra.run.smithi121.stderr:14 Jul 11:48:07 ntpd[22075]: can't open /var/log/ntpstats/loopstats.20230714: Permission denied 2023-07-14T11:48:08.113 INFO:teuthology.orchestra.run.smithi121.stdout: remote refid st t when poll reach delay offset jitter 2023-07-14T11:48:08.114 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================== 2023-07-14T11:48:08.114 INFO:teuthology.orchestra.run.smithi121.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-07-14T11:48:08.114 INFO:teuthology.orchestra.run.smithi121.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-07-14T11:48:08.114 INFO:teuthology.orchestra.run.smithi121.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-07-14T11:48:08.114 INFO:teuthology.orchestra.run.smithi121.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-07-14T11:48:08.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:08 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:08.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:08 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:08.944 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:08 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:08.944 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:08 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:08.944 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:08 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:08.945 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:08 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:08.945 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:08 ntpd[22104]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:08.945 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:08 ntpd[22104]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:08.945 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:08 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:08.946 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:08 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:08.946 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:08 ntpd[22104]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:08.946 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:08 ntpd[22104]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:09.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:09 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:09.942 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:09 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:09.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:09 ntpd[22104]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-07-14T11:48:09.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:09 ntpd[22104]: can't open /var/log/ntpstats/rawstats.20230714: Permission denied 2023-07-14T11:48:09.943 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:09 ntpd[22104]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-07-14T11:48:09.944 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:09 ntpd[22104]: can't open /var/log/ntpstats/peerstats.20230714: Permission denied 2023-07-14T11:48:09.944 INFO:teuthology.orchestra.run.smithi104.stdout:14 Jul 11:48:09 ntpd[22104]: ntpd: time slew +0.001791 s 2023-07-14T11:48:09.945 INFO:teuthology.orchestra.run.smithi104.stdout:ntpd: time slew +0.001791s 2023-07-14T11:48:09.945 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:09 ntpd[22104]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-07-14T11:48:09.946 INFO:teuthology.orchestra.run.smithi104.stderr:14 Jul 11:48:09 ntpd[22104]: can't open /var/log/ntpstats/loopstats.20230714: Permission denied 2023-07-14T11:48:10.069 INFO:teuthology.orchestra.run.smithi104.stdout: remote refid st t when poll reach delay offset jitter 2023-07-14T11:48:10.069 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================== 2023-07-14T11:48:10.070 INFO:teuthology.orchestra.run.smithi104.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-07-14T11:48:10.070 INFO:teuthology.orchestra.run.smithi104.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-07-14T11:48:10.070 INFO:teuthology.orchestra.run.smithi104.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-07-14T11:48:10.070 INFO:teuthology.orchestra.run.smithi104.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-07-14T11:48:10.071 INFO:teuthology.run_tasks:Running task install... 2023-07-14T11:48:10.081 DEBUG:teuthology.task.install:project ceph 2023-07-14T11:48:10.081 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'}} 2023-07-14T11:48:10.081 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'} 2023-07-14T11:48:10.082 INFO:teuthology.task.install:Using flavor: default 2023-07-14T11:48:10.088 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-14T11:48:10.089 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-14T11:48:10.090 DEBUG:teuthology.orchestra.run.smithi104:> sudo apt-key list | grep Ceph 2023-07-14T11:48:10.091 DEBUG:teuthology.orchestra.run.smithi121:> sudo apt-key list | grep Ceph 2023-07-14T11:48:10.317 INFO:teuthology.orchestra.run.smithi104.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-07-14T11:48:10.332 INFO:teuthology.orchestra.run.smithi121.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-07-14T11:48:10.464 INFO:teuthology.orchestra.run.smithi104.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-07-14T11:48:10.465 INFO:teuthology.orchestra.run.smithi104.stdout:uid [ unknown] Ceph.com (release key) 2023-07-14T11:48:10.466 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-07-14T11:48:10.466 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-07-14T11:48:10.467 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:48:10.471 INFO:teuthology.orchestra.run.smithi121.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-07-14T11:48:10.471 INFO:teuthology.orchestra.run.smithi121.stdout:uid [ unknown] Ceph.com (release key) 2023-07-14T11:48:10.473 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-07-14T11:48:10.473 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-07-14T11:48:10.473 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:48:10.843 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default/ 2023-07-14T11:48:10.843 INFO:teuthology.task.install.deb:Package version is 18.0.0-4757-g3fc0089c-1focal 2023-07-14T11:48:10.857 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default/ 2023-07-14T11:48:10.857 INFO:teuthology.task.install.deb:Package version is 18.0.0-4757-g3fc0089c-1focal 2023-07-14T11:48:11.153 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:48:11.154 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-07-14T11:48:11.159 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:48:11.159 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-07-14T11:48:11.186 DEBUG:teuthology.orchestra.run.smithi104:> sudo apt-get update 2023-07-14T11:48:11.194 DEBUG:teuthology.orchestra.run.smithi121:> sudo apt-get update 2023-07-14T11:48:11.542 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-07-14T11:48:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-07-14T11:48:11.642 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-07-14T11:48:11.654 INFO:teuthology.orchestra.run.smithi121.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-07-14T11:48:11.742 INFO:teuthology.orchestra.run.smithi121.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal InRelease 2023-07-14T11:48:11.743 INFO:teuthology.orchestra.run.smithi104.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal InRelease 2023-07-14T11:48:11.754 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-07-14T11:48:11.765 INFO:teuthology.orchestra.run.smithi121.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-07-14T11:48:11.811 INFO:teuthology.orchestra.run.smithi104.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-07-14T11:48:11.812 INFO:teuthology.orchestra.run.smithi121.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-07-14T11:48:11.873 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:6 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-07-14T11:48:11.885 INFO:teuthology.orchestra.run.smithi121.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal Release.gpg 2023-07-14T11:48:11.886 INFO:teuthology.orchestra.run.smithi104.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal Release.gpg 2023-07-14T11:48:11.888 INFO:teuthology.orchestra.run.smithi121.stdout:Hit:7 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-07-14T11:48:11.944 INFO:teuthology.orchestra.run.smithi121.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main i386 Packages [3,420 B] 2023-07-14T11:48:11.945 INFO:teuthology.orchestra.run.smithi104.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-07-14T11:48:12.033 INFO:teuthology.orchestra.run.smithi104.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main i386 Packages [3,420 B] 2023-07-14T11:48:12.033 INFO:teuthology.orchestra.run.smithi121.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-07-14T11:48:12.955 INFO:teuthology.orchestra.run.smithi104.stdout:Fetched 28.5 kB in 1s (19.8 kB/s) 2023-07-14T11:48:12.976 INFO:teuthology.orchestra.run.smithi121.stdout:Fetched 28.5 kB in 1s (19.7 kB/s) 2023-07-14T11:48:14.136 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2023-07-14T11:48:14.169 INFO:teuthology.orchestra.run.smithi121.stdout:Reading package lists... 2023-07-14T11:48:14.179 DEBUG:teuthology.orchestra.run.smithi104:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-4757-g3fc0089c-1focal cephadm=18.0.0-4757-g3fc0089c-1focal ceph-mds=18.0.0-4757-g3fc0089c-1focal ceph-mgr=18.0.0-4757-g3fc0089c-1focal ceph-common=18.0.0-4757-g3fc0089c-1focal ceph-fuse=18.0.0-4757-g3fc0089c-1focal ceph-test=18.0.0-4757-g3fc0089c-1focal ceph-volume=18.0.0-4757-g3fc0089c-1focal radosgw=18.0.0-4757-g3fc0089c-1focal python3-rados=18.0.0-4757-g3fc0089c-1focal python3-rgw=18.0.0-4757-g3fc0089c-1focal python3-cephfs=18.0.0-4757-g3fc0089c-1focal python3-rbd=18.0.0-4757-g3fc0089c-1focal libcephfs2=18.0.0-4757-g3fc0089c-1focal libcephfs-dev=18.0.0-4757-g3fc0089c-1focal librados2=18.0.0-4757-g3fc0089c-1focal librbd1=18.0.0-4757-g3fc0089c-1focal rbd-fuse=18.0.0-4757-g3fc0089c-1focal python3-cephfs=18.0.0-4757-g3fc0089c-1focal cephfs-shell=18.0.0-4757-g3fc0089c-1focal cephfs-top=18.0.0-4757-g3fc0089c-1focal cephfs-mirror=18.0.0-4757-g3fc0089c-1focal 2023-07-14T11:48:14.206 DEBUG:teuthology.orchestra.run.smithi121:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-4757-g3fc0089c-1focal cephadm=18.0.0-4757-g3fc0089c-1focal ceph-mds=18.0.0-4757-g3fc0089c-1focal ceph-mgr=18.0.0-4757-g3fc0089c-1focal ceph-common=18.0.0-4757-g3fc0089c-1focal ceph-fuse=18.0.0-4757-g3fc0089c-1focal ceph-test=18.0.0-4757-g3fc0089c-1focal ceph-volume=18.0.0-4757-g3fc0089c-1focal radosgw=18.0.0-4757-g3fc0089c-1focal python3-rados=18.0.0-4757-g3fc0089c-1focal python3-rgw=18.0.0-4757-g3fc0089c-1focal python3-cephfs=18.0.0-4757-g3fc0089c-1focal python3-rbd=18.0.0-4757-g3fc0089c-1focal libcephfs2=18.0.0-4757-g3fc0089c-1focal libcephfs-dev=18.0.0-4757-g3fc0089c-1focal librados2=18.0.0-4757-g3fc0089c-1focal librbd1=18.0.0-4757-g3fc0089c-1focal rbd-fuse=18.0.0-4757-g3fc0089c-1focal python3-cephfs=18.0.0-4757-g3fc0089c-1focal cephfs-shell=18.0.0-4757-g3fc0089c-1focal cephfs-top=18.0.0-4757-g3fc0089c-1focal cephfs-mirror=18.0.0-4757-g3fc0089c-1focal 2023-07-14T11:48:14.297 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2023-07-14T11:48:14.322 INFO:teuthology.orchestra.run.smithi121.stdout:Reading package lists... 2023-07-14T11:48:14.469 INFO:teuthology.orchestra.run.smithi104.stdout:Building dependency tree... 2023-07-14T11:48:14.470 INFO:teuthology.orchestra.run.smithi104.stdout:Reading state information... 2023-07-14T11:48:14.517 INFO:teuthology.orchestra.run.smithi121.stdout:Building dependency tree... 2023-07-14T11:48:14.517 INFO:teuthology.orchestra.run.smithi121.stdout:Reading state information... 2023-07-14T11:48:14.632 INFO:teuthology.orchestra.run.smithi104.stdout:The following packages were automatically installed and are no longer required: 2023-07-14T11:48:14.633 INFO:teuthology.orchestra.run.smithi104.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-07-14T11:48:14.633 INFO:teuthology.orchestra.run.smithi104.stdout:Use 'sudo apt autoremove' to remove them. 2023-07-14T11:48:14.633 INFO:teuthology.orchestra.run.smithi104.stdout:The following additional packages will be installed: 2023-07-14T11:48:14.633 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-07-14T11:48:14.634 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-07-14T11:48:14.634 INFO:teuthology.orchestra.run.smithi104.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-07-14T11:48:14.634 INFO:teuthology.orchestra.run.smithi104.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-07-14T11:48:14.634 INFO:teuthology.orchestra.run.smithi104.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-07-14T11:48:14.635 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-07-14T11:48:14.635 INFO:teuthology.orchestra.run.smithi104.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-07-14T11:48:14.635 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-07-14T11:48:14.635 INFO:teuthology.orchestra.run.smithi104.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-07-14T11:48:14.635 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2023-07-14T11:48:14.635 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-07-14T11:48:14.636 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-07-14T11:48:14.636 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-07-14T11:48:14.636 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-07-14T11:48:14.636 INFO:teuthology.orchestra.run.smithi104.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-07-14T11:48:14.636 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-07-14T11:48:14.637 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2023-07-14T11:48:14.637 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2023-07-14T11:48:14.637 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-07-14T11:48:14.637 INFO:teuthology.orchestra.run.smithi104.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-07-14T11:48:14.637 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-07-14T11:48:14.637 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-07-14T11:48:14.638 INFO:teuthology.orchestra.run.smithi104.stdout: qttranslations5-l10n socat xmlstarlet 2023-07-14T11:48:14.638 INFO:teuthology.orchestra.run.smithi104.stdout:Suggested packages: 2023-07-14T11:48:14.638 INFO:teuthology.orchestra.run.smithi104.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-07-14T11:48:14.638 INFO:teuthology.orchestra.run.smithi104.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-07-14T11:48:14.639 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-07-14T11:48:14.639 INFO:teuthology.orchestra.run.smithi104.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-07-14T11:48:14.639 INFO:teuthology.orchestra.run.smithi104.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-07-14T11:48:14.639 INFO:teuthology.orchestra.run.smithi104.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-07-14T11:48:14.639 INFO:teuthology.orchestra.run.smithi104.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-07-14T11:48:14.639 INFO:teuthology.orchestra.run.smithi104.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-07-14T11:48:14.639 INFO:teuthology.orchestra.run.smithi104.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-07-14T11:48:14.640 INFO:teuthology.orchestra.run.smithi104.stdout:Recommended packages: 2023-07-14T11:48:14.640 INFO:teuthology.orchestra.run.smithi104.stdout: btrfs-tools 2023-07-14T11:48:14.696 INFO:teuthology.orchestra.run.smithi121.stdout:The following packages were automatically installed and are no longer required: 2023-07-14T11:48:14.698 INFO:teuthology.orchestra.run.smithi121.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-07-14T11:48:14.698 INFO:teuthology.orchestra.run.smithi121.stdout:Use 'sudo apt autoremove' to remove them. 2023-07-14T11:48:14.699 INFO:teuthology.orchestra.run.smithi121.stdout:The following additional packages will be installed: 2023-07-14T11:48:14.699 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-07-14T11:48:14.699 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-07-14T11:48:14.699 INFO:teuthology.orchestra.run.smithi121.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-07-14T11:48:14.700 INFO:teuthology.orchestra.run.smithi121.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-07-14T11:48:14.700 INFO:teuthology.orchestra.run.smithi121.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-07-14T11:48:14.700 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-07-14T11:48:14.700 INFO:teuthology.orchestra.run.smithi121.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-07-14T11:48:14.701 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-07-14T11:48:14.701 INFO:teuthology.orchestra.run.smithi121.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-07-14T11:48:14.701 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2023-07-14T11:48:14.701 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-07-14T11:48:14.702 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-07-14T11:48:14.702 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-07-14T11:48:14.702 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-07-14T11:48:14.702 INFO:teuthology.orchestra.run.smithi121.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-07-14T11:48:14.703 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-07-14T11:48:14.703 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2023-07-14T11:48:14.703 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2023-07-14T11:48:14.704 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-07-14T11:48:14.704 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-07-14T11:48:14.704 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-07-14T11:48:14.704 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-07-14T11:48:14.705 INFO:teuthology.orchestra.run.smithi121.stdout: qttranslations5-l10n socat xmlstarlet 2023-07-14T11:48:14.705 INFO:teuthology.orchestra.run.smithi121.stdout:Suggested packages: 2023-07-14T11:48:14.706 INFO:teuthology.orchestra.run.smithi121.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-07-14T11:48:14.706 INFO:teuthology.orchestra.run.smithi121.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-07-14T11:48:14.706 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-07-14T11:48:14.707 INFO:teuthology.orchestra.run.smithi121.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-07-14T11:48:14.707 INFO:teuthology.orchestra.run.smithi121.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-07-14T11:48:14.707 INFO:teuthology.orchestra.run.smithi121.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-07-14T11:48:14.707 INFO:teuthology.orchestra.run.smithi121.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-07-14T11:48:14.708 INFO:teuthology.orchestra.run.smithi121.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-07-14T11:48:14.708 INFO:teuthology.orchestra.run.smithi121.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-07-14T11:48:14.708 INFO:teuthology.orchestra.run.smithi121.stdout:Recommended packages: 2023-07-14T11:48:14.708 INFO:teuthology.orchestra.run.smithi121.stdout: btrfs-tools 2023-07-14T11:48:14.837 INFO:teuthology.orchestra.run.smithi104.stdout:The following NEW packages will be installed: 2023-07-14T11:48:14.837 INFO:teuthology.orchestra.run.smithi104.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-07-14T11:48:14.837 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-07-14T11:48:14.837 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-07-14T11:48:14.837 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-07-14T11:48:14.837 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-07-14T11:48:14.838 INFO:teuthology.orchestra.run.smithi104.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-07-14T11:48:14.838 INFO:teuthology.orchestra.run.smithi104.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-07-14T11:48:14.840 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-07-14T11:48:14.840 INFO:teuthology.orchestra.run.smithi104.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-07-14T11:48:14.840 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-07-14T11:48:14.840 INFO:teuthology.orchestra.run.smithi104.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-07-14T11:48:14.841 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2023-07-14T11:48:14.841 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-07-14T11:48:14.841 INFO:teuthology.orchestra.run.smithi104.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-07-14T11:48:14.841 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-07-14T11:48:14.842 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-07-14T11:48:14.842 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-07-14T11:48:14.842 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-07-14T11:48:14.843 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-07-14T11:48:14.843 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-07-14T11:48:14.843 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-07-14T11:48:14.843 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-07-14T11:48:14.844 INFO:teuthology.orchestra.run.smithi104.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-07-14T11:48:14.844 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-07-14T11:48:14.844 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-07-14T11:48:14.844 INFO:teuthology.orchestra.run.smithi104.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-07-14T11:48:14.845 INFO:teuthology.orchestra.run.smithi104.stdout:The following packages will be upgraded: 2023-07-14T11:48:14.845 INFO:teuthology.orchestra.run.smithi104.stdout: librados2 librbd1 2023-07-14T11:48:14.905 INFO:teuthology.orchestra.run.smithi121.stdout:The following NEW packages will be installed: 2023-07-14T11:48:14.906 INFO:teuthology.orchestra.run.smithi121.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-07-14T11:48:14.906 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-07-14T11:48:14.906 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-07-14T11:48:14.907 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-07-14T11:48:14.907 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-07-14T11:48:14.908 INFO:teuthology.orchestra.run.smithi121.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-07-14T11:48:14.908 INFO:teuthology.orchestra.run.smithi121.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-07-14T11:48:14.908 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-07-14T11:48:14.908 INFO:teuthology.orchestra.run.smithi121.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-07-14T11:48:14.908 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-07-14T11:48:14.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-07-14T11:48:14.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2023-07-14T11:48:14.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-07-14T11:48:14.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-07-14T11:48:14.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-07-14T11:48:14.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-07-14T11:48:14.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-07-14T11:48:14.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-07-14T11:48:14.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-07-14T11:48:14.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-07-14T11:48:14.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-07-14T11:48:14.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-07-14T11:48:14.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-07-14T11:48:14.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-07-14T11:48:14.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-07-14T11:48:14.910 INFO:teuthology.orchestra.run.smithi121.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-07-14T11:48:14.911 INFO:teuthology.orchestra.run.smithi121.stdout:The following packages will be upgraded: 2023-07-14T11:48:14.911 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 librbd1 2023-07-14T11:48:15.011 INFO:teuthology.orchestra.run.smithi104.stdout:2 upgraded, 116 newly installed, 0 to remove and 210 not upgraded. 2023-07-14T11:48:15.011 INFO:teuthology.orchestra.run.smithi104.stdout:Need to get 210 MB of archives. 2023-07-14T11:48:15.012 INFO:teuthology.orchestra.run.smithi104.stdout:After this operation, 1,040 MB of additional disk space will be used. 2023-07-14T11:48:15.012 INFO:teuthology.orchestra.run.smithi104.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-07-14T11:48:15.069 INFO:teuthology.orchestra.run.smithi104.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-07-14T11:48:15.076 INFO:teuthology.orchestra.run.smithi121.stdout:2 upgraded, 116 newly installed, 0 to remove and 210 not upgraded. 2023-07-14T11:48:15.077 INFO:teuthology.orchestra.run.smithi121.stdout:Need to get 210 MB of archives. 2023-07-14T11:48:15.077 INFO:teuthology.orchestra.run.smithi121.stdout:After this operation, 1,040 MB of additional disk space will be used. 2023-07-14T11:48:15.077 INFO:teuthology.orchestra.run.smithi121.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-07-14T11:48:15.094 INFO:teuthology.orchestra.run.smithi104.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-07-14T11:48:15.136 INFO:teuthology.orchestra.run.smithi121.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-07-14T11:48:15.162 INFO:teuthology.orchestra.run.smithi121.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-07-14T11:48:15.181 INFO:teuthology.orchestra.run.smithi104.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-07-14T11:48:15.256 INFO:teuthology.orchestra.run.smithi121.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-07-14T11:48:15.280 INFO:teuthology.orchestra.run.smithi104.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.0.0-4757-g3fc0089c-1focal [3,427 kB] 2023-07-14T11:48:15.286 INFO:teuthology.orchestra.run.smithi104.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-07-14T11:48:15.300 INFO:teuthology.orchestra.run.smithi104.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-07-14T11:48:15.346 INFO:teuthology.orchestra.run.smithi104.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-07-14T11:48:15.362 INFO:teuthology.orchestra.run.smithi121.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-07-14T11:48:15.375 INFO:teuthology.orchestra.run.smithi121.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-07-14T11:48:15.391 INFO:teuthology.orchestra.run.smithi121.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.0.0-4757-g3fc0089c-1focal [3,427 kB] 2023-07-14T11:48:15.416 INFO:teuthology.orchestra.run.smithi121.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-07-14T11:48:15.486 INFO:teuthology.orchestra.run.smithi104.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-07-14T11:48:15.491 INFO:teuthology.orchestra.run.smithi104.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-07-14T11:48:15.516 INFO:teuthology.orchestra.run.smithi104.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-07-14T11:48:15.521 INFO:teuthology.orchestra.run.smithi104.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-07-14T11:48:15.557 INFO:teuthology.orchestra.run.smithi121.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-07-14T11:48:15.561 INFO:teuthology.orchestra.run.smithi121.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-07-14T11:48:15.577 INFO:teuthology.orchestra.run.smithi121.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-07-14T11:48:15.582 INFO:teuthology.orchestra.run.smithi121.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-07-14T11:48:15.605 INFO:teuthology.orchestra.run.smithi104.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-07-14T11:48:15.688 INFO:teuthology.orchestra.run.smithi121.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-07-14T11:48:15.831 INFO:teuthology.orchestra.run.smithi104.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-07-14T11:48:15.837 INFO:teuthology.orchestra.run.smithi104.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-07-14T11:48:15.840 INFO:teuthology.orchestra.run.smithi104.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.0.0-4757-g3fc0089c-1focal [3,992 kB] 2023-07-14T11:48:15.891 INFO:teuthology.orchestra.run.smithi121.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-07-14T11:48:15.903 INFO:teuthology.orchestra.run.smithi121.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-07-14T11:48:15.958 INFO:teuthology.orchestra.run.smithi121.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.0.0-4757-g3fc0089c-1focal [3,992 kB] 2023-07-14T11:48:15.973 INFO:teuthology.orchestra.run.smithi104.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-07-14T11:48:15.981 INFO:teuthology.orchestra.run.smithi104.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-07-14T11:48:16.003 INFO:teuthology.orchestra.run.smithi104.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-07-14T11:48:16.018 INFO:teuthology.orchestra.run.smithi104.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-07-14T11:48:16.020 INFO:teuthology.orchestra.run.smithi104.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-07-14T11:48:16.021 INFO:teuthology.orchestra.run.smithi104.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-07-14T11:48:16.023 INFO:teuthology.orchestra.run.smithi121.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-07-14T11:48:16.030 INFO:teuthology.orchestra.run.smithi104.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-07-14T11:48:16.031 INFO:teuthology.orchestra.run.smithi104.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-07-14T11:48:16.033 INFO:teuthology.orchestra.run.smithi121.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-07-14T11:48:16.044 INFO:teuthology.orchestra.run.smithi104.stdout:Get:25 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.0.0-4757-g3fc0089c-1focal [725 kB] 2023-07-14T11:48:16.065 INFO:teuthology.orchestra.run.smithi121.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-07-14T11:48:16.068 INFO:teuthology.orchestra.run.smithi121.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-07-14T11:48:16.069 INFO:teuthology.orchestra.run.smithi121.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-07-14T11:48:16.070 INFO:teuthology.orchestra.run.smithi121.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-07-14T11:48:16.106 INFO:teuthology.orchestra.run.smithi104.stdout:Get:26 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.0.0-4757-g3fc0089c-1focal [378 kB] 2023-07-14T11:48:16.115 INFO:teuthology.orchestra.run.smithi104.stdout:Get:27 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.0.0-4757-g3fc0089c-1focal [22.5 kB] 2023-07-14T11:48:16.116 INFO:teuthology.orchestra.run.smithi104.stdout:Get:28 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.0.0-4757-g3fc0089c-1focal [191 kB] 2023-07-14T11:48:16.119 INFO:teuthology.orchestra.run.smithi104.stdout:Get:29 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.0.0-4757-g3fc0089c-1focal [48.4 kB] 2023-07-14T11:48:16.121 INFO:teuthology.orchestra.run.smithi104.stdout:Get:30 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.0.0-4757-g3fc0089c-1focal [355 kB] 2023-07-14T11:48:16.129 INFO:teuthology.orchestra.run.smithi121.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-07-14T11:48:16.131 INFO:teuthology.orchestra.run.smithi121.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-07-14T11:48:16.142 INFO:teuthology.orchestra.run.smithi104.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-07-14T11:48:16.157 INFO:teuthology.orchestra.run.smithi104.stdout:Get:32 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.0.0-4757-g3fc0089c-1focal [6,774 kB] 2023-07-14T11:48:16.216 INFO:teuthology.orchestra.run.smithi121.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-07-14T11:48:16.254 INFO:teuthology.orchestra.run.smithi121.stdout:Get:26 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.0.0-4757-g3fc0089c-1focal [725 kB] 2023-07-14T11:48:16.275 INFO:teuthology.orchestra.run.smithi121.stdout:Get:27 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.0.0-4757-g3fc0089c-1focal [378 kB] 2023-07-14T11:48:16.309 INFO:teuthology.orchestra.run.smithi104.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-07-14T11:48:16.317 INFO:teuthology.orchestra.run.smithi104.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-07-14T11:48:16.333 INFO:teuthology.orchestra.run.smithi121.stdout:Get:28 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.0.0-4757-g3fc0089c-1focal [22.5 kB] 2023-07-14T11:48:16.334 INFO:teuthology.orchestra.run.smithi121.stdout:Get:29 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.0.0-4757-g3fc0089c-1focal [191 kB] 2023-07-14T11:48:16.342 INFO:teuthology.orchestra.run.smithi121.stdout:Get:30 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.0.0-4757-g3fc0089c-1focal [48.4 kB] 2023-07-14T11:48:16.344 INFO:teuthology.orchestra.run.smithi121.stdout:Get:31 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.0.0-4757-g3fc0089c-1focal [355 kB] 2023-07-14T11:48:16.367 INFO:teuthology.orchestra.run.smithi104.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-07-14T11:48:16.402 INFO:teuthology.orchestra.run.smithi121.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-07-14T11:48:16.411 INFO:teuthology.orchestra.run.smithi104.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-07-14T11:48:16.415 INFO:teuthology.orchestra.run.smithi104.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-07-14T11:48:16.417 INFO:teuthology.orchestra.run.smithi121.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-07-14T11:48:16.420 INFO:teuthology.orchestra.run.smithi121.stdout:Get:34 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.0.0-4757-g3fc0089c-1focal [6,774 kB] 2023-07-14T11:48:16.544 INFO:teuthology.orchestra.run.smithi121.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-07-14T11:48:16.576 INFO:teuthology.orchestra.run.smithi104.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-07-14T11:48:16.577 INFO:teuthology.orchestra.run.smithi104.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-07-14T11:48:16.578 INFO:teuthology.orchestra.run.smithi104.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-07-14T11:48:16.579 INFO:teuthology.orchestra.run.smithi104.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-07-14T11:48:16.624 INFO:teuthology.orchestra.run.smithi104.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-07-14T11:48:16.626 INFO:teuthology.orchestra.run.smithi121.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-07-14T11:48:16.627 INFO:teuthology.orchestra.run.smithi121.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-07-14T11:48:16.670 INFO:teuthology.orchestra.run.smithi104.stdout:Get:43 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.0.0-4757-g3fc0089c-1focal [103 kB] 2023-07-14T11:48:16.672 INFO:teuthology.orchestra.run.smithi104.stdout:Get:44 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.0.0-4757-g3fc0089c-1focal [466 kB] 2023-07-14T11:48:16.678 INFO:teuthology.orchestra.run.smithi104.stdout:Get:45 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.0.0-4757-g3fc0089c-1focal [25.6 MB] 2023-07-14T11:48:16.686 INFO:teuthology.orchestra.run.smithi121.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-07-14T11:48:16.687 INFO:teuthology.orchestra.run.smithi121.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-07-14T11:48:16.689 INFO:teuthology.orchestra.run.smithi121.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-07-14T11:48:16.691 INFO:teuthology.orchestra.run.smithi121.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-07-14T11:48:16.697 INFO:teuthology.orchestra.run.smithi121.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-07-14T11:48:16.704 INFO:teuthology.orchestra.run.smithi104.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-07-14T11:48:16.769 INFO:teuthology.orchestra.run.smithi121.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-07-14T11:48:16.868 INFO:teuthology.orchestra.run.smithi104.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-07-14T11:48:16.873 INFO:teuthology.orchestra.run.smithi104.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2023-07-14T11:48:16.900 INFO:teuthology.orchestra.run.smithi104.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-07-14T11:48:16.930 INFO:teuthology.orchestra.run.smithi104.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-07-14T11:48:16.936 INFO:teuthology.orchestra.run.smithi104.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-07-14T11:48:16.941 INFO:teuthology.orchestra.run.smithi104.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-07-14T11:48:16.943 INFO:teuthology.orchestra.run.smithi104.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-07-14T11:48:16.972 INFO:teuthology.orchestra.run.smithi104.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-07-14T11:48:16.984 INFO:teuthology.orchestra.run.smithi104.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-07-14T11:48:16.994 INFO:teuthology.orchestra.run.smithi104.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-07-14T11:48:17.014 INFO:teuthology.orchestra.run.smithi121.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-07-14T11:48:17.023 INFO:teuthology.orchestra.run.smithi121.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2023-07-14T11:48:17.060 INFO:teuthology.orchestra.run.smithi121.stdout:Get:46 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.0.0-4757-g3fc0089c-1focal [103 kB] 2023-07-14T11:48:17.061 INFO:teuthology.orchestra.run.smithi121.stdout:Get:47 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.0.0-4757-g3fc0089c-1focal [466 kB] 2023-07-14T11:48:17.069 INFO:teuthology.orchestra.run.smithi121.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-07-14T11:48:17.087 INFO:teuthology.orchestra.run.smithi104.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-07-14T11:48:17.100 INFO:teuthology.orchestra.run.smithi121.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-07-14T11:48:17.103 INFO:teuthology.orchestra.run.smithi121.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.0.0-4757-g3fc0089c-1focal [25.6 MB] 2023-07-14T11:48:17.105 INFO:teuthology.orchestra.run.smithi121.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-07-14T11:48:17.106 INFO:teuthology.orchestra.run.smithi121.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-07-14T11:48:17.107 INFO:teuthology.orchestra.run.smithi121.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-07-14T11:48:17.130 INFO:teuthology.orchestra.run.smithi121.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-07-14T11:48:17.142 INFO:teuthology.orchestra.run.smithi121.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-07-14T11:48:17.152 INFO:teuthology.orchestra.run.smithi121.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-07-14T11:48:17.172 INFO:teuthology.orchestra.run.smithi104.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-07-14T11:48:17.210 INFO:teuthology.orchestra.run.smithi121.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-07-14T11:48:17.283 INFO:teuthology.orchestra.run.smithi121.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-07-14T11:48:17.657 INFO:teuthology.orchestra.run.smithi104.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-07-14T11:48:17.686 INFO:teuthology.orchestra.run.smithi104.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-07-14T11:48:17.688 INFO:teuthology.orchestra.run.smithi104.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-07-14T11:48:17.740 INFO:teuthology.orchestra.run.smithi104.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-07-14T11:48:17.741 INFO:teuthology.orchestra.run.smithi104.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-07-14T11:48:17.741 INFO:teuthology.orchestra.run.smithi104.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-07-14T11:48:17.742 INFO:teuthology.orchestra.run.smithi104.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-07-14T11:48:17.742 INFO:teuthology.orchestra.run.smithi104.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-07-14T11:48:17.743 INFO:teuthology.orchestra.run.smithi104.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-07-14T11:48:17.828 INFO:teuthology.orchestra.run.smithi104.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-07-14T11:48:17.957 INFO:teuthology.orchestra.run.smithi121.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-07-14T11:48:18.010 INFO:teuthology.orchestra.run.smithi121.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-07-14T11:48:18.012 INFO:teuthology.orchestra.run.smithi121.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-07-14T11:48:18.020 INFO:teuthology.orchestra.run.smithi104.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-07-14T11:48:18.029 INFO:teuthology.orchestra.run.smithi104.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-07-14T11:48:18.047 INFO:teuthology.orchestra.run.smithi121.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-07-14T11:48:18.048 INFO:teuthology.orchestra.run.smithi121.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-07-14T11:48:18.048 INFO:teuthology.orchestra.run.smithi121.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-07-14T11:48:18.049 INFO:teuthology.orchestra.run.smithi121.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-07-14T11:48:18.050 INFO:teuthology.orchestra.run.smithi121.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-07-14T11:48:18.050 INFO:teuthology.orchestra.run.smithi104.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-07-14T11:48:18.050 INFO:teuthology.orchestra.run.smithi121.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-07-14T11:48:18.133 INFO:teuthology.orchestra.run.smithi121.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-07-14T11:48:18.134 INFO:teuthology.orchestra.run.smithi104.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-07-14T11:48:18.162 INFO:teuthology.orchestra.run.smithi104.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-07-14T11:48:18.179 INFO:teuthology.orchestra.run.smithi104.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-07-14T11:48:18.185 INFO:teuthology.orchestra.run.smithi104.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-07-14T11:48:18.216 INFO:teuthology.orchestra.run.smithi104.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-07-14T11:48:18.233 INFO:teuthology.orchestra.run.smithi104.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-07-14T11:48:18.234 INFO:teuthology.orchestra.run.smithi104.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-07-14T11:48:18.341 INFO:teuthology.orchestra.run.smithi121.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-07-14T11:48:18.349 INFO:teuthology.orchestra.run.smithi121.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-07-14T11:48:18.367 INFO:teuthology.orchestra.run.smithi121.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-07-14T11:48:18.436 INFO:teuthology.orchestra.run.smithi104.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-07-14T11:48:18.476 INFO:teuthology.orchestra.run.smithi121.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-07-14T11:48:18.520 INFO:teuthology.orchestra.run.smithi121.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-07-14T11:48:18.544 INFO:teuthology.orchestra.run.smithi121.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-07-14T11:48:18.550 INFO:teuthology.orchestra.run.smithi121.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-07-14T11:48:18.584 INFO:teuthology.orchestra.run.smithi121.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-07-14T11:48:18.613 INFO:teuthology.orchestra.run.smithi121.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-07-14T11:48:18.615 INFO:teuthology.orchestra.run.smithi121.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-07-14T11:48:18.810 INFO:teuthology.orchestra.run.smithi104.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2023-07-14T11:48:18.835 INFO:teuthology.orchestra.run.smithi121.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-07-14T11:48:18.856 INFO:teuthology.orchestra.run.smithi104.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-07-14T11:48:19.023 INFO:teuthology.orchestra.run.smithi104.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.0.0-4757-g3fc0089c-1focal [11.0 MB] 2023-07-14T11:48:19.124 INFO:teuthology.orchestra.run.smithi104.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-07-14T11:48:19.125 INFO:teuthology.orchestra.run.smithi104.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-07-14T11:48:19.126 INFO:teuthology.orchestra.run.smithi104.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-07-14T11:48:19.126 INFO:teuthology.orchestra.run.smithi104.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-07-14T11:48:19.127 INFO:teuthology.orchestra.run.smithi104.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-07-14T11:48:19.128 INFO:teuthology.orchestra.run.smithi104.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-07-14T11:48:19.163 INFO:teuthology.orchestra.run.smithi104.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-07-14T11:48:19.164 INFO:teuthology.orchestra.run.smithi104.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-07-14T11:48:19.198 INFO:teuthology.orchestra.run.smithi121.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2023-07-14T11:48:19.241 INFO:teuthology.orchestra.run.smithi121.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-07-14T11:48:19.271 INFO:teuthology.orchestra.run.smithi104.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-07-14T11:48:19.377 INFO:teuthology.orchestra.run.smithi104.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-07-14T11:48:19.469 INFO:teuthology.orchestra.run.smithi121.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-07-14T11:48:19.470 INFO:teuthology.orchestra.run.smithi121.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-07-14T11:48:19.471 INFO:teuthology.orchestra.run.smithi121.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-07-14T11:48:19.471 INFO:teuthology.orchestra.run.smithi121.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-07-14T11:48:19.472 INFO:teuthology.orchestra.run.smithi121.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-07-14T11:48:19.473 INFO:teuthology.orchestra.run.smithi121.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-07-14T11:48:19.500 INFO:teuthology.orchestra.run.smithi121.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-07-14T11:48:19.500 INFO:teuthology.orchestra.run.smithi121.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-07-14T11:48:19.518 INFO:teuthology.orchestra.run.smithi104.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-07-14T11:48:19.533 INFO:teuthology.orchestra.run.smithi104.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-07-14T11:48:19.540 INFO:teuthology.orchestra.run.smithi104.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-07-14T11:48:19.576 INFO:teuthology.orchestra.run.smithi104.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-07-14T11:48:19.579 INFO:teuthology.orchestra.run.smithi121.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-07-14T11:48:19.584 INFO:teuthology.orchestra.run.smithi104.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-07-14T11:48:19.648 INFO:teuthology.orchestra.run.smithi121.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-07-14T11:48:19.729 INFO:teuthology.orchestra.run.smithi121.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-07-14T11:48:19.750 INFO:teuthology.orchestra.run.smithi121.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-07-14T11:48:19.755 INFO:teuthology.orchestra.run.smithi121.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-07-14T11:48:19.779 INFO:teuthology.orchestra.run.smithi121.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-07-14T11:48:19.786 INFO:teuthology.orchestra.run.smithi121.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-07-14T11:48:20.081 INFO:teuthology.orchestra.run.smithi104.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.0.0-4757-g3fc0089c-1focal [223 kB] 2023-07-14T11:48:20.085 INFO:teuthology.orchestra.run.smithi104.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.0.0-4757-g3fc0089c-1focal [133 kB] 2023-07-14T11:48:20.092 INFO:teuthology.orchestra.run.smithi104.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.0.0-4757-g3fc0089c-1focal [1,552 kB] 2023-07-14T11:48:20.261 INFO:teuthology.orchestra.run.smithi104.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.0.0-4757-g3fc0089c-1focal [9,399 kB] 2023-07-14T11:48:21.097 INFO:teuthology.orchestra.run.smithi121.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.0.0-4757-g3fc0089c-1focal [11.0 MB] 2023-07-14T11:48:21.160 INFO:teuthology.orchestra.run.smithi104.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.0.0-4757-g3fc0089c-1focal [31.3 MB] 2023-07-14T11:48:22.509 INFO:teuthology.orchestra.run.smithi121.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.0.0-4757-g3fc0089c-1focal [223 kB] 2023-07-14T11:48:22.524 INFO:teuthology.orchestra.run.smithi121.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.0.0-4757-g3fc0089c-1focal [133 kB] 2023-07-14T11:48:22.534 INFO:teuthology.orchestra.run.smithi121.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.0.0-4757-g3fc0089c-1focal [1,552 kB] 2023-07-14T11:48:22.693 INFO:teuthology.orchestra.run.smithi121.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.0.0-4757-g3fc0089c-1focal [9,399 kB] 2023-07-14T11:48:23.612 INFO:teuthology.orchestra.run.smithi104.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.0.0-4757-g3fc0089c-1focal [3,812 B] 2023-07-14T11:48:23.612 INFO:teuthology.orchestra.run.smithi104.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.0.0-4757-g3fc0089c-1focal [834 kB] 2023-07-14T11:48:23.683 INFO:teuthology.orchestra.run.smithi104.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.0.0-4757-g3fc0089c-1focal [2,302 kB] 2023-07-14T11:48:23.901 INFO:teuthology.orchestra.run.smithi104.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.0.0-4757-g3fc0089c-1focal [217 kB] 2023-07-14T11:48:23.902 INFO:teuthology.orchestra.run.smithi121.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.0.0-4757-g3fc0089c-1focal [31.3 MB] 2023-07-14T11:48:23.930 INFO:teuthology.orchestra.run.smithi104.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.0.0-4757-g3fc0089c-1focal [124 kB] 2023-07-14T11:48:23.937 INFO:teuthology.orchestra.run.smithi104.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.0.0-4757-g3fc0089c-1focal [1,776 kB] 2023-07-14T11:48:24.072 INFO:teuthology.orchestra.run.smithi104.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.0.0-4757-g3fc0089c-1focal [7,879 kB] 2023-07-14T11:48:25.027 INFO:teuthology.orchestra.run.smithi104.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.0.0-4757-g3fc0089c-1focal [3,936 B] 2023-07-14T11:48:25.028 INFO:teuthology.orchestra.run.smithi104.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.0.0-4757-g3fc0089c-1focal [50.9 MB] 2023-07-14T11:48:28.628 INFO:teuthology.orchestra.run.smithi121.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.0.0-4757-g3fc0089c-1focal [3,812 B] 2023-07-14T11:48:28.629 INFO:teuthology.orchestra.run.smithi121.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.0.0-4757-g3fc0089c-1focal [834 kB] 2023-07-14T11:48:28.688 INFO:teuthology.orchestra.run.smithi121.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.0.0-4757-g3fc0089c-1focal [2,302 kB] 2023-07-14T11:48:28.926 INFO:teuthology.orchestra.run.smithi121.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.0.0-4757-g3fc0089c-1focal [217 kB] 2023-07-14T11:48:28.939 INFO:teuthology.orchestra.run.smithi121.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.0.0-4757-g3fc0089c-1focal [124 kB] 2023-07-14T11:48:28.944 INFO:teuthology.orchestra.run.smithi121.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.0.0-4757-g3fc0089c-1focal [1,776 kB] 2023-07-14T11:48:29.113 INFO:teuthology.orchestra.run.smithi121.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.0.0-4757-g3fc0089c-1focal [7,879 kB] 2023-07-14T11:48:29.451 INFO:teuthology.orchestra.run.smithi104.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.0.0-4757-g3fc0089c-1focal [220 kB] 2023-07-14T11:48:29.453 INFO:teuthology.orchestra.run.smithi104.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.0.0-4757-g3fc0089c-1focal [256 kB] 2023-07-14T11:48:29.456 INFO:teuthology.orchestra.run.smithi104.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.0.0-4757-g3fc0089c-1focal [23.3 kB] 2023-07-14T11:48:29.456 INFO:teuthology.orchestra.run.smithi104.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.0.0-4757-g3fc0089c-1focal [12.9 kB] 2023-07-14T11:48:29.457 INFO:teuthology.orchestra.run.smithi104.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.0.0-4757-g3fc0089c-1focal [28.0 kB] 2023-07-14T11:48:29.458 INFO:teuthology.orchestra.run.smithi104.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.0.0-4757-g3fc0089c-1focal [13.8 MB] 2023-07-14T11:48:29.986 INFO:teuthology.orchestra.run.smithi121.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.0.0-4757-g3fc0089c-1focal [3,936 B] 2023-07-14T11:48:29.986 INFO:teuthology.orchestra.run.smithi121.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.0.0-4757-g3fc0089c-1focal [50.9 MB] 2023-07-14T11:48:30.795 INFO:teuthology.orchestra.run.smithi104.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.0.0-4757-g3fc0089c-1focal [97.3 kB] 2023-07-14T11:48:31.722 INFO:teuthology.orchestra.run.smithi104.stdout:Fetched 210 MB in 16s (13.1 MB/s) 2023-07-14T11:48:31.886 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pyperclip. 2023-07-14T11:48:31.960 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 107758 files and directories currently installed.) 2023-07-14T11:48:31.963 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-07-14T11:48:32.010 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-07-14T11:48:32.378 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-wcwidth. 2023-07-14T11:48:32.398 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-07-14T11:48:32.435 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-07-14T11:48:32.822 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-cmd2. 2023-07-14T11:48:32.841 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-07-14T11:48:32.913 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-07-14T11:48:33.291 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-07-14T11:48:33.310 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-07-14T11:48:33.391 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-07-14T11:48:33.778 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-07-14T11:48:33.798 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-07-14T11:48:33.828 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-07-14T11:48:34.223 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-07-14T11:48:34.242 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-07-14T11:48:34.281 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-07-14T11:48:34.801 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-07-14T11:48:34.821 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-07-14T11:48:34.906 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-07-14T11:48:35.471 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-07-14T11:48:35.490 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-07-14T11:48:35.529 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-07-14T11:48:35.957 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-07-14T11:48:35.977 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-07-14T11:48:36.049 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-07-14T11:48:36.477 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-07-14T11:48:36.494 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-07-14T11:48:36.535 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-07-14T11:48:36.931 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../010-librbd1_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:37.055 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librbd1 (18.0.0-4757-g3fc0089c-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-07-14T11:48:37.804 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../011-librados2_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:38.160 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librados2 (18.0.0-4757-g3fc0089c-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-07-14T11:48:38.923 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libcephfs2. 2023-07-14T11:48:38.936 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:38.973 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libcephfs2 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:39.350 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-rados. 2023-07-14T11:48:39.363 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../013-python3-rados_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:39.409 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-rados (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:39.800 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-07-14T11:48:39.813 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:48:39.850 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-ceph-argparse (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:40.144 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-cephfs. 2023-07-14T11:48:40.163 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:40.202 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-cephfs (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:40.573 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-ceph-common. 2023-07-14T11:48:40.590 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:48:40.630 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-ceph-common (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:41.042 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-prettytable. 2023-07-14T11:48:41.060 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2023-07-14T11:48:41.133 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-07-14T11:48:41.288 INFO:teuthology.orchestra.run.smithi121.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.0.0-4757-g3fc0089c-1focal [220 kB] 2023-07-14T11:48:41.309 INFO:teuthology.orchestra.run.smithi121.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.0.0-4757-g3fc0089c-1focal [256 kB] 2023-07-14T11:48:41.361 INFO:teuthology.orchestra.run.smithi121.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.0.0-4757-g3fc0089c-1focal [23.3 kB] 2023-07-14T11:48:41.368 INFO:teuthology.orchestra.run.smithi121.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.0.0-4757-g3fc0089c-1focal [12.9 kB] 2023-07-14T11:48:41.369 INFO:teuthology.orchestra.run.smithi121.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.0.0-4757-g3fc0089c-1focal [28.0 kB] 2023-07-14T11:48:41.395 INFO:teuthology.orchestra.run.smithi121.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.0.0-4757-g3fc0089c-1focal [13.8 MB] 2023-07-14T11:48:41.411 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-rbd. 2023-07-14T11:48:41.430 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:41.469 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-rbd (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:41.864 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-07-14T11:48:41.882 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-07-14T11:48:41.930 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-07-14T11:48:42.325 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-07-14T11:48:42.338 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2023-07-14T11:48:42.375 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-07-14T11:48:42.770 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-07-14T11:48:42.789 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-07-14T11:48:42.828 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-07-14T11:48:43.264 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package librgw2. 2023-07-14T11:48:43.283 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../022-librgw2_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:43.322 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librgw2 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:44.256 INFO:teuthology.orchestra.run.smithi121.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.0.0-4757-g3fc0089c-1focal [97.3 kB] 2023-07-14T11:48:44.300 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-rgw. 2023-07-14T11:48:44.318 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../023-python3-rgw_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:44.358 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-rgw (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:44.769 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liboath0. 2023-07-14T11:48:44.786 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-07-14T11:48:44.827 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-07-14T11:48:45.197 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libradosstriper1. 2023-07-14T11:48:45.206 INFO:teuthology.orchestra.run.smithi121.stdout:Fetched 210 MB in 29s (7,142 kB/s) 2023-07-14T11:48:45.216 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../025-libradosstriper1_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:45.255 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libradosstriper1 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:45.367 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pyperclip. 2023-07-14T11:48:45.437 INFO:teuthology.orchestra.run.smithi121.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 107758 files and directories currently installed.) 2023-07-14T11:48:45.440 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-07-14T11:48:45.479 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-07-14T11:48:45.824 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-wcwidth. 2023-07-14T11:48:45.843 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-07-14T11:48:45.850 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-common. 2023-07-14T11:48:45.869 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../026-ceph-common_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:45.874 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-07-14T11:48:45.908 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-common (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:46.219 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-cmd2. 2023-07-14T11:48:46.237 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-07-14T11:48:46.277 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-07-14T11:48:46.630 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-07-14T11:48:46.649 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-07-14T11:48:46.763 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-07-14T11:48:47.133 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-07-14T11:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-07-14T11:48:47.205 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-07-14T11:48:47.545 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-07-14T11:48:47.563 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-07-14T11:48:47.603 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-07-14T11:48:48.106 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-07-14T11:48:48.126 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-07-14T11:48:48.188 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-07-14T11:48:49.173 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-base. 2023-07-14T11:48:49.191 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../027-ceph-base_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:49.218 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-07-14T11:48:49.236 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-07-14T11:48:49.252 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-base (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:49.276 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-07-14T11:48:49.688 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-07-14T11:48:49.707 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-07-14T11:48:49.752 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-07-14T11:48:50.174 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-07-14T11:48:50.192 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-07-14T11:48:50.232 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-07-14T11:48:50.663 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../010-librbd1_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:50.785 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-cherrypy3. 2023-07-14T11:48:50.794 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking librbd1 (18.0.0-4757-g3fc0089c-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-07-14T11:48:50.805 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2023-07-14T11:48:50.843 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-07-14T11:48:51.231 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-natsort. 2023-07-14T11:48:51.248 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2023-07-14T11:48:51.288 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-07-14T11:48:51.535 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../011-librados2_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:51.650 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-logutils. 2023-07-14T11:48:51.669 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2023-07-14T11:48:51.683 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking librados2 (18.0.0-4757-g3fc0089c-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-07-14T11:48:51.708 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-07-14T11:48:52.078 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-markupsafe. 2023-07-14T11:48:52.097 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-07-14T11:48:52.135 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-07-14T11:48:52.489 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-mako. 2023-07-14T11:48:52.492 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libcephfs2. 2023-07-14T11:48:52.506 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-07-14T11:48:52.511 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:52.546 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-07-14T11:48:52.550 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libcephfs2 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:52.937 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-rados. 2023-07-14T11:48:52.942 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-simplegeneric. 2023-07-14T11:48:52.956 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../013-python3-rados_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:52.960 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2023-07-14T11:48:52.995 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-rados (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:53.000 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-07-14T11:48:53.328 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-singledispatch. 2023-07-14T11:48:53.346 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-07-14T11:48:53.373 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-07-14T11:48:53.386 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-07-14T11:48:53.392 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:48:53.456 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-ceph-argparse (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:53.697 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-webob. 2023-07-14T11:48:53.700 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-cephfs. 2023-07-14T11:48:53.719 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:53.720 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2023-07-14T11:48:53.750 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-cephfs (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:53.763 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-07-14T11:48:54.075 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-soupsieve. 2023-07-14T11:48:54.094 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-07-14T11:48:54.120 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-ceph-common. 2023-07-14T11:48:54.124 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-07-14T11:48:54.139 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:48:54.170 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-ceph-common (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:54.477 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-bs4. 2023-07-14T11:48:54.490 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2023-07-14T11:48:54.527 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-07-14T11:48:54.539 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-prettytable. 2023-07-14T11:48:54.552 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2023-07-14T11:48:54.589 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-07-14T11:48:54.867 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-rbd. 2023-07-14T11:48:54.880 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:54.917 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-rbd (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:54.943 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-waitress. 2023-07-14T11:48:54.956 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-07-14T11:48:55.010 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-07-14T11:48:55.320 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-07-14T11:48:55.338 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-07-14T11:48:55.378 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-07-14T11:48:55.399 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-tempita. 2023-07-14T11:48:55.412 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2023-07-14T11:48:55.449 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-07-14T11:48:55.781 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-07-14T11:48:55.800 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2023-07-14T11:48:55.811 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-paste. 2023-07-14T11:48:55.830 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-07-14T11:48:55.839 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-07-14T11:48:55.869 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-07-14T11:48:56.217 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-07-14T11:48:56.230 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-07-14T11:48:56.272 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-07-14T11:48:56.275 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-07-14T11:48:56.290 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-07-14T11:48:56.330 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-07-14T11:48:56.686 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package librgw2. 2023-07-14T11:48:56.692 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pastedeploy. 2023-07-14T11:48:56.699 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../022-librgw2_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:56.710 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2023-07-14T11:48:56.736 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking librgw2 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:56.749 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-07-14T11:48:57.111 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-webtest. 2023-07-14T11:48:57.129 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2023-07-14T11:48:57.169 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-07-14T11:48:57.572 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pecan. 2023-07-14T11:48:57.585 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-07-14T11:48:57.630 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-07-14T11:48:57.747 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-rgw. 2023-07-14T11:48:57.766 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../023-python3-rgw_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:57.805 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-rgw (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:58.068 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-werkzeug. 2023-07-14T11:48:58.081 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-07-14T11:48:58.118 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-07-14T11:48:58.217 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package liboath0. 2023-07-14T11:48:58.235 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-07-14T11:48:58.274 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-07-14T11:48:58.463 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-07-14T11:48:58.481 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:48:58.521 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr-modules-core (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:58.620 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libradosstriper1. 2023-07-14T11:48:58.639 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../025-libradosstriper1_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:58.910 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libradosstriper1 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:58.992 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-07-14T11:48:59.011 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:59.050 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:59.349 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-common. 2023-07-14T11:48:59.367 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../026-ceph-common_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:48:59.406 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-common (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:48:59.437 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-cryptography. 2023-07-14T11:48:59.455 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../048-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-07-14T11:48:59.494 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-07-14T11:48:59.916 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-openssl. 2023-07-14T11:48:59.934 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../049-python3-openssl_19.0.0-1build1_all.deb ... 2023-07-14T11:48:59.973 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-07-14T11:49:00.402 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr. 2023-07-14T11:49:00.418 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../050-ceph-mgr_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:00.459 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:01.005 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mon. 2023-07-14T11:49:01.023 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../051-ceph-mon_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:01.063 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mon (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:01.978 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-base. 2023-07-14T11:49:01.991 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../027-ceph-base_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:02.049 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-base (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:02.358 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-osd. 2023-07-14T11:49:02.377 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../052-ceph-osd_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:02.449 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-osd (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:03.508 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-cherrypy3. 2023-07-14T11:49:03.527 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2023-07-14T11:49:03.566 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-07-14T11:49:03.953 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-natsort. 2023-07-14T11:49:03.972 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2023-07-14T11:49:04.011 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-07-14T11:49:04.397 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-logutils. 2023-07-14T11:49:04.417 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2023-07-14T11:49:04.455 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-07-14T11:49:04.826 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-markupsafe. 2023-07-14T11:49:04.843 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-07-14T11:49:04.883 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-07-14T11:49:05.210 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph. 2023-07-14T11:49:05.220 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-mako. 2023-07-14T11:49:05.222 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../053-ceph_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:05.239 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-07-14T11:49:05.260 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:05.278 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-07-14T11:49:05.638 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-fuse. 2023-07-14T11:49:05.658 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../054-ceph-fuse_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:05.681 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-simplegeneric. 2023-07-14T11:49:05.696 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-fuse (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:05.701 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2023-07-14T11:49:05.739 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-07-14T11:49:06.126 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-singledispatch. 2023-07-14T11:49:06.145 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-07-14T11:49:06.166 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mds. 2023-07-14T11:49:06.183 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-07-14T11:49:06.184 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../055-ceph-mds_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:06.224 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mds (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:06.562 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-webob. 2023-07-14T11:49:06.581 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2023-07-14T11:49:06.620 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-07-14T11:49:06.802 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package cephadm. 2023-07-14T11:49:06.821 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../056-cephadm_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:06.860 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking cephadm (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:06.973 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-soupsieve. 2023-07-14T11:49:06.992 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-07-14T11:49:07.031 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-07-14T11:49:07.254 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-asyncssh. 2023-07-14T11:49:07.273 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../057-python3-asyncssh_1.12.2-1_all.deb ... 2023-07-14T11:49:07.312 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2023-07-14T11:49:07.392 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-bs4. 2023-07-14T11:49:07.411 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2023-07-14T11:49:07.450 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-07-14T11:49:07.732 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-jinja2. 2023-07-14T11:49:07.751 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../058-python3-jinja2_2.10.1-2_all.deb ... 2023-07-14T11:49:07.790 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-07-14T11:49:07.837 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-waitress. 2023-07-14T11:49:07.856 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-07-14T11:49:07.927 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-07-14T11:49:08.193 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-07-14T11:49:08.212 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../059-ceph-mgr-cephadm_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:49:08.268 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr-cephadm (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:08.306 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-tempita. 2023-07-14T11:49:08.325 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2023-07-14T11:49:08.364 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-07-14T11:49:08.663 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-bcrypt. 2023-07-14T11:49:08.682 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../060-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-07-14T11:49:08.717 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-paste. 2023-07-14T11:49:08.720 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-07-14T11:49:08.735 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-07-14T11:49:08.775 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-07-14T11:49:09.162 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-07-14T11:49:09.181 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-07-14T11:49:09.220 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-07-14T11:49:09.282 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-jwt. 2023-07-14T11:49:09.301 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-07-14T11:49:09.340 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-07-14T11:49:09.598 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pastedeploy. 2023-07-14T11:49:09.617 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2023-07-14T11:49:09.656 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-07-14T11:49:09.709 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-repoze.lru. 2023-07-14T11:49:09.729 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../062-python3-repoze.lru_0.7-2_all.deb ... 2023-07-14T11:49:09.759 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-07-14T11:49:10.034 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-webtest. 2023-07-14T11:49:10.053 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2023-07-14T11:49:10.062 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libjs-underscore. 2023-07-14T11:49:10.082 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../063-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-07-14T11:49:10.092 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-07-14T11:49:10.120 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-07-14T11:49:10.456 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-07-14T11:49:10.470 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pecan. 2023-07-14T11:49:10.475 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../064-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-07-14T11:49:10.488 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-07-14T11:49:10.514 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-07-14T11:49:10.528 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-07-14T11:49:10.900 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-routes. 2023-07-14T11:49:10.916 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-werkzeug. 2023-07-14T11:49:10.920 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../065-python3-routes_2.4.1-2_all.deb ... 2023-07-14T11:49:10.936 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-07-14T11:49:10.958 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-07-14T11:49:10.999 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-07-14T11:49:11.345 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-07-14T11:49:11.361 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-07-14T11:49:11.362 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../066-ceph-mgr-dashboard_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:49:11.380 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:49:11.419 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mgr-modules-core (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:11.428 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr-dashboard (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:11.866 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-07-14T11:49:11.888 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:11.930 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:11.981 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-decorator. 2023-07-14T11:49:12.001 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../067-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-07-14T11:49:12.039 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-07-14T11:49:12.285 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-cryptography. 2023-07-14T11:49:12.304 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../048-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-07-14T11:49:12.351 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-07-14T11:49:12.417 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-07-14T11:49:12.436 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../068-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-07-14T11:49:12.475 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-07-14T11:49:12.788 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-openssl. 2023-07-14T11:49:12.807 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../049-python3-openssl_19.0.0-1build1_all.deb ... 2023-07-14T11:49:12.845 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-scipy. 2023-07-14T11:49:12.846 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-07-14T11:49:12.863 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../069-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-07-14T11:49:12.938 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-07-14T11:49:13.216 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mgr. 2023-07-14T11:49:13.234 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../050-ceph-mgr_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:13.274 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mgr (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:13.788 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mon. 2023-07-14T11:49:13.806 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../051-ceph-mon_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:13.845 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mon (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:14.362 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-07-14T11:49:14.384 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-07-14T11:49:14.419 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-07-14T11:49:15.016 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-joblib. 2023-07-14T11:49:15.035 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../071-python3-joblib_0.14.0-3_all.deb ... 2023-07-14T11:49:15.072 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-07-14T11:49:15.141 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-osd. 2023-07-14T11:49:15.160 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../052-ceph-osd_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:15.199 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-osd (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:15.484 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-sklearn. 2023-07-14T11:49:15.503 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../072-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-07-14T11:49:15.542 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-07-14T11:49:16.342 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-07-14T11:49:16.362 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../073-ceph-mgr-diskprediction-local_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:49:16.433 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:17.687 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-cachetools. 2023-07-14T11:49:17.706 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../074-python3-cachetools_4.0.0-1_all.deb ... 2023-07-14T11:49:17.778 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-07-14T11:49:17.927 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph. 2023-07-14T11:49:17.945 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../053-ceph_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:17.985 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:18.148 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pyasn1. 2023-07-14T11:49:18.167 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../075-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-07-14T11:49:18.205 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-07-14T11:49:18.371 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-fuse. 2023-07-14T11:49:18.388 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../054-ceph-fuse_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:18.429 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-fuse (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:18.601 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-07-14T11:49:18.619 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../076-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-07-14T11:49:18.658 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-07-14T11:49:18.924 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mds. 2023-07-14T11:49:18.940 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../055-ceph-mds_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:18.982 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mds (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:19.030 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-rsa. 2023-07-14T11:49:19.050 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../077-python3-rsa_4.0-3_all.deb ... 2023-07-14T11:49:19.087 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-rsa (4.0-3) ... 2023-07-14T11:49:19.491 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-google-auth. 2023-07-14T11:49:19.510 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../078-python3-google-auth_1.5.1-2_all.deb ... 2023-07-14T11:49:19.549 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-07-14T11:49:19.577 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package cephadm. 2023-07-14T11:49:19.594 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../056-cephadm_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:19.644 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking cephadm (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:19.944 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-blinker. 2023-07-14T11:49:19.962 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../079-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-07-14T11:49:20.001 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-07-14T11:49:20.039 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-asyncssh. 2023-07-14T11:49:20.055 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../057-python3-asyncssh_1.12.2-1_all.deb ... 2023-07-14T11:49:20.096 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2023-07-14T11:49:20.379 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-oauthlib. 2023-07-14T11:49:20.399 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../080-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-07-14T11:49:20.462 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-07-14T11:49:20.508 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-jinja2. 2023-07-14T11:49:20.525 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../058-python3-jinja2_2.10.1-2_all.deb ... 2023-07-14T11:49:20.566 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-07-14T11:49:20.944 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-07-14T11:49:20.951 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-07-14T11:49:20.961 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../059-ceph-mgr-cephadm_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:49:20.971 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../081-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-07-14T11:49:21.009 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-07-14T11:49:21.011 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mgr-cephadm (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:21.373 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-bcrypt. 2023-07-14T11:49:21.387 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-websocket. 2023-07-14T11:49:21.389 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../060-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-07-14T11:49:21.407 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../082-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-07-14T11:49:21.430 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-07-14T11:49:21.454 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-07-14T11:49:21.800 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-jwt. 2023-07-14T11:49:21.817 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-07-14T11:49:21.825 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-kubernetes. 2023-07-14T11:49:21.843 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../083-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-07-14T11:49:21.858 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-07-14T11:49:21.883 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-07-14T11:49:22.203 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-repoze.lru. 2023-07-14T11:49:22.221 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../062-python3-repoze.lru_0.7-2_all.deb ... 2023-07-14T11:49:22.294 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-07-14T11:49:22.553 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-07-14T11:49:22.568 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../084-ceph-mgr-k8sevents_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:49:22.581 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libjs-underscore. 2023-07-14T11:49:22.597 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../063-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-07-14T11:49:22.612 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:22.638 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-07-14T11:49:22.974 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-07-14T11:49:22.990 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libonig5:amd64. 2023-07-14T11:49:22.990 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../064-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-07-14T11:49:23.003 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../085-libonig5_6.9.4-1_amd64.deb ... 2023-07-14T11:49:23.032 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-07-14T11:49:23.048 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-07-14T11:49:23.427 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-routes. 2023-07-14T11:49:23.445 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../065-python3-routes_2.4.1-2_all.deb ... 2023-07-14T11:49:23.467 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libjq1:amd64. 2023-07-14T11:49:23.481 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../086-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-07-14T11:49:23.485 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-07-14T11:49:23.559 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-07-14T11:49:23.862 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package jq. 2023-07-14T11:49:23.882 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../087-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-07-14T11:49:23.888 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-07-14T11:49:23.909 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../066-ceph-mgr-dashboard_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:49:23.946 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mgr-dashboard (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:23.953 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-07-14T11:49:24.239 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package socat. 2023-07-14T11:49:24.258 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../088-socat_1.7.3.3-2_amd64.deb ... 2023-07-14T11:49:24.331 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking socat (1.7.3.3-2) ... 2023-07-14T11:49:24.533 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-decorator. 2023-07-14T11:49:24.550 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../067-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-07-14T11:49:24.591 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-07-14T11:49:24.659 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package xmlstarlet. 2023-07-14T11:49:24.678 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../089-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-07-14T11:49:24.742 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-07-14T11:49:24.961 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-07-14T11:49:24.978 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../068-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-07-14T11:49:25.019 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-07-14T11:49:25.176 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-test. 2023-07-14T11:49:25.194 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../090-ceph-test_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:25.233 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-test (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:25.380 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-scipy. 2023-07-14T11:49:25.397 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../069-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-07-14T11:49:25.438 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-07-14T11:49:26.847 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-07-14T11:49:26.865 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-07-14T11:49:26.905 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-07-14T11:49:27.542 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-joblib. 2023-07-14T11:49:27.558 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../071-python3-joblib_0.14.0-3_all.deb ... 2023-07-14T11:49:27.600 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-07-14T11:49:28.038 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-sklearn. 2023-07-14T11:49:28.046 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../072-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-07-14T11:49:28.087 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-07-14T11:49:28.871 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-07-14T11:49:28.879 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../073-ceph-mgr-diskprediction-local_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:49:28.920 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:29.445 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-volume. 2023-07-14T11:49:29.465 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../091-ceph-volume_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:49:29.503 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-volume (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:29.991 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package cephfs-mirror. 2023-07-14T11:49:30.011 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../092-cephfs-mirror_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:30.049 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking cephfs-mirror (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:30.215 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-cachetools. 2023-07-14T11:49:30.228 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../074-python3-cachetools_4.0.0-1_all.deb ... 2023-07-14T11:49:30.273 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-07-14T11:49:30.461 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-colorama. 2023-07-14T11:49:30.481 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../093-python3-colorama_0.4.3-1build1_all.deb ... 2023-07-14T11:49:30.518 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-07-14T11:49:30.642 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pyasn1. 2023-07-14T11:49:30.657 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../075-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-07-14T11:49:30.700 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-07-14T11:49:30.838 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package cephfs-shell. 2023-07-14T11:49:30.858 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../094-cephfs-shell_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:49:30.896 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking cephfs-shell (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:31.046 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-07-14T11:49:31.058 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../076-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-07-14T11:49:31.112 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-07-14T11:49:31.207 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package cephfs-top. 2023-07-14T11:49:31.220 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../095-cephfs-top_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:49:31.265 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking cephfs-top (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:31.465 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-rsa. 2023-07-14T11:49:31.477 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../077-python3-rsa_4.0-3_all.deb ... 2023-07-14T11:49:31.514 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-rsa (4.0-3) ... 2023-07-14T11:49:31.584 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package fonts-mathjax. 2023-07-14T11:49:31.598 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-07-14T11:49:31.642 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-07-14T11:49:31.893 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-google-auth. 2023-07-14T11:49:31.906 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../078-python3-google-auth_1.5.1-2_all.deb ... 2023-07-14T11:49:31.950 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-07-14T11:49:32.208 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libcephfs-dev. 2023-07-14T11:49:32.227 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../097-libcephfs-dev_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:32.266 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libcephfs-dev (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:32.338 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-blinker. 2023-07-14T11:49:32.345 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../079-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-07-14T11:49:32.387 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-07-14T11:49:32.741 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-oauthlib. 2023-07-14T11:49:32.745 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package nvme-cli. 2023-07-14T11:49:32.752 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../080-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-07-14T11:49:32.764 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-07-14T11:49:32.790 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-07-14T11:49:32.835 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-07-14T11:49:33.231 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-07-14T11:49:33.250 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2023-07-14T11:49:33.288 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2023-07-14T11:49:33.321 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-07-14T11:49:33.334 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../081-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-07-14T11:49:33.379 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-07-14T11:49:33.725 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libjs-mathjax. 2023-07-14T11:49:33.744 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-07-14T11:49:33.749 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-websocket. 2023-07-14T11:49:33.761 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../082-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-07-14T11:49:33.782 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-07-14T11:49:33.799 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-07-14T11:49:34.186 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-kubernetes. 2023-07-14T11:49:34.199 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../083-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-07-14T11:49:34.244 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-07-14T11:49:35.050 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-07-14T11:49:35.063 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../084-ceph-mgr-k8sevents_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:49:35.107 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:35.503 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libonig5:amd64. 2023-07-14T11:49:35.514 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../085-libonig5_6.9.4-1_amd64.deb ... 2023-07-14T11:49:35.552 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-07-14T11:49:35.867 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-natsort-doc. 2023-07-14T11:49:35.889 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2023-07-14T11:49:35.950 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-07-14T11:49:35.964 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libjq1:amd64. 2023-07-14T11:49:35.976 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../086-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-07-14T11:49:36.013 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-07-14T11:49:36.350 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package jq. 2023-07-14T11:49:36.353 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-atomicwrites. 2023-07-14T11:49:36.363 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../087-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-07-14T11:49:36.372 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../102-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-07-14T11:49:36.407 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-07-14T11:49:36.410 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-07-14T11:49:36.693 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package socat. 2023-07-14T11:49:36.708 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../088-socat_1.7.3.3-2_amd64.deb ... 2023-07-14T11:49:36.764 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-attr. 2023-07-14T11:49:36.776 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking socat (1.7.3.3-2) ... 2023-07-14T11:49:36.783 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../103-python3-attr_19.3.0-2_all.deb ... 2023-07-14T11:49:36.821 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-07-14T11:49:37.138 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package xmlstarlet. 2023-07-14T11:49:37.151 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../089-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-07-14T11:49:37.183 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-webencodings. 2023-07-14T11:49:37.195 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-07-14T11:49:37.202 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../104-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-07-14T11:49:37.241 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-07-14T11:49:37.602 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-html5lib. 2023-07-14T11:49:37.608 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-test. 2023-07-14T11:49:37.617 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../090-ceph-test_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:37.621 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../105-python3-html5lib_1.0.1-2_all.deb ... 2023-07-14T11:49:37.658 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-test (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:37.659 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-07-14T11:49:38.071 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-07-14T11:49:38.089 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../106-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-07-14T11:49:38.132 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-07-14T11:49:38.700 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-packaging. 2023-07-14T11:49:38.720 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../107-python3-packaging_20.3-1_all.deb ... 2023-07-14T11:49:38.757 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-packaging (20.3-1) ... 2023-07-14T11:49:39.102 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pastescript. 2023-07-14T11:49:39.122 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../108-python3-pastescript_2.0.2-4_all.deb ... 2023-07-14T11:49:39.160 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-07-14T11:49:39.844 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pluggy. 2023-07-14T11:49:39.858 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../109-python3-pluggy_0.13.0-2_all.deb ... 2023-07-14T11:49:39.904 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-07-14T11:49:40.275 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-psutil. 2023-07-14T11:49:40.294 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../110-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-07-14T11:49:40.333 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-07-14T11:49:40.728 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-py. 2023-07-14T11:49:40.750 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../111-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-07-14T11:49:40.794 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-07-14T11:49:41.172 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pyinotify. 2023-07-14T11:49:41.193 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-07-14T11:49:41.230 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-07-14T11:49:41.595 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pytest. 2023-07-14T11:49:41.615 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../113-python3-pytest_4.6.9-1_all.deb ... 2023-07-14T11:49:41.652 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-07-14T11:49:41.937 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-volume. 2023-07-14T11:49:41.951 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../091-ceph-volume_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:49:41.995 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-volume (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:42.056 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-simplejson. 2023-07-14T11:49:42.077 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-07-14T11:49:42.155 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-07-14T11:49:42.459 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package cephfs-mirror. 2023-07-14T11:49:42.475 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../092-cephfs-mirror_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:42.483 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-07-14T11:49:42.504 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-07-14T11:49:42.517 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking cephfs-mirror (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:42.574 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-07-14T11:49:42.887 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-colorama. 2023-07-14T11:49:42.902 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../093-python3-colorama_0.4.3-1build1_all.deb ... 2023-07-14T11:49:42.961 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-07-14T11:49:43.267 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package cephfs-shell. 2023-07-14T11:49:43.281 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../094-cephfs-shell_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:49:43.326 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking cephfs-shell (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:43.353 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package radosgw. 2023-07-14T11:49:43.374 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../116-radosgw_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:43.411 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking radosgw (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:43.638 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package cephfs-top. 2023-07-14T11:49:43.652 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../095-cephfs-top_18.0.0-4757-g3fc0089c-1focal_all.deb ... 2023-07-14T11:49:43.695 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking cephfs-top (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:44.015 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package fonts-mathjax. 2023-07-14T11:49:44.031 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-07-14T11:49:44.073 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-07-14T11:49:44.642 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libcephfs-dev. 2023-07-14T11:49:44.651 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../097-libcephfs-dev_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:44.694 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libcephfs-dev (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:44.806 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package rbd-fuse. 2023-07-14T11:49:44.826 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../117-rbd-fuse_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:44.863 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking rbd-fuse (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:45.155 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package nvme-cli. 2023-07-14T11:49:45.168 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-07-14T11:49:45.242 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-07-14T11:49:45.255 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-attr (19.3.0-2) ... 2023-07-14T11:49:45.626 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-07-14T11:49:45.640 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2023-07-14T11:49:45.641 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-07-14T11:49:45.683 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2023-07-14T11:49:45.800 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-07-14T11:49:45.926 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-07-14T11:49:46.121 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libjs-mathjax. 2023-07-14T11:49:46.135 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-07-14T11:49:46.169 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up cephadm (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:46.178 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-07-14T11:49:46.930 INFO:teuthology.orchestra.run.smithi104.stdout:Adding system user cephadm....done 2023-07-14T11:49:47.117 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-07-14T11:49:47.278 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-07-14T11:49:47.704 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-07-14T11:49:48.081 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-07-14T11:49:48.206 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-07-14T11:49:48.245 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python-natsort-doc. 2023-07-14T11:49:48.260 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2023-07-14T11:49:48.302 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-07-14T11:49:48.341 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-07-14T11:49:48.706 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-atomicwrites. 2023-07-14T11:49:48.720 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../102-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-07-14T11:49:48.725 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-07-14T11:49:48.763 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-07-14T11:49:49.117 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-attr. 2023-07-14T11:49:49.131 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../103-python3-attr_19.3.0-2_all.deb ... 2023-07-14T11:49:49.174 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-07-14T11:49:49.269 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-07-14T11:49:49.545 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-webencodings. 2023-07-14T11:49:49.560 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../104-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-07-14T11:49:49.628 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-07-14T11:49:49.653 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-07-14T11:49:49.982 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-html5lib. 2023-07-14T11:49:49.996 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../105-python3-html5lib_1.0.1-2_all.deb ... 2023-07-14T11:49:50.013 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-07-14T11:49:50.039 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-07-14T11:49:50.347 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-07-14T11:49:50.409 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-07-14T11:49:50.424 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../106-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-07-14T11:49:50.467 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-07-14T11:49:50.723 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-07-14T11:49:50.997 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-packaging. 2023-07-14T11:49:51.011 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../107-python3-packaging_20.3-1_all.deb ... 2023-07-14T11:49:51.054 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-packaging (20.3-1) ... 2023-07-14T11:49:51.149 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-ceph-argparse (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:51.399 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pastescript. 2023-07-14T11:49:51.413 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../108-python3-pastescript_2.0.2-4_all.deb ... 2023-07-14T11:49:51.457 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-07-14T11:49:51.534 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-07-14T11:49:51.660 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-07-14T11:49:51.844 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pluggy. 2023-07-14T11:49:51.858 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../109-python3-pluggy_0.13.0-2_all.deb ... 2023-07-14T11:49:51.901 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-07-14T11:49:52.002 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-07-14T11:49:52.282 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-psutil. 2023-07-14T11:49:52.296 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../110-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-07-14T11:49:52.350 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-07-14T11:49:52.370 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-07-14T11:49:52.746 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-07-14T11:49:52.753 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-py. 2023-07-14T11:49:52.768 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../111-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-07-14T11:49:52.811 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-07-14T11:49:52.873 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-07-14T11:49:53.156 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pyinotify. 2023-07-14T11:49:53.171 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-07-14T11:49:53.241 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-07-14T11:49:53.315 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-07-14T11:49:53.592 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pytest. 2023-07-14T11:49:53.610 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../113-python3-pytest_4.6.9-1_all.deb ... 2023-07-14T11:49:53.649 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-07-14T11:49:53.674 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-07-14T11:49:54.017 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-07-14T11:49:54.104 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-simplejson. 2023-07-14T11:49:54.122 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-07-14T11:49:54.161 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-07-14T11:49:54.419 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-07-14T11:49:54.499 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-07-14T11:49:54.518 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-07-14T11:49:54.556 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-07-14T11:49:54.769 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-07-14T11:49:55.181 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-07-14T11:49:55.328 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package radosgw. 2023-07-14T11:49:55.345 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../116-radosgw_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:55.385 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking radosgw (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:55.541 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-packaging (20.3-1) ... 2023-07-14T11:49:55.900 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-07-14T11:49:56.026 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-07-14T11:49:56.152 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-07-14T11:49:56.536 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-07-14T11:49:56.801 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package rbd-fuse. 2023-07-14T11:49:56.817 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../117-rbd-fuse_18.0.0-4757-g3fc0089c-1focal_amd64.deb ... 2023-07-14T11:49:56.859 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking rbd-fuse (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:57.046 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-07-14T11:49:57.212 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-attr (19.3.0-2) ... 2023-07-14T11:49:57.433 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-07-14T11:49:57.598 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-07-14T11:49:57.726 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-07-14T11:49:57.868 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-07-14T11:49:57.883 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-07-14T11:49:58.103 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up cephadm (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:58.312 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up socat (1.7.3.3-2) ... 2023-07-14T11:49:58.437 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-ceph-common (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:49:58.864 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-07-14T11:49:58.873 INFO:teuthology.orchestra.run.smithi121.stdout:Adding system user cephadm....done 2023-07-14T11:49:59.040 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-07-14T11:49:59.068 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-07-14T11:49:59.194 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-07-14T11:49:59.467 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-07-14T11:49:59.594 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-07-14T11:49:59.629 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-07-14T11:49:59.753 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-07-14T11:49:59.980 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-07-14T11:50:00.107 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-07-14T11:50:00.112 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-07-14T11:50:00.505 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-07-14T11:50:00.599 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-07-14T11:50:00.640 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-07-14T11:50:00.766 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-07-14T11:50:00.984 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-07-14T11:50:01.175 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-07-14T11:50:01.511 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-07-14T11:50:01.551 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-07-14T11:50:01.887 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-07-14T11:50:01.945 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-07-14T11:50:02.095 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-07-14T11:50:02.230 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-07-14T11:50:02.462 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-07-14T11:50:02.614 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-07-14T11:50:02.813 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-07-14T11:50:02.949 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-07-14T11:50:03.190 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-07-14T11:50:03.308 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-ceph-argparse (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:03.589 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-07-14T11:50:03.661 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-07-14T11:50:03.785 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-07-14T11:50:03.950 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-07-14T11:50:04.076 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-07-14T11:50:04.144 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-07-14T11:50:04.462 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-07-14T11:50:04.488 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-07-14T11:50:04.847 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-07-14T11:50:04.878 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-07-14T11:50:04.973 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-07-14T11:50:05.004 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-07-14T11:50:05.164 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-07-14T11:50:05.426 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-07-14T11:50:05.548 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-07-14T11:50:05.778 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-07-14T11:50:05.908 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-07-14T11:50:06.121 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-07-14T11:50:06.556 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-07-14T11:50:06.915 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-07-14T11:50:07.324 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-07-14T11:50:07.679 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-07-14T11:50:07.709 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-packaging (20.3-1) ... 2023-07-14T11:50:07.830 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-07-14T11:50:08.080 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-07-14T11:50:08.256 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-07-14T11:50:08.282 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-07-14T11:50:08.382 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-07-14T11:50:08.650 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-07-14T11:50:08.766 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-07-14T11:50:09.009 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-07-14T11:50:09.135 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-07-14T11:50:09.284 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-07-14T11:50:09.544 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-07-14T11:50:09.652 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-07-14T11:50:09.938 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2023-07-14T11:50:10.087 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-07-14T11:50:10.533 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up socat (1.7.3.3-2) ... 2023-07-14T11:50:10.606 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-07-14T11:50:10.659 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-ceph-common (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:11.024 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-07-14T11:50:11.052 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-07-14T11:50:11.194 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-07-14T11:50:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-07-14T11:50:11.789 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-07-14T11:50:11.948 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-07-14T11:50:12.245 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-07-14T11:50:12.299 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-07-14T11:50:12.371 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-07-14T11:50:12.530 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-07-14T11:50:12.656 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-rsa (4.0-3) ... 2023-07-14T11:50:12.717 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-07-14T11:50:12.868 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-07-14T11:50:12.994 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-07-14T11:50:13.007 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-07-14T11:50:13.135 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-07-14T11:50:13.396 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-07-14T11:50:13.494 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-07-14T11:50:13.739 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-07-14T11:50:13.844 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-07-14T11:50:14.099 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-07-14T11:50:14.220 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-07-14T11:50:14.224 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-07-14T11:50:14.575 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-07-14T11:50:14.664 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librados2 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:14.925 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2023-07-14T11:50:14.934 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-07-14T11:50:15.051 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librgw2 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:15.177 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libsqlite3-mod-ceph (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:15.302 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-07-14T11:50:15.336 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libcephfs2 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:15.462 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-routes (2.4.1-2) ... 2023-07-14T11:50:15.712 INFO:teuthology.orchestra.run.smithi121.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-07-14T11:50:15.822 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libradosstriper1 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:15.939 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-07-14T11:50:16.080 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-07-14T11:50:16.206 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-07-14T11:50:16.316 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librbd1 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:16.467 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr-modules-core (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:16.590 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-07-14T11:50:16.593 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-fuse (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:16.855 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-07-14T11:50:16.855 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-07-14T11:50:17.025 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-07-14T11:50:17.151 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-07-14T11:50:17.277 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-07-14T11:50:17.686 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-07-14T11:50:17.695 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libcephfs-dev (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:17.815 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-rados (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:17.966 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up cephfs-top (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:18.029 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-07-14T11:50:18.091 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-07-14T11:50:19.212 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-rbd (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:19.340 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up rbd-fuse (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:19.489 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-rgw (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:19.615 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-cephfs (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:19.741 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up cephfs-shell (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:19.793 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-07-14T11:50:19.900 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-common (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:19.919 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-07-14T11:50:20.396 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-07-14T11:50:20.624 INFO:teuthology.orchestra.run.smithi104.stdout:Adding group ceph....done 2023-07-14T11:50:20.772 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-07-14T11:50:21.090 INFO:teuthology.orchestra.run.smithi104.stdout:Adding system user ceph....done 2023-07-14T11:50:21.115 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-07-14T11:50:21.240 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-07-14T11:50:21.312 INFO:teuthology.orchestra.run.smithi104.stdout:Setting system user ceph properties....done 2023-07-14T11:50:21.341 INFO:teuthology.orchestra.run.smithi104.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-07-14T11:50:21.531 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-07-14T11:50:21.659 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-07-14T11:50:22.067 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-07-14T11:50:22.094 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2023-07-14T11:50:22.587 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-07-14T11:50:22.996 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-07-14T11:50:23.003 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up cephfs-mirror (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:23.270 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-07-14T11:50:23.270 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-07-14T11:50:24.146 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-test (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:24.243 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-07-14T11:50:24.272 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up radosgw (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:24.369 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-07-14T11:50:24.504 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-07-14T11:50:24.586 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-07-14T11:50:24.586 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-07-14T11:50:24.655 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-rsa (4.0-3) ... 2023-07-14T11:50:25.022 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-07-14T11:50:25.174 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-07-14T11:50:25.533 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-07-14T11:50:25.816 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-base (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:25.917 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-07-14T11:50:26.310 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-07-14T11:50:26.314 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-07-14T11:50:26.754 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up librados2 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:26.957 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2023-07-14T11:50:27.083 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up librgw2 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:27.135 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mds (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:27.242 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libsqlite3-mod-ceph (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:27.368 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libcephfs2 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:27.439 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-07-14T11:50:27.439 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-07-14T11:50:27.503 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-routes (2.4.1-2) ... 2023-07-14T11:50:27.855 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libradosstriper1 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:27.981 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-07-14T11:50:28.229 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:28.344 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up librbd1 (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:28.478 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mgr-modules-core (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:28.499 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-07-14T11:50:28.499 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-07-14T11:50:28.604 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-fuse (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:28.900 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-07-14T11:50:28.900 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-07-14T11:50:29.346 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-osd (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libcephfs-dev (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:29.906 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-rados (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:30.032 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up cephfs-top (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:30.147 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-07-14T11:50:30.147 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-07-14T11:50:30.150 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-07-14T11:50:31.073 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr-k8sevents (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:31.199 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:31.350 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mon (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:31.371 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-rbd (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:31.497 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up rbd-fuse (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:31.623 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-rgw (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:31.631 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-07-14T11:50:31.632 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-07-14T11:50:31.751 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-cephfs (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:31.911 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up cephfs-shell (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:32.037 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-common (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:32.509 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr-cephadm (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:32.652 INFO:teuthology.orchestra.run.smithi121.stdout:Adding group ceph....done 2023-07-14T11:50:32.677 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:32.819 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr-dashboard (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:33.003 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-volume (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:33.115 INFO:teuthology.orchestra.run.smithi121.stdout:Adding system user ceph....done 2023-07-14T11:50:33.130 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-07-14T11:50:33.290 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-07-14T11:50:33.379 INFO:teuthology.orchestra.run.smithi121.stdout:Setting system user ceph properties....done 2023-07-14T11:50:33.419 INFO:teuthology.orchestra.run.smithi121.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-07-14T11:50:33.614 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-07-14T11:50:33.791 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-07-14T11:50:34.136 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-07-14T11:50:35.073 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up cephfs-mirror (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:35.340 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-07-14T11:50:35.341 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-07-14T11:50:36.200 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-test (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:36.326 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up radosgw (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:36.671 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-07-14T11:50:36.672 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-07-14T11:50:37.946 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-base (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:38.425 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-07-14T11:50:38.902 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-07-14T11:50:39.306 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mds (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:39.583 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-07-14T11:50:39.583 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-07-14T11:50:39.671 INFO:teuthology.orchestra.run.smithi104.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-07-14T11:50:39.677 DEBUG:teuthology.orchestra.run.smithi104:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-07-14T11:50:39.802 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2023-07-14T11:50:39.999 INFO:teuthology.orchestra.run.smithi104.stdout:Building dependency tree... 2023-07-14T11:50:40.000 INFO:teuthology.orchestra.run.smithi104.stdout:Reading state information... 2023-07-14T11:50:40.183 INFO:teuthology.orchestra.run.smithi104.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-07-14T11:50:40.183 INFO:teuthology.orchestra.run.smithi104.stdout:flex is already the newest version (2.6.4-6.2). 2023-07-14T11:50:40.183 INFO:teuthology.orchestra.run.smithi104.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-07-14T11:50:40.184 INFO:teuthology.orchestra.run.smithi104.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-07-14T11:50:40.184 INFO:teuthology.orchestra.run.smithi104.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-07-14T11:50:40.184 INFO:teuthology.orchestra.run.smithi104.stdout:The following packages were automatically installed and are no longer required: 2023-07-14T11:50:40.184 INFO:teuthology.orchestra.run.smithi104.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-07-14T11:50:40.185 INFO:teuthology.orchestra.run.smithi104.stdout:Use 'sudo apt autoremove' to remove them. 2023-07-14T11:50:40.185 INFO:teuthology.orchestra.run.smithi104.stdout:The following additional packages will be installed: 2023-07-14T11:50:40.185 INFO:teuthology.orchestra.run.smithi104.stdout: libbluetooth3 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 2023-07-14T11:50:40.185 INFO:teuthology.orchestra.run.smithi104.stdout: libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 libuuid1 2023-07-14T11:50:40.186 INFO:teuthology.orchestra.run.smithi104.stdout: modemmanager network-manager-pptp postgresql-12 postgresql-client-12 2023-07-14T11:50:40.186 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-client-common ppp pptp-linux usb-modeswitch usb-modeswitch-data 2023-07-14T11:50:40.186 INFO:teuthology.orchestra.run.smithi104.stdout: wpasupplicant 2023-07-14T11:50:40.186 INFO:teuthology.orchestra.run.smithi104.stdout:Suggested packages: 2023-07-14T11:50:40.187 INFO:teuthology.orchestra.run.smithi104.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2023-07-14T11:50:40.187 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2023-07-14T11:50:40.187 INFO:teuthology.orchestra.run.smithi104.stdout: libengine-pkcs11-openssl 2023-07-14T11:50:40.276 INFO:teuthology.orchestra.run.smithi104.stdout:The following NEW packages will be installed: 2023-07-14T11:50:40.276 INFO:teuthology.orchestra.run.smithi104.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-07-14T11:50:40.276 INFO:teuthology.orchestra.run.smithi104.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2023-07-14T11:50:40.277 INFO:teuthology.orchestra.run.smithi104.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-07-14T11:50:40.277 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2023-07-14T11:50:40.277 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2023-07-14T11:50:40.277 INFO:teuthology.orchestra.run.smithi104.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-07-14T11:50:40.278 INFO:teuthology.orchestra.run.smithi104.stdout:The following packages will be upgraded: 2023-07-14T11:50:40.279 INFO:teuthology.orchestra.run.smithi104.stdout: libssl1.1 libuuid1 util-linux uuid-dev 2023-07-14T11:50:40.399 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mgr (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:40.484 INFO:teuthology.orchestra.run.smithi104.stdout:4 upgraded, 31 newly installed, 0 to remove and 206 not upgraded. 2023-07-14T11:50:40.484 INFO:teuthology.orchestra.run.smithi104.stdout:Need to get 25.1 MB of archives. 2023-07-14T11:50:40.485 INFO:teuthology.orchestra.run.smithi104.stdout:After this operation, 79.9 MB of additional disk space will be used. 2023-07-14T11:50:40.485 INFO:teuthology.orchestra.run.smithi104.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-07-14T11:50:40.688 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-07-14T11:50:40.689 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-07-14T11:50:41.044 INFO:teuthology.orchestra.run.smithi104.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-07-14T11:50:41.046 INFO:teuthology.orchestra.run.smithi104.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-07-14T11:50:41.047 INFO:teuthology.orchestra.run.smithi104.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-07-14T11:50:41.151 INFO:teuthology.orchestra.run.smithi104.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-07-14T11:50:41.156 INFO:teuthology.orchestra.run.smithi104.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-07-14T11:50:41.163 INFO:teuthology.orchestra.run.smithi104.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2023-07-14T11:50:41.168 INFO:teuthology.orchestra.run.smithi104.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-07-14T11:50:41.169 INFO:teuthology.orchestra.run.smithi104.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-07-14T11:50:41.172 INFO:teuthology.orchestra.run.smithi104.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2023-07-14T11:50:41.174 INFO:teuthology.orchestra.run.smithi104.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-07-14T11:50:41.327 INFO:teuthology.orchestra.run.smithi104.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-07-14T11:50:41.417 INFO:teuthology.orchestra.run.smithi104.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-07-14T11:50:41.559 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-osd (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:41.722 INFO:teuthology.orchestra.run.smithi104.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-07-14T11:50:41.729 INFO:teuthology.orchestra.run.smithi104.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-07-14T11:50:41.859 INFO:teuthology.orchestra.run.smithi104.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.15-0ubuntu0.20.04.1 [116 kB] 2023-07-14T11:50:41.884 INFO:teuthology.orchestra.run.smithi104.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-07-14T11:50:41.971 INFO:teuthology.orchestra.run.smithi104.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-07-14T11:50:41.972 INFO:teuthology.orchestra.run.smithi104.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-07-14T11:50:42.091 INFO:teuthology.orchestra.run.smithi104.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-07-14T11:50:42.091 INFO:teuthology.orchestra.run.smithi104.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-07-14T11:50:42.122 INFO:teuthology.orchestra.run.smithi104.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-07-14T11:50:42.364 INFO:teuthology.orchestra.run.smithi104.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-07-14T11:50:42.372 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-07-14T11:50:42.372 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-07-14T11:50:43.194 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mgr-k8sevents (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:43.226 INFO:teuthology.orchestra.run.smithi104.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-07-14T11:50:43.239 INFO:teuthology.orchestra.run.smithi104.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-07-14T11:50:43.241 INFO:teuthology.orchestra.run.smithi104.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-07-14T11:50:43.242 INFO:teuthology.orchestra.run.smithi104.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2023-07-14T11:50:43.244 INFO:teuthology.orchestra.run.smithi104.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.15-0ubuntu0.20.04.1 [1,052 kB] 2023-07-14T11:50:43.286 INFO:teuthology.orchestra.run.smithi104.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2023-07-14T11:50:43.292 INFO:teuthology.orchestra.run.smithi104.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.15-0ubuntu0.20.04.1 [13.5 MB] 2023-07-14T11:50:43.328 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:43.479 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mon (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:43.730 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-07-14T11:50:43.731 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-07-14T11:50:43.777 INFO:teuthology.orchestra.run.smithi104.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2023-07-14T11:50:43.777 INFO:teuthology.orchestra.run.smithi104.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2023-07-14T11:50:43.778 INFO:teuthology.orchestra.run.smithi104.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2023-07-14T11:50:43.972 INFO:teuthology.orchestra.run.smithi104.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-07-14T11:50:44.202 INFO:teuthology.orchestra.run.smithi104.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-07-14T11:50:44.580 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mgr-cephadm (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:44.764 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:44.863 INFO:teuthology.orchestra.run.smithi104.stdout:Preconfiguring packages ... 2023-07-14T11:50:44.890 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mgr-dashboard (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:45.041 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-volume (18.0.0-4757-g3fc0089c-1focal) ... 2023-07-14T11:50:45.100 INFO:teuthology.orchestra.run.smithi104.stdout:Fetched 25.1 MB in 4s (6,224 kB/s) 2023-07-14T11:50:45.202 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-07-14T11:50:45.361 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-07-14T11:50:45.435 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118869 files and directories currently installed.) 2023-07-14T11:50:45.438 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-07-14T11:50:45.660 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-07-14T11:50:45.854 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-07-14T11:50:47.282 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-07-14T11:50:49.301 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118869 files and directories currently installed.) 2023-07-14T11:50:49.305 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-07-14T11:50:49.468 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-07-14T11:50:49.930 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-07-14T11:50:50.141 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-07-14T11:50:50.496 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-07-14T11:50:50.907 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-07-14T11:50:51.096 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118869 files and directories currently installed.) 2023-07-14T11:50:51.099 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../00-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-07-14T11:50:51.232 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-07-14T11:50:51.697 INFO:teuthology.orchestra.run.smithi121.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-07-14T11:50:51.703 DEBUG:teuthology.orchestra.run.smithi121:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-07-14T11:50:51.823 INFO:teuthology.orchestra.run.smithi121.stdout:Reading package lists... 2023-07-14T11:50:52.024 INFO:teuthology.orchestra.run.smithi121.stdout:Building dependency tree... 2023-07-14T11:50:52.024 INFO:teuthology.orchestra.run.smithi121.stdout:Reading state information... 2023-07-14T11:50:52.113 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-07-14T11:50:52.134 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2_amd64.deb ... 2023-07-14T11:50:52.180 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-07-14T11:50:52.210 INFO:teuthology.orchestra.run.smithi121.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-07-14T11:50:52.211 INFO:teuthology.orchestra.run.smithi121.stdout:flex is already the newest version (2.6.4-6.2). 2023-07-14T11:50:52.211 INFO:teuthology.orchestra.run.smithi121.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-07-14T11:50:52.211 INFO:teuthology.orchestra.run.smithi121.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-07-14T11:50:52.211 INFO:teuthology.orchestra.run.smithi121.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-07-14T11:50:52.212 INFO:teuthology.orchestra.run.smithi121.stdout:The following packages were automatically installed and are no longer required: 2023-07-14T11:50:52.212 INFO:teuthology.orchestra.run.smithi121.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-07-14T11:50:52.213 INFO:teuthology.orchestra.run.smithi121.stdout:Use 'sudo apt autoremove' to remove them. 2023-07-14T11:50:52.213 INFO:teuthology.orchestra.run.smithi121.stdout:The following additional packages will be installed: 2023-07-14T11:50:52.213 INFO:teuthology.orchestra.run.smithi121.stdout: libbluetooth3 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 2023-07-14T11:50:52.213 INFO:teuthology.orchestra.run.smithi121.stdout: libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 libuuid1 2023-07-14T11:50:52.213 INFO:teuthology.orchestra.run.smithi121.stdout: modemmanager network-manager-pptp postgresql-12 postgresql-client-12 2023-07-14T11:50:52.213 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-client-common ppp pptp-linux usb-modeswitch usb-modeswitch-data 2023-07-14T11:50:52.213 INFO:teuthology.orchestra.run.smithi121.stdout: wpasupplicant 2023-07-14T11:50:52.214 INFO:teuthology.orchestra.run.smithi121.stdout:Suggested packages: 2023-07-14T11:50:52.214 INFO:teuthology.orchestra.run.smithi121.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2023-07-14T11:50:52.214 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2023-07-14T11:50:52.214 INFO:teuthology.orchestra.run.smithi121.stdout: libengine-pkcs11-openssl 2023-07-14T11:50:52.310 INFO:teuthology.orchestra.run.smithi121.stdout:The following NEW packages will be installed: 2023-07-14T11:50:52.311 INFO:teuthology.orchestra.run.smithi121.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-07-14T11:50:52.311 INFO:teuthology.orchestra.run.smithi121.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2023-07-14T11:50:52.311 INFO:teuthology.orchestra.run.smithi121.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-07-14T11:50:52.311 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2023-07-14T11:50:52.312 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2023-07-14T11:50:52.312 INFO:teuthology.orchestra.run.smithi121.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-07-14T11:50:52.312 INFO:teuthology.orchestra.run.smithi121.stdout:The following packages will be upgraded: 2023-07-14T11:50:52.313 INFO:teuthology.orchestra.run.smithi121.stdout: libssl1.1 libuuid1 util-linux uuid-dev 2023-07-14T11:50:52.574 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package dump. 2023-07-14T11:50:52.594 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../02-dump_0.4b46-6_amd64.deb ... 2023-07-14T11:50:52.632 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking dump (0.4b46-6) ... 2023-07-14T11:50:52.699 INFO:teuthology.orchestra.run.smithi121.stdout:4 upgraded, 31 newly installed, 0 to remove and 206 not upgraded. 2023-07-14T11:50:52.699 INFO:teuthology.orchestra.run.smithi121.stdout:Need to get 25.1 MB of archives. 2023-07-14T11:50:52.700 INFO:teuthology.orchestra.run.smithi121.stdout:After this operation, 79.9 MB of additional disk space will be used. 2023-07-14T11:50:52.700 INFO:teuthology.orchestra.run.smithi121.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-07-14T11:50:52.988 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package indent. 2023-07-14T11:50:53.008 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../03-indent_2.2.12-1_amd64.deb ... 2023-07-14T11:50:53.046 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking indent (2.2.12-1) ... 2023-07-14T11:50:53.407 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-07-14T11:50:53.428 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-5_amd64.deb ... 2023-07-14T11:50:53.465 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-07-14T11:50:53.861 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-07-14T11:50:53.880 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../05-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-07-14T11:50:53.919 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-07-14T11:50:53.998 INFO:teuthology.orchestra.run.smithi121.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-07-14T11:50:54.001 INFO:teuthology.orchestra.run.smithi121.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-07-14T11:50:54.003 INFO:teuthology.orchestra.run.smithi121.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-07-14T11:50:54.212 INFO:teuthology.orchestra.run.smithi121.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-07-14T11:50:54.217 INFO:teuthology.orchestra.run.smithi121.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-07-14T11:50:54.232 INFO:teuthology.orchestra.run.smithi121.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2023-07-14T11:50:54.243 INFO:teuthology.orchestra.run.smithi121.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-07-14T11:50:54.244 INFO:teuthology.orchestra.run.smithi121.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-07-14T11:50:54.247 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-07-14T11:50:54.251 INFO:teuthology.orchestra.run.smithi121.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2023-07-14T11:50:54.258 INFO:teuthology.orchestra.run.smithi121.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-07-14T11:50:54.267 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../06-libelf-dev_0.176-1.1build1_amd64.deb ... 2023-07-14T11:50:54.305 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2023-07-14T11:50:54.617 INFO:teuthology.orchestra.run.smithi121.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-07-14T11:50:54.708 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-07-14T11:50:54.729 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../07-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-07-14T11:50:54.766 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-07-14T11:50:54.800 INFO:teuthology.orchestra.run.smithi121.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-07-14T11:50:55.094 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libmbim-proxy. 2023-07-14T11:50:55.114 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../08-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-07-14T11:50:55.152 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-07-14T11:50:55.449 INFO:teuthology.orchestra.run.smithi121.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-07-14T11:50:55.464 INFO:teuthology.orchestra.run.smithi121.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-07-14T11:50:55.530 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-07-14T11:50:55.550 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../09-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-07-14T11:50:55.588 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-07-14T11:50:55.699 INFO:teuthology.orchestra.run.smithi121.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.15-0ubuntu0.20.04.1 [116 kB] 2023-07-14T11:50:55.742 INFO:teuthology.orchestra.run.smithi121.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-07-14T11:50:55.889 INFO:teuthology.orchestra.run.smithi121.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-07-14T11:50:55.890 INFO:teuthology.orchestra.run.smithi121.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-07-14T11:50:56.007 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libndp0:amd64. 2023-07-14T11:50:56.027 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../10-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-07-14T11:50:56.056 INFO:teuthology.orchestra.run.smithi121.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-07-14T11:50:56.057 INFO:teuthology.orchestra.run.smithi121.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-07-14T11:50:56.074 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-07-14T11:50:56.112 INFO:teuthology.orchestra.run.smithi121.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-07-14T11:50:56.469 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libnm0:amd64. 2023-07-14T11:50:56.473 INFO:teuthology.orchestra.run.smithi121.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-07-14T11:50:56.490 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../11-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-07-14T11:50:56.535 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-07-14T11:50:56.963 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libpq5:amd64. 2023-07-14T11:50:56.983 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../12-libpq5_12.15-0ubuntu0.20.04.1_amd64.deb ... 2023-07-14T11:50:57.021 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libpq5:amd64 (12.15-0ubuntu0.20.04.1) ... 2023-07-14T11:50:57.449 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-07-14T11:50:57.469 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../13-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-07-14T11:50:57.507 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-07-14T11:50:57.826 INFO:teuthology.orchestra.run.smithi121.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-07-14T11:50:57.841 INFO:teuthology.orchestra.run.smithi121.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-07-14T11:50:57.843 INFO:teuthology.orchestra.run.smithi121.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-07-14T11:50:57.844 INFO:teuthology.orchestra.run.smithi121.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2023-07-14T11:50:57.846 INFO:teuthology.orchestra.run.smithi121.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.15-0ubuntu0.20.04.1 [1,052 kB] 2023-07-14T11:50:57.885 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libqmi-proxy. 2023-07-14T11:50:57.906 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../14-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-07-14T11:50:57.910 INFO:teuthology.orchestra.run.smithi121.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2023-07-14T11:50:57.923 INFO:teuthology.orchestra.run.smithi121.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.15-0ubuntu0.20.04.1 [13.5 MB] 2023-07-14T11:50:57.951 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-07-14T11:50:58.263 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-07-14T11:50:58.283 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../15-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-07-14T11:50:58.321 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-07-14T11:50:58.715 INFO:teuthology.orchestra.run.smithi121.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2023-07-14T11:50:58.715 INFO:teuthology.orchestra.run.smithi121.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2023-07-14T11:50:58.715 INFO:teuthology.orchestra.run.smithi121.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2023-07-14T11:50:58.908 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-07-14T11:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../16-libteamdctl0_1.30-1_amd64.deb ... 2023-07-14T11:50:58.966 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-07-14T11:50:59.026 INFO:teuthology.orchestra.run.smithi121.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-07-14T11:50:59.394 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package modemmanager. 2023-07-14T11:50:59.406 INFO:teuthology.orchestra.run.smithi121.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-07-14T11:50:59.414 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../17-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-07-14T11:50:59.476 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-07-14T11:51:00.154 INFO:teuthology.orchestra.run.smithi121.stdout:Preconfiguring packages ... 2023-07-14T11:51:00.158 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package wpasupplicant. 2023-07-14T11:51:00.179 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../18-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-07-14T11:51:00.216 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-07-14T11:51:00.389 INFO:teuthology.orchestra.run.smithi121.stdout:Fetched 25.1 MB in 7s (3,442 kB/s) 2023-07-14T11:51:00.746 INFO:teuthology.orchestra.run.smithi121.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118869 files and directories currently installed.) 2023-07-14T11:51:00.750 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-07-14T11:51:00.805 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package network-manager. 2023-07-14T11:51:00.824 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../19-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-07-14T11:51:00.908 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-07-14T11:51:00.940 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-07-14T11:51:01.745 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ppp. 2023-07-14T11:51:01.764 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../20-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-07-14T11:51:02.229 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-07-14T11:51:02.551 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-07-14T11:51:02.608 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package pptp-linux. 2023-07-14T11:51:02.628 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../21-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-07-14T11:51:02.666 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-07-14T11:51:03.095 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package network-manager-pptp. 2023-07-14T11:51:03.115 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../22-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-07-14T11:51:03.217 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-07-14T11:51:03.606 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package postgresql-client-common. 2023-07-14T11:51:03.626 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../23-postgresql-client-common_214ubuntu0.1_all.deb ... 2023-07-14T11:51:03.664 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2023-07-14T11:51:04.044 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package postgresql-client-12. 2023-07-14T11:51:04.064 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../24-postgresql-client-12_12.15-0ubuntu0.20.04.1_amd64.deb ... 2023-07-14T11:51:04.101 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking postgresql-client-12 (12.15-0ubuntu0.20.04.1) ... 2023-07-14T11:51:04.566 INFO:teuthology.orchestra.run.smithi121.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118869 files and directories currently installed.) 2023-07-14T11:51:04.570 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-07-14T11:51:04.725 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-07-14T11:51:04.798 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package postgresql-common. 2023-07-14T11:51:04.820 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../25-postgresql-common_214ubuntu0.1_all.deb ... 2023-07-14T11:51:04.890 INFO:teuthology.orchestra.run.smithi104.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-07-14T11:51:04.918 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2023-07-14T11:51:05.209 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-07-14T11:51:05.381 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-07-14T11:51:05.529 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package postgresql-12. 2023-07-14T11:51:05.550 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../26-postgresql-12_12.15-0ubuntu0.20.04.1_amd64.deb ... 2023-07-14T11:51:05.602 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking postgresql-12 (12.15-0ubuntu0.20.04.1) ... 2023-07-14T11:51:05.775 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-07-14T11:51:06.361 INFO:teuthology.orchestra.run.smithi121.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118869 files and directories currently installed.) 2023-07-14T11:51:06.364 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../00-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-07-14T11:51:06.497 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-07-14T11:51:07.110 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-07-14T11:51:07.131 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2_amd64.deb ... 2023-07-14T11:51:07.173 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package postgresql. 2023-07-14T11:51:07.176 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-07-14T11:51:07.195 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../27-postgresql_12+214ubuntu0.1_all.deb ... 2023-07-14T11:51:07.240 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2023-07-14T11:51:07.542 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package postgresql-client. 2023-07-14T11:51:07.563 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../28-postgresql-client_12+214ubuntu0.1_all.deb ... 2023-07-14T11:51:07.571 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package dump. 2023-07-14T11:51:07.591 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../02-dump_0.4b46-6_amd64.deb ... 2023-07-14T11:51:07.600 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2023-07-14T11:51:07.629 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking dump (0.4b46-6) ... 2023-07-14T11:51:07.911 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package postgresql-contrib. 2023-07-14T11:51:07.932 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../29-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2023-07-14T11:51:07.969 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2023-07-14T11:51:07.983 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package indent. 2023-07-14T11:51:08.004 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../03-indent_2.2.12-1_amd64.deb ... 2023-07-14T11:51:08.049 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking indent (2.2.12-1) ... 2023-07-14T11:51:08.263 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-07-14T11:51:08.284 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../30-usb-modeswitch-data_20191128-3_all.deb ... 2023-07-14T11:51:08.321 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-07-14T11:51:08.428 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-07-14T11:51:08.448 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-5_amd64.deb ... 2023-07-14T11:51:08.485 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-07-14T11:51:08.766 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package usb-modeswitch. 2023-07-14T11:51:08.786 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../31-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-07-14T11:51:08.846 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-07-14T11:51:08.881 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-07-14T11:51:08.901 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../05-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-07-14T11:51:08.939 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-07-14T11:51:09.228 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2023-07-14T11:51:09.267 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-07-14T11:51:09.287 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../06-libelf-dev_0.176-1.1build1_amd64.deb ... 2023-07-14T11:51:09.325 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2023-07-14T11:51:09.471 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-07-14T11:51:09.711 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-07-14T11:51:09.732 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../07-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-07-14T11:51:09.803 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-07-14T11:51:09.856 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libpq5:amd64 (12.15-0ubuntu0.20.04.1) ... 2023-07-14T11:51:09.982 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-07-14T11:51:10.105 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libmbim-proxy. 2023-07-14T11:51:10.108 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-07-14T11:51:10.125 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../08-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-07-14T11:51:10.163 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-07-14T11:51:10.276 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-07-14T11:51:10.566 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-07-14T11:51:10.588 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../09-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-07-14T11:51:10.633 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-07-14T11:51:10.705 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-07-14T11:51:10.706 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-07-14T11:51:11.053 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libndp0:amd64. 2023-07-14T11:51:11.073 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../10-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-07-14T11:51:11.119 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-07-14T11:51:11.497 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libnm0:amd64. 2023-07-14T11:51:11.518 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../11-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-07-14T11:51:11.562 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up postgresql-client-12 (12.15-0ubuntu0.20.04.1) ... 2023-07-14T11:51:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-07-14T11:51:12.000 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libpq5:amd64. 2023-07-14T11:51:12.021 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../12-libpq5_12.15-0ubuntu0.20.04.1_amd64.deb ... 2023-07-14T11:51:12.058 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libpq5:amd64 (12.15-0ubuntu0.20.04.1) ... 2023-07-14T11:51:12.478 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-07-14T11:51:12.498 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../13-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-07-14T11:51:12.536 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-07-14T11:51:12.905 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libqmi-proxy. 2023-07-14T11:51:12.925 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../14-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-07-14T11:51:12.963 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-07-14T11:51:13.249 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-07-14T11:51:13.269 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../15-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-07-14T11:51:13.298 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-07-14T11:51:13.651 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-07-14T11:51:13.759 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-07-14T11:51:13.836 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-07-14T11:51:13.855 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../16-libteamdctl0_1.30-1_amd64.deb ... 2023-07-14T11:51:13.877 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-07-14T11:51:13.927 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-07-14T11:51:14.002 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2023-07-14T11:51:14.347 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package modemmanager. 2023-07-14T11:51:14.367 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../17-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-07-14T11:51:14.450 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-07-14T11:51:15.161 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package wpasupplicant. 2023-07-14T11:51:15.182 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../18-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-07-14T11:51:15.219 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-07-14T11:51:15.299 INFO:teuthology.orchestra.run.smithi104.stdout:Adding user postgres to group ssl-cert 2023-07-14T11:51:15.697 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:51:15.697 INFO:teuthology.orchestra.run.smithi104.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-07-14T11:51:15.791 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package network-manager. 2023-07-14T11:51:15.812 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../19-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-07-14T11:51:15.909 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-07-14T11:51:16.188 INFO:teuthology.orchestra.run.smithi104.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-07-14T11:51:16.188 INFO:teuthology.orchestra.run.smithi104.stdout:Removing obsolete dictionary files: 2023-07-14T11:51:16.698 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ppp. 2023-07-14T11:51:16.719 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../20-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-07-14T11:51:16.743 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-07-14T11:51:17.186 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-07-14T11:51:17.595 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package pptp-linux. 2023-07-14T11:51:17.616 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../21-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-07-14T11:51:17.661 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-07-14T11:51:18.073 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package network-manager-pptp. 2023-07-14T11:51:18.092 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../22-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-07-14T11:51:18.129 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-07-14T11:51:18.171 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-07-14T11:51:18.254 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-07-14T11:51:18.413 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2023-07-14T11:51:18.539 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-07-14T11:51:18.559 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package postgresql-client-common. 2023-07-14T11:51:18.580 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../23-postgresql-client-common_214ubuntu0.1_all.deb ... 2023-07-14T11:51:18.617 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2023-07-14T11:51:18.691 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-07-14T11:51:19.005 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package postgresql-client-12. 2023-07-14T11:51:19.025 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../24-postgresql-client-12_12.15-0ubuntu0.20.04.1_amd64.deb ... 2023-07-14T11:51:19.063 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking postgresql-client-12 (12.15-0ubuntu0.20.04.1) ... 2023-07-14T11:51:19.759 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package postgresql-common. 2023-07-14T11:51:19.779 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../25-postgresql-common_214ubuntu0.1_all.deb ... 2023-07-14T11:51:19.846 INFO:teuthology.orchestra.run.smithi121.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-07-14T11:51:19.870 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2023-07-14T11:51:19.958 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-07-14T11:51:20.404 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up dump (0.4b46-6) ... 2023-07-14T11:51:20.407 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package postgresql-12. 2023-07-14T11:51:20.428 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../26-postgresql-12_12.15-0ubuntu0.20.04.1_amd64.deb ... 2023-07-14T11:51:20.480 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking postgresql-12 (12.15-0ubuntu0.20.04.1) ... 2023-07-14T11:51:20.517 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-07-14T11:51:20.581 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-07-14T11:51:20.740 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-07-14T11:51:20.866 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up indent (2.2.12-1) ... 2023-07-14T11:51:20.992 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-07-14T11:51:21.210 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-07-14T11:51:21.336 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up postgresql-12 (12.15-0ubuntu0.20.04.1) ... 2023-07-14T11:51:21.864 INFO:teuthology.orchestra.run.smithi104.stdout:Creating new PostgreSQL cluster 12/main ... 2023-07-14T11:51:21.924 INFO:teuthology.orchestra.run.smithi104.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2023-07-14T11:51:21.991 INFO:teuthology.orchestra.run.smithi104.stdout:The files belonging to this database system will be owned by user "postgres". 2023-07-14T11:51:21.992 INFO:teuthology.orchestra.run.smithi104.stdout:This user must also own the server process. 2023-07-14T11:51:21.992 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:51:21.992 INFO:teuthology.orchestra.run.smithi104.stdout:The database cluster will be initialized with locale "en_US". 2023-07-14T11:51:21.992 INFO:teuthology.orchestra.run.smithi104.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-07-14T11:51:21.993 INFO:teuthology.orchestra.run.smithi104.stdout:The default text search configuration will be set to "english". 2023-07-14T11:51:21.993 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:51:21.993 INFO:teuthology.orchestra.run.smithi104.stdout:Data page checksums are disabled. 2023-07-14T11:51:21.993 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:51:21.993 INFO:teuthology.orchestra.run.smithi104.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2023-07-14T11:51:21.995 INFO:teuthology.orchestra.run.smithi104.stdout:creating subdirectories ... ok 2023-07-14T11:51:21.996 INFO:teuthology.orchestra.run.smithi104.stdout:selecting dynamic shared memory implementation ... posix 2023-07-14T11:51:22.034 INFO:teuthology.orchestra.run.smithi104.stdout:selecting default max_connections ... 100 2023-07-14T11:51:22.082 INFO:teuthology.orchestra.run.smithi104.stdout:selecting default shared_buffers ... 128MB 2023-07-14T11:51:22.102 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package postgresql. 2023-07-14T11:51:22.103 INFO:teuthology.orchestra.run.smithi104.stdout:selecting default time zone ... Etc/UTC 2023-07-14T11:51:22.104 INFO:teuthology.orchestra.run.smithi104.stdout:creating configuration files ... ok 2023-07-14T11:51:22.123 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../27-postgresql_12+214ubuntu0.1_all.deb ... 2023-07-14T11:51:22.193 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2023-07-14T11:51:22.294 INFO:teuthology.orchestra.run.smithi104.stdout:running bootstrap script ... ok 2023-07-14T11:51:22.837 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package postgresql-client. 2023-07-14T11:51:22.858 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../28-postgresql-client_12+214ubuntu0.1_all.deb ... 2023-07-14T11:51:22.895 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2023-07-14T11:51:22.918 INFO:teuthology.orchestra.run.smithi104.stdout:performing post-bootstrap initialization ... ok 2023-07-14T11:51:23.206 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package postgresql-contrib. 2023-07-14T11:51:23.225 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../29-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2023-07-14T11:51:23.264 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2023-07-14T11:51:23.428 INFO:teuthology.orchestra.run.smithi104.stdout:syncing data to disk ... ok 2023-07-14T11:51:23.428 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:51:23.428 INFO:teuthology.orchestra.run.smithi104.stdout:Success. You can now start the database server using: 2023-07-14T11:51:23.428 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:51:23.429 INFO:teuthology.orchestra.run.smithi104.stdout: pg_ctlcluster 12 main start 2023-07-14T11:51:23.429 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:51:23.550 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-07-14T11:51:23.570 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../30-usb-modeswitch-data_20191128-3_all.deb ... 2023-07-14T11:51:23.608 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-07-14T11:51:24.003 INFO:teuthology.orchestra.run.smithi104.stdout:Ver Cluster Port Status Owner Data directory Log file 2023-07-14T11:51:24.003 INFO:teuthology.orchestra.run.smithi104.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2023-07-14T11:51:24.053 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package usb-modeswitch. 2023-07-14T11:51:24.073 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../31-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-07-14T11:51:24.130 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-07-14T11:51:24.201 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-07-14T11:51:24.504 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2023-07-14T11:51:24.750 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-07-14T11:51:25.118 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libpq5:amd64 (12.15-0ubuntu0.20.04.1) ... 2023-07-14T11:51:25.277 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-07-14T11:51:25.403 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-07-14T11:51:25.529 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-07-14T11:51:26.000 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-07-14T11:51:26.000 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-07-14T11:51:26.899 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up postgresql-client-12 (12.15-0ubuntu0.20.04.1) ... 2023-07-14T11:51:27.868 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-07-14T11:51:27.994 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-07-14T11:51:28.196 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2023-07-14T11:51:28.321 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-07-14T11:51:28.998 INFO:teuthology.orchestra.run.smithi121.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-07-14T11:51:29.104 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-07-14T11:51:29.230 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-07-14T11:51:29.389 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2023-07-14T11:51:29.505 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:51:29.505 INFO:teuthology.orchestra.run.smithi104.stdout:The following network interfaces were found in /etc/network/interfaces 2023-07-14T11:51:29.506 INFO:teuthology.orchestra.run.smithi104.stdout:which means they are currently configured by ifupdown: 2023-07-14T11:51:29.506 INFO:teuthology.orchestra.run.smithi104.stdout:- enp3s0f1 2023-07-14T11:51:29.507 INFO:teuthology.orchestra.run.smithi104.stdout:If you want to manage those interfaces with NetworkManager instead 2023-07-14T11:51:29.507 INFO:teuthology.orchestra.run.smithi104.stdout:remove their configuration from /etc/network/interfaces. 2023-07-14T11:51:29.507 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:51:29.740 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-07-14T11:51:30.275 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-07-14T11:51:30.625 INFO:teuthology.orchestra.run.smithi121.stdout:Adding user postgres to group ssl-cert 2023-07-14T11:51:30.999 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:51:30.999 INFO:teuthology.orchestra.run.smithi121.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-07-14T11:51:31.181 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-07-14T11:51:31.488 INFO:teuthology.orchestra.run.smithi121.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-07-14T11:51:31.489 INFO:teuthology.orchestra.run.smithi121.stdout:Removing obsolete dictionary files: 2023-07-14T11:51:32.042 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-07-14T11:51:32.124 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-07-14T11:51:32.259 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-07-14T11:51:32.435 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2023-07-14T11:51:32.561 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2023-07-14T11:51:32.890 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-07-14T11:51:33.022 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-07-14T11:51:33.156 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-07-14T11:51:33.481 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-07-14T11:51:33.492 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-07-14T11:51:33.493 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-07-14T11:51:33.607 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-07-14T11:51:33.733 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2023-07-14T11:51:33.859 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-07-14T11:51:34.043 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-07-14T11:51:34.500 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-07-14T11:51:34.609 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-07-14T11:51:34.777 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-07-14T11:51:34.937 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-07-14T11:51:35.272 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-07-14T11:51:35.438 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-07-14T11:51:35.708 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up dump (0.4b46-6) ... 2023-07-14T11:51:35.815 INFO:teuthology.orchestra.run.smithi121.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-07-14T11:51:35.884 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-07-14T11:51:36.052 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-07-14T11:51:36.178 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up indent (2.2.12-1) ... 2023-07-14T11:51:36.304 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-07-14T11:51:36.488 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-07-14T11:51:36.656 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up postgresql-12 (12.15-0ubuntu0.20.04.1) ... 2023-07-14T11:51:37.170 INFO:teuthology.orchestra.run.smithi121.stdout:Creating new PostgreSQL cluster 12/main ... 2023-07-14T11:51:37.203 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2023-07-14T11:51:37.281 INFO:teuthology.orchestra.run.smithi121.stdout:The files belonging to this database system will be owned by user "postgres". 2023-07-14T11:51:37.282 INFO:teuthology.orchestra.run.smithi121.stdout:This user must also own the server process. 2023-07-14T11:51:37.282 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:51:37.283 INFO:teuthology.orchestra.run.smithi121.stdout:The database cluster will be initialized with locale "en_US". 2023-07-14T11:51:37.283 INFO:teuthology.orchestra.run.smithi121.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-07-14T11:51:37.283 INFO:teuthology.orchestra.run.smithi121.stdout:The default text search configuration will be set to "english". 2023-07-14T11:51:37.283 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:51:37.284 INFO:teuthology.orchestra.run.smithi121.stdout:Data page checksums are disabled. 2023-07-14T11:51:37.284 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:51:37.284 INFO:teuthology.orchestra.run.smithi121.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2023-07-14T11:51:37.285 INFO:teuthology.orchestra.run.smithi121.stdout:creating subdirectories ... ok 2023-07-14T11:51:37.285 INFO:teuthology.orchestra.run.smithi121.stdout:selecting dynamic shared memory implementation ... posix 2023-07-14T11:51:37.323 INFO:teuthology.orchestra.run.smithi121.stdout:selecting default max_connections ... 100 2023-07-14T11:51:37.380 INFO:teuthology.orchestra.run.smithi121.stdout:selecting default shared_buffers ... 128MB 2023-07-14T11:51:37.394 INFO:teuthology.orchestra.run.smithi121.stdout:selecting default time zone ... Etc/UTC 2023-07-14T11:51:37.395 INFO:teuthology.orchestra.run.smithi121.stdout:creating configuration files ... ok 2023-07-14T11:51:37.587 INFO:teuthology.orchestra.run.smithi121.stdout:running bootstrap script ... ok 2023-07-14T11:51:38.230 INFO:teuthology.orchestra.run.smithi121.stdout:performing post-bootstrap initialization ... ok 2023-07-14T11:51:38.767 INFO:teuthology.orchestra.run.smithi121.stdout:syncing data to disk ... ok 2023-07-14T11:51:38.767 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:51:38.767 INFO:teuthology.orchestra.run.smithi121.stdout:Success. You can now start the database server using: 2023-07-14T11:51:38.767 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:51:38.767 INFO:teuthology.orchestra.run.smithi121.stdout: pg_ctlcluster 12 main start 2023-07-14T11:51:38.768 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:51:39.350 INFO:teuthology.orchestra.run.smithi121.stdout:Ver Cluster Port Status Owner Data directory Log file 2023-07-14T11:51:39.350 INFO:teuthology.orchestra.run.smithi121.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2023-07-14T11:51:39.551 INFO:teuthology.orchestra.run.smithi121.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-07-14T11:51:43.222 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-07-14T11:51:43.348 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-07-14T11:51:43.516 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2023-07-14T11:51:43.641 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-07-14T11:51:44.857 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:51:44.857 INFO:teuthology.orchestra.run.smithi121.stdout:The following network interfaces were found in /etc/network/interfaces 2023-07-14T11:51:44.858 INFO:teuthology.orchestra.run.smithi121.stdout:which means they are currently configured by ifupdown: 2023-07-14T11:51:44.858 INFO:teuthology.orchestra.run.smithi121.stdout:- enp3s0f1 2023-07-14T11:51:44.858 INFO:teuthology.orchestra.run.smithi121.stdout:If you want to manage those interfaces with NetworkManager instead 2023-07-14T11:51:44.858 INFO:teuthology.orchestra.run.smithi121.stdout:remove their configuration from /etc/network/interfaces. 2023-07-14T11:51:44.858 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:51:45.088 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-07-14T11:51:45.637 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-07-14T11:51:46.611 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-07-14T11:51:47.569 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-07-14T11:51:47.712 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-07-14T11:51:47.914 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2023-07-14T11:51:48.039 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2023-07-14T11:51:48.165 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-07-14T11:51:48.291 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-07-14T11:51:48.459 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-07-14T11:51:48.599 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-07-14T11:51:48.741 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-07-14T11:51:48.775 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-07-14T11:51:48.776 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-07-14T11:51:49.786 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-07-14T11:51:49.912 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-07-14T11:51:49.985 INFO:teuthology.orchestra.run.smithi104.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-07-14T11:51:49.991 DEBUG:teuthology.parallel:result is None 2023-07-14T11:51:50.049 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-07-14T11:51:50.217 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-07-14T11:51:50.685 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-07-14T11:52:04.038 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-07-14T11:52:04.146 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-07-14T11:52:05.375 INFO:teuthology.orchestra.run.smithi121.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-07-14T11:52:05.381 DEBUG:teuthology.parallel:result is None 2023-07-14T11:52:05.382 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:52:05.729 DEBUG:teuthology.orchestra.run.smithi104:> dpkg-query -W -f '${Version}' ceph 2023-07-14T11:52:05.766 INFO:teuthology.orchestra.run.smithi104.stdout:18.0.0-4757-g3fc0089c-1focal 2023-07-14T11:52:05.766 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757-g3fc0089c-1focal 2023-07-14T11:52:05.767 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757-g3fc0089c-1focal is installed. 2023-07-14T11:52:05.767 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:52:06.051 DEBUG:teuthology.orchestra.run.smithi121:> dpkg-query -W -f '${Version}' ceph 2023-07-14T11:52:06.088 INFO:teuthology.orchestra.run.smithi121.stdout:18.0.0-4757-g3fc0089c-1focal 2023-07-14T11:52:06.089 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757-g3fc0089c-1focal 2023-07-14T11:52:06.089 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757-g3fc0089c-1focal is installed. 2023-07-14T11:52:06.090 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-14T11:52:06.090 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:06.091 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T11:52:06.124 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:06.125 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T11:52:06.165 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-14T11:52:06.165 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:06.166 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T11:52:06.203 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T11:52:06.284 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:06.284 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T11:52:06.318 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T11:52:06.393 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-14T11:52:06.394 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:06.394 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T11:52:06.426 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T11:52:06.503 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:06.503 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T11:52:06.537 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T11:52:06.617 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-14T11:52:06.618 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:06.618 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T11:52:06.652 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T11:52:06.731 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:06.731 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T11:52:06.765 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T11:52:06.845 INFO:teuthology.run_tasks:Running task ceph... 2023-07-14T11:52:06.916 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-07-14T11:52:06.917 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 777 /var/log/ceph 2023-07-14T11:52:06.924 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /var/log/ceph 2023-07-14T11:52:06.954 INFO:tasks.ceph:Disabling ceph logrotate... 2023-07-14T11:52:06.955 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-14T11:52:07.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-14T11:52:07.037 INFO:tasks.ceph:Creating extra log directories... 2023-07-14T11:52:07.038 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-14T11:52:07.080 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-14T11:52:07.128 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-07-14T11:52:07.129 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-07-14T11:52:07.129 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-07-14T11:52:07.130 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:07.130 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-14T11:52:07.165 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-14T11:52:07.287 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:07.288 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-14T11:52:07.308 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-14T11:52:07.450 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-07-14T11:52:07.451 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-07-14T11:52:07.451 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435', 'branch': 'wip-rishabh-2023Jul13', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm}', 'email': None, 'first_in_suite': False, 'job_id': '7337435', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-2023Jul13'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-2023Jul13', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-2023Jul13', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi104.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0=', 'smithi121.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_pool_perm']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'aa6a80e6cc655fff48452a7eca02070e61d074fc', 'timestamp': '2023-07-14_10:26:42', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-07-14T11:52:07.452 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-14T11:52:07.459 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-14T11:52:07.479 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m0777 -- /var/run/ceph 2023-07-14T11:52:07.525 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/run/ceph 2023-07-14T11:52:07.561 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:07.562 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T11:52:07.611 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T11:52:07.611 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 2023-07-14T11:52:07.673 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T11:52:07.674 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:52:07.674 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 495 Links: 1 2023-07-14T11:52:07.675 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:52:07.675 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2023-07-14 11:46:01.578618781 +0000 2023-07-14T11:52:07.675 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2023-07-14 11:46:01.074630404 +0000 2023-07-14T11:52:07.675 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2023-07-14 11:46:01.074630404 +0000 2023-07-14T11:52:07.676 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2023-07-14T11:52:07.676 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T11:52:07.745 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2023-07-14T11:52:07.745 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2023-07-14T11:52:07.745 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000721149 s, 710 kB/s 2023-07-14T11:52:07.747 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T11:52:07.817 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 2023-07-14T11:52:07.875 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T11:52:07.876 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:52:07.876 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 503 Links: 1 2023-07-14T11:52:07.876 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:52:07.876 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2023-07-14 11:46:02.366600609 +0000 2023-07-14T11:52:07.876 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2023-07-14 11:46:01.890611586 +0000 2023-07-14T11:52:07.877 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2023-07-14 11:46:01.890611586 +0000 2023-07-14T11:52:07.877 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2023-07-14T11:52:07.877 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T11:52:07.949 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2023-07-14T11:52:07.949 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2023-07-14T11:52:07.949 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000600717 s, 852 kB/s 2023-07-14T11:52:07.951 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T11:52:08.022 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 2023-07-14T11:52:08.081 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T11:52:08.082 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:52:08.082 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 509 Links: 1 2023-07-14T11:52:08.082 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:52:08.082 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2023-07-14 11:46:03.290579300 +0000 2023-07-14T11:52:08.082 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2023-07-14 11:46:02.650594059 +0000 2023-07-14T11:52:08.082 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2023-07-14 11:46:02.650594059 +0000 2023-07-14T11:52:08.082 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2023-07-14T11:52:08.083 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T11:52:08.154 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2023-07-14T11:52:08.154 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2023-07-14T11:52:08.154 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000707564 s, 724 kB/s 2023-07-14T11:52:08.156 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T11:52:08.222 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 2023-07-14T11:52:08.289 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T11:52:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:52:08.290 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 516 Links: 1 2023-07-14T11:52:08.290 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:52:08.290 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2023-07-14 11:46:04.082561035 +0000 2023-07-14T11:52:08.290 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2023-07-14 11:46:03.602572106 +0000 2023-07-14T11:52:08.290 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2023-07-14 11:46:03.602572106 +0000 2023-07-14T11:52:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2023-07-14T11:52:08.291 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T11:52:08.365 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2023-07-14T11:52:08.366 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2023-07-14T11:52:08.366 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000568231 s, 901 kB/s 2023-07-14T11:52:08.367 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T11:52:08.433 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-14T11:52:08.434 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:08.434 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T11:52:08.454 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T11:52:08.454 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_1 2023-07-14T11:52:08.518 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T11:52:08.519 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:52:08.519 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 498 Links: 1 2023-07-14T11:52:08.519 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:52:08.519 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-07-14 11:46:01.722513059 +0000 2023-07-14T11:52:08.519 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-07-14 11:46:01.718513150 +0000 2023-07-14T11:52:08.519 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-07-14 11:46:01.718513150 +0000 2023-07-14T11:52:08.520 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-07-14T11:52:08.520 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T11:52:08.591 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-07-14T11:52:08.592 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-07-14T11:52:08.592 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000673286 s, 760 kB/s 2023-07-14T11:52:08.593 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T11:52:08.664 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_2 2023-07-14T11:52:08.722 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T11:52:08.723 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:52:08.723 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 501 Links: 1 2023-07-14T11:52:08.723 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:52:08.723 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-07-14 11:46:02.530494435 +0000 2023-07-14T11:52:08.723 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-07-14 11:46:02.526494526 +0000 2023-07-14T11:52:08.723 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-07-14 11:46:02.526494526 +0000 2023-07-14T11:52:08.723 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-07-14T11:52:08.724 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T11:52:08.796 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-07-14T11:52:08.796 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-07-14T11:52:08.797 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000605514 s, 846 kB/s 2023-07-14T11:52:08.798 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T11:52:08.864 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_3 2023-07-14T11:52:08.928 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T11:52:08.929 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:52:08.929 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 510 Links: 1 2023-07-14T11:52:08.929 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:52:08.929 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-07-14 11:46:03.778465668 +0000 2023-07-14T11:52:08.930 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-07-14 11:46:03.306476548 +0000 2023-07-14T11:52:08.930 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-07-14 11:46:03.306476548 +0000 2023-07-14T11:52:08.930 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-07-14T11:52:08.931 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T11:52:09.000 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-07-14T11:52:09.000 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-07-14T11:52:09.001 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000694193 s, 738 kB/s 2023-07-14T11:52:09.003 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T11:52:09.070 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_4 2023-07-14T11:52:09.135 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T11:52:09.135 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:52:09.136 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 515 Links: 1 2023-07-14T11:52:09.136 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:52:09.136 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-07-14 11:46:04.514448704 +0000 2023-07-14T11:52:09.137 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-07-14 11:46:04.050459398 +0000 2023-07-14T11:52:09.137 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-07-14 11:46:04.050459398 +0000 2023-07-14T11:52:09.137 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-07-14T11:52:09.138 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T11:52:09.207 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-07-14T11:52:09.208 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-07-14T11:52:09.208 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000532848 s, 961 kB/s 2023-07-14T11:52:09.209 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T11:52:09.276 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-14T11:52:09.276 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi104.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-14T11:52:09.276 INFO:tasks.ceph:Generating config... 2023-07-14T11:52:09.278 INFO:tasks.ceph:[client] client mount timeout = 600 2023-07-14T11:52:09.278 INFO:tasks.ceph:[client] debug client = 20 2023-07-14T11:52:09.278 INFO:tasks.ceph:[client] debug ms = 1 2023-07-14T11:52:09.278 INFO:tasks.ceph:[client] pid file = 2023-07-14T11:52:09.279 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-07-14T11:52:09.279 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-07-14T11:52:09.279 INFO:tasks.ceph:[mds] debug mds = 20 2023-07-14T11:52:09.279 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-07-14T11:52:09.279 INFO:tasks.ceph:[mds] debug ms = 1 2023-07-14T11:52:09.280 INFO:tasks.ceph:[mds] mds debug frag = True 2023-07-14T11:52:09.280 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-07-14T11:52:09.280 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-07-14T11:52:09.280 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-07-14T11:52:09.280 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-07-14T11:52:09.281 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-07-14T11:52:09.281 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-07-14T11:52:09.281 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-07-14T11:52:09.281 INFO:tasks.ceph:[mgr] debug ms = 1 2023-07-14T11:52:09.281 INFO:tasks.ceph:[mon] debug mon = 20 2023-07-14T11:52:09.282 INFO:tasks.ceph:[mon] debug ms = 1 2023-07-14T11:52:09.282 INFO:tasks.ceph:[mon] debug paxos = 20 2023-07-14T11:52:09.282 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-07-14T11:52:09.282 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-07-14T11:52:09.282 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-07-14T11:52:09.283 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-07-14T11:52:09.283 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-07-14T11:52:09.283 INFO:tasks.ceph:[osd] debug ms = 1 2023-07-14T11:52:09.283 INFO:tasks.ceph:[osd] debug osd = 20 2023-07-14T11:52:09.283 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-07-14T11:52:09.283 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-07-14T11:52:09.284 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-07-14T11:52:09.284 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-07-14T11:52:09.284 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-07-14T11:52:09.284 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-07-14T11:52:09.284 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-07-14T11:52:09.285 INFO:tasks.ceph:Setting up mon.a... 2023-07-14T11:52:09.285 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-07-14T11:52:09.356 INFO:teuthology.orchestra.run.smithi104.stdout:creating /etc/ceph/ceph.keyring 2023-07-14T11:52:09.371 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-07-14T11:52:09.445 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-14T11:52:09.478 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.104'), ('mon.b', '172.21.15.121'), ('mon.c', '[v2:172.21.15.121:3301,v1:172.21.15.121:6790]')] 2023-07-14T11:52:09.479 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.104,172.21.15.121,[v2:172.21.15.121:3301,v1:172.21.15.121:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-07-14T11:52:09.481 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:09.481 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-07-14T11:52:09.542 DEBUG:teuthology.orchestra.run.smithi104:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.104 --add b 172.21.15.121 --addv c '[v2:172.21.15.121:3301,v1:172.21.15.121:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-07-14T11:52:09.659 INFO:teuthology.orchestra.run.smithi104.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-07-14T11:52:09.659 INFO:teuthology.orchestra.run.smithi104.stdout:monmaptool: generated fsid c5576993-99e1-43b1-a253-659a3cee5212 2023-07-14T11:52:09.659 INFO:teuthology.orchestra.run.smithi104.stdout:setting min_mon_release = pacific 2023-07-14T11:52:09.659 INFO:teuthology.orchestra.run.smithi104.stdout:epoch 0 2023-07-14T11:52:09.659 INFO:teuthology.orchestra.run.smithi104.stdout:fsid c5576993-99e1-43b1-a253-659a3cee5212 2023-07-14T11:52:09.660 INFO:teuthology.orchestra.run.smithi104.stdout:last_changed 2023-07-14T11:52:09.676918+0000 2023-07-14T11:52:09.660 INFO:teuthology.orchestra.run.smithi104.stdout:created 2023-07-14T11:52:09.676918+0000 2023-07-14T11:52:09.660 INFO:teuthology.orchestra.run.smithi104.stdout:min_mon_release 16 (pacific) 2023-07-14T11:52:09.660 INFO:teuthology.orchestra.run.smithi104.stdout:election_strategy: 1 2023-07-14T11:52:09.660 INFO:teuthology.orchestra.run.smithi104.stdout:0: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.a 2023-07-14T11:52:09.661 INFO:teuthology.orchestra.run.smithi104.stdout:1: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.b 2023-07-14T11:52:09.661 INFO:teuthology.orchestra.run.smithi104.stdout:2: [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] mon.c 2023-07-14T11:52:09.661 INFO:teuthology.orchestra.run.smithi104.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-07-14T11:52:09.662 DEBUG:teuthology.orchestra.run.smithi104:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-07-14T11:52:09.680 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c5576993-99e1-43b1-a253-659a3cee5212... 2023-07-14T11:52:09.681 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-14T11:52:09.729 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-14T11:52:09.802 INFO:teuthology.orchestra.run.smithi121.stdout:[global] 2023-07-14T11:52:09.803 INFO:teuthology.orchestra.run.smithi121.stdout: chdir = "" 2023-07-14T11:52:09.803 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-14T11:52:09.803 INFO:teuthology.orchestra.run.smithi121.stdout: auth supported = cephx 2023-07-14T11:52:09.804 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.804 INFO:teuthology.orchestra.run.smithi121.stdout: filestore xattr use omap = true 2023-07-14T11:52:09.804 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.804 INFO:teuthology.orchestra.run.smithi121.stdout: mon clock drift allowed = 1.000 2023-07-14T11:52:09.805 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.805 INFO:teuthology.orchestra.run.smithi121.stdout: osd crush chooseleaf type = 0 2023-07-14T11:52:09.805 INFO:teuthology.orchestra.run.smithi121.stdout: auth debug = true 2023-07-14T11:52:09.805 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.806 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on old message = true 2023-07-14T11:52:09.806 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on bug = true 2023-07-14T11:52:09.806 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.806 INFO:teuthology.orchestra.run.smithi121.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-14T11:52:09.807 INFO:teuthology.orchestra.run.smithi121.stdout: mon pg warn max object skew = 0 2023-07-14T11:52:09.807 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.807 INFO:teuthology.orchestra.run.smithi121.stdout: # disable pg_autoscaler by default for new pools 2023-07-14T11:52:09.807 INFO:teuthology.orchestra.run.smithi121.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-14T11:52:09.808 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.808 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default size = 2 2023-07-14T11:52:09.808 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.809 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow primary affinity = true 2023-07-14T11:52:09.809 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow pg remap = true 2023-07-14T11:52:09.809 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on legacy crush tunables = false 2023-07-14T11:52:09.810 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on crush straw calc version zero = false 2023-07-14T11:52:09.810 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on no sortbitwise = false 2023-07-14T11:52:09.810 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on osd down out interval zero = false 2023-07-14T11:52:09.810 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on too few osds = false 2023-07-14T11:52:09.811 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-14T11:52:09.811 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-14T11:52:09.811 INFO:teuthology.orchestra.run.smithi121.stdout: mon_allow_pool_size_one = true 2023-07-14T11:52:09.811 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.812 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-14T11:52:09.812 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.812 INFO:teuthology.orchestra.run.smithi121.stdout: osd default data pool replay window = 5 2023-07-14T11:52:09.812 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.813 INFO:teuthology.orchestra.run.smithi121.stdout: mon allow pool delete = true 2023-07-14T11:52:09.813 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.813 INFO:teuthology.orchestra.run.smithi121.stdout: mon cluster log file level = debug 2023-07-14T11:52:09.813 INFO:teuthology.orchestra.run.smithi121.stdout: debug asserts on shutdown = true 2023-07-14T11:52:09.813 INFO:teuthology.orchestra.run.smithi121.stdout: mon health detail to clog = false 2023-07-14T11:52:09.814 INFO:teuthology.orchestra.run.smithi121.stdout: mon host = "172.21.15.104,172.21.15.121,[v2:172.21.15.121:3301,v1:172.21.15.121:6790]" 2023-07-14T11:52:09.814 INFO:teuthology.orchestra.run.smithi121.stdout: fsid = c5576993-99e1-43b1-a253-659a3cee5212 2023-07-14T11:52:09.814 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.814 INFO:teuthology.orchestra.run.smithi121.stdout:[osd] 2023-07-14T11:52:09.815 INFO:teuthology.orchestra.run.smithi121.stdout: osd journal size = 100 2023-07-14T11:52:09.815 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.815 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub load threshold = 5.0 2023-07-14T11:52:09.815 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub max interval = 600 2023-07-14T11:52:09.816 INFO:teuthology.orchestra.run.smithi121.stdout: osd mclock profile = high_recovery_ops 2023-07-14T11:52:09.816 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.816 INFO:teuthology.orchestra.run.smithi121.stdout: osd recover clone overlap = true 2023-07-14T11:52:09.817 INFO:teuthology.orchestra.run.smithi121.stdout: osd recovery max chunk = 1048576 2023-07-14T11:52:09.817 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.817 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug shutdown = true 2023-07-14T11:52:09.817 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug op order = true 2023-07-14T11:52:09.818 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify stray on activate = true 2023-07-14T11:52:09.818 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.818 INFO:teuthology.orchestra.run.smithi121.stdout: osd open classes on start = true 2023-07-14T11:52:09.818 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug pg log writeout = true 2023-07-14T11:52:09.819 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.819 INFO:teuthology.orchestra.run.smithi121.stdout: osd deep scrub update digest min age = 30 2023-07-14T11:52:09.819 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.819 INFO:teuthology.orchestra.run.smithi121.stdout: osd map max advance = 10 2023-07-14T11:52:09.820 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.820 INFO:teuthology.orchestra.run.smithi121.stdout: journal zero on create = true 2023-07-14T11:52:09.820 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.820 INFO:teuthology.orchestra.run.smithi121.stdout: filestore ondisk finisher threads = 3 2023-07-14T11:52:09.821 INFO:teuthology.orchestra.run.smithi121.stdout: filestore apply finisher threads = 3 2023-07-14T11:52:09.821 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.821 INFO:teuthology.orchestra.run.smithi121.stdout: bdev debug aio = true 2023-07-14T11:52:09.821 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug misdirected ops = true 2023-07-14T11:52:09.822 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore block size = 96636764160 2023-07-14T11:52:09.822 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore fsck on mount = True 2023-07-14T11:52:09.822 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluefs = 20 2023-07-14T11:52:09.822 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluestore = 20 2023-07-14T11:52:09.823 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-07-14T11:52:09.823 INFO:teuthology.orchestra.run.smithi121.stdout: debug osd = 20 2023-07-14T11:52:09.823 INFO:teuthology.orchestra.run.smithi121.stdout: debug rocksdb = 10 2023-07-14T11:52:09.823 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd backfillfull_ratio = 0.85 2023-07-14T11:52:09.823 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd full ratio = 0.9 2023-07-14T11:52:09.824 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd nearfull ratio = 0.8 2023-07-14T11:52:09.824 INFO:teuthology.orchestra.run.smithi121.stdout: osd failsafe full ratio = 0.95 2023-07-14T11:52:09.824 INFO:teuthology.orchestra.run.smithi121.stdout: osd objectstore = bluestore 2023-07-14T11:52:09.824 INFO:teuthology.orchestra.run.smithi121.stdout: osd op complaint time = 180 2023-07-14T11:52:09.825 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.825 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr] 2023-07-14T11:52:09.825 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-07-14T11:52:09.825 INFO:teuthology.orchestra.run.smithi121.stdout: debug mgr = 20 2023-07-14T11:52:09.826 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2023-07-14T11:52:09.826 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2023-07-14T11:52:09.826 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2023-07-14T11:52:09.826 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2023-07-14T11:52:09.827 INFO:teuthology.orchestra.run.smithi121.stdout: mgr/telemetry/nag = false 2023-07-14T11:52:09.827 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.827 INFO:teuthology.orchestra.run.smithi121.stdout:[mon] 2023-07-14T11:52:09.827 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-07-14T11:52:09.828 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2023-07-14T11:52:09.828 INFO:teuthology.orchestra.run.smithi121.stdout: debug paxos = 20 2023-07-14T11:52:09.828 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2023-07-14T11:52:09.828 INFO:teuthology.orchestra.run.smithi121.stdout: mon data avail warn = 5 2023-07-14T11:52:09.829 INFO:teuthology.orchestra.run.smithi121.stdout: mon mgr mkfs grace = 240 2023-07-14T11:52:09.829 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2023-07-14T11:52:09.829 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd reporter subtree level = osd 2023-07-14T11:52:09.829 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd prime pg temp = true 2023-07-14T11:52:09.830 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2023-07-14T11:52:09.830 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.830 INFO:teuthology.orchestra.run.smithi121.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-14T11:52:09.830 INFO:teuthology.orchestra.run.smithi121.stdout: auth mon ticket ttl = 660 # 11m 2023-07-14T11:52:09.831 INFO:teuthology.orchestra.run.smithi121.stdout: auth service ticket ttl = 240 # 4m 2023-07-14T11:52:09.831 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.831 INFO:teuthology.orchestra.run.smithi121.stdout: # don't complain about insecure global_id in the test suite 2023-07-14T11:52:09.831 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-14T11:52:09.832 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-14T11:52:09.832 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.832 INFO:teuthology.orchestra.run.smithi121.stdout: # 1m isn't quite enough 2023-07-14T11:52:09.833 INFO:teuthology.orchestra.run.smithi121.stdout: mon_down_mkfs_grace = 2m 2023-07-14T11:52:09.833 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.833 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_filestore_osds = false 2023-07-14T11:52:09.833 INFO:teuthology.orchestra.run.smithi121.stdout: mon op complaint time = 120 2023-07-14T11:52:09.834 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-14T11:52:09.834 INFO:teuthology.orchestra.run.smithi121.stdout:[client] 2023-07-14T11:52:09.834 INFO:teuthology.orchestra.run.smithi121.stdout: rgw cache enabled = true 2023-07-14T11:52:09.834 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable ops log = true 2023-07-14T11:52:09.835 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable usage log = true 2023-07-14T11:52:09.835 INFO:teuthology.orchestra.run.smithi121.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-14T11:52:09.835 INFO:teuthology.orchestra.run.smithi121.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-14T11:52:09.835 INFO:teuthology.orchestra.run.smithi121.stdout: client mount timeout = 600 2023-07-14T11:52:09.836 INFO:teuthology.orchestra.run.smithi121.stdout: debug client = 20 2023-07-14T11:52:09.836 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-07-14T11:52:09.836 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = "" 2023-07-14T11:52:09.836 INFO:teuthology.orchestra.run.smithi121.stdout: rados mon op timeout = 900 2023-07-14T11:52:09.837 INFO:teuthology.orchestra.run.smithi121.stdout: rados osd op timeout = 900 2023-07-14T11:52:09.837 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.a] 2023-07-14T11:52:09.837 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.b] 2023-07-14T11:52:09.837 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.c] 2023-07-14T11:52:09.838 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.a] 2023-07-14T11:52:09.838 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.c] 2023-07-14T11:52:09.838 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.b] 2023-07-14T11:52:09.838 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.d] 2023-07-14T11:52:09.838 INFO:teuthology.orchestra.run.smithi121.stdout:[mds] 2023-07-14T11:52:09.838 INFO:teuthology.orchestra.run.smithi121.stdout: debug mds = 20 2023-07-14T11:52:09.838 INFO:teuthology.orchestra.run.smithi121.stdout: debug mds balancer = 20 2023-07-14T11:52:09.838 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-07-14T11:52:09.839 INFO:teuthology.orchestra.run.smithi121.stdout: mds debug frag = True 2023-07-14T11:52:09.839 INFO:teuthology.orchestra.run.smithi121.stdout: mds debug scatterstat = True 2023-07-14T11:52:09.839 INFO:teuthology.orchestra.run.smithi121.stdout: mds op complaint time = 180 2023-07-14T11:52:09.839 INFO:teuthology.orchestra.run.smithi121.stdout: mds verify scatter = True 2023-07-14T11:52:09.839 INFO:teuthology.orchestra.run.smithi121.stdout: osd op complaint time = 180 2023-07-14T11:52:09.839 INFO:teuthology.orchestra.run.smithi121.stdout: rados mon op timeout = 900 2023-07-14T11:52:09.839 INFO:teuthology.orchestra.run.smithi121.stdout: rados osd op timeout = 900 2023-07-14T11:52:09.841 INFO:teuthology.orchestra.run.smithi104.stdout:[global] 2023-07-14T11:52:09.841 INFO:teuthology.orchestra.run.smithi104.stdout: chdir = "" 2023-07-14T11:52:09.841 INFO:teuthology.orchestra.run.smithi104.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-14T11:52:09.841 INFO:teuthology.orchestra.run.smithi104.stdout: auth supported = cephx 2023-07-14T11:52:09.841 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.841 INFO:teuthology.orchestra.run.smithi104.stdout: filestore xattr use omap = true 2023-07-14T11:52:09.842 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.842 INFO:teuthology.orchestra.run.smithi104.stdout: mon clock drift allowed = 1.000 2023-07-14T11:52:09.842 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.842 INFO:teuthology.orchestra.run.smithi104.stdout: osd crush chooseleaf type = 0 2023-07-14T11:52:09.842 INFO:teuthology.orchestra.run.smithi104.stdout: auth debug = true 2023-07-14T11:52:09.842 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.842 INFO:teuthology.orchestra.run.smithi104.stdout: ms die on old message = true 2023-07-14T11:52:09.843 INFO:teuthology.orchestra.run.smithi104.stdout: ms die on bug = true 2023-07-14T11:52:09.843 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.843 INFO:teuthology.orchestra.run.smithi104.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-14T11:52:09.843 INFO:teuthology.orchestra.run.smithi104.stdout: mon pg warn max object skew = 0 2023-07-14T11:52:09.843 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.843 INFO:teuthology.orchestra.run.smithi104.stdout: # disable pg_autoscaler by default for new pools 2023-07-14T11:52:09.844 INFO:teuthology.orchestra.run.smithi104.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-14T11:52:09.844 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.844 INFO:teuthology.orchestra.run.smithi104.stdout: osd pool default size = 2 2023-07-14T11:52:09.844 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.844 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd allow primary affinity = true 2023-07-14T11:52:09.845 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd allow pg remap = true 2023-07-14T11:52:09.845 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on legacy crush tunables = false 2023-07-14T11:52:09.845 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on crush straw calc version zero = false 2023-07-14T11:52:09.845 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on no sortbitwise = false 2023-07-14T11:52:09.845 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on osd down out interval zero = false 2023-07-14T11:52:09.845 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on too few osds = false 2023-07-14T11:52:09.846 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-14T11:52:09.846 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-14T11:52:09.846 INFO:teuthology.orchestra.run.smithi104.stdout: mon_allow_pool_size_one = true 2023-07-14T11:52:09.846 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.848 INFO:teuthology.orchestra.run.smithi104.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-14T11:52:09.848 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.848 INFO:teuthology.orchestra.run.smithi104.stdout: osd default data pool replay window = 5 2023-07-14T11:52:09.848 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.848 INFO:teuthology.orchestra.run.smithi104.stdout: mon allow pool delete = true 2023-07-14T11:52:09.848 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.849 INFO:teuthology.orchestra.run.smithi104.stdout: mon cluster log file level = debug 2023-07-14T11:52:09.849 INFO:teuthology.orchestra.run.smithi104.stdout: debug asserts on shutdown = true 2023-07-14T11:52:09.849 INFO:teuthology.orchestra.run.smithi104.stdout: mon health detail to clog = false 2023-07-14T11:52:09.849 INFO:teuthology.orchestra.run.smithi104.stdout: mon host = "172.21.15.104,172.21.15.121,[v2:172.21.15.121:3301,v1:172.21.15.121:6790]" 2023-07-14T11:52:09.849 INFO:teuthology.orchestra.run.smithi104.stdout: fsid = c5576993-99e1-43b1-a253-659a3cee5212 2023-07-14T11:52:09.849 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.849 INFO:teuthology.orchestra.run.smithi104.stdout:[osd] 2023-07-14T11:52:09.849 INFO:teuthology.orchestra.run.smithi104.stdout: osd journal size = 100 2023-07-14T11:52:09.850 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.850 INFO:teuthology.orchestra.run.smithi104.stdout: osd scrub load threshold = 5.0 2023-07-14T11:52:09.850 INFO:teuthology.orchestra.run.smithi104.stdout: osd scrub max interval = 600 2023-07-14T11:52:09.850 INFO:teuthology.orchestra.run.smithi104.stdout: osd mclock profile = high_recovery_ops 2023-07-14T11:52:09.850 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.851 INFO:teuthology.orchestra.run.smithi104.stdout: osd recover clone overlap = true 2023-07-14T11:52:09.851 INFO:teuthology.orchestra.run.smithi104.stdout: osd recovery max chunk = 1048576 2023-07-14T11:52:09.851 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.851 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug shutdown = true 2023-07-14T11:52:09.852 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug op order = true 2023-07-14T11:52:09.852 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug verify stray on activate = true 2023-07-14T11:52:09.852 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.853 INFO:teuthology.orchestra.run.smithi104.stdout: osd open classes on start = true 2023-07-14T11:52:09.853 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug pg log writeout = true 2023-07-14T11:52:09.853 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.853 INFO:teuthology.orchestra.run.smithi104.stdout: osd deep scrub update digest min age = 30 2023-07-14T11:52:09.854 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.854 INFO:teuthology.orchestra.run.smithi104.stdout: osd map max advance = 10 2023-07-14T11:52:09.854 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.854 INFO:teuthology.orchestra.run.smithi104.stdout: journal zero on create = true 2023-07-14T11:52:09.855 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.855 INFO:teuthology.orchestra.run.smithi104.stdout: filestore ondisk finisher threads = 3 2023-07-14T11:52:09.855 INFO:teuthology.orchestra.run.smithi104.stdout: filestore apply finisher threads = 3 2023-07-14T11:52:09.856 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.856 INFO:teuthology.orchestra.run.smithi104.stdout: bdev debug aio = true 2023-07-14T11:52:09.856 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug misdirected ops = true 2023-07-14T11:52:09.856 INFO:teuthology.orchestra.run.smithi104.stdout: bluestore block size = 96636764160 2023-07-14T11:52:09.857 INFO:teuthology.orchestra.run.smithi104.stdout: bluestore fsck on mount = True 2023-07-14T11:52:09.857 INFO:teuthology.orchestra.run.smithi104.stdout: debug bluefs = 20 2023-07-14T11:52:09.857 INFO:teuthology.orchestra.run.smithi104.stdout: debug bluestore = 20 2023-07-14T11:52:09.857 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2023-07-14T11:52:09.857 INFO:teuthology.orchestra.run.smithi104.stdout: debug osd = 20 2023-07-14T11:52:09.857 INFO:teuthology.orchestra.run.smithi104.stdout: debug rocksdb = 10 2023-07-14T11:52:09.857 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd backfillfull_ratio = 0.85 2023-07-14T11:52:09.857 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd full ratio = 0.9 2023-07-14T11:52:09.858 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd nearfull ratio = 0.8 2023-07-14T11:52:09.858 INFO:teuthology.orchestra.run.smithi104.stdout: osd failsafe full ratio = 0.95 2023-07-14T11:52:09.858 INFO:teuthology.orchestra.run.smithi104.stdout: osd objectstore = bluestore 2023-07-14T11:52:09.858 INFO:teuthology.orchestra.run.smithi104.stdout: osd op complaint time = 180 2023-07-14T11:52:09.858 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.858 INFO:teuthology.orchestra.run.smithi104.stdout:[mgr] 2023-07-14T11:52:09.858 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2023-07-14T11:52:09.859 INFO:teuthology.orchestra.run.smithi104.stdout: debug mgr = 20 2023-07-14T11:52:09.859 INFO:teuthology.orchestra.run.smithi104.stdout: debug mon = 20 2023-07-14T11:52:09.859 INFO:teuthology.orchestra.run.smithi104.stdout: debug auth = 20 2023-07-14T11:52:09.859 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min pgs per osd = 4 2023-07-14T11:52:09.859 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min bytes per osd = 10 2023-07-14T11:52:09.859 INFO:teuthology.orchestra.run.smithi104.stdout: mgr/telemetry/nag = false 2023-07-14T11:52:09.859 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.859 INFO:teuthology.orchestra.run.smithi104.stdout:[mon] 2023-07-14T11:52:09.860 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2023-07-14T11:52:09.860 INFO:teuthology.orchestra.run.smithi104.stdout: debug mon = 20 2023-07-14T11:52:09.860 INFO:teuthology.orchestra.run.smithi104.stdout: debug paxos = 20 2023-07-14T11:52:09.860 INFO:teuthology.orchestra.run.smithi104.stdout: debug auth = 20 2023-07-14T11:52:09.860 INFO:teuthology.orchestra.run.smithi104.stdout: mon data avail warn = 5 2023-07-14T11:52:09.860 INFO:teuthology.orchestra.run.smithi104.stdout: mon mgr mkfs grace = 240 2023-07-14T11:52:09.860 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min pgs per osd = 4 2023-07-14T11:52:09.860 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd reporter subtree level = osd 2023-07-14T11:52:09.861 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd prime pg temp = true 2023-07-14T11:52:09.861 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min bytes per osd = 10 2023-07-14T11:52:09.861 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.861 INFO:teuthology.orchestra.run.smithi104.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-14T11:52:09.861 INFO:teuthology.orchestra.run.smithi104.stdout: auth mon ticket ttl = 660 # 11m 2023-07-14T11:52:09.861 INFO:teuthology.orchestra.run.smithi104.stdout: auth service ticket ttl = 240 # 4m 2023-07-14T11:52:09.861 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.861 INFO:teuthology.orchestra.run.smithi104.stdout: # don't complain about insecure global_id in the test suite 2023-07-14T11:52:09.862 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-14T11:52:09.862 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-14T11:52:09.862 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.862 INFO:teuthology.orchestra.run.smithi104.stdout: # 1m isn't quite enough 2023-07-14T11:52:09.862 INFO:teuthology.orchestra.run.smithi104.stdout: mon_down_mkfs_grace = 2m 2023-07-14T11:52:09.862 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.862 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_filestore_osds = false 2023-07-14T11:52:09.862 INFO:teuthology.orchestra.run.smithi104.stdout: mon op complaint time = 120 2023-07-14T11:52:09.863 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:09.863 INFO:teuthology.orchestra.run.smithi104.stdout:[client] 2023-07-14T11:52:09.863 INFO:teuthology.orchestra.run.smithi104.stdout: rgw cache enabled = true 2023-07-14T11:52:09.863 INFO:teuthology.orchestra.run.smithi104.stdout: rgw enable ops log = true 2023-07-14T11:52:09.863 INFO:teuthology.orchestra.run.smithi104.stdout: rgw enable usage log = true 2023-07-14T11:52:09.863 INFO:teuthology.orchestra.run.smithi104.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-14T11:52:09.863 INFO:teuthology.orchestra.run.smithi104.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-14T11:52:09.864 INFO:teuthology.orchestra.run.smithi104.stdout: client mount timeout = 600 2023-07-14T11:52:09.864 INFO:teuthology.orchestra.run.smithi104.stdout: debug client = 20 2023-07-14T11:52:09.864 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2023-07-14T11:52:09.864 INFO:teuthology.orchestra.run.smithi104.stdout: pid file = "" 2023-07-14T11:52:09.864 INFO:teuthology.orchestra.run.smithi104.stdout: rados mon op timeout = 900 2023-07-14T11:52:09.864 INFO:teuthology.orchestra.run.smithi104.stdout: rados osd op timeout = 900 2023-07-14T11:52:09.864 INFO:teuthology.orchestra.run.smithi104.stdout:[mon.a] 2023-07-14T11:52:09.865 INFO:teuthology.orchestra.run.smithi104.stdout:[mon.b] 2023-07-14T11:52:09.865 INFO:teuthology.orchestra.run.smithi104.stdout:[mon.c] 2023-07-14T11:52:09.865 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.a] 2023-07-14T11:52:09.865 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.c] 2023-07-14T11:52:09.865 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.b] 2023-07-14T11:52:09.865 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.d] 2023-07-14T11:52:09.865 INFO:teuthology.orchestra.run.smithi104.stdout:[mds] 2023-07-14T11:52:09.865 INFO:teuthology.orchestra.run.smithi104.stdout: debug mds = 20 2023-07-14T11:52:09.866 INFO:teuthology.orchestra.run.smithi104.stdout: debug mds balancer = 20 2023-07-14T11:52:09.866 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2023-07-14T11:52:09.866 INFO:teuthology.orchestra.run.smithi104.stdout: mds debug frag = True 2023-07-14T11:52:09.866 INFO:teuthology.orchestra.run.smithi104.stdout: mds debug scatterstat = True 2023-07-14T11:52:09.866 INFO:teuthology.orchestra.run.smithi104.stdout: mds op complaint time = 180 2023-07-14T11:52:09.866 INFO:teuthology.orchestra.run.smithi104.stdout: mds verify scatter = True 2023-07-14T11:52:09.866 INFO:teuthology.orchestra.run.smithi104.stdout: osd op complaint time = 180 2023-07-14T11:52:09.866 INFO:teuthology.orchestra.run.smithi104.stdout: rados mon op timeout = 900 2023-07-14T11:52:09.867 INFO:teuthology.orchestra.run.smithi104.stdout: rados osd op timeout = 900 2023-07-14T11:52:09.867 INFO:tasks.ceph:Creating admin key on mon.a... 2023-07-14T11:52:09.867 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-07-14T11:52:09.955 INFO:tasks.ceph:Copying monmap to all nodes... 2023-07-14T11:52:09.956 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:09.956 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-07-14T11:52:09.971 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:09.971 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-07-14T11:52:10.036 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:52:10.037 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:10.037 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-14T11:52:10.037 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-14T11:52:10.145 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:10.146 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-14T11:52:10.204 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:52:10.205 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:10.205 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-14T11:52:10.205 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-14T11:52:10.265 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:10.266 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-14T11:52:10.326 INFO:tasks.ceph:Setting up mon nodes... 2023-07-14T11:52:10.326 INFO:tasks.ceph:Setting up mgr nodes... 2023-07-14T11:52:10.327 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-07-14T11:52:10.428 INFO:teuthology.orchestra.run.smithi104.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-07-14T11:52:10.435 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-07-14T11:52:10.536 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-07-14T11:52:10.556 INFO:tasks.ceph:Setting up mds nodes... 2023-07-14T11:52:10.556 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-07-14T11:52:10.656 INFO:teuthology.orchestra.run.smithi104.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-07-14T11:52:10.662 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-07-14T11:52:10.694 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-07-14T11:52:10.841 INFO:teuthology.orchestra.run.smithi104.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-07-14T11:52:10.848 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-07-14T11:52:10.881 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-07-14T11:52:10.980 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-07-14T11:52:10.987 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-07-14T11:52:11.018 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-07-14T11:52:11.164 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-07-14T11:52:11.171 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-07-14T11:52:11.204 INFO:tasks.ceph_client:Setting up client nodes... 2023-07-14T11:52:11.205 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-14T11:52:11.282 INFO:teuthology.orchestra.run.smithi104.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-07-14T11:52:11.307 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-07-14T11:52:11.373 INFO:teuthology.orchestra.run.smithi104.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-07-14T11:52:11.400 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-07-14T11:52:11.477 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-07-14T11:52:11.504 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-07-14T11:52:11.578 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-07-14T11:52:11.606 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-07-14T11:52:11.606 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi104.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-14T11:52:11.606 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-07-14T11:52:11.641 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-14T11:52:11.641 INFO:tasks.ceph:role: osd.0 2023-07-14T11:52:11.641 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:52:11.642 DEBUG:teuthology.orchestra.run.smithi104:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-14T11:52:12.244 INFO:teuthology.orchestra.run.smithi104.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-14T11:52:12.245 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-14T11:52:12.245 INFO:teuthology.orchestra.run.smithi104.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-14T11:52:12.245 INFO:teuthology.orchestra.run.smithi104.stdout: = reflink=1 2023-07-14T11:52:12.245 INFO:teuthology.orchestra.run.smithi104.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-14T11:52:12.245 INFO:teuthology.orchestra.run.smithi104.stdout: = sunit=0 swidth=0 blks 2023-07-14T11:52:12.245 INFO:teuthology.orchestra.run.smithi104.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-14T11:52:12.245 INFO:teuthology.orchestra.run.smithi104.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-14T11:52:12.246 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-14T11:52:12.246 INFO:teuthology.orchestra.run.smithi104.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-14T11:52:12.249 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi104.front.sepia.ceph.com -o noatime 2023-07-14T11:52:12.249 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-07-14T11:52:12.389 DEBUG:teuthology.orchestra.run.smithi104:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-07-14T11:52:12.408 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: /sbin/restorecon: command not found 2023-07-14T11:52:12.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:52:12.410 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-07-14T11:52:12.485 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-14T11:52:12.486 INFO:tasks.ceph:role: osd.1 2023-07-14T11:52:12.486 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:52:12.486 DEBUG:teuthology.orchestra.run.smithi104:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-14T11:52:13.011 INFO:teuthology.orchestra.run.smithi104.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-14T11:52:13.011 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-14T11:52:13.011 INFO:teuthology.orchestra.run.smithi104.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-14T11:52:13.012 INFO:teuthology.orchestra.run.smithi104.stdout: = reflink=1 2023-07-14T11:52:13.012 INFO:teuthology.orchestra.run.smithi104.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-14T11:52:13.012 INFO:teuthology.orchestra.run.smithi104.stdout: = sunit=0 swidth=0 blks 2023-07-14T11:52:13.012 INFO:teuthology.orchestra.run.smithi104.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-14T11:52:13.012 INFO:teuthology.orchestra.run.smithi104.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-14T11:52:13.012 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-14T11:52:13.012 INFO:teuthology.orchestra.run.smithi104.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-14T11:52:13.016 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi104.front.sepia.ceph.com -o noatime 2023-07-14T11:52:13.016 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-07-14T11:52:13.086 DEBUG:teuthology.orchestra.run.smithi104:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-07-14T11:52:13.107 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: /sbin/restorecon: command not found 2023-07-14T11:52:13.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:52:13.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-07-14T11:52:13.183 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-14T11:52:13.183 INFO:tasks.ceph:role: osd.2 2023-07-14T11:52:13.183 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:52:13.184 DEBUG:teuthology.orchestra.run.smithi104:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-14T11:52:13.758 INFO:teuthology.orchestra.run.smithi104.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-14T11:52:13.759 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-14T11:52:13.759 INFO:teuthology.orchestra.run.smithi104.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-14T11:52:13.759 INFO:teuthology.orchestra.run.smithi104.stdout: = reflink=1 2023-07-14T11:52:13.759 INFO:teuthology.orchestra.run.smithi104.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-14T11:52:13.759 INFO:teuthology.orchestra.run.smithi104.stdout: = sunit=0 swidth=0 blks 2023-07-14T11:52:13.759 INFO:teuthology.orchestra.run.smithi104.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-14T11:52:13.760 INFO:teuthology.orchestra.run.smithi104.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-14T11:52:13.760 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-14T11:52:13.760 INFO:teuthology.orchestra.run.smithi104.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-14T11:52:13.762 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi104.front.sepia.ceph.com -o noatime 2023-07-14T11:52:13.762 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-07-14T11:52:13.903 DEBUG:teuthology.orchestra.run.smithi104:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-07-14T11:52:13.921 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: /sbin/restorecon: command not found 2023-07-14T11:52:13.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:52:13.924 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-07-14T11:52:14.005 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-14T11:52:14.005 INFO:tasks.ceph:role: osd.3 2023-07-14T11:52:14.006 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:52:14.006 DEBUG:teuthology.orchestra.run.smithi104:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-14T11:52:14.567 INFO:teuthology.orchestra.run.smithi104.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-14T11:52:14.568 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-14T11:52:14.568 INFO:teuthology.orchestra.run.smithi104.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-14T11:52:14.568 INFO:teuthology.orchestra.run.smithi104.stdout: = reflink=1 2023-07-14T11:52:14.568 INFO:teuthology.orchestra.run.smithi104.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-14T11:52:14.568 INFO:teuthology.orchestra.run.smithi104.stdout: = sunit=0 swidth=0 blks 2023-07-14T11:52:14.568 INFO:teuthology.orchestra.run.smithi104.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-14T11:52:14.568 INFO:teuthology.orchestra.run.smithi104.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-14T11:52:14.569 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-14T11:52:14.569 INFO:teuthology.orchestra.run.smithi104.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-14T11:52:14.572 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi104.front.sepia.ceph.com -o noatime 2023-07-14T11:52:14.573 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-07-14T11:52:14.718 DEBUG:teuthology.orchestra.run.smithi104:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-07-14T11:52:14.736 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: /sbin/restorecon: command not found 2023-07-14T11:52:14.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:52:14.738 DEBUG:teuthology.orchestra.run.smithi104:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-14T11:52:14.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:14.914+0000 7fab55ea8700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-07-14T11:52:14.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:14.918+0000 7fab55ea8700 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-07-14T11:52:14.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:14.918+0000 7fab55ea8700 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-14T11:52:14.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:14.918+0000 7fab55ea8700 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-14T11:52:14.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:14.918+0000 7fab55ea8700 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-14T11:52:14.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:14.922+0000 7fab55ea8700 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-07-14T11:52:17.972 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-07-14T11:52:18.008 DEBUG:teuthology.orchestra.run.smithi104:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-14T11:52:18.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:18.150+0000 7fb88e358700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-07-14T11:52:18.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:18.154+0000 7fb88e358700 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-07-14T11:52:18.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:18.154+0000 7fb88e358700 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-14T11:52:18.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:18.154+0000 7fb88e358700 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-14T11:52:18.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:18.154+0000 7fb88e358700 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-14T11:52:18.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:18.154+0000 7fb88e358700 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-07-14T11:52:21.196 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-07-14T11:52:21.229 DEBUG:teuthology.orchestra.run.smithi104:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-14T11:52:21.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:21.374+0000 7f9f41b9c700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-07-14T11:52:21.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:21.374+0000 7f9f41b9c700 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-07-14T11:52:21.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:21.374+0000 7f9f41b9c700 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-14T11:52:21.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:21.374+0000 7f9f41b9c700 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-14T11:52:21.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:21.374+0000 7f9f41b9c700 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-14T11:52:21.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:21.374+0000 7f9f41b9c700 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-07-14T11:52:24.442 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-07-14T11:52:24.472 DEBUG:teuthology.orchestra.run.smithi104:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-14T11:52:24.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:24.618+0000 7f8cfd7b6700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-07-14T11:52:24.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:24.618+0000 7f8cfd7b6700 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-07-14T11:52:24.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:24.618+0000 7f8cfd7b6700 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-14T11:52:24.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:24.618+0000 7f8cfd7b6700 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-14T11:52:24.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:24.618+0000 7f8cfd7b6700 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-14T11:52:24.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:24.618+0000 7f8cfd7b6700 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-07-14T11:52:27.668 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-07-14T11:52:27.701 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-07-14T11:52:27.734 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-14T11:52:27.734 INFO:tasks.ceph:role: osd.4 2023-07-14T11:52:27.734 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:52:27.735 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-14T11:52:28.339 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-14T11:52:28.340 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-14T11:52:28.340 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-14T11:52:28.340 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 2023-07-14T11:52:28.341 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-14T11:52:28.341 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-07-14T11:52:28.341 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-14T11:52:28.341 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-14T11:52:28.342 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-14T11:52:28.342 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-14T11:52:28.345 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-07-14T11:52:28.345 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-07-14T11:52:28.480 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-07-14T11:52:28.499 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: /sbin/restorecon: command not found 2023-07-14T11:52:28.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:52:28.501 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-07-14T11:52:28.579 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-14T11:52:28.580 INFO:tasks.ceph:role: osd.5 2023-07-14T11:52:28.580 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:52:28.580 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-14T11:52:29.094 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-14T11:52:29.095 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-14T11:52:29.095 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-14T11:52:29.095 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 2023-07-14T11:52:29.095 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-14T11:52:29.096 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-07-14T11:52:29.096 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-14T11:52:29.096 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-14T11:52:29.097 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-14T11:52:29.097 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-14T11:52:29.098 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-07-14T11:52:29.099 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-07-14T11:52:29.173 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-07-14T11:52:29.191 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: /sbin/restorecon: command not found 2023-07-14T11:52:29.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:52:29.193 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-07-14T11:52:29.271 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-14T11:52:29.271 INFO:tasks.ceph:role: osd.6 2023-07-14T11:52:29.271 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:52:29.271 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-14T11:52:29.850 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-14T11:52:29.851 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-14T11:52:29.851 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-14T11:52:29.851 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 2023-07-14T11:52:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-14T11:52:29.852 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-07-14T11:52:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-14T11:52:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-14T11:52:29.853 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-14T11:52:29.853 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-14T11:52:29.855 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-07-14T11:52:29.855 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-07-14T11:52:29.991 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-07-14T11:52:30.010 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: /sbin/restorecon: command not found 2023-07-14T11:52:30.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:52:30.012 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-07-14T11:52:30.086 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-14T11:52:30.087 INFO:tasks.ceph:role: osd.7 2023-07-14T11:52:30.087 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:52:30.087 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-14T11:52:30.671 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-14T11:52:30.671 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-14T11:52:30.671 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-14T11:52:30.671 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 2023-07-14T11:52:30.672 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-14T11:52:30.672 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-07-14T11:52:30.672 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-14T11:52:30.672 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-14T11:52:30.672 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-14T11:52:30.672 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-14T11:52:30.676 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-07-14T11:52:30.676 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-07-14T11:52:30.786 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-07-14T11:52:30.805 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: /sbin/restorecon: command not found 2023-07-14T11:52:30.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:52:30.807 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-14T11:52:30.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:30.977+0000 7f8f1555f700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-07-14T11:52:30.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:30.981+0000 7f8f1555f700 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-07-14T11:52:30.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:30.981+0000 7f8f1555f700 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-14T11:52:30.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:30.981+0000 7f8f1555f700 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-14T11:52:30.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:30.981+0000 7f8f1555f700 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-14T11:52:30.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:30.985+0000 7f8f1555f700 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-07-14T11:52:34.029 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-07-14T11:52:34.061 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-14T11:52:34.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:34.205+0000 7f750e944700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-07-14T11:52:34.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:34.205+0000 7f750e944700 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-07-14T11:52:34.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:34.205+0000 7f750e944700 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-14T11:52:34.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:34.205+0000 7f750e944700 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-14T11:52:34.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:34.205+0000 7f750e944700 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-14T11:52:34.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:34.209+0000 7f750e944700 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-07-14T11:52:37.245 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-07-14T11:52:37.275 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-14T11:52:37.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:37.417+0000 7fccaa9d3700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-07-14T11:52:37.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:37.417+0000 7fccaa9d3700 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-07-14T11:52:37.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:37.417+0000 7fccaa9d3700 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-14T11:52:37.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:37.417+0000 7fccaa9d3700 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-14T11:52:37.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:37.417+0000 7fccaa9d3700 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-14T11:52:37.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:37.421+0000 7fccaa9d3700 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-07-14T11:52:37.454 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:52:37.461 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:52:40.463 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-07-14T11:52:40.495 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-14T11:52:40.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:40.633+0000 7f1c0b3fa700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-07-14T11:52:40.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:40.637+0000 7f1c0b3fa700 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-07-14T11:52:40.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:40.637+0000 7f1c0b3fa700 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-14T11:52:40.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:40.637+0000 7f1c0b3fa700 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-14T11:52:40.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:40.637+0000 7f1c0b3fa700 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-14T11:52:40.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:40.637+0000 7f1c0b3fa700 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-07-14T11:52:43.686 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-07-14T11:52:43.718 INFO:tasks.ceph:Reading keys from all nodes... 2023-07-14T11:52:43.719 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:43.719 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-07-14T11:52:43.750 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:43.750 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-07-14T11:52:43.827 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:43.827 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-07-14T11:52:43.902 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:43.903 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-07-14T11:52:43.978 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:43.979 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-07-14T11:52:44.054 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:44.055 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-07-14T11:52:44.130 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:44.131 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-07-14T11:52:44.211 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:44.211 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-07-14T11:52:44.245 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:44.245 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-07-14T11:52:44.324 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:44.325 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-07-14T11:52:44.405 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:44.405 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-07-14T11:52:44.480 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:44.481 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-07-14T11:52:44.561 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:44.561 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-07-14T11:52:44.636 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:44.637 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-07-14T11:52:44.712 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:44.713 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-07-14T11:52:44.732 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:44.732 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-07-14T11:52:44.792 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:44.792 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-07-14T11:52:44.810 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:44.810 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-07-14T11:52:44.869 INFO:tasks.ceph:Adding keys to all mons... 2023-07-14T11:52:44.870 DEBUG:teuthology.orchestra.run.smithi104:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-14T11:52:44.874 DEBUG:teuthology.orchestra.run.smithi121:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-14T11:52:44.919 INFO:teuthology.orchestra.run.smithi104.stdout:[mgr.y] 2023-07-14T11:52:44.919 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDqNrFkmLyZGhAAe2/pYD9LAp8Y6JR+IxqDuA== 2023-07-14T11:52:44.919 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.a] 2023-07-14T11:52:44.919 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDqNrFkusAwKBAAd0vgoEvYzWoPG+qOPSx7xg== 2023-07-14T11:52:44.919 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.c] 2023-07-14T11:52:44.920 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDqNrFkRtpAMxAAq3jyn4VjOGmZrpCJVy6BYw== 2023-07-14T11:52:44.920 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.0] 2023-07-14T11:52:44.920 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDuNrFkvPfmNhAABqDrwKAMelrRtze4ZCW4fw== 2023-07-14T11:52:44.920 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.1] 2023-07-14T11:52:44.920 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDyNrFkNslaCRAAntw0JcsmGTwVodNy8b7LeQ== 2023-07-14T11:52:44.920 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.2] 2023-07-14T11:52:44.920 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQD1NrFkTkx9FhAAoxh/ll+DixRrVsqpDPdKZA== 2023-07-14T11:52:44.921 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.3] 2023-07-14T11:52:44.921 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQD4NrFkiGAHJRAA0o2NVYbQ5uGbsLaZmojHBA== 2023-07-14T11:52:44.921 INFO:teuthology.orchestra.run.smithi104.stdout:[mgr.x] 2023-07-14T11:52:44.921 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDqNrFkqhD3IBAAE14Ks6MpY1PyMfpbQcUgjQ== 2023-07-14T11:52:44.921 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.b] 2023-07-14T11:52:44.921 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDqNrFkjoJoOxAA4nNPKT2i5JIO7aF42mYIzg== 2023-07-14T11:52:44.921 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.d] 2023-07-14T11:52:44.921 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDrNrFkADXNChAA5ndYnLdDxZQwMJBBT4XBtQ== 2023-07-14T11:52:44.922 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.4] 2023-07-14T11:52:44.922 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQD+NrFkZ+13OhAAKNDNbO4EgSTYmT1sujX2TQ== 2023-07-14T11:52:44.922 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.5] 2023-07-14T11:52:44.922 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQACN7FkY69aDBAA6qbtQcxmW84CHeizN6bCvw== 2023-07-14T11:52:44.922 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.6] 2023-07-14T11:52:44.922 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQAFN7Fk6roFGRAA2A0Zt8ivszAYsaGf0wQH0Q== 2023-07-14T11:52:44.922 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.7] 2023-07-14T11:52:44.922 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQAIN7FkEVH+JRAAc5EcTI6vOzyNM1iXiESbjQ== 2023-07-14T11:52:44.923 INFO:teuthology.orchestra.run.smithi104.stdout:[client.0] 2023-07-14T11:52:44.923 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDrNrFkbrzjERAA+9BBzuauwUdhALUcDtd80A== 2023-07-14T11:52:44.923 INFO:teuthology.orchestra.run.smithi104.stdout:[client.2] 2023-07-14T11:52:44.923 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDrNrFkNmBZFxAA96kq+aaMbx6FQIJGUdN6xQ== 2023-07-14T11:52:44.923 INFO:teuthology.orchestra.run.smithi104.stdout:[client.1] 2023-07-14T11:52:44.923 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDrNrFk9gt2HRAAZ0izobs6vpgfpG6isDFRbQ== 2023-07-14T11:52:44.923 INFO:teuthology.orchestra.run.smithi104.stdout:[client.3] 2023-07-14T11:52:44.924 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDrNrFkiTBxIxAAfEZF/Xe7Kc/QZHkBmHFShg== 2023-07-14T11:52:44.941 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.y] 2023-07-14T11:52:44.942 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDqNrFkmLyZGhAAe2/pYD9LAp8Y6JR+IxqDuA== 2023-07-14T11:52:44.942 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.a] 2023-07-14T11:52:44.942 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDqNrFkusAwKBAAd0vgoEvYzWoPG+qOPSx7xg== 2023-07-14T11:52:44.942 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.c] 2023-07-14T11:52:44.942 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDqNrFkRtpAMxAAq3jyn4VjOGmZrpCJVy6BYw== 2023-07-14T11:52:44.942 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.0] 2023-07-14T11:52:44.942 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDuNrFkvPfmNhAABqDrwKAMelrRtze4ZCW4fw== 2023-07-14T11:52:44.943 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.1] 2023-07-14T11:52:44.943 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDyNrFkNslaCRAAntw0JcsmGTwVodNy8b7LeQ== 2023-07-14T11:52:44.943 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.2] 2023-07-14T11:52:44.943 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQD1NrFkTkx9FhAAoxh/ll+DixRrVsqpDPdKZA== 2023-07-14T11:52:44.943 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.3] 2023-07-14T11:52:44.943 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQD4NrFkiGAHJRAA0o2NVYbQ5uGbsLaZmojHBA== 2023-07-14T11:52:44.943 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.x] 2023-07-14T11:52:44.944 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDqNrFkqhD3IBAAE14Ks6MpY1PyMfpbQcUgjQ== 2023-07-14T11:52:44.944 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.b] 2023-07-14T11:52:44.944 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDqNrFkjoJoOxAA4nNPKT2i5JIO7aF42mYIzg== 2023-07-14T11:52:44.944 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.d] 2023-07-14T11:52:44.944 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDrNrFkADXNChAA5ndYnLdDxZQwMJBBT4XBtQ== 2023-07-14T11:52:44.944 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.4] 2023-07-14T11:52:44.944 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQD+NrFkZ+13OhAAKNDNbO4EgSTYmT1sujX2TQ== 2023-07-14T11:52:44.944 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.5] 2023-07-14T11:52:44.945 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQACN7FkY69aDBAA6qbtQcxmW84CHeizN6bCvw== 2023-07-14T11:52:44.945 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.6] 2023-07-14T11:52:44.945 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAFN7Fk6roFGRAA2A0Zt8ivszAYsaGf0wQH0Q== 2023-07-14T11:52:44.945 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.7] 2023-07-14T11:52:44.945 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAIN7FkEVH+JRAAc5EcTI6vOzyNM1iXiESbjQ== 2023-07-14T11:52:44.945 INFO:teuthology.orchestra.run.smithi121.stdout:[client.0] 2023-07-14T11:52:44.945 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDrNrFkbrzjERAA+9BBzuauwUdhALUcDtd80A== 2023-07-14T11:52:44.945 INFO:teuthology.orchestra.run.smithi121.stdout:[client.2] 2023-07-14T11:52:44.946 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDrNrFkNmBZFxAA96kq+aaMbx6FQIJGUdN6xQ== 2023-07-14T11:52:44.946 INFO:teuthology.orchestra.run.smithi121.stdout:[client.1] 2023-07-14T11:52:44.946 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDrNrFk9gt2HRAAZ0izobs6vpgfpG6isDFRbQ== 2023-07-14T11:52:44.946 INFO:teuthology.orchestra.run.smithi121.stdout:[client.3] 2023-07-14T11:52:44.946 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDrNrFkiTBxIxAAfEZF/Xe7Kc/QZHkBmHFShg== 2023-07-14T11:52:44.947 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-14T11:52:44.972 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-14T11:52:45.072 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-14T11:52:45.080 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-14T11:52:45.165 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-14T11:52:45.171 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-14T11:52:45.257 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:45.261 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:45.339 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:45.344 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:45.422 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:45.428 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:45.508 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:45.515 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:45.597 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-14T11:52:45.603 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-14T11:52:45.684 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-14T11:52:45.688 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-14T11:52:45.771 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-14T11:52:45.778 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-14T11:52:45.859 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:45.866 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:45.946 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:45.953 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:46.037 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:46.043 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:46.123 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:46.130 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-14T11:52:46.214 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-14T11:52:46.220 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-14T11:52:46.305 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-14T11:52:46.311 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-14T11:52:46.401 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-14T11:52:46.408 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-14T11:52:46.492 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-14T11:52:46.498 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-14T11:52:46.584 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-07-14T11:52:46.584 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-07-14T11:52:46.618 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-14T11:52:46.763 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-07-14T11:52:46.798 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-07-14T11:52:46.835 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-14T11:52:46.982 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-07-14T11:52:47.015 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-07-14T11:52:47.090 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-14T11:52:47.238 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-07-14T11:52:47.271 DEBUG:teuthology.orchestra.run.smithi104:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-14T11:52:47.278 DEBUG:teuthology.orchestra.run.smithi121:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-14T11:52:47.332 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-07-14T11:52:47.332 INFO:tasks.ceph.mon.a:Restarting daemon 2023-07-14T11:52:47.333 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-07-14T11:52:47.340 INFO:tasks.ceph.mon.a:Started 2023-07-14T11:52:47.340 INFO:tasks.ceph.mon.b:Restarting daemon 2023-07-14T11:52:47.340 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-07-14T11:52:47.382 INFO:tasks.ceph.mon.b:Started 2023-07-14T11:52:47.383 INFO:tasks.ceph.mon.c:Restarting daemon 2023-07-14T11:52:47.383 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-07-14T11:52:47.387 INFO:tasks.ceph.mon.c:Started 2023-07-14T11:52:47.387 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-07-14T11:52:47.388 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-07-14T11:52:47.388 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-07-14T11:52:47.396 INFO:tasks.ceph.mgr.y:Started 2023-07-14T11:52:47.396 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-07-14T11:52:47.396 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-07-14T11:52:47.402 INFO:tasks.ceph.mgr.x:Started 2023-07-14T11:52:47.402 INFO:tasks.ceph:Setting crush tunables to default 2023-07-14T11:52:47.403 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd crush tunables default 2023-07-14T11:52:47.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.574+0000 7f1c30d54700 1 Processor -- start 2023-07-14T11:52:47.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.574+0000 7f1c30d54700 1 -- start start 2023-07-14T11:52:47.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c30d54700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c2c0d0210 0x7f1c2c0d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c30d54700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1c2c0cf000 0x7f1c2c0cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c30d54700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2c071d10 con 0x7f1c2c0b1c50 2023-07-14T11:52:47.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c30d54700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c2c0cf9c0 con 0x7f1c2c0cf000 2023-07-14T11:52:47.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c30d54700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c15c420 con 0x7f1c2c0d0210 2023-07-14T11:52:47.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c29d9b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c2c0d0210 0x7f1c2c0d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c29d9b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c2c0d0210 0x7f1c2c0d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:54394/0 (socket says 172.21.15.104:54394) 2023-07-14T11:52:47.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c2ad9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1c2c0cf000 0x7f1c2c0cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c29d9b700 1 -- 172.21.15.104:0/3604836096 learned_addr learned my addr 172.21.15.104:0/3604836096 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:52:47.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c29d9b700 1 -- 172.21.15.104:0/3604836096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c2c0d0210 msgr2=0x7f1c2c0d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-07-14T11:52:47.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c29d9b700 1 -- 172.21.15.104:0/3604836096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c2c0d0210 msgr2=0x7f1c2c0d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-14T11:52:47.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c29d9b700 1 --2- 172.21.15.104:0/3604836096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c2c0d0210 0x7f1c2c0d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-14T11:52:47.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c29d9b700 1 --2- 172.21.15.104:0/3604836096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c2c0d0210 0x7f1c2c0d0640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:52:47.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c2ad9d700 1 -- 172.21.15.104:0/3604836096 >> v2:172.21.15.121:3300/0 conn(0x7f1c2c0cf000 msgr2=0x7f1c2c0cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-07-14T11:52:47.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c2ad9d700 1 -- 172.21.15.104:0/3604836096 >> v2:172.21.15.121:3300/0 conn(0x7f1c2c0cf000 msgr2=0x7f1c2c0cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-14T11:52:47.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c2ad9d700 1 --2- 172.21.15.104:0/3604836096 >> v2:172.21.15.121:3300/0 conn(0x7f1c2c0cf000 0x7f1c2c0cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-14T11:52:47.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.578+0000 7f1c2ad9d700 1 --2- 172.21.15.104:0/3604836096 >> v2:172.21.15.121:3300/0 conn(0x7f1c2c0cf000 0x7f1c2c0cf480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:52:47.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 -- 172.21.15.104:0/3604836096 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091587794 0 0) 0x7f1c1c001110 con 0x7f1c2c0b1c50 2023-07-14T11:52:47.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 -- 172.21.15.104:0/3604836096 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c140035d0 con 0x7f1c2c0b1c50 2023-07-14T11:52:47.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 -- 172.21.15.104:0/3604836096 <== mon.1 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2550033993 0 0) 0x7f1c140035d0 con 0x7f1c2c0b1c50 2023-07-14T11:52:47.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 -- 172.21.15.104:0/3604836096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c2c0d0210 msgr2=0x7f1c2c0d0640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:52:47.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 --2- 172.21.15.104:0/3604836096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c2c0d0210 0x7f1c2c0d0640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 -- 172.21.15.104:0/3604836096 >> v2:172.21.15.121:3300/0 conn(0x7f1c2c0cf000 msgr2=0x7f1c2c0cf480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:52:47.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 --2- 172.21.15.104:0/3604836096 >> v2:172.21.15.121:3300/0 conn(0x7f1c2c0cf000 0x7f1c2c0cf480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 -- 172.21.15.104:0/3604836096 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c070840 con 0x7f1c2c0b1c50 2023-07-14T11:52:47.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 -- 172.21.15.104:0/3604836096 <== mon.1 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f1c1c001800 con 0x7f1c2c0b1c50 2023-07-14T11:52:47.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 -- 172.21.15.104:0/3604836096 >> v1:172.21.15.121:6789/0 conn(0x7f1c2c0b1c50 legacy=0x7f1c2c071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:47.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 --2- 172.21.15.104:0/3604836096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c14003a20 0x7f1c14003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 --2- 172.21.15.104:0/3604836096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c14004870 0x7f1c14004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 --2- 172.21.15.104:0/3604836096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c2c0d0210 0x7f1c14005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 -- 172.21.15.104:0/3604836096 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c0cf9c0 con 0x7f1c14003a20 2023-07-14T11:52:47.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 -- 172.21.15.104:0/3604836096 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c15c420 con 0x7f1c14004870 2023-07-14T11:52:47.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c2959a700 1 -- 172.21.15.104:0/3604836096 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1400bb60 con 0x7f1c2c0d0210 2023-07-14T11:52:47.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c29d9b700 1 --2- 172.21.15.104:0/3604836096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c14004870 0x7f1c14004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c29d9b700 1 -- 172.21.15.104:0/3604836096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c2c0d0210 msgr2=0x7f1c14005220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T11:52:47.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c29d9b700 1 --2- 172.21.15.104:0/3604836096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c2c0d0210 0x7f1c14005220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c29d9b700 1 -- 172.21.15.104:0/3604836096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c14003a20 msgr2=0x7f1c14003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:47.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c29d9b700 1 --2- 172.21.15.104:0/3604836096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c14003a20 0x7f1c14003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c29d9b700 1 -- 172.21.15.104:0/3604836096 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c140035d0 con 0x7f1c14004870 2023-07-14T11:52:47.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.766+0000 7f1c29d9b700 1 --2- 172.21.15.104:0/3604836096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c14004870 0x7f1c14004ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009980 tx=0x7f1c140058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce163b1ef2b157d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:47.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.770+0000 7f1c2959a700 1 -- 172.21.15.104:0/3604836096 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2000c070 con 0x7f1c14004870 2023-07-14T11:52:47.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.770+0000 7f1c2959a700 1 -- 172.21.15.104:0/3604836096 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c20004020 con 0x7f1c14004870 2023-07-14T11:52:47.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.770+0000 7f1c2959a700 1 -- 172.21.15.104:0/3604836096 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c200043d0 con 0x7f1c14004870 2023-07-14T11:52:47.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.770+0000 7f1c30d54700 1 -- 172.21.15.104:0/3604836096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c14004870 msgr2=0x7f1c14004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:47.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.770+0000 7f1c30d54700 1 --2- 172.21.15.104:0/3604836096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c14004870 0x7f1c14004ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009980 tx=0x7f1c140058a0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.770+0000 7f1c2ad9d700 1 -- 172.21.15.104:0/3604836096 reap_dead start 2023-07-14T11:52:47.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.770+0000 7f1c30d54700 1 -- 172.21.15.104:0/3604836096 shutdown_connections 2023-07-14T11:52:47.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.770+0000 7f1c30d54700 1 -- 172.21.15.104:0/3604836096 >> 172.21.15.104:0/3604836096 conn(0x7f1c2c09b9c0 msgr2=0x7f1c2c0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:47.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.770+0000 7f1c30d54700 1 -- 172.21.15.104:0/3604836096 shutdown_connections 2023-07-14T11:52:47.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.770+0000 7f1c30d54700 1 -- 172.21.15.104:0/3604836096 wait complete. 2023-07-14T11:52:47.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.774+0000 7f1c30d54700 1 Processor -- start 2023-07-14T11:52:47.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.774+0000 7f1c30d54700 1 -- start start 2023-07-14T11:52:47.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c30d54700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c14004870 0x7f1c2c155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c30d54700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c2c1574f0 0x7f1c2c0b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c30d54700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c2c0b6460 0x7f1c2c15e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:47.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c30d54700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c0b68d0 con 0x7f1c2c0b6460 2023-07-14T11:52:47.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c30d54700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c156450 con 0x7f1c2c1574f0 2023-07-14T11:52:47.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c30d54700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c156730 con 0x7f1c14004870 2023-07-14T11:52:47.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c29d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c2c1574f0 0x7f1c2c0b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c29d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c2c1574f0 0x7f1c2c0b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:42334/0 (socket says 172.21.15.104:42334) 2023-07-14T11:52:47.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c29d9b700 1 -- 172.21.15.104:0/1575228976 learned_addr learned my addr 172.21.15.104:0/1575228976 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:52:47.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c2a59c700 1 --2- 172.21.15.104:0/1575228976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c14004870 0x7f1c2c155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c2ad9d700 1 --2- 172.21.15.104:0/1575228976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c2c0b6460 0x7f1c2c15e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:47.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c29d9b700 1 -- 172.21.15.104:0/1575228976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c14004870 msgr2=0x7f1c2c155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:47.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c29d9b700 1 --2- 172.21.15.104:0/1575228976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c14004870 0x7f1c2c155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c29d9b700 1 -- 172.21.15.104:0/1575228976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c2c0b6460 msgr2=0x7f1c2c15e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:47.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c29d9b700 1 --2- 172.21.15.104:0/1575228976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c2c0b6460 0x7f1c2c15e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c29d9b700 1 -- 172.21.15.104:0/1575228976 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c156a10 con 0x7f1c2c1574f0 2023-07-14T11:52:47.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c29d9b700 1 --2- 172.21.15.104:0/1575228976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c2c1574f0 0x7f1c2c0b5f20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009f70 tx=0x7f1c200047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:47.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c1affd700 1 -- 172.21.15.104:0/1575228976 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c200045c0 con 0x7f1c2c1574f0 2023-07-14T11:52:47.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c1affd700 1 -- 172.21.15.104:0/1575228976 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c20003de0 con 0x7f1c2c1574f0 2023-07-14T11:52:47.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c30d54700 1 -- 172.21.15.104:0/1575228976 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2c156cf0 con 0x7f1c2c1574f0 2023-07-14T11:52:47.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c1affd700 1 -- 172.21.15.104:0/1575228976 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20026680 con 0x7f1c2c1574f0 2023-07-14T11:52:47.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c30d54700 1 -- 172.21.15.104:0/1575228976 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2c157960 con 0x7f1c2c1574f0 2023-07-14T11:52:47.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c1affd700 1 -- 172.21.15.104:0/1575228976 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1c20026ae0 con 0x7f1c2c1574f0 2023-07-14T11:52:47.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.778+0000 7f1c1affd700 1 -- 172.21.15.104:0/1575228976 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1c2002fcd0 con 0x7f1c2c1574f0 2023-07-14T11:52:47.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.782+0000 7f1c30d54700 1 -- 172.21.15.104:0/1575228976 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2c0b08d0 con 0x7f1c2c1574f0 2023-07-14T11:52:47.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.786+0000 7f1c1affd700 1 -- 172.21.15.104:0/1575228976 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f1c20017030 con 0x7f1c2c1574f0 2023-07-14T11:52:47.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.858+0000 7f1c30d54700 1 -- 172.21.15.104:0/1575228976 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1c2c0d1d90 con 0x7f1c2c1574f0 2023-07-14T11:52:47.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.870+0000 7f1c1affd700 1 -- 172.21.15.104:0/1575228976 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1c2000f040 con 0x7f1c2c1574f0 2023-07-14T11:52:47.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.870+0000 7f1c30d54700 1 -- 172.21.15.104:0/1575228976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c2c1574f0 msgr2=0x7f1c2c0b5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:47.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.870+0000 7f1c30d54700 1 --2- 172.21.15.104:0/1575228976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c2c1574f0 0x7f1c2c0b5f20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009f70 tx=0x7f1c200047e0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.870+0000 7f1c30d54700 1 -- 172.21.15.104:0/1575228976 shutdown_connections 2023-07-14T11:52:47.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.870+0000 7f1c30d54700 1 --2- 172.21.15.104:0/1575228976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c2c0b6460 0x7f1c2c15e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.870+0000 7f1c30d54700 1 --2- 172.21.15.104:0/1575228976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c2c1574f0 0x7f1c2c0b5f20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.870+0000 7f1c30d54700 1 --2- 172.21.15.104:0/1575228976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c14004870 0x7f1c2c155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:47.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.870+0000 7f1c30d54700 1 -- 172.21.15.104:0/1575228976 >> 172.21.15.104:0/1575228976 conn(0x7f1c2c09b9c0 msgr2=0x7f1c2c0b6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:47.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.870+0000 7f1c30d54700 1 -- 172.21.15.104:0/1575228976 shutdown_connections 2023-07-14T11:52:47.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:47.870+0000 7f1c30d54700 1 -- 172.21.15.104:0/1575228976 wait complete. 2023-07-14T11:52:47.860 INFO:teuthology.orchestra.run.smithi104.stderr:adjusted tunables profile to default 2023-07-14T11:52:47.880 INFO:tasks.ceph:check_enable_crimson: False 2023-07-14T11:52:47.880 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-07-14T11:52:47.881 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:47.881 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-07-14T11:52:47.914 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:47.914 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-07-14T11:52:47.988 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:47.988 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-07-14T11:52:47.990 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:48.005+0000 7fe50f93f2c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T11:52:48.022 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:48.034+0000 7f4cc2ed62c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T11:52:48.053 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:52:48.053 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-07-14T11:52:48.130 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:48.131 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-07-14T11:52:48.163 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:48.164 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-07-14T11:52:48.237 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:48.238 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-07-14T11:52:48.311 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:52:48.312 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-07-14T11:52:48.384 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 0e83f7df-2e45-460d-a746-d48694cee08b 0 2023-07-14T11:52:48.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.605+0000 7fc529d45700 1 Processor -- start 2023-07-14T11:52:48.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.605+0000 7fc529d45700 1 -- start start 2023-07-14T11:52:48.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.609+0000 7fc529d45700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc524158a20 0x7fc5240cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:48.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.609+0000 7fc529d45700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52406c220 0x7fc5240cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:48.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.609+0000 7fc529d45700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52406d310 con 0x7fc5240cb820 2023-07-14T11:52:48.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.609+0000 7fc529d45700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc52406c700 con 0x7fc524158a20 2023-07-14T11:52:48.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.609+0000 7fc529d45700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52406c9e0 con 0x7fc52406c220 2023-07-14T11:52:48.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.609+0000 7fc51bfff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52406c220 0x7fc5240cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:48.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.609+0000 7fc528d43700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc524158a20 0x7fc5240cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:48.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.609+0000 7fc51bfff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52406c220 0x7fc5240cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:42034/0 (socket says 172.21.15.121:42034) 2023-07-14T11:52:48.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.609+0000 7fc51bfff700 1 -- 172.21.15.121:0/1997916905 learned_addr learned my addr 172.21.15.121:0/1997916905 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-14T11:52:48.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.609+0000 7fc51bfff700 1 -- 172.21.15.121:0/1997916905 >> v2:172.21.15.121:3300/0 conn(0x7fc524158a20 msgr2=0x7fc5240cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:48.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.609+0000 7fc51bfff700 1 --2- 172.21.15.121:0/1997916905 >> v2:172.21.15.121:3300/0 conn(0x7fc524158a20 0x7fc5240cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.609+0000 7fc51bfff700 1 -- 172.21.15.121:0/1997916905 >> v1:172.21.15.104:6789/0 conn(0x7fc5240cb820 legacy=0x7fc52406d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:48.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.609+0000 7fc51bfff700 1 -- 172.21.15.121:0/1997916905 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52406ccc0 con 0x7fc52406c220 2023-07-14T11:52:48.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.613+0000 7fc51bfff700 1 --2- 172.21.15.121:0/1997916905 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52406c220 0x7fc5240cbd20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc51400bbb0 tx=0x7fc51400bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d342749512e15b42 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:48.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.613+0000 7fc51b7fe700 1 -- 172.21.15.121:0/1997916905 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51400dd30 con 0x7fc52406c220 2023-07-14T11:52:48.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.613+0000 7fc51b7fe700 1 -- 172.21.15.121:0/1997916905 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51400de90 con 0x7fc52406c220 2023-07-14T11:52:48.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.613+0000 7fc51b7fe700 1 -- 172.21.15.121:0/1997916905 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514010690 con 0x7fc52406c220 2023-07-14T11:52:48.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.613+0000 7fc529d45700 1 -- 172.21.15.121:0/1997916905 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52406c220 msgr2=0x7fc5240cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:48.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.613+0000 7fc529d45700 1 --2- 172.21.15.121:0/1997916905 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52406c220 0x7fc5240cbd20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc51400bbb0 tx=0x7fc51400bec0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.613+0000 7fc529d45700 1 -- 172.21.15.121:0/1997916905 shutdown_connections 2023-07-14T11:52:48.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.613+0000 7fc529d45700 1 --2- 172.21.15.121:0/1997916905 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52406c220 0x7fc5240cbd20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.613+0000 7fc529d45700 1 --2- 172.21.15.121:0/1997916905 >> v2:172.21.15.121:3300/0 conn(0x7fc524158a20 0x7fc5240cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.613+0000 7fc529d45700 1 -- 172.21.15.121:0/1997916905 >> 172.21.15.121:0/1997916905 conn(0x7fc5240afc60 msgr2=0x7fc5240b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:48.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.613+0000 7fc529d45700 1 -- 172.21.15.121:0/1997916905 shutdown_connections 2023-07-14T11:52:48.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.613+0000 7fc529d45700 1 -- 172.21.15.121:0/1997916905 wait complete. 2023-07-14T11:52:48.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.617+0000 7fc529d45700 1 Processor -- start 2023-07-14T11:52:48.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.617+0000 7fc529d45700 1 -- start start 2023-07-14T11:52:48.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.617+0000 7fc529d45700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52406c220 0x7fc524152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:48.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.617+0000 7fc529d45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5240cb820 0x7fc524153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:48.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.617+0000 7fc529d45700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc524158a20 0x7fc5240cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:48.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.617+0000 7fc529d45700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc524155f20 con 0x7fc524158a20 2023-07-14T11:52:48.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.617+0000 7fc529d45700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5241592a0 con 0x7fc5240cb820 2023-07-14T11:52:48.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.617+0000 7fc529d45700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52415c090 con 0x7fc52406c220 2023-07-14T11:52:48.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.617+0000 7fc51bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5240cb820 0x7fc524153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:48.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.617+0000 7fc528d43700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52406c220 0x7fc524152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:48.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc528d43700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52406c220 0x7fc524152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:42046/0 (socket says 172.21.15.121:42046) 2023-07-14T11:52:48.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc528d43700 1 -- 172.21.15.121:0/2285809040 learned_addr learned my addr 172.21.15.121:0/2285809040 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-14T11:52:48.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc529544700 1 --2- 172.21.15.121:0/2285809040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc524158a20 0x7fc5240cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:48.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc51bfff700 1 -- 172.21.15.121:0/2285809040 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52406c220 msgr2=0x7fc524152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:48.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc51bfff700 1 --2- 172.21.15.121:0/2285809040 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52406c220 0x7fc524152da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc51bfff700 1 -- 172.21.15.121:0/2285809040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc524158a20 msgr2=0x7fc5240cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:48.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc51bfff700 1 --2- 172.21.15.121:0/2285809040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc524158a20 0x7fc5240cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc51bfff700 1 -- 172.21.15.121:0/2285809040 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51400b810 con 0x7fc5240cb820 2023-07-14T11:52:48.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc528d43700 1 --2- 172.21.15.121:0/2285809040 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52406c220 0x7fc524152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:52:48.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc529544700 1 --2- 172.21.15.121:0/2285809040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc524158a20 0x7fc5240cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:52:48.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc51bfff700 1 --2- 172.21.15.121:0/2285809040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5240cb820 0x7fc524153530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc514018000 tx=0x7fc514004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:48.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc5197fa700 1 -- 172.21.15.121:0/2285809040 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514003c90 con 0x7fc5240cb820 2023-07-14T11:52:48.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc5197fa700 1 -- 172.21.15.121:0/2285809040 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc514003e10 con 0x7fc5240cb820 2023-07-14T11:52:48.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc5197fa700 1 -- 172.21.15.121:0/2285809040 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5140106b0 con 0x7fc5240cb820 2023-07-14T11:52:48.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc529d45700 1 -- 172.21.15.121:0/2285809040 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5241532e0 con 0x7fc5240cb820 2023-07-14T11:52:48.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc529d45700 1 -- 172.21.15.121:0/2285809040 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52415bdb0 con 0x7fc5240cb820 2023-07-14T11:52:48.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc5197fa700 1 -- 172.21.15.121:0/2285809040 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc514010b10 con 0x7fc5240cb820 2023-07-14T11:52:48.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc5197fa700 1 -- 172.21.15.121:0/2285809040 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc514019400 con 0x7fc5240cb820 2023-07-14T11:52:48.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.621+0000 7fc529d45700 1 -- 172.21.15.121:0/2285809040 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc524156060 con 0x7fc5240cb820 2023-07-14T11:52:48.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.625+0000 7fc5197fa700 1 -- 172.21.15.121:0/2285809040 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fc524156060 con 0x7fc5240cb820 2023-07-14T11:52:48.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.705+0000 7fc529d45700 1 -- 172.21.15.121:0/2285809040 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0e83f7df-2e45-460d-a746-d48694cee08b", "id": 0} v 0) v1 -- 0x7fc52404e500 con 0x7fc5240cb820 2023-07-14T11:52:48.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.713+0000 7fc5197fa700 1 -- 172.21.15.121:0/2285809040 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0e83f7df-2e45-460d-a746-d48694cee08b", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc51402c430 con 0x7fc5240cb820 2023-07-14T11:52:48.700 INFO:teuthology.orchestra.run.smithi121.stdout:0 2023-07-14T11:52:48.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.717+0000 7fc529d45700 1 -- 172.21.15.121:0/2285809040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5240cb820 msgr2=0x7fc524153530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:48.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.717+0000 7fc529d45700 1 --2- 172.21.15.121:0/2285809040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5240cb820 0x7fc524153530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc514018000 tx=0x7fc514004770 comp rx=0 tx=0).stop 2023-07-14T11:52:48.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.717+0000 7fc529d45700 1 -- 172.21.15.121:0/2285809040 shutdown_connections 2023-07-14T11:52:48.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.717+0000 7fc529d45700 1 --2- 172.21.15.121:0/2285809040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc524158a20 0x7fc5240cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.717+0000 7fc529d45700 1 --2- 172.21.15.121:0/2285809040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5240cb820 0x7fc524153530 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.717+0000 7fc529d45700 1 --2- 172.21.15.121:0/2285809040 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52406c220 0x7fc524152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.717+0000 7fc529d45700 1 -- 172.21.15.121:0/2285809040 >> 172.21.15.121:0/2285809040 conn(0x7fc5240afc60 msgr2=0x7fc5240adb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:48.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.717+0000 7fc529d45700 1 -- 172.21.15.121:0/2285809040 shutdown_connections 2023-07-14T11:52:48.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.717+0000 7fc529d45700 1 -- 172.21.15.121:0/2285809040 wait complete. 2023-07-14T11:52:48.722 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new c4210db3-7801-45f7-89d2-811b2327e932 1 2023-07-14T11:52:48.740 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:48.757+0000 7fe50f93f2c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T11:52:48.769 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:48.782+0000 7f4cc2ed62c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T11:52:48.827 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:48.841+0000 7fe50f93f2c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T11:52:48.855 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:48.870+0000 7f4cc2ed62c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T11:52:48.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.893+0000 7fb262048700 1 Processor -- start 2023-07-14T11:52:48.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.893+0000 7fb262048700 1 -- start start 2023-07-14T11:52:48.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb262048700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb25c06a150 0x7fb25c06cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:48.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb262048700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb25c06d490 0x7fb25c0c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:48.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb262048700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb25c06a5d0 con 0x7fb25c158ac0 2023-07-14T11:52:48.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb262048700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25c06d890 con 0x7fb25c06a150 2023-07-14T11:52:48.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb262048700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c06b8f0 con 0x7fb25c06d490 2023-07-14T11:52:48.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb261046700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb25c158ac0 0x7fb25c0cda10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57500/0 (socket says 172.21.15.121:57500) 2023-07-14T11:52:48.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb261046700 1 -- 172.21.15.121:0/1270591022 learned_addr learned my addr 172.21.15.121:0/1270591022 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-07-14T11:52:48.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb260845700 1 --2- 172.21.15.121:0/1270591022 >> v2:172.21.15.121:3300/0 conn(0x7fb25c06a150 0x7fb25c06cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:48.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb261847700 1 --2- 172.21.15.121:0/1270591022 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb25c06d490 0x7fb25c0c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:48.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb253fff700 1 -- 172.21.15.121:0/1270591022 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308931993 0 0) 0x7fb25c06a5d0 con 0x7fb25c158ac0 2023-07-14T11:52:48.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb253fff700 1 -- 172.21.15.121:0/1270591022 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2480035d0 con 0x7fb25c158ac0 2023-07-14T11:52:48.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb261847700 1 -- 172.21.15.121:0/1270591022 >> v2:172.21.15.121:3300/0 conn(0x7fb25c06a150 msgr2=0x7fb25c06cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:48.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb261847700 1 --2- 172.21.15.121:0/1270591022 >> v2:172.21.15.121:3300/0 conn(0x7fb25c06a150 0x7fb25c06cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb261847700 1 -- 172.21.15.121:0/1270591022 >> v1:172.21.15.121:6789/0 conn(0x7fb25c158ac0 legacy=0x7fb25c0cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:48.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb261847700 1 -- 172.21.15.121:0/1270591022 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c06a850 con 0x7fb25c06d490 2023-07-14T11:52:48.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb261847700 1 --2- 172.21.15.121:0/1270591022 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb25c06d490 0x7fb25c0c9b30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb254009d90 tx=0x7fb25400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45712558f8d2a48e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:48.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb253fff700 1 -- 172.21.15.121:0/1270591022 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400e070 con 0x7fb25c06d490 2023-07-14T11:52:48.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb253fff700 1 -- 172.21.15.121:0/1270591022 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25400bae0 con 0x7fb25c06d490 2023-07-14T11:52:48.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.897+0000 7fb253fff700 1 -- 172.21.15.121:0/1270591022 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400be50 con 0x7fb25c06d490 2023-07-14T11:52:48.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.901+0000 7fb262048700 1 -- 172.21.15.121:0/1270591022 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb25c06d490 msgr2=0x7fb25c0c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:48.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.901+0000 7fb262048700 1 --2- 172.21.15.121:0/1270591022 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb25c06d490 0x7fb25c0c9b30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb254009d90 tx=0x7fb25400b330 comp rx=0 tx=0).stop 2023-07-14T11:52:48.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.901+0000 7fb262048700 1 -- 172.21.15.121:0/1270591022 shutdown_connections 2023-07-14T11:52:48.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.901+0000 7fb262048700 1 --2- 172.21.15.121:0/1270591022 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb25c06d490 0x7fb25c0c9b30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.901+0000 7fb262048700 1 --2- 172.21.15.121:0/1270591022 >> v2:172.21.15.121:3300/0 conn(0x7fb25c06a150 0x7fb25c06cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.901+0000 7fb262048700 1 -- 172.21.15.121:0/1270591022 >> 172.21.15.121:0/1270591022 conn(0x7fb25c06fe60 msgr2=0x7fb25c070260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:48.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.901+0000 7fb262048700 1 -- 172.21.15.121:0/1270591022 shutdown_connections 2023-07-14T11:52:48.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.901+0000 7fb262048700 1 -- 172.21.15.121:0/1270591022 wait complete. 2023-07-14T11:52:48.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.901+0000 7fb262048700 1 Processor -- start 2023-07-14T11:52:48.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.901+0000 7fb262048700 1 -- start start 2023-07-14T11:52:48.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb262048700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c06a150 0x7fb25c0c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:48.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb262048700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb25c06d490 0x7fb25c0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:48.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb262048700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb25c158ac0 0x7fb25c0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:48.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb262048700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0c5e60 con 0x7fb25c06d490 2023-07-14T11:52:48.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb262048700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0b4f70 con 0x7fb25c06a150 2023-07-14T11:52:48.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb262048700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0b4c90 con 0x7fb25c158ac0 2023-07-14T11:52:48.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb261046700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c06a150 0x7fb25c0c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:48.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb261046700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c06a150 0x7fb25c0c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52538/0 (socket says 172.21.15.121:52538) 2023-07-14T11:52:48.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb261046700 1 -- 172.21.15.121:0/4247619665 learned_addr learned my addr 172.21.15.121:0/4247619665 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-14T11:52:48.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb261847700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb25c158ac0 0x7fb25c0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:48.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb260845700 1 --2- 172.21.15.121:0/4247619665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb25c06d490 0x7fb25c0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:48.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb261046700 1 -- 172.21.15.121:0/4247619665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb25c158ac0 msgr2=0x7fb25c0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:48.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb261046700 1 --2- 172.21.15.121:0/4247619665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb25c158ac0 0x7fb25c0b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb261046700 1 -- 172.21.15.121:0/4247619665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb25c06d490 msgr2=0x7fb25c0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:48.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb261046700 1 --2- 172.21.15.121:0/4247619665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb25c06d490 0x7fb25c0b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb261046700 1 -- 172.21.15.121:0/4247619665 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb254009a70 con 0x7fb25c06a150 2023-07-14T11:52:48.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb261046700 1 --2- 172.21.15.121:0/4247619665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c06a150 0x7fb25c0c5920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb25800c4a0 tx=0x7fb25800c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:48.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb251ffb700 1 -- 172.21.15.121:0/4247619665 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258004ca0 con 0x7fb25c06a150 2023-07-14T11:52:48.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb251ffb700 1 -- 172.21.15.121:0/4247619665 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb258004e00 con 0x7fb25c06a150 2023-07-14T11:52:48.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb262048700 1 -- 172.21.15.121:0/4247619665 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25c0ac400 con 0x7fb25c06a150 2023-07-14T11:52:48.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb251ffb700 1 -- 172.21.15.121:0/4247619665 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258007310 con 0x7fb25c06a150 2023-07-14T11:52:48.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb251ffb700 1 -- 172.21.15.121:0/4247619665 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb2580133c0 con 0x7fb25c06a150 2023-07-14T11:52:48.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb262048700 1 -- 172.21.15.121:0/4247619665 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c0ac9c0 con 0x7fb25c06a150 2023-07-14T11:52:48.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.905+0000 7fb251ffb700 1 -- 172.21.15.121:0/4247619665 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fb258013c50 con 0x7fb25c06a150 2023-07-14T11:52:48.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.909+0000 7fb262048700 1 -- 172.21.15.121:0/4247619665 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25c0cc580 con 0x7fb25c06a150 2023-07-14T11:52:48.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.909+0000 7fb251ffb700 1 -- 172.21.15.121:0/4247619665 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fb258013e10 con 0x7fb25c06a150 2023-07-14T11:52:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.989+0000 7fb262048700 1 -- 172.21.15.121:0/4247619665 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c4210db3-7801-45f7-89d2-811b2327e932", "id": 1} v 0) v1 -- 0x7fb25c0acbe0 con 0x7fb25c06a150 2023-07-14T11:52:48.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:48.997+0000 7fb251ffb700 1 -- 172.21.15.121:0/4247619665 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c4210db3-7801-45f7-89d2-811b2327e932", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb2580254c0 con 0x7fb25c06a150 2023-07-14T11:52:48.983 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-07-14T11:52:48.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.001+0000 7fb262048700 1 -- 172.21.15.121:0/4247619665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c06a150 msgr2=0x7fb25c0c5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:48.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.001+0000 7fb262048700 1 --2- 172.21.15.121:0/4247619665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c06a150 0x7fb25c0c5920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb25800c4a0 tx=0x7fb25800c7b0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.001+0000 7fb262048700 1 -- 172.21.15.121:0/4247619665 shutdown_connections 2023-07-14T11:52:48.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.001+0000 7fb262048700 1 --2- 172.21.15.121:0/4247619665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb25c06d490 0x7fb25c0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.001+0000 7fb262048700 1 --2- 172.21.15.121:0/4247619665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb25c158ac0 0x7fb25c0b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.001+0000 7fb262048700 1 --2- 172.21.15.121:0/4247619665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c06a150 0x7fb25c0c5920 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.001+0000 7fb262048700 1 -- 172.21.15.121:0/4247619665 >> 172.21.15.121:0/4247619665 conn(0x7fb25c06fe60 msgr2=0x7fb25c0b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:48.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.001+0000 7fb262048700 1 -- 172.21.15.121:0/4247619665 shutdown_connections 2023-07-14T11:52:48.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.001+0000 7fb262048700 1 -- 172.21.15.121:0/4247619665 wait complete. 2023-07-14T11:52:49.006 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 85424c50-7fbb-43e9-8de1-e9acdc98e422 2 2023-07-14T11:52:49.013 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:49.029+0000 7fe50f93f2c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:52:49.038 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:49.054+0000 7f4cc2ed62c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:52:49.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.201+0000 7fbdfd1ae700 1 Processor -- start 2023-07-14T11:52:49.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.201+0000 7fbdfd1ae700 1 -- start start 2023-07-14T11:52:49.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.205+0000 7fbdfd1ae700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbdf815df10 0x7fbdf815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.205+0000 7fbdfd1ae700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbdf80c8950 0x7fbdf80c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.205+0000 7fbdfd1ae700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf8070a30 con 0x7fbdf80afd80 2023-07-14T11:52:49.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.205+0000 7fbdfd1ae700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdf80c92f0 con 0x7fbdf80c8950 2023-07-14T11:52:49.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.205+0000 7fbdfd1ae700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf815e830 con 0x7fbdf815df10 2023-07-14T11:52:49.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.205+0000 7fbdf659c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fbdf80afd80 0x7fbdf80711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57510/0 (socket says 172.21.15.121:57510) 2023-07-14T11:52:49.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.205+0000 7fbdf659c700 1 -- 172.21.15.121:0/3130624020 learned_addr learned my addr 172.21.15.121:0/3130624020 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-07-14T11:52:49.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.205+0000 7fbdf6d9d700 1 --2- 172.21.15.121:0/3130624020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbdf815df10 0x7fbdf815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.205+0000 7fbdf759e700 1 --2- 172.21.15.121:0/3130624020 >> v2:172.21.15.104:3300/0 conn(0x7fbdf80c8950 0x7fbdf80c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.205+0000 7fbdf5d9b700 1 -- 172.21.15.121:0/3130624020 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465945447 0 0) 0x7fbdf8070a30 con 0x7fbdf80afd80 2023-07-14T11:52:49.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.205+0000 7fbdf5d9b700 1 -- 172.21.15.121:0/3130624020 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbde00035d0 con 0x7fbdf80afd80 2023-07-14T11:52:49.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.205+0000 7fbdf6d9d700 1 -- 172.21.15.121:0/3130624020 >> v2:172.21.15.104:3300/0 conn(0x7fbdf80c8950 msgr2=0x7fbdf80c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:49.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.205+0000 7fbdf6d9d700 1 --2- 172.21.15.121:0/3130624020 >> v2:172.21.15.104:3300/0 conn(0x7fbdf80c8950 0x7fbdf80c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.205+0000 7fbdf6d9d700 1 -- 172.21.15.121:0/3130624020 >> v1:172.21.15.121:6789/0 conn(0x7fbdf80afd80 legacy=0x7fbdf80711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.205+0000 7fbdf6d9d700 1 -- 172.21.15.121:0/3130624020 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf80b0210 con 0x7fbdf815df10 2023-07-14T11:52:49.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.209+0000 7fbdf6d9d700 1 --2- 172.21.15.121:0/3130624020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbdf815df10 0x7fbdf815e2f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbde8009850 tx=0x7fbde8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe2f2f879efa2c2e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:49.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.209+0000 7fbdf5d9b700 1 -- 172.21.15.121:0/3130624020 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde8011830 con 0x7fbdf815df10 2023-07-14T11:52:49.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.209+0000 7fbdf5d9b700 1 -- 172.21.15.121:0/3130624020 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde8011990 con 0x7fbdf815df10 2023-07-14T11:52:49.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.209+0000 7fbdf5d9b700 1 -- 172.21.15.121:0/3130624020 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde8011d40 con 0x7fbdf815df10 2023-07-14T11:52:49.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.209+0000 7fbdfd1ae700 1 -- 172.21.15.121:0/3130624020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbdf815df10 msgr2=0x7fbdf815e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.209+0000 7fbdfd1ae700 1 --2- 172.21.15.121:0/3130624020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbdf815df10 0x7fbdf815e2f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbde8009850 tx=0x7fbde8009b60 comp rx=0 tx=0).stop 2023-07-14T11:52:49.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.209+0000 7fbdfd1ae700 1 -- 172.21.15.121:0/3130624020 shutdown_connections 2023-07-14T11:52:49.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.209+0000 7fbdfd1ae700 1 --2- 172.21.15.121:0/3130624020 >> v2:172.21.15.104:3300/0 conn(0x7fbdf80c8950 0x7fbdf80c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.209+0000 7fbdfd1ae700 1 --2- 172.21.15.121:0/3130624020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbdf815df10 0x7fbdf815e2f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.209+0000 7fbdfd1ae700 1 -- 172.21.15.121:0/3130624020 >> 172.21.15.121:0/3130624020 conn(0x7fbdf80cad80 msgr2=0x7fbdf80cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:49.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.209+0000 7fbdfd1ae700 1 -- 172.21.15.121:0/3130624020 shutdown_connections 2023-07-14T11:52:49.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.209+0000 7fbdfd1ae700 1 -- 172.21.15.121:0/3130624020 wait complete. 2023-07-14T11:52:49.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdfd1ae700 1 Processor -- start 2023-07-14T11:52:49.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdfd1ae700 1 -- start start 2023-07-14T11:52:49.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdfd1ae700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf80afd80 0x7fbdf80ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdfd1ae700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbdf80c8950 0x7fbdf80c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdfd1ae700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbdf80b2180 0x7fbdf80b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdfd1ae700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf80b2b60 con 0x7fbdf80afd80 2023-07-14T11:52:49.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdfd1ae700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf80ad4f0 con 0x7fbdf80b2180 2023-07-14T11:52:49.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdfd1ae700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf80c5100 con 0x7fbdf80c8950 2023-07-14T11:52:49.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdf759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbdf80b2180 0x7fbdf80b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdf759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbdf80b2180 0x7fbdf80b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52540/0 (socket says 172.21.15.121:52540) 2023-07-14T11:52:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdf759e700 1 -- 172.21.15.121:0/79014911 learned_addr learned my addr 172.21.15.121:0/79014911 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-14T11:52:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdf659c700 1 --2- 172.21.15.121:0/79014911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbdf80c8950 0x7fbdf80c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdf6d9d700 1 --2- 172.21.15.121:0/79014911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf80afd80 0x7fbdf80ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdf759e700 1 -- 172.21.15.121:0/79014911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbdf80c8950 msgr2=0x7fbdf80c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdf759e700 1 --2- 172.21.15.121:0/79014911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbdf80c8950 0x7fbdf80c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdf759e700 1 -- 172.21.15.121:0/79014911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf80afd80 msgr2=0x7fbdf80ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdf759e700 1 --2- 172.21.15.121:0/79014911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf80afd80 0x7fbdf80ab8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdf759e700 1 -- 172.21.15.121:0/79014911 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde80094b0 con 0x7fbdf80b2180 2023-07-14T11:52:49.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdf759e700 1 --2- 172.21.15.121:0/79014911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbdf80b2180 0x7fbdf80b2620 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000efc0 tx=0x7fbdf000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:49.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbddf7fe700 1 -- 172.21.15.121:0/79014911 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0009ea0 con 0x7fbdf80b2180 2023-07-14T11:52:49.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbddf7fe700 1 -- 172.21.15.121:0/79014911 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf000f040 con 0x7fbdf80b2180 2023-07-14T11:52:49.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdfd1ae700 1 -- 172.21.15.121:0/79014911 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf80cdec0 con 0x7fbdf80b2180 2023-07-14T11:52:49.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbddf7fe700 1 -- 172.21.15.121:0/79014911 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf000ce70 con 0x7fbdf80b2180 2023-07-14T11:52:49.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.213+0000 7fbdfd1ae700 1 -- 172.21.15.121:0/79014911 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf80ad210 con 0x7fbdf80b2180 2023-07-14T11:52:49.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.217+0000 7fbddf7fe700 1 -- 172.21.15.121:0/79014911 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbdf0014780 con 0x7fbdf80b2180 2023-07-14T11:52:49.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.217+0000 7fbddf7fe700 1 -- 172.21.15.121:0/79014911 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fbdf001d400 con 0x7fbdf80b2180 2023-07-14T11:52:49.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.217+0000 7fbdfd1ae700 1 -- 172.21.15.121:0/79014911 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf804dcb0 con 0x7fbdf80b2180 2023-07-14T11:52:49.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.221+0000 7fbddf7fe700 1 -- 172.21.15.121:0/79014911 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fbdf003bb20 con 0x7fbdf80b2180 2023-07-14T11:52:49.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.301+0000 7fbdfd1ae700 1 -- 172.21.15.121:0/79014911 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd new", "uuid": "85424c50-7fbb-43e9-8de1-e9acdc98e422", "id": 2} v 0) v1 -- 0x7fbdf80cbf60 con 0x7fbdf80b2180 2023-07-14T11:52:49.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.309+0000 7fbddf7fe700 1 -- 172.21.15.121:0/79014911 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "85424c50-7fbb-43e9-8de1-e9acdc98e422", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbdf002a030 con 0x7fbdf80b2180 2023-07-14T11:52:49.297 INFO:teuthology.orchestra.run.smithi121.stdout:2 2023-07-14T11:52:49.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.313+0000 7fbdfd1ae700 1 -- 172.21.15.121:0/79014911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbdf80b2180 msgr2=0x7fbdf80b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.313+0000 7fbdfd1ae700 1 --2- 172.21.15.121:0/79014911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbdf80b2180 0x7fbdf80b2620 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000efc0 tx=0x7fbdf000c3a0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.313+0000 7fbdfd1ae700 1 -- 172.21.15.121:0/79014911 shutdown_connections 2023-07-14T11:52:49.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.313+0000 7fbdfd1ae700 1 --2- 172.21.15.121:0/79014911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbdf80b2180 0x7fbdf80b2620 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.313+0000 7fbdfd1ae700 1 --2- 172.21.15.121:0/79014911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbdf80c8950 0x7fbdf80c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.313+0000 7fbdfd1ae700 1 --2- 172.21.15.121:0/79014911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf80afd80 0x7fbdf80ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.313+0000 7fbdfd1ae700 1 -- 172.21.15.121:0/79014911 >> 172.21.15.121:0/79014911 conn(0x7fbdf80cad80 msgr2=0x7fbdf8158b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:49.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.313+0000 7fbdfd1ae700 1 -- 172.21.15.121:0/79014911 shutdown_connections 2023-07-14T11:52:49.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.313+0000 7fbdfd1ae700 1 -- 172.21.15.121:0/79014911 wait complete. 2023-07-14T11:52:49.299 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:49.313+0000 7fe50f93f2c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T11:52:49.316 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 75db666c-73d2-4934-8417-4ce190953af0 3 2023-07-14T11:52:49.319 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:49.334+0000 7f4cc2ed62c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T11:52:49.379 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:49.393+0000 7fe50f93f2c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T11:52:49.399 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:49.414+0000 7f4cc2ed62c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T11:52:49.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.489+0000 7f9855a78700 1 Processor -- start 2023-07-14T11:52:49.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.489+0000 7f9855a78700 1 -- start start 2023-07-14T11:52:49.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.489+0000 7f9855a78700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98500cf7a0 0x7f98500d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.489+0000 7f9855a78700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f98500b1390 0x7f98500d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f9855a78700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98500af2e0 con 0x7f98500d21c0 2023-07-14T11:52:49.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f9855a78700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98500d0030 con 0x7f98500b1390 2023-07-14T11:52:49.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f9855a78700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98500af5c0 con 0x7f98500cf7a0 2023-07-14T11:52:49.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f984ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f98500b1390 0x7f98500d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f984f7fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98500cf7a0 0x7f98500d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f984ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f98500b1390 0x7f98500d0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44762/0 (socket says 172.21.15.121:44762) 2023-07-14T11:52:49.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f984ffff700 1 -- 172.21.15.121:0/3222290995 learned_addr learned my addr 172.21.15.121:0/3222290995 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-14T11:52:49.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f984ffff700 1 -- 172.21.15.121:0/3222290995 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98500cf7a0 msgr2=0x7f98500d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:49.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f984ffff700 1 --2- 172.21.15.121:0/3222290995 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98500cf7a0 0x7f98500d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f984ffff700 1 -- 172.21.15.121:0/3222290995 >> v1:172.21.15.104:6789/0 conn(0x7f98500d21c0 legacy=0x7f985006fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f984ffff700 1 -- 172.21.15.121:0/3222290995 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98500af8a0 con 0x7f98500b1390 2023-07-14T11:52:49.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f984f7fe700 1 --2- 172.21.15.121:0/3222290995 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98500cf7a0 0x7f98500d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:52:49.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f984ffff700 1 --2- 172.21.15.121:0/3222290995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98500b1390 0x7f98500d0750 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9848009e80 tx=0x7f984800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f2aa401b44184bc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:49.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f984e7fc700 1 -- 172.21.15.121:0/3222290995 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984800e070 con 0x7f98500b1390 2023-07-14T11:52:49.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f984e7fc700 1 -- 172.21.15.121:0/3222290995 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984800b8d0 con 0x7f98500b1390 2023-07-14T11:52:49.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f984e7fc700 1 -- 172.21.15.121:0/3222290995 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984800bc80 con 0x7f98500b1390 2023-07-14T11:52:49.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f9855a78700 1 -- 172.21.15.121:0/3222290995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98500b1390 msgr2=0x7f98500d0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f9855a78700 1 --2- 172.21.15.121:0/3222290995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98500b1390 0x7f98500d0750 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9848009e80 tx=0x7f984800b330 comp rx=0 tx=0).stop 2023-07-14T11:52:49.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f9855a78700 1 -- 172.21.15.121:0/3222290995 shutdown_connections 2023-07-14T11:52:49.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.493+0000 7f9855a78700 1 --2- 172.21.15.121:0/3222290995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98500b1390 0x7f98500d0750 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.497+0000 7f9855a78700 1 --2- 172.21.15.121:0/3222290995 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98500cf7a0 0x7f98500d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.497+0000 7f9855a78700 1 -- 172.21.15.121:0/3222290995 >> 172.21.15.121:0/3222290995 conn(0x7f985009b9c0 msgr2=0x7f98500d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:49.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.497+0000 7f9855a78700 1 -- 172.21.15.121:0/3222290995 shutdown_connections 2023-07-14T11:52:49.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.497+0000 7f9855a78700 1 -- 172.21.15.121:0/3222290995 wait complete. 2023-07-14T11:52:49.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f9855a78700 1 Processor -- start 2023-07-14T11:52:49.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f9855a78700 1 -- start start 2023-07-14T11:52:49.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f9855a78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98500b1390 0x7f9850159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f9855a78700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98500cf7a0 0x7f9850149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f9855a78700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98500d21c0 0x7f9850149d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f9855a78700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9850149010 con 0x7f98500cf7a0 2023-07-14T11:52:49.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f9855a78700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98500b7f00 con 0x7f98500b1390 2023-07-14T11:52:49.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f9855a78700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985015a160 con 0x7f98500d21c0 2023-07-14T11:52:49.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f984ffff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98500d21c0 0x7f9850149d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f984f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98500b1390 0x7f9850159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f984ffff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98500d21c0 0x7f9850149d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:39678/0 (socket says 172.21.15.121:39678) 2023-07-14T11:52:49.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f984ffff700 1 -- 172.21.15.121:0/2467564744 learned_addr learned my addr 172.21.15.121:0/2467564744 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-14T11:52:49.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f984f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98500b1390 0x7f9850159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44778/0 (socket says 172.21.15.121:44778) 2023-07-14T11:52:49.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f984effd700 1 --2- 172.21.15.121:0/2467564744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98500cf7a0 0x7f9850149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f984ffff700 1 -- 172.21.15.121:0/2467564744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98500b1390 msgr2=0x7f9850159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f984ffff700 1 --2- 172.21.15.121:0/2467564744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98500b1390 0x7f9850159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f984ffff700 1 -- 172.21.15.121:0/2467564744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98500cf7a0 msgr2=0x7f9850149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f984ffff700 1 --2- 172.21.15.121:0/2467564744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98500cf7a0 0x7f9850149840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f984ffff700 1 -- 172.21.15.121:0/2467564744 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98400094b0 con 0x7f98500d21c0 2023-07-14T11:52:49.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f984ffff700 1 --2- 172.21.15.121:0/2467564744 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98500d21c0 0x7f9850149d80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f985006eaf0 tx=0x7f9848004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:49.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f9837fff700 1 -- 172.21.15.121:0/2467564744 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848004800 con 0x7f98500d21c0 2023-07-14T11:52:49.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f9837fff700 1 -- 172.21.15.121:0/2467564744 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98480099b0 con 0x7f98500d21c0 2023-07-14T11:52:49.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f9855a78700 1 -- 172.21.15.121:0/2467564744 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9848009b30 con 0x7f98500d21c0 2023-07-14T11:52:49.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.501+0000 7f9837fff700 1 -- 172.21.15.121:0/2467564744 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984801b670 con 0x7f98500d21c0 2023-07-14T11:52:49.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.505+0000 7f9855a78700 1 -- 172.21.15.121:0/2467564744 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9850148670 con 0x7f98500d21c0 2023-07-14T11:52:49.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.505+0000 7f9837fff700 1 -- 172.21.15.121:0/2467564744 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f984801bad0 con 0x7f98500d21c0 2023-07-14T11:52:49.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.505+0000 7f9837fff700 1 -- 172.21.15.121:0/2467564744 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f9848024450 con 0x7f98500d21c0 2023-07-14T11:52:49.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.505+0000 7f9855a78700 1 -- 172.21.15.121:0/2467564744 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9850149150 con 0x7f98500d21c0 2023-07-14T11:52:49.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.509+0000 7f9837fff700 1 -- 172.21.15.121:0/2467564744 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f9848042b20 con 0x7f98500d21c0 2023-07-14T11:52:49.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.593+0000 7f9855a78700 1 -- 172.21.15.121:0/2467564744 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd new", "uuid": "75db666c-73d2-4934-8417-4ce190953af0", "id": 3} v 0) v1 -- 0x7f985004dcb0 con 0x7f98500d21c0 2023-07-14T11:52:49.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.601+0000 7f9837fff700 1 -- 172.21.15.121:0/2467564744 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "75db666c-73d2-4934-8417-4ce190953af0", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9848031030 con 0x7f98500d21c0 2023-07-14T11:52:49.589 INFO:teuthology.orchestra.run.smithi121.stdout:3 2023-07-14T11:52:49.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.605+0000 7f9855a78700 1 -- 172.21.15.121:0/2467564744 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98500d21c0 msgr2=0x7f9850149d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.605+0000 7f9855a78700 1 --2- 172.21.15.121:0/2467564744 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98500d21c0 0x7f9850149d80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f985006eaf0 tx=0x7f9848004af0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.605+0000 7f9855a78700 1 -- 172.21.15.121:0/2467564744 shutdown_connections 2023-07-14T11:52:49.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.605+0000 7f9855a78700 1 --2- 172.21.15.121:0/2467564744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98500cf7a0 0x7f9850149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.605+0000 7f9855a78700 1 --2- 172.21.15.121:0/2467564744 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98500d21c0 0x7f9850149d80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.605+0000 7f9855a78700 1 --2- 172.21.15.121:0/2467564744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98500b1390 0x7f9850159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.605+0000 7f9855a78700 1 -- 172.21.15.121:0/2467564744 >> 172.21.15.121:0/2467564744 conn(0x7f985009b9c0 msgr2=0x7f985015bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:49.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.605+0000 7f9855a78700 1 -- 172.21.15.121:0/2467564744 shutdown_connections 2023-07-14T11:52:49.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.605+0000 7f9855a78700 1 -- 172.21.15.121:0/2467564744 wait complete. 2023-07-14T11:52:49.617 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new d2ea3990-a14c-4c2e-9655-3928c83b1b5e 4 2023-07-14T11:52:49.640 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:49.657+0000 7fe50f93f2c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T11:52:49.652 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:49.666+0000 7f4cc2ed62c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T11:52:49.732 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:49.745+0000 7fe50f93f2c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T11:52:49.744 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:49.758+0000 7f4cc2ed62c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T11:52:49.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.793+0000 7f99c59d9700 1 Processor -- start 2023-07-14T11:52:49.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.793+0000 7f99c59d9700 1 -- start start 2023-07-14T11:52:49.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.793+0000 7f99c59d9700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99c00b0ee0 0x7f99c00ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.793+0000 7f99c59d9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f99c00cd090 0x7f99c00d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.793+0000 7f99c59d9700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c00cd4e0 con 0x7f99c00d0610 2023-07-14T11:52:49.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.793+0000 7f99c59d9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c00d1a60 con 0x7f99c00cd090 2023-07-14T11:52:49.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.793+0000 7f99c59d9700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00d1ef0 con 0x7f99c00b0ee0 2023-07-14T11:52:49.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.793+0000 7f99c49d7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99c00b0ee0 0x7f99c00ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99c49d7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99c00b0ee0 0x7f99c00ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:39688/0 (socket says 172.21.15.121:39688) 2023-07-14T11:52:49.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99c49d7700 1 -- 172.21.15.121:0/3660917776 learned_addr learned my addr 172.21.15.121:0/3660917776 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-14T11:52:49.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99bffff700 1 --2- 172.21.15.121:0/3660917776 >> v2:172.21.15.121:3300/0 conn(0x7f99c00cd090 0x7f99c00d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99c49d7700 1 -- 172.21.15.121:0/3660917776 >> v2:172.21.15.121:3300/0 conn(0x7f99c00cd090 msgr2=0x7f99c00d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:49.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99c49d7700 1 --2- 172.21.15.121:0/3660917776 >> v2:172.21.15.121:3300/0 conn(0x7f99c00cd090 0x7f99c00d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99c49d7700 1 -- 172.21.15.121:0/3660917776 >> v1:172.21.15.104:6789/0 conn(0x7f99c00d0610 legacy=0x7f99c015f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99c49d7700 1 -- 172.21.15.121:0/3660917776 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c00d0e50 con 0x7f99c00b0ee0 2023-07-14T11:52:49.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99c49d7700 1 --2- 172.21.15.121:0/3660917776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99c00b0ee0 0x7f99c00ccb50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f99b4009940 tx=0x7f99b4009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=549728ff4dadf5b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:49.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99bf7fe700 1 -- 172.21.15.121:0/3660917776 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b400ba20 con 0x7f99c00b0ee0 2023-07-14T11:52:49.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99bf7fe700 1 -- 172.21.15.121:0/3660917776 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b4009e90 con 0x7f99c00b0ee0 2023-07-14T11:52:49.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99bf7fe700 1 -- 172.21.15.121:0/3660917776 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b400bda0 con 0x7f99c00b0ee0 2023-07-14T11:52:49.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99c59d9700 1 -- 172.21.15.121:0/3660917776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99c00b0ee0 msgr2=0x7f99c00ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99c59d9700 1 --2- 172.21.15.121:0/3660917776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99c00b0ee0 0x7f99c00ccb50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f99b4009940 tx=0x7f99b4009c50 comp rx=0 tx=0).stop 2023-07-14T11:52:49.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99c59d9700 1 -- 172.21.15.121:0/3660917776 shutdown_connections 2023-07-14T11:52:49.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99c59d9700 1 --2- 172.21.15.121:0/3660917776 >> v2:172.21.15.121:3300/0 conn(0x7f99c00cd090 0x7f99c00d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99c59d9700 1 --2- 172.21.15.121:0/3660917776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99c00b0ee0 0x7f99c00ccb50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99c59d9700 1 -- 172.21.15.121:0/3660917776 >> 172.21.15.121:0/3660917776 conn(0x7f99c0060c40 msgr2=0x7f99c009b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:49.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99c59d9700 1 -- 172.21.15.121:0/3660917776 shutdown_connections 2023-07-14T11:52:49.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.797+0000 7f99c59d9700 1 -- 172.21.15.121:0/3660917776 wait complete. 2023-07-14T11:52:49.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.801+0000 7f99c59d9700 1 Processor -- start 2023-07-14T11:52:49.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.801+0000 7f99c59d9700 1 -- start start 2023-07-14T11:52:49.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.801+0000 7f99c59d9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c00b0ee0 0x7f99c00b7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.801+0000 7f99c59d9700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99c00cd090 0x7f99c0156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.801+0000 7f99c59d9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99c00d0610 0x7f99c015fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.801+0000 7f99c59d9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00b2db0 con 0x7f99c00b0ee0 2023-07-14T11:52:49.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.801+0000 7f99c59d9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0156ad0 con 0x7f99c00d0610 2023-07-14T11:52:49.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.801+0000 7f99c59d9700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00b21a0 con 0x7f99c00cd090 2023-07-14T11:52:49.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99c51d8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99c00d0610 0x7f99c015fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99c51d8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99c00d0610 0x7f99c015fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44790/0 (socket says 172.21.15.121:44790) 2023-07-14T11:52:49.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99c51d8700 1 -- 172.21.15.121:0/3000439488 learned_addr learned my addr 172.21.15.121:0/3000439488 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-14T11:52:49.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99bffff700 1 --2- 172.21.15.121:0/3000439488 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99c00cd090 0x7f99c0156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99c49d7700 1 --2- 172.21.15.121:0/3000439488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c00b0ee0 0x7f99c00b7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99c51d8700 1 -- 172.21.15.121:0/3000439488 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99c00cd090 msgr2=0x7f99c0156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99c51d8700 1 --2- 172.21.15.121:0/3000439488 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99c00cd090 0x7f99c0156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99c51d8700 1 -- 172.21.15.121:0/3000439488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c00b0ee0 msgr2=0x7f99c00b7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99c51d8700 1 --2- 172.21.15.121:0/3000439488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c00b0ee0 0x7f99c00b7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99c51d8700 1 -- 172.21.15.121:0/3000439488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b4009570 con 0x7f99c00d0610 2023-07-14T11:52:49.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99c49d7700 1 --2- 172.21.15.121:0/3000439488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c00b0ee0 0x7f99c00b7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:52:49.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99c51d8700 1 --2- 172.21.15.121:0/3000439488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99c00d0610 0x7f99c015fdc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f99b800cc50 tx=0x7f99b800cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:49.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99bd7fa700 1 -- 172.21.15.121:0/3000439488 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8003c20 con 0x7f99c00d0610 2023-07-14T11:52:49.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99bd7fa700 1 -- 172.21.15.121:0/3000439488 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b8003d80 con 0x7f99c00d0610 2023-07-14T11:52:49.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99c59d9700 1 -- 172.21.15.121:0/3000439488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c00b2480 con 0x7f99c00d0610 2023-07-14T11:52:49.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99bd7fa700 1 -- 172.21.15.121:0/3000439488 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b800ab60 con 0x7f99c00d0610 2023-07-14T11:52:49.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99c59d9700 1 -- 172.21.15.121:0/3000439488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c00b2a40 con 0x7f99c00d0610 2023-07-14T11:52:49.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99bd7fa700 1 -- 172.21.15.121:0/3000439488 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f99b80050d0 con 0x7f99c00d0610 2023-07-14T11:52:49.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99bd7fa700 1 -- 172.21.15.121:0/3000439488 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f99b8005890 con 0x7f99c00d0610 2023-07-14T11:52:49.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.805+0000 7f99c59d9700 1 -- 172.21.15.121:0/3000439488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c00b2f60 con 0x7f99c00d0610 2023-07-14T11:52:49.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.809+0000 7f99bd7fa700 1 -- 172.21.15.121:0/3000439488 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f99c00b2f60 con 0x7f99c00d0610 2023-07-14T11:52:49.826 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:49.841+0000 7fe50f93f2c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T11:52:49.836 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:49.850+0000 7f4cc2ed62c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T11:52:49.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.889+0000 7f99c59d9700 1 -- 172.21.15.121:0/3000439488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d2ea3990-a14c-4c2e-9655-3928c83b1b5e", "id": 4} v 0) v1 -- 0x7f99c00b2bd0 con 0x7f99c00d0610 2023-07-14T11:52:49.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.901+0000 7f99bd7fa700 1 -- 172.21.15.121:0/3000439488 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d2ea3990-a14c-4c2e-9655-3928c83b1b5e", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f99c00cd4e0 con 0x7f99c00d0610 2023-07-14T11:52:49.886 INFO:teuthology.orchestra.run.smithi121.stdout:4 2023-07-14T11:52:49.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.905+0000 7f99c59d9700 1 -- 172.21.15.121:0/3000439488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99c00d0610 msgr2=0x7f99c015fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.905+0000 7f99c59d9700 1 --2- 172.21.15.121:0/3000439488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99c00d0610 0x7f99c015fdc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f99b800cc50 tx=0x7f99b800cf60 comp rx=0 tx=0).stop 2023-07-14T11:52:49.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.905+0000 7f99c59d9700 1 -- 172.21.15.121:0/3000439488 shutdown_connections 2023-07-14T11:52:49.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.905+0000 7f99c59d9700 1 --2- 172.21.15.121:0/3000439488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99c00d0610 0x7f99c015fdc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.905+0000 7f99c59d9700 1 --2- 172.21.15.121:0/3000439488 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99c00cd090 0x7f99c0156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.905+0000 7f99c59d9700 1 --2- 172.21.15.121:0/3000439488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c00b0ee0 0x7f99c00b7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.905+0000 7f99c59d9700 1 -- 172.21.15.121:0/3000439488 >> 172.21.15.121:0/3000439488 conn(0x7f99c0060c40 msgr2=0x7f99c00cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:49.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.905+0000 7f99c59d9700 1 -- 172.21.15.121:0/3000439488 shutdown_connections 2023-07-14T11:52:49.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:49.905+0000 7f99c59d9700 1 -- 172.21.15.121:0/3000439488 wait complete. 2023-07-14T11:52:49.909 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c 5 2023-07-14T11:52:50.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb453f6700 1 Processor -- start 2023-07-14T11:52:50.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb453f6700 1 -- start start 2023-07-14T11:52:50.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb453f6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fcb400cd260 0x7fcb400d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb453f6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcb400d0610 0x7fcb400af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb453f6700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb400d1a60 con 0x7fcb400b0ee0 2023-07-14T11:52:50.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb453f6700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb400d1ef0 con 0x7fcb400cd260 2023-07-14T11:52:50.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb453f6700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb400d0e50 con 0x7fcb400d0610 2023-07-14T11:52:50.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb44bf5700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcb400d0610 0x7fcb400af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb3ffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fcb400b0ee0 0x7fcb400ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58892/0 (socket says 172.21.15.121:58892) 2023-07-14T11:52:50.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb3ffff700 1 -- 172.21.15.121:0/1093129312 learned_addr learned my addr 172.21.15.121:0/1093129312 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-07-14T11:52:50.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb3f7fe700 1 --2- 172.21.15.121:0/1093129312 >> v2:172.21.15.104:3300/0 conn(0x7fcb400cd260 0x7fcb400d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb44bf5700 1 -- 172.21.15.121:0/1093129312 >> v2:172.21.15.104:3300/0 conn(0x7fcb400cd260 msgr2=0x7fcb400d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:50.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb44bf5700 1 --2- 172.21.15.121:0/1093129312 >> v2:172.21.15.104:3300/0 conn(0x7fcb400cd260 0x7fcb400d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb44bf5700 1 -- 172.21.15.121:0/1093129312 >> v1:172.21.15.121:6789/0 conn(0x7fcb400b0ee0 legacy=0x7fcb400ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb44bf5700 1 -- 172.21.15.121:0/1093129312 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb400d1130 con 0x7fcb400d0610 2023-07-14T11:52:50.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.121+0000 7fcb44bf5700 1 --2- 172.21.15.121:0/1093129312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb400d0610 0x7fcb400af0c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcb4008c990 tx=0x7fcb3800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4d29c6eeaf079b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:50.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.125+0000 7fcb3effd700 1 -- 172.21.15.121:0/1093129312 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3800e070 con 0x7fcb400d0610 2023-07-14T11:52:50.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.125+0000 7fcb3effd700 1 -- 172.21.15.121:0/1093129312 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3800bb20 con 0x7fcb400d0610 2023-07-14T11:52:50.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.125+0000 7fcb3effd700 1 -- 172.21.15.121:0/1093129312 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb38012430 con 0x7fcb400d0610 2023-07-14T11:52:50.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.125+0000 7fcb453f6700 1 -- 172.21.15.121:0/1093129312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb400d0610 msgr2=0x7fcb400af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.125+0000 7fcb453f6700 1 --2- 172.21.15.121:0/1093129312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb400d0610 0x7fcb400af0c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcb4008c990 tx=0x7fcb3800b330 comp rx=0 tx=0).stop 2023-07-14T11:52:50.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.125+0000 7fcb453f6700 1 -- 172.21.15.121:0/1093129312 shutdown_connections 2023-07-14T11:52:50.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.125+0000 7fcb453f6700 1 --2- 172.21.15.121:0/1093129312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb400d0610 0x7fcb400af0c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.125+0000 7fcb453f6700 1 --2- 172.21.15.121:0/1093129312 >> v2:172.21.15.104:3300/0 conn(0x7fcb400cd260 0x7fcb400d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.125+0000 7fcb453f6700 1 -- 172.21.15.121:0/1093129312 >> 172.21.15.121:0/1093129312 conn(0x7fcb40060c40 msgr2=0x7fcb4009b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:50.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.125+0000 7fcb453f6700 1 -- 172.21.15.121:0/1093129312 shutdown_connections 2023-07-14T11:52:50.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.125+0000 7fcb453f6700 1 -- 172.21.15.121:0/1093129312 wait complete. 2023-07-14T11:52:50.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.129+0000 7fcb453f6700 1 Processor -- start 2023-07-14T11:52:50.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.129+0000 7fcb453f6700 1 -- start start 2023-07-14T11:52:50.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.129+0000 7fcb453f6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400b0ee0 0x7fcb400c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.129+0000 7fcb453f6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb400cd260 0x7fcb400cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb453f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb400d0610 0x7fcb400af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb453f6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb400ca710 con 0x7fcb400cd260 2023-07-14T11:52:50.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb453f6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb400c6190 con 0x7fcb400d0610 2023-07-14T11:52:50.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb453f6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb400c6470 con 0x7fcb400b0ee0 2023-07-14T11:52:50.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb3ffff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400b0ee0 0x7fcb400c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb3ffff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400b0ee0 0x7fcb400c8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:39696/0 (socket says 172.21.15.121:39696) 2023-07-14T11:52:50.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb3ffff700 1 -- 172.21.15.121:0/941359090 learned_addr learned my addr 172.21.15.121:0/941359090 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-14T11:52:50.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb44bf5700 1 --2- 172.21.15.121:0/941359090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb400d0610 0x7fcb400af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb3f7fe700 1 --2- 172.21.15.121:0/941359090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb400cd260 0x7fcb400cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb3ffff700 1 -- 172.21.15.121:0/941359090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb400d0610 msgr2=0x7fcb400af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb3ffff700 1 --2- 172.21.15.121:0/941359090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb400d0610 0x7fcb400af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb3ffff700 1 -- 172.21.15.121:0/941359090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb400cd260 msgr2=0x7fcb400cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb3ffff700 1 --2- 172.21.15.121:0/941359090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb400cd260 0x7fcb400cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb3ffff700 1 -- 172.21.15.121:0/941359090 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb38009a70 con 0x7fcb400b0ee0 2023-07-14T11:52:50.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb3ffff700 1 --2- 172.21.15.121:0/941359090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400b0ee0 0x7fcb400c8470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcb30001eb0 tx=0x7fcb3000c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:50.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb3cff9700 1 -- 172.21.15.121:0/941359090 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30005d90 con 0x7fcb400b0ee0 2023-07-14T11:52:50.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb3cff9700 1 -- 172.21.15.121:0/941359090 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb300070d0 con 0x7fcb400b0ee0 2023-07-14T11:52:50.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb453f6700 1 -- 172.21.15.121:0/941359090 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb400c6750 con 0x7fcb400b0ee0 2023-07-14T11:52:50.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb3cff9700 1 -- 172.21.15.121:0/941359090 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30007440 con 0x7fcb400b0ee0 2023-07-14T11:52:50.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb453f6700 1 -- 172.21.15.121:0/941359090 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb400ca3a0 con 0x7fcb400b0ee0 2023-07-14T11:52:50.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb3cff9700 1 -- 172.21.15.121:0/941359090 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcb30012400 con 0x7fcb400b0ee0 2023-07-14T11:52:50.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb3cff9700 1 -- 172.21.15.121:0/941359090 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fcb3001b930 con 0x7fcb400b0ee0 2023-07-14T11:52:50.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.133+0000 7fcb453f6700 1 -- 172.21.15.121:0/941359090 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4004e500 con 0x7fcb400b0ee0 2023-07-14T11:52:50.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.141+0000 7fcb3cff9700 1 -- 172.21.15.121:0/941359090 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fcb30017070 con 0x7fcb400b0ee0 2023-07-14T11:52:50.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.221+0000 7fcb453f6700 1 -- 172.21.15.121:0/941359090 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd new", "uuid": "52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c", "id": 5} v 0) v1 -- 0x7fcb400c6940 con 0x7fcb400b0ee0 2023-07-14T11:52:50.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.233+0000 7fcb3cff9700 1 -- 172.21.15.121:0/941359090 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcb3001bbc0 con 0x7fcb400b0ee0 2023-07-14T11:52:50.217 INFO:teuthology.orchestra.run.smithi121.stdout:5 2023-07-14T11:52:50.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.233+0000 7fcb453f6700 1 -- 172.21.15.121:0/941359090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400b0ee0 msgr2=0x7fcb400c8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.233+0000 7fcb453f6700 1 --2- 172.21.15.121:0/941359090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400b0ee0 0x7fcb400c8470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcb30001eb0 tx=0x7fcb3000c7b0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.233+0000 7fcb453f6700 1 -- 172.21.15.121:0/941359090 shutdown_connections 2023-07-14T11:52:50.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.233+0000 7fcb453f6700 1 --2- 172.21.15.121:0/941359090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb400cd260 0x7fcb400cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.233+0000 7fcb453f6700 1 --2- 172.21.15.121:0/941359090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb400d0610 0x7fcb400af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.233+0000 7fcb453f6700 1 --2- 172.21.15.121:0/941359090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400b0ee0 0x7fcb400c8470 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.233+0000 7fcb453f6700 1 -- 172.21.15.121:0/941359090 >> 172.21.15.121:0/941359090 conn(0x7fcb40060c40 msgr2=0x7fcb4015acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:50.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.233+0000 7fcb453f6700 1 -- 172.21.15.121:0/941359090 shutdown_connections 2023-07-14T11:52:50.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.233+0000 7fcb453f6700 1 -- 172.21.15.121:0/941359090 wait complete. 2023-07-14T11:52:50.237 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 8546ebcd-0a98-424c-b845-13ce61b9cea4 6 2023-07-14T11:52:50.343 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:50.357+0000 7fe50f93f2c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T11:52:50.355 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:50.370+0000 7f4cc2ed62c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T11:52:50.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.417+0000 7f56f6777700 1 Processor -- start 2023-07-14T11:52:50.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.417+0000 7f56f6777700 1 -- start start 2023-07-14T11:52:50.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f6777700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f56f015df10 0x7f56f015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f6777700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f56f00c8950 0x7f56f00c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f6777700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f00c9290 con 0x7f56f00afd80 2023-07-14T11:52:50.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f6777700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f015e830 con 0x7f56f015df10 2023-07-14T11:52:50.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f6777700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f0070a30 con 0x7f56f00c8950 2023-07-14T11:52:50.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56e7fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f56f00afd80 0x7f56f00711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58908/0 (socket says 172.21.15.121:58908) 2023-07-14T11:52:50.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56e7fff700 1 -- 172.21.15.121:0/820478249 learned_addr learned my addr 172.21.15.121:0/820478249 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-07-14T11:52:50.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f4d14700 1 --2- 172.21.15.121:0/820478249 >> v2:172.21.15.121:3300/0 conn(0x7f56f00c8950 0x7f56f00c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56effff700 1 --2- 172.21.15.121:0/820478249 >> v2:172.21.15.104:3300/0 conn(0x7f56f015df10 0x7f56f015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56ef7fe700 1 -- 172.21.15.121:0/820478249 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529242991 0 0) 0x7f56d8001110 con 0x7f56f00afd80 2023-07-14T11:52:50.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56ef7fe700 1 -- 172.21.15.121:0/820478249 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d00035d0 con 0x7f56f00afd80 2023-07-14T11:52:50.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f4d14700 1 -- 172.21.15.121:0/820478249 >> v2:172.21.15.104:3300/0 conn(0x7f56f015df10 msgr2=0x7f56f015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:50.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f4d14700 1 --2- 172.21.15.121:0/820478249 >> v2:172.21.15.104:3300/0 conn(0x7f56f015df10 0x7f56f015e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f4d14700 1 -- 172.21.15.121:0/820478249 >> v1:172.21.15.121:6789/0 conn(0x7f56f00afd80 legacy=0x7f56f00711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f4d14700 1 -- 172.21.15.121:0/820478249 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f00b0210 con 0x7f56f00c8950 2023-07-14T11:52:50.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56effff700 1 --2- 172.21.15.121:0/820478249 >> v2:172.21.15.104:3300/0 conn(0x7f56f015df10 0x7f56f015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:52:50.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f4d14700 1 --2- 172.21.15.121:0/820478249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56f00c8950 0x7f56f00c8d50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f56f00c88e0 tx=0x7f56e800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73cdca99c82b1d8c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:50.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56ef7fe700 1 -- 172.21.15.121:0/820478249 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e800e070 con 0x7f56f00c8950 2023-07-14T11:52:50.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56ef7fe700 1 -- 172.21.15.121:0/820478249 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e800bb20 con 0x7f56f00c8950 2023-07-14T11:52:50.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56ef7fe700 1 -- 172.21.15.121:0/820478249 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e800be90 con 0x7f56f00c8950 2023-07-14T11:52:50.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f6777700 1 -- 172.21.15.121:0/820478249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56f00c8950 msgr2=0x7f56f00c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f6777700 1 --2- 172.21.15.121:0/820478249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56f00c8950 0x7f56f00c8d50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f56f00c88e0 tx=0x7f56e800b330 comp rx=0 tx=0).stop 2023-07-14T11:52:50.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f6777700 1 -- 172.21.15.121:0/820478249 shutdown_connections 2023-07-14T11:52:50.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f6777700 1 --2- 172.21.15.121:0/820478249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56f00c8950 0x7f56f00c8d50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f6777700 1 --2- 172.21.15.121:0/820478249 >> v2:172.21.15.104:3300/0 conn(0x7f56f015df10 0x7f56f015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.421+0000 7f56f6777700 1 -- 172.21.15.121:0/820478249 >> 172.21.15.121:0/820478249 conn(0x7f56f00cad80 msgr2=0x7f56f00cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:50.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.425+0000 7f56f6777700 1 -- 172.21.15.121:0/820478249 shutdown_connections 2023-07-14T11:52:50.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.425+0000 7f56f6777700 1 -- 172.21.15.121:0/820478249 wait complete. 2023-07-14T11:52:50.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.425+0000 7f56f6777700 1 Processor -- start 2023-07-14T11:52:50.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.425+0000 7f56f6777700 1 -- start start 2023-07-14T11:52:50.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.425+0000 7f56f6777700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56f00afd80 0x7f56f00cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.425+0000 7f56f6777700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56f00c8950 0x7f56f01eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.425+0000 7f56f6777700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56f00c53d0 0x7f56f006d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.425+0000 7f56f6777700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00c60b0 con 0x7f56f00afd80 2023-07-14T11:52:50.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.425+0000 7f56f6777700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00cc410 con 0x7f56f00c53d0 2023-07-14T11:52:50.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.425+0000 7f56f6777700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f01f1af0 con 0x7f56f00c8950 2023-07-14T11:52:50.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56f4d14700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56f00c53d0 0x7f56f006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56e7fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56f00c8950 0x7f56f01eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56f4d14700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56f00c53d0 0x7f56f006d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44834/0 (socket says 172.21.15.121:44834) 2023-07-14T11:52:50.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56f4d14700 1 -- 172.21.15.121:0/908054301 learned_addr learned my addr 172.21.15.121:0/908054301 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-14T11:52:50.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56effff700 1 --2- 172.21.15.121:0/908054301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56f00afd80 0x7f56f00cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56f4d14700 1 -- 172.21.15.121:0/908054301 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56f00c8950 msgr2=0x7f56f01eeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56f4d14700 1 --2- 172.21.15.121:0/908054301 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56f00c8950 0x7f56f01eeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56f4d14700 1 -- 172.21.15.121:0/908054301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56f00afd80 msgr2=0x7f56f00cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56f4d14700 1 --2- 172.21.15.121:0/908054301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56f00afd80 0x7f56f00cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56f4d14700 1 -- 172.21.15.121:0/908054301 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e8009a70 con 0x7f56f00c53d0 2023-07-14T11:52:50.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56effff700 1 --2- 172.21.15.121:0/908054301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56f00afd80 0x7f56f00cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:52:50.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56f4d14700 1 --2- 172.21.15.121:0/908054301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56f00c53d0 0x7f56f006d740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f56e800b370 tx=0x7f56e8004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:50.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56ed7fa700 1 -- 172.21.15.121:0/908054301 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8004930 con 0x7f56f00c53d0 2023-07-14T11:52:50.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56ed7fa700 1 -- 172.21.15.121:0/908054301 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e8020070 con 0x7f56f00c53d0 2023-07-14T11:52:50.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56f6777700 1 -- 172.21.15.121:0/908054301 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f01f1dd0 con 0x7f56f00c53d0 2023-07-14T11:52:50.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56ed7fa700 1 -- 172.21.15.121:0/908054301 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e80276c0 con 0x7f56f00c53d0 2023-07-14T11:52:50.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56f6777700 1 -- 172.21.15.121:0/908054301 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f01f2390 con 0x7f56f00c53d0 2023-07-14T11:52:50.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56ed7fa700 1 -- 172.21.15.121:0/908054301 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f56e8027b20 con 0x7f56f00c53d0 2023-07-14T11:52:50.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56ed7fa700 1 -- 172.21.15.121:0/908054301 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f56e8022070 con 0x7f56f00c53d0 2023-07-14T11:52:50.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.429+0000 7f56f6777700 1 -- 172.21.15.121:0/908054301 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f004dcb0 con 0x7f56f00c53d0 2023-07-14T11:52:50.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.433+0000 7f56ed7fa700 1 -- 172.21.15.121:0/908054301 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f56e8012430 con 0x7f56f00c53d0 2023-07-14T11:52:50.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.509+0000 7f56f6777700 1 -- 172.21.15.121:0/908054301 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8546ebcd-0a98-424c-b845-13ce61b9cea4", "id": 6} v 0) v1 -- 0x7f56f01f4dd0 con 0x7f56f00c53d0 2023-07-14T11:52:50.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.521+0000 7f56ed7fa700 1 -- 172.21.15.121:0/908054301 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8546ebcd-0a98-424c-b845-13ce61b9cea4", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f56e8012890 con 0x7f56f00c53d0 2023-07-14T11:52:50.507 INFO:teuthology.orchestra.run.smithi121.stdout:6 2023-07-14T11:52:50.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.525+0000 7f56f6777700 1 -- 172.21.15.121:0/908054301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56f00c53d0 msgr2=0x7f56f006d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.525+0000 7f56f6777700 1 --2- 172.21.15.121:0/908054301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56f00c53d0 0x7f56f006d740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f56e800b370 tx=0x7f56e8004af0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.525+0000 7f56f6777700 1 -- 172.21.15.121:0/908054301 shutdown_connections 2023-07-14T11:52:50.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.525+0000 7f56f6777700 1 --2- 172.21.15.121:0/908054301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56f00c53d0 0x7f56f006d740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.525+0000 7f56f6777700 1 --2- 172.21.15.121:0/908054301 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56f00c8950 0x7f56f01eeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.525+0000 7f56f6777700 1 --2- 172.21.15.121:0/908054301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56f00afd80 0x7f56f00cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.525+0000 7f56f6777700 1 -- 172.21.15.121:0/908054301 >> 172.21.15.121:0/908054301 conn(0x7f56f00cad80 msgr2=0x7f56f00cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:50.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.525+0000 7f56f6777700 1 -- 172.21.15.121:0/908054301 shutdown_connections 2023-07-14T11:52:50.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.525+0000 7f56f6777700 1 -- 172.21.15.121:0/908054301 wait complete. 2023-07-14T11:52:50.521 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:50.537+0000 7fe50f93f2c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T11:52:50.528 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new c406079a-1e13-45eb-850c-f2eb69790571 7 2023-07-14T11:52:50.531 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:50.545+0000 7f4cc2ed62c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T11:52:50.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.705+0000 7f6fdbfff700 1 Processor -- start 2023-07-14T11:52:50.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.705+0000 7f6fdbfff700 1 -- start start 2023-07-14T11:52:50.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdbfff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fdc159c60 0x7f6fdc06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdbfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6fdc0cab00 0x7f6fdc0caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdbfff700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc0cb4c0 con 0x7f6fdc0b0770 2023-07-14T11:52:50.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdbfff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fdc0714f0 con 0x7f6fdc0cab00 2023-07-14T11:52:50.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdbfff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc06da00 con 0x7f6fdc159c60 2023-07-14T11:52:50.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fd3fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6fdc0b0770 0x7f6fdc070de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58914/0 (socket says 172.21.15.121:58914) 2023-07-14T11:52:50.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fd3fff700 1 -- 172.21.15.121:0/4251992190 learned_addr learned my addr 172.21.15.121:0/4251992190 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-07-14T11:52:50.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdb7fe700 1 --2- 172.21.15.121:0/4251992190 >> v2:172.21.15.121:3300/0 conn(0x7f6fdc0cab00 0x7f6fdc0caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdaffd700 1 --2- 172.21.15.121:0/4251992190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fdc159c60 0x7f6fdc06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdb7fe700 1 -- 172.21.15.121:0/4251992190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fdc159c60 msgr2=0x7f6fdc06d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:50.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdb7fe700 1 --2- 172.21.15.121:0/4251992190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fdc159c60 0x7f6fdc06d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdb7fe700 1 -- 172.21.15.121:0/4251992190 >> v1:172.21.15.121:6789/0 conn(0x7f6fdc0b0770 legacy=0x7f6fdc070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdb7fe700 1 -- 172.21.15.121:0/4251992190 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc06dce0 con 0x7f6fdc0cab00 2023-07-14T11:52:50.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdaffd700 1 --2- 172.21.15.121:0/4251992190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fdc159c60 0x7f6fdc06d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:52:50.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdb7fe700 1 --2- 172.21.15.121:0/4251992190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fdc0cab00 0x7f6fdc0caf80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009d90 tx=0x7f6fd400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d164637ae5f019d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:50.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fda7fc700 1 -- 172.21.15.121:0/4251992190 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd400e070 con 0x7f6fdc0cab00 2023-07-14T11:52:50.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fda7fc700 1 -- 172.21.15.121:0/4251992190 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd400bb20 con 0x7f6fdc0cab00 2023-07-14T11:52:50.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fda7fc700 1 -- 172.21.15.121:0/4251992190 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4012430 con 0x7f6fdc0cab00 2023-07-14T11:52:50.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdbfff700 1 -- 172.21.15.121:0/4251992190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fdc0cab00 msgr2=0x7f6fdc0caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdbfff700 1 --2- 172.21.15.121:0/4251992190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fdc0cab00 0x7f6fdc0caf80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009d90 tx=0x7f6fd400b330 comp rx=0 tx=0).stop 2023-07-14T11:52:50.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdbfff700 1 -- 172.21.15.121:0/4251992190 shutdown_connections 2023-07-14T11:52:50.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdbfff700 1 --2- 172.21.15.121:0/4251992190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fdc0cab00 0x7f6fdc0caf80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdbfff700 1 --2- 172.21.15.121:0/4251992190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fdc159c60 0x7f6fdc06d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdbfff700 1 -- 172.21.15.121:0/4251992190 >> 172.21.15.121:0/4251992190 conn(0x7f6fdc0afc60 msgr2=0x7f6fdc0b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:50.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.709+0000 7f6fdbfff700 1 -- 172.21.15.121:0/4251992190 shutdown_connections 2023-07-14T11:52:50.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.713+0000 7f6fdbfff700 1 -- 172.21.15.121:0/4251992190 wait complete. 2023-07-14T11:52:50.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.713+0000 7f6fdbfff700 1 Processor -- start 2023-07-14T11:52:50.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.713+0000 7f6fdbfff700 1 -- start start 2023-07-14T11:52:50.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fdbfff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fdc0b0770 0x7f6fdc1f4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fdbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fdc0cab00 0x7f6fdc0cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fdbfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fdc159c60 0x7f6fdc0cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fdbfff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc0c8e60 con 0x7f6fdc159c60 2023-07-14T11:52:50.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fdbfff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc0c9140 con 0x7f6fdc0cab00 2023-07-14T11:52:50.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fdbfff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc0aba90 con 0x7f6fdc0b0770 2023-07-14T11:52:50.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fdaffd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fdc0b0770 0x7f6fdc1f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fd3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fdc0cab00 0x7f6fdc0cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fdaffd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fdc0b0770 0x7f6fdc1f4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:39730/0 (socket says 172.21.15.121:39730) 2023-07-14T11:52:50.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fdaffd700 1 -- 172.21.15.121:0/776605481 learned_addr learned my addr 172.21.15.121:0/776605481 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-14T11:52:50.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fdb7fe700 1 --2- 172.21.15.121:0/776605481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fdc159c60 0x7f6fdc0cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fd3fff700 1 -- 172.21.15.121:0/776605481 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fdc0b0770 msgr2=0x7f6fdc1f4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fd3fff700 1 --2- 172.21.15.121:0/776605481 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fdc0b0770 0x7f6fdc1f4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fd3fff700 1 -- 172.21.15.121:0/776605481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fdc159c60 msgr2=0x7f6fdc0cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fd3fff700 1 --2- 172.21.15.121:0/776605481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fdc159c60 0x7f6fdc0cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fd3fff700 1 -- 172.21.15.121:0/776605481 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc0094b0 con 0x7f6fdc0cab00 2023-07-14T11:52:50.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fd3fff700 1 --2- 172.21.15.121:0/776605481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fdc0cab00 0x7f6fdc0cc8e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400c430 tx=0x7f6fc400c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:50.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fd37fe700 1 -- 172.21.15.121:0/776605481 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4005d90 con 0x7f6fdc0cab00 2023-07-14T11:52:50.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fd37fe700 1 -- 172.21.15.121:0/776605481 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc40070d0 con 0x7f6fdc0cab00 2023-07-14T11:52:50.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fdbfff700 1 -- 172.21.15.121:0/776605481 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd4009a70 con 0x7f6fdc0cab00 2023-07-14T11:52:50.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fd37fe700 1 -- 172.21.15.121:0/776605481 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4007440 con 0x7f6fdc0cab00 2023-07-14T11:52:50.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fdbfff700 1 -- 172.21.15.121:0/776605481 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc1f4560 con 0x7f6fdc0cab00 2023-07-14T11:52:50.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fd37fe700 1 -- 172.21.15.121:0/776605481 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6fc401c440 con 0x7f6fdc0cab00 2023-07-14T11:52:50.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.717+0000 7f6fd37fe700 1 -- 172.21.15.121:0/776605481 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f6fc401cd50 con 0x7f6fdc0cab00 2023-07-14T11:52:50.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.721+0000 7f6fdbfff700 1 -- 172.21.15.121:0/776605481 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fdc0ab8f0 con 0x7f6fdc0cab00 2023-07-14T11:52:50.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.721+0000 7f6fd37fe700 1 -- 172.21.15.121:0/776605481 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f6fdc0ab8f0 con 0x7f6fdc0cab00 2023-07-14T11:52:50.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.805+0000 7f6fdbfff700 1 -- 172.21.15.121:0/776605481 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c406079a-1e13-45eb-850c-f2eb69790571", "id": 7} v 0) v1 -- 0x7f6fdc04e500 con 0x7f6fdc0cab00 2023-07-14T11:52:50.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.817+0000 7f6fd37fe700 1 -- 172.21.15.121:0/776605481 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c406079a-1e13-45eb-850c-f2eb69790571", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6fc4021060 con 0x7f6fdc0cab00 2023-07-14T11:52:50.802 INFO:teuthology.orchestra.run.smithi121.stdout:7 2023-07-14T11:52:50.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.817+0000 7f6fdbfff700 1 -- 172.21.15.121:0/776605481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fdc0cab00 msgr2=0x7f6fdc0cc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.817+0000 7f6fdbfff700 1 --2- 172.21.15.121:0/776605481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fdc0cab00 0x7f6fdc0cc8e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400c430 tx=0x7f6fc400c740 comp rx=0 tx=0).stop 2023-07-14T11:52:50.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.817+0000 7f6fdbfff700 1 -- 172.21.15.121:0/776605481 shutdown_connections 2023-07-14T11:52:50.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.817+0000 7f6fdbfff700 1 --2- 172.21.15.121:0/776605481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fdc159c60 0x7f6fdc0cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.817+0000 7f6fdbfff700 1 --2- 172.21.15.121:0/776605481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fdc0cab00 0x7f6fdc0cc8e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.817+0000 7f6fdbfff700 1 --2- 172.21.15.121:0/776605481 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fdc0b0770 0x7f6fdc1f4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.817+0000 7f6fdbfff700 1 -- 172.21.15.121:0/776605481 >> 172.21.15.121:0/776605481 conn(0x7f6fdc0afc60 msgr2=0x7f6fdc070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:50.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.817+0000 7f6fdbfff700 1 -- 172.21.15.121:0/776605481 shutdown_connections 2023-07-14T11:52:50.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-14T11:52:50.817+0000 7f6fdbfff700 1 -- 172.21.15.121:0/776605481 wait complete. 2023-07-14T11:52:50.826 INFO:tasks.ceph.osd.0:Restarting daemon 2023-07-14T11:52:50.827 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-07-14T11:52:50.832 INFO:tasks.ceph.osd.0:Started 2023-07-14T11:52:50.832 INFO:tasks.ceph.osd.1:Restarting daemon 2023-07-14T11:52:50.833 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-07-14T11:52:50.837 INFO:tasks.ceph.osd.1:Started 2023-07-14T11:52:50.838 INFO:tasks.ceph.osd.2:Restarting daemon 2023-07-14T11:52:50.838 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-07-14T11:52:50.842 INFO:tasks.ceph.osd.2:Started 2023-07-14T11:52:50.843 INFO:tasks.ceph.osd.3:Restarting daemon 2023-07-14T11:52:50.843 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-07-14T11:52:50.847 INFO:tasks.ceph.osd.3:Started 2023-07-14T11:52:50.848 INFO:tasks.ceph.osd.4:Restarting daemon 2023-07-14T11:52:50.848 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-07-14T11:52:50.853 INFO:tasks.ceph.osd.4:Started 2023-07-14T11:52:50.854 INFO:tasks.ceph.osd.5:Restarting daemon 2023-07-14T11:52:50.854 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-07-14T11:52:50.859 INFO:tasks.ceph.osd.5:Started 2023-07-14T11:52:50.859 INFO:tasks.ceph.osd.6:Restarting daemon 2023-07-14T11:52:50.859 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-07-14T11:52:50.864 INFO:tasks.ceph.osd.6:Started 2023-07-14T11:52:50.865 INFO:tasks.ceph.osd.7:Restarting daemon 2023-07-14T11:52:50.865 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-07-14T11:52:50.870 INFO:tasks.ceph.osd.7:Started 2023-07-14T11:52:50.870 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:52:50.923 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:50.937+0000 7fe50f93f2c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T11:52:50.941 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:50.953+0000 7f4cc2ed62c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T11:52:51.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.057+0000 7f4fecb5f700 1 Processor -- start 2023-07-14T11:52:51.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.057+0000 7f4fecb5f700 1 -- start start 2023-07-14T11:52:51.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.061+0000 7f4fecb5f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fe80c9d90 0x7f4fe80ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:51.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.061+0000 7f4fecb5f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4fe8159f20 0x7f4fe80caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:51.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.061+0000 7f4fecb5f700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe80b0ef0 con 0x7f4fe80caf70 2023-07-14T11:52:51.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.061+0000 7f4fecb5f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe80ce9c0 con 0x7f4fe8159f20 2023-07-14T11:52:51.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.061+0000 7f4fecb5f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80cee50 con 0x7f4fe80c9d90 2023-07-14T11:52:51.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.061+0000 7f4fe7fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f4fe80caf70 0x7f4fe80c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42458/0 (socket says 172.21.15.104:42458) 2023-07-14T11:52:51.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.061+0000 7f4fe7fff700 1 -- 172.21.15.104:0/485367069 learned_addr learned my addr 172.21.15.104:0/485367069 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:52:51.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.061+0000 7f4fe77fe700 1 --2- 172.21.15.104:0/485367069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fe80c9d90 0x7f4fe80ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:51.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.061+0000 7f4fe6ffd700 1 --2- 172.21.15.104:0/485367069 >> v2:172.21.15.121:3300/0 conn(0x7f4fe8159f20 0x7f4fe80caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:51.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.061+0000 7f4fe77fe700 1 -- 172.21.15.104:0/485367069 >> v2:172.21.15.121:3300/0 conn(0x7f4fe8159f20 msgr2=0x7f4fe80caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:51.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.061+0000 7f4fe77fe700 1 --2- 172.21.15.104:0/485367069 >> v2:172.21.15.121:3300/0 conn(0x7f4fe8159f20 0x7f4fe80caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:51.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.061+0000 7f4fe77fe700 1 -- 172.21.15.104:0/485367069 >> v1:172.21.15.104:6789/0 conn(0x7f4fe80caf70 legacy=0x7f4fe80c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:51.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.061+0000 7f4fe77fe700 1 -- 172.21.15.104:0/485367069 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe80ca6a0 con 0x7f4fe80c9d90 2023-07-14T11:52:51.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.061+0000 7f4fe67fc700 1 -- 172.21.15.104:0/485367069 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113237891 0 0) 0x7f4fe80b0ef0 con 0x7f4fe80caf70 2023-07-14T11:52:51.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.061+0000 7f4fe77fe700 1 --2- 172.21.15.104:0/485367069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fe80c9d90 0x7f4fe80ca160 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009850 tx=0x7f4fd8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b8a2e152b88bdce server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:51.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.065+0000 7f4fe67fc700 1 -- 172.21.15.104:0/485367069 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8011830 con 0x7f4fe80c9d90 2023-07-14T11:52:51.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.065+0000 7f4fe67fc700 1 -- 172.21.15.104:0/485367069 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd8011990 con 0x7f4fe80c9d90 2023-07-14T11:52:51.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.065+0000 7f4fe67fc700 1 -- 172.21.15.104:0/485367069 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8011d40 con 0x7f4fe80c9d90 2023-07-14T11:52:51.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.065+0000 7f4fecb5f700 1 -- 172.21.15.104:0/485367069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fe80c9d90 msgr2=0x7f4fe80ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:51.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.065+0000 7f4fecb5f700 1 --2- 172.21.15.104:0/485367069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fe80c9d90 0x7f4fe80ca160 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009850 tx=0x7f4fd8009b60 comp rx=0 tx=0).stop 2023-07-14T11:52:51.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.065+0000 7f4fecb5f700 1 -- 172.21.15.104:0/485367069 shutdown_connections 2023-07-14T11:52:51.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.065+0000 7f4fecb5f700 1 --2- 172.21.15.104:0/485367069 >> v2:172.21.15.121:3300/0 conn(0x7f4fe8159f20 0x7f4fe80caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:51.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.065+0000 7f4fecb5f700 1 --2- 172.21.15.104:0/485367069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fe80c9d90 0x7f4fe80ca160 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:51.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.065+0000 7f4fecb5f700 1 -- 172.21.15.104:0/485367069 >> 172.21.15.104:0/485367069 conn(0x7f4fe806e960 msgr2=0x7f4fe806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:51.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.065+0000 7f4fecb5f700 1 -- 172.21.15.104:0/485367069 shutdown_connections 2023-07-14T11:52:51.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.065+0000 7f4fecb5f700 1 -- 172.21.15.104:0/485367069 wait complete. 2023-07-14T11:52:51.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.065+0000 7f4fecb5f700 1 Processor -- start 2023-07-14T11:52:51.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.065+0000 7f4fecb5f700 1 -- start start 2023-07-14T11:52:51.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fecb5f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe80c9d90 0x7f4fe80c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:51.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fecb5f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fe80caf70 0x7f4fe814d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:51.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fecb5f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fe8159f20 0x7f4fe8156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:51.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fecb5f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe8157dd0 con 0x7f4fe80c9d90 2023-07-14T11:52:51.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fecb5f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe81f2130 con 0x7f4fe8159f20 2023-07-14T11:52:51.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fecb5f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80cc4a0 con 0x7f4fe80caf70 2023-07-14T11:52:51.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fe77fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe80c9d90 0x7f4fe80c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:51.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fe77fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe80c9d90 0x7f4fe80c3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45262/0 (socket says 172.21.15.104:45262) 2023-07-14T11:52:51.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fe77fe700 1 -- 172.21.15.104:0/2192520661 learned_addr learned my addr 172.21.15.104:0/2192520661 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:52:51.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fe6ffd700 1 --2- 172.21.15.104:0/2192520661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fe80caf70 0x7f4fe814d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:51.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fe7fff700 1 --2- 172.21.15.104:0/2192520661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fe8159f20 0x7f4fe8156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:51.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fe77fe700 1 -- 172.21.15.104:0/2192520661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fe80caf70 msgr2=0x7f4fe814d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:51.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fe77fe700 1 --2- 172.21.15.104:0/2192520661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fe80caf70 0x7f4fe814d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:51.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fe77fe700 1 -- 172.21.15.104:0/2192520661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fe8159f20 msgr2=0x7f4fe8156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:51.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fe77fe700 1 --2- 172.21.15.104:0/2192520661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fe8159f20 0x7f4fe8156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:51.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fe77fe700 1 -- 172.21.15.104:0/2192520661 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd80094b0 con 0x7f4fe80c9d90 2023-07-14T11:52:51.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fe77fe700 1 --2- 172.21.15.104:0/2192520661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe80c9d90 0x7f4fe80c3180 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8014040 tx=0x7f4fd8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:51.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fcffff700 1 -- 172.21.15.104:0/2192520661 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8019a00 con 0x7f4fe80c9d90 2023-07-14T11:52:51.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fcffff700 1 -- 172.21.15.104:0/2192520661 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd801e070 con 0x7f4fe80c9d90 2023-07-14T11:52:51.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fecb5f700 1 -- 172.21.15.104:0/2192520661 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe80cc780 con 0x7f4fe80c9d90 2023-07-14T11:52:51.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fcffff700 1 -- 172.21.15.104:0/2192520661 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8019da0 con 0x7f4fe80c9d90 2023-07-14T11:52:51.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fecb5f700 1 -- 172.21.15.104:0/2192520661 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe814da30 con 0x7f4fe80c9d90 2023-07-14T11:52:51.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fcffff700 1 -- 172.21.15.104:0/2192520661 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4fd802d750 con 0x7f4fe80c9d90 2023-07-14T11:52:51.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.069+0000 7f4fcffff700 1 -- 172.21.15.104:0/2192520661 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4fd801ad20 con 0x7f4fe80c9d90 2023-07-14T11:52:51.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.073+0000 7f4fecb5f700 1 -- 172.21.15.104:0/2192520661 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe804e500 con 0x7f4fe80c9d90 2023-07-14T11:52:51.064 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.077+0000 7f4fcffff700 1 -- 172.21.15.104:0/2192520661 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f4fd8020070 con 0x7f4fe80c9d90 2023-07-14T11:52:51.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.145+0000 7f4fecb5f700 1 -- 172.21.15.104:0/2192520661 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fe8159740 con 0x7f4fe80c9d90 2023-07-14T11:52:51.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.145+0000 7f4fcffff700 1 -- 172.21.15.104:0/2192520661 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4fd8015020 con 0x7f4fe80c9d90 2023-07-14T11:52:51.133 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:52:51.134 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":10,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:52:50.810860+0000","last_up_change":"0.000000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:52:51.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.145+0000 7f4fecb5f700 1 -- 172.21.15.104:0/2192520661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe80c9d90 msgr2=0x7f4fe80c3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:51.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.145+0000 7f4fecb5f700 1 --2- 172.21.15.104:0/2192520661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe80c9d90 0x7f4fe80c3180 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8014040 tx=0x7f4fd8004770 comp rx=0 tx=0).stop 2023-07-14T11:52:51.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.149+0000 7f4fecb5f700 1 -- 172.21.15.104:0/2192520661 shutdown_connections 2023-07-14T11:52:51.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.149+0000 7f4fecb5f700 1 --2- 172.21.15.104:0/2192520661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fe8159f20 0x7f4fe8156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:51.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.149+0000 7f4fecb5f700 1 --2- 172.21.15.104:0/2192520661 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fe80caf70 0x7f4fe814d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:51.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.149+0000 7f4fecb5f700 1 --2- 172.21.15.104:0/2192520661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe80c9d90 0x7f4fe80c3180 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:51.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.149+0000 7f4fecb5f700 1 -- 172.21.15.104:0/2192520661 >> 172.21.15.104:0/2192520661 conn(0x7f4fe806e960 msgr2=0x7f4fe815bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:51.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.149+0000 7f4fecb5f700 1 -- 172.21.15.104:0/2192520661 shutdown_connections 2023-07-14T11:52:51.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:52:51.149+0000 7f4fecb5f700 1 -- 172.21.15.104:0/2192520661 wait complete. 2023-07-14T11:52:51.153 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-07-14T11:52:51.153 INFO:tasks.ceph:Waiting for OSDs to come up 2023-07-14T11:52:51.174 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:51.189+0000 7fe50f93f2c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:52:51.190 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:51.201+0000 7f4cc2ed62c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:52:51.264 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:51.277+0000 7fe50f93f2c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T11:52:51.281 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:51.293+0000 7f4cc2ed62c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T11:52:51.343 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:51.357+0000 7fe50f93f2c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T11:52:51.361 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:51.373+0000 7f4cc2ed62c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T11:52:51.520 INFO:tasks.ceph.osd.5.smithi121.stderr:2023-07-14T11:52:51.533+0000 7f4a761e6700 -1 Falling back to public interface 2023-07-14T11:52:51.520 INFO:tasks.ceph.osd.7.smithi121.stderr:2023-07-14T11:52:51.537+0000 7f2e77e22700 -1 Falling back to public interface 2023-07-14T11:52:51.530 INFO:tasks.ceph.osd.2.smithi104.stderr:2023-07-14T11:52:51.541+0000 7fa2d5a70700 -1 Falling back to public interface 2023-07-14T11:52:51.530 INFO:tasks.ceph.osd.0.smithi104.stderr:2023-07-14T11:52:51.541+0000 7ff40328f700 -1 Falling back to public interface 2023-07-14T11:52:51.544 INFO:tasks.ceph.osd.3.smithi104.stderr:2023-07-14T11:52:51.557+0000 7f42ea575700 -1 Falling back to public interface 2023-07-14T11:52:51.545 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-07-14T11:52:51.561+0000 7f13e4df3700 -1 Falling back to public interface 2023-07-14T11:52:51.553 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-07-14T11:52:51.569+0000 7fd4d7f72700 -1 Falling back to public interface 2023-07-14T11:52:51.555 INFO:tasks.ceph.osd.1.smithi104.stderr:2023-07-14T11:52:51.569+0000 7f5f7d10f700 -1 Falling back to public interface 2023-07-14T11:52:51.957 DEBUG:teuthology.orchestra.run.smithi104:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-14T11:52:52.109 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.121+0000 7f4eafa89700 1 Processor -- start 2023-07-14T11:52:52.109 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.121+0000 7f4eafa89700 1 -- start start 2023-07-14T11:52:52.110 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.121+0000 7f4eafa89700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ea8160650 0x7f4ea80abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:52.111 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4eafa89700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4ea80c1930 0x7f4ea80cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:52.111 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4eafa89700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea80b2000 con 0x7f4ea806ce90 2023-07-14T11:52:52.111 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4eafa89700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea80b22e0 con 0x7f4ea80c1930 2023-07-14T11:52:52.111 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4eafa89700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea80b25c0 con 0x7f4ea8160650 2023-07-14T11:52:52.111 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4ead024700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f4ea806ce90 0x7f4ea80c1220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42464/0 (socket says 172.21.15.104:42464) 2023-07-14T11:52:52.111 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4ead024700 1 -- 172.21.15.104:0/2383166164 learned_addr learned my addr 172.21.15.104:0/2383166164 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:52:52.112 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4eae026700 1 --2- 172.21.15.104:0/2383166164 >> v2:172.21.15.121:3300/0 conn(0x7f4ea80c1930 0x7f4ea80cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:52.112 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4ead825700 1 --2- 172.21.15.104:0/2383166164 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ea8160650 0x7f4ea80abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:52.113 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4eac823700 1 -- 172.21.15.104:0/2383166164 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723850988 0 0) 0x7f4ea80b2000 con 0x7f4ea806ce90 2023-07-14T11:52:52.113 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4eac823700 1 -- 172.21.15.104:0/2383166164 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e940035d0 con 0x7f4ea806ce90 2023-07-14T11:52:52.113 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4eac823700 1 -- 172.21.15.104:0/2383166164 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2474063854 0 0) 0x7f4e940035d0 con 0x7f4ea806ce90 2023-07-14T11:52:52.113 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4ead825700 1 -- 172.21.15.104:0/2383166164 >> v2:172.21.15.121:3300/0 conn(0x7f4ea80c1930 msgr2=0x7f4ea80cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:52.113 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4ead825700 1 --2- 172.21.15.104:0/2383166164 >> v2:172.21.15.121:3300/0 conn(0x7f4ea80c1930 0x7f4ea80cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:52.113 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4ead825700 1 -- 172.21.15.104:0/2383166164 >> v1:172.21.15.104:6789/0 conn(0x7f4ea806ce90 legacy=0x7f4ea80c1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:52.113 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4ead825700 1 -- 172.21.15.104:0/2383166164 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea80b28a0 con 0x7f4ea8160650 2023-07-14T11:52:52.113 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4ead825700 1 --2- 172.21.15.104:0/2383166164 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ea8160650 0x7f4ea80abe80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009de0 tx=0x7f4ea400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2954e2a6e285a6f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:52.114 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4eac823700 1 -- 172.21.15.104:0/2383166164 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea400e070 con 0x7f4ea8160650 2023-07-14T11:52:52.115 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4eac823700 1 -- 172.21.15.104:0/2383166164 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea4003a40 con 0x7f4ea8160650 2023-07-14T11:52:52.115 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4eac823700 1 -- 172.21.15.104:0/2383166164 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98001670 con 0x7f4ea8160650 2023-07-14T11:52:52.115 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4eafa89700 1 -- 172.21.15.104:0/2383166164 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ea8160650 msgr2=0x7f4ea80abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:52.115 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.125+0000 7f4eafa89700 1 --2- 172.21.15.104:0/2383166164 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ea8160650 0x7f4ea80abe80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009de0 tx=0x7f4ea400b330 comp rx=0 tx=0).stop 2023-07-14T11:52:52.115 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.129+0000 7f4eafa89700 1 -- 172.21.15.104:0/2383166164 shutdown_connections 2023-07-14T11:52:52.115 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.129+0000 7f4eafa89700 1 --2- 172.21.15.104:0/2383166164 >> v2:172.21.15.121:3300/0 conn(0x7f4ea80c1930 0x7f4ea80cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:52.115 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.129+0000 7f4eafa89700 1 --2- 172.21.15.104:0/2383166164 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ea8160650 0x7f4ea80abe80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:52.116 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.129+0000 7f4eafa89700 1 -- 172.21.15.104:0/2383166164 >> 172.21.15.104:0/2383166164 conn(0x7f4ea80cacd0 msgr2=0x7f4ea80cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:52.116 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.129+0000 7f4eafa89700 1 -- 172.21.15.104:0/2383166164 shutdown_connections 2023-07-14T11:52:52.116 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.129+0000 7f4eafa89700 1 -- 172.21.15.104:0/2383166164 wait complete. 2023-07-14T11:52:52.117 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.129+0000 7f4eafa89700 1 Processor -- start 2023-07-14T11:52:52.118 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.129+0000 7f4eafa89700 1 -- start start 2023-07-14T11:52:52.119 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4eafa89700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ea806ce90 0x7f4ea815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:52.120 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4eafa89700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ea80c1930 0x7f4ea80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:52.120 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4eafa89700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ea8160650 0x7f4ea80ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:52.120 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4eafa89700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea81e8f20 con 0x7f4ea806ce90 2023-07-14T11:52:52.120 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4eafa89700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea806ab70 con 0x7f4ea80c1930 2023-07-14T11:52:52.120 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4eafa89700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea806ae50 con 0x7f4ea8160650 2023-07-14T11:52:52.121 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4ead825700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ea806ce90 0x7f4ea815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:52.122 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4ead825700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ea806ce90 0x7f4ea815d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45276/0 (socket says 172.21.15.104:45276) 2023-07-14T11:52:52.122 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4ead825700 1 -- 172.21.15.104:0/1990084769 learned_addr learned my addr 172.21.15.104:0/1990084769 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:52:52.122 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4eae026700 1 --2- 172.21.15.104:0/1990084769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ea8160650 0x7f4ea80ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:52.122 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4ead024700 1 --2- 172.21.15.104:0/1990084769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ea80c1930 0x7f4ea80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:52.122 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4ead825700 1 -- 172.21.15.104:0/1990084769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ea8160650 msgr2=0x7f4ea80ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:52.122 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4ead825700 1 --2- 172.21.15.104:0/1990084769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ea8160650 0x7f4ea80ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:52.122 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4ead825700 1 -- 172.21.15.104:0/1990084769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ea80c1930 msgr2=0x7f4ea80c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:52.122 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4ead825700 1 --2- 172.21.15.104:0/1990084769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ea80c1930 0x7f4ea80c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:52.123 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4ead825700 1 -- 172.21.15.104:0/1990084769 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea4009a70 con 0x7f4ea806ce90 2023-07-14T11:52:52.123 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4ead024700 1 --2- 172.21.15.104:0/1990084769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ea80c1930 0x7f4ea80c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:52:52.123 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4ead825700 1 --2- 172.21.15.104:0/1990084769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ea806ce90 0x7f4ea815d9e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009790 tx=0x7f4ea4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:52.123 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4e9e7fc700 1 -- 172.21.15.104:0/1990084769 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4004580 con 0x7f4ea806ce90 2023-07-14T11:52:52.123 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4e9e7fc700 1 -- 172.21.15.104:0/1990084769 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea40046e0 con 0x7f4ea806ce90 2023-07-14T11:52:52.123 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4eafa89700 1 -- 172.21.15.104:0/1990084769 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea806b130 con 0x7f4ea806ce90 2023-07-14T11:52:52.123 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4e9e7fc700 1 -- 172.21.15.104:0/1990084769 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4012430 con 0x7f4ea806ce90 2023-07-14T11:52:52.124 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4eafa89700 1 -- 172.21.15.104:0/1990084769 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea81e95c0 con 0x7f4ea806ce90 2023-07-14T11:52:52.124 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4e9e7fc700 1 -- 172.21.15.104:0/1990084769 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4ea4012890 con 0x7f4ea806ce90 2023-07-14T11:52:52.124 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.133+0000 7f4e9e7fc700 1 -- 172.21.15.104:0/1990084769 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4ea401c410 con 0x7f4ea806ce90 2023-07-14T11:52:52.124 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.137+0000 7f4eafa89700 1 -- 172.21.15.104:0/1990084769 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea81e9420 con 0x7f4ea806ce90 2023-07-14T11:52:52.127 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.141+0000 7f4e9e7fc700 1 -- 172.21.15.104:0/1990084769 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f4ea81e9420 con 0x7f4ea806ce90 2023-07-14T11:52:52.155 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:52.169+0000 7f4cc2ed62c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T11:52:52.191 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.205+0000 7f4eafa89700 1 -- 172.21.15.104:0/1990084769 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ea804dcb0 con 0x7f4ea806ce90 2023-07-14T11:52:52.192 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.205+0000 7f4e9e7fc700 1 -- 172.21.15.104:0/1990084769 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4ea4031020 con 0x7f4ea806ce90 2023-07-14T11:52:52.193 INFO:teuthology.misc.health.smithi104.stdout: 2023-07-14T11:52:52.193 INFO:teuthology.misc.health.smithi104.stdout:{"epoch":10,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:52:50.810860+0000","last_up_change":"0.000000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:52:52.194 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.205+0000 7f4eafa89700 1 -- 172.21.15.104:0/1990084769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ea806ce90 msgr2=0x7f4ea815d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:52.194 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.205+0000 7f4eafa89700 1 --2- 172.21.15.104:0/1990084769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ea806ce90 0x7f4ea815d9e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009790 tx=0x7f4ea4004af0 comp rx=0 tx=0).stop 2023-07-14T11:52:52.194 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.205+0000 7f4eafa89700 1 -- 172.21.15.104:0/1990084769 shutdown_connections 2023-07-14T11:52:52.195 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.205+0000 7f4eafa89700 1 --2- 172.21.15.104:0/1990084769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ea8160650 0x7f4ea80ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:52.195 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.205+0000 7f4eafa89700 1 --2- 172.21.15.104:0/1990084769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ea80c1930 0x7f4ea80c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:52.195 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.205+0000 7f4eafa89700 1 --2- 172.21.15.104:0/1990084769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ea806ce90 0x7f4ea815d9e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:52.195 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.205+0000 7f4eafa89700 1 -- 172.21.15.104:0/1990084769 >> 172.21.15.104:0/1990084769 conn(0x7f4ea80cacd0 msgr2=0x7f4ea815f840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:52.195 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.205+0000 7f4eafa89700 1 -- 172.21.15.104:0/1990084769 shutdown_connections 2023-07-14T11:52:52.195 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:52.205+0000 7f4eafa89700 1 -- 172.21.15.104:0/1990084769 wait complete. 2023-07-14T11:52:52.198 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:52.213+0000 7fe50f93f2c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T11:52:52.212 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-14T11:52:52.237 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:52.249+0000 7f4cc2ed62c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T11:52:52.289 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:52.305+0000 7fe50f93f2c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T11:52:52.408 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:52.421+0000 7f4cc2ed62c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T11:52:52.476 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:52.489+0000 7fe50f93f2c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T11:52:52.492 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:52.505+0000 7f4cc2ed62c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T11:52:52.562 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:52.577+0000 7fe50f93f2c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T11:52:52.588 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:52.601+0000 7f4cc2ed62c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T11:52:52.656 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:52.669+0000 7fe50f93f2c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T11:52:52.795 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:52.809+0000 7f4cc2ed62c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T11:52:52.860 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:52.873+0000 7fe50f93f2c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T11:52:53.043 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:53.057+0000 7f4cc2ed62c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T11:52:53.101 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:53.117+0000 7fe50f93f2c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T11:52:53.834 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:53.845+0000 7f4cc2ed62c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T11:52:53.886 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-07-14T11:52:53.900+0000 7fe50f93f2c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T11:52:54.871 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:54.885+0000 7f4caa18a700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-07-14T11:52:54.872 INFO:tasks.ceph.mgr.y.smithi104.stderr:2023-07-14T11:52:54.885+0000 7f4caa18a700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-14T11:52:55.487 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-07-14T11:52:55.500+0000 7f13e4df3700 -1 osd.6 0 log_to_monitors true 2023-07-14T11:52:55.521 INFO:tasks.ceph.osd.5.smithi121.stderr:2023-07-14T11:52:55.536+0000 7f4a761e6700 -1 osd.5 0 log_to_monitors true 2023-07-14T11:52:55.526 INFO:tasks.ceph.osd.0.smithi104.stderr:2023-07-14T11:52:55.541+0000 7ff40328f700 -1 osd.0 0 log_to_monitors true 2023-07-14T11:52:55.527 INFO:tasks.ceph.osd.2.smithi104.stderr:2023-07-14T11:52:55.541+0000 7fa2d5a70700 -1 osd.2 0 log_to_monitors true 2023-07-14T11:52:55.542 INFO:tasks.ceph.osd.1.smithi104.stderr:2023-07-14T11:52:55.557+0000 7f5f7d10f700 -1 osd.1 0 log_to_monitors true 2023-07-14T11:52:55.558 INFO:tasks.ceph.osd.3.smithi104.stderr:2023-07-14T11:52:55.569+0000 7f42ea575700 -1 osd.3 0 log_to_monitors true 2023-07-14T11:52:55.744 INFO:tasks.ceph.osd.7.smithi121.stderr:2023-07-14T11:52:55.756+0000 7f2e77e22700 -1 osd.7 0 log_to_monitors true 2023-07-14T11:52:55.756 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-07-14T11:52:55.772+0000 7fd4d7f72700 -1 osd.4 0 log_to_monitors true 2023-07-14T11:52:58.990 INFO:tasks.ceph.osd.7.smithi121.stderr:2023-07-14T11:52:59.004+0000 7f2e73c61700 -1 osd.7 0 waiting for initial osdmap 2023-07-14T11:52:59.005 INFO:tasks.ceph.osd.7.smithi121.stderr:2023-07-14T11:52:59.020+0000 7f2e6b5f0700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:52:59.018 DEBUG:teuthology.orchestra.run.smithi104:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-14T11:52:59.019 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-07-14T11:52:59.032+0000 7fd4d4738700 -1 osd.4 0 waiting for initial osdmap 2023-07-14T11:52:59.028 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-07-14T11:52:59.044+0000 7fd4cb7f0700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:52:59.230 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.241+0000 7fe9dc918700 1 Processor -- start 2023-07-14T11:52:59.231 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.241+0000 7fe9dc918700 1 -- start start 2023-07-14T11:52:59.234 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.245+0000 7fe9dc918700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe9d8157e20 0x7fe9d806fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:59.234 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.245+0000 7fe9dc918700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9d815c3b0 0x7fe9d80701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:59.236 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.245+0000 7fe9dc918700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d80ccd90 con 0x7fe9d80cb5a0 2023-07-14T11:52:59.236 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.245+0000 7fe9dc918700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d80cd070 con 0x7fe9d8157e20 2023-07-14T11:52:59.236 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.245+0000 7fe9dc918700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80cd350 con 0x7fe9d815c3b0 2023-07-14T11:52:59.236 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.245+0000 7fe9d6d9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fe9d80cb5a0 0x7fe9d80cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42478/0 (socket says 172.21.15.104:42478) 2023-07-14T11:52:59.236 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.245+0000 7fe9d6d9d700 1 -- 172.21.15.104:0/3857939872 learned_addr learned my addr 172.21.15.104:0/3857939872 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:52:59.237 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.245+0000 7fe9d759e700 1 --2- 172.21.15.104:0/3857939872 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9d815c3b0 0x7fe9d80701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:59.237 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.245+0000 7fe9d5d9b700 1 -- 172.21.15.104:0/3857939872 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649533454 0 0) 0x7fe9d80ccd90 con 0x7fe9d80cb5a0 2023-07-14T11:52:59.237 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.245+0000 7fe9d5d9b700 1 -- 172.21.15.104:0/3857939872 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9c40035d0 con 0x7fe9d80cb5a0 2023-07-14T11:52:59.237 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d5d9b700 1 -- 172.21.15.104:0/3857939872 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1621206965 0 0) 0x7fe9c40035d0 con 0x7fe9d80cb5a0 2023-07-14T11:52:59.238 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d5d9b700 1 -- 172.21.15.104:0/3857939872 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9d815c3b0 msgr2=0x7fe9d80701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:59.238 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d5d9b700 1 --2- 172.21.15.104:0/3857939872 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9d815c3b0 0x7fe9d80701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:59.239 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d5d9b700 1 -- 172.21.15.104:0/3857939872 >> v2:172.21.15.104:3300/0 conn(0x7fe9d8157e20 msgr2=0x7fe9d806fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:59.239 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d659c700 1 --2- 172.21.15.104:0/3857939872 >> v2:172.21.15.104:3300/0 conn(0x7fe9d8157e20 0x7fe9d806fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:59.239 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d5d9b700 1 --2- 172.21.15.104:0/3857939872 >> v2:172.21.15.104:3300/0 conn(0x7fe9d8157e20 0x7fe9d806fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:59.239 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d5d9b700 1 -- 172.21.15.104:0/3857939872 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d806f760 con 0x7fe9d80cb5a0 2023-07-14T11:52:59.239 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d5d9b700 1 -- 172.21.15.104:0/3857939872 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7fe9d0001670 con 0x7fe9d80cb5a0 2023-07-14T11:52:59.239 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d5d9b700 1 -- 172.21.15.104:0/3857939872 >> v1:172.21.15.104:6789/0 conn(0x7fe9d80cb5a0 legacy=0x7fe9d80cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:59.239 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d5d9b700 1 --2- 172.21.15.104:0/3857939872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9c4003a20 0x7fe9c4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:59.240 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d5d9b700 1 --2- 172.21.15.104:0/3857939872 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9c4004870 0x7fe9c4004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:59.240 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d5d9b700 1 --2- 172.21.15.104:0/3857939872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9c4005220 0x7fe9c400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:59.240 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d5d9b700 1 -- 172.21.15.104:0/3857939872 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80cd070 con 0x7fe9c4003a20 2023-07-14T11:52:59.240 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d5d9b700 1 -- 172.21.15.104:0/3857939872 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80cd350 con 0x7fe9c4005220 2023-07-14T11:52:59.240 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d5d9b700 1 -- 172.21.15.104:0/3857939872 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c400c070 con 0x7fe9c4004870 2023-07-14T11:52:59.240 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d759e700 1 --2- 172.21.15.104:0/3857939872 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9d815c3b0 0x7fe9d80701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:52:59.240 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d659c700 1 --2- 172.21.15.104:0/3857939872 >> v2:172.21.15.104:3300/0 conn(0x7fe9d8157e20 0x7fe9d806fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:52:59.241 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d759e700 1 --2- 172.21.15.104:0/3857939872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9c4005220 0x7fe9c400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:59.241 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d6d9d700 1 --2- 172.21.15.104:0/3857939872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9c4003a20 0x7fe9c4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:59.241 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d659c700 1 --2- 172.21.15.104:0/3857939872 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9c4004870 0x7fe9c4004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:59.241 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d6d9d700 1 -- 172.21.15.104:0/3857939872 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9c4004870 msgr2=0x7fe9c4004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:59.241 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d6d9d700 1 --2- 172.21.15.104:0/3857939872 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9c4004870 0x7fe9c4004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:59.241 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d6d9d700 1 -- 172.21.15.104:0/3857939872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9c4005220 msgr2=0x7fe9c400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:52:59.241 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.249+0000 7fe9d6d9d700 1 --2- 172.21.15.104:0/3857939872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9c4005220 0x7fe9c400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:59.241 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.253+0000 7fe9d6d9d700 1 -- 172.21.15.104:0/3857939872 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c40035d0 con 0x7fe9c4003a20 2023-07-14T11:52:59.242 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.253+0000 7fe9d6d9d700 1 --2- 172.21.15.104:0/3857939872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9c4003a20 0x7fe9c4003df0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000faf0 tx=0x7fe9c40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9add99e7a9d29f76 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:59.242 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.253+0000 7fe9d5d9b700 1 -- 172.21.15.104:0/3857939872 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0014070 con 0x7fe9c4003a20 2023-07-14T11:52:59.242 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.253+0000 7fe9d5d9b700 1 -- 172.21.15.104:0/3857939872 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d000fe60 con 0x7fe9c4003a20 2023-07-14T11:52:59.243 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.253+0000 7fe9dc918700 1 -- 172.21.15.104:0/3857939872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9c4003a20 msgr2=0x7fe9c4003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:59.243 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.253+0000 7fe9dc918700 1 --2- 172.21.15.104:0/3857939872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9c4003a20 0x7fe9c4003df0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000faf0 tx=0x7fe9c40058a0 comp rx=0 tx=0).stop 2023-07-14T11:52:59.243 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.253+0000 7fe9d759e700 1 -- 172.21.15.104:0/3857939872 reap_dead start 2023-07-14T11:52:59.243 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.253+0000 7fe9dc918700 1 -- 172.21.15.104:0/3857939872 shutdown_connections 2023-07-14T11:52:59.243 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.253+0000 7fe9dc918700 1 -- 172.21.15.104:0/3857939872 >> 172.21.15.104:0/3857939872 conn(0x7fe9d80caa90 msgr2=0x7fe9d80cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:59.249 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.257+0000 7fe9dc918700 1 -- 172.21.15.104:0/3857939872 shutdown_connections 2023-07-14T11:52:59.250 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.257+0000 7fe9dc918700 1 -- 172.21.15.104:0/3857939872 wait complete. 2023-07-14T11:52:59.250 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.261+0000 7fe9dc918700 1 Processor -- start 2023-07-14T11:52:59.250 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.261+0000 7fe9dc918700 1 -- start start 2023-07-14T11:52:59.250 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.261+0000 7fe9dc918700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9d814dd60 0x7fe9d8099c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:59.251 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.261+0000 7fe9dc918700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d814e130 0x7fe9d80c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:59.252 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.261+0000 7fe9dc918700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9d81f4810 0x7fe9d814a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:59.252 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.261+0000 7fe9dc918700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d809b000 con 0x7fe9d81f4810 2023-07-14T11:52:59.252 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.261+0000 7fe9dc918700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d809a150 con 0x7fe9d814e130 2023-07-14T11:52:59.252 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.261+0000 7fe9dc918700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d809a3f0 con 0x7fe9d814dd60 2023-07-14T11:52:59.252 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.261+0000 7fe9d6d9d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9d814dd60 0x7fe9d8099c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:59.253 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.261+0000 7fe9d6d9d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9d814dd60 0x7fe9d8099c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:37084/0 (socket says 172.21.15.104:37084) 2023-07-14T11:52:59.253 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.265+0000 7fe9d6d9d700 1 -- 172.21.15.104:0/1412068040 learned_addr learned my addr 172.21.15.104:0/1412068040 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:52:59.254 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.265+0000 7fe9d659c700 1 --2- 172.21.15.104:0/1412068040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d814e130 0x7fe9d80c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:59.255 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.265+0000 7fe9d6d9d700 1 -- 172.21.15.104:0/1412068040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d814e130 msgr2=0x7fe9d80c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:59.255 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.265+0000 7fe9d759e700 1 --2- 172.21.15.104:0/1412068040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9d81f4810 0x7fe9d814a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:59.255 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.265+0000 7fe9d6d9d700 1 --2- 172.21.15.104:0/1412068040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d814e130 0x7fe9d80c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:59.255 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.265+0000 7fe9d6d9d700 1 -- 172.21.15.104:0/1412068040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9d81f4810 msgr2=0x7fe9d814a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:59.255 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.265+0000 7fe9d6d9d700 1 --2- 172.21.15.104:0/1412068040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9d81f4810 0x7fe9d814a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:59.255 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.265+0000 7fe9d6d9d700 1 -- 172.21.15.104:0/1412068040 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d809a6d0 con 0x7fe9d814dd60 2023-07-14T11:52:59.255 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.265+0000 7fe9d759e700 1 --2- 172.21.15.104:0/1412068040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9d81f4810 0x7fe9d814a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:52:59.256 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.265+0000 7fe9d6d9d700 1 --2- 172.21.15.104:0/1412068040 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9d814dd60 0x7fe9d8099c10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0000c00 tx=0x7fe9d0004a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:59.256 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.265+0000 7fe9cf7fe700 1 -- 172.21.15.104:0/1412068040 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d00051d0 con 0x7fe9d814dd60 2023-07-14T11:52:59.256 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.265+0000 7fe9cf7fe700 1 -- 172.21.15.104:0/1412068040 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d0005350 con 0x7fe9d814dd60 2023-07-14T11:52:59.256 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.265+0000 7fe9cf7fe700 1 -- 172.21.15.104:0/1412068040 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0005700 con 0x7fe9d814dd60 2023-07-14T11:52:59.257 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.265+0000 7fe9dc918700 1 -- 172.21.15.104:0/1412068040 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d809a9b0 con 0x7fe9d814dd60 2023-07-14T11:52:59.257 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.265+0000 7fe9dc918700 1 -- 172.21.15.104:0/1412068040 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d80c6690 con 0x7fe9d814dd60 2023-07-14T11:52:59.257 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.269+0000 7fe9cf7fe700 1 -- 172.21.15.104:0/1412068040 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 4) v1 ==== 85602+0+0 (secure 0 0 0) 0x7fe9d0005860 con 0x7fe9d814dd60 2023-07-14T11:52:59.257 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.269+0000 7fe9cf7fe700 1 --2- 172.21.15.104:0/1412068040 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe9b8066330 0x7fe9b80687e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:59.258 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.269+0000 7fe9cf7fe700 1 -- 172.21.15.104:0/1412068040 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fe9d0083b30 con 0x7fe9d814dd60 2023-07-14T11:52:59.258 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.273+0000 7fe9d659c700 1 --2- 172.21.15.104:0/1412068040 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe9b8066330 0x7fe9b80687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:59.258 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.273+0000 7fe9cd7fa700 1 -- 172.21.15.104:0/1412068040 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b0005310 con 0x7fe9d814dd60 2023-07-14T11:52:59.266 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.277+0000 7fe9d659c700 1 --2- 172.21.15.104:0/1412068040 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe9b8066330 0x7fe9b80687e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8099330 tx=0x7fe9c0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:59.274 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.285+0000 7fe9cf7fe700 1 -- 172.21.15.104:0/1412068040 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe9d0051ff0 con 0x7fe9d814dd60 2023-07-14T11:52:59.481 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-07-14T11:52:59.496+0000 7f13e15b9700 -1 osd.6 0 waiting for initial osdmap 2023-07-14T11:52:59.494 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-07-14T11:52:59.508+0000 7f13d85f0700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:52:59.535 INFO:tasks.ceph.osd.5.smithi121.stderr:2023-07-14T11:52:59.548+0000 7f4a72025700 -1 osd.5 0 waiting for initial osdmap 2023-07-14T11:52:59.547 INFO:tasks.ceph.osd.5.smithi121.stderr:2023-07-14T11:52:59.560+0000 7f4a699f0700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:52:59.561 INFO:tasks.ceph.osd.3.smithi104.stderr:2023-07-14T11:52:59.573+0000 7f42e63b4700 -1 osd.3 0 waiting for initial osdmap 2023-07-14T11:52:59.570 INFO:tasks.ceph.osd.3.smithi104.stderr:2023-07-14T11:52:59.585+0000 7f42dddf0700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:52:59.583 INFO:tasks.ceph.osd.1.smithi104.stderr:2023-07-14T11:52:59.597+0000 7f5f78f4e700 -1 osd.1 0 waiting for initial osdmap 2023-07-14T11:52:59.590 INFO:tasks.ceph.osd.1.smithi104.stderr:2023-07-14T11:52:59.605+0000 7f5f709f0700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:52:59.598 INFO:tasks.ceph.osd.0.smithi104.stderr:2023-07-14T11:52:59.609+0000 7ff3ff0ce700 -1 osd.0 0 waiting for initial osdmap 2023-07-14T11:52:59.607 INFO:tasks.ceph.osd.0.smithi104.stderr:2023-07-14T11:52:59.621+0000 7ff3f69f0700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:52:59.613 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.625+0000 7fe9cd7fa700 1 -- 172.21.15.104:0/1412068040 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9b00055c0 con 0x7fe9d814dd60 2023-07-14T11:52:59.614 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.629+0000 7fe9cf7fe700 1 -- 172.21.15.104:0/1412068040 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fe9d0055610 con 0x7fe9d814dd60 2023-07-14T11:52:59.614 INFO:teuthology.misc.health.smithi104.stdout: 2023-07-14T11:52:59.615 INFO:teuthology.misc.health.smithi104.stdout:{"epoch":12,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:52:56.932682+0000","last_up_change":"0.000000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:52:59.616 INFO:tasks.ceph.osd.2.smithi104.stderr:2023-07-14T11:52:59.629+0000 7fa2d18af700 -1 osd.2 0 waiting for initial osdmap 2023-07-14T11:52:59.618 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.629+0000 7fe9cd7fa700 1 -- 172.21.15.104:0/1412068040 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe9b8066330 msgr2=0x7fe9b80687e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:59.619 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.629+0000 7fe9cd7fa700 1 --2- 172.21.15.104:0/1412068040 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe9b8066330 0x7fe9b80687e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8099330 tx=0x7fe9c0008040 comp rx=0 tx=0).stop 2023-07-14T11:52:59.619 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.629+0000 7fe9cd7fa700 1 -- 172.21.15.104:0/1412068040 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9d814dd60 msgr2=0x7fe9d8099c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:59.619 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.629+0000 7fe9cd7fa700 1 --2- 172.21.15.104:0/1412068040 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9d814dd60 0x7fe9d8099c10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0000c00 tx=0x7fe9d0004a70 comp rx=0 tx=0).stop 2023-07-14T11:52:59.619 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.633+0000 7fe9cd7fa700 1 -- 172.21.15.104:0/1412068040 shutdown_connections 2023-07-14T11:52:59.619 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.633+0000 7fe9cd7fa700 1 --2- 172.21.15.104:0/1412068040 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe9b8066330 0x7fe9b80687e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:59.619 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.633+0000 7fe9cd7fa700 1 --2- 172.21.15.104:0/1412068040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9d81f4810 0x7fe9d814a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:59.620 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.633+0000 7fe9cd7fa700 1 --2- 172.21.15.104:0/1412068040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d814e130 0x7fe9d80c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:59.620 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.633+0000 7fe9cd7fa700 1 --2- 172.21.15.104:0/1412068040 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9d814dd60 0x7fe9d8099c10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:59.620 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.633+0000 7fe9cd7fa700 1 -- 172.21.15.104:0/1412068040 >> 172.21.15.104:0/1412068040 conn(0x7fe9d80caa90 msgr2=0x7fe9d80c9700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:59.620 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.633+0000 7fe9cd7fa700 1 -- 172.21.15.104:0/1412068040 shutdown_connections 2023-07-14T11:52:59.620 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:52:59.633+0000 7fe9cd7fa700 1 -- 172.21.15.104:0/1412068040 wait complete. 2023-07-14T11:52:59.624 INFO:tasks.ceph.osd.2.smithi104.stderr:2023-07-14T11:52:59.637+0000 7fa2c91f0700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:52:59.635 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-14T11:53:06.441 DEBUG:teuthology.orchestra.run.smithi104:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:06.601 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.613+0000 7ff48759e700 1 Processor -- start 2023-07-14T11:53:06.602 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.613+0000 7ff48759e700 1 -- start start 2023-07-14T11:53:06.603 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff48759e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff48806d000 0x7ff48815a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:06.604 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff48759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4880711b0 0x7ff488156c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:06.604 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff48759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff488157190 0x7ff48815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:06.604 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff48759e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4880ad8e0 con 0x7ff48806d000 2023-07-14T11:53:06.604 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff48759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48815aea0 con 0x7ff488157190 2023-07-14T11:53:06.604 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff48759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff488070110 con 0x7ff4880711b0 2023-07-14T11:53:06.605 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff48659c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff48806d000 0x7ff48815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:06.605 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff48659c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff48806d000 0x7ff48815a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49180/0 (socket says 172.21.15.104:49180) 2023-07-14T11:53:06.606 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff48659c700 1 -- 172.21.15.104:0/2108104353 learned_addr learned my addr 172.21.15.104:0/2108104353 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:06.606 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff486d9d700 1 --2- 172.21.15.104:0/2108104353 >> v2:172.21.15.121:3300/0 conn(0x7ff488157190 0x7ff48815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:06.606 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff47dd9b700 1 --2- 172.21.15.104:0/2108104353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4880711b0 0x7ff488156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:06.606 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff48659c700 1 -- 172.21.15.104:0/2108104353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4880711b0 msgr2=0x7ff488156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:06.606 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff48659c700 1 --2- 172.21.15.104:0/2108104353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4880711b0 0x7ff488156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:06.606 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff48659c700 1 -- 172.21.15.104:0/2108104353 >> v2:172.21.15.121:3300/0 conn(0x7ff488157190 msgr2=0x7ff48815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:06.606 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff48659c700 1 --2- 172.21.15.104:0/2108104353 >> v2:172.21.15.121:3300/0 conn(0x7ff488157190 0x7ff48815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:06.607 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff48659c700 1 -- 172.21.15.104:0/2108104353 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4880703f0 con 0x7ff48806d000 2023-07-14T11:53:06.607 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff486d9d700 1 --2- 172.21.15.104:0/2108104353 >> v2:172.21.15.121:3300/0 conn(0x7ff488157190 0x7ff48815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:06.607 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff48659c700 1 --2- 172.21.15.104:0/2108104353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff48806d000 0x7ff48815a960 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff4880c5000 tx=0x7ff47800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba12513728e1021 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:06.608 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff485d9b700 1 -- 172.21.15.104:0/2108104353 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47800e070 con 0x7ff48806d000 2023-07-14T11:53:06.608 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff485d9b700 1 -- 172.21.15.104:0/2108104353 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47800b9c0 con 0x7ff48806d000 2023-07-14T11:53:06.608 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.617+0000 7ff485d9b700 1 -- 172.21.15.104:0/2108104353 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47800bd70 con 0x7ff48806d000 2023-07-14T11:53:06.608 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.621+0000 7ff48759e700 1 -- 172.21.15.104:0/2108104353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff48806d000 msgr2=0x7ff48815a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:06.608 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.621+0000 7ff48759e700 1 --2- 172.21.15.104:0/2108104353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff48806d000 0x7ff48815a960 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff4880c5000 tx=0x7ff47800b330 comp rx=0 tx=0).stop 2023-07-14T11:53:06.608 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.621+0000 7ff48759e700 1 -- 172.21.15.104:0/2108104353 shutdown_connections 2023-07-14T11:53:06.609 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.621+0000 7ff48759e700 1 --2- 172.21.15.104:0/2108104353 >> v2:172.21.15.121:3300/0 conn(0x7ff488157190 0x7ff48815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:06.609 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.621+0000 7ff48759e700 1 --2- 172.21.15.104:0/2108104353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4880711b0 0x7ff488156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:06.609 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.621+0000 7ff48759e700 1 --2- 172.21.15.104:0/2108104353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff48806d000 0x7ff48815a960 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:06.609 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.621+0000 7ff48759e700 1 -- 172.21.15.104:0/2108104353 >> 172.21.15.104:0/2108104353 conn(0x7ff4880caa90 msgr2=0x7ff4880cae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:06.609 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.621+0000 7ff48759e700 1 -- 172.21.15.104:0/2108104353 shutdown_connections 2023-07-14T11:53:06.609 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.621+0000 7ff48759e700 1 -- 172.21.15.104:0/2108104353 wait complete. 2023-07-14T11:53:06.610 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.621+0000 7ff48759e700 1 Processor -- start 2023-07-14T11:53:06.610 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff48759e700 1 -- start start 2023-07-14T11:53:06.612 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff48759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4880711b0 0x7ff4881f1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:06.612 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff48759e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff488157190 0x7ff488069fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:06.612 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff48759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4880c3ee0 0x7ff48806a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:06.612 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff48759e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff488158e70 con 0x7ff488157190 2023-07-14T11:53:06.613 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff48759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4881f15a0 con 0x7ff4880711b0 2023-07-14T11:53:06.613 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff48759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4880c6220 con 0x7ff4880c3ee0 2023-07-14T11:53:06.613 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff47dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff488157190 0x7ff488069fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:06.614 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff47dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff488157190 0x7ff488069fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49188/0 (socket says 172.21.15.104:49188) 2023-07-14T11:53:06.614 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff47dd9b700 1 -- 172.21.15.104:0/720878329 learned_addr learned my addr 172.21.15.104:0/720878329 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:06.614 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff48659c700 1 --2- 172.21.15.104:0/720878329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4880711b0 0x7ff4881f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:06.614 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff486d9d700 1 --2- 172.21.15.104:0/720878329 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4880c3ee0 0x7ff48806a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:06.614 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff47dd9b700 1 -- 172.21.15.104:0/720878329 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4880c3ee0 msgr2=0x7ff48806a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:06.614 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff47dd9b700 1 --2- 172.21.15.104:0/720878329 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4880c3ee0 0x7ff48806a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:06.615 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff47dd9b700 1 -- 172.21.15.104:0/720878329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4880711b0 msgr2=0x7ff4881f1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:06.615 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff47dd9b700 1 --2- 172.21.15.104:0/720878329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4880711b0 0x7ff4881f1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:06.615 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff47dd9b700 1 -- 172.21.15.104:0/720878329 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff478009b30 con 0x7ff488157190 2023-07-14T11:53:06.615 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff47dd9b700 1 --2- 172.21.15.104:0/720878329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff488157190 0x7ff488069fd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff47000e810 tx=0x7ff47000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:06.616 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.625+0000 7ff47f7fe700 1 -- 172.21.15.104:0/720878329 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47000c070 con 0x7ff488157190 2023-07-14T11:53:06.616 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.629+0000 7ff47f7fe700 1 -- 172.21.15.104:0/720878329 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff470013610 con 0x7ff488157190 2023-07-14T11:53:06.616 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.629+0000 7ff48759e700 1 -- 172.21.15.104:0/720878329 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4881f1840 con 0x7ff488157190 2023-07-14T11:53:06.616 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.629+0000 7ff47f7fe700 1 -- 172.21.15.104:0/720878329 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4700139a0 con 0x7ff488157190 2023-07-14T11:53:06.616 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.629+0000 7ff48759e700 1 -- 172.21.15.104:0/720878329 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4881f1e00 con 0x7ff488157190 2023-07-14T11:53:06.616 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.629+0000 7ff48759e700 1 -- 172.21.15.104:0/720878329 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4881f2020 con 0x7ff488157190 2023-07-14T11:53:06.618 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.629+0000 7ff47f7fe700 1 -- 172.21.15.104:0/720878329 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff470013b00 con 0x7ff488157190 2023-07-14T11:53:06.619 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.633+0000 7ff47f7fe700 1 --2- 172.21.15.104:0/720878329 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff4680662f0 0x7ff4680687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:06.619 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.633+0000 7ff47f7fe700 1 -- 172.21.15.104:0/720878329 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff470083bb0 con 0x7ff488157190 2023-07-14T11:53:06.620 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.633+0000 7ff48659c700 1 --2- 172.21.15.104:0/720878329 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff4680662f0 0x7ff4680687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:06.620 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.633+0000 7ff48659c700 1 --2- 172.21.15.104:0/720878329 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff4680662f0 0x7ff4680687a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff47800b370 tx=0x7ff478009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:06.625 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.637+0000 7ff47f7fe700 1 -- 172.21.15.104:0/720878329 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff4700514f0 con 0x7ff488157190 2023-07-14T11:53:06.797 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.809+0000 7ff48759e700 1 -- 172.21.15.104:0/720878329 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff48804e500 con 0x7ff488157190 2023-07-14T11:53:06.799 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.813+0000 7ff47f7fe700 1 -- 172.21.15.104:0/720878329 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12886 (secure 0 0 0) 0x7ff470054b10 con 0x7ff488157190 2023-07-14T11:53:06.799 INFO:teuthology.misc.health.smithi104.stdout: 2023-07-14T11:53:06.800 INFO:teuthology.misc.health.smithi104.stdout:{"epoch":17,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:04.082354+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:06.802 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.813+0000 7ff48759e700 1 -- 172.21.15.104:0/720878329 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff4680662f0 msgr2=0x7ff4680687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:06.802 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.813+0000 7ff48759e700 1 --2- 172.21.15.104:0/720878329 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff4680662f0 0x7ff4680687a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff47800b370 tx=0x7ff478009000 comp rx=0 tx=0).stop 2023-07-14T11:53:06.802 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.813+0000 7ff48759e700 1 -- 172.21.15.104:0/720878329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff488157190 msgr2=0x7ff488069fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:06.802 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.813+0000 7ff48759e700 1 --2- 172.21.15.104:0/720878329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff488157190 0x7ff488069fd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff47000e810 tx=0x7ff47000eb20 comp rx=0 tx=0).stop 2023-07-14T11:53:06.802 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.813+0000 7ff48759e700 1 -- 172.21.15.104:0/720878329 shutdown_connections 2023-07-14T11:53:06.802 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.813+0000 7ff48759e700 1 --2- 172.21.15.104:0/720878329 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff4680662f0 0x7ff4680687a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:06.803 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.813+0000 7ff48759e700 1 --2- 172.21.15.104:0/720878329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff488157190 0x7ff488069fd0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:06.803 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.813+0000 7ff48759e700 1 --2- 172.21.15.104:0/720878329 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4880c3ee0 0x7ff48806a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:06.803 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.813+0000 7ff48759e700 1 --2- 172.21.15.104:0/720878329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4880711b0 0x7ff4881f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:06.803 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.813+0000 7ff48759e700 1 -- 172.21.15.104:0/720878329 >> 172.21.15.104:0/720878329 conn(0x7ff4880caa90 msgr2=0x7ff4880ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:06.803 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.813+0000 7ff48759e700 1 -- 172.21.15.104:0/720878329 shutdown_connections 2023-07-14T11:53:06.804 INFO:teuthology.misc.health.smithi104.stderr:2023-07-14T11:53:06.813+0000 7ff48759e700 1 -- 172.21.15.104:0/720878329 wait complete. 2023-07-14T11:53:06.823 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-07-14T11:53:06.823 INFO:tasks.ceph:Creating RBD pool 2023-07-14T11:53:06.824 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd pool create rbd 8 2023-07-14T11:53:06.987 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.001+0000 7f3d04078700 1 Processor -- start 2023-07-14T11:53:06.987 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.001+0000 7f3d04078700 1 -- start start 2023-07-14T11:53:06.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.001+0000 7f3d04078700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3cfc0b0490 0x7f3cfc0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:06.990 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.001+0000 7f3d04078700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3cfc160610 0x7f3cfc0ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:06.990 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.001+0000 7f3d04078700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cfc163fe0 con 0x7f3cfc160240 2023-07-14T11:53:06.990 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.001+0000 7f3d04078700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cfc0c8c30 con 0x7f3cfc160610 2023-07-14T11:53:06.990 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.001+0000 7f3d04078700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0ca9e0 con 0x7f3cfc0b0490 2023-07-14T11:53:06.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.001+0000 7f3d02615700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3cfc160610 0x7f3cfc0ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:06.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.001+0000 7f3d01613700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3cfc0b0490 0x7f3cfc0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:06.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d01613700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3cfc0b0490 0x7f3cfc0cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:44932/0 (socket says 172.21.15.104:44932) 2023-07-14T11:53:06.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d02615700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3cfc160610 0x7f3cfc0ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49202/0 (socket says 172.21.15.104:49202) 2023-07-14T11:53:06.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d01613700 1 -- 172.21.15.104:0/561363746 learned_addr learned my addr 172.21.15.104:0/561363746 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:06.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d01613700 1 -- 172.21.15.104:0/561363746 >> v2:172.21.15.104:3300/0 conn(0x7f3cfc160610 msgr2=0x7f3cfc0ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:06.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d01613700 1 --2- 172.21.15.104:0/561363746 >> v2:172.21.15.104:3300/0 conn(0x7f3cfc160610 0x7f3cfc0ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:06.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d01613700 1 -- 172.21.15.104:0/561363746 >> v1:172.21.15.121:6789/0 conn(0x7f3cfc160240 legacy=0x7f3cfc0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:06.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d01613700 1 -- 172.21.15.104:0/561363746 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc164230 con 0x7f3cfc0b0490 2023-07-14T11:53:06.993 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d01613700 1 --2- 172.21.15.104:0/561363746 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3cfc0b0490 0x7f3cfc0cdc20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009910 tx=0x7f3cec009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b9c0c241552f07f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:06.993 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d00e12700 1 -- 172.21.15.104:0/561363746 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec011830 con 0x7f3cfc0b0490 2023-07-14T11:53:06.993 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d00e12700 1 -- 172.21.15.104:0/561363746 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cec011990 con 0x7f3cfc0b0490 2023-07-14T11:53:06.993 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d00e12700 1 -- 172.21.15.104:0/561363746 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec011d40 con 0x7f3cfc0b0490 2023-07-14T11:53:06.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d04078700 1 -- 172.21.15.104:0/561363746 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3cfc0b0490 msgr2=0x7f3cfc0cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:06.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d04078700 1 --2- 172.21.15.104:0/561363746 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3cfc0b0490 0x7f3cfc0cdc20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009910 tx=0x7f3cec009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:06.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d04078700 1 -- 172.21.15.104:0/561363746 shutdown_connections 2023-07-14T11:53:06.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d04078700 1 --2- 172.21.15.104:0/561363746 >> v2:172.21.15.104:3300/0 conn(0x7f3cfc160610 0x7f3cfc0ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:06.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d04078700 1 --2- 172.21.15.104:0/561363746 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3cfc0b0490 0x7f3cfc0cdc20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:06.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d04078700 1 -- 172.21.15.104:0/561363746 >> 172.21.15.104:0/561363746 conn(0x7f3cfc0cad80 msgr2=0x7f3cfc0cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:06.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d04078700 1 -- 172.21.15.104:0/561363746 shutdown_connections 2023-07-14T11:53:06.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.005+0000 7f3d04078700 1 -- 172.21.15.104:0/561363746 wait complete. 2023-07-14T11:53:06.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.009+0000 7f3d04078700 1 Processor -- start 2023-07-14T11:53:06.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.009+0000 7f3d04078700 1 -- start start 2023-07-14T11:53:06.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d04078700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cfc0b0490 0x7f3cfc15e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:06.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d04078700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3cfc160240 0x7f3cfc0aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:06.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d04078700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cfc160610 0x7f3cfc0af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:06.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d04078700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc15ecb0 con 0x7f3cfc0b0490 2023-07-14T11:53:06.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d04078700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc160b10 con 0x7f3cfc160610 2023-07-14T11:53:06.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d04078700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc160df0 con 0x7f3cfc160240 2023-07-14T11:53:07.000 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d01e14700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cfc0b0490 0x7f3cfc15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:07.000 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d01e14700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cfc0b0490 0x7f3cfc15e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49212/0 (socket says 172.21.15.104:49212) 2023-07-14T11:53:07.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d01e14700 1 -- 172.21.15.104:0/3503209115 learned_addr learned my addr 172.21.15.104:0/3503209115 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:07.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d01613700 1 --2- 172.21.15.104:0/3503209115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3cfc160240 0x7f3cfc0aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:07.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d02615700 1 --2- 172.21.15.104:0/3503209115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cfc160610 0x7f3cfc0af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:07.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d01e14700 1 -- 172.21.15.104:0/3503209115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3cfc160240 msgr2=0x7f3cfc0aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:07.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d01e14700 1 --2- 172.21.15.104:0/3503209115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3cfc160240 0x7f3cfc0aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:07.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d01e14700 1 -- 172.21.15.104:0/3503209115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cfc160610 msgr2=0x7f3cfc0af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:07.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d01e14700 1 --2- 172.21.15.104:0/3503209115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cfc160610 0x7f3cfc0af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:07.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d01e14700 1 -- 172.21.15.104:0/3503209115 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec009570 con 0x7f3cfc0b0490 2023-07-14T11:53:07.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d01e14700 1 --2- 172.21.15.104:0/3503209115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cfc0b0490 0x7f3cfc15e770 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800c500 tx=0x7f3cf800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:07.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3cf27fc700 1 -- 172.21.15.104:0/3503209115 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf80052d0 con 0x7f3cfc0b0490 2023-07-14T11:53:07.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3cf27fc700 1 -- 172.21.15.104:0/3503209115 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf8005430 con 0x7f3cfc0b0490 2023-07-14T11:53:07.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d04078700 1 -- 172.21.15.104:0/3503209115 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cfc1610d0 con 0x7f3cfc0b0490 2023-07-14T11:53:07.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3cf27fc700 1 -- 172.21.15.104:0/3503209115 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf80055a0 con 0x7f3cfc0b0490 2023-07-14T11:53:07.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.013+0000 7f3d04078700 1 -- 172.21.15.104:0/3503209115 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cfc161690 con 0x7f3cfc0b0490 2023-07-14T11:53:07.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.017+0000 7f3d04078700 1 -- 172.21.15.104:0/3503209115 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cfc04dcb0 con 0x7f3cfc0b0490 2023-07-14T11:53:07.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.017+0000 7f3cf27fc700 1 -- 172.21.15.104:0/3503209115 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3cf80036a0 con 0x7f3cfc0b0490 2023-07-14T11:53:07.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.021+0000 7f3cf27fc700 1 --2- 172.21.15.104:0/3503209115 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3ce0066320 0x7f3ce00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:07.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.021+0000 7f3cf27fc700 1 -- 172.21.15.104:0/3503209115 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3cf8082bf0 con 0x7f3cfc0b0490 2023-07-14T11:53:07.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.021+0000 7f3d01613700 1 --2- 172.21.15.104:0/3503209115 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3ce0066320 0x7f3ce00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:07.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.021+0000 7f3d01613700 1 --2- 172.21.15.104:0/3503209115 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3ce0066320 0x7f3ce00687d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0b0880 tx=0x7f3cec008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:07.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.025+0000 7f3cf27fc700 1 -- 172.21.15.104:0/3503209115 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f3cf80504a0 con 0x7f3cfc0b0490 2023-07-14T11:53:07.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:07.201+0000 7f3d04078700 1 -- 172.21.15.104:0/3503209115 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f3cfc0b0bc0 con 0x7f3cfc0b0490 2023-07-14T11:53:07.557 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:53:07.564 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:53:08.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.125+0000 7f3cf27fc700 1 -- 172.21.15.104:0/3503209115 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v18) v1 ==== 109+0+0 (secure 0 0 0) 0x7f3cf8053ac0 con 0x7f3cfc0b0490 2023-07-14T11:53:08.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.129+0000 7f3d04078700 1 -- 172.21.15.104:0/3503209115 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3ce0066320 msgr2=0x7f3ce00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:08.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.129+0000 7f3d04078700 1 --2- 172.21.15.104:0/3503209115 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3ce0066320 0x7f3ce00687d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0b0880 tx=0x7f3cec008030 comp rx=0 tx=0).stop 2023-07-14T11:53:08.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.129+0000 7f3d04078700 1 -- 172.21.15.104:0/3503209115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cfc0b0490 msgr2=0x7f3cfc15e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:08.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.129+0000 7f3d04078700 1 --2- 172.21.15.104:0/3503209115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cfc0b0490 0x7f3cfc15e770 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800c500 tx=0x7f3cf800c8c0 comp rx=0 tx=0).stop 2023-07-14T11:53:08.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.129+0000 7f3d04078700 1 -- 172.21.15.104:0/3503209115 shutdown_connections 2023-07-14T11:53:08.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.129+0000 7f3d04078700 1 --2- 172.21.15.104:0/3503209115 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3ce0066320 0x7f3ce00687d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:08.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.129+0000 7f3d04078700 1 --2- 172.21.15.104:0/3503209115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cfc160610 0x7f3cfc0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:08.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.129+0000 7f3d04078700 1 --2- 172.21.15.104:0/3503209115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3cfc160240 0x7f3cfc0aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:08.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.129+0000 7f3d04078700 1 --2- 172.21.15.104:0/3503209115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cfc0b0490 0x7f3cfc15e770 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:08.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.129+0000 7f3d04078700 1 -- 172.21.15.104:0/3503209115 >> 172.21.15.104:0/3503209115 conn(0x7f3cfc0cad80 msgr2=0x7f3cfc0adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:08.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.129+0000 7f3d04078700 1 -- 172.21.15.104:0/3503209115 shutdown_connections 2023-07-14T11:53:08.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.129+0000 7f3d04078700 1 -- 172.21.15.104:0/3503209115 wait complete. 2023-07-14T11:53:08.119 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'rbd' created 2023-07-14T11:53:08.136 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-07-14T11:53:08.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.321+0000 7f407ed5d700 1 Processor -- start 2023-07-14T11:53:08.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.321+0000 7f407ed5d700 1 -- start start 2023-07-14T11:53:08.309 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.321+0000 7f407ed5d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40780cd260 0x7f40780d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:08.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.321+0000 7f407ed5d700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40780d1a60 con 0x7f40780b0ee0 2023-07-14T11:53:08.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.321+0000 7f407ed5d700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40780d1ef0 con 0x7f40780d0610 2023-07-14T11:53:08.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.321+0000 7f407ed5d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40780d0e50 con 0x7f40780cd260 2023-07-14T11:53:08.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.321+0000 7f407dd5b700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f40780b0ee0 0x7f40780ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:41564/0 (socket says 172.21.15.104:41564) 2023-07-14T11:53:08.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.321+0000 7f407dd5b700 1 -- 172.21.15.104:0/891362728 learned_addr learned my addr 172.21.15.104:0/891362728 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:08.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.325+0000 7f407d55a700 1 --2- 172.21.15.104:0/891362728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40780cd260 0x7f40780d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:08.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.325+0000 7f407cd59700 1 -- 172.21.15.104:0/891362728 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685660497 0 0) 0x7f40780d1a60 con 0x7f40780b0ee0 2023-07-14T11:53:08.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.325+0000 7f407cd59700 1 -- 172.21.15.104:0/891362728 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40640035d0 con 0x7f40780b0ee0 2023-07-14T11:53:08.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.325+0000 7f407cd59700 1 -- 172.21.15.104:0/891362728 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862061578 0 0) 0x7f40780d1ef0 con 0x7f40780d0610 2023-07-14T11:53:08.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.325+0000 7f407cd59700 1 -- 172.21.15.104:0/891362728 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40780d1a60 con 0x7f40780d0610 2023-07-14T11:53:08.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.325+0000 7f407d55a700 1 -- 172.21.15.104:0/891362728 >> v1:172.21.15.121:6789/0 conn(0x7f40780d0610 legacy=0x7f40780af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:08.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.325+0000 7f407d55a700 1 -- 172.21.15.104:0/891362728 >> v1:172.21.15.104:6789/0 conn(0x7f40780b0ee0 legacy=0x7f40780ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:08.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.325+0000 7f407d55a700 1 -- 172.21.15.104:0/891362728 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40780d1130 con 0x7f40780cd260 2023-07-14T11:53:08.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.325+0000 7f407d55a700 1 --2- 172.21.15.104:0/891362728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40780cd260 0x7f40780d00d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4068009880 tx=0x7f4068009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=109371a0bdc48e32 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:08.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.325+0000 7f407cd59700 1 -- 172.21.15.104:0/891362728 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068011830 con 0x7f40780cd260 2023-07-14T11:53:08.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.325+0000 7f407cd59700 1 -- 172.21.15.104:0/891362728 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4068011990 con 0x7f40780cd260 2023-07-14T11:53:08.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.325+0000 7f407cd59700 1 -- 172.21.15.104:0/891362728 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068011d20 con 0x7f40780cd260 2023-07-14T11:53:08.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.329+0000 7f407ed5d700 1 -- 172.21.15.104:0/891362728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40780cd260 msgr2=0x7f40780d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:08.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.329+0000 7f407ed5d700 1 --2- 172.21.15.104:0/891362728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40780cd260 0x7f40780d00d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4068009880 tx=0x7f4068009b90 comp rx=0 tx=0).stop 2023-07-14T11:53:08.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.329+0000 7f407ed5d700 1 -- 172.21.15.104:0/891362728 shutdown_connections 2023-07-14T11:53:08.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.329+0000 7f407ed5d700 1 --2- 172.21.15.104:0/891362728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40780cd260 0x7f40780d00d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:08.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.329+0000 7f407ed5d700 1 -- 172.21.15.104:0/891362728 >> 172.21.15.104:0/891362728 conn(0x7f4078060c40 msgr2=0x7f407809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:08.316 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.329+0000 7f407ed5d700 1 -- 172.21.15.104:0/891362728 shutdown_connections 2023-07-14T11:53:08.316 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.329+0000 7f407ed5d700 1 -- 172.21.15.104:0/891362728 wait complete. 2023-07-14T11:53:08.317 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.329+0000 7f407ed5d700 1 Processor -- start 2023-07-14T11:53:08.318 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.329+0000 7f407ed5d700 1 -- start start 2023-07-14T11:53:08.319 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407ed5d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40780b0ee0 0x7f40780c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:08.320 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407ed5d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40780cd260 0x7f407814a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:08.320 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407ed5d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40780d0610 0x7f40780c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:08.320 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407ed5d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40780cc730 con 0x7f40780cd260 2023-07-14T11:53:08.320 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407ed5d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40780c9180 con 0x7f40780d0610 2023-07-14T11:53:08.320 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407ed5d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40781f21a0 con 0x7f40780b0ee0 2023-07-14T11:53:08.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407d55a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40780cd260 0x7f407814a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:08.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407d55a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40780cd260 0x7f407814a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49224/0 (socket says 172.21.15.104:49224) 2023-07-14T11:53:08.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407d55a700 1 -- 172.21.15.104:0/3660620514 learned_addr learned my addr 172.21.15.104:0/3660620514 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:08.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407dd5b700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40780b0ee0 0x7f40780c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:08.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407e55c700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40780d0610 0x7f40780c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:08.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407d55a700 1 -- 172.21.15.104:0/3660620514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40780b0ee0 msgr2=0x7f40780c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:08.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407d55a700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40780b0ee0 0x7f40780c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:08.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407d55a700 1 -- 172.21.15.104:0/3660620514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40780d0610 msgr2=0x7f40780c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:08.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407d55a700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40780d0610 0x7f40780c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:08.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407d55a700 1 -- 172.21.15.104:0/3660620514 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40680094b0 con 0x7f40780cd260 2023-07-14T11:53:08.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407dd5b700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40780b0ee0 0x7f40780c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:08.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407e55c700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40780d0610 0x7f40780c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:08.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407d55a700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40780cd260 0x7f407814a640 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4068009bd0 tx=0x7f4068004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:08.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f406e7fc700 1 -- 172.21.15.104:0/3660620514 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068019950 con 0x7f40780cd260 2023-07-14T11:53:08.324 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f406e7fc700 1 -- 172.21.15.104:0/3660620514 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406801e070 con 0x7f40780cd260 2023-07-14T11:53:08.324 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407ed5d700 1 -- 172.21.15.104:0/3660620514 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40780c8160 con 0x7f40780cd260 2023-07-14T11:53:08.324 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f406e7fc700 1 -- 172.21.15.104:0/3660620514 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068019cf0 con 0x7f40780cd260 2023-07-14T11:53:08.324 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.333+0000 7f407ed5d700 1 -- 172.21.15.104:0/3660620514 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40780c7e10 con 0x7f40780cd260 2023-07-14T11:53:08.324 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.337+0000 7f407ed5d700 1 -- 172.21.15.104:0/3660620514 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40780c8960 con 0x7f40780cd260 2023-07-14T11:53:08.325 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.337+0000 7f406e7fc700 1 -- 172.21.15.104:0/3660620514 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f406802c3f0 con 0x7f40780cd260 2023-07-14T11:53:08.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.337+0000 7f406e7fc700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f405c0662f0 0x7f405c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:08.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.341+0000 7f406e7fc700 1 -- 172.21.15.104:0/3660620514 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f4068084500 con 0x7f40780cd260 2023-07-14T11:53:08.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.341+0000 7f407dd5b700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f405c0662f0 0x7f405c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:08.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.341+0000 7f407dd5b700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f405c0662f0 0x7f405c0687a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4074000f60 tx=0x7f4074009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:08.333 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.345+0000 7f406e7fc700 1 -- 172.21.15.104:0/3660620514 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f40780c8960 con 0x7f40780cd260 2023-07-14T11:53:08.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:08.521+0000 7f407ed5d700 1 -- 172.21.15.104:0/3660620514 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f40780c8960 con 0x7f40780cd260 2023-07-14T11:53:09.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.133+0000 7f406e7fc700 1 -- 172.21.15.104:0/3660620514 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v19) v1 ==== 173+0+0 (secure 0 0 0) 0x7f40780c8960 con 0x7f40780cd260 2023-07-14T11:53:09.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.137+0000 7f407ed5d700 1 -- 172.21.15.104:0/3660620514 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f405c0662f0 msgr2=0x7f405c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:09.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.137+0000 7f407ed5d700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f405c0662f0 0x7f405c0687a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4074000f60 tx=0x7f4074009040 comp rx=0 tx=0).stop 2023-07-14T11:53:09.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.137+0000 7f407ed5d700 1 -- 172.21.15.104:0/3660620514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40780cd260 msgr2=0x7f407814a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:09.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.137+0000 7f407ed5d700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40780cd260 0x7f407814a640 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4068009bd0 tx=0x7f4068004770 comp rx=0 tx=0).stop 2023-07-14T11:53:09.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.137+0000 7f407ed5d700 1 -- 172.21.15.104:0/3660620514 shutdown_connections 2023-07-14T11:53:09.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.137+0000 7f407ed5d700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f405c0662f0 0x7f405c0687a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.137+0000 7f407ed5d700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40780cd260 0x7f407814a640 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.137+0000 7f407ed5d700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40780d0610 0x7f40780c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.137+0000 7f407ed5d700 1 --2- 172.21.15.104:0/3660620514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40780b0ee0 0x7f40780c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.137+0000 7f407ed5d700 1 -- 172.21.15.104:0/3660620514 >> 172.21.15.104:0/3660620514 conn(0x7f4078060c40 msgr2=0x7f40780b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:09.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.137+0000 7f407ed5d700 1 -- 172.21.15.104:0/3660620514 shutdown_connections 2023-07-14T11:53:09.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.137+0000 7f407ed5d700 1 -- 172.21.15.104:0/3660620514 wait complete. 2023-07-14T11:53:09.127 INFO:teuthology.orchestra.run.smithi104.stderr:enabled application 'rbd' on pool 'rbd' 2023-07-14T11:53:09.147 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-07-14T11:53:09.148 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-14T11:53:09.148 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-14T11:53:09.151 INFO:tasks.ceph.mds.a:Started 2023-07-14T11:53:09.152 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-14T11:53:09.152 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-14T11:53:09.157 INFO:tasks.ceph.mds.c:Started 2023-07-14T11:53:09.157 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-14T11:53:09.157 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-14T11:53:09.161 INFO:tasks.ceph.mds.b:Started 2023-07-14T11:53:09.162 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-14T11:53:09.162 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-14T11:53:09.165 INFO:tasks.ceph.mds.d:Started 2023-07-14T11:53:09.165 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-07-14T11:53:09.166 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:09.268 INFO:tasks.ceph.mds.a.smithi104.stdout:starting mds.a at 2023-07-14T11:53:09.270 INFO:tasks.ceph.mds.c.smithi104.stdout:starting mds.c at 2023-07-14T11:53:09.286 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2023-07-14T11:53:09.289 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2023-07-14T11:53:09.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.345+0000 7fd779915700 1 Processor -- start 2023-07-14T11:53:09.333 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.345+0000 7fd779915700 1 -- start start 2023-07-14T11:53:09.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.345+0000 7fd779915700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7740b38b0 0x7fd7740b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:09.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.345+0000 7fd779915700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fd7740ccb10 0x7fd7740ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:09.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.345+0000 7fd779915700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7740b3e20 con 0x7fd77415f670 2023-07-14T11:53:09.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.345+0000 7fd779915700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7740d0b20 con 0x7fd7740ccb10 2023-07-14T11:53:09.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.345+0000 7fd779915700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740d0560 con 0x7fd7740b38b0 2023-07-14T11:53:09.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.345+0000 7fd778913700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fd77415f670 0x7fd7740b1e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:41574/0 (socket says 172.21.15.104:41574) 2023-07-14T11:53:09.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd779114700 1 --2- 172.21.15.104:0/1698323094 >> v2:172.21.15.104:3300/0 conn(0x7fd7740ccb10 0x7fd7740ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:09.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd773fff700 1 --2- 172.21.15.104:0/1698323094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7740b38b0 0x7fd7740b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:09.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.345+0000 7fd778913700 1 -- 172.21.15.104:0/1698323094 learned_addr learned my addr 172.21.15.104:0/1698323094 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:09.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd779114700 1 -- 172.21.15.104:0/1698323094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7740b38b0 msgr2=0x7fd7740b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:09.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd779114700 1 --2- 172.21.15.104:0/1698323094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7740b38b0 0x7fd7740b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd779114700 1 -- 172.21.15.104:0/1698323094 >> v1:172.21.15.104:6789/0 conn(0x7fd77415f670 legacy=0x7fd7740b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:09.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd7737fe700 1 -- 172.21.15.104:0/1698323094 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007271864 0 0) 0x7fd7740b3e20 con 0x7fd77415f670 2023-07-14T11:53:09.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd779114700 1 -- 172.21.15.104:0/1698323094 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7740d0840 con 0x7fd7740ccb10 2023-07-14T11:53:09.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd773fff700 1 --2- 172.21.15.104:0/1698323094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7740b38b0 0x7fd7740b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:09.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd779114700 1 --2- 172.21.15.104:0/1698323094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7740ccb10 0x7fd7740ccf50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009d90 tx=0x7fd76c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124bfc362c06de07 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:09.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd7737fe700 1 -- 172.21.15.104:0/1698323094 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c00e070 con 0x7fd7740ccb10 2023-07-14T11:53:09.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd7737fe700 1 -- 172.21.15.104:0/1698323094 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76c00bb20 con 0x7fd7740ccb10 2023-07-14T11:53:09.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd7737fe700 1 -- 172.21.15.104:0/1698323094 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c012430 con 0x7fd7740ccb10 2023-07-14T11:53:09.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd779915700 1 -- 172.21.15.104:0/1698323094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7740ccb10 msgr2=0x7fd7740ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:09.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd779915700 1 --2- 172.21.15.104:0/1698323094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7740ccb10 0x7fd7740ccf50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009d90 tx=0x7fd76c00b330 comp rx=0 tx=0).stop 2023-07-14T11:53:09.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd779915700 1 -- 172.21.15.104:0/1698323094 shutdown_connections 2023-07-14T11:53:09.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd779915700 1 --2- 172.21.15.104:0/1698323094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7740ccb10 0x7fd7740ccf50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd779915700 1 --2- 172.21.15.104:0/1698323094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7740b38b0 0x7fd7740b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd779915700 1 -- 172.21.15.104:0/1698323094 >> 172.21.15.104:0/1698323094 conn(0x7fd774060c40 msgr2=0x7fd77409b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:09.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd779915700 1 -- 172.21.15.104:0/1698323094 shutdown_connections 2023-07-14T11:53:09.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.349+0000 7fd779915700 1 -- 172.21.15.104:0/1698323094 wait complete. 2023-07-14T11:53:09.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.353+0000 7fd779915700 1 Processor -- start 2023-07-14T11:53:09.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.353+0000 7fd779915700 1 -- start start 2023-07-14T11:53:09.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779915700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7740b38b0 0x7fd77415c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:09.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779915700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7740ccb10 0x7fd7740b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:09.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779915700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd77415f670 0x7fd774070370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:09.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779915700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740c70a0 con 0x7fd77415f670 2023-07-14T11:53:09.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779915700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77415c8d0 con 0x7fd7740ccb10 2023-07-14T11:53:09.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779915700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740b5600 con 0x7fd7740b38b0 2023-07-14T11:53:09.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779114700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd77415f670 0x7fd774070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:09.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779114700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd77415f670 0x7fd774070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49266/0 (socket says 172.21.15.104:49266) 2023-07-14T11:53:09.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779114700 1 -- 172.21.15.104:0/2264867664 learned_addr learned my addr 172.21.15.104:0/2264867664 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:09.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd778913700 1 --2- 172.21.15.104:0/2264867664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7740b38b0 0x7fd77415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:09.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd773fff700 1 --2- 172.21.15.104:0/2264867664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7740ccb10 0x7fd7740b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:09.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779114700 1 -- 172.21.15.104:0/2264867664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7740b38b0 msgr2=0x7fd77415c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:09.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779114700 1 --2- 172.21.15.104:0/2264867664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7740b38b0 0x7fd77415c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779114700 1 -- 172.21.15.104:0/2264867664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7740ccb10 msgr2=0x7fd7740b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:09.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779114700 1 --2- 172.21.15.104:0/2264867664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7740ccb10 0x7fd7740b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779114700 1 -- 172.21.15.104:0/2264867664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7600094b0 con 0x7fd77415f670 2023-07-14T11:53:09.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779114700 1 --2- 172.21.15.104:0/2264867664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd77415f670 0x7fd774070370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7740d0040 tx=0x7fd76c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:09.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd7717fa700 1 -- 172.21.15.104:0/2264867664 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c012430 con 0x7fd77415f670 2023-07-14T11:53:09.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd7717fa700 1 -- 172.21.15.104:0/2264867664 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76c020070 con 0x7fd77415f670 2023-07-14T11:53:09.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd7717fa700 1 -- 172.21.15.104:0/2264867664 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c004280 con 0x7fd77415f670 2023-07-14T11:53:09.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779915700 1 -- 172.21.15.104:0/2264867664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76c009a70 con 0x7fd77415f670 2023-07-14T11:53:09.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.357+0000 7fd779915700 1 -- 172.21.15.104:0/2264867664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7740c75f0 con 0x7fd77415f670 2023-07-14T11:53:09.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.361+0000 7fd779915700 1 -- 172.21.15.104:0/2264867664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd77404e500 con 0x7fd77415f670 2023-07-14T11:53:09.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.361+0000 7fd7717fa700 1 -- 172.21.15.104:0/2264867664 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd76c0043e0 con 0x7fd77415f670 2023-07-14T11:53:09.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.365+0000 7fd7717fa700 1 --2- 172.21.15.104:0/2264867664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd7580662f0 0x7fd7580687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:09.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.365+0000 7fd7717fa700 1 -- 172.21.15.104:0/2264867664 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd76c022070 con 0x7fd77415f670 2023-07-14T11:53:09.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.365+0000 7fd778913700 1 --2- 172.21.15.104:0/2264867664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd7580662f0 0x7fd7580687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:09.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.365+0000 7fd778913700 1 --2- 172.21.15.104:0/2264867664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd7580662f0 0x7fd7580687a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd768000f60 tx=0x7fd768007cb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:09.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.369+0000 7fd7717fa700 1 -- 172.21.15.104:0/2264867664 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fd76c052e70 con 0x7fd77415f670 2023-07-14T11:53:09.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.549+0000 7fd779915700 1 -- 172.21.15.104:0/2264867664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7740b21a0 con 0x7fd77415f670 2023-07-14T11:53:09.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.549+0000 7fd7717fa700 1 -- 172.21.15.104:0/2264867664 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14685 (secure 0 0 0) 0x7fd76c056490 con 0x7fd77415f670 2023-07-14T11:53:09.536 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:09.537 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":19,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:09.127826+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:09.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.553+0000 7fd779915700 1 -- 172.21.15.104:0/2264867664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd7580662f0 msgr2=0x7fd7580687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:09.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.553+0000 7fd779915700 1 --2- 172.21.15.104:0/2264867664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd7580662f0 0x7fd7580687a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd768000f60 tx=0x7fd768007cb0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.553+0000 7fd779915700 1 -- 172.21.15.104:0/2264867664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd77415f670 msgr2=0x7fd774070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:09.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.553+0000 7fd779915700 1 --2- 172.21.15.104:0/2264867664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd77415f670 0x7fd774070370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7740d0040 tx=0x7fd76c004af0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.553+0000 7fd779915700 1 -- 172.21.15.104:0/2264867664 shutdown_connections 2023-07-14T11:53:09.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.553+0000 7fd779915700 1 --2- 172.21.15.104:0/2264867664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd7580662f0 0x7fd7580687a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.553+0000 7fd779915700 1 --2- 172.21.15.104:0/2264867664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd77415f670 0x7fd774070370 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.553+0000 7fd779915700 1 --2- 172.21.15.104:0/2264867664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7740ccb10 0x7fd7740b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.553+0000 7fd779915700 1 --2- 172.21.15.104:0/2264867664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7740b38b0 0x7fd77415c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.553+0000 7fd779915700 1 -- 172.21.15.104:0/2264867664 >> 172.21.15.104:0/2264867664 conn(0x7fd774060c40 msgr2=0x7fd7740af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:09.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.553+0000 7fd779915700 1 -- 172.21.15.104:0/2264867664 shutdown_connections 2023-07-14T11:53:09.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.553+0000 7fd779915700 1 -- 172.21.15.104:0/2264867664 wait complete. 2023-07-14T11:53:09.559 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:53:09.560 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:53:09.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.757+0000 7f83eabc6700 1 Processor -- start 2023-07-14T11:53:09.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.757+0000 7f83eabc6700 1 -- start start 2023-07-14T11:53:09.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.757+0000 7f83eabc6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83e415df10 0x7f83e415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:09.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.757+0000 7f83eabc6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f83e40afd80 0x7f83e40711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:09.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.757+0000 7f83eabc6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f83e40716e0 0x7f83e40c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:09.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.757+0000 7f83eabc6700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e415e830 con 0x7f83e40afd80 2023-07-14T11:53:09.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.757+0000 7f83eabc6700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e4070a30 con 0x7f83e40716e0 2023-07-14T11:53:09.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.757+0000 7f83eabc6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e40b0210 con 0x7f83e415df10 2023-07-14T11:53:09.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.757+0000 7f83e3fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f83e40afd80 0x7f83e40711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:09.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.757+0000 7f83e8962700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83e415df10 0x7f83e415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:09.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.757+0000 7f83e3fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f83e40afd80 0x7f83e40711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49270/0 (socket says 172.21.15.104:49270) 2023-07-14T11:53:09.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.757+0000 7f83e3fff700 1 -- 172.21.15.104:0/69481407 learned_addr learned my addr 172.21.15.104:0/69481407 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:09.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.757+0000 7f83e9163700 1 --2- 172.21.15.104:0/69481407 >> v2:172.21.15.121:3300/0 conn(0x7f83e40716e0 0x7f83e40c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:09.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.757+0000 7f83e3fff700 1 -- 172.21.15.104:0/69481407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83e415df10 msgr2=0x7f83e415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:09.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83e3fff700 1 --2- 172.21.15.104:0/69481407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83e415df10 0x7f83e415e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83e3fff700 1 -- 172.21.15.104:0/69481407 >> v2:172.21.15.121:3300/0 conn(0x7f83e40716e0 msgr2=0x7f83e40c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:09.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83e3fff700 1 --2- 172.21.15.104:0/69481407 >> v2:172.21.15.121:3300/0 conn(0x7f83e40716e0 0x7f83e40c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83e3fff700 1 -- 172.21.15.104:0/69481407 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e40b04f0 con 0x7f83e40afd80 2023-07-14T11:53:09.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83e3fff700 1 --2- 172.21.15.104:0/69481407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83e40afd80 0x7f83e40711a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f83e40adca0 tx=0x7f83d4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7446bb56ce516baa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:09.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83e37fe700 1 -- 172.21.15.104:0/69481407 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d4011830 con 0x7f83e40afd80 2023-07-14T11:53:09.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83e37fe700 1 -- 172.21.15.104:0/69481407 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d4011990 con 0x7f83e40afd80 2023-07-14T11:53:09.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83e37fe700 1 -- 172.21.15.104:0/69481407 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d4011d20 con 0x7f83e40afd80 2023-07-14T11:53:09.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83eabc6700 1 -- 172.21.15.104:0/69481407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83e40afd80 msgr2=0x7f83e40711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:09.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83eabc6700 1 --2- 172.21.15.104:0/69481407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83e40afd80 0x7f83e40711a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f83e40adca0 tx=0x7f83d4009bf0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83eabc6700 1 -- 172.21.15.104:0/69481407 shutdown_connections 2023-07-14T11:53:09.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83eabc6700 1 --2- 172.21.15.104:0/69481407 >> v2:172.21.15.121:3300/0 conn(0x7f83e40716e0 0x7f83e40c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83eabc6700 1 --2- 172.21.15.104:0/69481407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83e40afd80 0x7f83e40711a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83eabc6700 1 --2- 172.21.15.104:0/69481407 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83e415df10 0x7f83e415e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83eabc6700 1 -- 172.21.15.104:0/69481407 >> 172.21.15.104:0/69481407 conn(0x7f83e40cad80 msgr2=0x7f83e40cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:09.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83eabc6700 1 -- 172.21.15.104:0/69481407 shutdown_connections 2023-07-14T11:53:09.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.761+0000 7f83eabc6700 1 -- 172.21.15.104:0/69481407 wait complete. 2023-07-14T11:53:09.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.765+0000 7f83eabc6700 1 Processor -- start 2023-07-14T11:53:09.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.765+0000 7f83eabc6700 1 -- start start 2023-07-14T11:53:09.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.765+0000 7f83eabc6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83e40716e0 0x7f83e41492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:09.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83eabc6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e40afd80 0x7f83e414be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:09.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83eabc6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83e41f1e30 0x7f83e414a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:09.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83eabc6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e414a550 con 0x7f83e40716e0 2023-07-14T11:53:09.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83eabc6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e414c340 con 0x7f83e40afd80 2023-07-14T11:53:09.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83eabc6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e4149940 con 0x7f83e41f1e30 2023-07-14T11:53:09.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83e8962700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83e40716e0 0x7f83e41492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:09.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83e8962700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83e40716e0 0x7f83e41492b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49278/0 (socket says 172.21.15.104:49278) 2023-07-14T11:53:09.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83e8962700 1 -- 172.21.15.104:0/3031363422 learned_addr learned my addr 172.21.15.104:0/3031363422 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:09.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83e9163700 1 --2- 172.21.15.104:0/3031363422 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83e41f1e30 0x7f83e414a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:09.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83e3fff700 1 --2- 172.21.15.104:0/3031363422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e40afd80 0x7f83e414be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:09.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83e8962700 1 -- 172.21.15.104:0/3031363422 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83e41f1e30 msgr2=0x7f83e414a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:09.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83e8962700 1 --2- 172.21.15.104:0/3031363422 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83e41f1e30 0x7f83e414a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83e8962700 1 -- 172.21.15.104:0/3031363422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e40afd80 msgr2=0x7f83e414be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:09.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83e8962700 1 --2- 172.21.15.104:0/3031363422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e40afd80 0x7f83e414be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83e8962700 1 -- 172.21.15.104:0/3031363422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d4009570 con 0x7f83e40716e0 2023-07-14T11:53:09.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83e8962700 1 --2- 172.21.15.104:0/3031363422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83e40716e0 0x7f83e41492b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f83d800e840 tx=0x7f83d800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:09.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83e17fa700 1 -- 172.21.15.104:0/3031363422 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d80095a0 con 0x7f83e40716e0 2023-07-14T11:53:09.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83e17fa700 1 -- 172.21.15.104:0/3031363422 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d800c690 con 0x7f83e40716e0 2023-07-14T11:53:09.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83eabc6700 1 -- 172.21.15.104:0/3031363422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e4149c20 con 0x7f83e40716e0 2023-07-14T11:53:09.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83e17fa700 1 -- 172.21.15.104:0/3031363422 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d800ca00 con 0x7f83e40716e0 2023-07-14T11:53:09.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83eabc6700 1 -- 172.21.15.104:0/3031363422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e414a1e0 con 0x7f83e40716e0 2023-07-14T11:53:09.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.769+0000 7f83eabc6700 1 -- 172.21.15.104:0/3031363422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e404dcb0 con 0x7f83e40716e0 2023-07-14T11:53:09.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.773+0000 7f83e17fa700 1 -- 172.21.15.104:0/3031363422 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f83d800cb60 con 0x7f83e40716e0 2023-07-14T11:53:09.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.773+0000 7f83e17fa700 1 --2- 172.21.15.104:0/3031363422 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83d00662f0 0x7f83d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:09.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.773+0000 7f83e17fa700 1 -- 172.21.15.104:0/3031363422 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f83d8083900 con 0x7f83e40716e0 2023-07-14T11:53:09.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.773+0000 7f83e3fff700 1 --2- 172.21.15.104:0/3031363422 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83d00662f0 0x7f83d00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:09.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.773+0000 7f83e3fff700 1 --2- 172.21.15.104:0/3031363422 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83d00662f0 0x7f83d00687a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f83d4000c00 tx=0x7f83d4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:09.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.781+0000 7f83e17fa700 1 -- 172.21.15.104:0/3031363422 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f83d8054bf0 con 0x7f83e40716e0 2023-07-14T11:53:09.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.953+0000 7f83eabc6700 1 -- 172.21.15.104:0/3031363422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83e4158d50 con 0x7f83e40716e0 2023-07-14T11:53:09.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.953+0000 7f83e17fa700 1 -- 172.21.15.104:0/3031363422 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83d80170a0 con 0x7f83e40716e0 2023-07-14T11:53:09.940 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:53:09.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.957+0000 7f83eabc6700 1 -- 172.21.15.104:0/3031363422 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83d00662f0 msgr2=0x7f83d00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:09.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.957+0000 7f83eabc6700 1 --2- 172.21.15.104:0/3031363422 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83d00662f0 0x7f83d00687a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f83d4000c00 tx=0x7f83d4008030 comp rx=0 tx=0).stop 2023-07-14T11:53:09.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.957+0000 7f83eabc6700 1 -- 172.21.15.104:0/3031363422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83e40716e0 msgr2=0x7f83e41492b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:09.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.957+0000 7f83eabc6700 1 --2- 172.21.15.104:0/3031363422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83e40716e0 0x7f83e41492b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f83d800e840 tx=0x7f83d800ec00 comp rx=0 tx=0).stop 2023-07-14T11:53:09.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.957+0000 7f83eabc6700 1 -- 172.21.15.104:0/3031363422 shutdown_connections 2023-07-14T11:53:09.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.957+0000 7f83eabc6700 1 --2- 172.21.15.104:0/3031363422 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83d00662f0 0x7f83d00687a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.957+0000 7f83eabc6700 1 --2- 172.21.15.104:0/3031363422 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83e41f1e30 0x7f83e414a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.957+0000 7f83eabc6700 1 --2- 172.21.15.104:0/3031363422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e40afd80 0x7f83e414be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.957+0000 7f83eabc6700 1 --2- 172.21.15.104:0/3031363422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83e40716e0 0x7f83e41492b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:09.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.957+0000 7f83eabc6700 1 -- 172.21.15.104:0/3031363422 >> 172.21.15.104:0/3031363422 conn(0x7f83e40cad80 msgr2=0x7f83e4157fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:09.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.957+0000 7f83eabc6700 1 -- 172.21.15.104:0/3031363422 shutdown_connections 2023-07-14T11:53:09.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:09.957+0000 7f83eabc6700 1 -- 172.21.15.104:0/3031363422 wait complete. 2023-07-14T11:53:09.966 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:53:10.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.153+0000 7fa3d2903700 1 Processor -- start 2023-07-14T11:53:10.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.153+0000 7fa3d2903700 1 -- start start 2023-07-14T11:53:10.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.153+0000 7fa3d2903700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa3cc0d07e0 0x7fa3cc1669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.157+0000 7fa3d2903700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3cc0d1a60 con 0x7fa3cc0b0ee0 2023-07-14T11:53:10.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.157+0000 7fa3d2903700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3cc0d1ef0 con 0x7fa3cc0cd260 2023-07-14T11:53:10.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.157+0000 7fa3d2903700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3cc0d0c60 con 0x7fa3cc0d07e0 2023-07-14T11:53:10.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.157+0000 7fa3d1901700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fa3cc0b0ee0 0x7fa3cc0ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:41590/0 (socket says 172.21.15.104:41590) 2023-07-14T11:53:10.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.157+0000 7fa3d1901700 1 -- 172.21.15.104:0/1330254950 learned_addr learned my addr 172.21.15.104:0/1330254950 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:10.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.157+0000 7fa3d2102700 1 --2- 172.21.15.104:0/1330254950 >> v2:172.21.15.121:3300/0 conn(0x7fa3cc0d07e0 0x7fa3cc1669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.157+0000 7fa3d2102700 1 -- 172.21.15.104:0/1330254950 >> v1:172.21.15.121:6789/0 conn(0x7fa3cc0cd260 legacy=0x7fa3cc0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.157+0000 7fa3d2102700 1 -- 172.21.15.104:0/1330254950 >> v1:172.21.15.104:6789/0 conn(0x7fa3cc0b0ee0 legacy=0x7fa3cc0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.157+0000 7fa3d2102700 1 -- 172.21.15.104:0/1330254950 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc0d0e50 con 0x7fa3cc0d07e0 2023-07-14T11:53:10.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.157+0000 7fa3d2102700 1 --2- 172.21.15.104:0/1330254950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc0d07e0 0x7fa3cc1669b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009d90 tx=0x7fa3c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42b1f8c5d9c04499 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:10.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.157+0000 7fa3d08ff700 1 -- 172.21.15.104:0/1330254950 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c400e070 con 0x7fa3cc0d07e0 2023-07-14T11:53:10.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.157+0000 7fa3d08ff700 1 -- 172.21.15.104:0/1330254950 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c400bbd0 con 0x7fa3cc0d07e0 2023-07-14T11:53:10.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.157+0000 7fa3d08ff700 1 -- 172.21.15.104:0/1330254950 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4012430 con 0x7fa3cc0d07e0 2023-07-14T11:53:10.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.161+0000 7fa3d2903700 1 -- 172.21.15.104:0/1330254950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc0d07e0 msgr2=0x7fa3cc1669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.161+0000 7fa3d2903700 1 --2- 172.21.15.104:0/1330254950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc0d07e0 0x7fa3cc1669b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009d90 tx=0x7fa3c400b330 comp rx=0 tx=0).stop 2023-07-14T11:53:10.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.161+0000 7fa3d2903700 1 -- 172.21.15.104:0/1330254950 shutdown_connections 2023-07-14T11:53:10.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.161+0000 7fa3d2903700 1 --2- 172.21.15.104:0/1330254950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc0d07e0 0x7fa3cc1669b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.161+0000 7fa3d2903700 1 -- 172.21.15.104:0/1330254950 >> 172.21.15.104:0/1330254950 conn(0x7fa3cc060c40 msgr2=0x7fa3cc09b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:10.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.161+0000 7fa3d2903700 1 -- 172.21.15.104:0/1330254950 shutdown_connections 2023-07-14T11:53:10.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.161+0000 7fa3d2903700 1 -- 172.21.15.104:0/1330254950 wait complete. 2023-07-14T11:53:10.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.161+0000 7fa3d2903700 1 Processor -- start 2023-07-14T11:53:10.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.161+0000 7fa3d2903700 1 -- start start 2023-07-14T11:53:10.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d2903700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3cc0b0ee0 0x7fa3cc15b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d2903700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc0cd260 0x7fa3cc0ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d2903700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc0d07e0 0x7fa3cc1f4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d2903700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc0c7570 con 0x7fa3cc0b0ee0 2023-07-14T11:53:10.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d2903700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc0caad0 con 0x7fa3cc0cd260 2023-07-14T11:53:10.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d2903700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc0cadb0 con 0x7fa3cc0d07e0 2023-07-14T11:53:10.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d1901700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3cc0b0ee0 0x7fa3cc15b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d1901700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3cc0b0ee0 0x7fa3cc15b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49280/0 (socket says 172.21.15.104:49280) 2023-07-14T11:53:10.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d1901700 1 -- 172.21.15.104:0/1111995618 learned_addr learned my addr 172.21.15.104:0/1111995618 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:10.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d2102700 1 --2- 172.21.15.104:0/1111995618 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc0d07e0 0x7fa3cc1f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d1100700 1 --2- 172.21.15.104:0/1111995618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc0cd260 0x7fa3cc0ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d1901700 1 -- 172.21.15.104:0/1111995618 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc0d07e0 msgr2=0x7fa3cc1f4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d1901700 1 --2- 172.21.15.104:0/1111995618 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc0d07e0 0x7fa3cc1f4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d1901700 1 -- 172.21.15.104:0/1111995618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc0cd260 msgr2=0x7fa3cc0ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d1901700 1 --2- 172.21.15.104:0/1111995618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc0cd260 0x7fa3cc0ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d1901700 1 -- 172.21.15.104:0/1111995618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c4009a70 con 0x7fa3cc0b0ee0 2023-07-14T11:53:10.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d1901700 1 --2- 172.21.15.104:0/1111995618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3cc0b0ee0 0x7fa3cc15b670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8001eb0 tx=0x7fa3c800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:10.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3c27fc700 1 -- 172.21.15.104:0/1111995618 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8005d90 con 0x7fa3cc0b0ee0 2023-07-14T11:53:10.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3c27fc700 1 -- 172.21.15.104:0/1111995618 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c80070d0 con 0x7fa3cc0b0ee0 2023-07-14T11:53:10.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d2903700 1 -- 172.21.15.104:0/1111995618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc0c7200 con 0x7fa3cc0b0ee0 2023-07-14T11:53:10.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3c27fc700 1 -- 172.21.15.104:0/1111995618 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8007440 con 0x7fa3cc0b0ee0 2023-07-14T11:53:10.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d2903700 1 -- 172.21.15.104:0/1111995618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc0ca710 con 0x7fa3cc0b0ee0 2023-07-14T11:53:10.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.165+0000 7fa3d2903700 1 -- 172.21.15.104:0/1111995618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3cc0c7060 con 0x7fa3cc0b0ee0 2023-07-14T11:53:10.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.169+0000 7fa3c27fc700 1 -- 172.21.15.104:0/1111995618 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa3c80075a0 con 0x7fa3cc0b0ee0 2023-07-14T11:53:10.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.169+0000 7fa3c27fc700 1 --2- 172.21.15.104:0/1111995618 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3b0066320 0x7fa3b00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.169+0000 7fa3c27fc700 1 -- 172.21.15.104:0/1111995618 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa3c8082c40 con 0x7fa3cc0b0ee0 2023-07-14T11:53:10.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.169+0000 7fa3d1100700 1 --2- 172.21.15.104:0/1111995618 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3b0066320 0x7fa3b00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.169+0000 7fa3d1100700 1 --2- 172.21.15.104:0/1111995618 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3b0066320 0x7fa3b00687d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc160910 tx=0x7fa3bc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:10.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.177+0000 7fa3c27fc700 1 -- 172.21.15.104:0/1111995618 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fa3cc0c7060 con 0x7fa3cc0b0ee0 2023-07-14T11:53:10.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.353+0000 7fa3d2903700 1 -- 172.21.15.104:0/1111995618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa3cc0c7060 con 0x7fa3cc0b0ee0 2023-07-14T11:53:10.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.353+0000 7fa3c27fc700 1 -- 172.21.15.104:0/1111995618 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa3cc0c7060 con 0x7fa3cc0b0ee0 2023-07-14T11:53:10.340 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:53:10.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.357+0000 7fa3d2903700 1 -- 172.21.15.104:0/1111995618 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3b0066320 msgr2=0x7fa3b00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.357+0000 7fa3d2903700 1 --2- 172.21.15.104:0/1111995618 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3b0066320 0x7fa3b00687d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc160910 tx=0x7fa3bc009040 comp rx=0 tx=0).stop 2023-07-14T11:53:10.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.357+0000 7fa3d2903700 1 -- 172.21.15.104:0/1111995618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3cc0b0ee0 msgr2=0x7fa3cc15b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.357+0000 7fa3d2903700 1 --2- 172.21.15.104:0/1111995618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3cc0b0ee0 0x7fa3cc15b670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8001eb0 tx=0x7fa3c800c7b0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.357+0000 7fa3d2903700 1 -- 172.21.15.104:0/1111995618 shutdown_connections 2023-07-14T11:53:10.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.357+0000 7fa3d2903700 1 --2- 172.21.15.104:0/1111995618 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3b0066320 0x7fa3b00687d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.357+0000 7fa3d2903700 1 --2- 172.21.15.104:0/1111995618 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc0d07e0 0x7fa3cc1f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.357+0000 7fa3d2903700 1 --2- 172.21.15.104:0/1111995618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc0cd260 0x7fa3cc0ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.357+0000 7fa3d2903700 1 --2- 172.21.15.104:0/1111995618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3cc0b0ee0 0x7fa3cc15b670 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.357+0000 7fa3d2903700 1 -- 172.21.15.104:0/1111995618 >> 172.21.15.104:0/1111995618 conn(0x7fa3cc060c40 msgr2=0x7fa3cc0b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:10.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.357+0000 7fa3d2903700 1 -- 172.21.15.104:0/1111995618 shutdown_connections 2023-07-14T11:53:10.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.357+0000 7fa3d2903700 1 -- 172.21.15.104:0/1111995618 wait complete. 2023-07-14T11:53:10.365 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:53:10.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.549+0000 7ffa03315700 1 Processor -- start 2023-07-14T11:53:10.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.549+0000 7ffa03315700 1 -- start start 2023-07-14T11:53:10.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.549+0000 7ffa03315700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff9fc159f20 0x7ff9fc0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.549+0000 7ffa03315700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9fc0b0ef0 con 0x7ff9fc0caf70 2023-07-14T11:53:10.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.549+0000 7ffa03315700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9fc0ce9c0 con 0x7ff9fc0c9d90 2023-07-14T11:53:10.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.549+0000 7ffa03315700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9fc0cee50 con 0x7ff9fc159f20 2023-07-14T11:53:10.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.549+0000 7ffa02b14700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ff9fc0caf70 0x7ff9fc15d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:38134/0 (socket says 172.21.15.104:38134) 2023-07-14T11:53:10.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.549+0000 7ffa02b14700 1 -- 172.21.15.104:0/2660247022 learned_addr learned my addr 172.21.15.104:0/2660247022 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:10.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.549+0000 7ffa01b12700 1 --2- 172.21.15.104:0/2660247022 >> v2:172.21.15.104:3300/0 conn(0x7ff9fc159f20 0x7ff9fc0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.549+0000 7ffa01311700 1 -- 172.21.15.104:0/2660247022 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859823278 0 0) 0x7ff9fc0ce9c0 con 0x7ff9fc0c9d90 2023-07-14T11:53:10.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.549+0000 7ffa01311700 1 -- 172.21.15.104:0/2660247022 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9e80035d0 con 0x7ff9fc0c9d90 2023-07-14T11:53:10.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.549+0000 7ffa01311700 1 -- 172.21.15.104:0/2660247022 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983769735 0 0) 0x7ff9fc0b0ef0 con 0x7ff9fc0caf70 2023-07-14T11:53:10.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.549+0000 7ffa01311700 1 -- 172.21.15.104:0/2660247022 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9fc0ce9c0 con 0x7ff9fc0caf70 2023-07-14T11:53:10.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.553+0000 7ffa01b12700 1 -- 172.21.15.104:0/2660247022 >> v1:172.21.15.121:6789/0 conn(0x7ff9fc0c9d90 legacy=0x7ff9fc0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.553+0000 7ffa01b12700 1 -- 172.21.15.104:0/2660247022 >> v1:172.21.15.104:6789/0 conn(0x7ff9fc0caf70 legacy=0x7ff9fc15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.553+0000 7ffa01b12700 1 -- 172.21.15.104:0/2660247022 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9fc0cb4f0 con 0x7ff9fc159f20 2023-07-14T11:53:10.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.553+0000 7ffa01b12700 1 --2- 172.21.15.104:0/2660247022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc159f20 0x7ff9fc0caa30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009850 tx=0x7ff9ec009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcf556582a3facb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:10.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.553+0000 7ffa01311700 1 -- 172.21.15.104:0/2660247022 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec011830 con 0x7ff9fc159f20 2023-07-14T11:53:10.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.553+0000 7ffa01311700 1 -- 172.21.15.104:0/2660247022 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ec011990 con 0x7ff9fc159f20 2023-07-14T11:53:10.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.553+0000 7ffa01311700 1 -- 172.21.15.104:0/2660247022 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec011d40 con 0x7ff9fc159f20 2023-07-14T11:53:10.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.553+0000 7ffa03315700 1 -- 172.21.15.104:0/2660247022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc159f20 msgr2=0x7ff9fc0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.553+0000 7ffa03315700 1 --2- 172.21.15.104:0/2660247022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc159f20 0x7ff9fc0caa30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009850 tx=0x7ff9ec009b60 comp rx=0 tx=0).stop 2023-07-14T11:53:10.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.553+0000 7ffa03315700 1 -- 172.21.15.104:0/2660247022 shutdown_connections 2023-07-14T11:53:10.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.553+0000 7ffa03315700 1 --2- 172.21.15.104:0/2660247022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc159f20 0x7ff9fc0caa30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.553+0000 7ffa03315700 1 -- 172.21.15.104:0/2660247022 >> 172.21.15.104:0/2660247022 conn(0x7ff9fc06e960 msgr2=0x7ff9fc06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:10.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.553+0000 7ffa03315700 1 -- 172.21.15.104:0/2660247022 shutdown_connections 2023-07-14T11:53:10.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.553+0000 7ffa03315700 1 -- 172.21.15.104:0/2660247022 wait complete. 2023-07-14T11:53:10.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa03315700 1 Processor -- start 2023-07-14T11:53:10.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa03315700 1 -- start start 2023-07-14T11:53:10.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa03315700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9fc0c9d90 0x7ff9fc151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa03315700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc0caf70 0x7ff9fc153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa03315700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9fc159f20 0x7ff9fc1509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa03315700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc1514a0 con 0x7ff9fc0caf70 2023-07-14T11:53:10.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa03315700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc0b3130 con 0x7ff9fc0c9d90 2023-07-14T11:53:10.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa03315700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc150f20 con 0x7ff9fc159f20 2023-07-14T11:53:10.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa01b12700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc0caf70 0x7ff9fc153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa01b12700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc0caf70 0x7ff9fc153870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36626/0 (socket says 172.21.15.104:36626) 2023-07-14T11:53:10.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa01b12700 1 -- 172.21.15.104:0/3576584628 learned_addr learned my addr 172.21.15.104:0/3576584628 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:10.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa02313700 1 --2- 172.21.15.104:0/3576584628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9fc0c9d90 0x7ff9fc151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa02b14700 1 --2- 172.21.15.104:0/3576584628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9fc159f20 0x7ff9fc1509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa01b12700 1 -- 172.21.15.104:0/3576584628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9fc159f20 msgr2=0x7ff9fc1509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa01b12700 1 --2- 172.21.15.104:0/3576584628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9fc159f20 0x7ff9fc1509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa01b12700 1 -- 172.21.15.104:0/3576584628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9fc0c9d90 msgr2=0x7ff9fc151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa01b12700 1 --2- 172.21.15.104:0/3576584628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9fc0c9d90 0x7ff9fc151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa01b12700 1 -- 172.21.15.104:0/3576584628 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0094b0 con 0x7ff9fc0caf70 2023-07-14T11:53:10.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.557+0000 7ffa02b14700 1 --2- 172.21.15.104:0/3576584628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9fc159f20 0x7ff9fc1509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:10.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.561+0000 7ffa01b12700 1 --2- 172.21.15.104:0/3576584628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc0caf70 0x7ff9fc153870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009eb0 tx=0x7ff9ec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:10.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.561+0000 7ff9f2ffd700 1 -- 172.21.15.104:0/3576584628 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec01aa10 con 0x7ff9fc0caf70 2023-07-14T11:53:10.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.561+0000 7ff9f2ffd700 1 -- 172.21.15.104:0/3576584628 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ec01f070 con 0x7ff9fc0caf70 2023-07-14T11:53:10.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.561+0000 7ffa03315700 1 -- 172.21.15.104:0/3576584628 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc0b2800 con 0x7ff9fc0caf70 2023-07-14T11:53:10.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.561+0000 7ff9f2ffd700 1 -- 172.21.15.104:0/3576584628 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec01adf0 con 0x7ff9fc0caf70 2023-07-14T11:53:10.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.561+0000 7ffa03315700 1 -- 172.21.15.104:0/3576584628 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9fc0b2dc0 con 0x7ff9fc0caf70 2023-07-14T11:53:10.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.561+0000 7ffa03315700 1 -- 172.21.15.104:0/3576584628 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9fc0b32c0 con 0x7ff9fc0caf70 2023-07-14T11:53:10.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.565+0000 7ff9f2ffd700 1 -- 172.21.15.104:0/3576584628 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff9ec01b430 con 0x7ff9fc0caf70 2023-07-14T11:53:10.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.565+0000 7ff9f2ffd700 1 --2- 172.21.15.104:0/3576584628 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9e00662f0 0x7ff9e00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.565+0000 7ff9f2ffd700 1 -- 172.21.15.104:0/3576584628 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff9ec085270 con 0x7ff9fc0caf70 2023-07-14T11:53:10.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.565+0000 7ffa02313700 1 --2- 172.21.15.104:0/3576584628 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9e00662f0 0x7ff9e00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.565+0000 7ffa02313700 1 --2- 172.21.15.104:0/3576584628 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9e00662f0 0x7ff9e00687a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80011e0 tx=0x7ff9f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:10.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.569+0000 7ff9f2ffd700 1 -- 172.21.15.104:0/3576584628 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff9ec052ab0 con 0x7ff9fc0caf70 2023-07-14T11:53:10.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.777+0000 7ffa03315700 1 -- 172.21.15.104:0/3576584628 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9fc04e500 con 0x7ff9fc0caf70 2023-07-14T11:53:10.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.777+0000 7ff9f2ffd700 1 -- 172.21.15.104:0/3576584628 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3536 (secure 0 0 0) 0x7ff9ec033de0 con 0x7ff9fc0caf70 2023-07-14T11:53:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:10.765 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2103358277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2103358277},{"type":"v1","addr":"172.21.15.121:6833","nonce":2103358277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1841065287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1841065287},{"type":"v1","addr":"172.21.15.121:6835","nonce":1841065287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3342053518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3342053518},{"type":"v1","addr":"172.21.15.104:6835","nonce":3342053518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2666329866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2666329866},{"type":"v1","addr":"172.21.15.104:6837","nonce":2666329866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-14T11:53:10.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.781+0000 7ffa03315700 1 -- 172.21.15.104:0/3576584628 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9e00662f0 msgr2=0x7ff9e00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.781+0000 7ffa03315700 1 --2- 172.21.15.104:0/3576584628 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9e00662f0 0x7ff9e00687a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80011e0 tx=0x7ff9f8009040 comp rx=0 tx=0).stop 2023-07-14T11:53:10.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.781+0000 7ffa03315700 1 -- 172.21.15.104:0/3576584628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc0caf70 msgr2=0x7ff9fc153870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.781+0000 7ffa03315700 1 --2- 172.21.15.104:0/3576584628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc0caf70 0x7ff9fc153870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009eb0 tx=0x7ff9ec004770 comp rx=0 tx=0).stop 2023-07-14T11:53:10.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.781+0000 7ffa03315700 1 -- 172.21.15.104:0/3576584628 shutdown_connections 2023-07-14T11:53:10.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.781+0000 7ffa03315700 1 --2- 172.21.15.104:0/3576584628 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9e00662f0 0x7ff9e00687a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.781+0000 7ffa03315700 1 --2- 172.21.15.104:0/3576584628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9fc0caf70 0x7ff9fc153870 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.781+0000 7ffa03315700 1 --2- 172.21.15.104:0/3576584628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9fc159f20 0x7ff9fc1509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.781+0000 7ffa03315700 1 --2- 172.21.15.104:0/3576584628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9fc0c9d90 0x7ff9fc151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.781+0000 7ffa03315700 1 -- 172.21.15.104:0/3576584628 >> 172.21.15.104:0/3576584628 conn(0x7ff9fc06e960 msgr2=0x7ff9fc15c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:10.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.781+0000 7ffa03315700 1 -- 172.21.15.104:0/3576584628 shutdown_connections 2023-07-14T11:53:10.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.781+0000 7ffa03315700 1 -- 172.21.15.104:0/3576584628 wait complete. 2023-07-14T11:53:10.771 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 3 2023-07-14T11:53:10.789 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:10.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.977+0000 7f169d72d700 1 Processor -- start 2023-07-14T11:53:10.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.977+0000 7f169d72d700 1 -- start start 2023-07-14T11:53:10.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.977+0000 7f169d72d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1698159b70 0x7f169806ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.977+0000 7f169d72d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1698159f40 0x7f169806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.977+0000 7f169d72d700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16980cba60 con 0x7f169806e230 2023-07-14T11:53:10.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.977+0000 7f169d72d700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16980cbd40 con 0x7f1698159b70 2023-07-14T11:53:10.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.977+0000 7f169d72d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16980cc020 con 0x7f1698159f40 2023-07-14T11:53:10.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f1696ffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1698159b70 0x7f169806ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f1696ffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1698159b70 0x7f169806ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36634/0 (socket says 172.21.15.104:36634) 2023-07-14T11:53:10.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f1696ffd700 1 -- 172.21.15.104:0/4226066594 learned_addr learned my addr 172.21.15.104:0/4226066594 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:10.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f168ffff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1698159f40 0x7f169806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f1696ffd700 1 -- 172.21.15.104:0/4226066594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1698159f40 msgr2=0x7f169806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:10.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f1696ffd700 1 --2- 172.21.15.104:0/4226066594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1698159f40 0x7f169806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f1696ffd700 1 -- 172.21.15.104:0/4226066594 >> v1:172.21.15.121:6789/0 conn(0x7f169806e230 legacy=0x7f169806f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f1696ffd700 1 -- 172.21.15.104:0/4226066594 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169806d7d0 con 0x7f1698159b70 2023-07-14T11:53:10.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f1696ffd700 1 --2- 172.21.15.104:0/4226066594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1698159b70 0x7f169806ec00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1688009910 tx=0x7f1688009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ef7c46b7d2fe33 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:10.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f16967fc700 1 -- 172.21.15.104:0/4226066594 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688011830 con 0x7f1698159b70 2023-07-14T11:53:10.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f16967fc700 1 -- 172.21.15.104:0/4226066594 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1688011990 con 0x7f1698159b70 2023-07-14T11:53:10.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f16967fc700 1 -- 172.21.15.104:0/4226066594 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688011d40 con 0x7f1698159b70 2023-07-14T11:53:10.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f169d72d700 1 -- 172.21.15.104:0/4226066594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1698159b70 msgr2=0x7f169806ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f169d72d700 1 --2- 172.21.15.104:0/4226066594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1698159b70 0x7f169806ec00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1688009910 tx=0x7f1688009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:10.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f169d72d700 1 -- 172.21.15.104:0/4226066594 shutdown_connections 2023-07-14T11:53:10.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f169d72d700 1 --2- 172.21.15.104:0/4226066594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1698159f40 0x7f169806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f169d72d700 1 --2- 172.21.15.104:0/4226066594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1698159b70 0x7f169806ec00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f169d72d700 1 -- 172.21.15.104:0/4226066594 >> 172.21.15.104:0/4226066594 conn(0x7f16980cc500 msgr2=0x7f16980cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:10.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f169d72d700 1 -- 172.21.15.104:0/4226066594 shutdown_connections 2023-07-14T11:53:10.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.981+0000 7f169d72d700 1 -- 172.21.15.104:0/4226066594 wait complete. 2023-07-14T11:53:10.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.985+0000 7f169d72d700 1 Processor -- start 2023-07-14T11:53:10.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.985+0000 7f169d72d700 1 -- start start 2023-07-14T11:53:10.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.985+0000 7f169d72d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169806e230 0x7f1698147720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.985+0000 7f169d72d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1698159b70 0x7f16980c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f169d72d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1698159f40 0x7f1698145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f169d72d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16980c87a0 con 0x7f1698159b70 2023-07-14T11:53:10.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f169d72d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16981486a0 con 0x7f169806e230 2023-07-14T11:53:10.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f169d72d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16980c7330 con 0x7f1698159f40 2023-07-14T11:53:10.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f168ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1698159b70 0x7f16980c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f168ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1698159b70 0x7f16980c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36640/0 (socket says 172.21.15.104:36640) 2023-07-14T11:53:10.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f168ffff700 1 -- 172.21.15.104:0/2764566337 learned_addr learned my addr 172.21.15.104:0/2764566337 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:10.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f1697fff700 1 --2- 172.21.15.104:0/2764566337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1698159f40 0x7f1698145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f1696ffd700 1 --2- 172.21.15.104:0/2764566337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169806e230 0x7f1698147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f168ffff700 1 -- 172.21.15.104:0/2764566337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1698159f40 msgr2=0x7f1698145690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f168ffff700 1 --2- 172.21.15.104:0/2764566337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1698159f40 0x7f1698145690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f168ffff700 1 -- 172.21.15.104:0/2764566337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169806e230 msgr2=0x7f1698147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f168ffff700 1 --2- 172.21.15.104:0/2764566337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169806e230 0x7f1698147720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f168ffff700 1 -- 172.21.15.104:0/2764566337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1688009570 con 0x7f1698159b70 2023-07-14T11:53:10.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f168ffff700 1 --2- 172.21.15.104:0/2764566337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1698159b70 0x7f16980c6df0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f168000e9b0 tx=0x7f168000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:10.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f168f7fe700 1 -- 172.21.15.104:0/2764566337 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680009710 con 0x7f1698159b70 2023-07-14T11:53:10.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f168f7fe700 1 -- 172.21.15.104:0/2764566337 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168000c850 con 0x7f1698159b70 2023-07-14T11:53:10.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f168f7fe700 1 -- 172.21.15.104:0/2764566337 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168000cbc0 con 0x7f1698159b70 2023-07-14T11:53:10.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f169d72d700 1 -- 172.21.15.104:0/2764566337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1698147e00 con 0x7f1698159b70 2023-07-14T11:53:10.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f169d72d700 1 -- 172.21.15.104:0/2764566337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16981483c0 con 0x7f1698159b70 2023-07-14T11:53:10.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.989+0000 7f169d72d700 1 -- 172.21.15.104:0/2764566337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169804dcb0 con 0x7f1698159b70 2023-07-14T11:53:10.979 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.993+0000 7f168f7fe700 1 -- 172.21.15.104:0/2764566337 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f168000cd20 con 0x7f1698159b70 2023-07-14T11:53:10.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.993+0000 7f168f7fe700 1 --2- 172.21.15.104:0/2764566337 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16780662f0 0x7f16780687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.993+0000 7f168f7fe700 1 -- 172.21.15.104:0/2764566337 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1680083e70 con 0x7f1698159b70 2023-07-14T11:53:10.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.993+0000 7f1696ffd700 1 --2- 172.21.15.104:0/2764566337 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16780662f0 0x7f16780687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.982 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:10.997+0000 7f1696ffd700 1 --2- 172.21.15.104:0/2764566337 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16780662f0 0x7f16780687a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1688014040 tx=0x7f1688008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:10.987 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.001+0000 7f168f7fe700 1 -- 172.21.15.104:0/2764566337 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f16800550b0 con 0x7f1698159b70 2023-07-14T11:53:11.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.177+0000 7f169d72d700 1 -- 172.21.15.104:0/2764566337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16980b2420 con 0x7f1698159b70 2023-07-14T11:53:11.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.177+0000 7f168f7fe700 1 -- 172.21.15.104:0/2764566337 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+14672 (secure 0 0 0) 0x7f1680017090 con 0x7f1698159b70 2023-07-14T11:53:11.165 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:11.166 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":20,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:10.141551+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:11.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.181+0000 7f169d72d700 1 -- 172.21.15.104:0/2764566337 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16780662f0 msgr2=0x7f16780687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.181+0000 7f169d72d700 1 --2- 172.21.15.104:0/2764566337 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16780662f0 0x7f16780687a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1688014040 tx=0x7f1688008030 comp rx=0 tx=0).stop 2023-07-14T11:53:11.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.181+0000 7f169d72d700 1 -- 172.21.15.104:0/2764566337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1698159b70 msgr2=0x7f16980c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.181+0000 7f169d72d700 1 --2- 172.21.15.104:0/2764566337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1698159b70 0x7f16980c6df0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f168000e9b0 tx=0x7f168000ed70 comp rx=0 tx=0).stop 2023-07-14T11:53:11.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.181+0000 7f169d72d700 1 -- 172.21.15.104:0/2764566337 shutdown_connections 2023-07-14T11:53:11.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.181+0000 7f169d72d700 1 --2- 172.21.15.104:0/2764566337 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16780662f0 0x7f16780687a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.181+0000 7f169d72d700 1 --2- 172.21.15.104:0/2764566337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1698159b70 0x7f16980c6df0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.181+0000 7f169d72d700 1 --2- 172.21.15.104:0/2764566337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1698159f40 0x7f1698145690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.181+0000 7f169d72d700 1 --2- 172.21.15.104:0/2764566337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169806e230 0x7f1698147720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.181+0000 7f169d72d700 1 -- 172.21.15.104:0/2764566337 >> 172.21.15.104:0/2764566337 conn(0x7f16980cc500 msgr2=0x7f16980cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:11.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.181+0000 7f169d72d700 1 -- 172.21.15.104:0/2764566337 shutdown_connections 2023-07-14T11:53:11.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.181+0000 7f169d72d700 1 -- 172.21.15.104:0/2764566337 wait complete. 2023-07-14T11:53:11.189 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:53:11.189 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:53:11.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.385+0000 7fef6043b700 1 Processor -- start 2023-07-14T11:53:11.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.385+0000 7fef6043b700 1 -- start start 2023-07-14T11:53:11.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.385+0000 7fef6043b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fef580cf710 0x7fef580d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:11.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.385+0000 7fef6043b700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef58071d10 con 0x7fef580b1c50 2023-07-14T11:53:11.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.385+0000 7fef6043b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef5815c420 con 0x7fef580d0210 2023-07-14T11:53:11.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.385+0000 7fef6043b700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef58070840 con 0x7fef580cf710 2023-07-14T11:53:11.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.385+0000 7fef5e9d8700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fef580cf710 0x7fef580d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:11.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.385+0000 7fef5e9d8700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fef580cf710 0x7fef580d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36646/0 (socket says 172.21.15.104:36646) 2023-07-14T11:53:11.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.385+0000 7fef5e9d8700 1 -- 172.21.15.104:0/3822304650 learned_addr learned my addr 172.21.15.104:0/3822304650 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:11.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef5d1d5700 1 -- 172.21.15.104:0/3822304650 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333184648 0 0) 0x7fef5815c420 con 0x7fef580d0210 2023-07-14T11:53:11.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef5d1d5700 1 -- 172.21.15.104:0/3822304650 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef440035d0 con 0x7fef580d0210 2023-07-14T11:53:11.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef5e9d8700 1 -- 172.21.15.104:0/3822304650 >> v1:172.21.15.121:6789/0 conn(0x7fef580d0210 legacy=0x7fef580cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef5e9d8700 1 -- 172.21.15.104:0/3822304650 >> v1:172.21.15.104:6789/0 conn(0x7fef580b1c50 legacy=0x7fef58071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef5d1d5700 1 -- 172.21.15.104:0/3822304650 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019860969 0 0) 0x7fef54001110 con 0x7fef580b1c50 2023-07-14T11:53:11.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef5e9d8700 1 -- 172.21.15.104:0/3822304650 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef58070b20 con 0x7fef580cf710 2023-07-14T11:53:11.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef5e9d8700 1 --2- 172.21.15.104:0/3822304650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef580cf710 0x7fef580d0640 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef50009e50 tx=0x7fef5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ccc0be49d29026 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:11.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef5d1d5700 1 -- 172.21.15.104:0/3822304650 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5000e070 con 0x7fef580cf710 2023-07-14T11:53:11.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef5d1d5700 1 -- 172.21.15.104:0/3822304650 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5000b910 con 0x7fef580cf710 2023-07-14T11:53:11.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef5d1d5700 1 -- 172.21.15.104:0/3822304650 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5000bcc0 con 0x7fef580cf710 2023-07-14T11:53:11.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef6043b700 1 -- 172.21.15.104:0/3822304650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef580cf710 msgr2=0x7fef580d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef6043b700 1 --2- 172.21.15.104:0/3822304650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef580cf710 0x7fef580d0640 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef50009e50 tx=0x7fef5000b330 comp rx=0 tx=0).stop 2023-07-14T11:53:11.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef6043b700 1 -- 172.21.15.104:0/3822304650 shutdown_connections 2023-07-14T11:53:11.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef6043b700 1 --2- 172.21.15.104:0/3822304650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef580cf710 0x7fef580d0640 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef6043b700 1 -- 172.21.15.104:0/3822304650 >> 172.21.15.104:0/3822304650 conn(0x7fef5809b9c0 msgr2=0x7fef580d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:11.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef6043b700 1 -- 172.21.15.104:0/3822304650 shutdown_connections 2023-07-14T11:53:11.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.389+0000 7fef6043b700 1 -- 172.21.15.104:0/3822304650 wait complete. 2023-07-14T11:53:11.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef6043b700 1 Processor -- start 2023-07-14T11:53:11.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef6043b700 1 -- start start 2023-07-14T11:53:11.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef6043b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef580b1c50 0x7fef58159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:11.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef6043b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef580cf710 0x7fef581e5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:11.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef6043b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef580d0210 0x7fef5809f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:11.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef6043b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef581e5770 con 0x7fef580b1c50 2023-07-14T11:53:11.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef6043b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef58159cd0 con 0x7fef580d0210 2023-07-14T11:53:11.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef6043b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef581e6140 con 0x7fef580cf710 2023-07-14T11:53:11.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef5e1d7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef580b1c50 0x7fef58159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:11.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef5e1d7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef580b1c50 0x7fef58159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36650/0 (socket says 172.21.15.104:36650) 2023-07-14T11:53:11.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef5e1d7700 1 -- 172.21.15.104:0/3577816174 learned_addr learned my addr 172.21.15.104:0/3577816174 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:11.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef5e9d8700 1 --2- 172.21.15.104:0/3577816174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef580d0210 0x7fef5809f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:11.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef5d9d6700 1 --2- 172.21.15.104:0/3577816174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef580cf710 0x7fef581e5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:11.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef5e9d8700 1 -- 172.21.15.104:0/3577816174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef580cf710 msgr2=0x7fef581e5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef5e9d8700 1 --2- 172.21.15.104:0/3577816174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef580cf710 0x7fef581e5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef5e9d8700 1 -- 172.21.15.104:0/3577816174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef580b1c50 msgr2=0x7fef58159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef5e9d8700 1 --2- 172.21.15.104:0/3577816174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef580b1c50 0x7fef58159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef5e9d8700 1 -- 172.21.15.104:0/3577816174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef50009b30 con 0x7fef580d0210 2023-07-14T11:53:11.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.393+0000 7fef5e1d7700 1 --2- 172.21.15.104:0/3577816174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef580b1c50 0x7fef58159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:11.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.397+0000 7fef5e9d8700 1 --2- 172.21.15.104:0/3577816174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef580d0210 0x7fef5809f300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fef50015000 tx=0x7fef50004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:11.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.397+0000 7fef4effd700 1 -- 172.21.15.104:0/3577816174 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5001c640 con 0x7fef580d0210 2023-07-14T11:53:11.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.397+0000 7fef4effd700 1 -- 172.21.15.104:0/3577816174 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5001c7c0 con 0x7fef580d0210 2023-07-14T11:53:11.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.397+0000 7fef6043b700 1 -- 172.21.15.104:0/3577816174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef581e4b60 con 0x7fef580d0210 2023-07-14T11:53:11.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.397+0000 7fef4effd700 1 -- 172.21.15.104:0/3577816174 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5001cb70 con 0x7fef580d0210 2023-07-14T11:53:11.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.397+0000 7fef6043b700 1 -- 172.21.15.104:0/3577816174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef581e5120 con 0x7fef580d0210 2023-07-14T11:53:11.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.397+0000 7fef6043b700 1 -- 172.21.15.104:0/3577816174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef581e58e0 con 0x7fef580d0210 2023-07-14T11:53:11.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.401+0000 7fef4effd700 1 -- 172.21.15.104:0/3577816174 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fef5001ccd0 con 0x7fef580d0210 2023-07-14T11:53:11.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.401+0000 7fef4effd700 1 --2- 172.21.15.104:0/3577816174 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fef3c066320 0x7fef3c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:11.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.401+0000 7fef4effd700 1 -- 172.21.15.104:0/3577816174 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fef50085170 con 0x7fef580d0210 2023-07-14T11:53:11.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.401+0000 7fef5e1d7700 1 --2- 172.21.15.104:0/3577816174 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fef3c066320 0x7fef3c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:11.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.401+0000 7fef5e1d7700 1 --2- 172.21.15.104:0/3577816174 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fef3c066320 0x7fef3c0687d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef54000f60 tx=0x7fef54009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:11.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.409+0000 7fef4effd700 1 -- 172.21.15.104:0/3577816174 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fef50052930 con 0x7fef580d0210 2023-07-14T11:53:11.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.581+0000 7fef6043b700 1 -- 172.21.15.104:0/3577816174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fef5804dcb0 con 0x7fef580d0210 2023-07-14T11:53:11.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.585+0000 7fef4effd700 1 -- 172.21.15.104:0/3577816174 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fef50008260 con 0x7fef580d0210 2023-07-14T11:53:11.571 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:53:11.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.589+0000 7fef6043b700 1 -- 172.21.15.104:0/3577816174 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fef3c066320 msgr2=0x7fef3c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.589+0000 7fef6043b700 1 --2- 172.21.15.104:0/3577816174 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fef3c066320 0x7fef3c0687d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef54000f60 tx=0x7fef54009040 comp rx=0 tx=0).stop 2023-07-14T11:53:11.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.589+0000 7fef6043b700 1 -- 172.21.15.104:0/3577816174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef580d0210 msgr2=0x7fef5809f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.589+0000 7fef6043b700 1 --2- 172.21.15.104:0/3577816174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef580d0210 0x7fef5809f300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fef50015000 tx=0x7fef50004af0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.589+0000 7fef6043b700 1 -- 172.21.15.104:0/3577816174 shutdown_connections 2023-07-14T11:53:11.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.589+0000 7fef6043b700 1 --2- 172.21.15.104:0/3577816174 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fef3c066320 0x7fef3c0687d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.589+0000 7fef6043b700 1 --2- 172.21.15.104:0/3577816174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef580d0210 0x7fef5809f300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.589+0000 7fef6043b700 1 --2- 172.21.15.104:0/3577816174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef580cf710 0x7fef581e5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.589+0000 7fef6043b700 1 --2- 172.21.15.104:0/3577816174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef580b1c50 0x7fef58159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.589+0000 7fef6043b700 1 -- 172.21.15.104:0/3577816174 >> 172.21.15.104:0/3577816174 conn(0x7fef5809b9c0 msgr2=0x7fef580b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:11.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.589+0000 7fef6043b700 1 -- 172.21.15.104:0/3577816174 shutdown_connections 2023-07-14T11:53:11.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.589+0000 7fef6043b700 1 -- 172.21.15.104:0/3577816174 wait complete. 2023-07-14T11:53:11.597 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:53:11.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.789+0000 7ff92d3d9700 1 Processor -- start 2023-07-14T11:53:11.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.789+0000 7ff92d3d9700 1 -- start start 2023-07-14T11:53:11.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff92d3d9700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9280d07e0 0x7ff9281669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:11.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff92d3d9700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9280d1a60 con 0x7ff9280b0ee0 2023-07-14T11:53:11.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff92d3d9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9280d1ef0 con 0x7ff9280cd260 2023-07-14T11:53:11.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff92d3d9700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9280d0c60 con 0x7ff9280d07e0 2023-07-14T11:53:11.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff927fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ff9280b0ee0 0x7ff9280ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:38152/0 (socket says 172.21.15.104:38152) 2023-07-14T11:53:11.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff927fff700 1 -- 172.21.15.104:0/3857738172 learned_addr learned my addr 172.21.15.104:0/3857738172 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:11.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff92cbd8700 1 --2- 172.21.15.104:0/3857738172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9280d07e0 0x7ff9281669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:11.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff926ffd700 1 -- 172.21.15.104:0/3857738172 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875281422 0 0) 0x7ff9280d1a60 con 0x7ff9280b0ee0 2023-07-14T11:53:11.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff926ffd700 1 -- 172.21.15.104:0/3857738172 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9100035d0 con 0x7ff9280b0ee0 2023-07-14T11:53:11.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff92cbd8700 1 -- 172.21.15.104:0/3857738172 >> v1:172.21.15.121:6789/0 conn(0x7ff9280cd260 legacy=0x7ff9280d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff92cbd8700 1 -- 172.21.15.104:0/3857738172 >> v1:172.21.15.104:6789/0 conn(0x7ff9280b0ee0 legacy=0x7ff9280ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff92cbd8700 1 -- 172.21.15.104:0/3857738172 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9280d0e50 con 0x7ff9280d07e0 2023-07-14T11:53:11.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff92cbd8700 1 --2- 172.21.15.104:0/3857738172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9280d07e0 0x7ff9281669b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff920009dc0 tx=0x7ff92000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89da961ab4014c1b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:11.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff926ffd700 1 -- 172.21.15.104:0/3857738172 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92000e070 con 0x7ff9280d07e0 2023-07-14T11:53:11.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff926ffd700 1 -- 172.21.15.104:0/3857738172 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92000bb20 con 0x7ff9280d07e0 2023-07-14T11:53:11.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.793+0000 7ff926ffd700 1 -- 172.21.15.104:0/3857738172 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92000be90 con 0x7ff9280d07e0 2023-07-14T11:53:11.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.797+0000 7ff92d3d9700 1 -- 172.21.15.104:0/3857738172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9280d07e0 msgr2=0x7ff9281669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.797+0000 7ff92d3d9700 1 --2- 172.21.15.104:0/3857738172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9280d07e0 0x7ff9281669b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff920009dc0 tx=0x7ff92000b330 comp rx=0 tx=0).stop 2023-07-14T11:53:11.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.797+0000 7ff92d3d9700 1 -- 172.21.15.104:0/3857738172 shutdown_connections 2023-07-14T11:53:11.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.797+0000 7ff92d3d9700 1 --2- 172.21.15.104:0/3857738172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9280d07e0 0x7ff9281669b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.797+0000 7ff92d3d9700 1 -- 172.21.15.104:0/3857738172 >> 172.21.15.104:0/3857738172 conn(0x7ff928060c40 msgr2=0x7ff92809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:11.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.797+0000 7ff92d3d9700 1 -- 172.21.15.104:0/3857738172 shutdown_connections 2023-07-14T11:53:11.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.797+0000 7ff92d3d9700 1 -- 172.21.15.104:0/3857738172 wait complete. 2023-07-14T11:53:11.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.797+0000 7ff92d3d9700 1 Processor -- start 2023-07-14T11:53:11.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff92d3d9700 1 -- start start 2023-07-14T11:53:11.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff92d3d9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9280b0ee0 0x7ff9281f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:11.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff92d3d9700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9280cd260 0x7ff9281f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:11.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff92d3d9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9280d07e0 0x7ff9280c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:11.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff92d3d9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9281f2320 con 0x7ff9280b0ee0 2023-07-14T11:53:11.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff92d3d9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9281f4ab0 con 0x7ff9280d07e0 2023-07-14T11:53:11.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff92d3d9700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9281f4d90 con 0x7ff9280cd260 2023-07-14T11:53:11.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff927fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9280b0ee0 0x7ff9281f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:11.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff927fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9280b0ee0 0x7ff9281f1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36662/0 (socket says 172.21.15.104:36662) 2023-07-14T11:53:11.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff927fff700 1 -- 172.21.15.104:0/2906180886 learned_addr learned my addr 172.21.15.104:0/2906180886 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:11.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff92cbd8700 1 --2- 172.21.15.104:0/2906180886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9280d07e0 0x7ff9280c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:11.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff9277fe700 1 --2- 172.21.15.104:0/2906180886 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9280cd260 0x7ff9281f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:11.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff927fff700 1 -- 172.21.15.104:0/2906180886 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9280cd260 msgr2=0x7ff9281f0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff927fff700 1 --2- 172.21.15.104:0/2906180886 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9280cd260 0x7ff9281f0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff927fff700 1 -- 172.21.15.104:0/2906180886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9280d07e0 msgr2=0x7ff9280c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff927fff700 1 --2- 172.21.15.104:0/2906180886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9280d07e0 0x7ff9280c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff927fff700 1 -- 172.21.15.104:0/2906180886 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff920009a70 con 0x7ff9280b0ee0 2023-07-14T11:53:11.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff927fff700 1 --2- 172.21.15.104:0/2906180886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9280b0ee0 0x7ff9281f1de0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff91800c4d0 tx=0x7ff91800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:11.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff924ff9700 1 -- 172.21.15.104:0/2906180886 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918005d90 con 0x7ff9280b0ee0 2023-07-14T11:53:11.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff924ff9700 1 -- 172.21.15.104:0/2906180886 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9180070d0 con 0x7ff9280b0ee0 2023-07-14T11:53:11.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff92d3d9700 1 -- 172.21.15.104:0/2906180886 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9281f5070 con 0x7ff9280b0ee0 2023-07-14T11:53:11.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff924ff9700 1 -- 172.21.15.104:0/2906180886 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918007480 con 0x7ff9280b0ee0 2023-07-14T11:53:11.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.801+0000 7ff92d3d9700 1 -- 172.21.15.104:0/2906180886 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9281f5630 con 0x7ff9280b0ee0 2023-07-14T11:53:11.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.805+0000 7ff92d3d9700 1 -- 172.21.15.104:0/2906180886 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92804e500 con 0x7ff9280b0ee0 2023-07-14T11:53:11.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.805+0000 7ff924ff9700 1 -- 172.21.15.104:0/2906180886 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff9180075e0 con 0x7ff9280b0ee0 2023-07-14T11:53:11.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.809+0000 7ff924ff9700 1 --2- 172.21.15.104:0/2906180886 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9140662f0 0x7ff9140687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:11.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.809+0000 7ff924ff9700 1 -- 172.21.15.104:0/2906180886 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff918082d60 con 0x7ff9280b0ee0 2023-07-14T11:53:11.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.809+0000 7ff9277fe700 1 --2- 172.21.15.104:0/2906180886 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9140662f0 0x7ff9140687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:11.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.809+0000 7ff9277fe700 1 --2- 172.21.15.104:0/2906180886 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9140662f0 0x7ff9140687a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff92815c4c0 tx=0x7ff91c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:11.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.813+0000 7ff924ff9700 1 -- 172.21.15.104:0/2906180886 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff9180505a0 con 0x7ff9280b0ee0 2023-07-14T11:53:11.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.981+0000 7ff92d3d9700 1 -- 172.21.15.104:0/2906180886 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff9280b18d0 con 0x7ff9280b0ee0 2023-07-14T11:53:11.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.985+0000 7ff924ff9700 1 -- 172.21.15.104:0/2906180886 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff918053bc0 con 0x7ff9280b0ee0 2023-07-14T11:53:11.971 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:53:11.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.989+0000 7ff92d3d9700 1 -- 172.21.15.104:0/2906180886 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9140662f0 msgr2=0x7ff9140687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.989+0000 7ff92d3d9700 1 --2- 172.21.15.104:0/2906180886 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9140662f0 0x7ff9140687a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff92815c4c0 tx=0x7ff91c009040 comp rx=0 tx=0).stop 2023-07-14T11:53:11.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.989+0000 7ff92d3d9700 1 -- 172.21.15.104:0/2906180886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9280b0ee0 msgr2=0x7ff9281f1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:11.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.989+0000 7ff92d3d9700 1 --2- 172.21.15.104:0/2906180886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9280b0ee0 0x7ff9281f1de0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff91800c4d0 tx=0x7ff91800c7e0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.989+0000 7ff92d3d9700 1 -- 172.21.15.104:0/2906180886 shutdown_connections 2023-07-14T11:53:11.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.989+0000 7ff92d3d9700 1 --2- 172.21.15.104:0/2906180886 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9140662f0 0x7ff9140687a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.989+0000 7ff92d3d9700 1 --2- 172.21.15.104:0/2906180886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9280d07e0 0x7ff9280c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.989+0000 7ff92d3d9700 1 --2- 172.21.15.104:0/2906180886 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9280cd260 0x7ff9281f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.989+0000 7ff92d3d9700 1 --2- 172.21.15.104:0/2906180886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9280b0ee0 0x7ff9281f1de0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:11.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.989+0000 7ff92d3d9700 1 -- 172.21.15.104:0/2906180886 >> 172.21.15.104:0/2906180886 conn(0x7ff928060c40 msgr2=0x7ff9280b55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:11.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.989+0000 7ff92d3d9700 1 -- 172.21.15.104:0/2906180886 shutdown_connections 2023-07-14T11:53:11.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:11.989+0000 7ff92d3d9700 1 -- 172.21.15.104:0/2906180886 wait complete. 2023-07-14T11:53:11.996 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-14T11:53:12.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.177+0000 7f479759e700 1 Processor -- start 2023-07-14T11:53:12.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.177+0000 7f479759e700 1 -- start start 2023-07-14T11:53:12.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.177+0000 7f479759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f479806dc20 0x7f47980cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:12.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.177+0000 7f479759e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f47980c9820 0x7f47980ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:12.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.177+0000 7f479759e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4798159d60 con 0x7f4798158fe0 2023-07-14T11:53:12.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.177+0000 7f479759e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47980cd250 con 0x7f47980c9820 2023-07-14T11:53:12.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.177+0000 7f479759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47980c8a60 con 0x7f479806dc20 2023-07-14T11:53:12.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.177+0000 7f478ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f47980c9820 0x7f47980ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:12.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.177+0000 7f478ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f47980c9820 0x7f47980ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36664/0 (socket says 172.21.15.104:36664) 2023-07-14T11:53:12.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.177+0000 7f478ffff700 1 -- 172.21.15.104:0/4157152731 learned_addr learned my addr 172.21.15.104:0/4157152731 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:12.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.177+0000 7f4796d9d700 1 --1- 172.21.15.104:0/4157152731 >> v1:172.21.15.104:6789/0 conn(0x7f4798158fe0 0x7f4798159480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:38166/0 (socket says 172.21.15.104:38166) 2023-07-14T11:53:12.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.177+0000 7f479659c700 1 --2- 172.21.15.104:0/4157152731 >> v2:172.21.15.121:3300/0 conn(0x7f479806dc20 0x7f47980cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:12.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f478ffff700 1 -- 172.21.15.104:0/4157152731 >> v2:172.21.15.121:3300/0 conn(0x7f479806dc20 msgr2=0x7f47980cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:12.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f478ffff700 1 --2- 172.21.15.104:0/4157152731 >> v2:172.21.15.121:3300/0 conn(0x7f479806dc20 0x7f47980cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f478ffff700 1 -- 172.21.15.104:0/4157152731 >> v1:172.21.15.104:6789/0 conn(0x7f4798158fe0 legacy=0x7f4798159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:12.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f478ffff700 1 -- 172.21.15.104:0/4157152731 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47980c8d40 con 0x7f47980c9820 2023-07-14T11:53:12.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f478ffff700 1 --2- 172.21.15.104:0/4157152731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47980c9820 0x7f47980ce410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f479806a5d0 tx=0x7f478800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bb1d36f20c4f1d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:12.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f4795d9b700 1 -- 172.21.15.104:0/4157152731 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478800dd30 con 0x7f47980c9820 2023-07-14T11:53:12.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f4795d9b700 1 -- 172.21.15.104:0/4157152731 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478800de90 con 0x7f47980c9820 2023-07-14T11:53:12.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f4795d9b700 1 -- 172.21.15.104:0/4157152731 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788010690 con 0x7f47980c9820 2023-07-14T11:53:12.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f479759e700 1 -- 172.21.15.104:0/4157152731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47980c9820 msgr2=0x7f47980ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:12.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f479759e700 1 --2- 172.21.15.104:0/4157152731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47980c9820 0x7f47980ce410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f479806a5d0 tx=0x7f478800be90 comp rx=0 tx=0).stop 2023-07-14T11:53:12.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f479759e700 1 -- 172.21.15.104:0/4157152731 shutdown_connections 2023-07-14T11:53:12.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f479759e700 1 --2- 172.21.15.104:0/4157152731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47980c9820 0x7f47980ce410 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f479759e700 1 --2- 172.21.15.104:0/4157152731 >> v2:172.21.15.121:3300/0 conn(0x7f479806dc20 0x7f47980cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f479759e700 1 -- 172.21.15.104:0/4157152731 >> 172.21.15.104:0/4157152731 conn(0x7f479806e960 msgr2=0x7f479806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:12.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f479759e700 1 -- 172.21.15.104:0/4157152731 shutdown_connections 2023-07-14T11:53:12.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.181+0000 7f479759e700 1 -- 172.21.15.104:0/4157152731 wait complete. 2023-07-14T11:53:12.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.185+0000 7f479759e700 1 Processor -- start 2023-07-14T11:53:12.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.185+0000 7f479759e700 1 -- start start 2023-07-14T11:53:12.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.185+0000 7f479759e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f479806dc20 0x7f479814e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:12.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.185+0000 7f479759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47980c9820 0x7f47980b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:12.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.185+0000 7f479759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4798158fe0 0x7f4798153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:12.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.185+0000 7f479759e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47980b22e0 con 0x7f479806dc20 2023-07-14T11:53:12.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.185+0000 7f479759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47980b25c0 con 0x7f4798158fe0 2023-07-14T11:53:12.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.185+0000 7f479759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47980b28a0 con 0x7f47980c9820 2023-07-14T11:53:12.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.185+0000 7f479659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f479806dc20 0x7f479814e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:12.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.185+0000 7f479659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f479806dc20 0x7f479814e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36680/0 (socket says 172.21.15.104:36680) 2023-07-14T11:53:12.176 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.185+0000 7f479659c700 1 -- 172.21.15.104:0/2978855023 learned_addr learned my addr 172.21.15.104:0/2978855023 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:12.176 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.189+0000 7f4796d9d700 1 --2- 172.21.15.104:0/2978855023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4798158fe0 0x7f4798153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:12.176 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.189+0000 7f479659c700 1 -- 172.21.15.104:0/2978855023 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47980c9820 msgr2=0x7f47980b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:12.176 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.189+0000 7f479659c700 1 --2- 172.21.15.104:0/2978855023 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47980c9820 0x7f47980b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.176 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.189+0000 7f479659c700 1 -- 172.21.15.104:0/2978855023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4798158fe0 msgr2=0x7f4798153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:12.176 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.189+0000 7f479659c700 1 --2- 172.21.15.104:0/2978855023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4798158fe0 0x7f4798153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.176 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.189+0000 7f479659c700 1 -- 172.21.15.104:0/2978855023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478800b810 con 0x7f479806dc20 2023-07-14T11:53:12.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.189+0000 7f479659c700 1 --2- 172.21.15.104:0/2978855023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f479806dc20 0x7f479814e920 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f478000e9d0 tx=0x7f478000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:12.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.189+0000 7f478effd700 1 -- 172.21.15.104:0/2978855023 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780009720 con 0x7f479806dc20 2023-07-14T11:53:12.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.189+0000 7f478effd700 1 -- 172.21.15.104:0/2978855023 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478000c850 con 0x7f479806dc20 2023-07-14T11:53:12.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.189+0000 7f479759e700 1 -- 172.21.15.104:0/2978855023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47980b2b80 con 0x7f479806dc20 2023-07-14T11:53:12.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.189+0000 7f478effd700 1 -- 172.21.15.104:0/2978855023 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478000cbc0 con 0x7f479806dc20 2023-07-14T11:53:12.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.189+0000 7f479759e700 1 -- 172.21.15.104:0/2978855023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479814ee60 con 0x7f479806dc20 2023-07-14T11:53:12.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.189+0000 7f479759e700 1 -- 172.21.15.104:0/2978855023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47980ae000 con 0x7f479806dc20 2023-07-14T11:53:12.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.189+0000 7f478effd700 1 -- 172.21.15.104:0/2978855023 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f478000cd20 con 0x7f479806dc20 2023-07-14T11:53:12.179 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.193+0000 7f478effd700 1 --2- 172.21.15.104:0/2978855023 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f47780662f0 0x7f47780687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:12.179 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.193+0000 7f478effd700 1 -- 172.21.15.104:0/2978855023 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4780083b90 con 0x7f479806dc20 2023-07-14T11:53:12.180 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.193+0000 7f478ffff700 1 --2- 172.21.15.104:0/2978855023 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f47780662f0 0x7f47780687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:12.180 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.193+0000 7f478ffff700 1 --2- 172.21.15.104:0/2978855023 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f47780662f0 0x7f47780687a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f47980c9c10 tx=0x7f4788012000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:12.186 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.197+0000 7f478effd700 1 -- 172.21.15.104:0/2978855023 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f47980ae000 con 0x7f479806dc20 2023-07-14T11:53:12.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.373+0000 7f479759e700 1 -- 172.21.15.104:0/2978855023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f47980ae000 con 0x7f479806dc20 2023-07-14T11:53:12.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.377+0000 7f478effd700 1 -- 172.21.15.104:0/2978855023 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) v1 ==== 84+0+126 (secure 0 0 0) 0x7f47980ae000 con 0x7f479806dc20 2023-07-14T11:53:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:12.363 INFO:teuthology.orchestra.run.smithi104.stdout:[ 2023-07-14T11:53:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:53:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "poolnum": 1, 2023-07-14T11:53:12.363 INFO:teuthology.orchestra.run.smithi104.stdout: "poolname": ".mgr" 2023-07-14T11:53:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:53:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:53:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: "poolnum": 2, 2023-07-14T11:53:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: "poolname": "rbd" 2023-07-14T11:53:12.364 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:53:12.364 INFO:teuthology.orchestra.run.smithi104.stdout:] 2023-07-14T11:53:12.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.377+0000 7f479759e700 1 -- 172.21.15.104:0/2978855023 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f47780662f0 msgr2=0x7f47780687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:12.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.377+0000 7f479759e700 1 --2- 172.21.15.104:0/2978855023 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f47780662f0 0x7f47780687a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f47980c9c10 tx=0x7f4788012000 comp rx=0 tx=0).stop 2023-07-14T11:53:12.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.377+0000 7f479759e700 1 -- 172.21.15.104:0/2978855023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f479806dc20 msgr2=0x7f479814e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:12.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.377+0000 7f479759e700 1 --2- 172.21.15.104:0/2978855023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f479806dc20 0x7f479814e920 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f478000e9d0 tx=0x7f478000ed90 comp rx=0 tx=0).stop 2023-07-14T11:53:12.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.377+0000 7f479759e700 1 -- 172.21.15.104:0/2978855023 shutdown_connections 2023-07-14T11:53:12.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.377+0000 7f479759e700 1 --2- 172.21.15.104:0/2978855023 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f47780662f0 0x7f47780687a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.377+0000 7f479759e700 1 --2- 172.21.15.104:0/2978855023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4798158fe0 0x7f4798153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.377+0000 7f479759e700 1 --2- 172.21.15.104:0/2978855023 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47980c9820 0x7f47980b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.377+0000 7f479759e700 1 --2- 172.21.15.104:0/2978855023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f479806dc20 0x7f479814e920 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.377+0000 7f479759e700 1 -- 172.21.15.104:0/2978855023 >> 172.21.15.104:0/2978855023 conn(0x7f479806e960 msgr2=0x7f47981535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:12.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.377+0000 7f479759e700 1 -- 172.21.15.104:0/2978855023 shutdown_connections 2023-07-14T11:53:12.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.377+0000 7f479759e700 1 -- 172.21.15.104:0/2978855023 wait complete. 2023-07-14T11:53:12.387 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-14T11:53:12.388 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-14T11:53:12.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.577+0000 7f94f2e27700 1 Processor -- start 2023-07-14T11:53:12.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.577+0000 7f94f2e27700 1 -- start start 2023-07-14T11:53:12.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.581+0000 7f94f2e27700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f94ec0af770 0x7f94ec0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:12.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.581+0000 7f94f2e27700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec0afe80 con 0x7f94ec0ccb20 2023-07-14T11:53:12.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.581+0000 7f94f2e27700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec15c360 con 0x7f94ec0b1320 2023-07-14T11:53:12.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.581+0000 7f94f2e27700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ec070740 con 0x7f94ec0af770 2023-07-14T11:53:12.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.581+0000 7f94f1e25700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f94ec0af770 0x7f94ec0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:12.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.581+0000 7f94f1624700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f94ec0ccb20 0x7f94ec0ca8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:38182/0 (socket says 172.21.15.104:38182) 2023-07-14T11:53:12.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.581+0000 7f94f1624700 1 -- 172.21.15.104:0/3533818565 learned_addr learned my addr 172.21.15.104:0/3533818565 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:12.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.585+0000 7f94f1e25700 1 -- 172.21.15.104:0/3533818565 >> v1:172.21.15.121:6789/0 conn(0x7f94ec0b1320 legacy=0x7f94ec0cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:12.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.585+0000 7f94f1e25700 1 -- 172.21.15.104:0/3533818565 >> v1:172.21.15.104:6789/0 conn(0x7f94ec0ccb20 legacy=0x7f94ec0ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:12.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.585+0000 7f94f1e25700 1 -- 172.21.15.104:0/3533818565 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec0cb710 con 0x7f94ec0af770 2023-07-14T11:53:12.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.585+0000 7f94f1e25700 1 --2- 172.21.15.104:0/3533818565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94ec0af770 0x7f94ec0cc310 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009880 tx=0x7f94e8009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba91315e3a3e6d61 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:12.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.585+0000 7f94f0e23700 1 -- 172.21.15.104:0/3533818565 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8011830 con 0x7f94ec0af770 2023-07-14T11:53:12.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.585+0000 7f94f0e23700 1 -- 172.21.15.104:0/3533818565 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e8011990 con 0x7f94ec0af770 2023-07-14T11:53:12.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.585+0000 7f94f0e23700 1 -- 172.21.15.104:0/3533818565 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8011d20 con 0x7f94ec0af770 2023-07-14T11:53:12.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.585+0000 7f94f2e27700 1 -- 172.21.15.104:0/3533818565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94ec0af770 msgr2=0x7f94ec0cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:12.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.585+0000 7f94f2e27700 1 --2- 172.21.15.104:0/3533818565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94ec0af770 0x7f94ec0cc310 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009880 tx=0x7f94e8009b90 comp rx=0 tx=0).stop 2023-07-14T11:53:12.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.585+0000 7f94f2e27700 1 -- 172.21.15.104:0/3533818565 shutdown_connections 2023-07-14T11:53:12.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.585+0000 7f94f2e27700 1 --2- 172.21.15.104:0/3533818565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94ec0af770 0x7f94ec0cc310 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.585+0000 7f94f2e27700 1 -- 172.21.15.104:0/3533818565 >> 172.21.15.104:0/3533818565 conn(0x7f94ec08a000 msgr2=0x7f94ec0d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:12.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.585+0000 7f94f2e27700 1 -- 172.21.15.104:0/3533818565 shutdown_connections 2023-07-14T11:53:12.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.585+0000 7f94f2e27700 1 -- 172.21.15.104:0/3533818565 wait complete. 2023-07-14T11:53:12.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2e27700 1 Processor -- start 2023-07-14T11:53:12.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2e27700 1 -- start start 2023-07-14T11:53:12.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2e27700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94ec0af770 0x7f94ec0c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:12.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2e27700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94ec0b1320 0x7f94ec0ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:12.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2e27700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94ec0ccb20 0x7f94ec0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:12.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2e27700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec0b42f0 con 0x7f94ec0ccb20 2023-07-14T11:53:12.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2e27700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec0b45d0 con 0x7f94ec0af770 2023-07-14T11:53:12.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2e27700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec0b48b0 con 0x7f94ec0b1320 2023-07-14T11:53:12.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2626700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94ec0ccb20 0x7f94ec0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:12.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2626700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94ec0ccb20 0x7f94ec0b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36688/0 (socket says 172.21.15.104:36688) 2023-07-14T11:53:12.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2626700 1 -- 172.21.15.104:0/731803148 learned_addr learned my addr 172.21.15.104:0/731803148 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:12.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f1624700 1 --2- 172.21.15.104:0/731803148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94ec0b1320 0x7f94ec0ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:12.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f1e25700 1 --2- 172.21.15.104:0/731803148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94ec0af770 0x7f94ec0c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:12.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2626700 1 -- 172.21.15.104:0/731803148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94ec0b1320 msgr2=0x7f94ec0ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:12.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2626700 1 --2- 172.21.15.104:0/731803148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94ec0b1320 0x7f94ec0ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2626700 1 -- 172.21.15.104:0/731803148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94ec0af770 msgr2=0x7f94ec0c47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:12.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2626700 1 --2- 172.21.15.104:0/731803148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94ec0af770 0x7f94ec0c47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.589+0000 7f94f2626700 1 -- 172.21.15.104:0/731803148 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e80094b0 con 0x7f94ec0ccb20 2023-07-14T11:53:12.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.593+0000 7f94f2626700 1 --2- 172.21.15.104:0/731803148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94ec0ccb20 0x7f94ec0b5390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f94e400cc50 tx=0x7f94e400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:12.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.593+0000 7f94e27fc700 1 -- 172.21.15.104:0/731803148 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4003c20 con 0x7f94ec0ccb20 2023-07-14T11:53:12.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.593+0000 7f94e27fc700 1 -- 172.21.15.104:0/731803148 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e4003d80 con 0x7f94ec0ccb20 2023-07-14T11:53:12.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.593+0000 7f94f2e27700 1 -- 172.21.15.104:0/731803148 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ec0b4b90 con 0x7f94ec0ccb20 2023-07-14T11:53:12.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.593+0000 7f94e27fc700 1 -- 172.21.15.104:0/731803148 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e400aab0 con 0x7f94ec0ccb20 2023-07-14T11:53:12.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.593+0000 7f94f2e27700 1 -- 172.21.15.104:0/731803148 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec0c4d20 con 0x7f94ec0ccb20 2023-07-14T11:53:12.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.593+0000 7f94f2e27700 1 -- 172.21.15.104:0/731803148 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ec0b50b0 con 0x7f94ec0ccb20 2023-07-14T11:53:12.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.597+0000 7f94e27fc700 1 -- 172.21.15.104:0/731803148 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f94e400ac10 con 0x7f94ec0ccb20 2023-07-14T11:53:12.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.597+0000 7f94e27fc700 1 --2- 172.21.15.104:0/731803148 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94d00662f0 0x7f94d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:12.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.597+0000 7f94e27fc700 1 -- 172.21.15.104:0/731803148 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f94e4083a40 con 0x7f94ec0ccb20 2023-07-14T11:53:12.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.597+0000 7f94f1e25700 1 --2- 172.21.15.104:0/731803148 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94d00662f0 0x7f94d00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:12.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.597+0000 7f94f1e25700 1 --2- 172.21.15.104:0/731803148 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94d00662f0 0x7f94d00687a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f94e8000c00 tx=0x7f94e8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:12.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.601+0000 7f94e27fc700 1 -- 172.21.15.104:0/731803148 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f94ec0b50b0 con 0x7f94ec0ccb20 2023-07-14T11:53:12.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:12.777+0000 7f94f2e27700 1 -- 172.21.15.104:0/731803148 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f94ec0b50b0 con 0x7f94ec0ccb20 2023-07-14T11:53:13.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.545+0000 7f94e27fc700 1 -- 172.21.15.104:0/731803148 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v21) v1 ==== 138+0+0 (secure 0 0 0) 0x7f94ec0b50b0 con 0x7f94ec0ccb20 2023-07-14T11:53:13.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.549+0000 7f94f2e27700 1 -- 172.21.15.104:0/731803148 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94d00662f0 msgr2=0x7f94d00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:13.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.549+0000 7f94f2e27700 1 --2- 172.21.15.104:0/731803148 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94d00662f0 0x7f94d00687a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f94e8000c00 tx=0x7f94e8008030 comp rx=0 tx=0).stop 2023-07-14T11:53:13.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.549+0000 7f94f2e27700 1 -- 172.21.15.104:0/731803148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94ec0ccb20 msgr2=0x7f94ec0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:13.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.549+0000 7f94f2e27700 1 --2- 172.21.15.104:0/731803148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94ec0ccb20 0x7f94ec0b5390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f94e400cc50 tx=0x7f94e400cf60 comp rx=0 tx=0).stop 2023-07-14T11:53:13.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.553+0000 7f94f2e27700 1 -- 172.21.15.104:0/731803148 shutdown_connections 2023-07-14T11:53:13.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.553+0000 7f94f2e27700 1 --2- 172.21.15.104:0/731803148 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94d00662f0 0x7f94d00687a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:13.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.553+0000 7f94f2e27700 1 --2- 172.21.15.104:0/731803148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94ec0ccb20 0x7f94ec0b5390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:13.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.553+0000 7f94f2e27700 1 --2- 172.21.15.104:0/731803148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94ec0b1320 0x7f94ec0ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:13.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.553+0000 7f94f2e27700 1 --2- 172.21.15.104:0/731803148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94ec0af770 0x7f94ec0c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:13.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.553+0000 7f94f2e27700 1 -- 172.21.15.104:0/731803148 >> 172.21.15.104:0/731803148 conn(0x7f94ec08a000 msgr2=0x7f94ec0cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:13.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.553+0000 7f94f2e27700 1 -- 172.21.15.104:0/731803148 shutdown_connections 2023-07-14T11:53:13.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.553+0000 7f94f2e27700 1 -- 172.21.15.104:0/731803148 wait complete. 2023-07-14T11:53:13.544 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'cephfs_metadata' created 2023-07-14T11:53:13.563 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-07-14T11:53:13.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.761+0000 7f12d3fff700 1 Processor -- start 2023-07-14T11:53:13.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.761+0000 7f12d3fff700 1 -- start start 2023-07-14T11:53:13.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.761+0000 7f12d3fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f12d41459c0 0x7f12d40ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:13.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.761+0000 7f12d3fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f12d40ccbd0 0x7f12d40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:13.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.761+0000 7f12d3fff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d4145d90 con 0x7f12d40d0150 2023-07-14T11:53:13.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.761+0000 7f12d3fff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d40cd650 con 0x7f12d40ccbd0 2023-07-14T11:53:13.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.761+0000 7f12d3fff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d40cd930 con 0x7f12d41459c0 2023-07-14T11:53:13.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.761+0000 7f12d27fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f12d40ccbd0 0x7f12d40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:13.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.761+0000 7f12d27fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f12d40ccbd0 0x7f12d40cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36690/0 (socket says 172.21.15.104:36690) 2023-07-14T11:53:13.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.761+0000 7f12d27fc700 1 -- 172.21.15.104:0/815116879 learned_addr learned my addr 172.21.15.104:0/815116879 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:13.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.761+0000 7f12d2ffd700 1 --2- 172.21.15.104:0/815116879 >> v2:172.21.15.121:3300/0 conn(0x7f12d41459c0 0x7f12d40ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:13.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.761+0000 7f12d1ffb700 1 -- 172.21.15.104:0/815116879 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541374446 0 0) 0x7f12d4145d90 con 0x7f12d40d0150 2023-07-14T11:53:13.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.761+0000 7f12d27fc700 1 -- 172.21.15.104:0/815116879 >> v2:172.21.15.121:3300/0 conn(0x7f12d41459c0 msgr2=0x7f12d40ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:13.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.761+0000 7f12d27fc700 1 --2- 172.21.15.104:0/815116879 >> v2:172.21.15.121:3300/0 conn(0x7f12d41459c0 0x7f12d40ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:13.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.765+0000 7f12d27fc700 1 -- 172.21.15.104:0/815116879 >> v1:172.21.15.104:6789/0 conn(0x7f12d40d0150 legacy=0x7f12d40b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:13.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.765+0000 7f12d27fc700 1 -- 172.21.15.104:0/815116879 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d40cdc10 con 0x7f12d40ccbd0 2023-07-14T11:53:13.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.765+0000 7f12d27fc700 1 --2- 172.21.15.104:0/815116879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12d40ccbd0 0x7f12d40cd050 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009910 tx=0x7f12c8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301f54bdf16c6d95 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:13.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.765+0000 7f12d1ffb700 1 -- 172.21.15.104:0/815116879 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8011830 con 0x7f12d40ccbd0 2023-07-14T11:53:13.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.765+0000 7f12d1ffb700 1 -- 172.21.15.104:0/815116879 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c8011990 con 0x7f12d40ccbd0 2023-07-14T11:53:13.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.765+0000 7f12d1ffb700 1 -- 172.21.15.104:0/815116879 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8011d40 con 0x7f12d40ccbd0 2023-07-14T11:53:13.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.765+0000 7f12d3fff700 1 -- 172.21.15.104:0/815116879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12d40ccbd0 msgr2=0x7f12d40cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:13.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.765+0000 7f12d3fff700 1 --2- 172.21.15.104:0/815116879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12d40ccbd0 0x7f12d40cd050 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009910 tx=0x7f12c8009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:13.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.765+0000 7f12d3fff700 1 -- 172.21.15.104:0/815116879 shutdown_connections 2023-07-14T11:53:13.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.765+0000 7f12d3fff700 1 --2- 172.21.15.104:0/815116879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12d40ccbd0 0x7f12d40cd050 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:13.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.765+0000 7f12d3fff700 1 --2- 172.21.15.104:0/815116879 >> v2:172.21.15.121:3300/0 conn(0x7f12d41459c0 0x7f12d40ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:13.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.765+0000 7f12d3fff700 1 -- 172.21.15.104:0/815116879 >> 172.21.15.104:0/815116879 conn(0x7f12d4060c40 msgr2=0x7f12d409b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:13.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.765+0000 7f12d3fff700 1 -- 172.21.15.104:0/815116879 shutdown_connections 2023-07-14T11:53:13.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.765+0000 7f12d3fff700 1 -- 172.21.15.104:0/815116879 wait complete. 2023-07-14T11:53:13.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.769+0000 7f12d3fff700 1 Processor -- start 2023-07-14T11:53:13.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.769+0000 7f12d3fff700 1 -- start start 2023-07-14T11:53:13.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.769+0000 7f12d3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12d40ccbd0 0x7f12d40c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:13.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.769+0000 7f12d3fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12d40d0150 0x7f12d40c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:13.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.769+0000 7f12d3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12d41459c0 0x7f12d40c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:13.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.769+0000 7f12d3fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40c8000 con 0x7f12d40ccbd0 2023-07-14T11:53:13.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.769+0000 7f12d3fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40c7a20 con 0x7f12d41459c0 2023-07-14T11:53:13.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.769+0000 7f12d3fff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40c5820 con 0x7f12d40d0150 2023-07-14T11:53:13.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.769+0000 7f12d2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12d40ccbd0 0x7f12d40c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:13.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.769+0000 7f12d2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12d40ccbd0 0x7f12d40c52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36696/0 (socket says 172.21.15.104:36696) 2023-07-14T11:53:13.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.769+0000 7f12d2ffd700 1 -- 172.21.15.104:0/3065214426 learned_addr learned my addr 172.21.15.104:0/3065214426 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:13.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.769+0000 7f12d27fc700 1 --2- 172.21.15.104:0/3065214426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12d40d0150 0x7f12d40c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:13.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.769+0000 7f12d37fe700 1 --2- 172.21.15.104:0/3065214426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12d41459c0 0x7f12d40c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:13.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.773+0000 7f12d2ffd700 1 -- 172.21.15.104:0/3065214426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12d40d0150 msgr2=0x7f12d40c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:13.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.773+0000 7f12d2ffd700 1 --2- 172.21.15.104:0/3065214426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12d40d0150 0x7f12d40c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:13.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.773+0000 7f12d2ffd700 1 -- 172.21.15.104:0/3065214426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12d41459c0 msgr2=0x7f12d40c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:13.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.773+0000 7f12d2ffd700 1 --2- 172.21.15.104:0/3065214426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12d41459c0 0x7f12d40c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:13.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.773+0000 7f12d2ffd700 1 -- 172.21.15.104:0/3065214426 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c8009570 con 0x7f12d40ccbd0 2023-07-14T11:53:13.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.773+0000 7f12d27fc700 1 --2- 172.21.15.104:0/3065214426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12d40d0150 0x7f12d40c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:13.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.773+0000 7f12d2ffd700 1 --2- 172.21.15.104:0/3065214426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12d40ccbd0 0x7f12d40c52e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f12c400e9b0 tx=0x7f12c400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:13.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.773+0000 7f12bb7fe700 1 -- 172.21.15.104:0/3065214426 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4009670 con 0x7f12d40ccbd0 2023-07-14T11:53:13.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.773+0000 7f12bb7fe700 1 -- 172.21.15.104:0/3065214426 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c400c850 con 0x7f12d40ccbd0 2023-07-14T11:53:13.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.773+0000 7f12d3fff700 1 -- 172.21.15.104:0/3065214426 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d40c6af0 con 0x7f12d40ccbd0 2023-07-14T11:53:13.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.773+0000 7f12bb7fe700 1 -- 172.21.15.104:0/3065214426 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c400cbc0 con 0x7f12d40ccbd0 2023-07-14T11:53:13.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.773+0000 7f12d3fff700 1 -- 172.21.15.104:0/3065214426 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d415acf0 con 0x7f12d40ccbd0 2023-07-14T11:53:13.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.773+0000 7f12d3fff700 1 -- 172.21.15.104:0/3065214426 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d40cded0 con 0x7f12d40ccbd0 2023-07-14T11:53:13.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.777+0000 7f12bb7fe700 1 -- 172.21.15.104:0/3065214426 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f12c400cd20 con 0x7f12d40ccbd0 2023-07-14T11:53:13.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.781+0000 7f12bb7fe700 1 --2- 172.21.15.104:0/3065214426 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12c00662f0 0x7f12c00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:13.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.781+0000 7f12bb7fe700 1 -- 172.21.15.104:0/3065214426 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f12c40842b0 con 0x7f12d40ccbd0 2023-07-14T11:53:13.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.781+0000 7f12d27fc700 1 --2- 172.21.15.104:0/3065214426 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12c00662f0 0x7f12c00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:13.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.781+0000 7f12d27fc700 1 --2- 172.21.15.104:0/3065214426 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12c00662f0 0x7f12c00687a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f12d4070ff0 tx=0x7f12c8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:13.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.785+0000 7f12bb7fe700 1 -- 172.21.15.104:0/3065214426 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f12c40518f0 con 0x7f12d40ccbd0 2023-07-14T11:53:13.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:13.961+0000 7f12d3fff700 1 -- 172.21.15.104:0/3065214426 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f12d40cded0 con 0x7f12d40ccbd0 2023-07-14T11:53:14.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.561+0000 7f12bb7fe700 1 -- 172.21.15.104:0/3065214426 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v22) v1 ==== 171+0+0 (secure 0 0 0) 0x7f12d40cded0 con 0x7f12d40ccbd0 2023-07-14T11:53:14.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.585+0000 7f12d3fff700 1 -- 172.21.15.104:0/3065214426 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12c00662f0 msgr2=0x7f12c00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:14.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.585+0000 7f12d3fff700 1 --2- 172.21.15.104:0/3065214426 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12c00662f0 0x7f12c00687a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f12d4070ff0 tx=0x7f12c8008030 comp rx=0 tx=0).stop 2023-07-14T11:53:14.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.585+0000 7f12d3fff700 1 -- 172.21.15.104:0/3065214426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12d40ccbd0 msgr2=0x7f12d40c52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:14.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.585+0000 7f12d3fff700 1 --2- 172.21.15.104:0/3065214426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12d40ccbd0 0x7f12d40c52e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f12c400e9b0 tx=0x7f12c400ed70 comp rx=0 tx=0).stop 2023-07-14T11:53:14.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.589+0000 7f12d3fff700 1 -- 172.21.15.104:0/3065214426 shutdown_connections 2023-07-14T11:53:14.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.589+0000 7f12d3fff700 1 --2- 172.21.15.104:0/3065214426 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12c00662f0 0x7f12c00687a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:14.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.589+0000 7f12d3fff700 1 --2- 172.21.15.104:0/3065214426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12d41459c0 0x7f12d40c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:14.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.589+0000 7f12d3fff700 1 --2- 172.21.15.104:0/3065214426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12d40d0150 0x7f12d40c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:14.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.589+0000 7f12d3fff700 1 --2- 172.21.15.104:0/3065214426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12d40ccbd0 0x7f12d40c52e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:14.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.589+0000 7f12d3fff700 1 -- 172.21.15.104:0/3065214426 >> 172.21.15.104:0/3065214426 conn(0x7f12d4060c40 msgr2=0x7f12d4071540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:14.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.589+0000 7f12d3fff700 1 -- 172.21.15.104:0/3065214426 shutdown_connections 2023-07-14T11:53:14.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.589+0000 7f12d3fff700 1 -- 172.21.15.104:0/3065214426 wait complete. 2023-07-14T11:53:14.579 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'cephfs_data' created 2023-07-14T11:53:14.597 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-14T11:53:14.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.793+0000 7f6fe88bb700 1 Processor -- start 2023-07-14T11:53:14.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.793+0000 7f6fe88bb700 1 -- start start 2023-07-14T11:53:14.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe88bb700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fe40b0ee0 0x7f6fe40ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:14.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe88bb700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe40cd4e0 con 0x7f6fe40cd090 2023-07-14T11:53:14.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe88bb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe40d1a60 con 0x7f6fe40d07e0 2023-07-14T11:53:14.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe88bb700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40d0c60 con 0x7f6fe40b0ee0 2023-07-14T11:53:14.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe259c700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f6fe40cd090 0x7f6fe40d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:38194/0 (socket says 172.21.15.104:38194) 2023-07-14T11:53:14.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe259c700 1 -- 172.21.15.104:0/299288173 learned_addr learned my addr 172.21.15.104:0/299288173 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:14.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe2d9d700 1 --2- 172.21.15.104:0/299288173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fe40b0ee0 0x7f6fe40ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:14.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe1d9b700 1 -- 172.21.15.104:0/299288173 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117502876 0 0) 0x7f6fd8001110 con 0x7f6fe40cd090 2023-07-14T11:53:14.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe1d9b700 1 -- 172.21.15.104:0/299288173 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fcc0035d0 con 0x7f6fe40cd090 2023-07-14T11:53:14.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe1d9b700 1 -- 172.21.15.104:0/299288173 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443659509 0 0) 0x7f6fe40d1a60 con 0x7f6fe40d07e0 2023-07-14T11:53:14.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe1d9b700 1 -- 172.21.15.104:0/299288173 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fd8001110 con 0x7f6fe40d07e0 2023-07-14T11:53:14.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe2d9d700 1 -- 172.21.15.104:0/299288173 >> v1:172.21.15.121:6789/0 conn(0x7f6fe40d07e0 legacy=0x7f6fe40af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:14.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe2d9d700 1 -- 172.21.15.104:0/299288173 >> v1:172.21.15.104:6789/0 conn(0x7f6fe40cd090 legacy=0x7f6fe40d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:14.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe2d9d700 1 -- 172.21.15.104:0/299288173 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe40d1ef0 con 0x7f6fe40b0ee0 2023-07-14T11:53:14.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe2d9d700 1 --2- 172.21.15.104:0/299288173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fe40b0ee0 0x7f6fe40ccb50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40098b0 tx=0x7f6fd4009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c4beb4be75b3cd4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:14.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.797+0000 7f6fe1d9b700 1 -- 172.21.15.104:0/299288173 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4011830 con 0x7f6fe40b0ee0 2023-07-14T11:53:14.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.801+0000 7f6fe1d9b700 1 -- 172.21.15.104:0/299288173 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd4011990 con 0x7f6fe40b0ee0 2023-07-14T11:53:14.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.801+0000 7f6fe1d9b700 1 -- 172.21.15.104:0/299288173 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4011d20 con 0x7f6fe40b0ee0 2023-07-14T11:53:14.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.801+0000 7f6fe88bb700 1 -- 172.21.15.104:0/299288173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fe40b0ee0 msgr2=0x7f6fe40ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:14.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.801+0000 7f6fe88bb700 1 --2- 172.21.15.104:0/299288173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fe40b0ee0 0x7f6fe40ccb50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40098b0 tx=0x7f6fd4009bc0 comp rx=0 tx=0).stop 2023-07-14T11:53:14.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.801+0000 7f6fe88bb700 1 -- 172.21.15.104:0/299288173 shutdown_connections 2023-07-14T11:53:14.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.801+0000 7f6fe88bb700 1 --2- 172.21.15.104:0/299288173 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fe40b0ee0 0x7f6fe40ccb50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:14.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.801+0000 7f6fe88bb700 1 -- 172.21.15.104:0/299288173 >> 172.21.15.104:0/299288173 conn(0x7f6fe4060c40 msgr2=0x7f6fe409b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:14.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.801+0000 7f6fe88bb700 1 -- 172.21.15.104:0/299288173 shutdown_connections 2023-07-14T11:53:14.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.801+0000 7f6fe88bb700 1 -- 172.21.15.104:0/299288173 wait complete. 2023-07-14T11:53:14.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.801+0000 7f6fe88bb700 1 Processor -- start 2023-07-14T11:53:14.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.801+0000 7f6fe88bb700 1 -- start start 2023-07-14T11:53:14.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe88bb700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fe40b0ee0 0x7f6fe414ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:14.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe88bb700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fe40cd090 0x7f6fe40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:14.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe88bb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fe40d07e0 0x7f6fe4070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:14.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe88bb700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40afa20 con 0x7f6fe40b0ee0 2023-07-14T11:53:14.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe88bb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40b3380 con 0x7f6fe40d07e0 2023-07-14T11:53:14.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe88bb700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40b2770 con 0x7f6fe40cd090 2023-07-14T11:53:14.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe2d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fe40b0ee0 0x7f6fe414ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:14.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe2d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fe40b0ee0 0x7f6fe414ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36710/0 (socket says 172.21.15.104:36710) 2023-07-14T11:53:14.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe2d9d700 1 -- 172.21.15.104:0/1740220542 learned_addr learned my addr 172.21.15.104:0/1740220542 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:14.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe359e700 1 --2- 172.21.15.104:0/1740220542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fe40d07e0 0x7f6fe4070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:14.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe259c700 1 --2- 172.21.15.104:0/1740220542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fe40cd090 0x7f6fe40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:14.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe2d9d700 1 -- 172.21.15.104:0/1740220542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fe40cd090 msgr2=0x7f6fe40706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:14.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe2d9d700 1 --2- 172.21.15.104:0/1740220542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fe40cd090 0x7f6fe40706b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:14.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe2d9d700 1 -- 172.21.15.104:0/1740220542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fe40d07e0 msgr2=0x7f6fe4070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:14.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe2d9d700 1 --2- 172.21.15.104:0/1740220542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fe40d07e0 0x7f6fe4070bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:14.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe2d9d700 1 -- 172.21.15.104:0/1740220542 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd40094b0 con 0x7f6fe40b0ee0 2023-07-14T11:53:14.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe359e700 1 --2- 172.21.15.104:0/1740220542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fe40d07e0 0x7f6fe4070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:14.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe2d9d700 1 --2- 172.21.15.104:0/1740220542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fe40b0ee0 0x7f6fe414ba30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4000c00 tx=0x7f6fd4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:14.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fcb7fe700 1 -- 172.21.15.104:0/1740220542 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd401a990 con 0x7f6fe40b0ee0 2023-07-14T11:53:14.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe88bb700 1 -- 172.21.15.104:0/1740220542 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe40b2a50 con 0x7f6fe40b0ee0 2023-07-14T11:53:14.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fcb7fe700 1 -- 172.21.15.104:0/1740220542 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd401f070 con 0x7f6fe40b0ee0 2023-07-14T11:53:14.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fcb7fe700 1 -- 172.21.15.104:0/1740220542 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd401ad10 con 0x7f6fe40b0ee0 2023-07-14T11:53:14.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.805+0000 7f6fe88bb700 1 -- 172.21.15.104:0/1740220542 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe40b3010 con 0x7f6fe40b0ee0 2023-07-14T11:53:14.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.809+0000 7f6fe88bb700 1 -- 172.21.15.104:0/1740220542 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe404e500 con 0x7f6fe40b0ee0 2023-07-14T11:53:14.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.809+0000 7f6fcb7fe700 1 -- 172.21.15.104:0/1740220542 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6fd402e450 con 0x7f6fe40b0ee0 2023-07-14T11:53:14.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.809+0000 7f6fcb7fe700 1 --2- 172.21.15.104:0/1740220542 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6fd00662f0 0x7f6fd00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:14.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.813+0000 7f6fcb7fe700 1 -- 172.21.15.104:0/1740220542 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f6fd4084e80 con 0x7f6fe40b0ee0 2023-07-14T11:53:14.799 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.813+0000 7f6fe259c700 1 --2- 172.21.15.104:0/1740220542 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6fd00662f0 0x7f6fd00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:14.799 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.813+0000 7f6fe259c700 1 --2- 172.21.15.104:0/1740220542 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6fd00662f0 0x7f6fd00687a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4071270 tx=0x7f6fd8007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:14.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:14.817+0000 7f6fcb7fe700 1 -- 172.21.15.104:0/1740220542 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f6fd4052240 con 0x7f6fe40b0ee0 2023-07-14T11:53:15.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.021+0000 7f6fe88bb700 1 -- 172.21.15.104:0/1740220542 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6fe40cccd0 con 0x7f6fe40b0ee0 2023-07-14T11:53:15.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.569+0000 7f6fcb7fe700 1 -- 172.21.15.104:0/1740220542 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6fd40210c0 con 0x7f6fe40b0ee0 2023-07-14T11:53:15.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.589+0000 7f6fe88bb700 1 -- 172.21.15.104:0/1740220542 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6fd00662f0 msgr2=0x7f6fd00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:15.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.589+0000 7f6fe88bb700 1 --2- 172.21.15.104:0/1740220542 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6fd00662f0 0x7f6fd00687a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4071270 tx=0x7f6fd8007de0 comp rx=0 tx=0).stop 2023-07-14T11:53:15.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.589+0000 7f6fe88bb700 1 -- 172.21.15.104:0/1740220542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fe40b0ee0 msgr2=0x7f6fe414ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:15.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.589+0000 7f6fe88bb700 1 --2- 172.21.15.104:0/1740220542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fe40b0ee0 0x7f6fe414ba30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4000c00 tx=0x7f6fd4004770 comp rx=0 tx=0).stop 2023-07-14T11:53:15.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.589+0000 7f6fe88bb700 1 -- 172.21.15.104:0/1740220542 shutdown_connections 2023-07-14T11:53:15.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.589+0000 7f6fe88bb700 1 --2- 172.21.15.104:0/1740220542 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6fd00662f0 0x7f6fd00687a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:15.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.589+0000 7f6fe88bb700 1 --2- 172.21.15.104:0/1740220542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fe40d07e0 0x7f6fe4070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:15.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.589+0000 7f6fe88bb700 1 --2- 172.21.15.104:0/1740220542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6fe40cd090 0x7f6fe40706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:15.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.589+0000 7f6fe88bb700 1 --2- 172.21.15.104:0/1740220542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fe40b0ee0 0x7f6fe414ba30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:15.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.589+0000 7f6fe88bb700 1 -- 172.21.15.104:0/1740220542 >> 172.21.15.104:0/1740220542 conn(0x7f6fe4060c40 msgr2=0x7f6fe4160110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:15.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.589+0000 7f6fe88bb700 1 -- 172.21.15.104:0/1740220542 shutdown_connections 2023-07-14T11:53:15.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.593+0000 7f6fe88bb700 1 -- 172.21.15.104:0/1740220542 wait complete. 2023-07-14T11:53:15.582 INFO:teuthology.orchestra.run.smithi104.stderr:new fs with metadata pool 3 and data pool 4 2023-07-14T11:53:15.608 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-07-14T11:53:15.608 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-07-14T11:53:15.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.797+0000 7ffad5dd5700 1 Processor -- start 2023-07-14T11:53:15.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.797+0000 7ffad5dd5700 1 -- start start 2023-07-14T11:53:15.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffad5dd5700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffad00d02d0 0x7ffad00d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:15.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffad5dd5700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ffad00b40f0 0x7ffad00cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:15.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffad5dd5700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad00d0be0 con 0x7ffad00cd830 2023-07-14T11:53:15.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffad5dd5700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad0071bb0 con 0x7ffad00b40f0 2023-07-14T11:53:15.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffad5dd5700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad0071310 con 0x7ffad00d02d0 2023-07-14T11:53:15.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffaceffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ffad00b40f0 0x7ffad00cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:15.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffaceffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ffad00b40f0 0x7ffad00cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36726/0 (socket says 172.21.15.104:36726) 2023-07-14T11:53:15.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffaceffd700 1 -- 172.21.15.104:0/3441330025 learned_addr learned my addr 172.21.15.104:0/3441330025 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:15.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffacf7fe700 1 --2- 172.21.15.104:0/3441330025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffad00d02d0 0x7ffad00d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:15.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffaceffd700 1 -- 172.21.15.104:0/3441330025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffad00d02d0 msgr2=0x7ffad00d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:15.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffaceffd700 1 --2- 172.21.15.104:0/3441330025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffad00d02d0 0x7ffad00d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:15.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffaceffd700 1 -- 172.21.15.104:0/3441330025 >> v1:172.21.15.104:6789/0 conn(0x7ffad00cd830 legacy=0x7ffad00cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:15.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffaceffd700 1 -- 172.21.15.104:0/3441330025 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad00715f0 con 0x7ffad00b40f0 2023-07-14T11:53:15.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffaceffd700 1 --2- 172.21.15.104:0/3441330025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffad00b40f0 0x7ffad00cd2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009910 tx=0x7ffac4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261e4e2886c1fb9b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:15.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7fface7fc700 1 -- 172.21.15.104:0/3441330025 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac400ba60 con 0x7ffad00b40f0 2023-07-14T11:53:15.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7fface7fc700 1 -- 172.21.15.104:0/3441330025 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac400bbc0 con 0x7ffad00b40f0 2023-07-14T11:53:15.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7fface7fc700 1 -- 172.21.15.104:0/3441330025 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac4009e60 con 0x7ffad00b40f0 2023-07-14T11:53:15.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffad5dd5700 1 -- 172.21.15.104:0/3441330025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffad00b40f0 msgr2=0x7ffad00cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:15.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffad5dd5700 1 --2- 172.21.15.104:0/3441330025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffad00b40f0 0x7ffad00cd2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009910 tx=0x7ffac4009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:15.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffad5dd5700 1 -- 172.21.15.104:0/3441330025 shutdown_connections 2023-07-14T11:53:15.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffad5dd5700 1 --2- 172.21.15.104:0/3441330025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffad00b40f0 0x7ffad00cd2f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:15.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffad5dd5700 1 --2- 172.21.15.104:0/3441330025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffad00d02d0 0x7ffad00d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:15.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffad5dd5700 1 -- 172.21.15.104:0/3441330025 >> 172.21.15.104:0/3441330025 conn(0x7ffad009b9e0 msgr2=0x7ffad00d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:15.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffad5dd5700 1 -- 172.21.15.104:0/3441330025 shutdown_connections 2023-07-14T11:53:15.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.801+0000 7ffad5dd5700 1 -- 172.21.15.104:0/3441330025 wait complete. 2023-07-14T11:53:15.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.805+0000 7ffad5dd5700 1 Processor -- start 2023-07-14T11:53:15.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.805+0000 7ffad5dd5700 1 -- start start 2023-07-14T11:53:15.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.805+0000 7ffad5dd5700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffad00b40f0 0x7ffad0154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:15.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.805+0000 7ffad5dd5700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffad00cd830 0x7ffad00cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:15.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.805+0000 7ffad5dd5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffad00d02d0 0x7ffad00cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:15.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.805+0000 7ffad5dd5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad0155200 con 0x7ffad00d02d0 2023-07-14T11:53:15.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.805+0000 7ffad5dd5700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad01562b0 con 0x7ffad00cd830 2023-07-14T11:53:15.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.805+0000 7ffad5dd5700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad0156590 con 0x7ffad00b40f0 2023-07-14T11:53:15.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.805+0000 7ffacffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffad00d02d0 0x7ffad00cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:15.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.805+0000 7ffaceffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffad00cd830 0x7ffad00cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:15.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.805+0000 7ffacffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffad00d02d0 0x7ffad00cc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36736/0 (socket says 172.21.15.104:36736) 2023-07-14T11:53:15.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.805+0000 7ffacffff700 1 -- 172.21.15.104:0/1591483467 learned_addr learned my addr 172.21.15.104:0/1591483467 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:15.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.805+0000 7ffacf7fe700 1 --2- 172.21.15.104:0/1591483467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffad00b40f0 0x7ffad0154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:15.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.809+0000 7ffacffff700 1 -- 172.21.15.104:0/1591483467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffad00b40f0 msgr2=0x7ffad0154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:15.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.809+0000 7ffacffff700 1 --2- 172.21.15.104:0/1591483467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffad00b40f0 0x7ffad0154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:15.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.809+0000 7ffacffff700 1 -- 172.21.15.104:0/1591483467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffad00cd830 msgr2=0x7ffad00cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:15.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.809+0000 7ffacffff700 1 --2- 172.21.15.104:0/1591483467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffad00cd830 0x7ffad00cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:15.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.809+0000 7ffacffff700 1 -- 172.21.15.104:0/1591483467 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac4009570 con 0x7ffad00d02d0 2023-07-14T11:53:15.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.809+0000 7ffacffff700 1 --2- 172.21.15.104:0/1591483467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffad00d02d0 0x7ffad00cc070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffac800cc50 tx=0x7ffac800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:15.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.809+0000 7ffab7fff700 1 -- 172.21.15.104:0/1591483467 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8003c20 con 0x7ffad00d02d0 2023-07-14T11:53:15.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.809+0000 7ffab7fff700 1 -- 172.21.15.104:0/1591483467 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac8003d80 con 0x7ffad00d02d0 2023-07-14T11:53:15.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.809+0000 7ffab7fff700 1 -- 172.21.15.104:0/1591483467 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac800aab0 con 0x7ffad00d02d0 2023-07-14T11:53:15.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.809+0000 7ffad5dd5700 1 -- 172.21.15.104:0/1591483467 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad0156870 con 0x7ffad00d02d0 2023-07-14T11:53:15.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.809+0000 7ffad5dd5700 1 -- 172.21.15.104:0/1591483467 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad0156e30 con 0x7ffad00d02d0 2023-07-14T11:53:15.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.809+0000 7ffad5dd5700 1 -- 172.21.15.104:0/1591483467 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad004dcb0 con 0x7ffad00d02d0 2023-07-14T11:53:15.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.813+0000 7ffab7fff700 1 -- 172.21.15.104:0/1591483467 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ffac800ac10 con 0x7ffad00d02d0 2023-07-14T11:53:15.799 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.813+0000 7ffab7fff700 1 --2- 172.21.15.104:0/1591483467 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffabc0662f0 0x7ffabc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:15.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.813+0000 7ffab7fff700 1 -- 172.21.15.104:0/1591483467 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffac8083c80 con 0x7ffad00d02d0 2023-07-14T11:53:15.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.813+0000 7ffacf7fe700 1 --2- 172.21.15.104:0/1591483467 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffabc0662f0 0x7ffabc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:15.801 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.813+0000 7ffacf7fe700 1 --2- 172.21.15.104:0/1591483467 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffabc0662f0 0x7ffabc0687a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009a00 tx=0x7ffac0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:15.806 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:15.821+0000 7ffab7fff700 1 -- 172.21.15.104:0/1591483467 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ffac80510f0 con 0x7ffad00d02d0 2023-07-14T11:53:15.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.001+0000 7ffad5dd5700 1 -- 172.21.15.104:0/1591483467 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7ffad0071030 con 0x7ffad00d02d0 2023-07-14T11:53:16.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.569+0000 7ffab7fff700 1 -- 172.21.15.104:0/1591483467 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v24) v1 ==== 157+0+0 (secure 0 0 0) 0x7ffac8054710 con 0x7ffad00d02d0 2023-07-14T11:53:16.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.569+0000 7ffad5dd5700 1 -- 172.21.15.104:0/1591483467 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffabc0662f0 msgr2=0x7ffabc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:16.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.569+0000 7ffad5dd5700 1 --2- 172.21.15.104:0/1591483467 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffabc0662f0 0x7ffabc0687a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009a00 tx=0x7ffac0008040 comp rx=0 tx=0).stop 2023-07-14T11:53:16.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.569+0000 7ffad5dd5700 1 -- 172.21.15.104:0/1591483467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffad00d02d0 msgr2=0x7ffad00cc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:16.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.569+0000 7ffad5dd5700 1 --2- 172.21.15.104:0/1591483467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffad00d02d0 0x7ffad00cc070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffac800cc50 tx=0x7ffac800cf60 comp rx=0 tx=0).stop 2023-07-14T11:53:16.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.569+0000 7ffad5dd5700 1 -- 172.21.15.104:0/1591483467 shutdown_connections 2023-07-14T11:53:16.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.569+0000 7ffad5dd5700 1 --2- 172.21.15.104:0/1591483467 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffabc0662f0 0x7ffabc0687a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:16.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.569+0000 7ffad5dd5700 1 --2- 172.21.15.104:0/1591483467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffad00d02d0 0x7ffad00cc070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:16.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.569+0000 7ffad5dd5700 1 --2- 172.21.15.104:0/1591483467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffad00cd830 0x7ffad00cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:16.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.569+0000 7ffad5dd5700 1 --2- 172.21.15.104:0/1591483467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffad00b40f0 0x7ffad0154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:16.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.569+0000 7ffad5dd5700 1 -- 172.21.15.104:0/1591483467 >> 172.21.15.104:0/1591483467 conn(0x7ffad009b9e0 msgr2=0x7ffad00d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:16.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.573+0000 7ffad5dd5700 1 -- 172.21.15.104:0/1591483467 shutdown_connections 2023-07-14T11:53:16.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.573+0000 7ffad5dd5700 1 -- 172.21.15.104:0/1591483467 wait complete. 2023-07-14T11:53:16.579 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-07-14T11:53:16.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.769+0000 7fbca23d5700 1 Processor -- start 2023-07-14T11:53:16.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.769+0000 7fbca23d5700 1 -- start start 2023-07-14T11:53:16.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.769+0000 7fbca23d5700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbc9c071430 0x7fbc9c166d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:16.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.769+0000 7fbca23d5700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc9c06fa70 con 0x7fbc9c0cd530 2023-07-14T11:53:16.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.769+0000 7fbca23d5700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc9c06c2b0 con 0x7fbc9c15bf20 2023-07-14T11:53:16.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.769+0000 7fbca23d5700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc9c15d300 con 0x7fbc9c071430 2023-07-14T11:53:16.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.769+0000 7fbca0972700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbc9c071430 0x7fbc9c166d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:16.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.769+0000 7fbca0972700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbc9c071430 0x7fbc9c166d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36746/0 (socket says 172.21.15.104:36746) 2023-07-14T11:53:16.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.769+0000 7fbca0972700 1 -- 172.21.15.104:0/2627638901 learned_addr learned my addr 172.21.15.104:0/2627638901 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:16.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.769+0000 7fbc9b7fe700 1 --1- 172.21.15.104:0/2627638901 >> v1:172.21.15.104:6789/0 conn(0x7fbc9c0cd530 0x7fbc9c070d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:38214/0 (socket says 172.21.15.104:38214) 2023-07-14T11:53:16.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.773+0000 7fbca0972700 1 -- 172.21.15.104:0/2627638901 >> v1:172.21.15.121:6789/0 conn(0x7fbc9c15bf20 legacy=0x7fbc9c0cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:16.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.773+0000 7fbca0972700 1 -- 172.21.15.104:0/2627638901 >> v1:172.21.15.104:6789/0 conn(0x7fbc9c0cd530 legacy=0x7fbc9c070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:16.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.773+0000 7fbca0972700 1 -- 172.21.15.104:0/2627638901 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c071900 con 0x7fbc9c071430 2023-07-14T11:53:16.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.773+0000 7fbca0972700 1 --2- 172.21.15.104:0/2627638901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc9c071430 0x7fbc9c166d10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009910 tx=0x7fbc94009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2d78eae805e89f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:16.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.773+0000 7fbc9affd700 1 -- 172.21.15.104:0/2627638901 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9400ba60 con 0x7fbc9c071430 2023-07-14T11:53:16.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.773+0000 7fbc9affd700 1 -- 172.21.15.104:0/2627638901 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9400bbc0 con 0x7fbc9c071430 2023-07-14T11:53:16.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.773+0000 7fbc9affd700 1 -- 172.21.15.104:0/2627638901 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94009e60 con 0x7fbc9c071430 2023-07-14T11:53:16.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.773+0000 7fbca23d5700 1 -- 172.21.15.104:0/2627638901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc9c071430 msgr2=0x7fbc9c166d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:16.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.773+0000 7fbca23d5700 1 --2- 172.21.15.104:0/2627638901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc9c071430 0x7fbc9c166d10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009910 tx=0x7fbc94009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:16.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.773+0000 7fbca23d5700 1 -- 172.21.15.104:0/2627638901 shutdown_connections 2023-07-14T11:53:16.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.773+0000 7fbca23d5700 1 --2- 172.21.15.104:0/2627638901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc9c071430 0x7fbc9c166d10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:16.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.773+0000 7fbca23d5700 1 -- 172.21.15.104:0/2627638901 >> 172.21.15.104:0/2627638901 conn(0x7fbc9c06a0c0 msgr2=0x7fbc9c06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:16.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.773+0000 7fbca23d5700 1 -- 172.21.15.104:0/2627638901 shutdown_connections 2023-07-14T11:53:16.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.773+0000 7fbca23d5700 1 -- 172.21.15.104:0/2627638901 wait complete. 2023-07-14T11:53:16.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca23d5700 1 Processor -- start 2023-07-14T11:53:16.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca23d5700 1 -- start start 2023-07-14T11:53:16.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca23d5700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc9c071430 0x7fbc9c1e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:16.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca23d5700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc9c0cd530 0x7fbc9c1e70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:16.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca23d5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc9c15bf20 0x7fbc9c152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:16.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca23d5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c1e6030 con 0x7fbc9c15bf20 2023-07-14T11:53:16.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca23d5700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c1e6310 con 0x7fbc9c0cd530 2023-07-14T11:53:16.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca23d5700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c1e65f0 con 0x7fbc9c071430 2023-07-14T11:53:16.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca0972700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc9c15bf20 0x7fbc9c152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:16.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca0972700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc9c15bf20 0x7fbc9c152a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36754/0 (socket says 172.21.15.104:36754) 2023-07-14T11:53:16.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca0972700 1 -- 172.21.15.104:0/1016077069 learned_addr learned my addr 172.21.15.104:0/1016077069 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:16.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbc9bfff700 1 --2- 172.21.15.104:0/1016077069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc9c071430 0x7fbc9c1e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:16.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbc9b7fe700 1 --2- 172.21.15.104:0/1016077069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc9c0cd530 0x7fbc9c1e70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:16.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca0972700 1 -- 172.21.15.104:0/1016077069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc9c071430 msgr2=0x7fbc9c1e57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:16.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca0972700 1 --2- 172.21.15.104:0/1016077069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc9c071430 0x7fbc9c1e57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:16.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca0972700 1 -- 172.21.15.104:0/1016077069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc9c0cd530 msgr2=0x7fbc9c1e70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:16.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca0972700 1 --2- 172.21.15.104:0/1016077069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc9c0cd530 0x7fbc9c1e70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:16.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbca0972700 1 -- 172.21.15.104:0/1016077069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc94009570 con 0x7fbc9c15bf20 2023-07-14T11:53:16.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.777+0000 7fbc9b7fe700 1 --2- 172.21.15.104:0/1016077069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc9c0cd530 0x7fbc9c1e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:16.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.781+0000 7fbca0972700 1 --2- 172.21.15.104:0/1016077069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc9c15bf20 0x7fbc9c152a60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009c60 tx=0x7fbc94004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:16.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.781+0000 7fbc98ff9700 1 -- 172.21.15.104:0/1016077069 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc940036c0 con 0x7fbc9c15bf20 2023-07-14T11:53:16.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.781+0000 7fbc98ff9700 1 -- 172.21.15.104:0/1016077069 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9401e070 con 0x7fbc9c15bf20 2023-07-14T11:53:16.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.781+0000 7fbca23d5700 1 -- 172.21.15.104:0/1016077069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9c1e68d0 con 0x7fbc9c15bf20 2023-07-14T11:53:16.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.781+0000 7fbc98ff9700 1 -- 172.21.15.104:0/1016077069 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94003a80 con 0x7fbc9c15bf20 2023-07-14T11:53:16.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.781+0000 7fbca23d5700 1 -- 172.21.15.104:0/1016077069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9c1e5d20 con 0x7fbc9c15bf20 2023-07-14T11:53:16.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.781+0000 7fbca23d5700 1 -- 172.21.15.104:0/1016077069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc9c1e6df0 con 0x7fbc9c15bf20 2023-07-14T11:53:16.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.781+0000 7fbc98ff9700 1 -- 172.21.15.104:0/1016077069 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fbc94003be0 con 0x7fbc9c15bf20 2023-07-14T11:53:16.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.785+0000 7fbc98ff9700 1 --2- 172.21.15.104:0/1016077069 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbc880662f0 0x7fbc880687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:16.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.785+0000 7fbc98ff9700 1 -- 172.21.15.104:0/1016077069 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fbc94020070 con 0x7fbc9c15bf20 2023-07-14T11:53:16.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.785+0000 7fbc9bfff700 1 --2- 172.21.15.104:0/1016077069 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbc880662f0 0x7fbc880687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:16.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.785+0000 7fbc9bfff700 1 --2- 172.21.15.104:0/1016077069 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbc880662f0 0x7fbc880687a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0012e0 tx=0x7fbc8c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:16.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.789+0000 7fbc98ff9700 1 -- 172.21.15.104:0/1016077069 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fbc9c1e6df0 con 0x7fbc9c15bf20 2023-07-14T11:53:16.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:16.961+0000 7fbca23d5700 1 -- 172.21.15.104:0/1016077069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbc9c1e6df0 con 0x7fbc9c15bf20 2023-07-14T11:53:18.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.633+0000 7fbc98ff9700 1 -- 172.21.15.104:0/1016077069 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v26) v1 ==== 228+0+0 (secure 0 0 0) 0x7fbc9c1e6df0 con 0x7fbc9c15bf20 2023-07-14T11:53:18.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.645+0000 7fbca23d5700 1 -- 172.21.15.104:0/1016077069 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbc880662f0 msgr2=0x7fbc880687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:18.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.645+0000 7fbca23d5700 1 --2- 172.21.15.104:0/1016077069 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbc880662f0 0x7fbc880687a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0012e0 tx=0x7fbc8c009040 comp rx=0 tx=0).stop 2023-07-14T11:53:18.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.645+0000 7fbca23d5700 1 -- 172.21.15.104:0/1016077069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc9c15bf20 msgr2=0x7fbc9c152a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:18.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.645+0000 7fbca23d5700 1 --2- 172.21.15.104:0/1016077069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc9c15bf20 0x7fbc9c152a60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009c60 tx=0x7fbc94004770 comp rx=0 tx=0).stop 2023-07-14T11:53:18.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.645+0000 7fbca23d5700 1 -- 172.21.15.104:0/1016077069 shutdown_connections 2023-07-14T11:53:18.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.645+0000 7fbca23d5700 1 --2- 172.21.15.104:0/1016077069 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbc880662f0 0x7fbc880687a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.645+0000 7fbca23d5700 1 --2- 172.21.15.104:0/1016077069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc9c15bf20 0x7fbc9c152a60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.645+0000 7fbca23d5700 1 --2- 172.21.15.104:0/1016077069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc9c0cd530 0x7fbc9c1e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.645+0000 7fbca23d5700 1 --2- 172.21.15.104:0/1016077069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc9c071430 0x7fbc9c1e57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.645+0000 7fbca23d5700 1 -- 172.21.15.104:0/1016077069 >> 172.21.15.104:0/1016077069 conn(0x7fbc9c06a0c0 msgr2=0x7fbc9c156e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:18.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.645+0000 7fbca23d5700 1 -- 172.21.15.104:0/1016077069 shutdown_connections 2023-07-14T11:53:18.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.645+0000 7fbca23d5700 1 -- 172.21.15.104:0/1016077069 wait complete. 2023-07-14T11:53:18.634 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'cephfs_data_ec' created 2023-07-14T11:53:18.652 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-07-14T11:53:18.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.841+0000 7f9e3aadf700 1 Processor -- start 2023-07-14T11:53:18.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.841+0000 7f9e3aadf700 1 -- start start 2023-07-14T11:53:18.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3aadf700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9e340d02d0 0x7f9e340d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:18.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3aadf700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9e340b40f0 0x7f9e340cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:18.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3aadf700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e340cd830 0x7f9e340cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:18.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3aadf700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e34071bb0 con 0x7f9e340d02d0 2023-07-14T11:53:18.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3aadf700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e340d0be0 con 0x7f9e340b40f0 2023-07-14T11:53:18.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3aadf700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e34071310 con 0x7f9e340cd830 2023-07-14T11:53:18.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3887b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9e340d02d0 0x7f9e340d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:18.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3887b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9e340d02d0 0x7f9e340d06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36764/0 (socket says 172.21.15.104:36764) 2023-07-14T11:53:18.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3887b700 1 -- 172.21.15.104:0/1636841757 learned_addr learned my addr 172.21.15.104:0/1636841757 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:18.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e33fff700 1 --2- 172.21.15.104:0/1636841757 >> v2:172.21.15.121:3300/0 conn(0x7f9e340b40f0 0x7f9e340cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:18.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3887b700 1 -- 172.21.15.104:0/1636841757 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e340cd830 msgr2=0x7f9e340cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:18.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3907c700 1 --2- 172.21.15.104:0/1636841757 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e340cd830 0x7f9e340cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:18.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3887b700 1 --2- 172.21.15.104:0/1636841757 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e340cd830 0x7f9e340cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3887b700 1 -- 172.21.15.104:0/1636841757 >> v2:172.21.15.121:3300/0 conn(0x7f9e340b40f0 msgr2=0x7f9e340cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:18.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3887b700 1 --2- 172.21.15.104:0/1636841757 >> v2:172.21.15.121:3300/0 conn(0x7f9e340b40f0 0x7f9e340cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3887b700 1 -- 172.21.15.104:0/1636841757 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e340715f0 con 0x7f9e340d02d0 2023-07-14T11:53:18.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3907c700 1 --2- 172.21.15.104:0/1636841757 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e340cd830 0x7f9e340cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:18.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e33fff700 1 --2- 172.21.15.104:0/1636841757 >> v2:172.21.15.121:3300/0 conn(0x7f9e340b40f0 0x7f9e340cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:18.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.845+0000 7f9e3887b700 1 --2- 172.21.15.104:0/1636841757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e340d02d0 0x7f9e340d06a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9e3406e640 tx=0x7f9e28009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de7a7792fac0323 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:18.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.849+0000 7f9e337fe700 1 -- 172.21.15.104:0/1636841757 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2800b8a0 con 0x7f9e340d02d0 2023-07-14T11:53:18.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.849+0000 7f9e337fe700 1 -- 172.21.15.104:0/1636841757 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2800ba00 con 0x7f9e340d02d0 2023-07-14T11:53:18.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.849+0000 7f9e337fe700 1 -- 172.21.15.104:0/1636841757 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2800bd90 con 0x7f9e340d02d0 2023-07-14T11:53:18.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.849+0000 7f9e3aadf700 1 -- 172.21.15.104:0/1636841757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e340d02d0 msgr2=0x7f9e340d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:18.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.849+0000 7f9e3aadf700 1 --2- 172.21.15.104:0/1636841757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e340d02d0 0x7f9e340d06a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9e3406e640 tx=0x7f9e28009bf0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.849+0000 7f9e3aadf700 1 -- 172.21.15.104:0/1636841757 shutdown_connections 2023-07-14T11:53:18.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.849+0000 7f9e3aadf700 1 --2- 172.21.15.104:0/1636841757 >> v2:172.21.15.121:3300/0 conn(0x7f9e340b40f0 0x7f9e340cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.849+0000 7f9e3aadf700 1 --2- 172.21.15.104:0/1636841757 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e340cd830 0x7f9e340cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.849+0000 7f9e3aadf700 1 --2- 172.21.15.104:0/1636841757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e340d02d0 0x7f9e340d06a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.849+0000 7f9e3aadf700 1 -- 172.21.15.104:0/1636841757 >> 172.21.15.104:0/1636841757 conn(0x7f9e3409b9e0 msgr2=0x7f9e340d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:18.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.849+0000 7f9e3aadf700 1 -- 172.21.15.104:0/1636841757 shutdown_connections 2023-07-14T11:53:18.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.849+0000 7f9e3aadf700 1 -- 172.21.15.104:0/1636841757 wait complete. 2023-07-14T11:53:18.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.853+0000 7f9e3aadf700 1 Processor -- start 2023-07-14T11:53:18.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.853+0000 7f9e3aadf700 1 -- start start 2023-07-14T11:53:18.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.853+0000 7f9e3aadf700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e340b40f0 0x7f9e341f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:18.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.853+0000 7f9e3aadf700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e340cd830 0x7f9e3415d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:18.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.853+0000 7f9e3aadf700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e340d02d0 0x7f9e341f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:18.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.853+0000 7f9e3aadf700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e341f6700 con 0x7f9e340b40f0 2023-07-14T11:53:18.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.853+0000 7f9e3aadf700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340cab50 con 0x7f9e340d02d0 2023-07-14T11:53:18.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.853+0000 7f9e3aadf700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340cae30 con 0x7f9e340cd830 2023-07-14T11:53:18.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.853+0000 7f9e3887b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e340b40f0 0x7f9e341f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:18.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.853+0000 7f9e3887b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e340b40f0 0x7f9e341f61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36780/0 (socket says 172.21.15.104:36780) 2023-07-14T11:53:18.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.853+0000 7f9e3887b700 1 -- 172.21.15.104:0/1961626065 learned_addr learned my addr 172.21.15.104:0/1961626065 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:18.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.853+0000 7f9e33fff700 1 --2- 172.21.15.104:0/1961626065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e340cd830 0x7f9e3415d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:18.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.853+0000 7f9e3907c700 1 --2- 172.21.15.104:0/1961626065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e340d02d0 0x7f9e341f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:18.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.857+0000 7f9e3887b700 1 -- 172.21.15.104:0/1961626065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e340cd830 msgr2=0x7f9e3415d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:18.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.857+0000 7f9e3887b700 1 --2- 172.21.15.104:0/1961626065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e340cd830 0x7f9e3415d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.857+0000 7f9e3887b700 1 -- 172.21.15.104:0/1961626065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e340d02d0 msgr2=0x7f9e341f8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:18.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.857+0000 7f9e3887b700 1 --2- 172.21.15.104:0/1961626065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e340d02d0 0x7f9e341f8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.857+0000 7f9e3887b700 1 -- 172.21.15.104:0/1961626065 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e28009570 con 0x7f9e340b40f0 2023-07-14T11:53:18.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.857+0000 7f9e3887b700 1 --2- 172.21.15.104:0/1961626065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e340b40f0 0x7f9e341f61c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9e34072e40 tx=0x7f9e28004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:18.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.857+0000 7f9e317fa700 1 -- 172.21.15.104:0/1961626065 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e280045a0 con 0x7f9e340b40f0 2023-07-14T11:53:18.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.857+0000 7f9e317fa700 1 -- 172.21.15.104:0/1961626065 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2801f070 con 0x7f9e340b40f0 2023-07-14T11:53:18.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.857+0000 7f9e3aadf700 1 -- 172.21.15.104:0/1961626065 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e340cb110 con 0x7f9e340b40f0 2023-07-14T11:53:18.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.857+0000 7f9e317fa700 1 -- 172.21.15.104:0/1961626065 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28003900 con 0x7f9e340b40f0 2023-07-14T11:53:18.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.857+0000 7f9e3aadf700 1 -- 172.21.15.104:0/1961626065 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e340cb6d0 con 0x7f9e340b40f0 2023-07-14T11:53:18.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.857+0000 7f9e3aadf700 1 -- 172.21.15.104:0/1961626065 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3404dcb0 con 0x7f9e340b40f0 2023-07-14T11:53:18.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.861+0000 7f9e317fa700 1 -- 172.21.15.104:0/1961626065 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9e28021030 con 0x7f9e340b40f0 2023-07-14T11:53:18.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.861+0000 7f9e317fa700 1 --2- 172.21.15.104:0/1961626065 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e180662f0 0x7f9e180687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:18.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.861+0000 7f9e317fa700 1 -- 172.21.15.104:0/1961626065 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f9e28085950 con 0x7f9e340b40f0 2023-07-14T11:53:18.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.861+0000 7f9e33fff700 1 --2- 172.21.15.104:0/1961626065 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e180662f0 0x7f9e180687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:18.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.865+0000 7f9e33fff700 1 --2- 172.21.15.104:0/1961626065 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e180662f0 0x7f9e180687a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e3415dd40 tx=0x7f9e20008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:18.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:18.869+0000 7f9e317fa700 1 -- 172.21.15.104:0/1961626065 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f9e28052af0 con 0x7f9e340b40f0 2023-07-14T11:53:19.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.037+0000 7f9e3aadf700 1 -- 172.21.15.104:0/1961626065 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f9e340c58e0 con 0x7f9e340b40f0 2023-07-14T11:53:19.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.637+0000 7f9e317fa700 1 -- 172.21.15.104:0/1961626065 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9e28056110 con 0x7f9e340b40f0 2023-07-14T11:53:19.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.649+0000 7f9e3aadf700 1 -- 172.21.15.104:0/1961626065 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e180662f0 msgr2=0x7f9e180687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:19.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.649+0000 7f9e3aadf700 1 --2- 172.21.15.104:0/1961626065 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e180662f0 0x7f9e180687a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e3415dd40 tx=0x7f9e20008040 comp rx=0 tx=0).stop 2023-07-14T11:53:19.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.649+0000 7f9e3aadf700 1 -- 172.21.15.104:0/1961626065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e340b40f0 msgr2=0x7f9e341f61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:19.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.649+0000 7f9e3aadf700 1 --2- 172.21.15.104:0/1961626065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e340b40f0 0x7f9e341f61c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9e34072e40 tx=0x7f9e28004770 comp rx=0 tx=0).stop 2023-07-14T11:53:19.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.653+0000 7f9e3aadf700 1 -- 172.21.15.104:0/1961626065 shutdown_connections 2023-07-14T11:53:19.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.657+0000 7f9e3aadf700 1 --2- 172.21.15.104:0/1961626065 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e180662f0 0x7f9e180687a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:19.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.657+0000 7f9e3aadf700 1 --2- 172.21.15.104:0/1961626065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e340d02d0 0x7f9e341f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:19.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.657+0000 7f9e3aadf700 1 --2- 172.21.15.104:0/1961626065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e340cd830 0x7f9e3415d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:19.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.657+0000 7f9e3aadf700 1 --2- 172.21.15.104:0/1961626065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e340b40f0 0x7f9e341f61c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:19.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.657+0000 7f9e3aadf700 1 -- 172.21.15.104:0/1961626065 >> 172.21.15.104:0/1961626065 conn(0x7f9e3409b9e0 msgr2=0x7f9e340d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:19.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.657+0000 7f9e3aadf700 1 -- 172.21.15.104:0/1961626065 shutdown_connections 2023-07-14T11:53:19.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.657+0000 7f9e3aadf700 1 -- 172.21.15.104:0/1961626065 wait complete. 2023-07-14T11:53:19.652 INFO:teuthology.orchestra.run.smithi104.stderr:set pool 5 allow_ec_overwrites to true 2023-07-14T11:53:19.674 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-07-14T11:53:19.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.853+0000 7f2950f9e700 1 Processor -- start 2023-07-14T11:53:19.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f2950f9e700 1 -- start start 2023-07-14T11:53:19.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f2950f9e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f294c160000 0x7f294c06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:19.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f2950f9e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f294c1603d0 0x7f294c0c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:19.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f2950f9e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c0cdfb0 con 0x7f294c0cd7f0 2023-07-14T11:53:19.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f2950f9e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294c070e40 con 0x7f294c160000 2023-07-14T11:53:19.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f2950f9e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294c0cdcd0 con 0x7f294c1603d0 2023-07-14T11:53:19.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f294b7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f294c160000 0x7f294c06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:19.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f294bfff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f294c0cd7f0 0x7f294c0c0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:38222/0 (socket says 172.21.15.104:38222) 2023-07-14T11:53:19.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f294b7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f294c160000 0x7f294c06dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36796/0 (socket says 172.21.15.104:36796) 2023-07-14T11:53:19.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f294bfff700 1 -- 172.21.15.104:0/362966248 learned_addr learned my addr 172.21.15.104:0/362966248 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:19.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f2942ffd700 1 --2- 172.21.15.104:0/362966248 >> v2:172.21.15.121:3300/0 conn(0x7f294c1603d0 0x7f294c0c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:19.847 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f294b7fe700 1 -- 172.21.15.104:0/362966248 >> v2:172.21.15.121:3300/0 conn(0x7f294c1603d0 msgr2=0x7f294c0c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:19.847 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f294b7fe700 1 --2- 172.21.15.104:0/362966248 >> v2:172.21.15.121:3300/0 conn(0x7f294c1603d0 0x7f294c0c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:19.847 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f294b7fe700 1 -- 172.21.15.104:0/362966248 >> v1:172.21.15.104:6789/0 conn(0x7f294c0cd7f0 legacy=0x7f294c0c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:19.847 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f294affd700 1 -- 172.21.15.104:0/362966248 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119649823 0 0) 0x7f2944001110 con 0x7f294c0cd7f0 2023-07-14T11:53:19.847 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.857+0000 7f294b7fe700 1 -- 172.21.15.104:0/362966248 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c070780 con 0x7f294c160000 2023-07-14T11:53:19.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.861+0000 7f294b7fe700 1 --2- 172.21.15.104:0/362966248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f294c160000 0x7f294c06dd10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f294c071a20 tx=0x7f293c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5967e22500c1b000 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:19.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.861+0000 7f294affd700 1 -- 172.21.15.104:0/362966248 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c011830 con 0x7f294c160000 2023-07-14T11:53:19.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.861+0000 7f294affd700 1 -- 172.21.15.104:0/362966248 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c011990 con 0x7f294c160000 2023-07-14T11:53:19.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.861+0000 7f294affd700 1 -- 172.21.15.104:0/362966248 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c011d20 con 0x7f294c160000 2023-07-14T11:53:19.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.861+0000 7f2950f9e700 1 -- 172.21.15.104:0/362966248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f294c160000 msgr2=0x7f294c06dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:19.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.861+0000 7f2950f9e700 1 --2- 172.21.15.104:0/362966248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f294c160000 0x7f294c06dd10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f294c071a20 tx=0x7f293c009b30 comp rx=0 tx=0).stop 2023-07-14T11:53:19.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.861+0000 7f2950f9e700 1 -- 172.21.15.104:0/362966248 shutdown_connections 2023-07-14T11:53:19.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.861+0000 7f2950f9e700 1 --2- 172.21.15.104:0/362966248 >> v2:172.21.15.121:3300/0 conn(0x7f294c1603d0 0x7f294c0c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:19.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.861+0000 7f2950f9e700 1 --2- 172.21.15.104:0/362966248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f294c160000 0x7f294c06dd10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:19.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.861+0000 7f2950f9e700 1 -- 172.21.15.104:0/362966248 >> 172.21.15.104:0/362966248 conn(0x7f294c0afc60 msgr2=0x7f294c0b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:19.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.861+0000 7f2950f9e700 1 -- 172.21.15.104:0/362966248 shutdown_connections 2023-07-14T11:53:19.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.861+0000 7f2950f9e700 1 -- 172.21.15.104:0/362966248 wait complete. 2023-07-14T11:53:19.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.865+0000 7f2950f9e700 1 Processor -- start 2023-07-14T11:53:19.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.865+0000 7f2950f9e700 1 -- start start 2023-07-14T11:53:19.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f2950f9e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f294c0cd7f0 0x7f294c0b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:19.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f2950f9e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f294c160000 0x7f294c0b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:19.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f2950f9e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f294c1603d0 0x7f294c0b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:19.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f2950f9e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c161170 con 0x7f294c0cd7f0 2023-07-14T11:53:19.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f2950f9e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c160840 con 0x7f294c160000 2023-07-14T11:53:19.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f2950f9e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c160b20 con 0x7f294c1603d0 2023-07-14T11:53:19.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f294b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f294c0cd7f0 0x7f294c0b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:19.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f294b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f294c0cd7f0 0x7f294c0b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36812/0 (socket says 172.21.15.104:36812) 2023-07-14T11:53:19.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f294b7fe700 1 -- 172.21.15.104:0/22241524 learned_addr learned my addr 172.21.15.104:0/22241524 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:19.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f2942ffd700 1 --2- 172.21.15.104:0/22241524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f294c160000 0x7f294c0b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:19.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f294bfff700 1 --2- 172.21.15.104:0/22241524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f294c1603d0 0x7f294c0b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:19.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f294b7fe700 1 -- 172.21.15.104:0/22241524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f294c1603d0 msgr2=0x7f294c0b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:19.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f294b7fe700 1 --2- 172.21.15.104:0/22241524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f294c1603d0 0x7f294c0b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:19.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f294b7fe700 1 -- 172.21.15.104:0/22241524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f294c160000 msgr2=0x7f294c0b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:19.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f294b7fe700 1 --2- 172.21.15.104:0/22241524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f294c160000 0x7f294c0b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:19.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f294b7fe700 1 -- 172.21.15.104:0/22241524 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c0094b0 con 0x7f294c0cd7f0 2023-07-14T11:53:19.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f2942ffd700 1 --2- 172.21.15.104:0/22241524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f294c160000 0x7f294c0b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:19.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f294b7fe700 1 --2- 172.21.15.104:0/22241524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f294c0cd7f0 0x7f294c0b2af0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f293c009fd0 tx=0x7f293c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:19.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f2948ff9700 1 -- 172.21.15.104:0/22241524 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c004480 con 0x7f294c0cd7f0 2023-07-14T11:53:19.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f2948ff9700 1 -- 172.21.15.104:0/22241524 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c01f070 con 0x7f294c0cd7f0 2023-07-14T11:53:19.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f2948ff9700 1 -- 172.21.15.104:0/22241524 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c0038c0 con 0x7f294c0cd7f0 2023-07-14T11:53:19.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f2950f9e700 1 -- 172.21.15.104:0/22241524 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c160e00 con 0x7f294c0cd7f0 2023-07-14T11:53:19.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.869+0000 7f2950f9e700 1 -- 172.21.15.104:0/22241524 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294c0b1e80 con 0x7f294c0cd7f0 2023-07-14T11:53:19.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.873+0000 7f2950f9e700 1 -- 172.21.15.104:0/22241524 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294c04e500 con 0x7f294c0cd7f0 2023-07-14T11:53:19.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.873+0000 7f2948ff9700 1 -- 172.21.15.104:0/22241524 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f293c021030 con 0x7f294c0cd7f0 2023-07-14T11:53:19.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.873+0000 7f2948ff9700 1 --2- 172.21.15.104:0/22241524 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f292c0662f0 0x7f292c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:19.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.873+0000 7f2948ff9700 1 -- 172.21.15.104:0/22241524 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f293c085500 con 0x7f294c0cd7f0 2023-07-14T11:53:19.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.877+0000 7f2942ffd700 1 --2- 172.21.15.104:0/22241524 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f292c0662f0 0x7f292c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:19.863 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.877+0000 7f2942ffd700 1 --2- 172.21.15.104:0/22241524 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f292c0662f0 0x7f292c0687a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f294c155c50 tx=0x7f293400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:19.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:19.881+0000 7f2948ff9700 1 -- 172.21.15.104:0/22241524 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f293c0527c0 con 0x7f294c0cd7f0 2023-07-14T11:53:20.072 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.085+0000 7f2950f9e700 1 -- 172.21.15.104:0/22241524 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f294c0b40d0 con 0x7f294c0cd7f0 2023-07-14T11:53:20.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.665+0000 7f2948ff9700 1 -- 172.21.15.104:0/22241524 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f293c055de0 con 0x7f294c0cd7f0 2023-07-14T11:53:20.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.669+0000 7f2950f9e700 1 -- 172.21.15.104:0/22241524 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f292c0662f0 msgr2=0x7f292c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:20.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.669+0000 7f2950f9e700 1 --2- 172.21.15.104:0/22241524 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f292c0662f0 0x7f292c0687a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f294c155c50 tx=0x7f293400b040 comp rx=0 tx=0).stop 2023-07-14T11:53:20.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.669+0000 7f2950f9e700 1 -- 172.21.15.104:0/22241524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f294c0cd7f0 msgr2=0x7f294c0b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:20.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.669+0000 7f2950f9e700 1 --2- 172.21.15.104:0/22241524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f294c0cd7f0 0x7f294c0b2af0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f293c009fd0 tx=0x7f293c004770 comp rx=0 tx=0).stop 2023-07-14T11:53:20.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.669+0000 7f2950f9e700 1 -- 172.21.15.104:0/22241524 shutdown_connections 2023-07-14T11:53:20.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.669+0000 7f2950f9e700 1 --2- 172.21.15.104:0/22241524 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f292c0662f0 0x7f292c0687a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:20.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.669+0000 7f2950f9e700 1 --2- 172.21.15.104:0/22241524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f294c1603d0 0x7f294c0b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:20.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.669+0000 7f2950f9e700 1 --2- 172.21.15.104:0/22241524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f294c160000 0x7f294c0b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:20.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.669+0000 7f2950f9e700 1 --2- 172.21.15.104:0/22241524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f294c0cd7f0 0x7f294c0b2af0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:20.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.669+0000 7f2950f9e700 1 -- 172.21.15.104:0/22241524 >> 172.21.15.104:0/22241524 conn(0x7f294c0afc60 msgr2=0x7f294c15e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:20.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.669+0000 7f2950f9e700 1 -- 172.21.15.104:0/22241524 shutdown_connections 2023-07-14T11:53:20.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.669+0000 7f2950f9e700 1 -- 172.21.15.104:0/22241524 wait complete. 2023-07-14T11:53:20.660 INFO:teuthology.orchestra.run.smithi104.stderr:added data pool 5 to fsmap 2023-07-14T11:53:20.680 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:53:20.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.873+0000 7f314f40f700 1 Processor -- start 2023-07-14T11:53:20.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.873+0000 7f314f40f700 1 -- start start 2023-07-14T11:53:20.864 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314f40f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3148159e30 0x7f314815e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:20.864 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314f40f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f31480b06f0 0x7f31480ad750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:20.864 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314f40f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314815a200 con 0x7f31480caf40 2023-07-14T11:53:20.864 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314f40f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31480cf2a0 con 0x7f3148159e30 2023-07-14T11:53:20.864 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314f40f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31480cdcd0 con 0x7f31480b06f0 2023-07-14T11:53:20.865 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314e40d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3148159e30 0x7f314815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:20.865 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314e40d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3148159e30 0x7f314815e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46212/0 (socket says 172.21.15.104:46212) 2023-07-14T11:53:20.865 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314e40d700 1 -- 172.21.15.104:0/4233035126 learned_addr learned my addr 172.21.15.104:0/4233035126 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:20.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314dc0c700 1 --2- 172.21.15.104:0/4233035126 >> v2:172.21.15.121:3300/0 conn(0x7f31480b06f0 0x7f31480ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:20.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314e40d700 1 -- 172.21.15.104:0/4233035126 >> v2:172.21.15.121:3300/0 conn(0x7f31480b06f0 msgr2=0x7f31480ad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:20.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314e40d700 1 --2- 172.21.15.104:0/4233035126 >> v2:172.21.15.121:3300/0 conn(0x7f31480b06f0 0x7f31480ad750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:20.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314e40d700 1 -- 172.21.15.104:0/4233035126 >> v1:172.21.15.121:6789/0 conn(0x7f31480caf40 legacy=0x7f31480cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:20.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314e40d700 1 -- 172.21.15.104:0/4233035126 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31480cdfb0 con 0x7f3148159e30 2023-07-14T11:53:20.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314e40d700 1 --2- 172.21.15.104:0/4233035126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3148159e30 0x7f314815e240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f314806b2d0 tx=0x7f314400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbce43ccd6544e17 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:20.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314d40b700 1 -- 172.21.15.104:0/4233035126 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314400e070 con 0x7f3148159e30 2023-07-14T11:53:20.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314d40b700 1 -- 172.21.15.104:0/4233035126 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3144003a40 con 0x7f3148159e30 2023-07-14T11:53:20.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314d40b700 1 -- 172.21.15.104:0/4233035126 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3144003df0 con 0x7f3148159e30 2023-07-14T11:53:20.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314f40f700 1 -- 172.21.15.104:0/4233035126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3148159e30 msgr2=0x7f314815e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:20.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.877+0000 7f314f40f700 1 --2- 172.21.15.104:0/4233035126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3148159e30 0x7f314815e240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f314806b2d0 tx=0x7f314400b330 comp rx=0 tx=0).stop 2023-07-14T11:53:20.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.881+0000 7f314f40f700 1 -- 172.21.15.104:0/4233035126 shutdown_connections 2023-07-14T11:53:20.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.881+0000 7f314f40f700 1 --2- 172.21.15.104:0/4233035126 >> v2:172.21.15.121:3300/0 conn(0x7f31480b06f0 0x7f31480ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:20.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.881+0000 7f314f40f700 1 --2- 172.21.15.104:0/4233035126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3148159e30 0x7f314815e240 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:20.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.881+0000 7f314f40f700 1 -- 172.21.15.104:0/4233035126 >> 172.21.15.104:0/4233035126 conn(0x7f3148069a10 msgr2=0x7f3148070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:20.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.881+0000 7f314f40f700 1 -- 172.21.15.104:0/4233035126 shutdown_connections 2023-07-14T11:53:20.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.881+0000 7f314f40f700 1 -- 172.21.15.104:0/4233035126 wait complete. 2023-07-14T11:53:20.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.881+0000 7f314f40f700 1 Processor -- start 2023-07-14T11:53:20.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.881+0000 7f314f40f700 1 -- start start 2023-07-14T11:53:20.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.881+0000 7f314f40f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31480b06f0 0x7f31481f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:20.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.881+0000 7f314f40f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31480caf40 0x7f31481f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:20.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.881+0000 7f314f40f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3148159e30 0x7f314815c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:20.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.881+0000 7f314f40f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3148155390 con 0x7f3148159e30 2023-07-14T11:53:20.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.881+0000 7f314f40f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31480c35b0 con 0x7f31480b06f0 2023-07-14T11:53:20.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.881+0000 7f314f40f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31481f2140 con 0x7f31480caf40 2023-07-14T11:53:20.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f314e40d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31480b06f0 0x7f31481f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:20.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f314dc0c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31480caf40 0x7f31481f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:20.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f314e40d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31480b06f0 0x7f31481f4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:49326/0 (socket says 172.21.15.104:49326) 2023-07-14T11:53:20.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f314ec0e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3148159e30 0x7f314815c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:20.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f314e40d700 1 -- 172.21.15.104:0/1190751353 learned_addr learned my addr 172.21.15.104:0/1190751353 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:20.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f314ec0e700 1 -- 172.21.15.104:0/1190751353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31480caf40 msgr2=0x7f31481f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:20.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f314ec0e700 1 --2- 172.21.15.104:0/1190751353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31480caf40 0x7f31481f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:20.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f314ec0e700 1 -- 172.21.15.104:0/1190751353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31480b06f0 msgr2=0x7f31481f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:20.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f314ec0e700 1 --2- 172.21.15.104:0/1190751353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31480b06f0 0x7f31481f4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:20.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f314ec0e700 1 -- 172.21.15.104:0/1190751353 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3144009b30 con 0x7f3148159e30 2023-07-14T11:53:20.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f314ec0e700 1 --2- 172.21.15.104:0/1190751353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3148159e30 0x7f314815c5c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f314000c410 tx=0x7f314000c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:20.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f313effd700 1 -- 172.21.15.104:0/1190751353 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140012620 con 0x7f3148159e30 2023-07-14T11:53:20.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f313effd700 1 -- 172.21.15.104:0/1190751353 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31400127c0 con 0x7f3148159e30 2023-07-14T11:53:20.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f314f40f700 1 -- 172.21.15.104:0/1190751353 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31481f2420 con 0x7f3148159e30 2023-07-14T11:53:20.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f313effd700 1 -- 172.21.15.104:0/1190751353 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140012b70 con 0x7f3148159e30 2023-07-14T11:53:20.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f314f40f700 1 -- 172.21.15.104:0/1190751353 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31480c3ae0 con 0x7f3148159e30 2023-07-14T11:53:20.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.885+0000 7f314f40f700 1 -- 172.21.15.104:0/1190751353 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31481f2610 con 0x7f3148159e30 2023-07-14T11:53:20.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.889+0000 7f313effd700 1 -- 172.21.15.104:0/1190751353 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3140012cd0 con 0x7f3148159e30 2023-07-14T11:53:20.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.889+0000 7f313effd700 1 --2- 172.21.15.104:0/1190751353 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f312c0662f0 0x7f312c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:20.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.889+0000 7f313effd700 1 -- 172.21.15.104:0/1190751353 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3140083b80 con 0x7f3148159e30 2023-07-14T11:53:20.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.889+0000 7f314e40d700 1 --2- 172.21.15.104:0/1190751353 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f312c0662f0 0x7f312c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:20.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.889+0000 7f314e40d700 1 --2- 172.21.15.104:0/1190751353 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f312c0662f0 0x7f312c0687a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f314400b370 tx=0x7f3144009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:20.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:20.897+0000 7f313effd700 1 -- 172.21.15.104:0/1190751353 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f3140050c70 con 0x7f3148159e30 2023-07-14T11:53:21.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.101+0000 7f314f40f700 1 -- 172.21.15.104:0/1190751353 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31480c3cd0 con 0x7f3148159e30 2023-07-14T11:53:21.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.101+0000 7f313effd700 1 -- 172.21.15.104:0/1190751353 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4717 (secure 0 0 0) 0x7f3140083e60 con 0x7f3148159e30 2023-07-14T11:53:21.088 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:21.088 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2103358277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2103358277},{"type":"v1","addr":"172.21.15.121:6833","nonce":2103358277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1841065287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1841065287},{"type":"v1","addr":"172.21.15.121:6835","nonce":1841065287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3342053518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3342053518},{"type":"v1","addr":"172.21.15.104:6835","nonce":3342053518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:53:15.029045+0000","modified":"2023-07-14T11:53:20.652622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/2666329866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2666329866},{"type":"v1","addr":"172.21.15.104:6837","nonce":2666329866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-14T11:53:21.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.101+0000 7f314f40f700 1 -- 172.21.15.104:0/1190751353 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f312c0662f0 msgr2=0x7f312c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:21.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.101+0000 7f314f40f700 1 --2- 172.21.15.104:0/1190751353 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f312c0662f0 0x7f312c0687a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f314400b370 tx=0x7f3144009000 comp rx=0 tx=0).stop 2023-07-14T11:53:21.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.101+0000 7f314f40f700 1 -- 172.21.15.104:0/1190751353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3148159e30 msgr2=0x7f314815c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:21.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.101+0000 7f314f40f700 1 --2- 172.21.15.104:0/1190751353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3148159e30 0x7f314815c5c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f314000c410 tx=0x7f314000c720 comp rx=0 tx=0).stop 2023-07-14T11:53:21.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.101+0000 7f314f40f700 1 -- 172.21.15.104:0/1190751353 shutdown_connections 2023-07-14T11:53:21.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.101+0000 7f314f40f700 1 --2- 172.21.15.104:0/1190751353 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f312c0662f0 0x7f312c0687a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.101+0000 7f314f40f700 1 --2- 172.21.15.104:0/1190751353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3148159e30 0x7f314815c5c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.101+0000 7f314f40f700 1 --2- 172.21.15.104:0/1190751353 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31480caf40 0x7f31481f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.105+0000 7f314f40f700 1 --2- 172.21.15.104:0/1190751353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31480b06f0 0x7f31481f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.105+0000 7f314f40f700 1 -- 172.21.15.104:0/1190751353 >> 172.21.15.104:0/1190751353 conn(0x7f3148069a10 msgr2=0x7f314806e960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:21.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.105+0000 7f314f40f700 1 -- 172.21.15.104:0/1190751353 shutdown_connections 2023-07-14T11:53:21.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.105+0000 7f314f40f700 1 -- 172.21.15.104:0/1190751353 wait complete. 2023-07-14T11:53:21.092 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 7 2023-07-14T11:53:21.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:21.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.293+0000 7fb25691f700 1 Processor -- start 2023-07-14T11:53:21.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25691f700 1 -- start start 2023-07-14T11:53:21.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25691f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb2500c9d90 0x7fb2500ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:21.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25691f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb250159f20 0x7fb2500caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:21.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25691f700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2500b0ef0 con 0x7fb2500caf70 2023-07-14T11:53:21.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25691f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2500ce9c0 con 0x7fb2500c9d90 2023-07-14T11:53:21.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25691f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2500cee50 con 0x7fb250159f20 2023-07-14T11:53:21.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25591d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb2500c9d90 0x7fb2500ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:21.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25611e700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb2500caf70 0x7fb2500c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:51128/0 (socket says 172.21.15.104:51128) 2023-07-14T11:53:21.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25611e700 1 -- 172.21.15.104:0/694561392 learned_addr learned my addr 172.21.15.104:0/694561392 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:21.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25511c700 1 --2- 172.21.15.104:0/694561392 >> v2:172.21.15.121:3300/0 conn(0x7fb250159f20 0x7fb2500caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:21.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25591d700 1 -- 172.21.15.104:0/694561392 >> v2:172.21.15.121:3300/0 conn(0x7fb250159f20 msgr2=0x7fb2500caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:21.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25591d700 1 --2- 172.21.15.104:0/694561392 >> v2:172.21.15.121:3300/0 conn(0x7fb250159f20 0x7fb2500caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25591d700 1 -- 172.21.15.104:0/694561392 >> v1:172.21.15.104:6789/0 conn(0x7fb2500caf70 legacy=0x7fb2500c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:21.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25591d700 1 -- 172.21.15.104:0/694561392 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2500ca6a0 con 0x7fb2500c9d90 2023-07-14T11:53:21.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25491b700 1 -- 172.21.15.104:0/694561392 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348538377 0 0) 0x7fb2500b0ef0 con 0x7fb2500caf70 2023-07-14T11:53:21.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.297+0000 7fb25591d700 1 --2- 172.21.15.104:0/694561392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2500c9d90 0x7fb2500ca160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb25006aa00 tx=0x7fb24c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4cce6d1bd5b476a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:21.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.301+0000 7fb25491b700 1 -- 172.21.15.104:0/694561392 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c011830 con 0x7fb2500c9d90 2023-07-14T11:53:21.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.301+0000 7fb25491b700 1 -- 172.21.15.104:0/694561392 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24c011990 con 0x7fb2500c9d90 2023-07-14T11:53:21.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.301+0000 7fb25491b700 1 -- 172.21.15.104:0/694561392 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c011d40 con 0x7fb2500c9d90 2023-07-14T11:53:21.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.301+0000 7fb25691f700 1 -- 172.21.15.104:0/694561392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2500c9d90 msgr2=0x7fb2500ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:21.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.301+0000 7fb25691f700 1 --2- 172.21.15.104:0/694561392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2500c9d90 0x7fb2500ca160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb25006aa00 tx=0x7fb24c009b30 comp rx=0 tx=0).stop 2023-07-14T11:53:21.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.301+0000 7fb25691f700 1 -- 172.21.15.104:0/694561392 shutdown_connections 2023-07-14T11:53:21.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.301+0000 7fb25691f700 1 --2- 172.21.15.104:0/694561392 >> v2:172.21.15.121:3300/0 conn(0x7fb250159f20 0x7fb2500caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.301+0000 7fb25691f700 1 --2- 172.21.15.104:0/694561392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2500c9d90 0x7fb2500ca160 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.301+0000 7fb25691f700 1 -- 172.21.15.104:0/694561392 >> 172.21.15.104:0/694561392 conn(0x7fb25006e960 msgr2=0x7fb25006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:21.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.301+0000 7fb25691f700 1 -- 172.21.15.104:0/694561392 shutdown_connections 2023-07-14T11:53:21.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.301+0000 7fb25691f700 1 -- 172.21.15.104:0/694561392 wait complete. 2023-07-14T11:53:21.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.305+0000 7fb25691f700 1 Processor -- start 2023-07-14T11:53:21.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.305+0000 7fb25691f700 1 -- start start 2023-07-14T11:53:21.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.305+0000 7fb25691f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2500c9d90 0x7fb2500c4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:21.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.305+0000 7fb25691f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2500caf70 0x7fb2500c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:21.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.305+0000 7fb25691f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb250159f20 0x7fb2500c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:21.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.305+0000 7fb25691f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2500c2ae0 con 0x7fb2500c9d90 2023-07-14T11:53:21.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.305+0000 7fb25691f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2500c2d10 con 0x7fb250159f20 2023-07-14T11:53:21.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.305+0000 7fb25691f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2500c2ff0 con 0x7fb2500caf70 2023-07-14T11:53:21.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.305+0000 7fb25591d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2500c9d90 0x7fb2500c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:21.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.305+0000 7fb25591d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2500c9d90 0x7fb2500c4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46244/0 (socket says 172.21.15.104:46244) 2023-07-14T11:53:21.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.305+0000 7fb25591d700 1 -- 172.21.15.104:0/1354601457 learned_addr learned my addr 172.21.15.104:0/1354601457 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:21.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.305+0000 7fb25511c700 1 --2- 172.21.15.104:0/1354601457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2500caf70 0x7fb2500c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:21.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.305+0000 7fb25611e700 1 --2- 172.21.15.104:0/1354601457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb250159f20 0x7fb2500c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:21.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.309+0000 7fb25591d700 1 -- 172.21.15.104:0/1354601457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2500caf70 msgr2=0x7fb2500c25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:21.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.309+0000 7fb25591d700 1 --2- 172.21.15.104:0/1354601457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2500caf70 0x7fb2500c25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.309+0000 7fb25591d700 1 -- 172.21.15.104:0/1354601457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb250159f20 msgr2=0x7fb2500c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:21.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.309+0000 7fb25591d700 1 --2- 172.21.15.104:0/1354601457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb250159f20 0x7fb2500c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.309+0000 7fb25591d700 1 -- 172.21.15.104:0/1354601457 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c0094b0 con 0x7fb2500c9d90 2023-07-14T11:53:21.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.309+0000 7fb25591d700 1 --2- 172.21.15.104:0/1354601457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2500c9d90 0x7fb2500c4040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb24c014040 tx=0x7fb24c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:21.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.309+0000 7fb2467fc700 1 -- 172.21.15.104:0/1354601457 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c019980 con 0x7fb2500c9d90 2023-07-14T11:53:21.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.309+0000 7fb2467fc700 1 -- 172.21.15.104:0/1354601457 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24c01e070 con 0x7fb2500c9d90 2023-07-14T11:53:21.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.309+0000 7fb2467fc700 1 -- 172.21.15.104:0/1354601457 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c019d40 con 0x7fb2500c9d90 2023-07-14T11:53:21.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.309+0000 7fb25691f700 1 -- 172.21.15.104:0/1354601457 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2500c32d0 con 0x7fb2500c9d90 2023-07-14T11:53:21.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.309+0000 7fb25691f700 1 -- 172.21.15.104:0/1354601457 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb250156370 con 0x7fb2500c9d90 2023-07-14T11:53:21.298 INFO:tasks.ceph.mds.c.smithi104.stderr:2023-07-14T11:53:21.309+0000 7f5b769da700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-14T11:53:21.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.309+0000 7fb25691f700 1 -- 172.21.15.104:0/1354601457 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25004e500 con 0x7fb2500c9d90 2023-07-14T11:53:21.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.313+0000 7fb2467fc700 1 -- 172.21.15.104:0/1354601457 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb24c02d450 con 0x7fb2500c9d90 2023-07-14T11:53:21.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.313+0000 7fb2467fc700 1 --2- 172.21.15.104:0/1354601457 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2340662f0 0x7fb2340687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:21.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.313+0000 7fb2467fc700 1 -- 172.21.15.104:0/1354601457 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb24c084fe0 con 0x7fb2500c9d90 2023-07-14T11:53:21.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.313+0000 7fb25511c700 1 --2- 172.21.15.104:0/1354601457 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2340662f0 0x7fb2340687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:21.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.313+0000 7fb25511c700 1 --2- 172.21.15.104:0/1354601457 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2340662f0 0x7fb2340687a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb240009a00 tx=0x7fb240008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:21.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.321+0000 7fb2467fc700 1 -- 172.21.15.104:0/1354601457 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb24c0521a0 con 0x7fb2500c9d90 2023-07-14T11:53:21.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.497+0000 7fb25691f700 1 -- 172.21.15.104:0/1354601457 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2500aeb30 con 0x7fb2500c9d90 2023-07-14T11:53:21.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.497+0000 7fb2467fc700 1 -- 172.21.15.104:0/1354601457 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20156 (secure 0 0 0) 0x7fb24c015020 con 0x7fb2500c9d90 2023-07-14T11:53:21.488 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:21.488 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":28,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:20.091434+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:21.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.501+0000 7fb25691f700 1 -- 172.21.15.104:0/1354601457 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2340662f0 msgr2=0x7fb2340687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:21.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.501+0000 7fb25691f700 1 --2- 172.21.15.104:0/1354601457 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2340662f0 0x7fb2340687a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb240009a00 tx=0x7fb240008040 comp rx=0 tx=0).stop 2023-07-14T11:53:21.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.501+0000 7fb25691f700 1 -- 172.21.15.104:0/1354601457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2500c9d90 msgr2=0x7fb2500c4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:21.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.501+0000 7fb25691f700 1 --2- 172.21.15.104:0/1354601457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2500c9d90 0x7fb2500c4040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb24c014040 tx=0x7fb24c004770 comp rx=0 tx=0).stop 2023-07-14T11:53:21.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.501+0000 7fb25691f700 1 -- 172.21.15.104:0/1354601457 shutdown_connections 2023-07-14T11:53:21.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.501+0000 7fb25691f700 1 --2- 172.21.15.104:0/1354601457 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2340662f0 0x7fb2340687a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.501+0000 7fb25691f700 1 --2- 172.21.15.104:0/1354601457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb250159f20 0x7fb2500c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.501+0000 7fb25691f700 1 --2- 172.21.15.104:0/1354601457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2500caf70 0x7fb2500c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.501+0000 7fb25691f700 1 --2- 172.21.15.104:0/1354601457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2500c9d90 0x7fb2500c4040 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.501+0000 7fb25691f700 1 -- 172.21.15.104:0/1354601457 >> 172.21.15.104:0/1354601457 conn(0x7fb25006e960 msgr2=0x7fb2500af050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:21.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.501+0000 7fb25691f700 1 -- 172.21.15.104:0/1354601457 shutdown_connections 2023-07-14T11:53:21.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.501+0000 7fb25691f700 1 -- 172.21.15.104:0/1354601457 wait complete. 2023-07-14T11:53:21.511 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:21.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.693+0000 7fb148045700 1 Processor -- start 2023-07-14T11:53:21.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.697+0000 7fb148045700 1 -- start start 2023-07-14T11:53:21.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.697+0000 7fb148045700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb1400cddb0 0x7fb140070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:21.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.697+0000 7fb148045700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb140070ea0 0x7fb1400afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:21.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.697+0000 7fb148045700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1400c8900 con 0x7fb140160640 2023-07-14T11:53:21.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.697+0000 7fb148045700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1400c8fc0 con 0x7fb140070ea0 2023-07-14T11:53:21.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.697+0000 7fb148045700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1400712e0 con 0x7fb1400cddb0 2023-07-14T11:53:21.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.697+0000 7fb1465e2700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb140070ea0 0x7fb1400afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:21.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.697+0000 7fb1465e2700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb140070ea0 0x7fb1400afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46256/0 (socket says 172.21.15.104:46256) 2023-07-14T11:53:21.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.697+0000 7fb1465e2700 1 -- 172.21.15.104:0/3471898160 learned_addr learned my addr 172.21.15.104:0/3471898160 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:21.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.697+0000 7fb1455e0700 1 --2- 172.21.15.104:0/3471898160 >> v2:172.21.15.121:3300/0 conn(0x7fb1400cddb0 0x7fb140070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:21.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.697+0000 7fb1465e2700 1 -- 172.21.15.104:0/3471898160 >> v2:172.21.15.121:3300/0 conn(0x7fb1400cddb0 msgr2=0x7fb140070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:21.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.697+0000 7fb1465e2700 1 --2- 172.21.15.104:0/3471898160 >> v2:172.21.15.121:3300/0 conn(0x7fb1400cddb0 0x7fb140070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.697+0000 7fb1465e2700 1 -- 172.21.15.104:0/3471898160 >> v1:172.21.15.121:6789/0 conn(0x7fb140160640 legacy=0x7fb140164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:21.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.697+0000 7fb144ddf700 1 -- 172.21.15.104:0/3471898160 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073474932 0 0) 0x7fb1400c8900 con 0x7fb140160640 2023-07-14T11:53:21.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.697+0000 7fb1465e2700 1 -- 172.21.15.104:0/3471898160 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1400caae0 con 0x7fb140070ea0 2023-07-14T11:53:21.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.701+0000 7fb1465e2700 1 --2- 172.21.15.104:0/3471898160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140070ea0 0x7fb1400afd30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb14006bc70 tx=0x7fb13800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b590dc2e9c8e385f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:21.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.701+0000 7fb144ddf700 1 -- 172.21.15.104:0/3471898160 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13800e070 con 0x7fb140070ea0 2023-07-14T11:53:21.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.701+0000 7fb144ddf700 1 -- 172.21.15.104:0/3471898160 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13800b8d0 con 0x7fb140070ea0 2023-07-14T11:53:21.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.701+0000 7fb144ddf700 1 -- 172.21.15.104:0/3471898160 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13800bc80 con 0x7fb140070ea0 2023-07-14T11:53:21.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.701+0000 7fb148045700 1 -- 172.21.15.104:0/3471898160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140070ea0 msgr2=0x7fb1400afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:21.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.701+0000 7fb148045700 1 --2- 172.21.15.104:0/3471898160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140070ea0 0x7fb1400afd30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb14006bc70 tx=0x7fb13800b330 comp rx=0 tx=0).stop 2023-07-14T11:53:21.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.701+0000 7fb148045700 1 -- 172.21.15.104:0/3471898160 shutdown_connections 2023-07-14T11:53:21.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.701+0000 7fb148045700 1 --2- 172.21.15.104:0/3471898160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140070ea0 0x7fb1400afd30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.701+0000 7fb148045700 1 --2- 172.21.15.104:0/3471898160 >> v2:172.21.15.121:3300/0 conn(0x7fb1400cddb0 0x7fb140070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.701+0000 7fb148045700 1 -- 172.21.15.104:0/3471898160 >> 172.21.15.104:0/3471898160 conn(0x7fb1400cad30 msgr2=0x7fb1400cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:21.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.701+0000 7fb148045700 1 -- 172.21.15.104:0/3471898160 shutdown_connections 2023-07-14T11:53:21.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.701+0000 7fb148045700 1 -- 172.21.15.104:0/3471898160 wait complete. 2023-07-14T11:53:21.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb148045700 1 Processor -- start 2023-07-14T11:53:21.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb148045700 1 -- start start 2023-07-14T11:53:21.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb148045700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140070ea0 0x7fb140159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:21.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb148045700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1400cddb0 0x7fb14015bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:21.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb148045700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb140160640 0x7fb140147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:21.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb148045700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb140070540 con 0x7fb140070ea0 2023-07-14T11:53:21.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb148045700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14015aa90 con 0x7fb1400cddb0 2023-07-14T11:53:21.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb148045700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14015ad10 con 0x7fb140160640 2023-07-14T11:53:21.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb145de1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140070ea0 0x7fb140159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:21.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb145de1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140070ea0 0x7fb140159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46264/0 (socket says 172.21.15.104:46264) 2023-07-14T11:53:21.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb145de1700 1 -- 172.21.15.104:0/2184969221 learned_addr learned my addr 172.21.15.104:0/2184969221 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:21.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb1455e0700 1 --2- 172.21.15.104:0/2184969221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1400cddb0 0x7fb14015bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:21.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb1465e2700 1 --2- 172.21.15.104:0/2184969221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb140160640 0x7fb140147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:21.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb145de1700 1 -- 172.21.15.104:0/2184969221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb140160640 msgr2=0x7fb140147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:21.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb145de1700 1 --2- 172.21.15.104:0/2184969221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb140160640 0x7fb140147e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb145de1700 1 -- 172.21.15.104:0/2184969221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1400cddb0 msgr2=0x7fb14015bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:21.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb145de1700 1 --2- 172.21.15.104:0/2184969221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1400cddb0 0x7fb14015bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb145de1700 1 -- 172.21.15.104:0/2184969221 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb138009b30 con 0x7fb140070ea0 2023-07-14T11:53:21.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb145de1700 1 --2- 172.21.15.104:0/2184969221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140070ea0 0x7fb140159030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00c550 tx=0x7fb13c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:21.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb1367fc700 1 -- 172.21.15.104:0/2184969221 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c0054e0 con 0x7fb140070ea0 2023-07-14T11:53:21.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb1367fc700 1 -- 172.21.15.104:0/2184969221 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13c0036a0 con 0x7fb140070ea0 2023-07-14T11:53:21.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb1367fc700 1 -- 172.21.15.104:0/2184969221 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c003a10 con 0x7fb140070ea0 2023-07-14T11:53:21.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.705+0000 7fb148045700 1 -- 172.21.15.104:0/2184969221 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14015afd0 con 0x7fb140070ea0 2023-07-14T11:53:21.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.709+0000 7fb148045700 1 -- 172.21.15.104:0/2184969221 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14015b590 con 0x7fb140070ea0 2023-07-14T11:53:21.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.709+0000 7fb148045700 1 -- 172.21.15.104:0/2184969221 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb14004dcb0 con 0x7fb140070ea0 2023-07-14T11:53:21.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.709+0000 7fb1367fc700 1 -- 172.21.15.104:0/2184969221 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb13c003b70 con 0x7fb140070ea0 2023-07-14T11:53:21.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.713+0000 7fb1367fc700 1 --2- 172.21.15.104:0/2184969221 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1240662f0 0x7fb1240687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:21.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.713+0000 7fb1367fc700 1 -- 172.21.15.104:0/2184969221 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb13c083360 con 0x7fb140070ea0 2023-07-14T11:53:21.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.713+0000 7fb1455e0700 1 --2- 172.21.15.104:0/2184969221 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1240662f0 0x7fb1240687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:21.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.713+0000 7fb1455e0700 1 --2- 172.21.15.104:0/2184969221 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1240662f0 0x7fb1240687a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb140158790 tx=0x7fb130009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:21.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.717+0000 7fb1367fc700 1 -- 172.21.15.104:0/2184969221 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb13c050500 con 0x7fb140070ea0 2023-07-14T11:53:21.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.889+0000 7fb148045700 1 -- 172.21.15.104:0/2184969221 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1401646b0 con 0x7fb140070ea0 2023-07-14T11:53:21.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.893+0000 7fb1367fc700 1 -- 172.21.15.104:0/2184969221 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20156 (secure 0 0 0) 0x7fb13c053b20 con 0x7fb140070ea0 2023-07-14T11:53:21.879 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:21.879 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":28,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:20.091434+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:21.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.893+0000 7fb148045700 1 -- 172.21.15.104:0/2184969221 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1240662f0 msgr2=0x7fb1240687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:21.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.893+0000 7fb148045700 1 --2- 172.21.15.104:0/2184969221 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1240662f0 0x7fb1240687a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb140158790 tx=0x7fb130009240 comp rx=0 tx=0).stop 2023-07-14T11:53:21.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.893+0000 7fb148045700 1 -- 172.21.15.104:0/2184969221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140070ea0 msgr2=0x7fb140159030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:21.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.893+0000 7fb148045700 1 --2- 172.21.15.104:0/2184969221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140070ea0 0x7fb140159030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00c550 tx=0x7fb13c00c860 comp rx=0 tx=0).stop 2023-07-14T11:53:21.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.893+0000 7fb148045700 1 -- 172.21.15.104:0/2184969221 shutdown_connections 2023-07-14T11:53:21.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.893+0000 7fb148045700 1 --2- 172.21.15.104:0/2184969221 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1240662f0 0x7fb1240687a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.893+0000 7fb148045700 1 --2- 172.21.15.104:0/2184969221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb140160640 0x7fb140147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.893+0000 7fb148045700 1 --2- 172.21.15.104:0/2184969221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1400cddb0 0x7fb14015bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.893+0000 7fb148045700 1 --2- 172.21.15.104:0/2184969221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140070ea0 0x7fb140159030 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.893+0000 7fb148045700 1 -- 172.21.15.104:0/2184969221 >> 172.21.15.104:0/2184969221 conn(0x7fb1400cad30 msgr2=0x7fb14006c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:21.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.893+0000 7fb148045700 1 -- 172.21.15.104:0/2184969221 shutdown_connections 2023-07-14T11:53:21.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:21.893+0000 7fb148045700 1 -- 172.21.15.104:0/2184969221 wait complete. 2023-07-14T11:53:21.924 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:53:22.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.113+0000 7fad6c9c5700 1 Processor -- start 2023-07-14T11:53:22.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.113+0000 7fad6c9c5700 1 -- start start 2023-07-14T11:53:22.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.113+0000 7fad6c9c5700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad680b0de0 0x7fad681669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:22.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.113+0000 7fad6c9c5700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad680af0a0 con 0x7fad6815bff0 2023-07-14T11:53:22.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.113+0000 7fad6c9c5700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad680cf720 con 0x7fad680cd240 2023-07-14T11:53:22.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.113+0000 7fad6c9c5700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad680af2f0 con 0x7fad680b0de0 2023-07-14T11:53:22.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.113+0000 7fad66d9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fad6815bff0 0x7fad680ccb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:51136/0 (socket says 172.21.15.104:51136) 2023-07-14T11:53:22.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.113+0000 7fad66d9d700 1 -- 172.21.15.104:0/2471125860 learned_addr learned my addr 172.21.15.104:0/2471125860 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:22.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.113+0000 7fad6759e700 1 --2- 172.21.15.104:0/2471125860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad680b0de0 0x7fad681669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:22.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.113+0000 7fad65d9b700 1 -- 172.21.15.104:0/2471125860 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884176552 0 0) 0x7fad680af0a0 con 0x7fad6815bff0 2023-07-14T11:53:22.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.113+0000 7fad65d9b700 1 -- 172.21.15.104:0/2471125860 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad500035d0 con 0x7fad6815bff0 2023-07-14T11:53:22.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.113+0000 7fad65d9b700 1 -- 172.21.15.104:0/2471125860 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103051188 0 0) 0x7fad680cf720 con 0x7fad680cd240 2023-07-14T11:53:22.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.113+0000 7fad65d9b700 1 -- 172.21.15.104:0/2471125860 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad680af0a0 con 0x7fad680cd240 2023-07-14T11:53:22.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.117+0000 7fad6759e700 1 -- 172.21.15.104:0/2471125860 >> v1:172.21.15.121:6789/0 conn(0x7fad680cd240 legacy=0x7fad680ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:22.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.117+0000 7fad65d9b700 1 -- 172.21.15.104:0/2471125860 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2809796047 0 0) 0x7fad500035d0 con 0x7fad6815bff0 2023-07-14T11:53:22.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.117+0000 7fad6759e700 1 -- 172.21.15.104:0/2471125860 >> v1:172.21.15.104:6789/0 conn(0x7fad6815bff0 legacy=0x7fad680ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:22.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.117+0000 7fad6759e700 1 -- 172.21.15.104:0/2471125860 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad680af5d0 con 0x7fad680b0de0 2023-07-14T11:53:22.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.117+0000 7fad6759e700 1 --2- 172.21.15.104:0/2471125860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad680b0de0 0x7fad681669d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad60009dc0 tx=0x7fad6000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15b990f43cd15c1d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:22.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.117+0000 7fad65d9b700 1 -- 172.21.15.104:0/2471125860 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6000e070 con 0x7fad680b0de0 2023-07-14T11:53:22.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.117+0000 7fad65d9b700 1 -- 172.21.15.104:0/2471125860 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad60003a40 con 0x7fad680b0de0 2023-07-14T11:53:22.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.117+0000 7fad65d9b700 1 -- 172.21.15.104:0/2471125860 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad58001670 con 0x7fad680b0de0 2023-07-14T11:53:22.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.117+0000 7fad6c9c5700 1 -- 172.21.15.104:0/2471125860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad680b0de0 msgr2=0x7fad681669d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:22.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.117+0000 7fad6c9c5700 1 --2- 172.21.15.104:0/2471125860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad680b0de0 0x7fad681669d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad60009dc0 tx=0x7fad6000b330 comp rx=0 tx=0).stop 2023-07-14T11:53:22.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.117+0000 7fad6c9c5700 1 -- 172.21.15.104:0/2471125860 shutdown_connections 2023-07-14T11:53:22.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.117+0000 7fad6c9c5700 1 --2- 172.21.15.104:0/2471125860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad680b0de0 0x7fad681669d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.117+0000 7fad6c9c5700 1 -- 172.21.15.104:0/2471125860 >> 172.21.15.104:0/2471125860 conn(0x7fad68060c40 msgr2=0x7fad6809b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:22.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.117+0000 7fad6c9c5700 1 -- 172.21.15.104:0/2471125860 shutdown_connections 2023-07-14T11:53:22.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.117+0000 7fad6c9c5700 1 -- 172.21.15.104:0/2471125860 wait complete. 2023-07-14T11:53:22.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.121+0000 7fad6c9c5700 1 Processor -- start 2023-07-14T11:53:22.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.121+0000 7fad6c9c5700 1 -- start start 2023-07-14T11:53:22.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad6c9c5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad680b0de0 0x7fad6815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:22.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad6c9c5700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad680cd240 0x7fad68159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:22.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad6c9c5700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad6815bff0 0x7fad6815e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:22.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad6c9c5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6815d2b0 con 0x7fad680b0de0 2023-07-14T11:53:22.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad6c9c5700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6815a350 con 0x7fad680cd240 2023-07-14T11:53:22.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad6c9c5700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad681442e0 con 0x7fad6815bff0 2023-07-14T11:53:22.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad66d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad680b0de0 0x7fad6815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:22.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad66d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad680b0de0 0x7fad6815c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46274/0 (socket says 172.21.15.104:46274) 2023-07-14T11:53:22.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad66d9d700 1 -- 172.21.15.104:0/2305807643 learned_addr learned my addr 172.21.15.104:0/2305807643 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:22.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad6759e700 1 --2- 172.21.15.104:0/2305807643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad6815bff0 0x7fad6815e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:22.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad6659c700 1 --2- 172.21.15.104:0/2305807643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad680cd240 0x7fad68159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:22.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad66d9d700 1 -- 172.21.15.104:0/2305807643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad6815bff0 msgr2=0x7fad6815e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:22.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad66d9d700 1 --2- 172.21.15.104:0/2305807643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad6815bff0 0x7fad6815e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad66d9d700 1 -- 172.21.15.104:0/2305807643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad680cd240 msgr2=0x7fad68159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:22.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad66d9d700 1 --2- 172.21.15.104:0/2305807643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad680cd240 0x7fad68159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad66d9d700 1 -- 172.21.15.104:0/2305807643 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad60009a70 con 0x7fad680b0de0 2023-07-14T11:53:22.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad66d9d700 1 --2- 172.21.15.104:0/2305807643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad680b0de0 0x7fad6815c3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fad5800c730 tx=0x7fad5800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:22.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad4f7fe700 1 -- 172.21.15.104:0/2305807643 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad58004ca0 con 0x7fad680b0de0 2023-07-14T11:53:22.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad4f7fe700 1 -- 172.21.15.104:0/2305807643 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad58004e00 con 0x7fad680b0de0 2023-07-14T11:53:22.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad4f7fe700 1 -- 172.21.15.104:0/2305807643 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad58007310 con 0x7fad680b0de0 2023-07-14T11:53:22.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad6c9c5700 1 -- 172.21.15.104:0/2305807643 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6815ca10 con 0x7fad680b0de0 2023-07-14T11:53:22.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.125+0000 7fad6c9c5700 1 -- 172.21.15.104:0/2305807643 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6815cfd0 con 0x7fad680b0de0 2023-07-14T11:53:22.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.129+0000 7fad6c9c5700 1 -- 172.21.15.104:0/2305807643 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad6804e500 con 0x7fad680b0de0 2023-07-14T11:53:22.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.129+0000 7fad4f7fe700 1 -- 172.21.15.104:0/2305807643 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fad58007470 con 0x7fad680b0de0 2023-07-14T11:53:22.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.133+0000 7fad4f7fe700 1 --2- 172.21.15.104:0/2305807643 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fad540662f0 0x7fad540687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:22.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.133+0000 7fad4f7fe700 1 -- 172.21.15.104:0/2305807643 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fad58083cb0 con 0x7fad680b0de0 2023-07-14T11:53:22.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.133+0000 7fad6659c700 1 --2- 172.21.15.104:0/2305807643 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fad540662f0 0x7fad540687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:22.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.133+0000 7fad6659c700 1 --2- 172.21.15.104:0/2305807643 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fad540662f0 0x7fad540687a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fad680d17b0 tx=0x7fad5c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:22.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.137+0000 7fad4f7fe700 1 -- 172.21.15.104:0/2305807643 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fad580548d0 con 0x7fad680b0de0 2023-07-14T11:53:22.333 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.345+0000 7fad6c9c5700 1 -- 172.21.15.104:0/2305807643 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad680cce90 con 0x7fad680b0de0 2023-07-14T11:53:22.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.345+0000 7fad4f7fe700 1 -- 172.21.15.104:0/2305807643 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4717 (secure 0 0 0) 0x7fad58054460 con 0x7fad680b0de0 2023-07-14T11:53:22.334 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:22.334 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2103358277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2103358277},{"type":"v1","addr":"172.21.15.121:6833","nonce":2103358277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1841065287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1841065287},{"type":"v1","addr":"172.21.15.121:6835","nonce":1841065287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3342053518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3342053518},{"type":"v1","addr":"172.21.15.104:6835","nonce":3342053518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:53:15.029045+0000","modified":"2023-07-14T11:53:20.652622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/2666329866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2666329866},{"type":"v1","addr":"172.21.15.104:6837","nonce":2666329866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-14T11:53:22.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.349+0000 7fad6c9c5700 1 -- 172.21.15.104:0/2305807643 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fad540662f0 msgr2=0x7fad540687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:22.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.349+0000 7fad6c9c5700 1 --2- 172.21.15.104:0/2305807643 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fad540662f0 0x7fad540687a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fad680d17b0 tx=0x7fad5c009040 comp rx=0 tx=0).stop 2023-07-14T11:53:22.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.349+0000 7fad6c9c5700 1 -- 172.21.15.104:0/2305807643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad680b0de0 msgr2=0x7fad6815c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:22.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.349+0000 7fad6c9c5700 1 --2- 172.21.15.104:0/2305807643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad680b0de0 0x7fad6815c3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fad5800c730 tx=0x7fad5800ca40 comp rx=0 tx=0).stop 2023-07-14T11:53:22.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.349+0000 7fad6c9c5700 1 -- 172.21.15.104:0/2305807643 shutdown_connections 2023-07-14T11:53:22.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.349+0000 7fad6c9c5700 1 --2- 172.21.15.104:0/2305807643 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fad540662f0 0x7fad540687a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.349+0000 7fad6c9c5700 1 --2- 172.21.15.104:0/2305807643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fad6815bff0 0x7fad6815e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.349+0000 7fad6c9c5700 1 --2- 172.21.15.104:0/2305807643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad680cd240 0x7fad68159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.349+0000 7fad6c9c5700 1 --2- 172.21.15.104:0/2305807643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad680b0de0 0x7fad6815c3c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.349+0000 7fad6c9c5700 1 -- 172.21.15.104:0/2305807643 >> 172.21.15.104:0/2305807643 conn(0x7fad68060c40 msgr2=0x7fad680d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:22.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.349+0000 7fad6c9c5700 1 -- 172.21.15.104:0/2305807643 shutdown_connections 2023-07-14T11:53:22.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.349+0000 7fad6c9c5700 1 -- 172.21.15.104:0/2305807643 wait complete. 2023-07-14T11:53:22.339 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 7 2023-07-14T11:53:22.355 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:22.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.545+0000 7f742f30e700 1 Processor -- start 2023-07-14T11:53:22.532 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.545+0000 7f742f30e700 1 -- start start 2023-07-14T11:53:22.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.545+0000 7f742f30e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f742815bff0 0x7f74280ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:22.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.545+0000 7f742f30e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74280cd070 0x7f74280ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:22.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742f30e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f74280ceff0 0x7f74280d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:22.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742f30e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74280cd4c0 con 0x7f74280ceff0 2023-07-14T11:53:22.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742f30e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74280af0a0 con 0x7f742815bff0 2023-07-14T11:53:22.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742f30e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74280cf470 con 0x7f74280cd070 2023-07-14T11:53:22.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742eb0d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f74280ceff0 0x7f74280d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:22.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742eb0d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f74280ceff0 0x7f74280d0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46286/0 (socket says 172.21.15.104:46286) 2023-07-14T11:53:22.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742eb0d700 1 -- 172.21.15.104:0/851238324 learned_addr learned my addr 172.21.15.104:0/851238324 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:22.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742eb0d700 1 -- 172.21.15.104:0/851238324 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74280cd070 msgr2=0x7f74280ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:22.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742db0b700 1 --2- 172.21.15.104:0/851238324 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74280cd070 0x7f74280ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:22.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742e30c700 1 --2- 172.21.15.104:0/851238324 >> v2:172.21.15.121:3300/0 conn(0x7f742815bff0 0x7f74280ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:22.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742eb0d700 1 --2- 172.21.15.104:0/851238324 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74280cd070 0x7f74280ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742eb0d700 1 -- 172.21.15.104:0/851238324 >> v2:172.21.15.121:3300/0 conn(0x7f742815bff0 msgr2=0x7f74280ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:22.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742eb0d700 1 --2- 172.21.15.104:0/851238324 >> v2:172.21.15.121:3300/0 conn(0x7f742815bff0 0x7f74280ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742eb0d700 1 -- 172.21.15.104:0/851238324 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74280cf720 con 0x7f74280ceff0 2023-07-14T11:53:22.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742e30c700 1 --2- 172.21.15.104:0/851238324 >> v2:172.21.15.121:3300/0 conn(0x7f742815bff0 0x7f74280ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:22.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742db0b700 1 --2- 172.21.15.104:0/851238324 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74280cd070 0x7f74280ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:22.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742eb0d700 1 --2- 172.21.15.104:0/851238324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74280ceff0 0x7f74280d0020 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f742808ac70 tx=0x7f742000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992209682ea99a54 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:22.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742d30a700 1 -- 172.21.15.104:0/851238324 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742000e070 con 0x7f74280ceff0 2023-07-14T11:53:22.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742d30a700 1 -- 172.21.15.104:0/851238324 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742000b9c0 con 0x7f74280ceff0 2023-07-14T11:53:22.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742d30a700 1 -- 172.21.15.104:0/851238324 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742000bd70 con 0x7f74280ceff0 2023-07-14T11:53:22.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742f30e700 1 -- 172.21.15.104:0/851238324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74280ceff0 msgr2=0x7f74280d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:22.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742f30e700 1 --2- 172.21.15.104:0/851238324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74280ceff0 0x7f74280d0020 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f742808ac70 tx=0x7f742000b330 comp rx=0 tx=0).stop 2023-07-14T11:53:22.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742f30e700 1 -- 172.21.15.104:0/851238324 shutdown_connections 2023-07-14T11:53:22.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742f30e700 1 --2- 172.21.15.104:0/851238324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74280ceff0 0x7f74280d0020 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742f30e700 1 --2- 172.21.15.104:0/851238324 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74280cd070 0x7f74280ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742f30e700 1 --2- 172.21.15.104:0/851238324 >> v2:172.21.15.121:3300/0 conn(0x7f742815bff0 0x7f74280ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742f30e700 1 -- 172.21.15.104:0/851238324 >> 172.21.15.104:0/851238324 conn(0x7f7428060c40 msgr2=0x7f742809b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:22.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742f30e700 1 -- 172.21.15.104:0/851238324 shutdown_connections 2023-07-14T11:53:22.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.549+0000 7f742f30e700 1 -- 172.21.15.104:0/851238324 wait complete. 2023-07-14T11:53:22.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.553+0000 7f742f30e700 1 Processor -- start 2023-07-14T11:53:22.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.553+0000 7f742f30e700 1 -- start start 2023-07-14T11:53:22.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.553+0000 7f742f30e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74280cd070 0x7f742815ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:22.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.553+0000 7f742f30e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74280ceff0 0x7f74280c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:22.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.553+0000 7f742f30e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f742815bff0 0x7f74280c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:22.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.553+0000 7f742f30e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742815cfc0 con 0x7f74280cd070 2023-07-14T11:53:22.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.553+0000 7f742f30e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742815f250 con 0x7f742815bff0 2023-07-14T11:53:22.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.553+0000 7f742f30e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74280b53b0 con 0x7f74280ceff0 2023-07-14T11:53:22.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.553+0000 7f742e30c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74280cd070 0x7f742815ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:22.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.553+0000 7f742e30c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74280cd070 0x7f742815ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46292/0 (socket says 172.21.15.104:46292) 2023-07-14T11:53:22.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.553+0000 7f742e30c700 1 -- 172.21.15.104:0/2728017603 learned_addr learned my addr 172.21.15.104:0/2728017603 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:22.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.553+0000 7f742eb0d700 1 --2- 172.21.15.104:0/2728017603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f742815bff0 0x7f74280c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:22.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.553+0000 7f742db0b700 1 --2- 172.21.15.104:0/2728017603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74280ceff0 0x7f74280c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:22.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.557+0000 7f742e30c700 1 -- 172.21.15.104:0/2728017603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74280ceff0 msgr2=0x7f74280c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:22.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.557+0000 7f742e30c700 1 --2- 172.21.15.104:0/2728017603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74280ceff0 0x7f74280c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.557+0000 7f742e30c700 1 -- 172.21.15.104:0/2728017603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f742815bff0 msgr2=0x7f74280c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:22.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.557+0000 7f742e30c700 1 --2- 172.21.15.104:0/2728017603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f742815bff0 0x7f74280c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.557+0000 7f742e30c700 1 -- 172.21.15.104:0/2728017603 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7420009b30 con 0x7f74280cd070 2023-07-14T11:53:22.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.557+0000 7f742e30c700 1 --2- 172.21.15.104:0/2728017603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74280cd070 0x7f742815ca80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f742400c7c0 tx=0x7f742400cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:22.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.557+0000 7f741effd700 1 -- 172.21.15.104:0/2728017603 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424011800 con 0x7f74280cd070 2023-07-14T11:53:22.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.557+0000 7f741effd700 1 -- 172.21.15.104:0/2728017603 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7424011960 con 0x7f74280cd070 2023-07-14T11:53:22.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.557+0000 7f741effd700 1 -- 172.21.15.104:0/2728017603 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424011cf0 con 0x7f74280cd070 2023-07-14T11:53:22.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.557+0000 7f742f30e700 1 -- 172.21.15.104:0/2728017603 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74280b5690 con 0x7f74280cd070 2023-07-14T11:53:22.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.557+0000 7f742f30e700 1 -- 172.21.15.104:0/2728017603 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74280b5c50 con 0x7f74280cd070 2023-07-14T11:53:22.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.557+0000 7f742f30e700 1 -- 172.21.15.104:0/2728017603 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742804e500 con 0x7f74280cd070 2023-07-14T11:53:22.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.557+0000 7f741effd700 1 -- 172.21.15.104:0/2728017603 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7424021350 con 0x7f74280cd070 2023-07-14T11:53:22.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.561+0000 7f741effd700 1 --2- 172.21.15.104:0/2728017603 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f740c0662f0 0x7f740c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:22.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.561+0000 7f741effd700 1 -- 172.21.15.104:0/2728017603 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7424083e40 con 0x7f74280cd070 2023-07-14T11:53:22.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.561+0000 7f742db0b700 1 --2- 172.21.15.104:0/2728017603 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f740c0662f0 0x7f740c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:22.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.561+0000 7f742db0b700 1 --2- 172.21.15.104:0/2728017603 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f740c0662f0 0x7f740c0687a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f74280cce50 tx=0x7f7418008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:22.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.565+0000 7f741effd700 1 -- 172.21.15.104:0/2728017603 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f7424054a70 con 0x7f74280cd070 2023-07-14T11:53:22.729 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.741+0000 7f742f30e700 1 -- 172.21.15.104:0/2728017603 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74280cc8a0 con 0x7f74280cd070 2023-07-14T11:53:22.732 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.745+0000 7f741effd700 1 -- 172.21.15.104:0/2728017603 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20156 (secure 0 0 0) 0x7f7424054600 con 0x7f74280cd070 2023-07-14T11:53:22.733 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:22.733 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":28,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:20.091434+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:22.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.749+0000 7f742f30e700 1 -- 172.21.15.104:0/2728017603 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f740c0662f0 msgr2=0x7f740c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:22.736 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.749+0000 7f742f30e700 1 --2- 172.21.15.104:0/2728017603 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f740c0662f0 0x7f740c0687a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f74280cce50 tx=0x7f7418008040 comp rx=0 tx=0).stop 2023-07-14T11:53:22.736 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.749+0000 7f742f30e700 1 -- 172.21.15.104:0/2728017603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74280cd070 msgr2=0x7f742815ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:22.736 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.749+0000 7f742f30e700 1 --2- 172.21.15.104:0/2728017603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74280cd070 0x7f742815ca80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f742400c7c0 tx=0x7f742400cad0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.749+0000 7f742f30e700 1 -- 172.21.15.104:0/2728017603 shutdown_connections 2023-07-14T11:53:22.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.749+0000 7f742f30e700 1 --2- 172.21.15.104:0/2728017603 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f740c0662f0 0x7f740c0687a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.749+0000 7f742f30e700 1 --2- 172.21.15.104:0/2728017603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f742815bff0 0x7f74280c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.749+0000 7f742f30e700 1 --2- 172.21.15.104:0/2728017603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74280ceff0 0x7f74280c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.749+0000 7f742f30e700 1 --2- 172.21.15.104:0/2728017603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74280cd070 0x7f742815ca80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.749+0000 7f742f30e700 1 -- 172.21.15.104:0/2728017603 >> 172.21.15.104:0/2728017603 conn(0x7f7428060c40 msgr2=0x7f7428071180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:22.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.749+0000 7f742f30e700 1 -- 172.21.15.104:0/2728017603 shutdown_connections 2023-07-14T11:53:22.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:22.749+0000 7f742f30e700 1 -- 172.21.15.104:0/2728017603 wait complete. 2023-07-14T11:53:23.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-14T11:53:15.029045+0000', 'modified': '2023-07-14T11:53:20.652622+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4275}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4275': {'gid': 4275, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6837/2666329866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 2666329866}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 2666329866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-07-14T11:53:23.159 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-14T11:53:23.160 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-14T11:53:23.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.357+0000 7f2692f24700 1 Processor -- start 2023-07-14T11:53:23.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.357+0000 7f2692f24700 1 -- start start 2023-07-14T11:53:23.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.357+0000 7f2692f24700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f268c1599d0 0x7f268c071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.357+0000 7f2692f24700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f268c15ccf0 0x7f268c0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.357+0000 7f2692f24700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f268c0caa20 0x7f268c0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.357+0000 7f2692f24700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c0cde10 con 0x7f268c1599d0 2023-07-14T11:53:23.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.357+0000 7f2692f24700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c0cb3c0 con 0x7f268c0caa20 2023-07-14T11:53:23.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.357+0000 7f2692f24700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0c8bf0 con 0x7f268c15ccf0 2023-07-14T11:53:23.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2692723700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f268c0caa20 0x7f268c0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2691721700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f268c15ccf0 0x7f268c0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2691721700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f268c15ccf0 0x7f268c0c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:48498/0 (socket says 172.21.15.104:48498) 2023-07-14T11:53:23.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2691721700 1 -- 172.21.15.104:0/2505468271 learned_addr learned my addr 172.21.15.104:0/2505468271 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:23.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2691f22700 1 --2- 172.21.15.104:0/2505468271 >> v2:172.21.15.104:3300/0 conn(0x7f268c1599d0 0x7f268c071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2691721700 1 -- 172.21.15.104:0/2505468271 >> v2:172.21.15.121:3300/0 conn(0x7f268c0caa20 msgr2=0x7f268c0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:23.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2691721700 1 --2- 172.21.15.104:0/2505468271 >> v2:172.21.15.121:3300/0 conn(0x7f268c0caa20 0x7f268c0cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2691721700 1 -- 172.21.15.104:0/2505468271 >> v2:172.21.15.104:3300/0 conn(0x7f268c1599d0 msgr2=0x7f268c071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:23.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2691721700 1 --2- 172.21.15.104:0/2505468271 >> v2:172.21.15.104:3300/0 conn(0x7f268c1599d0 0x7f268c071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2691721700 1 -- 172.21.15.104:0/2505468271 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c0cd520 con 0x7f268c15ccf0 2023-07-14T11:53:23.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2691721700 1 --2- 172.21.15.104:0/2505468271 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f268c15ccf0 0x7f268c0c86b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f268c0720b0 tx=0x7f267c009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a64ef77c7b7af745 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:23.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2690f20700 1 -- 172.21.15.104:0/2505468271 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c011830 con 0x7f268c15ccf0 2023-07-14T11:53:23.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2690f20700 1 -- 172.21.15.104:0/2505468271 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c011990 con 0x7f268c15ccf0 2023-07-14T11:53:23.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2690f20700 1 -- 172.21.15.104:0/2505468271 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c011d20 con 0x7f268c15ccf0 2023-07-14T11:53:23.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2692f24700 1 -- 172.21.15.104:0/2505468271 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f268c15ccf0 msgr2=0x7f268c0c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:23.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2692f24700 1 --2- 172.21.15.104:0/2505468271 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f268c15ccf0 0x7f268c0c86b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f268c0720b0 tx=0x7f267c009bf0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2692f24700 1 -- 172.21.15.104:0/2505468271 shutdown_connections 2023-07-14T11:53:23.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2692f24700 1 --2- 172.21.15.104:0/2505468271 >> v2:172.21.15.121:3300/0 conn(0x7f268c0caa20 0x7f268c0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2692f24700 1 --2- 172.21.15.104:0/2505468271 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f268c15ccf0 0x7f268c0c86b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2692f24700 1 --2- 172.21.15.104:0/2505468271 >> v2:172.21.15.104:3300/0 conn(0x7f268c1599d0 0x7f268c071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2692f24700 1 -- 172.21.15.104:0/2505468271 >> 172.21.15.104:0/2505468271 conn(0x7f268c06e960 msgr2=0x7f268c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:23.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2692f24700 1 -- 172.21.15.104:0/2505468271 shutdown_connections 2023-07-14T11:53:23.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.361+0000 7f2692f24700 1 -- 172.21.15.104:0/2505468271 wait complete. 2023-07-14T11:53:23.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.365+0000 7f2692f24700 1 Processor -- start 2023-07-14T11:53:23.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.365+0000 7f2692f24700 1 -- start start 2023-07-14T11:53:23.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.365+0000 7f2692f24700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f268c0caa20 0x7f268c1f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.365+0000 7f2692f24700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f268c1599d0 0x7f268c0b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.365+0000 7f2692f24700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f268c15ccf0 0x7f268c1531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.365+0000 7f2692f24700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0aeab0 con 0x7f268c1599d0 2023-07-14T11:53:23.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.365+0000 7f2692f24700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c1f5e60 con 0x7f268c15ccf0 2023-07-14T11:53:23.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.365+0000 7f2692f24700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0b36a0 con 0x7f268c0caa20 2023-07-14T11:53:23.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.365+0000 7f2691721700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f268c1599d0 0x7f268c0b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.365+0000 7f2691721700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f268c1599d0 0x7f268c0b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46310/0 (socket says 172.21.15.104:46310) 2023-07-14T11:53:23.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.365+0000 7f2691721700 1 -- 172.21.15.104:0/1245354368 learned_addr learned my addr 172.21.15.104:0/1245354368 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:23.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.365+0000 7f2692723700 1 --2- 172.21.15.104:0/1245354368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f268c15ccf0 0x7f268c1531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.365+0000 7f2691f22700 1 --2- 172.21.15.104:0/1245354368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f268c0caa20 0x7f268c1f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.369+0000 7f2691721700 1 -- 172.21.15.104:0/1245354368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f268c0caa20 msgr2=0x7f268c1f5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:23.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.369+0000 7f2691721700 1 --2- 172.21.15.104:0/1245354368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f268c0caa20 0x7f268c1f5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.369+0000 7f2691721700 1 -- 172.21.15.104:0/1245354368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f268c15ccf0 msgr2=0x7f268c1531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:23.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.369+0000 7f2691721700 1 --2- 172.21.15.104:0/1245354368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f268c15ccf0 0x7f268c1531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.369+0000 7f2691721700 1 -- 172.21.15.104:0/1245354368 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267c009570 con 0x7f268c1599d0 2023-07-14T11:53:23.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.369+0000 7f2691721700 1 --2- 172.21.15.104:0/1245354368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f268c1599d0 0x7f268c0b1640 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f267c009f40 tx=0x7f267c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:23.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.369+0000 7f26827fc700 1 -- 172.21.15.104:0/1245354368 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c004240 con 0x7f268c1599d0 2023-07-14T11:53:23.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.369+0000 7f26827fc700 1 -- 172.21.15.104:0/1245354368 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c01f070 con 0x7f268c1599d0 2023-07-14T11:53:23.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.369+0000 7f2692f24700 1 -- 172.21.15.104:0/1245354368 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268c0b3980 con 0x7f268c1599d0 2023-07-14T11:53:23.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.369+0000 7f26827fc700 1 -- 172.21.15.104:0/1245354368 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c004600 con 0x7f268c1599d0 2023-07-14T11:53:23.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.369+0000 7f2692f24700 1 -- 172.21.15.104:0/1245354368 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268c0aed00 con 0x7f268c1599d0 2023-07-14T11:53:23.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.373+0000 7f26827fc700 1 -- 172.21.15.104:0/1245354368 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f267c021030 con 0x7f268c1599d0 2023-07-14T11:53:23.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.373+0000 7f26827fc700 1 --2- 172.21.15.104:0/1245354368 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f26700662f0 0x7f26700687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.373+0000 7f26827fc700 1 -- 172.21.15.104:0/1245354368 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f267c084fa0 con 0x7f268c1599d0 2023-07-14T11:53:23.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.373+0000 7f2691f22700 1 --2- 172.21.15.104:0/1245354368 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f26700662f0 0x7f26700687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.373+0000 7f2691f22700 1 --2- 172.21.15.104:0/1245354368 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f26700662f0 0x7f26700687a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f268c0b32f0 tx=0x7f2688009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:23.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.377+0000 7f269740b740 1 Processor -- start 2023-07-14T11:53:23.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.377+0000 7f269740b740 1 -- start start 2023-07-14T11:53:23.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.377+0000 7f269740b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447eb0 0x17008b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.377+0000 7f269740b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1447ae0 0x1700df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.377+0000 7f269740b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1447710 0x16b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.377+0000 7f269740b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x15d4270 con 0x1447eb0 2023-07-14T11:53:23.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.377+0000 7f269740b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x16273a0 con 0x1447ae0 2023-07-14T11:53:23.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.377+0000 7f269740b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x1620600 con 0x1447710 2023-07-14T11:53:23.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.377+0000 7f2691f22700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447eb0 0x17008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.377+0000 7f2691f22700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447eb0 0x17008b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46324/0 (socket says 172.21.15.104:46324) 2023-07-14T11:53:23.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.377+0000 7f2691f22700 1 -- 172.21.15.104:0/115733884 learned_addr learned my addr 172.21.15.104:0/115733884 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:23.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f2691f22700 1 -- 172.21.15.104:0/115733884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1447710 msgr2=0x16b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:23.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f2691f22700 1 --2- 172.21.15.104:0/115733884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1447710 0x16b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f2691f22700 1 -- 172.21.15.104:0/115733884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1447ae0 msgr2=0x1700df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:23.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f2691f22700 1 --2- 172.21.15.104:0/115733884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1447ae0 0x1700df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f2691f22700 1 -- 172.21.15.104:0/115733884 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x14939e0 con 0x1447eb0 2023-07-14T11:53:23.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f2691f22700 1 --2- 172.21.15.104:0/115733884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447eb0 0x17008b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2688011c50 tx=0x7f2688009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f453549ec4679fb7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:23.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f266f7fe700 1 -- 172.21.15.104:0/115733884 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688014a30 con 0x1447eb0 2023-07-14T11:53:23.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f266f7fe700 1 -- 172.21.15.104:0/115733884 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2688014b90 con 0x1447eb0 2023-07-14T11:53:23.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f266f7fe700 1 -- 172.21.15.104:0/115733884 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688004030 con 0x1447eb0 2023-07-14T11:53:23.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f269740b740 1 -- 172.21.15.104:0/115733884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447eb0 msgr2=0x17008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:23.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f269740b740 1 --2- 172.21.15.104:0/115733884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447eb0 0x17008b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2688011c50 tx=0x7f2688009f90 comp rx=0 tx=0).stop 2023-07-14T11:53:23.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f269740b740 1 -- 172.21.15.104:0/115733884 shutdown_connections 2023-07-14T11:53:23.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f269740b740 1 --2- 172.21.15.104:0/115733884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1447710 0x16b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f269740b740 1 --2- 172.21.15.104:0/115733884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1447ae0 0x1700df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f269740b740 1 --2- 172.21.15.104:0/115733884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447eb0 0x17008b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f269740b740 1 -- 172.21.15.104:0/115733884 >> 172.21.15.104:0/115733884 conn(0x16b1b60 msgr2=0x16bc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:23.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f269740b740 1 -- 172.21.15.104:0/115733884 shutdown_connections 2023-07-14T11:53:23.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f269740b740 1 -- 172.21.15.104:0/115733884 wait complete. 2023-07-14T11:53:23.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f269740b740 1 -- start start 2023-07-14T11:53:23.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.381+0000 7f269740b740 1 Processor -- start 2023-07-14T11:53:23.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f269740b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1447710 0x16e9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f269740b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f269740b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1447eb0 0x170a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f269740b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x15e2b00 con 0x1447ae0 2023-07-14T11:53:23.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f269740b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x15e1470 con 0x1447eb0 2023-07-14T11:53:23.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f269740b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x14ccb20 con 0x1447710 2023-07-14T11:53:23.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f269740b740 10 client.0 _ll_register_callbacks cb 0x1646530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:53:23.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f2691721700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f2691721700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46330/0 (socket says 172.21.15.104:46330) 2023-07-14T11:53:23.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f2691721700 1 -- 172.21.15.104:0/2847537722 learned_addr learned my addr 172.21.15.104:0/2847537722 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:23.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f2692723700 1 --2- 172.21.15.104:0/2847537722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1447eb0 0x170a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f2691f22700 1 --2- 172.21.15.104:0/2847537722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1447710 0x16e9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f2691721700 1 -- 172.21.15.104:0/2847537722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1447710 msgr2=0x16e9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:23.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f2691721700 1 --2- 172.21.15.104:0/2847537722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1447710 0x16e9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f2691721700 1 -- 172.21.15.104:0/2847537722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1447eb0 msgr2=0x170a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:23.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f2691721700 1 --2- 172.21.15.104:0/2847537722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1447eb0 0x170a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f2691721700 1 -- 172.21.15.104:0/2847537722 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2688011900 con 0x1447ae0 2023-07-14T11:53:23.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f2691721700 1 --2- 172.21.15.104:0/2847537722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e8c60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f267c052440 tx=0x7f267c059f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff29d6d9ab0d80e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:23.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f266cff9700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-14T11:53:23.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f266cff9700 1 -- 172.21.15.104:0/2847537722 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c055630 con 0x1447ae0 2023-07-14T11:53:23.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f266cff9700 1 -- 172.21.15.104:0/2847537722 <== mon.0 v2:172.21.15.104:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f267c05de00 con 0x1447ae0 2023-07-14T11:53:23.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f269740b740 1 -- 172.21.15.104:0/2847537722 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x16195e0 con 0x1447ae0 2023-07-14T11:53:23.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f266cff9700 1 -- 172.21.15.104:0/2847537722 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f267c05e020 con 0x1447ae0 2023-07-14T11:53:23.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f269740b740 10 client.4446 fetch_fsmap learned FSMap version 7 2023-07-14T11:53:23.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f269740b740 1 -- 172.21.15.104:0/2847537722 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x15e6250 con 0x1447ae0 2023-07-14T11:53:23.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f266cff9700 1 -- 172.21.15.104:0/2847537722 <== mon.0 v2:172.21.15.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c076070 con 0x1447ae0 2023-07-14T11:53:23.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f266cff9700 1 -- 172.21.15.104:0/2847537722 <== mon.0 v2:172.21.15.104:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f267c022a60 con 0x1447ae0 2023-07-14T11:53:23.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f269740b740 10 client.4446 fetch_fsmap finished waiting for FSMap version 7 2023-07-14T11:53:23.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.385+0000 7f269740b740 10 client.4446 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-14T11:53:23.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f269740b740 20 client.4446 populate_metadata read hostname 'smithi104' 2023-07-14T11:53:23.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f269740b740 1 --2- 172.21.15.104:0/2847537722 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0x1492790 0x16e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f269740b740 4 client.4446 mds_command: new command op to 4275 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:53:23.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f269740b740 1 -- 172.21.15.104:0/2847537722 --> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1619300 con 0x1492790 2023-07-14T11:53:23.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f2691f22700 1 --2- 172.21.15.104:0/2847537722 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0x1492790 0x16e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f2691f22700 1 --2- 172.21.15.104:0/2847537722 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0x1492790 0x16e6b70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b5ddfc0f2b68d8 server_cookie=524f1c964680a208 in_seq=0 out_seq=0 2023-07-14T11:53:23.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f266cff9700 10 client.4446 ms_handle_connect on v2:172.21.15.104:6836/2666329866 2023-07-14T11:53:23.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f266cff9700 1 -- 172.21.15.104:0/2847537722 <== mds.0 v2:172.21.15.104:6836/2666329866 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x1619300 con 0x1492790 2023-07-14T11:53:23.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f266cff9700 10 client.4446 handle_command_reply: tid=0 2023-07-14T11:53:23.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f269740b740 1 client.4446 shutdown 2023-07-14T11:53:23.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f269740b740 1 -- 172.21.15.104:0/2847537722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 msgr2=0x16e8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:23.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f269740b740 1 --2- 172.21.15.104:0/2847537722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e8c60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f267c052440 tx=0x7f267c059f00 comp rx=0 tx=0).stop 2023-07-14T11:53:23.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f269740b740 1 -- 172.21.15.104:0/2847537722 shutdown_connections 2023-07-14T11:53:23.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f269740b740 1 --2- 172.21.15.104:0/2847537722 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0x1492790 0x16e6b70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f269740b740 1 --2- 172.21.15.104:0/2847537722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e8c60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f269740b740 1 --2- 172.21.15.104:0/2847537722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1447eb0 0x170a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f269740b740 1 --2- 172.21.15.104:0/2847537722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1447710 0x16e9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f266cff9700 0 client.4446 ms_handle_reset on v2:172.21.15.104:6836/2666329866 2023-07-14T11:53:23.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f269740b740 1 -- 172.21.15.104:0/2847537722 >> 172.21.15.104:0/2847537722 conn(0x16b1b60 msgr2=0x16bc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:23.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f269740b740 1 -- 172.21.15.104:0/2847537722 shutdown_connections 2023-07-14T11:53:23.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.389+0000 7f269740b740 1 -- 172.21.15.104:0/2847537722 wait complete. 2023-07-14T11:53:23.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.393+0000 7f269740b740 20 client.4446 trim_cache size 0 max 16384 2023-07-14T11:53:23.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.409+0000 7f269740b740 1 Processor -- start 2023-07-14T11:53:23.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 -- start start 2023-07-14T11:53:23.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16b1b60 0x16e8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1447710 0x16e45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x1623160 con 0x1447ae0 2023-07-14T11:53:23.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x161f0f0 con 0x16b1b60 2023-07-14T11:53:23.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x15bef30 con 0x1447710 2023-07-14T11:53:23.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f2692723700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f2692723700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46334/0 (socket says 172.21.15.104:46334) 2023-07-14T11:53:23.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f2692723700 1 -- 172.21.15.104:0/1299407354 learned_addr learned my addr 172.21.15.104:0/1299407354 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:23.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f2691f22700 1 --2- 172.21.15.104:0/1299407354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16b1b60 0x16e8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f2691721700 1 --2- 172.21.15.104:0/1299407354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1447710 0x16e45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f2692723700 1 -- 172.21.15.104:0/1299407354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1447710 msgr2=0x16e45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:23.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f2692723700 1 --2- 172.21.15.104:0/1299407354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1447710 0x16e45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f2692723700 1 -- 172.21.15.104:0/1299407354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16b1b60 msgr2=0x16e8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:23.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f2692723700 1 --2- 172.21.15.104:0/1299407354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16b1b60 0x16e8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f2692723700 1 -- 172.21.15.104:0/1299407354 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x16ea290 con 0x1447ae0 2023-07-14T11:53:23.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f2691f22700 1 --2- 172.21.15.104:0/1299407354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16b1b60 0x16e8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:23.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f2691721700 1 --2- 172.21.15.104:0/1299407354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1447710 0x16e45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:23.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f2692723700 1 --2- 172.21.15.104:0/1299407354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e4b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2684014380 tx=0x7f2684009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3475f9a508da51ce server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:23.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f266ffff700 1 -- 172.21.15.104:0/1299407354 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684010040 con 0x1447ae0 2023-07-14T11:53:23.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f266ffff700 1 -- 172.21.15.104:0/1299407354 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2684003c40 con 0x1447ae0 2023-07-14T11:53:23.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f266ffff700 1 -- 172.21.15.104:0/1299407354 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684014ee0 con 0x1447ae0 2023-07-14T11:53:23.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 -- 172.21.15.104:0/1299407354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 msgr2=0x16e4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:23.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 --2- 172.21.15.104:0/1299407354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e4b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2684014380 tx=0x7f2684009750 comp rx=0 tx=0).stop 2023-07-14T11:53:23.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 -- 172.21.15.104:0/1299407354 shutdown_connections 2023-07-14T11:53:23.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 --2- 172.21.15.104:0/1299407354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e4b00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 --2- 172.21.15.104:0/1299407354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1447710 0x16e45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 --2- 172.21.15.104:0/1299407354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16b1b60 0x16e8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 -- 172.21.15.104:0/1299407354 >> 172.21.15.104:0/1299407354 conn(0x170a210 msgr2=0x16fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:23.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 -- 172.21.15.104:0/1299407354 shutdown_connections 2023-07-14T11:53:23.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.413+0000 7f269740b740 1 -- 172.21.15.104:0/1299407354 wait complete. 2023-07-14T11:53:23.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f269740b740 1 -- start start 2023-07-14T11:53:23.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f269740b740 1 Processor -- start 2023-07-14T11:53:23.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f269740b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1447710 0x16e8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f269740b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f269740b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x16b1b60 0x170fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f269740b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x15deea0 con 0x1447ae0 2023-07-14T11:53:23.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f269740b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x161f490 con 0x1447710 2023-07-14T11:53:23.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f269740b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x161de10 con 0x16b1b60 2023-07-14T11:53:23.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f269740b740 10 client.0 _ll_register_callbacks cb 0x15d8800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:53:23.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f2691721700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f2691721700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46338/0 (socket says 172.21.15.104:46338) 2023-07-14T11:53:23.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f2691721700 1 -- 172.21.15.104:0/246852466 learned_addr learned my addr 172.21.15.104:0/246852466 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:23.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f2691f22700 1 --2- 172.21.15.104:0/246852466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1447710 0x16e8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f2691721700 1 -- 172.21.15.104:0/246852466 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x16b1b60 msgr2=0x170fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:23.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f2691721700 1 --2- 172.21.15.104:0/246852466 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x16b1b60 0x170fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f2691721700 1 -- 172.21.15.104:0/246852466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1447710 msgr2=0x16e8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:23.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f2691721700 1 --2- 172.21.15.104:0/246852466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1447710 0x16e8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f2691721700 1 -- 172.21.15.104:0/246852466 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2684009d90 con 0x1447ae0 2023-07-14T11:53:23.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.417+0000 7f2691f22700 1 --2- 172.21.15.104:0/246852466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1447710 0x16e8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:23.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f2691721700 1 --2- 172.21.15.104:0/246852466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e49c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f267c05dd60 tx=0x7f267c025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc602b4449f9541 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:23.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f266cff9700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-14T11:53:23.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f266cff9700 1 -- 172.21.15.104:0/246852466 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c055430 con 0x1447ae0 2023-07-14T11:53:23.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 1 -- 172.21.15.104:0/246852466 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2688014b90 con 0x1447ae0 2023-07-14T11:53:23.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f266cff9700 1 -- 172.21.15.104:0/246852466 <== mon.0 v2:172.21.15.104:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f267c055590 con 0x1447ae0 2023-07-14T11:53:23.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f266cff9700 1 -- 172.21.15.104:0/246852466 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f267c05de00 con 0x1447ae0 2023-07-14T11:53:23.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 10 client.4452 fetch_fsmap learned FSMap version 7 2023-07-14T11:53:23.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 1 -- 172.21.15.104:0/246852466 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x16ff7f0 con 0x1447ae0 2023-07-14T11:53:23.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f266cff9700 1 -- 172.21.15.104:0/246852466 <== mon.0 v2:172.21.15.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2688014b90 con 0x1447ae0 2023-07-14T11:53:23.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f266cff9700 1 -- 172.21.15.104:0/246852466 <== mon.0 v2:172.21.15.104:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f267c051d30 con 0x1447ae0 2023-07-14T11:53:23.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 10 client.4452 fetch_fsmap finished waiting for FSMap version 7 2023-07-14T11:53:23.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 10 client.4452 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-14T11:53:23.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 20 client.4452 populate_metadata read hostname 'smithi104' 2023-07-14T11:53:23.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 1 --2- 172.21.15.104:0/246852466 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0x1447eb0 0x1767e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 4 client.4452 mds_command: new command op to 4275 tid=0[{"prefix": "status"}] 2023-07-14T11:53:23.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 1 -- 172.21.15.104:0/246852466 --> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] -- command(tid 0: {"prefix": "status"}) v1 -- 0x149f4e0 con 0x1447eb0 2023-07-14T11:53:23.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f2691f22700 1 --2- 172.21.15.104:0/246852466 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0x1447eb0 0x1767e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f2691f22700 1 --2- 172.21.15.104:0/246852466 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0x1447eb0 0x1767e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=688858bbfcd452d server_cookie=8b7aa4f393dea380 in_seq=0 out_seq=0 2023-07-14T11:53:23.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f266cff9700 10 client.4452 ms_handle_connect on v2:172.21.15.104:6836/2666329866 2023-07-14T11:53:23.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f266cff9700 1 -- 172.21.15.104:0/246852466 <== mds.0 v2:172.21.15.104:6836/2666329866 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x149f4e0 con 0x1447eb0 2023-07-14T11:53:23.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f266cff9700 10 client.4452 handle_command_reply: tid=0 2023-07-14T11:53:23.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 1 client.4452 shutdown 2023-07-14T11:53:23.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 1 -- 172.21.15.104:0/246852466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 msgr2=0x16e49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:23.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 1 --2- 172.21.15.104:0/246852466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e49c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f267c05dd60 tx=0x7f267c025000 comp rx=0 tx=0).stop 2023-07-14T11:53:23.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 1 -- 172.21.15.104:0/246852466 shutdown_connections 2023-07-14T11:53:23.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 1 --2- 172.21.15.104:0/246852466 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0x1447eb0 0x1767e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 1 --2- 172.21.15.104:0/246852466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1447ae0 0x16e49c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 1 --2- 172.21.15.104:0/246852466 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x16b1b60 0x170fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f266cff9700 0 client.4452 ms_handle_reset on v2:172.21.15.104:6836/2666329866 2023-07-14T11:53:23.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 1 --2- 172.21.15.104:0/246852466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1447710 0x16e8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 1 -- 172.21.15.104:0/246852466 >> 172.21.15.104:0/246852466 conn(0x170a210 msgr2=0x16bc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:23.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 1 -- 172.21.15.104:0/246852466 shutdown_connections 2023-07-14T11:53:23.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.421+0000 7f269740b740 1 -- 172.21.15.104:0/246852466 wait complete. 2023-07-14T11:53:23.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.425+0000 7f269740b740 20 client.4452 trim_cache size 0 max 16384 2023-07-14T11:53:23.420 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:53:23.420 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_fsid": "c5576993-99e1-43b1-a253-659a3cee5212", 2023-07-14T11:53:23.421 INFO:teuthology.orchestra.run.smithi104.stdout: "whoami": 0, 2023-07-14T11:53:23.421 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 4275, 2023-07-14T11:53:23.421 INFO:teuthology.orchestra.run.smithi104.stdout: "want_state": "up:active", 2023-07-14T11:53:23.421 INFO:teuthology.orchestra.run.smithi104.stdout: "state": "up:active", 2023-07-14T11:53:23.421 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs", 2023-07-14T11:53:23.422 INFO:teuthology.orchestra.run.smithi104.stdout: "rank_uptime": 7.8634516809999999, 2023-07-14T11:53:23.422 INFO:teuthology.orchestra.run.smithi104.stdout: "mdsmap_epoch": 7, 2023-07-14T11:53:23.422 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_epoch": 28, 2023-07-14T11:53:23.422 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_epoch_barrier": 28, 2023-07-14T11:53:23.422 INFO:teuthology.orchestra.run.smithi104.stdout: "uptime": 14.134011182 2023-07-14T11:53:23.422 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:53:23.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.425+0000 7f2692f24700 1 -- 172.21.15.104:0/1245354368 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f26700662f0 msgr2=0x7f26700687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:23.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.425+0000 7f2692f24700 1 --2- 172.21.15.104:0/1245354368 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f26700662f0 0x7f26700687a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f268c0b32f0 tx=0x7f2688009240 comp rx=0 tx=0).stop 2023-07-14T11:53:23.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.425+0000 7f2692f24700 1 -- 172.21.15.104:0/1245354368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f268c1599d0 msgr2=0x7f268c0b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:23.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.425+0000 7f2692f24700 1 --2- 172.21.15.104:0/1245354368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f268c1599d0 0x7f268c0b1640 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f267c009f40 tx=0x7f267c004770 comp rx=0 tx=0).stop 2023-07-14T11:53:23.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.425+0000 7f2692f24700 1 -- 172.21.15.104:0/1245354368 shutdown_connections 2023-07-14T11:53:23.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.425+0000 7f2692f24700 1 --2- 172.21.15.104:0/1245354368 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f26700662f0 0x7f26700687a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.425+0000 7f2692f24700 1 --2- 172.21.15.104:0/1245354368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f268c1599d0 0x7f268c0b1640 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.425+0000 7f2692f24700 1 --2- 172.21.15.104:0/1245354368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f268c15ccf0 0x7f268c1531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.425+0000 7f2692f24700 1 --2- 172.21.15.104:0/1245354368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f268c0caa20 0x7f268c1f5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.425+0000 7f2692f24700 1 -- 172.21.15.104:0/1245354368 >> 172.21.15.104:0/1245354368 conn(0x7f268c06e960 msgr2=0x7f268c06d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:23.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.425+0000 7f2692f24700 1 -- 172.21.15.104:0/1245354368 shutdown_connections 2023-07-14T11:53:23.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.425+0000 7f2692f24700 1 -- 172.21.15.104:0/1245354368 wait complete. 2023-07-14T11:53:23.432 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-14T11:53:23.433 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:53:23.433 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-14T11:53:23.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.629+0000 7fd68155d700 1 Processor -- start 2023-07-14T11:53:23.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.629+0000 7fd68155d700 1 -- start start 2023-07-14T11:53:23.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.629+0000 7fd68155d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c06dc20 0x7fd67c0cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.629+0000 7fd68155d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd67c0c9820 0x7fd67c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.629+0000 7fd68155d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fd67c158fe0 0x7fd67c159480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.629+0000 7fd68155d700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67c159d60 con 0x7fd67c158fe0 2023-07-14T11:53:23.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.629+0000 7fd68155d700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67c0c8a60 con 0x7fd67c0c9820 2023-07-14T11:53:23.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.629+0000 7fd68155d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c0c8d40 con 0x7fd67c06dc20 2023-07-14T11:53:23.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.629+0000 7fd680d5c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fd67c158fe0 0x7fd67c159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.629+0000 7fd680d5c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fd67c158fe0 0x7fd67c159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46340/0 (socket says 172.21.15.104:46340) 2023-07-14T11:53:23.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.629+0000 7fd680d5c700 1 -- 172.21.15.104:0/240546133 learned_addr learned my addr 172.21.15.104:0/240546133 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:23.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.629+0000 7fd67bfff700 1 --2- 172.21.15.104:0/240546133 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c06dc20 0x7fd67c0cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.629+0000 7fd673fff700 1 --2- 172.21.15.104:0/240546133 >> v2:172.21.15.121:3300/0 conn(0x7fd67c0c9820 0x7fd67c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd680d5c700 1 -- 172.21.15.104:0/240546133 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c06dc20 msgr2=0x7fd67c0cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:23.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd680d5c700 1 --2- 172.21.15.104:0/240546133 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c06dc20 0x7fd67c0cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd680d5c700 1 -- 172.21.15.104:0/240546133 >> v2:172.21.15.121:3300/0 conn(0x7fd67c0c9820 msgr2=0x7fd67c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:23.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd680d5c700 1 --2- 172.21.15.104:0/240546133 >> v2:172.21.15.121:3300/0 conn(0x7fd67c0c9820 0x7fd67c0ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd680d5c700 1 -- 172.21.15.104:0/240546133 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67c0c9020 con 0x7fd67c158fe0 2023-07-14T11:53:23.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd673fff700 1 --2- 172.21.15.104:0/240546133 >> v2:172.21.15.121:3300/0 conn(0x7fd67c0c9820 0x7fd67c0ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:23.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd680d5c700 1 --2- 172.21.15.104:0/240546133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd67c158fe0 0x7fd67c159480 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd674009e50 tx=0x7fd67400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c27777bb58f1781 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:23.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd67b7fe700 1 -- 172.21.15.104:0/240546133 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67400e070 con 0x7fd67c158fe0 2023-07-14T11:53:23.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd67b7fe700 1 -- 172.21.15.104:0/240546133 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67400b8d0 con 0x7fd67c158fe0 2023-07-14T11:53:23.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd67b7fe700 1 -- 172.21.15.104:0/240546133 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67400bc80 con 0x7fd67c158fe0 2023-07-14T11:53:23.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd68155d700 1 -- 172.21.15.104:0/240546133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd67c158fe0 msgr2=0x7fd67c159480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:23.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd68155d700 1 --2- 172.21.15.104:0/240546133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd67c158fe0 0x7fd67c159480 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd674009e50 tx=0x7fd67400b330 comp rx=0 tx=0).stop 2023-07-14T11:53:23.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd68155d700 1 -- 172.21.15.104:0/240546133 shutdown_connections 2023-07-14T11:53:23.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd68155d700 1 --2- 172.21.15.104:0/240546133 >> v2:172.21.15.121:3300/0 conn(0x7fd67c0c9820 0x7fd67c0ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd68155d700 1 --2- 172.21.15.104:0/240546133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd67c158fe0 0x7fd67c159480 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd68155d700 1 --2- 172.21.15.104:0/240546133 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c06dc20 0x7fd67c0cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd68155d700 1 -- 172.21.15.104:0/240546133 >> 172.21.15.104:0/240546133 conn(0x7fd67c06e960 msgr2=0x7fd67c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:23.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd68155d700 1 -- 172.21.15.104:0/240546133 shutdown_connections 2023-07-14T11:53:23.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.633+0000 7fd68155d700 1 -- 172.21.15.104:0/240546133 wait complete. 2023-07-14T11:53:23.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.637+0000 7fd68155d700 1 Processor -- start 2023-07-14T11:53:23.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.637+0000 7fd68155d700 1 -- start start 2023-07-14T11:53:23.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.637+0000 7fd68155d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd67c06dc20 0x7fd67c143a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.637+0000 7fd68155d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c0c9820 0x7fd67c144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.637+0000 7fd68155d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd67c158fe0 0x7fd67c0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.637+0000 7fd68155d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c1403b0 con 0x7fd67c158fe0 2023-07-14T11:53:23.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.637+0000 7fd68155d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c144dd0 con 0x7fd67c06dc20 2023-07-14T11:53:23.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.637+0000 7fd68155d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c144af0 con 0x7fd67c0c9820 2023-07-14T11:53:23.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd680d5c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd67c158fe0 0x7fd67c0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd680d5c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd67c158fe0 0x7fd67c0ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46350/0 (socket says 172.21.15.104:46350) 2023-07-14T11:53:23.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd680d5c700 1 -- 172.21.15.104:0/3757580568 learned_addr learned my addr 172.21.15.104:0/3757580568 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:23.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd673fff700 1 --2- 172.21.15.104:0/3757580568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c0c9820 0x7fd67c144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd67bfff700 1 --2- 172.21.15.104:0/3757580568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd67c06dc20 0x7fd67c143a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd680d5c700 1 -- 172.21.15.104:0/3757580568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c0c9820 msgr2=0x7fd67c144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:23.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd680d5c700 1 --2- 172.21.15.104:0/3757580568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c0c9820 0x7fd67c144240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd680d5c700 1 -- 172.21.15.104:0/3757580568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd67c06dc20 msgr2=0x7fd67c143a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:23.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd680d5c700 1 --2- 172.21.15.104:0/3757580568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd67c06dc20 0x7fd67c143a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd680d5c700 1 -- 172.21.15.104:0/3757580568 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd674009b30 con 0x7fd67c158fe0 2023-07-14T11:53:23.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd673fff700 1 --2- 172.21.15.104:0/3757580568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c0c9820 0x7fd67c144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:23.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd680d5c700 1 --2- 172.21.15.104:0/3757580568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd67c158fe0 0x7fd67c0ccac0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd67c071ce0 tx=0x7fd674004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:23.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd6797fa700 1 -- 172.21.15.104:0/3757580568 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd674004450 con 0x7fd67c158fe0 2023-07-14T11:53:23.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd6797fa700 1 -- 172.21.15.104:0/3757580568 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6740045d0 con 0x7fd67c158fe0 2023-07-14T11:53:23.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd68155d700 1 -- 172.21.15.104:0/3757580568 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd67c0c23b0 con 0x7fd67c158fe0 2023-07-14T11:53:23.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd6797fa700 1 -- 172.21.15.104:0/3757580568 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd674004980 con 0x7fd67c158fe0 2023-07-14T11:53:23.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd68155d700 1 -- 172.21.15.104:0/3757580568 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67c0c2970 con 0x7fd67c158fe0 2023-07-14T11:53:23.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.641+0000 7fd68155d700 1 -- 172.21.15.104:0/3757580568 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67c0c2210 con 0x7fd67c158fe0 2023-07-14T11:53:23.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.645+0000 7fd6797fa700 1 -- 172.21.15.104:0/3757580568 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd67401c410 con 0x7fd67c158fe0 2023-07-14T11:53:23.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.645+0000 7fd6797fa700 1 --2- 172.21.15.104:0/3757580568 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd65c0662f0 0x7fd65c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:23.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.645+0000 7fd6797fa700 1 -- 172.21.15.104:0/3757580568 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd674085850 con 0x7fd67c158fe0 2023-07-14T11:53:23.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.645+0000 7fd67bfff700 1 --2- 172.21.15.104:0/3757580568 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd65c0662f0 0x7fd65c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:23.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.649+0000 7fd67bfff700 1 --2- 172.21.15.104:0/3757580568 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd65c0662f0 0x7fd65c0687a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0c5090 tx=0x7fd66c009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:23.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.653+0000 7fd6797fa700 1 -- 172.21.15.104:0/3757580568 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fd67c0c2210 con 0x7fd67c158fe0 2023-07-14T11:53:23.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.865+0000 7fd68155d700 1 -- 172.21.15.104:0/3757580568 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd67c0c2210 con 0x7fd67c158fe0 2023-07-14T11:53:23.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.865+0000 7fd6797fa700 1 -- 172.21.15.104:0/3757580568 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd67c0c2210 con 0x7fd67c158fe0 2023-07-14T11:53:23.852 INFO:teuthology.orchestra.run.smithi104.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-14T11:53:23.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.865+0000 7fd68155d700 1 -- 172.21.15.104:0/3757580568 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd65c0662f0 msgr2=0x7fd65c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:23.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.865+0000 7fd68155d700 1 --2- 172.21.15.104:0/3757580568 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd65c0662f0 0x7fd65c0687a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0c5090 tx=0x7fd66c009240 comp rx=0 tx=0).stop 2023-07-14T11:53:23.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.865+0000 7fd68155d700 1 -- 172.21.15.104:0/3757580568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd67c158fe0 msgr2=0x7fd67c0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:23.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.865+0000 7fd68155d700 1 --2- 172.21.15.104:0/3757580568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd67c158fe0 0x7fd67c0ccac0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd67c071ce0 tx=0x7fd674004af0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.865+0000 7fd68155d700 1 -- 172.21.15.104:0/3757580568 shutdown_connections 2023-07-14T11:53:23.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.865+0000 7fd68155d700 1 --2- 172.21.15.104:0/3757580568 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd65c0662f0 0x7fd65c0687a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.865+0000 7fd68155d700 1 --2- 172.21.15.104:0/3757580568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd67c158fe0 0x7fd67c0ccac0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.865+0000 7fd68155d700 1 --2- 172.21.15.104:0/3757580568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c0c9820 0x7fd67c144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.865+0000 7fd68155d700 1 --2- 172.21.15.104:0/3757580568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd67c06dc20 0x7fd67c143a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:23.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.865+0000 7fd68155d700 1 -- 172.21.15.104:0/3757580568 >> 172.21.15.104:0/3757580568 conn(0x7fd67c06e960 msgr2=0x7fd67c152280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:23.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.865+0000 7fd68155d700 1 -- 172.21.15.104:0/3757580568 shutdown_connections 2023-07-14T11:53:23.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:23.865+0000 7fd68155d700 1 -- 172.21.15.104:0/3757580568 wait complete. 2023-07-14T11:53:23.876 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-14T11:53:23.876 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T11:53:23.877 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2023-07-14T11:53:23.877 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-07-14T11:53:23.877 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:53:23.877 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-14T11:53:23.878 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T11:53:23.878 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T11:53:23.878 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-07-14T11:53:23.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:23.879 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2023-07-14T11:53:23.896 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:53:23.897 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:53:23.897 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:53:23.897 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:23.897 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2023-07-14T11:53:23.897 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:23.897 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:53:23.897 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:23.898 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2023-07-14T11:53:23.898 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:53:23.898 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:23.898 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2023-07-14T11:53:23.899 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:53:23.899 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:23.899 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T11:53:23.899 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:23.899 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-07-14T11:53:23.899 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42392sec preferred_lft 42392sec 2023-07-14T11:53:23.899 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2023-07-14T11:53:23.899 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:23.900 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-07-14T11:53:23.900 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:a2:a1:53:16 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:23.900 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-07-14T11:53:23.900 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:23.901 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-14T11:53:23.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:23.902 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:53:23.902 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:53:23.902 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link add name ceph-brx type bridge 2023-07-14T11:53:23.903 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip addr flush dev ceph-brx 2023-07-14T11:53:23.903 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set ceph-brx up 2023-07-14T11:53:23.903 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-14T11:53:23.904 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:53:24.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:24.059 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T11:53:24.088 INFO:teuthology.orchestra.run.smithi104.stdout:1 2023-07-14T11:53:24.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:24.091 DEBUG:teuthology.orchestra.run.smithi104:> route 2023-07-14T11:53:24.177 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2023-07-14T11:53:24.178 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T11:53:24.178 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-07-14T11:53:24.178 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T11:53:24.178 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-07-14T11:53:24.178 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-07-14T11:53:24.179 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-07-14T11:53:24.179 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T11:53:24.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:24.180 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:53:24.180 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:53:24.180 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T11:53:24.181 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T11:53:24.181 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T11:53:24.181 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:53:24.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:24.318 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2023-07-14T11:53:24.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:24.337 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list-id 2023-07-14T11:53:24.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:24.400 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:53:24.400 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:53:24.400 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-07-14T11:53:24.401 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-07-14T11:53:24.401 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:53:24.514 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-07-14T11:53:24.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:24.515 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:53:24.515 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:53:24.515 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-07-14T11:53:24.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-14T11:53:24.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-07-14T11:53:24.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-07-14T11:53:24.517 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-07-14T11:53:24.517 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:53:24.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:24.693 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:53:24.693 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:53:24.694 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2023-07-14T11:53:24.694 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set dev brx.0 master ceph-brx 2023-07-14T11:53:24.694 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:53:24.762 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-07-14T11:53:24.763 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:53:24.764 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:53:24.781 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-07-14T11:53:24.781 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:53:24.782 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:53:24.838 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:53:24.920 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:53:24.920 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:53:24.995 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:53:24.999 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:53:24.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:25.000 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:53:25.061 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-14T11:53:25.061 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-07-14T11:53:25.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:53:25.142 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:53:25.143 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:53:25.219 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:53:25.222 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:53:25.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:25.223 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:53:25.258 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi104.stderr:2023-07-14T11:53:25.273+0000 7feefd37f640 -1 init, newargv = 0x561326dff860 newargc=15 2023-07-14T11:53:25.259 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi104.stderr:ceph-fuse[41139]: starting ceph client 2023-07-14T11:53:25.274 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi104.stderr:ceph-fuse[41139]: starting fuse 2023-07-14T11:53:26.372 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:53:26.407 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:53:26.407 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:53:26.481 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:53:26.483 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:53:26.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:26.485 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:53:26.545 INFO:teuthology.orchestra.run.smithi104.stdout:49 2023-07-14T11:53:26.545 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-07-14T11:53:26.545 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T11:53:26.546 DEBUG:teuthology.orchestra.run.smithi104:> import glob 2023-07-14T11:53:26.546 DEBUG:teuthology.orchestra.run.smithi104:> import re 2023-07-14T11:53:26.546 DEBUG:teuthology.orchestra.run.smithi104:> import os 2023-07-14T11:53:26.547 DEBUG:teuthology.orchestra.run.smithi104:> import subprocess 2023-07-14T11:53:26.547 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:53:26.547 DEBUG:teuthology.orchestra.run.smithi104:> def _find_admin_socket(client_name): 2023-07-14T11:53:26.548 DEBUG:teuthology.orchestra.run.smithi104:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-07-14T11:53:26.548 DEBUG:teuthology.orchestra.run.smithi104:> files = glob.glob(asok_path) 2023-07-14T11:53:26.548 DEBUG:teuthology.orchestra.run.smithi104:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-07-14T11:53:26.549 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:53:26.549 DEBUG:teuthology.orchestra.run.smithi104:> # Given a non-glob path, it better be there 2023-07-14T11:53:26.549 DEBUG:teuthology.orchestra.run.smithi104:> if "*" not in asok_path: 2023-07-14T11:53:26.550 DEBUG:teuthology.orchestra.run.smithi104:> assert(len(files) == 1) 2023-07-14T11:53:26.550 DEBUG:teuthology.orchestra.run.smithi104:> return files[0] 2023-07-14T11:53:26.550 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:53:26.550 DEBUG:teuthology.orchestra.run.smithi104:> for f in files: 2023-07-14T11:53:26.551 DEBUG:teuthology.orchestra.run.smithi104:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-14T11:53:26.551 DEBUG:teuthology.orchestra.run.smithi104:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-14T11:53:26.551 DEBUG:teuthology.orchestra.run.smithi104:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-14T11:53:26.552 DEBUG:teuthology.orchestra.run.smithi104:> contents = proc_f.read() 2023-07-14T11:53:26.552 DEBUG:teuthology.orchestra.run.smithi104:> if mountpoint in contents: 2023-07-14T11:53:26.552 DEBUG:teuthology.orchestra.run.smithi104:> return f 2023-07-14T11:53:26.553 DEBUG:teuthology.orchestra.run.smithi104:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-14T11:53:26.553 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:53:26.553 DEBUG:teuthology.orchestra.run.smithi104:> print(_find_admin_socket("client.admin")) 2023-07-14T11:53:26.554 DEBUG:teuthology.orchestra.run.smithi104:> ' 2023-07-14T11:53:26.695 INFO:teuthology.orchestra.run.smithi104.stdout:/var/run/ceph/ceph-client.admin.41139.asok 2023-07-14T11:53:26.882 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41139.asok 2023-07-14T11:53:26.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:26.883 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41139.asok status 2023-07-14T11:53:27.079 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:53:27.079 INFO:teuthology.orchestra.run.smithi104.stdout: "metadata": { 2023-07-14T11:53:27.080 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-14T11:53:27.080 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:53:27.080 INFO:teuthology.orchestra.run.smithi104.stdout: "entity_id": "admin", 2023-07-14T11:53:27.080 INFO:teuthology.orchestra.run.smithi104.stdout: "hostname": "smithi104", 2023-07-14T11:53:27.080 INFO:teuthology.orchestra.run.smithi104.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-07-14T11:53:27.080 INFO:teuthology.orchestra.run.smithi104.stdout: "pid": "41139", 2023-07-14T11:53:27.081 INFO:teuthology.orchestra.run.smithi104.stdout: "root": "/" 2023-07-14T11:53:27.081 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:53:27.081 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_count": 0, 2023-07-14T11:53:27.081 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_pinned_count": 0, 2023-07-14T11:53:27.081 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 4467, 2023-07-14T11:53:27.081 INFO:teuthology.orchestra.run.smithi104.stdout: "inst": { 2023-07-14T11:53:27.081 INFO:teuthology.orchestra.run.smithi104.stdout: "name": { 2023-07-14T11:53:27.082 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "client", 2023-07-14T11:53:27.082 INFO:teuthology.orchestra.run.smithi104.stdout: "num": 4467 2023-07-14T11:53:27.082 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:53:27.082 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:53:27.082 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:53:27.082 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:53:27.082 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 637960466 2023-07-14T11:53:27.083 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:53:27.083 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:53:27.083 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:53:27.083 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:53:27.083 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:53:27.083 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 637960466 2023-07-14T11:53:27.083 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:53:27.083 INFO:teuthology.orchestra.run.smithi104.stdout: "inst_str": "client.4467 192.168.0.1:0/637960466", 2023-07-14T11:53:27.084 INFO:teuthology.orchestra.run.smithi104.stdout: "addr_str": "192.168.0.1:0/637960466", 2023-07-14T11:53:27.084 INFO:teuthology.orchestra.run.smithi104.stdout: "inode_count": 1, 2023-07-14T11:53:27.084 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_epoch": 7, 2023-07-14T11:53:27.084 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch": 28, 2023-07-14T11:53:27.084 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch_barrier": 0, 2023-07-14T11:53:27.084 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklisted": false, 2023-07-14T11:53:27.084 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs" 2023-07-14T11:53:27.085 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:53:27.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:27.095 DEBUG:teuthology.orchestra.run.smithi104:> stat --file-system '--printf=%T 2023-07-14T11:53:27.095 DEBUG:teuthology.orchestra.run.smithi104:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:53:27.113 INFO:teuthology.orchestra.run.smithi104.stdout:fuseblk 2023-07-14T11:53:27.113 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:53:27.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:27.114 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:53:27.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:27.193 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-07-14T11:53:27.298 INFO:teuthology.orchestra.run.smithi104.stdout:# file: . 2023-07-14T11:53:27.298 INFO:teuthology.orchestra.run.smithi104.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-07-14T11:53:27.298 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:27.299 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:53:27.300 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:53:27.313 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:27.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.549+0000 7f4768546700 1 Processor -- start 2023-07-14T11:53:27.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.549+0000 7f4768546700 1 -- start start 2023-07-14T11:53:27.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4768546700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f47600d01a0 0x7f47600cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:27.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4768546700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47601604e0 0x7f47600b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:27.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4768546700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47600ce200 con 0x7f47600cf730 2023-07-14T11:53:27.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4768546700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47600b1af0 con 0x7f47600d01a0 2023-07-14T11:53:27.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4768546700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4760070ae0 con 0x7f47601604e0 2023-07-14T11:53:27.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4766ae3700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f47600cf730 0x7f47600d0570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:51150/0 (socket says 172.21.15.104:51150) 2023-07-14T11:53:27.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4766ae3700 1 -- 172.21.15.104:0/1561783084 learned_addr learned my addr 172.21.15.104:0/1561783084 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:27.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f47662e2700 1 --2- 172.21.15.104:0/1561783084 >> v2:172.21.15.104:3300/0 conn(0x7f47600d01a0 0x7f47600cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:27.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4765ae1700 1 --2- 172.21.15.104:0/1561783084 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47601604e0 0x7f47600b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:27.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f47662e2700 1 -- 172.21.15.104:0/1561783084 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47601604e0 msgr2=0x7f47600b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:27.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f47662e2700 1 --2- 172.21.15.104:0/1561783084 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47601604e0 0x7f47600b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:27.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f47662e2700 1 -- 172.21.15.104:0/1561783084 >> v1:172.21.15.104:6789/0 conn(0x7f47600cf730 legacy=0x7f47600d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:27.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f47662e2700 1 -- 172.21.15.104:0/1561783084 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47600ce430 con 0x7f47600d01a0 2023-07-14T11:53:27.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f47662e2700 1 --2- 172.21.15.104:0/1561783084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47600d01a0 0x7f47600cf1f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f475c009850 tx=0x7f475c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d135fde920a02 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:27.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f47652e0700 1 -- 172.21.15.104:0/1561783084 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c011830 con 0x7f47600d01a0 2023-07-14T11:53:27.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f47652e0700 1 -- 172.21.15.104:0/1561783084 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475c011990 con 0x7f47600d01a0 2023-07-14T11:53:27.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f47652e0700 1 -- 172.21.15.104:0/1561783084 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c011d40 con 0x7f47600d01a0 2023-07-14T11:53:27.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4768546700 1 -- 172.21.15.104:0/1561783084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47600d01a0 msgr2=0x7f47600cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:27.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4768546700 1 --2- 172.21.15.104:0/1561783084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47600d01a0 0x7f47600cf1f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f475c009850 tx=0x7f475c009b60 comp rx=0 tx=0).stop 2023-07-14T11:53:27.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4768546700 1 -- 172.21.15.104:0/1561783084 shutdown_connections 2023-07-14T11:53:27.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4768546700 1 --2- 172.21.15.104:0/1561783084 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47601604e0 0x7f47600b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:27.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4768546700 1 --2- 172.21.15.104:0/1561783084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47600d01a0 0x7f47600cf1f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:27.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4768546700 1 -- 172.21.15.104:0/1561783084 >> 172.21.15.104:0/1561783084 conn(0x7f476006fc80 msgr2=0x7f4760070080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:27.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4768546700 1 -- 172.21.15.104:0/1561783084 shutdown_connections 2023-07-14T11:53:27.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.553+0000 7f4768546700 1 -- 172.21.15.104:0/1561783084 wait complete. 2023-07-14T11:53:27.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f4768546700 1 Processor -- start 2023-07-14T11:53:27.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f4768546700 1 -- start start 2023-07-14T11:53:27.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f4768546700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47600cf730 0x7f47600b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:27.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f4768546700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47600d01a0 0x7f47600b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:27.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f4768546700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47601604e0 0x7f47600b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:27.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f4768546700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4760143450 con 0x7f47601604e0 2023-07-14T11:53:27.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f4768546700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47600b4eb0 con 0x7f47600cf730 2023-07-14T11:53:27.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f4768546700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47600c7fc0 con 0x7f47600d01a0 2023-07-14T11:53:27.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f4766ae3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47601604e0 0x7f47600b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:27.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f4766ae3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47601604e0 0x7f47600b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46368/0 (socket says 172.21.15.104:46368) 2023-07-14T11:53:27.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f4766ae3700 1 -- 172.21.15.104:0/2304065484 learned_addr learned my addr 172.21.15.104:0/2304065484 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:27.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f47662e2700 1 --2- 172.21.15.104:0/2304065484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47600cf730 0x7f47600b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:27.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f4765ae1700 1 --2- 172.21.15.104:0/2304065484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47600d01a0 0x7f47600b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:27.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f47662e2700 1 -- 172.21.15.104:0/2304065484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47600d01a0 msgr2=0x7f47600b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:27.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f47662e2700 1 --2- 172.21.15.104:0/2304065484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47600d01a0 0x7f47600b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:27.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f47662e2700 1 -- 172.21.15.104:0/2304065484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47601604e0 msgr2=0x7f47600b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:27.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f47662e2700 1 --2- 172.21.15.104:0/2304065484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47601604e0 0x7f47600b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:27.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.557+0000 7f47662e2700 1 -- 172.21.15.104:0/2304065484 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c0094b0 con 0x7f47600cf730 2023-07-14T11:53:27.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.561+0000 7f47662e2700 1 --2- 172.21.15.104:0/2304065484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47600cf730 0x7f47600b7d00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f475c016000 tx=0x7f475c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:27.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.561+0000 7f4756ffd700 1 -- 172.21.15.104:0/2304065484 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c004610 con 0x7f47600cf730 2023-07-14T11:53:27.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.561+0000 7f4756ffd700 1 -- 172.21.15.104:0/2304065484 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475c01e070 con 0x7f47600cf730 2023-07-14T11:53:27.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.561+0000 7f4768546700 1 -- 172.21.15.104:0/2304065484 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47600c82a0 con 0x7f47600cf730 2023-07-14T11:53:27.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.561+0000 7f4756ffd700 1 -- 172.21.15.104:0/2304065484 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0038c0 con 0x7f47600cf730 2023-07-14T11:53:27.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.561+0000 7f4768546700 1 -- 172.21.15.104:0/2304065484 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47600c8860 con 0x7f47600cf730 2023-07-14T11:53:27.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.561+0000 7f4768546700 1 -- 172.21.15.104:0/2304065484 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476004dcb0 con 0x7f47600cf730 2023-07-14T11:53:27.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.565+0000 7f4756ffd700 1 -- 172.21.15.104:0/2304065484 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f475c003a20 con 0x7f47600cf730 2023-07-14T11:53:27.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.565+0000 7f4756ffd700 1 --2- 172.21.15.104:0/2304065484 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f47440662f0 0x7f47440687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:27.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.565+0000 7f4756ffd700 1 -- 172.21.15.104:0/2304065484 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f475c020070 con 0x7f47600cf730 2023-07-14T11:53:27.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.565+0000 7f4765ae1700 1 --2- 172.21.15.104:0/2304065484 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f47440662f0 0x7f47440687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:27.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.565+0000 7f4765ae1700 1 --2- 172.21.15.104:0/2304065484 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f47440662f0 0x7f47440687a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f476015b230 tx=0x7f4750009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:27.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.573+0000 7f4756ffd700 1 -- 172.21.15.104:0/2304065484 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f475c052c80 con 0x7f47600cf730 2023-07-14T11:53:27.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.753+0000 7f4768546700 1 -- 172.21.15.104:0/2304065484 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47600c89f0 con 0x7f47600cf730 2023-07-14T11:53:27.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.757+0000 7f4756ffd700 1 -- 172.21.15.104:0/2304065484 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20156 (secure 0 0 0) 0x7f475c056260 con 0x7f47600cf730 2023-07-14T11:53:27.745 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:27.745 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":28,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:20.091434+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:27.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.761+0000 7f4768546700 1 -- 172.21.15.104:0/2304065484 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f47440662f0 msgr2=0x7f47440687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:27.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.761+0000 7f4768546700 1 --2- 172.21.15.104:0/2304065484 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f47440662f0 0x7f47440687a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f476015b230 tx=0x7f4750009240 comp rx=0 tx=0).stop 2023-07-14T11:53:27.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.761+0000 7f4768546700 1 -- 172.21.15.104:0/2304065484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47600cf730 msgr2=0x7f47600b7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:27.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.761+0000 7f4768546700 1 --2- 172.21.15.104:0/2304065484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47600cf730 0x7f47600b7d00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f475c016000 tx=0x7f475c004770 comp rx=0 tx=0).stop 2023-07-14T11:53:27.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.761+0000 7f4768546700 1 -- 172.21.15.104:0/2304065484 shutdown_connections 2023-07-14T11:53:27.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.761+0000 7f4768546700 1 --2- 172.21.15.104:0/2304065484 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f47440662f0 0x7f47440687a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:27.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.761+0000 7f4768546700 1 --2- 172.21.15.104:0/2304065484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47601604e0 0x7f47600b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:27.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.761+0000 7f4768546700 1 --2- 172.21.15.104:0/2304065484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47600d01a0 0x7f47600b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:27.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.761+0000 7f4768546700 1 --2- 172.21.15.104:0/2304065484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47600cf730 0x7f47600b7d00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:27.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.761+0000 7f4768546700 1 -- 172.21.15.104:0/2304065484 >> 172.21.15.104:0/2304065484 conn(0x7f476006fc80 msgr2=0x7f476015f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:27.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.761+0000 7f4768546700 1 -- 172.21.15.104:0/2304065484 shutdown_connections 2023-07-14T11:53:27.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.761+0000 7f4768546700 1 -- 172.21.15.104:0/2304065484 wait complete. 2023-07-14T11:53:27.770 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:53:27.771 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:53:27.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.957+0000 7fb10e1c3700 1 Processor -- start 2023-07-14T11:53:27.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.957+0000 7fb10e1c3700 1 -- start start 2023-07-14T11:53:27.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.961+0000 7fb10e1c3700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb1080b0ee0 0x7fb1080ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:27.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.961+0000 7fb10e1c3700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1080cd4e0 con 0x7fb1080cd090 2023-07-14T11:53:27.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.961+0000 7fb10e1c3700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1080d1a60 con 0x7fb1080d07e0 2023-07-14T11:53:27.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.961+0000 7fb10e1c3700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1080d0c60 con 0x7fb1080b0ee0 2023-07-14T11:53:27.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.961+0000 7fb10d1c1700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb1080b0ee0 0x7fb1080ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:27.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.961+0000 7fb10c9c0700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb1080cd090 0x7fb1080d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:51158/0 (socket says 172.21.15.104:51158) 2023-07-14T11:53:27.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.961+0000 7fb10c9c0700 1 -- 172.21.15.104:0/4277448529 learned_addr learned my addr 172.21.15.104:0/4277448529 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:27.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.961+0000 7fb0fffff700 1 -- 172.21.15.104:0/4277448529 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441809927 0 0) 0x7fb0f8001110 con 0x7fb1080cd090 2023-07-14T11:53:27.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.961+0000 7fb0fffff700 1 -- 172.21.15.104:0/4277448529 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0f40035d0 con 0x7fb1080cd090 2023-07-14T11:53:27.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.961+0000 7fb10d1c1700 1 -- 172.21.15.104:0/4277448529 >> v1:172.21.15.121:6789/0 conn(0x7fb1080d07e0 legacy=0x7fb1080af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:27.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.965+0000 7fb0fffff700 1 -- 172.21.15.104:0/4277448529 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737725298 0 0) 0x7fb1080d1a60 con 0x7fb1080d07e0 2023-07-14T11:53:27.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.965+0000 7fb10d1c1700 1 -- 172.21.15.104:0/4277448529 >> v1:172.21.15.104:6789/0 conn(0x7fb1080cd090 legacy=0x7fb1080d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:27.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.965+0000 7fb10d1c1700 1 -- 172.21.15.104:0/4277448529 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1080d1ef0 con 0x7fb1080b0ee0 2023-07-14T11:53:27.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.965+0000 7fb10d1c1700 1 --2- 172.21.15.104:0/4277448529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1080b0ee0 0x7fb1080ccb50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb104009850 tx=0x7fb104009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3332bf77c881c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:27.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.965+0000 7fb0fffff700 1 -- 172.21.15.104:0/4277448529 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104011830 con 0x7fb1080b0ee0 2023-07-14T11:53:27.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.965+0000 7fb0fffff700 1 -- 172.21.15.104:0/4277448529 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb104011990 con 0x7fb1080b0ee0 2023-07-14T11:53:27.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.965+0000 7fb0fffff700 1 -- 172.21.15.104:0/4277448529 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104011d40 con 0x7fb1080b0ee0 2023-07-14T11:53:27.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.965+0000 7fb10e1c3700 1 -- 172.21.15.104:0/4277448529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1080b0ee0 msgr2=0x7fb1080ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:27.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.965+0000 7fb10e1c3700 1 --2- 172.21.15.104:0/4277448529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1080b0ee0 0x7fb1080ccb50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb104009850 tx=0x7fb104009b60 comp rx=0 tx=0).stop 2023-07-14T11:53:27.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.965+0000 7fb10e1c3700 1 -- 172.21.15.104:0/4277448529 shutdown_connections 2023-07-14T11:53:27.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.965+0000 7fb10e1c3700 1 --2- 172.21.15.104:0/4277448529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1080b0ee0 0x7fb1080ccb50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:27.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.965+0000 7fb10e1c3700 1 -- 172.21.15.104:0/4277448529 >> 172.21.15.104:0/4277448529 conn(0x7fb108060c40 msgr2=0x7fb10809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:27.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.965+0000 7fb10e1c3700 1 -- 172.21.15.104:0/4277448529 shutdown_connections 2023-07-14T11:53:27.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.965+0000 7fb10e1c3700 1 -- 172.21.15.104:0/4277448529 wait complete. 2023-07-14T11:53:27.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10e1c3700 1 Processor -- start 2023-07-14T11:53:27.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10e1c3700 1 -- start start 2023-07-14T11:53:27.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10e1c3700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1080b0ee0 0x7fb10815e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:27.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10e1c3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1080cd090 0x7fb1080b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:27.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10e1c3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1080d07e0 0x7fb1080b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:27.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10e1c3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1080b1af0 con 0x7fb1080d07e0 2023-07-14T11:53:27.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10e1c3700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1080b2670 con 0x7fb1080cd090 2023-07-14T11:53:27.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10e1c3700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1080b1d40 con 0x7fb1080b0ee0 2023-07-14T11:53:27.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10d9c2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1080d07e0 0x7fb1080b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:27.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10d9c2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1080d07e0 0x7fb1080b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46392/0 (socket says 172.21.15.104:46392) 2023-07-14T11:53:27.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10d9c2700 1 -- 172.21.15.104:0/3947141032 learned_addr learned my addr 172.21.15.104:0/3947141032 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:27.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10d1c1700 1 --2- 172.21.15.104:0/3947141032 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1080b0ee0 0x7fb10815e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:27.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10c9c0700 1 --2- 172.21.15.104:0/3947141032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1080cd090 0x7fb1080b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:27.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10d9c2700 1 -- 172.21.15.104:0/3947141032 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1080b0ee0 msgr2=0x7fb10815e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:27.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10d9c2700 1 --2- 172.21.15.104:0/3947141032 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1080b0ee0 0x7fb10815e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:27.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10d9c2700 1 -- 172.21.15.104:0/3947141032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1080cd090 msgr2=0x7fb1080b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:27.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10d9c2700 1 --2- 172.21.15.104:0/3947141032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1080cd090 0x7fb1080b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:27.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10d9c2700 1 -- 172.21.15.104:0/3947141032 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1040094b0 con 0x7fb1080d07e0 2023-07-14T11:53:27.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10d9c2700 1 --2- 172.21.15.104:0/3947141032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1080d07e0 0x7fb1080b4970 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb10000cc70 tx=0x7fb10000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:27.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb0fdffb700 1 -- 172.21.15.104:0/3947141032 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb100003e50 con 0x7fb1080d07e0 2023-07-14T11:53:27.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb0fdffb700 1 -- 172.21.15.104:0/3947141032 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10000ce70 con 0x7fb1080d07e0 2023-07-14T11:53:27.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb0fdffb700 1 -- 172.21.15.104:0/3947141032 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10000abf0 con 0x7fb1080d07e0 2023-07-14T11:53:27.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10e1c3700 1 -- 172.21.15.104:0/3947141032 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1080b2020 con 0x7fb1080d07e0 2023-07-14T11:53:27.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.969+0000 7fb10e1c3700 1 -- 172.21.15.104:0/3947141032 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1080b2b00 con 0x7fb1080d07e0 2023-07-14T11:53:27.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.973+0000 7fb10e1c3700 1 -- 172.21.15.104:0/3947141032 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10804e500 con 0x7fb1080d07e0 2023-07-14T11:53:27.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.973+0000 7fb0fdffb700 1 -- 172.21.15.104:0/3947141032 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb10000ad50 con 0x7fb1080d07e0 2023-07-14T11:53:27.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.977+0000 7fb0fdffb700 1 --2- 172.21.15.104:0/3947141032 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb0ec066320 0x7fb0ec0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:27.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.977+0000 7fb0fdffb700 1 -- 172.21.15.104:0/3947141032 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb100083b60 con 0x7fb1080d07e0 2023-07-14T11:53:27.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.977+0000 7fb10d1c1700 1 --2- 172.21.15.104:0/3947141032 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb0ec066320 0x7fb0ec0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:27.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.977+0000 7fb10d1c1700 1 --2- 172.21.15.104:0/3947141032 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb0ec066320 0x7fb0ec0687d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb1040090b0 tx=0x7fb10401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:27.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:27.981+0000 7fb0fdffb700 1 -- 172.21.15.104:0/3947141032 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb100050ce0 con 0x7fb1080d07e0 2023-07-14T11:53:28.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.153+0000 7fb10e1c3700 1 -- 172.21.15.104:0/3947141032 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1080b18d0 con 0x7fb1080d07e0 2023-07-14T11:53:28.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.153+0000 7fb0fdffb700 1 -- 172.21.15.104:0/3947141032 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb100017020 con 0x7fb1080d07e0 2023-07-14T11:53:28.139 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:53:28.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.153+0000 7fb10e1c3700 1 -- 172.21.15.104:0/3947141032 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb0ec066320 msgr2=0x7fb0ec0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:28.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.153+0000 7fb10e1c3700 1 --2- 172.21.15.104:0/3947141032 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb0ec066320 0x7fb0ec0687d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb1040090b0 tx=0x7fb10401c040 comp rx=0 tx=0).stop 2023-07-14T11:53:28.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.153+0000 7fb10e1c3700 1 -- 172.21.15.104:0/3947141032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1080d07e0 msgr2=0x7fb1080b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:28.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.153+0000 7fb10e1c3700 1 --2- 172.21.15.104:0/3947141032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1080d07e0 0x7fb1080b4970 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb10000cc70 tx=0x7fb10000a340 comp rx=0 tx=0).stop 2023-07-14T11:53:28.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.153+0000 7fb10e1c3700 1 -- 172.21.15.104:0/3947141032 shutdown_connections 2023-07-14T11:53:28.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.153+0000 7fb10e1c3700 1 --2- 172.21.15.104:0/3947141032 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb0ec066320 0x7fb0ec0687d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.153+0000 7fb10e1c3700 1 --2- 172.21.15.104:0/3947141032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1080d07e0 0x7fb1080b4970 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.153+0000 7fb10e1c3700 1 --2- 172.21.15.104:0/3947141032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1080cd090 0x7fb1080b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.153+0000 7fb10e1c3700 1 --2- 172.21.15.104:0/3947141032 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1080b0ee0 0x7fb10815e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.153+0000 7fb10e1c3700 1 -- 172.21.15.104:0/3947141032 >> 172.21.15.104:0/3947141032 conn(0x7fb108060c40 msgr2=0x7fb10809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:28.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.157+0000 7fb10e1c3700 1 -- 172.21.15.104:0/3947141032 shutdown_connections 2023-07-14T11:53:28.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.157+0000 7fb10e1c3700 1 -- 172.21.15.104:0/3947141032 wait complete. 2023-07-14T11:53:28.162 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:53:28.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.361+0000 7f48d759e700 1 Processor -- start 2023-07-14T11:53:28.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.361+0000 7f48d759e700 1 -- start start 2023-07-14T11:53:28.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.365+0000 7f48d759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48d8069ca0 0x7f48d80ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:28.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.365+0000 7f48d759e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f48d806aaf0 0x7f48d806e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:28.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.365+0000 7f48d759e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d815fd90 con 0x7f48d806a680 2023-07-14T11:53:28.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.365+0000 7f48d759e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d8159eb0 con 0x7f48d806aaf0 2023-07-14T11:53:28.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.365+0000 7f48d759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d815fa90 con 0x7f48d8069ca0 2023-07-14T11:53:28.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.365+0000 7f48d6d9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f48d806aaf0 0x7f48d806e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:28.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.365+0000 7f48d659c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48d8069ca0 0x7f48d80ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:28.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.365+0000 7f48d5d9b700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f48d806a680 0x7f48d8160220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:33772/0 (socket says 172.21.15.104:33772) 2023-07-14T11:53:28.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.365+0000 7f48d5d9b700 1 -- 172.21.15.104:0/2595161559 learned_addr learned my addr 172.21.15.104:0/2595161559 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:28.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.365+0000 7f48d559a700 1 -- 172.21.15.104:0/2595161559 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001003481 0 0) 0x7f48d815fd90 con 0x7f48d806a680 2023-07-14T11:53:28.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.365+0000 7f48d559a700 1 -- 172.21.15.104:0/2595161559 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48c00035d0 con 0x7f48d806a680 2023-07-14T11:53:28.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.365+0000 7f48d6d9d700 1 -- 172.21.15.104:0/2595161559 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48d8069ca0 msgr2=0x7f48d80ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:28.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.365+0000 7f48d6d9d700 1 --2- 172.21.15.104:0/2595161559 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48d8069ca0 0x7f48d80ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.365+0000 7f48d6d9d700 1 -- 172.21.15.104:0/2595161559 >> v1:172.21.15.121:6789/0 conn(0x7f48d806a680 legacy=0x7f48d8160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:28.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.369+0000 7f48d6d9d700 1 -- 172.21.15.104:0/2595161559 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d8160930 con 0x7f48d806aaf0 2023-07-14T11:53:28.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.369+0000 7f48d6d9d700 1 --2- 172.21.15.104:0/2595161559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48d806aaf0 0x7f48d806e960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009850 tx=0x7f48d0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e08e1cd1a1f897e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:28.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.369+0000 7f48d559a700 1 -- 172.21.15.104:0/2595161559 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0011830 con 0x7f48d806aaf0 2023-07-14T11:53:28.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.369+0000 7f48d559a700 1 -- 172.21.15.104:0/2595161559 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d0011990 con 0x7f48d806aaf0 2023-07-14T11:53:28.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.369+0000 7f48d559a700 1 -- 172.21.15.104:0/2595161559 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0011d00 con 0x7f48d806aaf0 2023-07-14T11:53:28.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.369+0000 7f48d759e700 1 -- 172.21.15.104:0/2595161559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48d806aaf0 msgr2=0x7f48d806e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:28.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.369+0000 7f48d759e700 1 --2- 172.21.15.104:0/2595161559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48d806aaf0 0x7f48d806e960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009850 tx=0x7f48d0009b60 comp rx=0 tx=0).stop 2023-07-14T11:53:28.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.369+0000 7f48d759e700 1 -- 172.21.15.104:0/2595161559 shutdown_connections 2023-07-14T11:53:28.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.369+0000 7f48d759e700 1 --2- 172.21.15.104:0/2595161559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48d806aaf0 0x7f48d806e960 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.369+0000 7f48d759e700 1 --2- 172.21.15.104:0/2595161559 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48d8069ca0 0x7f48d80ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.369+0000 7f48d759e700 1 -- 172.21.15.104:0/2595161559 >> 172.21.15.104:0/2595161559 conn(0x7f48d806fa70 msgr2=0x7f48d806fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:28.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.369+0000 7f48d759e700 1 -- 172.21.15.104:0/2595161559 shutdown_connections 2023-07-14T11:53:28.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.369+0000 7f48d759e700 1 -- 172.21.15.104:0/2595161559 wait complete. 2023-07-14T11:53:28.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.373+0000 7f48d759e700 1 Processor -- start 2023-07-14T11:53:28.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.373+0000 7f48d759e700 1 -- start start 2023-07-14T11:53:28.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.373+0000 7f48d759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8069ca0 0x7f48d80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:28.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.373+0000 7f48d759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48d806a680 0x7f48d80c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:28.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.373+0000 7f48d759e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48d806aaf0 0x7f48d8156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:28.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.373+0000 7f48d759e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80af790 con 0x7f48d806aaf0 2023-07-14T11:53:28.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.373+0000 7f48d759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80c82b0 con 0x7f48d8069ca0 2023-07-14T11:53:28.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.373+0000 7f48d759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80b26f0 con 0x7f48d806a680 2023-07-14T11:53:28.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48d6d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48d806aaf0 0x7f48d8156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:28.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48d6d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48d806aaf0 0x7f48d8156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46402/0 (socket says 172.21.15.104:46402) 2023-07-14T11:53:28.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48d6d9d700 1 -- 172.21.15.104:0/3537168587 learned_addr learned my addr 172.21.15.104:0/3537168587 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:28.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48d659c700 1 --2- 172.21.15.104:0/3537168587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8069ca0 0x7f48d80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:28.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48d5d9b700 1 --2- 172.21.15.104:0/3537168587 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48d806a680 0x7f48d80c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:28.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48d6d9d700 1 -- 172.21.15.104:0/3537168587 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48d806a680 msgr2=0x7f48d80c7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:28.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48d6d9d700 1 --2- 172.21.15.104:0/3537168587 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48d806a680 0x7f48d80c7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48d6d9d700 1 -- 172.21.15.104:0/3537168587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8069ca0 msgr2=0x7f48d80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:28.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48d6d9d700 1 --2- 172.21.15.104:0/3537168587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8069ca0 0x7f48d80b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48d6d9d700 1 -- 172.21.15.104:0/3537168587 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d00094b0 con 0x7f48d806aaf0 2023-07-14T11:53:28.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48d659c700 1 --2- 172.21.15.104:0/3537168587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8069ca0 0x7f48d80b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:28.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48d6d9d700 1 --2- 172.21.15.104:0/3537168587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48d806aaf0 0x7f48d8156280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f48d00090b0 tx=0x7f48d0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:28.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48c6ffd700 1 -- 172.21.15.104:0/3537168587 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0004590 con 0x7f48d806aaf0 2023-07-14T11:53:28.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48c6ffd700 1 -- 172.21.15.104:0/3537168587 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d00036c0 con 0x7f48d806aaf0 2023-07-14T11:53:28.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48c6ffd700 1 -- 172.21.15.104:0/3537168587 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0003a30 con 0x7f48d806aaf0 2023-07-14T11:53:28.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48d759e700 1 -- 172.21.15.104:0/3537168587 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d80b29d0 con 0x7f48d806aaf0 2023-07-14T11:53:28.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48d759e700 1 -- 172.21.15.104:0/3537168587 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d80b2cb0 con 0x7f48d806aaf0 2023-07-14T11:53:28.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.377+0000 7f48d759e700 1 -- 172.21.15.104:0/3537168587 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d80af8d0 con 0x7f48d806aaf0 2023-07-14T11:53:28.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.381+0000 7f48c6ffd700 1 -- 172.21.15.104:0/3537168587 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f48d0003b90 con 0x7f48d806aaf0 2023-07-14T11:53:28.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.381+0000 7f48c6ffd700 1 --2- 172.21.15.104:0/3537168587 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f48b8066320 0x7f48b80687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:28.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.381+0000 7f48c6ffd700 1 -- 172.21.15.104:0/3537168587 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f48d0021030 con 0x7f48d806aaf0 2023-07-14T11:53:28.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.381+0000 7f48d659c700 1 --2- 172.21.15.104:0/3537168587 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f48b8066320 0x7f48b80687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:28.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.385+0000 7f48d659c700 1 --2- 172.21.15.104:0/3537168587 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f48b8066320 0x7f48b80687d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f48c80074c0 tx=0x7f48c800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:28.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.385+0000 7f48c6ffd700 1 -- 172.21.15.104:0/3537168587 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f48d80af8d0 con 0x7f48d806aaf0 2023-07-14T11:53:28.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.553+0000 7f48d759e700 1 -- 172.21.15.104:0/3537168587 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f48d80af8d0 con 0x7f48d806aaf0 2023-07-14T11:53:28.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.557+0000 7f48c6ffd700 1 -- 172.21.15.104:0/3537168587 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7f48d80af8d0 con 0x7f48d806aaf0 2023-07-14T11:53:28.543 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:53:28.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.557+0000 7f48d759e700 1 -- 172.21.15.104:0/3537168587 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f48b8066320 msgr2=0x7f48b80687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:28.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.557+0000 7f48d759e700 1 --2- 172.21.15.104:0/3537168587 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f48b8066320 0x7f48b80687d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f48c80074c0 tx=0x7f48c800b040 comp rx=0 tx=0).stop 2023-07-14T11:53:28.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.557+0000 7f48d759e700 1 -- 172.21.15.104:0/3537168587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48d806aaf0 msgr2=0x7f48d8156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:28.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.557+0000 7f48d759e700 1 --2- 172.21.15.104:0/3537168587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48d806aaf0 0x7f48d8156280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f48d00090b0 tx=0x7f48d0004770 comp rx=0 tx=0).stop 2023-07-14T11:53:28.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.557+0000 7f48d759e700 1 -- 172.21.15.104:0/3537168587 shutdown_connections 2023-07-14T11:53:28.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.557+0000 7f48d759e700 1 --2- 172.21.15.104:0/3537168587 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f48b8066320 0x7f48b80687d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.557+0000 7f48d759e700 1 --2- 172.21.15.104:0/3537168587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48d806aaf0 0x7f48d8156280 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.557+0000 7f48d759e700 1 --2- 172.21.15.104:0/3537168587 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f48d806a680 0x7f48d80c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.557+0000 7f48d759e700 1 --2- 172.21.15.104:0/3537168587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8069ca0 0x7f48d80b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.557+0000 7f48d759e700 1 -- 172.21.15.104:0/3537168587 >> 172.21.15.104:0/3537168587 conn(0x7f48d806fa70 msgr2=0x7f48d80c97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:28.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.557+0000 7f48d759e700 1 -- 172.21.15.104:0/3537168587 shutdown_connections 2023-07-14T11:53:28.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.557+0000 7f48d759e700 1 -- 172.21.15.104:0/3537168587 wait complete. 2023-07-14T11:53:28.567 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:53:28.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.753+0000 7f76da88c700 1 Processor -- start 2023-07-14T11:53:28.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.753+0000 7f76da88c700 1 -- start start 2023-07-14T11:53:28.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.753+0000 7f76da88c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76d40b0ee0 0x7f76d40ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:28.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.753+0000 7f76da88c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f76d40cd090 0x7f76d40d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:28.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.753+0000 7f76da88c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f76d40d0610 0x7f76d415f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:28.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.753+0000 7f76da88c700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d40cd4e0 con 0x7f76d40d0610 2023-07-14T11:53:28.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.753+0000 7f76da88c700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d40d1a60 con 0x7f76d40cd090 2023-07-14T11:53:28.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.753+0000 7f76da88c700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40d1ef0 con 0x7f76d40b0ee0 2023-07-14T11:53:28.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.753+0000 7f76da08b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f76d40d0610 0x7f76d415f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:28.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.753+0000 7f76d9089700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f76d40cd090 0x7f76d40d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:28.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.753+0000 7f76d988a700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76d40b0ee0 0x7f76d40ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:28.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76d9089700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f76d40cd090 0x7f76d40d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:49510/0 (socket says 172.21.15.104:49510) 2023-07-14T11:53:28.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76d9089700 1 -- 172.21.15.104:0/1111291682 learned_addr learned my addr 172.21.15.104:0/1111291682 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:28.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76d9089700 1 -- 172.21.15.104:0/1111291682 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76d40b0ee0 msgr2=0x7f76d40ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:28.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76d9089700 1 --2- 172.21.15.104:0/1111291682 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76d40b0ee0 0x7f76d40ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76d9089700 1 -- 172.21.15.104:0/1111291682 >> v2:172.21.15.104:3300/0 conn(0x7f76d40d0610 msgr2=0x7f76d415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:28.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76d9089700 1 --2- 172.21.15.104:0/1111291682 >> v2:172.21.15.104:3300/0 conn(0x7f76d40d0610 0x7f76d415f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76d9089700 1 -- 172.21.15.104:0/1111291682 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d40d0e50 con 0x7f76d40cd090 2023-07-14T11:53:28.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76d9089700 1 --2- 172.21.15.104:0/1111291682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76d40cd090 0x7f76d40d00d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009910 tx=0x7f76c4009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca71524681df3c06 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:28.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76d8888700 1 -- 172.21.15.104:0/1111291682 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4011830 con 0x7f76d40cd090 2023-07-14T11:53:28.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76d8888700 1 -- 172.21.15.104:0/1111291682 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c4011990 con 0x7f76d40cd090 2023-07-14T11:53:28.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76d8888700 1 -- 172.21.15.104:0/1111291682 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4011d40 con 0x7f76d40cd090 2023-07-14T11:53:28.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76da88c700 1 -- 172.21.15.104:0/1111291682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76d40cd090 msgr2=0x7f76d40d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:28.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76da88c700 1 --2- 172.21.15.104:0/1111291682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76d40cd090 0x7f76d40d00d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009910 tx=0x7f76c4009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:28.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76da88c700 1 -- 172.21.15.104:0/1111291682 shutdown_connections 2023-07-14T11:53:28.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76da88c700 1 --2- 172.21.15.104:0/1111291682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76d40cd090 0x7f76d40d00d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76da88c700 1 --2- 172.21.15.104:0/1111291682 >> v2:172.21.15.104:3300/0 conn(0x7f76d40d0610 0x7f76d415f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76da88c700 1 --2- 172.21.15.104:0/1111291682 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76d40b0ee0 0x7f76d40ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.757+0000 7f76da88c700 1 -- 172.21.15.104:0/1111291682 >> 172.21.15.104:0/1111291682 conn(0x7f76d4060c40 msgr2=0x7f76d409b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:28.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.761+0000 7f76da88c700 1 -- 172.21.15.104:0/1111291682 shutdown_connections 2023-07-14T11:53:28.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.761+0000 7f76da88c700 1 -- 172.21.15.104:0/1111291682 wait complete. 2023-07-14T11:53:28.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.761+0000 7f76da88c700 1 Processor -- start 2023-07-14T11:53:28.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76da88c700 1 -- start start 2023-07-14T11:53:28.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76da88c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76d40b0ee0 0x7f76d415aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:28.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76da88c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76d40cd090 0x7f76d40b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:28.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76da88c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76d40d0610 0x7f76d40b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:28.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76da88c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40b3540 con 0x7f76d40b0ee0 2023-07-14T11:53:28.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76da88c700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40b28c0 con 0x7f76d40cd090 2023-07-14T11:53:28.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76da88c700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40b2ba0 con 0x7f76d40d0610 2023-07-14T11:53:28.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76d988a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76d40b0ee0 0x7f76d415aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:28.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76d988a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76d40b0ee0 0x7f76d415aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46408/0 (socket says 172.21.15.104:46408) 2023-07-14T11:53:28.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76d988a700 1 -- 172.21.15.104:0/4073000110 learned_addr learned my addr 172.21.15.104:0/4073000110 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:28.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76d9089700 1 --2- 172.21.15.104:0/4073000110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76d40cd090 0x7f76d40b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:28.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76da08b700 1 --2- 172.21.15.104:0/4073000110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76d40d0610 0x7f76d40b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:28.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76d988a700 1 -- 172.21.15.104:0/4073000110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76d40d0610 msgr2=0x7f76d40b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:28.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76d988a700 1 --2- 172.21.15.104:0/4073000110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76d40d0610 0x7f76d40b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76d988a700 1 -- 172.21.15.104:0/4073000110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76d40cd090 msgr2=0x7f76d40b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:28.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76d988a700 1 --2- 172.21.15.104:0/4073000110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76d40cd090 0x7f76d40b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76d988a700 1 -- 172.21.15.104:0/4073000110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c4009570 con 0x7f76d40b0ee0 2023-07-14T11:53:28.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76d988a700 1 --2- 172.21.15.104:0/4073000110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76d40b0ee0 0x7f76d415aae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f76d000e7c0 tx=0x7f76d000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:28.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76ca7fc700 1 -- 172.21.15.104:0/4073000110 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0013680 con 0x7f76d40b0ee0 2023-07-14T11:53:28.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76ca7fc700 1 -- 172.21.15.104:0/4073000110 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d00137e0 con 0x7f76d40b0ee0 2023-07-14T11:53:28.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76da88c700 1 -- 172.21.15.104:0/4073000110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d40b2e80 con 0x7f76d40b0ee0 2023-07-14T11:53:28.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.765+0000 7f76ca7fc700 1 -- 172.21.15.104:0/4073000110 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0013b90 con 0x7f76d40b0ee0 2023-07-14T11:53:28.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.769+0000 7f76da88c700 1 -- 172.21.15.104:0/4073000110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d40a0040 con 0x7f76d40b0ee0 2023-07-14T11:53:28.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.769+0000 7f76da88c700 1 -- 172.21.15.104:0/4073000110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d40b37b0 con 0x7f76d40b0ee0 2023-07-14T11:53:28.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.769+0000 7f76ca7fc700 1 -- 172.21.15.104:0/4073000110 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f76d0013cf0 con 0x7f76d40b0ee0 2023-07-14T11:53:28.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.773+0000 7f76ca7fc700 1 --2- 172.21.15.104:0/4073000110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f76b80662f0 0x7f76b80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:28.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.773+0000 7f76ca7fc700 1 -- 172.21.15.104:0/4073000110 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f76d0083eb0 con 0x7f76d40b0ee0 2023-07-14T11:53:28.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.773+0000 7f76d9089700 1 --2- 172.21.15.104:0/4073000110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f76b80662f0 0x7f76b80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:28.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.773+0000 7f76d9089700 1 --2- 172.21.15.104:0/4073000110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f76b80662f0 0x7f76b80687a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f76c4014040 tx=0x7f76c4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:28.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.777+0000 7f76ca7fc700 1 -- 172.21.15.104:0/4073000110 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f76d0050fb0 con 0x7f76d40b0ee0 2023-07-14T11:53:28.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.949+0000 7f76da88c700 1 -- 172.21.15.104:0/4073000110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76d404e500 con 0x7f76d40b0ee0 2023-07-14T11:53:28.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.953+0000 7f76ca7fc700 1 -- 172.21.15.104:0/4073000110 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f76d00545d0 con 0x7f76d40b0ee0 2023-07-14T11:53:28.939 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:53:28.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.953+0000 7f76da88c700 1 -- 172.21.15.104:0/4073000110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f76b80662f0 msgr2=0x7f76b80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:28.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.953+0000 7f76da88c700 1 --2- 172.21.15.104:0/4073000110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f76b80662f0 0x7f76b80687a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f76c4014040 tx=0x7f76c4008030 comp rx=0 tx=0).stop 2023-07-14T11:53:28.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.953+0000 7f76da88c700 1 -- 172.21.15.104:0/4073000110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76d40b0ee0 msgr2=0x7f76d415aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:28.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.953+0000 7f76da88c700 1 --2- 172.21.15.104:0/4073000110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76d40b0ee0 0x7f76d415aae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f76d000e7c0 tx=0x7f76d000ead0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.953+0000 7f76da88c700 1 -- 172.21.15.104:0/4073000110 shutdown_connections 2023-07-14T11:53:28.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.953+0000 7f76da88c700 1 --2- 172.21.15.104:0/4073000110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f76b80662f0 0x7f76b80687a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.953+0000 7f76da88c700 1 --2- 172.21.15.104:0/4073000110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76d40d0610 0x7f76d40b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.953+0000 7f76da88c700 1 --2- 172.21.15.104:0/4073000110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76d40cd090 0x7f76d40b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.953+0000 7f76da88c700 1 --2- 172.21.15.104:0/4073000110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76d40b0ee0 0x7f76d415aae0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:28.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.953+0000 7f76da88c700 1 -- 172.21.15.104:0/4073000110 >> 172.21.15.104:0/4073000110 conn(0x7f76d4060c40 msgr2=0x7f76d40af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:28.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.953+0000 7f76da88c700 1 -- 172.21.15.104:0/4073000110 shutdown_connections 2023-07-14T11:53:28.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:28.953+0000 7f76da88c700 1 -- 172.21.15.104:0/4073000110 wait complete. 2023-07-14T11:53:28.962 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:53:29.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.157+0000 7fea94c40700 1 Processor -- start 2023-07-14T11:53:29.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.157+0000 7fea94c40700 1 -- start start 2023-07-14T11:53:29.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.161+0000 7fea94c40700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fea900d07e0 0x7fea901669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.161+0000 7fea94c40700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea900d1a60 con 0x7fea900b0ee0 2023-07-14T11:53:29.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.161+0000 7fea94c40700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea900d1ef0 con 0x7fea900cd260 2023-07-14T11:53:29.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.161+0000 7fea94c40700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea900d0c60 con 0x7fea900d07e0 2023-07-14T11:53:29.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.161+0000 7fea8f7fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fea900b0ee0 0x7fea900ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:51162/0 (socket says 172.21.15.104:51162) 2023-07-14T11:53:29.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.161+0000 7fea8f7fe700 1 -- 172.21.15.104:0/3697618297 learned_addr learned my addr 172.21.15.104:0/3697618297 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:29.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.161+0000 7fea8ffff700 1 --2- 172.21.15.104:0/3697618297 >> v2:172.21.15.121:3300/0 conn(0x7fea900d07e0 0x7fea901669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.161+0000 7fea8e7fc700 1 -- 172.21.15.104:0/3697618297 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253988710 0 0) 0x7fea900d1a60 con 0x7fea900b0ee0 2023-07-14T11:53:29.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.161+0000 7fea8e7fc700 1 -- 172.21.15.104:0/3697618297 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea780035d0 con 0x7fea900b0ee0 2023-07-14T11:53:29.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.161+0000 7fea8ffff700 1 -- 172.21.15.104:0/3697618297 >> v1:172.21.15.121:6789/0 conn(0x7fea900cd260 legacy=0x7fea900d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.161+0000 7fea8ffff700 1 -- 172.21.15.104:0/3697618297 >> v1:172.21.15.104:6789/0 conn(0x7fea900b0ee0 legacy=0x7fea900ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.161+0000 7fea8ffff700 1 -- 172.21.15.104:0/3697618297 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea900d0e50 con 0x7fea900d07e0 2023-07-14T11:53:29.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.161+0000 7fea8ffff700 1 --2- 172.21.15.104:0/3697618297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea900d07e0 0x7fea901669b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fea88009d90 tx=0x7fea8800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=965baccf5ff08841 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:29.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.161+0000 7fea8e7fc700 1 -- 172.21.15.104:0/3697618297 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8800e070 con 0x7fea900d07e0 2023-07-14T11:53:29.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.161+0000 7fea8e7fc700 1 -- 172.21.15.104:0/3697618297 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8800bb20 con 0x7fea900d07e0 2023-07-14T11:53:29.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.165+0000 7fea8e7fc700 1 -- 172.21.15.104:0/3697618297 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88012430 con 0x7fea900d07e0 2023-07-14T11:53:29.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.165+0000 7fea94c40700 1 -- 172.21.15.104:0/3697618297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea900d07e0 msgr2=0x7fea901669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.165+0000 7fea94c40700 1 --2- 172.21.15.104:0/3697618297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea900d07e0 0x7fea901669b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fea88009d90 tx=0x7fea8800b330 comp rx=0 tx=0).stop 2023-07-14T11:53:29.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.165+0000 7fea94c40700 1 -- 172.21.15.104:0/3697618297 shutdown_connections 2023-07-14T11:53:29.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.165+0000 7fea94c40700 1 --2- 172.21.15.104:0/3697618297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea900d07e0 0x7fea901669b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.165+0000 7fea94c40700 1 -- 172.21.15.104:0/3697618297 >> 172.21.15.104:0/3697618297 conn(0x7fea90060c40 msgr2=0x7fea9009b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:29.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.165+0000 7fea94c40700 1 -- 172.21.15.104:0/3697618297 shutdown_connections 2023-07-14T11:53:29.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.165+0000 7fea94c40700 1 -- 172.21.15.104:0/3697618297 wait complete. 2023-07-14T11:53:29.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.165+0000 7fea94c40700 1 Processor -- start 2023-07-14T11:53:29.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea94c40700 1 -- start start 2023-07-14T11:53:29.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea94c40700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fea900b0ee0 0x7fea900c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea94c40700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea900cd260 0x7fea901f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea94c40700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea900d07e0 0x7fea901f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea94c40700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea900c8c70 con 0x7fea900cd260 2023-07-14T11:53:29.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea94c40700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea901584d0 con 0x7fea900d07e0 2023-07-14T11:53:29.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea94c40700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea901f5920 con 0x7fea900b0ee0 2023-07-14T11:53:29.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea8effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea900cd260 0x7fea901f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea8effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea900cd260 0x7fea901f53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46412/0 (socket says 172.21.15.104:46412) 2023-07-14T11:53:29.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea8f7fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fea900b0ee0 0x7fea900c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea8effd700 1 -- 172.21.15.104:0/172787735 learned_addr learned my addr 172.21.15.104:0/172787735 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:29.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea8ffff700 1 --2- 172.21.15.104:0/172787735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea900d07e0 0x7fea901f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea8effd700 1 -- 172.21.15.104:0/172787735 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fea900b0ee0 msgr2=0x7fea900c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea8effd700 1 --2- 172.21.15.104:0/172787735 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fea900b0ee0 0x7fea900c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea8effd700 1 -- 172.21.15.104:0/172787735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea900d07e0 msgr2=0x7fea901f0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea8effd700 1 --2- 172.21.15.104:0/172787735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea900d07e0 0x7fea901f0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea8effd700 1 -- 172.21.15.104:0/172787735 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea88009a70 con 0x7fea900cd260 2023-07-14T11:53:29.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea8ffff700 1 --2- 172.21.15.104:0/172787735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea900d07e0 0x7fea901f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:29.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea8effd700 1 --2- 172.21.15.104:0/172787735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea900cd260 0x7fea901f53e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fea8400c4d0 tx=0x7fea8400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:29.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.169+0000 7fea77fff700 1 -- 172.21.15.104:0/172787735 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea840052d0 con 0x7fea900cd260 2023-07-14T11:53:29.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.173+0000 7fea94c40700 1 -- 172.21.15.104:0/172787735 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea900c97d0 con 0x7fea900cd260 2023-07-14T11:53:29.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.173+0000 7fea77fff700 1 -- 172.21.15.104:0/172787735 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea84005430 con 0x7fea900cd260 2023-07-14T11:53:29.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.173+0000 7fea77fff700 1 -- 172.21.15.104:0/172787735 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea840055a0 con 0x7fea900cd260 2023-07-14T11:53:29.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.173+0000 7fea94c40700 1 -- 172.21.15.104:0/172787735 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea900ca010 con 0x7fea900cd260 2023-07-14T11:53:29.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.173+0000 7fea94c40700 1 -- 172.21.15.104:0/172787735 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea900ca230 con 0x7fea900cd260 2023-07-14T11:53:29.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.173+0000 7fea77fff700 1 -- 172.21.15.104:0/172787735 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fea840038c0 con 0x7fea900cd260 2023-07-14T11:53:29.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.177+0000 7fea77fff700 1 --2- 172.21.15.104:0/172787735 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fea7c0662f0 0x7fea7c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.177+0000 7fea77fff700 1 -- 172.21.15.104:0/172787735 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fea84083310 con 0x7fea900cd260 2023-07-14T11:53:29.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.177+0000 7fea8f7fe700 1 --2- 172.21.15.104:0/172787735 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fea7c0662f0 0x7fea7c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.177+0000 7fea8f7fe700 1 --2- 172.21.15.104:0/172787735 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fea7c0662f0 0x7fea7c0687a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fea80007fd0 tx=0x7fea80007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:29.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.181+0000 7fea77fff700 1 -- 172.21.15.104:0/172787735 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fea84050400 con 0x7fea900cd260 2023-07-14T11:53:29.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.353+0000 7fea94c40700 1 -- 172.21.15.104:0/172787735 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fea9004e500 con 0x7fea900cd260 2023-07-14T11:53:29.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.353+0000 7fea77fff700 1 -- 172.21.15.104:0/172787735 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fea84053a20 con 0x7fea900cd260 2023-07-14T11:53:29.342 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:53:29.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.357+0000 7fea94c40700 1 -- 172.21.15.104:0/172787735 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fea7c0662f0 msgr2=0x7fea7c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.357+0000 7fea94c40700 1 --2- 172.21.15.104:0/172787735 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fea7c0662f0 0x7fea7c0687a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fea80007fd0 tx=0x7fea80007de0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.357+0000 7fea94c40700 1 -- 172.21.15.104:0/172787735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea900cd260 msgr2=0x7fea901f53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.357+0000 7fea94c40700 1 --2- 172.21.15.104:0/172787735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea900cd260 0x7fea901f53e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fea8400c4d0 tx=0x7fea8400c7e0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.357+0000 7fea94c40700 1 -- 172.21.15.104:0/172787735 shutdown_connections 2023-07-14T11:53:29.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.357+0000 7fea94c40700 1 --2- 172.21.15.104:0/172787735 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fea7c0662f0 0x7fea7c0687a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.357+0000 7fea94c40700 1 --2- 172.21.15.104:0/172787735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea900cd260 0x7fea901f53e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.357+0000 7fea94c40700 1 --2- 172.21.15.104:0/172787735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea900d07e0 0x7fea901f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.357+0000 7fea94c40700 1 --2- 172.21.15.104:0/172787735 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fea900b0ee0 0x7fea900c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.357+0000 7fea94c40700 1 -- 172.21.15.104:0/172787735 >> 172.21.15.104:0/172787735 conn(0x7fea90060c40 msgr2=0x7fea900b6200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:29.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.357+0000 7fea94c40700 1 -- 172.21.15.104:0/172787735 shutdown_connections 2023-07-14T11:53:29.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.357+0000 7fea94c40700 1 -- 172.21.15.104:0/172787735 wait complete. 2023-07-14T11:53:29.364 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:53:29.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.549+0000 7f4b8cf5b700 1 Processor -- start 2023-07-14T11:53:29.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.549+0000 7f4b8cf5b700 1 -- start start 2023-07-14T11:53:29.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b8cf5b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4b88160240 0x7f4b880cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b8cf5b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b880b0490 0x7f4b880ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b8cf5b700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b88163fe0 con 0x7f4b88160610 2023-07-14T11:53:29.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b8cf5b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b880c8c30 con 0x7f4b88160240 2023-07-14T11:53:29.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b8cf5b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b880ca9e0 con 0x7f4b880b0490 2023-07-14T11:53:29.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b7dd9b700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f4b88160610 0x7f4b880afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:51178/0 (socket says 172.21.15.104:51178) 2023-07-14T11:53:29.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b7dd9b700 1 -- 172.21.15.104:0/3225762832 learned_addr learned my addr 172.21.15.104:0/3225762832 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:29.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b8659c700 1 --2- 172.21.15.104:0/3225762832 >> v2:172.21.15.121:3300/0 conn(0x7f4b88160240 0x7f4b880cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b86d9d700 1 --2- 172.21.15.104:0/3225762832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b880b0490 0x7f4b880ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 -- 172.21.15.104:0/3225762832 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433464393 0 0) 0x7f4b88163fe0 con 0x7f4b88160610 2023-07-14T11:53:29.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 -- 172.21.15.104:0/3225762832 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b740035d0 con 0x7f4b88160610 2023-07-14T11:53:29.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 -- 172.21.15.104:0/3225762832 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2300840817 0 0) 0x7f4b740035d0 con 0x7f4b88160610 2023-07-14T11:53:29.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 -- 172.21.15.104:0/3225762832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b880b0490 msgr2=0x7f4b880ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:29.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 --2- 172.21.15.104:0/3225762832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b880b0490 0x7f4b880ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 -- 172.21.15.104:0/3225762832 >> v2:172.21.15.121:3300/0 conn(0x7f4b88160240 msgr2=0x7f4b880cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:29.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 --2- 172.21.15.104:0/3225762832 >> v2:172.21.15.121:3300/0 conn(0x7f4b88160240 0x7f4b880cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 -- 172.21.15.104:0/3225762832 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b88164230 con 0x7f4b88160610 2023-07-14T11:53:29.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 -- 172.21.15.104:0/3225762832 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f4b700019f0 con 0x7f4b88160610 2023-07-14T11:53:29.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b86d9d700 1 --2- 172.21.15.104:0/3225762832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b880b0490 0x7f4b880ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:29.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 -- 172.21.15.104:0/3225762832 >> v1:172.21.15.104:6789/0 conn(0x7f4b88160610 legacy=0x7f4b880afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 --2- 172.21.15.104:0/3225762832 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b74003a20 0x7f4b74003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 --2- 172.21.15.104:0/3225762832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b74004870 0x7f4b74004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 --2- 172.21.15.104:0/3225762832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b880b0490 0x7f4b74005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 -- 172.21.15.104:0/3225762832 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b880c8c30 con 0x7f4b74003a20 2023-07-14T11:53:29.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 -- 172.21.15.104:0/3225762832 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b880ca9e0 con 0x7f4b74004870 2023-07-14T11:53:29.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b85d9b700 1 -- 172.21.15.104:0/3225762832 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7400ba30 con 0x7f4b880b0490 2023-07-14T11:53:29.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b8659c700 1 --2- 172.21.15.104:0/3225762832 >> v2:172.21.15.121:3300/0 conn(0x7f4b88160240 0x7f4b880cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:29.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b8659c700 1 --2- 172.21.15.104:0/3225762832 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b74003a20 0x7f4b74003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.553+0000 7f4b7dd9b700 1 --2- 172.21.15.104:0/3225762832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b74004870 0x7f4b74004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b86d9d700 1 --2- 172.21.15.104:0/3225762832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b880b0490 0x7f4b74005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b8659c700 1 -- 172.21.15.104:0/3225762832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b880b0490 msgr2=0x7f4b74005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:29.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b8659c700 1 --2- 172.21.15.104:0/3225762832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b880b0490 0x7f4b74005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b8659c700 1 -- 172.21.15.104:0/3225762832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b74004870 msgr2=0x7f4b74004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:29.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b8659c700 1 --2- 172.21.15.104:0/3225762832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b74004870 0x7f4b74004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b8659c700 1 -- 172.21.15.104:0/3225762832 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b800094b0 con 0x7f4b74003a20 2023-07-14T11:53:29.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b8659c700 1 --2- 172.21.15.104:0/3225762832 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b74003a20 0x7f4b74003e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009d20 tx=0x7f4b740058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292877639acb1ac7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:29.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b85d9b700 1 -- 172.21.15.104:0/3225762832 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78009e70 con 0x7f4b74003a20 2023-07-14T11:53:29.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b85d9b700 1 -- 172.21.15.104:0/3225762832 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7800e9a0 con 0x7f4b74003a20 2023-07-14T11:53:29.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b85d9b700 1 -- 172.21.15.104:0/3225762832 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7800ed10 con 0x7f4b74003a20 2023-07-14T11:53:29.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b8cf5b700 1 -- 172.21.15.104:0/3225762832 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b74003a20 msgr2=0x7f4b74003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b8cf5b700 1 --2- 172.21.15.104:0/3225762832 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b74003a20 0x7f4b74003e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009d20 tx=0x7f4b740058a0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b86d9d700 1 -- 172.21.15.104:0/3225762832 reap_dead start 2023-07-14T11:53:29.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b8cf5b700 1 -- 172.21.15.104:0/3225762832 shutdown_connections 2023-07-14T11:53:29.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b8cf5b700 1 -- 172.21.15.104:0/3225762832 >> 172.21.15.104:0/3225762832 conn(0x7f4b880cad80 msgr2=0x7f4b880cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:29.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b8cf5b700 1 -- 172.21.15.104:0/3225762832 shutdown_connections 2023-07-14T11:53:29.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.557+0000 7f4b8cf5b700 1 -- 172.21.15.104:0/3225762832 wait complete. 2023-07-14T11:53:29.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.561+0000 7f4b8cf5b700 1 Processor -- start 2023-07-14T11:53:29.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.561+0000 7f4b8cf5b700 1 -- start start 2023-07-14T11:53:29.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b8cf5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b74003a20 0x7f4b881f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b8cf5b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b8815c0d0 0x7f4b8815c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b8cf5b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b88161710 0x7f4b88161b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b8cf5b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8815ca80 con 0x7f4b8815c0d0 2023-07-14T11:53:29.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b8cf5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b880c8260 con 0x7f4b74003a20 2023-07-14T11:53:29.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b8cf5b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b881f5c30 con 0x7f4b88161710 2023-07-14T11:53:29.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b7dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b8815c0d0 0x7f4b8815c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b7dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b8815c0d0 0x7f4b8815c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46434/0 (socket says 172.21.15.104:46434) 2023-07-14T11:53:29.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b7dd9b700 1 -- 172.21.15.104:0/997486632 learned_addr learned my addr 172.21.15.104:0/997486632 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:29.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b86d9d700 1 --2- 172.21.15.104:0/997486632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b88161710 0x7f4b88161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b8659c700 1 --2- 172.21.15.104:0/997486632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b74003a20 0x7f4b881f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b7dd9b700 1 -- 172.21.15.104:0/997486632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b88161710 msgr2=0x7f4b88161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b7dd9b700 1 --2- 172.21.15.104:0/997486632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b88161710 0x7f4b88161b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b7dd9b700 1 -- 172.21.15.104:0/997486632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b74003a20 msgr2=0x7f4b881f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b7dd9b700 1 --2- 172.21.15.104:0/997486632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b74003a20 0x7f4b881f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b7dd9b700 1 -- 172.21.15.104:0/997486632 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b880c71c0 con 0x7f4b8815c0d0 2023-07-14T11:53:29.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b8659c700 1 --2- 172.21.15.104:0/997486632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b74003a20 0x7f4b881f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:29.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b7dd9b700 1 --2- 172.21.15.104:0/997486632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b8815c0d0 0x7f4b8815c540 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4b70001bb0 tx=0x7f4b7000a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:29.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b7f7fe700 1 -- 172.21.15.104:0/997486632 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7000ab80 con 0x7f4b8815c0d0 2023-07-14T11:53:29.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b7f7fe700 1 -- 172.21.15.104:0/997486632 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7000ace0 con 0x7f4b8815c0d0 2023-07-14T11:53:29.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b8cf5b700 1 -- 172.21.15.104:0/997486632 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b880c74a0 con 0x7f4b8815c0d0 2023-07-14T11:53:29.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b7f7fe700 1 -- 172.21.15.104:0/997486632 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b70013630 con 0x7f4b8815c0d0 2023-07-14T11:53:29.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.565+0000 7f4b8cf5b700 1 -- 172.21.15.104:0/997486632 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b880c7a60 con 0x7f4b8815c0d0 2023-07-14T11:53:29.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.569+0000 7f4b8cf5b700 1 -- 172.21.15.104:0/997486632 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8804dcb0 con 0x7f4b8815c0d0 2023-07-14T11:53:29.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.569+0000 7f4b7f7fe700 1 -- 172.21.15.104:0/997486632 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f4b70013790 con 0x7f4b8815c0d0 2023-07-14T11:53:29.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.573+0000 7f4b7f7fe700 1 --2- 172.21.15.104:0/997486632 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4b68066320 0x7f4b680687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.573+0000 7f4b7f7fe700 1 -- 172.21.15.104:0/997486632 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4b70084ac0 con 0x7f4b8815c0d0 2023-07-14T11:53:29.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.573+0000 7f4b8659c700 1 --2- 172.21.15.104:0/997486632 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4b68066320 0x7f4b680687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.573+0000 7f4b8659c700 1 --2- 172.21.15.104:0/997486632 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4b68066320 0x7f4b680687d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4b780090b0 tx=0x7f4b7801e040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:29.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.577+0000 7f4b7f7fe700 1 -- 172.21.15.104:0/997486632 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f4b70051c40 con 0x7f4b8815c0d0 2023-07-14T11:53:29.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.749+0000 7f4b8cf5b700 1 -- 172.21.15.104:0/997486632 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4b880b0bc0 con 0x7f4b8815c0d0 2023-07-14T11:53:29.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.749+0000 7f4b7f7fe700 1 -- 172.21.15.104:0/997486632 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4b700170a0 con 0x7f4b8815c0d0 2023-07-14T11:53:29.738 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:53:29.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.753+0000 7f4b8cf5b700 1 -- 172.21.15.104:0/997486632 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4b68066320 msgr2=0x7f4b680687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.753+0000 7f4b8cf5b700 1 --2- 172.21.15.104:0/997486632 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4b68066320 0x7f4b680687d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4b780090b0 tx=0x7f4b7801e040 comp rx=0 tx=0).stop 2023-07-14T11:53:29.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.753+0000 7f4b8cf5b700 1 -- 172.21.15.104:0/997486632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b8815c0d0 msgr2=0x7f4b8815c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.753+0000 7f4b8cf5b700 1 --2- 172.21.15.104:0/997486632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b8815c0d0 0x7f4b8815c540 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4b70001bb0 tx=0x7f4b7000a270 comp rx=0 tx=0).stop 2023-07-14T11:53:29.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.753+0000 7f4b8cf5b700 1 -- 172.21.15.104:0/997486632 shutdown_connections 2023-07-14T11:53:29.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.753+0000 7f4b8cf5b700 1 --2- 172.21.15.104:0/997486632 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4b68066320 0x7f4b680687d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.753+0000 7f4b8cf5b700 1 --2- 172.21.15.104:0/997486632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b8815c0d0 0x7f4b8815c540 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.753+0000 7f4b8cf5b700 1 --2- 172.21.15.104:0/997486632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b88161710 0x7f4b88161b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.753+0000 7f4b8cf5b700 1 --2- 172.21.15.104:0/997486632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b74003a20 0x7f4b881f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.753+0000 7f4b8cf5b700 1 -- 172.21.15.104:0/997486632 >> 172.21.15.104:0/997486632 conn(0x7f4b880cad80 msgr2=0x7f4b8815e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:29.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.753+0000 7f4b8cf5b700 1 -- 172.21.15.104:0/997486632 shutdown_connections 2023-07-14T11:53:29.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.753+0000 7f4b8cf5b700 1 -- 172.21.15.104:0/997486632 wait complete. 2023-07-14T11:53:29.764 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:53:29.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.949+0000 7f036ea53700 1 Processor -- start 2023-07-14T11:53:29.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.949+0000 7f036ea53700 1 -- start start 2023-07-14T11:53:29.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036ea53700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f036815ffa0 0x7f036806d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036ea53700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f03680cb180 0x7f03680ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036ea53700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03680ca8a0 con 0x7f036806dc50 2023-07-14T11:53:29.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036ea53700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03680cbcd0 con 0x7f036815ffa0 2023-07-14T11:53:29.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036ea53700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03680cbf70 con 0x7f03680cb180 2023-07-14T11:53:29.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036da51700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f036815ffa0 0x7f036806d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036da51700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f036815ffa0 0x7f036806d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46436/0 (socket says 172.21.15.104:46436) 2023-07-14T11:53:29.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036da51700 1 -- 172.21.15.104:0/238451346 learned_addr learned my addr 172.21.15.104:0/238451346 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:29.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036e252700 1 --2- 172.21.15.104:0/238451346 >> v2:172.21.15.121:3300/0 conn(0x7f03680cb180 0x7f03680ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036da51700 1 -- 172.21.15.104:0/238451346 >> v2:172.21.15.121:3300/0 conn(0x7f03680cb180 msgr2=0x7f03680ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:29.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036da51700 1 --2- 172.21.15.104:0/238451346 >> v2:172.21.15.121:3300/0 conn(0x7f03680cb180 0x7f03680ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036da51700 1 -- 172.21.15.104:0/238451346 >> v1:172.21.15.121:6789/0 conn(0x7f036806dc50 legacy=0x7f03680caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:29.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036da51700 1 -- 172.21.15.104:0/238451346 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03680cc250 con 0x7f036815ffa0 2023-07-14T11:53:29.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036da51700 1 --2- 172.21.15.104:0/238451346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f036815ffa0 0x7f036806d710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0368163c80 tx=0x7f0364009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc974ab52f134ef server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:29.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036ca4f700 1 -- 172.21.15.104:0/238451346 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364004230 con 0x7f036815ffa0 2023-07-14T11:53:29.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036ca4f700 1 -- 172.21.15.104:0/238451346 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0364004390 con 0x7f036815ffa0 2023-07-14T11:53:29.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036ca4f700 1 -- 172.21.15.104:0/238451346 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03640036c0 con 0x7f036815ffa0 2023-07-14T11:53:29.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036ea53700 1 -- 172.21.15.104:0/238451346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f036815ffa0 msgr2=0x7f036806d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036ea53700 1 --2- 172.21.15.104:0/238451346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f036815ffa0 0x7f036806d710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0368163c80 tx=0x7f0364009bf0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036ea53700 1 -- 172.21.15.104:0/238451346 shutdown_connections 2023-07-14T11:53:29.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036ea53700 1 --2- 172.21.15.104:0/238451346 >> v2:172.21.15.121:3300/0 conn(0x7f03680cb180 0x7f03680ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036ea53700 1 --2- 172.21.15.104:0/238451346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f036815ffa0 0x7f036806d710 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036ea53700 1 -- 172.21.15.104:0/238451346 >> 172.21.15.104:0/238451346 conn(0x7f036806e960 msgr2=0x7f036806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:29.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.953+0000 7f036ea53700 1 -- 172.21.15.104:0/238451346 shutdown_connections 2023-07-14T11:53:29.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.957+0000 7f036ea53700 1 -- 172.21.15.104:0/238451346 wait complete. 2023-07-14T11:53:29.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.957+0000 7f036ea53700 1 Processor -- start 2023-07-14T11:53:29.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.957+0000 7f036ea53700 1 -- start start 2023-07-14T11:53:29.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036ea53700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f036806dc50 0x7f03680c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036ea53700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03680cb180 0x7f03680ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036ea53700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036815ffa0 0x7f03680ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036ea53700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03681f6610 con 0x7f03680cb180 2023-07-14T11:53:29.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036ea53700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03680c1a50 con 0x7f036815ffa0 2023-07-14T11:53:29.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036ea53700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03680c1d30 con 0x7f036806dc50 2023-07-14T11:53:29.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036d250700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03680cb180 0x7f03680ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036d250700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03680cb180 0x7f03680ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46448/0 (socket says 172.21.15.104:46448) 2023-07-14T11:53:29.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036d250700 1 -- 172.21.15.104:0/642742206 learned_addr learned my addr 172.21.15.104:0/642742206 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:29.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036da51700 1 --2- 172.21.15.104:0/642742206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f036806dc50 0x7f03680c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036e252700 1 --2- 172.21.15.104:0/642742206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036815ffa0 0x7f03680ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036d250700 1 -- 172.21.15.104:0/642742206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f036806dc50 msgr2=0x7f03680c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036d250700 1 --2- 172.21.15.104:0/642742206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f036806dc50 0x7f03680c6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036d250700 1 -- 172.21.15.104:0/642742206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036815ffa0 msgr2=0x7f03680ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:29.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036d250700 1 --2- 172.21.15.104:0/642742206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036815ffa0 0x7f03680ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:29.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036d250700 1 -- 172.21.15.104:0/642742206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0364009570 con 0x7f03680cb180 2023-07-14T11:53:29.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036e252700 1 --2- 172.21.15.104:0/642742206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036815ffa0 0x7f03680ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:29.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036d250700 1 --2- 172.21.15.104:0/642742206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03680cb180 0x7f03680ae480 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f035800b450 tx=0x7f035800b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:29.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f035e7fc700 1 -- 172.21.15.104:0/642742206 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358004020 con 0x7f03680cb180 2023-07-14T11:53:29.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f035e7fc700 1 -- 172.21.15.104:0/642742206 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0358004180 con 0x7f03680cb180 2023-07-14T11:53:29.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036ea53700 1 -- 172.21.15.104:0/642742206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03680c2010 con 0x7f03680cb180 2023-07-14T11:53:29.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f035e7fc700 1 -- 172.21.15.104:0/642742206 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358004530 con 0x7f03680cb180 2023-07-14T11:53:29.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.961+0000 7f036ea53700 1 -- 172.21.15.104:0/642742206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03680c2810 con 0x7f03680cb180 2023-07-14T11:53:29.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.965+0000 7f036ea53700 1 -- 172.21.15.104:0/642742206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03680c8500 con 0x7f03680cb180 2023-07-14T11:53:29.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.965+0000 7f035e7fc700 1 -- 172.21.15.104:0/642742206 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f03580036a0 con 0x7f03680cb180 2023-07-14T11:53:29.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.965+0000 7f035e7fc700 1 --2- 172.21.15.104:0/642742206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f034c0662f0 0x7f034c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:29.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.965+0000 7f035e7fc700 1 -- 172.21.15.104:0/642742206 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0358083c20 con 0x7f03680cb180 2023-07-14T11:53:29.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.965+0000 7f036da51700 1 --2- 172.21.15.104:0/642742206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f034c0662f0 0x7f034c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:29.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.969+0000 7f036da51700 1 --2- 172.21.15.104:0/642742206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f034c0662f0 0x7f034c0687a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0368163c80 tx=0x7f0364008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:29.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:29.973+0000 7f035e7fc700 1 -- 172.21.15.104:0/642742206 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f0358050c90 con 0x7f03680cb180 2023-07-14T11:53:30.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.157+0000 7f036ea53700 1 -- 172.21.15.104:0/642742206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f03680c2530 con 0x7f03680cb180 2023-07-14T11:53:30.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.157+0000 7f035e7fc700 1 -- 172.21.15.104:0/642742206 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f03680c2530 con 0x7f03680cb180 2023-07-14T11:53:30.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.157+0000 7f036ea53700 1 -- 172.21.15.104:0/642742206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f034c0662f0 msgr2=0x7f034c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.157+0000 7f036ea53700 1 --2- 172.21.15.104:0/642742206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f034c0662f0 0x7f034c0687a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0368163c80 tx=0x7f0364008030 comp rx=0 tx=0).stop 2023-07-14T11:53:30.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.157+0000 7f036ea53700 1 -- 172.21.15.104:0/642742206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03680cb180 msgr2=0x7f03680ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.157+0000 7f036ea53700 1 --2- 172.21.15.104:0/642742206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03680cb180 0x7f03680ae480 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f035800b450 tx=0x7f035800b760 comp rx=0 tx=0).stop 2023-07-14T11:53:30.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.157+0000 7f036ea53700 1 -- 172.21.15.104:0/642742206 shutdown_connections 2023-07-14T11:53:30.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.157+0000 7f036ea53700 1 --2- 172.21.15.104:0/642742206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f034c0662f0 0x7f034c0687a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.157+0000 7f036ea53700 1 --2- 172.21.15.104:0/642742206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03680cb180 0x7f03680ae480 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.157+0000 7f036ea53700 1 --2- 172.21.15.104:0/642742206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036815ffa0 0x7f03680ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.157+0000 7f036ea53700 1 --2- 172.21.15.104:0/642742206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f036806dc50 0x7f03680c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.157+0000 7f036ea53700 1 -- 172.21.15.104:0/642742206 >> 172.21.15.104:0/642742206 conn(0x7f036806e960 msgr2=0x7f03680c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:30.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.161+0000 7f036ea53700 1 -- 172.21.15.104:0/642742206 shutdown_connections 2023-07-14T11:53:30.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.161+0000 7f036ea53700 1 -- 172.21.15.104:0/642742206 wait complete. 2023-07-14T11:53:30.149 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:53:30.169 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:53:30.170 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:53:30.185 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:30.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.421+0000 7f8782a48700 1 Processor -- start 2023-07-14T11:53:30.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.421+0000 7f8782a48700 1 -- start start 2023-07-14T11:53:30.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.425+0000 7f8782a48700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f877c159f20 0x7f877c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:30.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.425+0000 7f8782a48700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f877c0caf70 0x7f877c15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:30.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.425+0000 7f8782a48700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f877c0b0ef0 con 0x7f877c0c9d90 2023-07-14T11:53:30.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.425+0000 7f8782a48700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877c0ce9c0 con 0x7f877c159f20 2023-07-14T11:53:30.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.425+0000 7f8782a48700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877c0cee50 con 0x7f877c0caf70 2023-07-14T11:53:30.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.425+0000 7f8781a46700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f877c0c9d90 0x7f877c0ca160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35790/0 (socket says 172.21.15.104:35790) 2023-07-14T11:53:30.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.425+0000 7f8781a46700 1 -- 172.21.15.104:0/25997401 learned_addr learned my addr 172.21.15.104:0/25997401 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:30.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.425+0000 7f8781245700 1 --2- 172.21.15.104:0/25997401 >> v2:172.21.15.104:3300/0 conn(0x7f877c159f20 0x7f877c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:30.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.425+0000 7f8782247700 1 --2- 172.21.15.104:0/25997401 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f877c0caf70 0x7f877c15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:30.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.425+0000 7f8780a44700 1 -- 172.21.15.104:0/25997401 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097250574 0 0) 0x7f877c0b0ef0 con 0x7f877c0c9d90 2023-07-14T11:53:30.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.425+0000 7f8780a44700 1 -- 172.21.15.104:0/25997401 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87640035d0 con 0x7f877c0c9d90 2023-07-14T11:53:30.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.425+0000 7f8781245700 1 -- 172.21.15.104:0/25997401 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f877c0caf70 msgr2=0x7f877c15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:30.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.425+0000 7f8781245700 1 --2- 172.21.15.104:0/25997401 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f877c0caf70 0x7f877c15d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.425+0000 7f8781245700 1 -- 172.21.15.104:0/25997401 >> v1:172.21.15.104:6789/0 conn(0x7f877c0c9d90 legacy=0x7f877c0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.425+0000 7f8781245700 1 -- 172.21.15.104:0/25997401 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c0cb4f0 con 0x7f877c159f20 2023-07-14T11:53:30.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.429+0000 7f8781245700 1 --2- 172.21.15.104:0/25997401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f877c159f20 0x7f877c0caa30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8770009850 tx=0x7f8770009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02d4496f731887e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:30.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.429+0000 7f8780a44700 1 -- 172.21.15.104:0/25997401 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770011830 con 0x7f877c159f20 2023-07-14T11:53:30.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.429+0000 7f8780a44700 1 -- 172.21.15.104:0/25997401 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8770011990 con 0x7f877c159f20 2023-07-14T11:53:30.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.429+0000 7f8780a44700 1 -- 172.21.15.104:0/25997401 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770011d00 con 0x7f877c159f20 2023-07-14T11:53:30.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.429+0000 7f8782a48700 1 -- 172.21.15.104:0/25997401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f877c159f20 msgr2=0x7f877c0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.429+0000 7f8782a48700 1 --2- 172.21.15.104:0/25997401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f877c159f20 0x7f877c0caa30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8770009850 tx=0x7f8770009b60 comp rx=0 tx=0).stop 2023-07-14T11:53:30.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.429+0000 7f8782a48700 1 -- 172.21.15.104:0/25997401 shutdown_connections 2023-07-14T11:53:30.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.429+0000 7f8782a48700 1 --2- 172.21.15.104:0/25997401 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f877c0caf70 0x7f877c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.429+0000 7f8782a48700 1 --2- 172.21.15.104:0/25997401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f877c159f20 0x7f877c0caa30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.429+0000 7f8782a48700 1 -- 172.21.15.104:0/25997401 >> 172.21.15.104:0/25997401 conn(0x7f877c06e960 msgr2=0x7f877c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:30.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.429+0000 7f8782a48700 1 -- 172.21.15.104:0/25997401 shutdown_connections 2023-07-14T11:53:30.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.429+0000 7f8782a48700 1 -- 172.21.15.104:0/25997401 wait complete. 2023-07-14T11:53:30.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.433+0000 7f8782a48700 1 Processor -- start 2023-07-14T11:53:30.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.433+0000 7f8782a48700 1 -- start start 2023-07-14T11:53:30.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.433+0000 7f8782a48700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f877c0c9d90 0x7f877c154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:30.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.433+0000 7f8782a48700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f877c0caf70 0x7f877c156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:30.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.433+0000 7f8782a48700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f877c159f20 0x7f877c0b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:30.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.433+0000 7f8782a48700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c156f50 con 0x7f877c0c9d90 2023-07-14T11:53:30.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.433+0000 7f8782a48700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c157230 con 0x7f877c159f20 2023-07-14T11:53:30.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.433+0000 7f8782a48700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877c157510 con 0x7f877c0caf70 2023-07-14T11:53:30.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.433+0000 7f8781a46700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f877c0c9d90 0x7f877c154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:30.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f8781a46700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f877c0c9d90 0x7f877c154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42656/0 (socket says 172.21.15.104:42656) 2023-07-14T11:53:30.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f8781a46700 1 -- 172.21.15.104:0/2249763646 learned_addr learned my addr 172.21.15.104:0/2249763646 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:30.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f8782247700 1 --2- 172.21.15.104:0/2249763646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f877c159f20 0x7f877c0b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:30.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f8781245700 1 --2- 172.21.15.104:0/2249763646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f877c0caf70 0x7f877c156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:30.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f8781a46700 1 -- 172.21.15.104:0/2249763646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f877c0caf70 msgr2=0x7f877c156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f8781a46700 1 --2- 172.21.15.104:0/2249763646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f877c0caf70 0x7f877c156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f8781a46700 1 -- 172.21.15.104:0/2249763646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f877c159f20 msgr2=0x7f877c0b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f8781a46700 1 --2- 172.21.15.104:0/2249763646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f877c159f20 0x7f877c0b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f8781a46700 1 -- 172.21.15.104:0/2249763646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87700094b0 con 0x7f877c0c9d90 2023-07-14T11:53:30.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f8781a46700 1 --2- 172.21.15.104:0/2249763646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f877c0c9d90 0x7f877c154dd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f876c00c4d0 tx=0x7f876c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:30.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f877a7fc700 1 -- 172.21.15.104:0/2249763646 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c005d90 con 0x7f877c0c9d90 2023-07-14T11:53:30.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f877a7fc700 1 -- 172.21.15.104:0/2249763646 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c00ce80 con 0x7f877c0c9d90 2023-07-14T11:53:30.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f8782a48700 1 -- 172.21.15.104:0/2249763646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877c1577f0 con 0x7f877c0c9d90 2023-07-14T11:53:30.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f877a7fc700 1 -- 172.21.15.104:0/2249763646 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c0072d0 con 0x7f877c0c9d90 2023-07-14T11:53:30.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f8782a48700 1 -- 172.21.15.104:0/2249763646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877c0c3c00 con 0x7f877c0c9d90 2023-07-14T11:53:30.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.437+0000 7f8782a48700 1 -- 172.21.15.104:0/2249763646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877c157d10 con 0x7f877c0c9d90 2023-07-14T11:53:30.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.441+0000 7f877a7fc700 1 -- 172.21.15.104:0/2249763646 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f876c007430 con 0x7f877c0c9d90 2023-07-14T11:53:30.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.441+0000 7f877a7fc700 1 --2- 172.21.15.104:0/2249763646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87680662f0 0x7f87680687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:30.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.441+0000 7f877a7fc700 1 -- 172.21.15.104:0/2249763646 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f876c0832b0 con 0x7f877c0c9d90 2023-07-14T11:53:30.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.441+0000 7f8781245700 1 --2- 172.21.15.104:0/2249763646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87680662f0 0x7f87680687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:30.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.441+0000 7f8781245700 1 --2- 172.21.15.104:0/2249763646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87680662f0 0x7f87680687a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87700090b0 tx=0x7f8770008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:30.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.449+0000 7f877a7fc700 1 -- 172.21.15.104:0/2249763646 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f876c050460 con 0x7f877c0c9d90 2023-07-14T11:53:30.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.633+0000 7f8782a48700 1 -- 172.21.15.104:0/2249763646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f877c0b05d0 con 0x7f877c0c9d90 2023-07-14T11:53:30.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.637+0000 7f877a7fc700 1 -- 172.21.15.104:0/2249763646 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20156 (secure 0 0 0) 0x7f876c053a80 con 0x7f877c0c9d90 2023-07-14T11:53:30.626 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:30.626 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":28,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:20.091434+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:30.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.641+0000 7f8782a48700 1 -- 172.21.15.104:0/2249763646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87680662f0 msgr2=0x7f87680687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.641+0000 7f8782a48700 1 --2- 172.21.15.104:0/2249763646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87680662f0 0x7f87680687a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87700090b0 tx=0x7f8770008030 comp rx=0 tx=0).stop 2023-07-14T11:53:30.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.641+0000 7f8782a48700 1 -- 172.21.15.104:0/2249763646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f877c0c9d90 msgr2=0x7f877c154dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.641+0000 7f8782a48700 1 --2- 172.21.15.104:0/2249763646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f877c0c9d90 0x7f877c154dd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f876c00c4d0 tx=0x7f876c00c7e0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.641+0000 7f8782a48700 1 -- 172.21.15.104:0/2249763646 shutdown_connections 2023-07-14T11:53:30.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.641+0000 7f8782a48700 1 --2- 172.21.15.104:0/2249763646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87680662f0 0x7f87680687a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.641+0000 7f8782a48700 1 --2- 172.21.15.104:0/2249763646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f877c159f20 0x7f877c0b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.641+0000 7f8782a48700 1 --2- 172.21.15.104:0/2249763646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f877c0caf70 0x7f877c156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.641+0000 7f8782a48700 1 --2- 172.21.15.104:0/2249763646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f877c0c9d90 0x7f877c154dd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.641+0000 7f8782a48700 1 -- 172.21.15.104:0/2249763646 >> 172.21.15.104:0/2249763646 conn(0x7f877c06e960 msgr2=0x7f877c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:30.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.641+0000 7f8782a48700 1 -- 172.21.15.104:0/2249763646 shutdown_connections 2023-07-14T11:53:30.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.641+0000 7f8782a48700 1 -- 172.21.15.104:0/2249763646 wait complete. 2023-07-14T11:53:30.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:53:30.651 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:53:30.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.841+0000 7f5560346700 1 Processor -- start 2023-07-14T11:53:30.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.841+0000 7f5560346700 1 -- start start 2023-07-14T11:53:30.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.845+0000 7f5560346700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f55580d0210 0x7f55580d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:30.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.845+0000 7f5560346700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5558071d10 con 0x7f55580b1c50 2023-07-14T11:53:30.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.845+0000 7f5560346700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555815c420 con 0x7f55580cf000 2023-07-14T11:53:30.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.845+0000 7f5560346700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5558070840 con 0x7f55580d0210 2023-07-14T11:53:30.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.845+0000 7f555e0e2700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f55580b1c50 0x7f5558071600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35796/0 (socket says 172.21.15.104:35796) 2023-07-14T11:53:30.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.845+0000 7f555d8e1700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f55580d0210 0x7f55580d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:30.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.845+0000 7f555e0e2700 1 -- 172.21.15.104:0/3933383472 learned_addr learned my addr 172.21.15.104:0/3933383472 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:30.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.845+0000 7f555d8e1700 1 -- 172.21.15.104:0/3933383472 >> v1:172.21.15.121:6789/0 conn(0x7f55580cf000 legacy=0x7f55580ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.845+0000 7f555d8e1700 1 -- 172.21.15.104:0/3933383472 >> v1:172.21.15.104:6789/0 conn(0x7f55580b1c50 legacy=0x7f5558071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.845+0000 7f555d8e1700 1 -- 172.21.15.104:0/3933383472 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5558070b20 con 0x7f55580d0210 2023-07-14T11:53:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.845+0000 7f555d0e0700 1 -- 172.21.15.104:0/3933383472 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888963242 0 0) 0x7f5554001110 con 0x7f55580b1c50 2023-07-14T11:53:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.845+0000 7f555d8e1700 1 --2- 172.21.15.104:0/3933383472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55580d0210 0x7f55580d0640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5548009850 tx=0x7f5548009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d1fe7499eadad6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.845+0000 7f555d0e0700 1 -- 172.21.15.104:0/3933383472 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548011830 con 0x7f55580d0210 2023-07-14T11:53:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.845+0000 7f555d0e0700 1 -- 172.21.15.104:0/3933383472 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5548011990 con 0x7f55580d0210 2023-07-14T11:53:30.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.845+0000 7f555d0e0700 1 -- 172.21.15.104:0/3933383472 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548011d20 con 0x7f55580d0210 2023-07-14T11:53:30.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.849+0000 7f5560346700 1 -- 172.21.15.104:0/3933383472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55580d0210 msgr2=0x7f55580d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.849+0000 7f5560346700 1 --2- 172.21.15.104:0/3933383472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55580d0210 0x7f55580d0640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5548009850 tx=0x7f5548009b60 comp rx=0 tx=0).stop 2023-07-14T11:53:30.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.849+0000 7f5560346700 1 -- 172.21.15.104:0/3933383472 shutdown_connections 2023-07-14T11:53:30.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.849+0000 7f5560346700 1 --2- 172.21.15.104:0/3933383472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55580d0210 0x7f55580d0640 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.849+0000 7f5560346700 1 -- 172.21.15.104:0/3933383472 >> 172.21.15.104:0/3933383472 conn(0x7f555809b9c0 msgr2=0x7f55580d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:30.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.849+0000 7f5560346700 1 -- 172.21.15.104:0/3933383472 shutdown_connections 2023-07-14T11:53:30.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.849+0000 7f5560346700 1 -- 172.21.15.104:0/3933383472 wait complete. 2023-07-14T11:53:30.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.849+0000 7f5560346700 1 Processor -- start 2023-07-14T11:53:30.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.849+0000 7f5560346700 1 -- start start 2023-07-14T11:53:30.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f5560346700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55580b1c50 0x7f55580cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:30.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f5560346700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55580cf000 0x7f555809f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:30.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f5560346700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55580d0210 0x7f555809f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:30.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f5560346700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555809d380 con 0x7f55580b1c50 2023-07-14T11:53:30.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f5560346700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555809dc20 con 0x7f55580d0210 2023-07-14T11:53:30.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f5560346700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55580ce3c0 con 0x7f55580cf000 2023-07-14T11:53:30.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f555e0e2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55580b1c50 0x7f55580cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:30.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f555e0e2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55580b1c50 0x7f55580cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42664/0 (socket says 172.21.15.104:42664) 2023-07-14T11:53:30.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f555e0e2700 1 -- 172.21.15.104:0/833097391 learned_addr learned my addr 172.21.15.104:0/833097391 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:30.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f555d8e1700 1 --2- 172.21.15.104:0/833097391 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55580cf000 0x7f555809f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:30.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f555e8e3700 1 --2- 172.21.15.104:0/833097391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55580d0210 0x7f555809f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:30.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f555e0e2700 1 -- 172.21.15.104:0/833097391 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55580cf000 msgr2=0x7f555809f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f555e0e2700 1 --2- 172.21.15.104:0/833097391 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55580cf000 0x7f555809f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f555e0e2700 1 -- 172.21.15.104:0/833097391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55580d0210 msgr2=0x7f555809f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f555e0e2700 1 --2- 172.21.15.104:0/833097391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55580d0210 0x7f555809f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f555e0e2700 1 -- 172.21.15.104:0/833097391 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55480094b0 con 0x7f55580b1c50 2023-07-14T11:53:30.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f555e0e2700 1 --2- 172.21.15.104:0/833097391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55580b1c50 0x7f55580cdae0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f555400c550 tx=0x7f555400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:30.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f554effd700 1 -- 172.21.15.104:0/833097391 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554004ca0 con 0x7f55580b1c50 2023-07-14T11:53:30.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f554effd700 1 -- 172.21.15.104:0/833097391 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5554004e00 con 0x7f55580b1c50 2023-07-14T11:53:30.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f5560346700 1 -- 172.21.15.104:0/833097391 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555809d5d0 con 0x7f55580b1c50 2023-07-14T11:53:30.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f554effd700 1 -- 172.21.15.104:0/833097391 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554007310 con 0x7f55580b1c50 2023-07-14T11:53:30.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.853+0000 7f5560346700 1 -- 172.21.15.104:0/833097391 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55580ce020 con 0x7f55580b1c50 2023-07-14T11:53:30.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.857+0000 7f5560346700 1 -- 172.21.15.104:0/833097391 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55580b1850 con 0x7f55580b1c50 2023-07-14T11:53:30.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.857+0000 7f554effd700 1 -- 172.21.15.104:0/833097391 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5554007470 con 0x7f55580b1c50 2023-07-14T11:53:30.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.857+0000 7f554effd700 1 --2- 172.21.15.104:0/833097391 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f553c066320 0x7f553c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:30.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.857+0000 7f554effd700 1 -- 172.21.15.104:0/833097391 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5554083af0 con 0x7f55580b1c50 2023-07-14T11:53:30.847 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.861+0000 7f555d8e1700 1 --2- 172.21.15.104:0/833097391 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f553c066320 0x7f553c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:30.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.861+0000 7f555d8e1700 1 --2- 172.21.15.104:0/833097391 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f553c066320 0x7f553c0687d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f55480090b0 tx=0x7f554801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:30.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:30.865+0000 7f554effd700 1 -- 172.21.15.104:0/833097391 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f5554050c70 con 0x7f55580b1c50 2023-07-14T11:53:31.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.037+0000 7f5560346700 1 -- 172.21.15.104:0/833097391 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5558071d10 con 0x7f55580b1c50 2023-07-14T11:53:31.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.037+0000 7f554effd700 1 -- 172.21.15.104:0/833097391 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5558071d10 con 0x7f55580b1c50 2023-07-14T11:53:31.025 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:53:31.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.041+0000 7f5560346700 1 -- 172.21.15.104:0/833097391 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f553c066320 msgr2=0x7f553c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:31.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.041+0000 7f5560346700 1 --2- 172.21.15.104:0/833097391 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f553c066320 0x7f553c0687d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f55480090b0 tx=0x7f554801c040 comp rx=0 tx=0).stop 2023-07-14T11:53:31.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.041+0000 7f5560346700 1 -- 172.21.15.104:0/833097391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55580b1c50 msgr2=0x7f55580cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:31.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.041+0000 7f5560346700 1 --2- 172.21.15.104:0/833097391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55580b1c50 0x7f55580cdae0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f555400c550 tx=0x7f555400c860 comp rx=0 tx=0).stop 2023-07-14T11:53:31.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.041+0000 7f5560346700 1 -- 172.21.15.104:0/833097391 shutdown_connections 2023-07-14T11:53:31.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.041+0000 7f5560346700 1 --2- 172.21.15.104:0/833097391 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f553c066320 0x7f553c0687d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.041+0000 7f5560346700 1 --2- 172.21.15.104:0/833097391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55580d0210 0x7f555809f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.041+0000 7f5560346700 1 --2- 172.21.15.104:0/833097391 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55580cf000 0x7f555809f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.041+0000 7f5560346700 1 --2- 172.21.15.104:0/833097391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55580b1c50 0x7f55580cdae0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.041+0000 7f5560346700 1 -- 172.21.15.104:0/833097391 >> 172.21.15.104:0/833097391 conn(0x7f555809b9c0 msgr2=0x7f55580b7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:31.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.041+0000 7f5560346700 1 -- 172.21.15.104:0/833097391 shutdown_connections 2023-07-14T11:53:31.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.041+0000 7f5560346700 1 -- 172.21.15.104:0/833097391 wait complete. 2023-07-14T11:53:31.047 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:53:31.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.237+0000 7f5be2115700 1 Processor -- start 2023-07-14T11:53:31.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.237+0000 7f5be2115700 1 -- start start 2023-07-14T11:53:31.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5be2115700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5bdc0b0ee0 0x7f5bdc0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:31.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5be2115700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f5bdc0d07e0 0x7f5bdc0af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:31.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5be2115700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bdc0cd4e0 con 0x7f5bdc0cd090 2023-07-14T11:53:31.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5be2115700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bdc0d0c60 con 0x7f5bdc0d07e0 2023-07-14T11:53:31.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5be2115700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bdc0d1a60 con 0x7f5bdc0b0ee0 2023-07-14T11:53:31.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5be0912700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f5bdc0cd090 0x7f5bdc0d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35802/0 (socket says 172.21.15.104:35802) 2023-07-14T11:53:31.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5be0912700 1 -- 172.21.15.104:0/800093736 learned_addr learned my addr 172.21.15.104:0/800093736 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:31.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5be1113700 1 --2- 172.21.15.104:0/800093736 >> v2:172.21.15.121:3300/0 conn(0x7f5bdc0b0ee0 0x7f5bdc0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:31.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5be1914700 1 --2- 172.21.15.104:0/800093736 >> v2:172.21.15.104:3300/0 conn(0x7f5bdc0d07e0 0x7f5bdc0af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:31.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5bd3fff700 1 -- 172.21.15.104:0/800093736 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518133149 0 0) 0x7f5bcc001110 con 0x7f5bdc0cd090 2023-07-14T11:53:31.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5bd3fff700 1 -- 172.21.15.104:0/800093736 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bc80035d0 con 0x7f5bdc0cd090 2023-07-14T11:53:31.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5be1914700 1 -- 172.21.15.104:0/800093736 >> v2:172.21.15.121:3300/0 conn(0x7f5bdc0b0ee0 msgr2=0x7f5bdc0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:31.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5be1914700 1 --2- 172.21.15.104:0/800093736 >> v2:172.21.15.121:3300/0 conn(0x7f5bdc0b0ee0 0x7f5bdc0ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5be1914700 1 -- 172.21.15.104:0/800093736 >> v1:172.21.15.104:6789/0 conn(0x7f5bdc0cd090 legacy=0x7f5bdc0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:31.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5be1914700 1 -- 172.21.15.104:0/800093736 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc0d1ef0 con 0x7f5bdc0d07e0 2023-07-14T11:53:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.241+0000 7f5be1113700 1 --2- 172.21.15.104:0/800093736 >> v2:172.21.15.121:3300/0 conn(0x7f5bdc0b0ee0 0x7f5bdc0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.245+0000 7f5be1914700 1 --2- 172.21.15.104:0/800093736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc0d07e0 0x7f5bdc0af0c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc08b380 tx=0x7f5bd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f7766c19244a23 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.245+0000 7f5bd3fff700 1 -- 172.21.15.104:0/800093736 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd400e070 con 0x7f5bdc0d07e0 2023-07-14T11:53:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.245+0000 7f5bd3fff700 1 -- 172.21.15.104:0/800093736 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd400bb20 con 0x7f5bdc0d07e0 2023-07-14T11:53:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.245+0000 7f5bd3fff700 1 -- 172.21.15.104:0/800093736 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4012430 con 0x7f5bdc0d07e0 2023-07-14T11:53:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.245+0000 7f5be2115700 1 -- 172.21.15.104:0/800093736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc0d07e0 msgr2=0x7f5bdc0af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.245+0000 7f5be2115700 1 --2- 172.21.15.104:0/800093736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc0d07e0 0x7f5bdc0af0c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc08b380 tx=0x7f5bd400b330 comp rx=0 tx=0).stop 2023-07-14T11:53:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.245+0000 7f5be2115700 1 -- 172.21.15.104:0/800093736 shutdown_connections 2023-07-14T11:53:31.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.245+0000 7f5be2115700 1 --2- 172.21.15.104:0/800093736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc0d07e0 0x7f5bdc0af0c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.245+0000 7f5be2115700 1 --2- 172.21.15.104:0/800093736 >> v2:172.21.15.121:3300/0 conn(0x7f5bdc0b0ee0 0x7f5bdc0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.245+0000 7f5be2115700 1 -- 172.21.15.104:0/800093736 >> 172.21.15.104:0/800093736 conn(0x7f5bdc060c40 msgr2=0x7f5bdc09b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:31.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.245+0000 7f5be2115700 1 -- 172.21.15.104:0/800093736 shutdown_connections 2023-07-14T11:53:31.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.245+0000 7f5be2115700 1 -- 172.21.15.104:0/800093736 wait complete. 2023-07-14T11:53:31.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.249+0000 7f5be2115700 1 Processor -- start 2023-07-14T11:53:31.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.249+0000 7f5be2115700 1 -- start start 2023-07-14T11:53:31.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.249+0000 7f5be2115700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc0b0ee0 0x7f5bdc1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:31.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.249+0000 7f5be2115700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bdc0cd090 0x7f5bdc09fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:31.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.249+0000 7f5be2115700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bdc0d07e0 0x7f5bdc0c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:31.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.249+0000 7f5be2115700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc156bf0 con 0x7f5bdc0b0ee0 2023-07-14T11:53:31.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.249+0000 7f5be2115700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0b1fd0 con 0x7f5bdc0d07e0 2023-07-14T11:53:31.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.249+0000 7f5be2115700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0b22b0 con 0x7f5bdc0cd090 2023-07-14T11:53:31.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.249+0000 7f5be1113700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc0b0ee0 0x7f5bdc1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:31.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5be1113700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc0b0ee0 0x7f5bdc1566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42690/0 (socket says 172.21.15.104:42690) 2023-07-14T11:53:31.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5be1113700 1 -- 172.21.15.104:0/3576198536 learned_addr learned my addr 172.21.15.104:0/3576198536 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:31.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5be0912700 1 --2- 172.21.15.104:0/3576198536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bdc0cd090 0x7f5bdc09fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:31.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5be1914700 1 --2- 172.21.15.104:0/3576198536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bdc0d07e0 0x7f5bdc0c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:31.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5be1113700 1 -- 172.21.15.104:0/3576198536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bdc0cd090 msgr2=0x7f5bdc09fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:31.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5be1113700 1 --2- 172.21.15.104:0/3576198536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bdc0cd090 0x7f5bdc09fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5be1113700 1 -- 172.21.15.104:0/3576198536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bdc0d07e0 msgr2=0x7f5bdc0c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:31.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5be1113700 1 --2- 172.21.15.104:0/3576198536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bdc0d07e0 0x7f5bdc0c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5be1113700 1 -- 172.21.15.104:0/3576198536 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd80094b0 con 0x7f5bdc0b0ee0 2023-07-14T11:53:31.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5be1914700 1 --2- 172.21.15.104:0/3576198536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bdc0d07e0 0x7f5bdc0c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:31.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5be1113700 1 --2- 172.21.15.104:0/3576198536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc0b0ee0 0x7f5bdc1566b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800ebe0 tx=0x7f5bd8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:31.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5bd1ffb700 1 -- 172.21.15.104:0/3576198536 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd800c8b0 con 0x7f5bdc0b0ee0 2023-07-14T11:53:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5bd1ffb700 1 -- 172.21.15.104:0/3576198536 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd800ca10 con 0x7f5bdc0b0ee0 2023-07-14T11:53:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5be2115700 1 -- 172.21.15.104:0/3576198536 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd4009a70 con 0x7f5bdc0b0ee0 2023-07-14T11:53:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5bd1ffb700 1 -- 172.21.15.104:0/3576198536 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd800cda0 con 0x7f5bdc0b0ee0 2023-07-14T11:53:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.253+0000 7f5be2115700 1 -- 172.21.15.104:0/3576198536 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc0b2870 con 0x7f5bdc0b0ee0 2023-07-14T11:53:31.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.257+0000 7f5be2115700 1 -- 172.21.15.104:0/3576198536 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bdc04e500 con 0x7f5bdc0b0ee0 2023-07-14T11:53:31.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.257+0000 7f5bd1ffb700 1 -- 172.21.15.104:0/3576198536 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5bd8013470 con 0x7f5bdc0b0ee0 2023-07-14T11:53:31.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.257+0000 7f5bd1ffb700 1 --2- 172.21.15.104:0/3576198536 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5bc00662f0 0x7f5bc00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:31.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.257+0000 7f5bd1ffb700 1 -- 172.21.15.104:0/3576198536 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5bd8084110 con 0x7f5bdc0b0ee0 2023-07-14T11:53:31.247 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.257+0000 7f5be0912700 1 --2- 172.21.15.104:0/3576198536 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5bc00662f0 0x7f5bc00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:31.247 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.261+0000 7f5be0912700 1 --2- 172.21.15.104:0/3576198536 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5bc00662f0 0x7f5bc00687a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc15a040 tx=0x7f5bcc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:31.251 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.265+0000 7f5bd1ffb700 1 -- 172.21.15.104:0/3576198536 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f5bd80512a0 con 0x7f5bdc0b0ee0 2023-07-14T11:53:31.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.437+0000 7f5be2115700 1 -- 172.21.15.104:0/3576198536 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5bdc0b02e0 con 0x7f5bdc0b0ee0 2023-07-14T11:53:31.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.441+0000 7f5bd1ffb700 1 -- 172.21.15.104:0/3576198536 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5bd80548c0 con 0x7f5bdc0b0ee0 2023-07-14T11:53:31.427 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:53:31.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.441+0000 7f5be2115700 1 -- 172.21.15.104:0/3576198536 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5bc00662f0 msgr2=0x7f5bc00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:31.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.441+0000 7f5be2115700 1 --2- 172.21.15.104:0/3576198536 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5bc00662f0 0x7f5bc00687a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc15a040 tx=0x7f5bcc009040 comp rx=0 tx=0).stop 2023-07-14T11:53:31.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.441+0000 7f5be2115700 1 -- 172.21.15.104:0/3576198536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc0b0ee0 msgr2=0x7f5bdc1566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:31.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.441+0000 7f5be2115700 1 --2- 172.21.15.104:0/3576198536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc0b0ee0 0x7f5bdc1566b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800ebe0 tx=0x7f5bd8009e30 comp rx=0 tx=0).stop 2023-07-14T11:53:31.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.441+0000 7f5be2115700 1 -- 172.21.15.104:0/3576198536 shutdown_connections 2023-07-14T11:53:31.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.441+0000 7f5be2115700 1 --2- 172.21.15.104:0/3576198536 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5bc00662f0 0x7f5bc00687a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.441+0000 7f5be2115700 1 --2- 172.21.15.104:0/3576198536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bdc0d07e0 0x7f5bdc0c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.441+0000 7f5be2115700 1 --2- 172.21.15.104:0/3576198536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bdc0cd090 0x7f5bdc09fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.441+0000 7f5be2115700 1 --2- 172.21.15.104:0/3576198536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bdc0b0ee0 0x7f5bdc1566b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.441+0000 7f5be2115700 1 -- 172.21.15.104:0/3576198536 >> 172.21.15.104:0/3576198536 conn(0x7f5bdc060c40 msgr2=0x7f5bdc0af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:31.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.445+0000 7f5be2115700 1 -- 172.21.15.104:0/3576198536 shutdown_connections 2023-07-14T11:53:31.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.445+0000 7f5be2115700 1 -- 172.21.15.104:0/3576198536 wait complete. 2023-07-14T11:53:31.454 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:53:31.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.637+0000 7f529d1ca700 1 Processor -- start 2023-07-14T11:53:31.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.637+0000 7f529d1ca700 1 -- start start 2023-07-14T11:53:31.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.637+0000 7f529d1ca700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5298159c60 0x7f529806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:31.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.637+0000 7f529d1ca700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f52980b0770 0x7f5298070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:31.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.637+0000 7f529d1ca700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529806da00 con 0x7f52980cab00 2023-07-14T11:53:31.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.637+0000 7f529d1ca700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529806dce0 con 0x7f52980b0770 2023-07-14T11:53:31.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.637+0000 7f529d1ca700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529806dfc0 con 0x7f5298159c60 2023-07-14T11:53:31.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.637+0000 7f529c9c9700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f52980cab00 0x7f52980c1ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35808/0 (socket says 172.21.15.104:35808) 2023-07-14T11:53:31.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.637+0000 7f5297fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5298159c60 0x7f529806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:31.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.637+0000 7f528f7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f52980b0770 0x7f5298070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:31.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.637+0000 7f529c9c9700 1 -- 172.21.15.104:0/444114810 learned_addr learned my addr 172.21.15.104:0/444114810 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:31.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f5297fff700 1 -- 172.21.15.104:0/444114810 >> v2:172.21.15.104:3300/0 conn(0x7f52980b0770 msgr2=0x7f5298070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:31.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f5297fff700 1 --2- 172.21.15.104:0/444114810 >> v2:172.21.15.104:3300/0 conn(0x7f52980b0770 0x7f5298070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f5297fff700 1 -- 172.21.15.104:0/444114810 >> v1:172.21.15.104:6789/0 conn(0x7f52980cab00 legacy=0x7f52980c1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:31.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f5297fff700 1 -- 172.21.15.104:0/444114810 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52980705e0 con 0x7f5298159c60 2023-07-14T11:53:31.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f52977fe700 1 -- 172.21.15.104:0/444114810 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106680634 0 0) 0x7f529806da00 con 0x7f52980cab00 2023-07-14T11:53:31.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f5297fff700 1 --2- 172.21.15.104:0/444114810 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5298159c60 0x7f529806d3b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5288009850 tx=0x7f5288009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82fb53b2787cdaff server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:31.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f52977fe700 1 -- 172.21.15.104:0/444114810 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288011830 con 0x7f5298159c60 2023-07-14T11:53:31.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f52977fe700 1 -- 172.21.15.104:0/444114810 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5288011990 con 0x7f5298159c60 2023-07-14T11:53:31.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f52977fe700 1 -- 172.21.15.104:0/444114810 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288011d40 con 0x7f5298159c60 2023-07-14T11:53:31.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f529d1ca700 1 -- 172.21.15.104:0/444114810 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5298159c60 msgr2=0x7f529806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:31.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f529d1ca700 1 --2- 172.21.15.104:0/444114810 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5298159c60 0x7f529806d3b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5288009850 tx=0x7f5288009b60 comp rx=0 tx=0).stop 2023-07-14T11:53:31.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f529d1ca700 1 -- 172.21.15.104:0/444114810 shutdown_connections 2023-07-14T11:53:31.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f529d1ca700 1 --2- 172.21.15.104:0/444114810 >> v2:172.21.15.104:3300/0 conn(0x7f52980b0770 0x7f5298070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f529d1ca700 1 --2- 172.21.15.104:0/444114810 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5298159c60 0x7f529806d3b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f529d1ca700 1 -- 172.21.15.104:0/444114810 >> 172.21.15.104:0/444114810 conn(0x7f52980afc60 msgr2=0x7f52980b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:31.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f529d1ca700 1 -- 172.21.15.104:0/444114810 shutdown_connections 2023-07-14T11:53:31.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.641+0000 7f529d1ca700 1 -- 172.21.15.104:0/444114810 wait complete. 2023-07-14T11:53:31.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.645+0000 7f529d1ca700 1 Processor -- start 2023-07-14T11:53:31.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.645+0000 7f529d1ca700 1 -- start start 2023-07-14T11:53:31.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f529d1ca700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52980b0770 0x7f52980c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:31.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f529d1ca700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52980cab00 0x7f52981ecd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:31.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f529d1ca700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5298159c60 0x7f52981ed280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:31.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f529d1ca700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52980c6440 con 0x7f52980cab00 2023-07-14T11:53:31.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f529d1ca700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52980c6720 con 0x7f52980b0770 2023-07-14T11:53:31.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f529d1ca700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52980c6a00 con 0x7f5298159c60 2023-07-14T11:53:31.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f528f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52980cab00 0x7f52981ecd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:31.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f528f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52980cab00 0x7f52981ecd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42704/0 (socket says 172.21.15.104:42704) 2023-07-14T11:53:31.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f528f7fe700 1 -- 172.21.15.104:0/3415412383 learned_addr learned my addr 172.21.15.104:0/3415412383 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:31.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f5297fff700 1 --2- 172.21.15.104:0/3415412383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52980b0770 0x7f52980c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:31.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f529c9c9700 1 --2- 172.21.15.104:0/3415412383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5298159c60 0x7f52981ed280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:31.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f528f7fe700 1 -- 172.21.15.104:0/3415412383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5298159c60 msgr2=0x7f52981ed280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:31.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f528f7fe700 1 --2- 172.21.15.104:0/3415412383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5298159c60 0x7f52981ed280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f528f7fe700 1 -- 172.21.15.104:0/3415412383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52980b0770 msgr2=0x7f52980c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:31.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f528f7fe700 1 --2- 172.21.15.104:0/3415412383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52980b0770 0x7f52980c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f528f7fe700 1 -- 172.21.15.104:0/3415412383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52880094b0 con 0x7f52980cab00 2023-07-14T11:53:31.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f528f7fe700 1 --2- 172.21.15.104:0/3415412383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52980cab00 0x7f52981ecd40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f528000e9d0 tx=0x7f528000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:31.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f52957fa700 1 -- 172.21.15.104:0/3415412383 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280009720 con 0x7f52980cab00 2023-07-14T11:53:31.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f52957fa700 1 -- 172.21.15.104:0/3415412383 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528000ee70 con 0x7f52980cab00 2023-07-14T11:53:31.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f529d1ca700 1 -- 172.21.15.104:0/3415412383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52980a7580 con 0x7f52980cab00 2023-07-14T11:53:31.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f52957fa700 1 -- 172.21.15.104:0/3415412383 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528000ca50 con 0x7f52980cab00 2023-07-14T11:53:31.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.649+0000 7f529d1ca700 1 -- 172.21.15.104:0/3415412383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52980c6ce0 con 0x7f52980cab00 2023-07-14T11:53:31.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.653+0000 7f529d1ca700 1 -- 172.21.15.104:0/3415412383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529804e500 con 0x7f52980cab00 2023-07-14T11:53:31.641 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.653+0000 7f52957fa700 1 -- 172.21.15.104:0/3415412383 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f528000cbb0 con 0x7f52980cab00 2023-07-14T11:53:31.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.657+0000 7f52957fa700 1 --2- 172.21.15.104:0/3415412383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52780662f0 0x7f52780687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:31.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.657+0000 7f52957fa700 1 -- 172.21.15.104:0/3415412383 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f52800840a0 con 0x7f52980cab00 2023-07-14T11:53:31.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.657+0000 7f5297fff700 1 --2- 172.21.15.104:0/3415412383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52780662f0 0x7f52780687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:31.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.657+0000 7f5297fff700 1 --2- 172.21.15.104:0/3415412383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52780662f0 0x7f52780687a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f52880090b0 tx=0x7f528801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:31.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.661+0000 7f52957fa700 1 -- 172.21.15.104:0/3415412383 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f5280054c20 con 0x7f52980cab00 2023-07-14T11:53:31.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.837+0000 7f529d1ca700 1 -- 172.21.15.104:0/3415412383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52980c9030 con 0x7f52980cab00 2023-07-14T11:53:31.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.837+0000 7f52957fa700 1 -- 172.21.15.104:0/3415412383 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f528000ce60 con 0x7f52980cab00 2023-07-14T11:53:31.826 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:53:31.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.841+0000 7f529d1ca700 1 -- 172.21.15.104:0/3415412383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52780662f0 msgr2=0x7f52780687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:31.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.841+0000 7f529d1ca700 1 --2- 172.21.15.104:0/3415412383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52780662f0 0x7f52780687a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f52880090b0 tx=0x7f528801c040 comp rx=0 tx=0).stop 2023-07-14T11:53:31.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.841+0000 7f529d1ca700 1 -- 172.21.15.104:0/3415412383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52980cab00 msgr2=0x7f52981ecd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:31.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.841+0000 7f529d1ca700 1 --2- 172.21.15.104:0/3415412383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52980cab00 0x7f52981ecd40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f528000e9d0 tx=0x7f528000ece0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.841+0000 7f529d1ca700 1 -- 172.21.15.104:0/3415412383 shutdown_connections 2023-07-14T11:53:31.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.841+0000 7f529d1ca700 1 --2- 172.21.15.104:0/3415412383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52780662f0 0x7f52780687a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.841+0000 7f529d1ca700 1 --2- 172.21.15.104:0/3415412383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52980cab00 0x7f52981ecd40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.841+0000 7f529d1ca700 1 --2- 172.21.15.104:0/3415412383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5298159c60 0x7f52981ed280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.841+0000 7f529d1ca700 1 --2- 172.21.15.104:0/3415412383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52980b0770 0x7f52980c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:31.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.841+0000 7f529d1ca700 1 -- 172.21.15.104:0/3415412383 >> 172.21.15.104:0/3415412383 conn(0x7f52980afc60 msgr2=0x7f52980c9310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:31.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.841+0000 7f529d1ca700 1 -- 172.21.15.104:0/3415412383 shutdown_connections 2023-07-14T11:53:31.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:31.841+0000 7f529d1ca700 1 -- 172.21.15.104:0/3415412383 wait complete. 2023-07-14T11:53:31.851 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:53:32.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.041+0000 7f1c37427700 1 Processor -- start 2023-07-14T11:53:32.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.041+0000 7f1c37427700 1 -- start start 2023-07-14T11:53:32.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.041+0000 7f1c37427700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1c300b0ee0 0x7f1c300ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.041+0000 7f1c37427700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c300cd4e0 con 0x7f1c300cd090 2023-07-14T11:53:32.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.041+0000 7f1c37427700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c300d1a60 con 0x7f1c300d07e0 2023-07-14T11:53:32.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.041+0000 7f1c37427700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c300d0c60 con 0x7f1c300b0ee0 2023-07-14T11:53:32.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.041+0000 7f1c35c24700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f1c300cd090 0x7f1c300d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35822/0 (socket says 172.21.15.104:35822) 2023-07-14T11:53:32.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.041+0000 7f1c35c24700 1 -- 172.21.15.104:0/3817305256 learned_addr learned my addr 172.21.15.104:0/3817305256 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:32.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.041+0000 7f1c36425700 1 --2- 172.21.15.104:0/3817305256 >> v2:172.21.15.121:3300/0 conn(0x7f1c300b0ee0 0x7f1c300ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.045+0000 7f1c36425700 1 -- 172.21.15.104:0/3817305256 >> v1:172.21.15.121:6789/0 conn(0x7f1c300d07e0 legacy=0x7f1c300af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.045+0000 7f1c36425700 1 -- 172.21.15.104:0/3817305256 >> v1:172.21.15.104:6789/0 conn(0x7f1c300cd090 legacy=0x7f1c300d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.045+0000 7f1c36425700 1 -- 172.21.15.104:0/3817305256 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c300d1ef0 con 0x7f1c300b0ee0 2023-07-14T11:53:32.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.045+0000 7f1c36425700 1 --2- 172.21.15.104:0/3817305256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c300b0ee0 0x7f1c300ccb50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c009850 tx=0x7f1c2c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=375c46e438f56ff server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:32.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.045+0000 7f1c35423700 1 -- 172.21.15.104:0/3817305256 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c011830 con 0x7f1c300b0ee0 2023-07-14T11:53:32.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.045+0000 7f1c35423700 1 -- 172.21.15.104:0/3817305256 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c2c011990 con 0x7f1c300b0ee0 2023-07-14T11:53:32.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.045+0000 7f1c35423700 1 -- 172.21.15.104:0/3817305256 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c011d20 con 0x7f1c300b0ee0 2023-07-14T11:53:32.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.045+0000 7f1c37427700 1 -- 172.21.15.104:0/3817305256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c300b0ee0 msgr2=0x7f1c300ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.045+0000 7f1c37427700 1 --2- 172.21.15.104:0/3817305256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c300b0ee0 0x7f1c300ccb50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c009850 tx=0x7f1c2c009b60 comp rx=0 tx=0).stop 2023-07-14T11:53:32.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.045+0000 7f1c37427700 1 -- 172.21.15.104:0/3817305256 shutdown_connections 2023-07-14T11:53:32.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.045+0000 7f1c37427700 1 --2- 172.21.15.104:0/3817305256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c300b0ee0 0x7f1c300ccb50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.045+0000 7f1c37427700 1 -- 172.21.15.104:0/3817305256 >> 172.21.15.104:0/3817305256 conn(0x7f1c30060c40 msgr2=0x7f1c3009b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:32.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.045+0000 7f1c37427700 1 -- 172.21.15.104:0/3817305256 shutdown_connections 2023-07-14T11:53:32.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.045+0000 7f1c37427700 1 -- 172.21.15.104:0/3817305256 wait complete. 2023-07-14T11:53:32.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.049+0000 7f1c37427700 1 Processor -- start 2023-07-14T11:53:32.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.049+0000 7f1c37427700 1 -- start start 2023-07-14T11:53:32.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c37427700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c300b0ee0 0x7f1c30070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c37427700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c300cd090 0x7f1c301f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c37427700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c300d07e0 0x7f1c300c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c37427700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c30071210 con 0x7f1c300d07e0 2023-07-14T11:53:32.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c37427700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c300714e0 con 0x7f1c300b0ee0 2023-07-14T11:53:32.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c37427700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c300b2b40 con 0x7f1c300cd090 2023-07-14T11:53:32.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c36c26700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c300d07e0 0x7f1c300c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c36c26700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c300d07e0 0x7f1c300c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42710/0 (socket says 172.21.15.104:42710) 2023-07-14T11:53:32.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c36c26700 1 -- 172.21.15.104:0/1865524935 learned_addr learned my addr 172.21.15.104:0/1865524935 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:32.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c36425700 1 --2- 172.21.15.104:0/1865524935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c300b0ee0 0x7f1c30070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c35c24700 1 --2- 172.21.15.104:0/1865524935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c300cd090 0x7f1c301f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c36c26700 1 -- 172.21.15.104:0/1865524935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c300cd090 msgr2=0x7f1c301f2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c36c26700 1 --2- 172.21.15.104:0/1865524935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c300cd090 0x7f1c301f2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c36c26700 1 -- 172.21.15.104:0/1865524935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c300b0ee0 msgr2=0x7f1c30070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c36c26700 1 --2- 172.21.15.104:0/1865524935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c300b0ee0 0x7f1c30070cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c36c26700 1 -- 172.21.15.104:0/1865524935 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c0094b0 con 0x7f1c300d07e0 2023-07-14T11:53:32.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c36c26700 1 --2- 172.21.15.104:0/1865524935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c300d07e0 0x7f1c300c6010 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800cc50 tx=0x7f1c2800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:32.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c26ffd700 1 -- 172.21.15.104:0/1865524935 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28003c20 con 0x7f1c300d07e0 2023-07-14T11:53:32.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c26ffd700 1 -- 172.21.15.104:0/1865524935 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c28003d80 con 0x7f1c300d07e0 2023-07-14T11:53:32.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c26ffd700 1 -- 172.21.15.104:0/1865524935 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2800aab0 con 0x7f1c300d07e0 2023-07-14T11:53:32.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c37427700 1 -- 172.21.15.104:0/1865524935 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c300b2e20 con 0x7f1c300d07e0 2023-07-14T11:53:32.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.053+0000 7f1c37427700 1 -- 172.21.15.104:0/1865524935 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c300b33e0 con 0x7f1c300d07e0 2023-07-14T11:53:32.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.057+0000 7f1c37427700 1 -- 172.21.15.104:0/1865524935 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3004e500 con 0x7f1c300d07e0 2023-07-14T11:53:32.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.057+0000 7f1c26ffd700 1 -- 172.21.15.104:0/1865524935 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1c2800ac10 con 0x7f1c300d07e0 2023-07-14T11:53:32.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.061+0000 7f1c26ffd700 1 --2- 172.21.15.104:0/1865524935 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1c140662f0 0x7f1c140687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.061+0000 7f1c26ffd700 1 -- 172.21.15.104:0/1865524935 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1c28084080 con 0x7f1c300d07e0 2023-07-14T11:53:32.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.061+0000 7f1c36425700 1 --2- 172.21.15.104:0/1865524935 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1c140662f0 0x7f1c140687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.061+0000 7f1c36425700 1 --2- 172.21.15.104:0/1865524935 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1c140662f0 0x7f1c140687a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c014040 tx=0x7f1c2c01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:32.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.065+0000 7f1c26ffd700 1 -- 172.21.15.104:0/1865524935 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f1c280b4720 con 0x7f1c300d07e0 2023-07-14T11:53:32.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.237+0000 7f1c37427700 1 -- 172.21.15.104:0/1865524935 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1c300b4d50 con 0x7f1c300d07e0 2023-07-14T11:53:32.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.237+0000 7f1c26ffd700 1 -- 172.21.15.104:0/1865524935 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1c28051170 con 0x7f1c300d07e0 2023-07-14T11:53:32.226 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:53:32.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.241+0000 7f1c37427700 1 -- 172.21.15.104:0/1865524935 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1c140662f0 msgr2=0x7f1c140687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.241+0000 7f1c37427700 1 --2- 172.21.15.104:0/1865524935 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1c140662f0 0x7f1c140687a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c014040 tx=0x7f1c2c01c040 comp rx=0 tx=0).stop 2023-07-14T11:53:32.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.241+0000 7f1c37427700 1 -- 172.21.15.104:0/1865524935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c300d07e0 msgr2=0x7f1c300c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.241+0000 7f1c37427700 1 --2- 172.21.15.104:0/1865524935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c300d07e0 0x7f1c300c6010 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800cc50 tx=0x7f1c2800cf60 comp rx=0 tx=0).stop 2023-07-14T11:53:32.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.241+0000 7f1c37427700 1 -- 172.21.15.104:0/1865524935 shutdown_connections 2023-07-14T11:53:32.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.241+0000 7f1c37427700 1 --2- 172.21.15.104:0/1865524935 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1c140662f0 0x7f1c140687a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.241+0000 7f1c37427700 1 --2- 172.21.15.104:0/1865524935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c300d07e0 0x7f1c300c6010 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.241+0000 7f1c37427700 1 --2- 172.21.15.104:0/1865524935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c300cd090 0x7f1c301f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.241+0000 7f1c37427700 1 --2- 172.21.15.104:0/1865524935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c300b0ee0 0x7f1c30070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.241+0000 7f1c37427700 1 -- 172.21.15.104:0/1865524935 >> 172.21.15.104:0/1865524935 conn(0x7f1c30060c40 msgr2=0x7f1c300af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:32.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.241+0000 7f1c37427700 1 -- 172.21.15.104:0/1865524935 shutdown_connections 2023-07-14T11:53:32.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.241+0000 7f1c37427700 1 -- 172.21.15.104:0/1865524935 wait complete. 2023-07-14T11:53:32.251 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:53:32.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.453+0000 7f1ddbb08700 1 Processor -- start 2023-07-14T11:53:32.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.453+0000 7f1ddbb08700 1 -- start start 2023-07-14T11:53:32.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.453+0000 7f1ddbb08700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1dd415c200 0x7f1dd40cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.453+0000 7f1ddbb08700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1dd40ad0f0 0x7f1dd40cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.453+0000 7f1ddbb08700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd40cbed0 con 0x7f1dd40afd80 2023-07-14T11:53:32.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.453+0000 7f1ddbb08700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd4071820 con 0x7f1dd415c200 2023-07-14T11:53:32.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.453+0000 7f1ddbb08700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd40ad760 con 0x7f1dd40ad0f0 2023-07-14T11:53:32.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1dd98a4700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f1dd40afd80 0x7f1dd40b0150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:47562/0 (socket says 172.21.15.104:47562) 2023-07-14T11:53:32.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1dd98a4700 1 -- 172.21.15.104:0/3059706215 learned_addr learned my addr 172.21.15.104:0/3059706215 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:32.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1dda0a5700 1 --2- 172.21.15.104:0/3059706215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1dd40ad0f0 0x7f1dd40cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1dd90a3700 1 --2- 172.21.15.104:0/3059706215 >> v2:172.21.15.121:3300/0 conn(0x7f1dd415c200 0x7f1dd40cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1dd88a2700 1 -- 172.21.15.104:0/3059706215 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080116889 0 0) 0x7f1dd40cbed0 con 0x7f1dd40afd80 2023-07-14T11:53:32.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1dd88a2700 1 -- 172.21.15.104:0/3059706215 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dc00035d0 con 0x7f1dd40afd80 2023-07-14T11:53:32.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1dda0a5700 1 -- 172.21.15.104:0/3059706215 >> v2:172.21.15.121:3300/0 conn(0x7f1dd415c200 msgr2=0x7f1dd40cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:32.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1dda0a5700 1 --2- 172.21.15.104:0/3059706215 >> v2:172.21.15.121:3300/0 conn(0x7f1dd415c200 0x7f1dd40cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1dda0a5700 1 -- 172.21.15.104:0/3059706215 >> v1:172.21.15.121:6789/0 conn(0x7f1dd40afd80 legacy=0x7f1dd40b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1dda0a5700 1 -- 172.21.15.104:0/3059706215 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd40ada40 con 0x7f1dd40ad0f0 2023-07-14T11:53:32.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1dda0a5700 1 --2- 172.21.15.104:0/3059706215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1dd40ad0f0 0x7f1dd40cdae0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009dc0 tx=0x7f1dcc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b196062a9a884b22 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:32.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1dd88a2700 1 -- 172.21.15.104:0/3059706215 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00e070 con 0x7f1dd40ad0f0 2023-07-14T11:53:32.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1dd88a2700 1 -- 172.21.15.104:0/3059706215 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dcc00bae0 con 0x7f1dd40ad0f0 2023-07-14T11:53:32.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1dd88a2700 1 -- 172.21.15.104:0/3059706215 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00be90 con 0x7f1dd40ad0f0 2023-07-14T11:53:32.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1ddbb08700 1 -- 172.21.15.104:0/3059706215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1dd40ad0f0 msgr2=0x7f1dd40cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.457+0000 7f1ddbb08700 1 --2- 172.21.15.104:0/3059706215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1dd40ad0f0 0x7f1dd40cdae0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009dc0 tx=0x7f1dcc00b330 comp rx=0 tx=0).stop 2023-07-14T11:53:32.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.461+0000 7f1ddbb08700 1 -- 172.21.15.104:0/3059706215 shutdown_connections 2023-07-14T11:53:32.447 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.461+0000 7f1ddbb08700 1 --2- 172.21.15.104:0/3059706215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1dd40ad0f0 0x7f1dd40cdae0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.447 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.461+0000 7f1ddbb08700 1 --2- 172.21.15.104:0/3059706215 >> v2:172.21.15.121:3300/0 conn(0x7f1dd415c200 0x7f1dd40cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.447 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.461+0000 7f1ddbb08700 1 -- 172.21.15.104:0/3059706215 >> 172.21.15.104:0/3059706215 conn(0x7f1dd40cad80 msgr2=0x7f1dd40cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:32.447 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.461+0000 7f1ddbb08700 1 -- 172.21.15.104:0/3059706215 shutdown_connections 2023-07-14T11:53:32.447 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.461+0000 7f1ddbb08700 1 -- 172.21.15.104:0/3059706215 wait complete. 2023-07-14T11:53:32.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.461+0000 7f1ddbb08700 1 Processor -- start 2023-07-14T11:53:32.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1ddbb08700 1 -- start start 2023-07-14T11:53:32.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1ddbb08700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1dd40ad0f0 0x7f1dd40c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1ddbb08700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd40afd80 0x7f1dd41f1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1ddbb08700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1dd415c200 0x7f1dd40c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1ddbb08700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd40c4d90 con 0x7f1dd40afd80 2023-07-14T11:53:32.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1ddbb08700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd415e160 con 0x7f1dd415c200 2023-07-14T11:53:32.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1ddbb08700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd40c27b0 con 0x7f1dd40ad0f0 2023-07-14T11:53:32.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1dd90a3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd40afd80 0x7f1dd41f1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1dd90a3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd40afd80 0x7f1dd41f1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42720/0 (socket says 172.21.15.104:42720) 2023-07-14T11:53:32.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1dd90a3700 1 -- 172.21.15.104:0/730051552 learned_addr learned my addr 172.21.15.104:0/730051552 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:32.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1dd98a4700 1 --2- 172.21.15.104:0/730051552 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1dd40ad0f0 0x7f1dd40c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1dda0a5700 1 --2- 172.21.15.104:0/730051552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1dd415c200 0x7f1dd40c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1dd90a3700 1 -- 172.21.15.104:0/730051552 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1dd40ad0f0 msgr2=0x7f1dd40c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1dd90a3700 1 --2- 172.21.15.104:0/730051552 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1dd40ad0f0 0x7f1dd40c2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1dd90a3700 1 -- 172.21.15.104:0/730051552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1dd415c200 msgr2=0x7f1dd40c29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1dd90a3700 1 --2- 172.21.15.104:0/730051552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1dd415c200 0x7f1dd40c29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1dd90a3700 1 -- 172.21.15.104:0/730051552 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc009a70 con 0x7f1dd40afd80 2023-07-14T11:53:32.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1dd90a3700 1 --2- 172.21.15.104:0/730051552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd40afd80 0x7f1dd41f1360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400e9a0 tx=0x7f1dc400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:32.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1dca7fc700 1 -- 172.21.15.104:0/730051552 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4009720 con 0x7f1dd40afd80 2023-07-14T11:53:32.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1dca7fc700 1 -- 172.21.15.104:0/730051552 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc4013680 con 0x7f1dd40afd80 2023-07-14T11:53:32.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1dca7fc700 1 -- 172.21.15.104:0/730051552 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4013a10 con 0x7f1dd40afd80 2023-07-14T11:53:32.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1ddbb08700 1 -- 172.21.15.104:0/730051552 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd415cc60 con 0x7f1dd40afd80 2023-07-14T11:53:32.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.465+0000 7f1ddbb08700 1 -- 172.21.15.104:0/730051552 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd415c980 con 0x7f1dd40afd80 2023-07-14T11:53:32.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.469+0000 7f1ddbb08700 1 -- 172.21.15.104:0/730051552 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd41f18a0 con 0x7f1dd40afd80 2023-07-14T11:53:32.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.469+0000 7f1dca7fc700 1 -- 172.21.15.104:0/730051552 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1dc4013b70 con 0x7f1dd40afd80 2023-07-14T11:53:32.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.473+0000 7f1dca7fc700 1 --2- 172.21.15.104:0/730051552 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1db80662f0 0x7f1db80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.459 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.473+0000 7f1dca7fc700 1 -- 172.21.15.104:0/730051552 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1dc4083f10 con 0x7f1dd40afd80 2023-07-14T11:53:32.459 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.473+0000 7f1dd98a4700 1 --2- 172.21.15.104:0/730051552 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1db80662f0 0x7f1db80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.473+0000 7f1dd98a4700 1 --2- 172.21.15.104:0/730051552 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1db80662f0 0x7f1db80687a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0007fd0 tx=0x7f1dd0007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:32.465 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.477+0000 7f1dca7fc700 1 -- 172.21.15.104:0/730051552 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f1dc4054b30 con 0x7f1dd40afd80 2023-07-14T11:53:32.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.649+0000 7f1ddbb08700 1 -- 172.21.15.104:0/730051552 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1dd404dcb0 con 0x7f1dd40afd80 2023-07-14T11:53:32.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.649+0000 7f1dca7fc700 1 -- 172.21.15.104:0/730051552 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1dc40546c0 con 0x7f1dd40afd80 2023-07-14T11:53:32.637 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:53:32.642 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.653+0000 7f1ddbb08700 1 -- 172.21.15.104:0/730051552 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1db80662f0 msgr2=0x7f1db80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.642 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.653+0000 7f1ddbb08700 1 --2- 172.21.15.104:0/730051552 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1db80662f0 0x7f1db80687a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0007fd0 tx=0x7f1dd0007de0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.642 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.653+0000 7f1ddbb08700 1 -- 172.21.15.104:0/730051552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd40afd80 msgr2=0x7f1dd41f1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.642 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.653+0000 7f1ddbb08700 1 --2- 172.21.15.104:0/730051552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd40afd80 0x7f1dd41f1360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400e9a0 tx=0x7f1dc400ecb0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.653+0000 7f1ddbb08700 1 -- 172.21.15.104:0/730051552 shutdown_connections 2023-07-14T11:53:32.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.653+0000 7f1ddbb08700 1 --2- 172.21.15.104:0/730051552 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1db80662f0 0x7f1db80687a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.653+0000 7f1ddbb08700 1 --2- 172.21.15.104:0/730051552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd40afd80 0x7f1dd41f1360 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.653+0000 7f1ddbb08700 1 --2- 172.21.15.104:0/730051552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1dd415c200 0x7f1dd40c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.653+0000 7f1ddbb08700 1 --2- 172.21.15.104:0/730051552 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1dd40ad0f0 0x7f1dd40c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.653+0000 7f1ddbb08700 1 -- 172.21.15.104:0/730051552 >> 172.21.15.104:0/730051552 conn(0x7f1dd40cad80 msgr2=0x7f1dd40ab8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:32.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.657+0000 7f1ddbb08700 1 -- 172.21.15.104:0/730051552 shutdown_connections 2023-07-14T11:53:32.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.657+0000 7f1ddbb08700 1 -- 172.21.15.104:0/730051552 wait complete. 2023-07-14T11:53:32.663 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:53:32.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.853+0000 7f2ef76c6700 1 Processor -- start 2023-07-14T11:53:32.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.853+0000 7f2ef76c6700 1 -- start start 2023-07-14T11:53:32.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.853+0000 7f2ef76c6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ef006d860 0x7f2ef00cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.853+0000 7f2ef76c6700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef00cc1b0 con 0x7f2ef006be70 2023-07-14T11:53:32.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.853+0000 7f2ef76c6700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef00cd690 con 0x7f2ef00c1e00 2023-07-14T11:53:32.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.853+0000 7f2ef76c6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00cd970 con 0x7f2ef006d860 2023-07-14T11:53:32.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ef4c61700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f2ef006be70 0x7f2ef006dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35834/0 (socket says 172.21.15.104:35834) 2023-07-14T11:53:32.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ef4c61700 1 -- 172.21.15.104:0/1935316656 learned_addr learned my addr 172.21.15.104:0/1935316656 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:32.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ef5462700 1 --2- 172.21.15.104:0/1935316656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ef006d860 0x7f2ef00cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ee7fff700 1 -- 172.21.15.104:0/1935316656 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321207743 0 0) 0x7f2ef00cc1b0 con 0x7f2ef006be70 2023-07-14T11:53:32.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ef5462700 1 -- 172.21.15.104:0/1935316656 >> v1:172.21.15.121:6789/0 conn(0x7f2ef00c1e00 legacy=0x7f2ef00cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ef5462700 1 -- 172.21.15.104:0/1935316656 >> v1:172.21.15.104:6789/0 conn(0x7f2ef006be70 legacy=0x7f2ef006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ef5462700 1 -- 172.21.15.104:0/1935316656 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef00cdc50 con 0x7f2ef006d860 2023-07-14T11:53:32.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ef5462700 1 --2- 172.21.15.104:0/1935316656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ef006d860 0x7f2ef00cb840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0098b0 tx=0x7f2eec009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5adade54c30b275d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:32.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ee7fff700 1 -- 172.21.15.104:0/1935316656 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec011830 con 0x7f2ef006d860 2023-07-14T11:53:32.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ee7fff700 1 -- 172.21.15.104:0/1935316656 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eec011990 con 0x7f2ef006d860 2023-07-14T11:53:32.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ee7fff700 1 -- 172.21.15.104:0/1935316656 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec011d20 con 0x7f2ef006d860 2023-07-14T11:53:32.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ef76c6700 1 -- 172.21.15.104:0/1935316656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ef006d860 msgr2=0x7f2ef00cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ef76c6700 1 --2- 172.21.15.104:0/1935316656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ef006d860 0x7f2ef00cb840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0098b0 tx=0x7f2eec009bc0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ef76c6700 1 -- 172.21.15.104:0/1935316656 shutdown_connections 2023-07-14T11:53:32.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ef76c6700 1 --2- 172.21.15.104:0/1935316656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ef006d860 0x7f2ef00cb840 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ef76c6700 1 -- 172.21.15.104:0/1935316656 >> 172.21.15.104:0/1935316656 conn(0x7f2ef00cad30 msgr2=0x7f2ef00cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:32.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ef76c6700 1 -- 172.21.15.104:0/1935316656 shutdown_connections 2023-07-14T11:53:32.847 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.857+0000 7f2ef76c6700 1 -- 172.21.15.104:0/1935316656 wait complete. 2023-07-14T11:53:32.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.861+0000 7f2ef76c6700 1 Processor -- start 2023-07-14T11:53:32.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.861+0000 7f2ef76c6700 1 -- start start 2023-07-14T11:53:32.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.861+0000 7f2ef76c6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ef006be70 0x7f2ef00acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.861+0000 7f2ef76c6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ef006d860 0x7f2ef00c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.861+0000 7f2ef76c6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ef00c1e00 0x7f2ef00c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.861+0000 7f2ef76c6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00c6720 con 0x7f2ef00c1e00 2023-07-14T11:53:32.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.861+0000 7f2ef76c6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef015ab50 con 0x7f2ef006be70 2023-07-14T11:53:32.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.861+0000 7f2ef76c6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef015ada0 con 0x7f2ef006d860 2023-07-14T11:53:32.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.861+0000 7f2ef5c63700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ef00c1e00 0x7f2ef00c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.861+0000 7f2ef5c63700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ef00c1e00 0x7f2ef00c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42722/0 (socket says 172.21.15.104:42722) 2023-07-14T11:53:32.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.861+0000 7f2ef5c63700 1 -- 172.21.15.104:0/1252895860 learned_addr learned my addr 172.21.15.104:0/1252895860 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:32.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.861+0000 7f2ef5462700 1 --2- 172.21.15.104:0/1252895860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ef006be70 0x7f2ef00acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.861+0000 7f2ef4c61700 1 --2- 172.21.15.104:0/1252895860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ef006d860 0x7f2ef00c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.865+0000 7f2ef5c63700 1 -- 172.21.15.104:0/1252895860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ef006d860 msgr2=0x7f2ef00c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.865+0000 7f2ef5c63700 1 --2- 172.21.15.104:0/1252895860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ef006d860 0x7f2ef00c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.865+0000 7f2ef5c63700 1 -- 172.21.15.104:0/1252895860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ef006be70 msgr2=0x7f2ef00acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:32.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.865+0000 7f2ef5c63700 1 --2- 172.21.15.104:0/1252895860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ef006be70 0x7f2ef00acc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:32.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.865+0000 7f2ef5c63700 1 -- 172.21.15.104:0/1252895860 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec0094b0 con 0x7f2ef00c1e00 2023-07-14T11:53:32.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.865+0000 7f2ef5c63700 1 --2- 172.21.15.104:0/1252895860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ef00c1e00 0x7f2ef00c77f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800cb50 tx=0x7f2ee800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:32.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.865+0000 7f2ee5ffb700 1 -- 172.21.15.104:0/1252895860 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8003c20 con 0x7f2ef00c1e00 2023-07-14T11:53:32.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.865+0000 7f2ee5ffb700 1 -- 172.21.15.104:0/1252895860 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee8003d80 con 0x7f2ef00c1e00 2023-07-14T11:53:32.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.865+0000 7f2ef76c6700 1 -- 172.21.15.104:0/1252895860 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef015b080 con 0x7f2ef00c1e00 2023-07-14T11:53:32.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.865+0000 7f2ee5ffb700 1 -- 172.21.15.104:0/1252895860 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee800aac0 con 0x7f2ef00c1e00 2023-07-14T11:53:32.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.865+0000 7f2ef76c6700 1 -- 172.21.15.104:0/1252895860 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef015b640 con 0x7f2ef00c1e00 2023-07-14T11:53:32.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.865+0000 7f2ef76c6700 1 -- 172.21.15.104:0/1252895860 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef004dcb0 con 0x7f2ef00c1e00 2023-07-14T11:53:32.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.869+0000 7f2ee5ffb700 1 -- 172.21.15.104:0/1252895860 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2ee800ac20 con 0x7f2ef00c1e00 2023-07-14T11:53:32.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.869+0000 7f2ee5ffb700 1 --2- 172.21.15.104:0/1252895860 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2ed40662f0 0x7f2ed40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:32.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.869+0000 7f2ee5ffb700 1 -- 172.21.15.104:0/1252895860 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2ee8083eb0 con 0x7f2ef00c1e00 2023-07-14T11:53:32.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.869+0000 7f2ef5462700 1 --2- 172.21.15.104:0/1252895860 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2ed40662f0 0x7f2ed40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:32.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.869+0000 7f2ef5462700 1 --2- 172.21.15.104:0/1252895860 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2ed40662f0 0x7f2ed40687a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0090b0 tx=0x7f2eec01d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:32.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:32.873+0000 7f2ee5ffb700 1 -- 172.21.15.104:0/1252895860 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f2ee8051180 con 0x7f2ef00c1e00 2023-07-14T11:53:33.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:33.057+0000 7f2ef76c6700 1 -- 172.21.15.104:0/1252895860 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2ef00b0470 con 0x7f2ef00c1e00 2023-07-14T11:53:33.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:33.057+0000 7f2ee5ffb700 1 -- 172.21.15.104:0/1252895860 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2ee80547a0 con 0x7f2ef00c1e00 2023-07-14T11:53:33.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:33.061+0000 7f2ef76c6700 1 -- 172.21.15.104:0/1252895860 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2ed40662f0 msgr2=0x7f2ed40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:33.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:33.061+0000 7f2ef76c6700 1 --2- 172.21.15.104:0/1252895860 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2ed40662f0 0x7f2ed40687a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0090b0 tx=0x7f2eec01d040 comp rx=0 tx=0).stop 2023-07-14T11:53:33.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:33.061+0000 7f2ef76c6700 1 -- 172.21.15.104:0/1252895860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ef00c1e00 msgr2=0x7f2ef00c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:33.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:33.061+0000 7f2ef76c6700 1 --2- 172.21.15.104:0/1252895860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ef00c1e00 0x7f2ef00c77f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800cb50 tx=0x7f2ee800ce60 comp rx=0 tx=0).stop 2023-07-14T11:53:33.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:33.061+0000 7f2ef76c6700 1 -- 172.21.15.104:0/1252895860 shutdown_connections 2023-07-14T11:53:33.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:33.061+0000 7f2ef76c6700 1 --2- 172.21.15.104:0/1252895860 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2ed40662f0 0x7f2ed40687a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:33.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:33.061+0000 7f2ef76c6700 1 --2- 172.21.15.104:0/1252895860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ef00c1e00 0x7f2ef00c77f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:33.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:33.061+0000 7f2ef76c6700 1 --2- 172.21.15.104:0/1252895860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ef006d860 0x7f2ef00c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:33.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:33.061+0000 7f2ef76c6700 1 --2- 172.21.15.104:0/1252895860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ef006be70 0x7f2ef00acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:33.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:33.061+0000 7f2ef76c6700 1 -- 172.21.15.104:0/1252895860 >> 172.21.15.104:0/1252895860 conn(0x7f2ef00cad30 msgr2=0x7f2ef0158bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:33.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:33.061+0000 7f2ef76c6700 1 -- 172.21.15.104:0/1252895860 shutdown_connections 2023-07-14T11:53:33.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:33.061+0000 7f2ef76c6700 1 -- 172.21.15.104:0/1252895860 wait complete. 2023-07-14T11:53:33.050 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:53:33.068 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi104.front.sepia.ceph.com... 2023-07-14T11:53:33.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:33.069 DEBUG:teuthology.orchestra.run.smithi104:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:53:33.108 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi104.stderr:ceph-fuse[41139]: fuse finished with error 0 and tester_r 0 2023-07-14T11:53:33.113 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:53:37.634 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:53:37.642 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:53:39.214 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:53:39.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:39.215 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-07-14T11:53:39.251 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:39.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.492+0000 7fcfc9b7e700 1 Processor -- start 2023-07-14T11:53:39.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.492+0000 7fcfc9b7e700 1 -- start start 2023-07-14T11:53:39.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.496+0000 7fcfc9b7e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfc40c9d90 0x7fcfc40ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:39.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.496+0000 7fcfc9b7e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc40b0ef0 con 0x7fcfc415d7f0 2023-07-14T11:53:39.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.496+0000 7fcfc9b7e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc40ce9c0 con 0x7fcfc4159f20 2023-07-14T11:53:39.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.496+0000 7fcfc9b7e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc40cee50 con 0x7fcfc40c9d90 2023-07-14T11:53:39.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.496+0000 7fcfc937d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fcfc415d7f0 0x7fcfc415dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35848/0 (socket says 172.21.15.104:35848) 2023-07-14T11:53:39.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.496+0000 7fcfc937d700 1 -- 172.21.15.104:0/540550810 learned_addr learned my addr 172.21.15.104:0/540550810 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:39.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.496+0000 7fcfc8b7c700 1 --2- 172.21.15.104:0/540550810 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfc40c9d90 0x7fcfc40ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:39.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.496+0000 7fcfbb7fe700 1 -- 172.21.15.104:0/540550810 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4169887915 0 0) 0x7fcfc40b0ef0 con 0x7fcfc415d7f0 2023-07-14T11:53:39.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.496+0000 7fcfbb7fe700 1 -- 172.21.15.104:0/540550810 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfac0035d0 con 0x7fcfc415d7f0 2023-07-14T11:53:39.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.496+0000 7fcfbb7fe700 1 -- 172.21.15.104:0/540550810 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209420605 0 0) 0x7fcfc40ce9c0 con 0x7fcfc4159f20 2023-07-14T11:53:39.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.496+0000 7fcfc8b7c700 1 -- 172.21.15.104:0/540550810 >> v1:172.21.15.121:6789/0 conn(0x7fcfc4159f20 legacy=0x7fcfc40caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:39.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.496+0000 7fcfc8b7c700 1 -- 172.21.15.104:0/540550810 >> v1:172.21.15.104:6789/0 conn(0x7fcfc415d7f0 legacy=0x7fcfc415dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:39.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.496+0000 7fcfc8b7c700 1 -- 172.21.15.104:0/540550810 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc40ca6a0 con 0x7fcfc40c9d90 2023-07-14T11:53:39.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.500+0000 7fcfc8b7c700 1 --2- 172.21.15.104:0/540550810 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfc40c9d90 0x7fcfc40ca160 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009880 tx=0x7fcfc0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1f14b4d494ade8f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:39.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.500+0000 7fcfbb7fe700 1 -- 172.21.15.104:0/540550810 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0011830 con 0x7fcfc40c9d90 2023-07-14T11:53:39.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.500+0000 7fcfbb7fe700 1 -- 172.21.15.104:0/540550810 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc0011990 con 0x7fcfc40c9d90 2023-07-14T11:53:39.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.500+0000 7fcfbb7fe700 1 -- 172.21.15.104:0/540550810 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0011d20 con 0x7fcfc40c9d90 2023-07-14T11:53:39.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.500+0000 7fcfc9b7e700 1 -- 172.21.15.104:0/540550810 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfc40c9d90 msgr2=0x7fcfc40ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:39.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.500+0000 7fcfc9b7e700 1 --2- 172.21.15.104:0/540550810 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfc40c9d90 0x7fcfc40ca160 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009880 tx=0x7fcfc0009b90 comp rx=0 tx=0).stop 2023-07-14T11:53:39.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.500+0000 7fcfc9b7e700 1 -- 172.21.15.104:0/540550810 shutdown_connections 2023-07-14T11:53:39.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.500+0000 7fcfc9b7e700 1 --2- 172.21.15.104:0/540550810 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfc40c9d90 0x7fcfc40ca160 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.500+0000 7fcfc9b7e700 1 -- 172.21.15.104:0/540550810 >> 172.21.15.104:0/540550810 conn(0x7fcfc406e960 msgr2=0x7fcfc406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:39.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.500+0000 7fcfc9b7e700 1 -- 172.21.15.104:0/540550810 shutdown_connections 2023-07-14T11:53:39.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.500+0000 7fcfc9b7e700 1 -- 172.21.15.104:0/540550810 wait complete. 2023-07-14T11:53:39.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.504+0000 7fcfc9b7e700 1 Processor -- start 2023-07-14T11:53:39.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.504+0000 7fcfc9b7e700 1 -- start start 2023-07-14T11:53:39.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.504+0000 7fcfc9b7e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc40c9d90 0x7fcfc414dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:39.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.504+0000 7fcfc9b7e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfc4159f20 0x7fcfc4148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:39.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.504+0000 7fcfc9b7e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfc415d7f0 0x7fcfc40b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:39.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.504+0000 7fcfc9b7e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc4149950 con 0x7fcfc40c9d90 2023-07-14T11:53:39.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.504+0000 7fcfc9b7e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc40c2db0 con 0x7fcfc415d7f0 2023-07-14T11:53:39.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.504+0000 7fcfc9b7e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc41e3570 con 0x7fcfc4159f20 2023-07-14T11:53:39.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.504+0000 7fcfc8b7c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc40c9d90 0x7fcfc414dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:39.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.504+0000 7fcfc8b7c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc40c9d90 0x7fcfc414dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42734/0 (socket says 172.21.15.104:42734) 2023-07-14T11:53:39.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.504+0000 7fcfc8b7c700 1 -- 172.21.15.104:0/900234291 learned_addr learned my addr 172.21.15.104:0/900234291 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:39.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.504+0000 7fcfc937d700 1 --2- 172.21.15.104:0/900234291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfc415d7f0 0x7fcfc40b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:39.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.504+0000 7fcfbbfff700 1 --2- 172.21.15.104:0/900234291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfc4159f20 0x7fcfc4148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:39.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.508+0000 7fcfc8b7c700 1 -- 172.21.15.104:0/900234291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfc4159f20 msgr2=0x7fcfc4148bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:39.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.508+0000 7fcfc8b7c700 1 --2- 172.21.15.104:0/900234291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfc4159f20 0x7fcfc4148bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.508+0000 7fcfc8b7c700 1 -- 172.21.15.104:0/900234291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfc415d7f0 msgr2=0x7fcfc40b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:39.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.508+0000 7fcfc8b7c700 1 --2- 172.21.15.104:0/900234291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfc415d7f0 0x7fcfc40b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.508+0000 7fcfc8b7c700 1 -- 172.21.15.104:0/900234291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc00094b0 con 0x7fcfc40c9d90 2023-07-14T11:53:39.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.508+0000 7fcfc937d700 1 --2- 172.21.15.104:0/900234291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfc415d7f0 0x7fcfc40b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:39.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.508+0000 7fcfc8b7c700 1 --2- 172.21.15.104:0/900234291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc40c9d90 0x7fcfc414dca0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0013040 tx=0x7fcfc0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:39.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.508+0000 7fcfb97fa700 1 -- 172.21.15.104:0/900234291 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0019a80 con 0x7fcfc40c9d90 2023-07-14T11:53:39.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.508+0000 7fcfb97fa700 1 -- 172.21.15.104:0/900234291 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc001e070 con 0x7fcfc40c9d90 2023-07-14T11:53:39.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.508+0000 7fcfb97fa700 1 -- 172.21.15.104:0/900234291 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0019e00 con 0x7fcfc40c9d90 2023-07-14T11:53:39.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.508+0000 7fcfc9b7e700 1 -- 172.21.15.104:0/900234291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc41e3850 con 0x7fcfc40c9d90 2023-07-14T11:53:39.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.508+0000 7fcfc9b7e700 1 -- 172.21.15.104:0/900234291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc40c32e0 con 0x7fcfc40c9d90 2023-07-14T11:53:39.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.508+0000 7fcfc9b7e700 1 -- 172.21.15.104:0/900234291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc40c34d0 con 0x7fcfc40c9d90 2023-07-14T11:53:39.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.512+0000 7fcfb97fa700 1 -- 172.21.15.104:0/900234291 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fcfc002d450 con 0x7fcfc40c9d90 2023-07-14T11:53:39.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.512+0000 7fcfb97fa700 1 --2- 172.21.15.104:0/900234291 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcfb00662f0 0x7fcfb00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:39.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.512+0000 7fcfb97fa700 1 -- 172.21.15.104:0/900234291 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fcfc0085100 con 0x7fcfc40c9d90 2023-07-14T11:53:39.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.512+0000 7fcfbbfff700 1 --2- 172.21.15.104:0/900234291 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcfb00662f0 0x7fcfb00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:39.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.512+0000 7fcfbbfff700 1 --2- 172.21.15.104:0/900234291 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcfb00662f0 0x7fcfb00687a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcfc415bd70 tx=0x7fcfb4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:39.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.520+0000 7fcfb97fa700 1 -- 172.21.15.104:0/900234291 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fcfc00521f0 con 0x7fcfc40c9d90 2023-07-14T11:53:39.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.704+0000 7fcfc9b7e700 1 -- 172.21.15.104:0/900234291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfc41e39e0 con 0x7fcfc40c9d90 2023-07-14T11:53:39.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.708+0000 7fcfb97fa700 1 -- 172.21.15.104:0/900234291 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20156 (secure 0 0 0) 0x7fcfc0020090 con 0x7fcfc40c9d90 2023-07-14T11:53:39.692 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:39.692 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":28,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:20.091434+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:39.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.712+0000 7fcfc9b7e700 1 -- 172.21.15.104:0/900234291 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcfb00662f0 msgr2=0x7fcfb00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:39.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.712+0000 7fcfc9b7e700 1 --2- 172.21.15.104:0/900234291 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcfb00662f0 0x7fcfb00687a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcfc415bd70 tx=0x7fcfb4009040 comp rx=0 tx=0).stop 2023-07-14T11:53:39.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.712+0000 7fcfc9b7e700 1 -- 172.21.15.104:0/900234291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc40c9d90 msgr2=0x7fcfc414dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:39.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.712+0000 7fcfc9b7e700 1 --2- 172.21.15.104:0/900234291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc40c9d90 0x7fcfc414dca0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0013040 tx=0x7fcfc0004770 comp rx=0 tx=0).stop 2023-07-14T11:53:39.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.712+0000 7fcfc9b7e700 1 -- 172.21.15.104:0/900234291 shutdown_connections 2023-07-14T11:53:39.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.712+0000 7fcfc9b7e700 1 --2- 172.21.15.104:0/900234291 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcfb00662f0 0x7fcfb00687a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.712+0000 7fcfc9b7e700 1 --2- 172.21.15.104:0/900234291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfc415d7f0 0x7fcfc40b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.712+0000 7fcfc9b7e700 1 --2- 172.21.15.104:0/900234291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfc4159f20 0x7fcfc4148bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.712+0000 7fcfc9b7e700 1 --2- 172.21.15.104:0/900234291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc40c9d90 0x7fcfc414dca0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.712+0000 7fcfc9b7e700 1 -- 172.21.15.104:0/900234291 >> 172.21.15.104:0/900234291 conn(0x7fcfc406e960 msgr2=0x7fcfc41544c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:39.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.712+0000 7fcfc9b7e700 1 -- 172.21.15.104:0/900234291 shutdown_connections 2023-07-14T11:53:39.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.712+0000 7fcfc9b7e700 1 -- 172.21.15.104:0/900234291 wait complete. 2023-07-14T11:53:39.717 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:39.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.904+0000 7f7ff7464700 1 Processor -- start 2023-07-14T11:53:39.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.904+0000 7f7ff7464700 1 -- start start 2023-07-14T11:53:39.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff7464700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7ff015ffa0 0x7f7ff006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:39.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff7464700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff006dc50 0x7f7ff00caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:39.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff7464700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7ff0160370 0x7f7ff00cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:39.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff7464700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff00ca8a0 con 0x7f7ff0160370 2023-07-14T11:53:39.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff7464700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff00cbcd0 con 0x7f7ff015ffa0 2023-07-14T11:53:39.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff7464700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff00cbf70 con 0x7f7ff006dc50 2023-07-14T11:53:39.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff5c61700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff006dc50 0x7f7ff00caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:39.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff6c63700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7ff0160370 0x7f7ff00cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:39.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff6462700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7ff015ffa0 0x7f7ff006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:39.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff5c61700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff006dc50 0x7f7ff00caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:40398/0 (socket says 172.21.15.104:40398) 2023-07-14T11:53:39.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff5c61700 1 -- 172.21.15.104:0/1745165059 learned_addr learned my addr 172.21.15.104:0/1745165059 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:39.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff6462700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7ff015ffa0 0x7f7ff006d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:37622/0 (socket says 172.21.15.104:37622) 2023-07-14T11:53:39.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff5c61700 1 -- 172.21.15.104:0/1745165059 >> v2:172.21.15.121:3300/0 conn(0x7f7ff015ffa0 msgr2=0x7f7ff006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:39.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff5c61700 1 --2- 172.21.15.104:0/1745165059 >> v2:172.21.15.121:3300/0 conn(0x7f7ff015ffa0 0x7f7ff006d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff5c61700 1 -- 172.21.15.104:0/1745165059 >> v2:172.21.15.104:3300/0 conn(0x7f7ff0160370 msgr2=0x7f7ff00cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:39.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff5c61700 1 --2- 172.21.15.104:0/1745165059 >> v2:172.21.15.104:3300/0 conn(0x7f7ff0160370 0x7f7ff00cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff5c61700 1 -- 172.21.15.104:0/1745165059 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff00cc250 con 0x7f7ff006dc50 2023-07-14T11:53:39.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff6c63700 1 --2- 172.21.15.104:0/1745165059 >> v2:172.21.15.104:3300/0 conn(0x7f7ff0160370 0x7f7ff00cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:39.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff6462700 1 --2- 172.21.15.104:0/1745165059 >> v2:172.21.15.121:3300/0 conn(0x7f7ff015ffa0 0x7f7ff006d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:39.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff5c61700 1 --2- 172.21.15.104:0/1745165059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff006dc50 0x7f7ff00caa70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00caa40 tx=0x7f7fe0009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da66b0652df65e5b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:39.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff5460700 1 -- 172.21.15.104:0/1745165059 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0011830 con 0x7f7ff006dc50 2023-07-14T11:53:39.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff5460700 1 -- 172.21.15.104:0/1745165059 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe0011990 con 0x7f7ff006dc50 2023-07-14T11:53:39.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.908+0000 7f7ff5460700 1 -- 172.21.15.104:0/1745165059 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0011d20 con 0x7f7ff006dc50 2023-07-14T11:53:39.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.912+0000 7f7ff7464700 1 -- 172.21.15.104:0/1745165059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff006dc50 msgr2=0x7f7ff00caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:39.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.912+0000 7f7ff7464700 1 --2- 172.21.15.104:0/1745165059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff006dc50 0x7f7ff00caa70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00caa40 tx=0x7f7fe0009bf0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.912+0000 7f7ff7464700 1 -- 172.21.15.104:0/1745165059 shutdown_connections 2023-07-14T11:53:39.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.912+0000 7f7ff7464700 1 --2- 172.21.15.104:0/1745165059 >> v2:172.21.15.104:3300/0 conn(0x7f7ff0160370 0x7f7ff00cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.912+0000 7f7ff7464700 1 --2- 172.21.15.104:0/1745165059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff006dc50 0x7f7ff00caa70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.912+0000 7f7ff7464700 1 --2- 172.21.15.104:0/1745165059 >> v2:172.21.15.121:3300/0 conn(0x7f7ff015ffa0 0x7f7ff006d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.912+0000 7f7ff7464700 1 -- 172.21.15.104:0/1745165059 >> 172.21.15.104:0/1745165059 conn(0x7f7ff006e960 msgr2=0x7f7ff006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:39.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.912+0000 7f7ff7464700 1 -- 172.21.15.104:0/1745165059 shutdown_connections 2023-07-14T11:53:39.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.912+0000 7f7ff7464700 1 -- 172.21.15.104:0/1745165059 wait complete. 2023-07-14T11:53:39.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.912+0000 7f7ff7464700 1 Processor -- start 2023-07-14T11:53:39.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.912+0000 7f7ff7464700 1 -- start start 2023-07-14T11:53:39.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff7464700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ff006dc50 0x7f7ff01f64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:39.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff7464700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ff015ffa0 0x7f7ff01613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:39.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff7464700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff0160370 0x7f7ff01f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:39.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff7464700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff00c6cc0 con 0x7f7ff015ffa0 2023-07-14T11:53:39.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff7464700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff0161930 con 0x7f7ff006dc50 2023-07-14T11:53:39.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff7464700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff01f8cd0 con 0x7f7ff0160370 2023-07-14T11:53:39.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff5c61700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ff015ffa0 0x7f7ff01613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:39.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff5c61700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ff015ffa0 0x7f7ff01613f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42746/0 (socket says 172.21.15.104:42746) 2023-07-14T11:53:39.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff5c61700 1 -- 172.21.15.104:0/762634456 learned_addr learned my addr 172.21.15.104:0/762634456 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:39.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff6462700 1 --2- 172.21.15.104:0/762634456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ff006dc50 0x7f7ff01f64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:39.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff6c63700 1 --2- 172.21.15.104:0/762634456 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff0160370 0x7f7ff01f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:39.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff6462700 1 -- 172.21.15.104:0/762634456 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff0160370 msgr2=0x7f7ff01f8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:39.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff6462700 1 --2- 172.21.15.104:0/762634456 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff0160370 0x7f7ff01f8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff6462700 1 -- 172.21.15.104:0/762634456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ff015ffa0 msgr2=0x7f7ff01613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:39.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff6462700 1 --2- 172.21.15.104:0/762634456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ff015ffa0 0x7f7ff01613f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff6462700 1 -- 172.21.15.104:0/762634456 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec0094b0 con 0x7f7ff006dc50 2023-07-14T11:53:39.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff5c61700 1 --2- 172.21.15.104:0/762634456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ff015ffa0 0x7f7ff01613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:39.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff6462700 1 --2- 172.21.15.104:0/762634456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ff006dc50 0x7f7ff01f64f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00e9b0 tx=0x7f7fec00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:39.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7fe6ffd700 1 -- 172.21.15.104:0/762634456 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec00c640 con 0x7f7ff006dc50 2023-07-14T11:53:39.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7fe6ffd700 1 -- 172.21.15.104:0/762634456 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fec00c7a0 con 0x7f7ff006dc50 2023-07-14T11:53:39.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff7464700 1 -- 172.21.15.104:0/762634456 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe8009a70 con 0x7f7ff006dc50 2023-07-14T11:53:39.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7fe6ffd700 1 -- 172.21.15.104:0/762634456 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec00cb50 con 0x7f7ff006dc50 2023-07-14T11:53:39.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.916+0000 7f7ff7464700 1 -- 172.21.15.104:0/762634456 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe0009570 con 0x7f7ff006dc50 2023-07-14T11:53:39.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.920+0000 7f7ff7464700 1 -- 172.21.15.104:0/762634456 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff0161ed0 con 0x7f7ff006dc50 2023-07-14T11:53:39.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.920+0000 7f7fe6ffd700 1 -- 172.21.15.104:0/762634456 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7fec00ccb0 con 0x7f7ff006dc50 2023-07-14T11:53:39.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.924+0000 7f7fe6ffd700 1 --2- 172.21.15.104:0/762634456 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7fd40662f0 0x7f7fd40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:39.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.924+0000 7f7fe6ffd700 1 -- 172.21.15.104:0/762634456 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7fec083f90 con 0x7f7ff006dc50 2023-07-14T11:53:39.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.924+0000 7f7ff5c61700 1 --2- 172.21.15.104:0/762634456 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7fd40662f0 0x7f7fd40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:39.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.924+0000 7f7ff5c61700 1 --2- 172.21.15.104:0/762634456 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7fd40662f0 0x7f7fd40687a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00c6160 tx=0x7f7fe0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:39.916 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:39.932+0000 7f7fe6ffd700 1 -- 172.21.15.104:0/762634456 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f7fec051080 con 0x7f7ff006dc50 2023-07-14T11:53:40.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.104+0000 7f7ff7464700 1 -- 172.21.15.104:0/762634456 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ff0071620 con 0x7f7ff006dc50 2023-07-14T11:53:40.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.108+0000 7f7fe6ffd700 1 -- 172.21.15.104:0/762634456 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20156 (secure 0 0 0) 0x7f7fec0546a0 con 0x7f7ff006dc50 2023-07-14T11:53:40.093 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:40.093 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":28,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:20.091434+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:40.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.112+0000 7f7ff7464700 1 -- 172.21.15.104:0/762634456 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7fd40662f0 msgr2=0x7f7fd40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:40.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.112+0000 7f7ff7464700 1 --2- 172.21.15.104:0/762634456 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7fd40662f0 0x7f7fd40687a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00c6160 tx=0x7f7fe0008030 comp rx=0 tx=0).stop 2023-07-14T11:53:40.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.112+0000 7f7ff7464700 1 -- 172.21.15.104:0/762634456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ff006dc50 msgr2=0x7f7ff01f64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:40.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.112+0000 7f7ff7464700 1 --2- 172.21.15.104:0/762634456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ff006dc50 0x7f7ff01f64f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00e9b0 tx=0x7f7fec00ecc0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.112+0000 7f7ff7464700 1 -- 172.21.15.104:0/762634456 shutdown_connections 2023-07-14T11:53:40.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.112+0000 7f7ff7464700 1 --2- 172.21.15.104:0/762634456 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7fd40662f0 0x7f7fd40687a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.112+0000 7f7ff7464700 1 --2- 172.21.15.104:0/762634456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ff015ffa0 0x7f7ff01613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.112+0000 7f7ff7464700 1 --2- 172.21.15.104:0/762634456 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ff0160370 0x7f7ff01f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.112+0000 7f7ff7464700 1 --2- 172.21.15.104:0/762634456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ff006dc50 0x7f7ff01f64f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.112+0000 7f7ff7464700 1 -- 172.21.15.104:0/762634456 >> 172.21.15.104:0/762634456 conn(0x7f7ff006e960 msgr2=0x7f7ff00cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:40.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.112+0000 7f7ff7464700 1 -- 172.21.15.104:0/762634456 shutdown_connections 2023-07-14T11:53:40.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.112+0000 7f7ff7464700 1 -- 172.21.15.104:0/762634456 wait complete. 2023-07-14T11:53:40.119 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-14T11:53:40.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.312+0000 7f32e1e03700 1 Processor -- start 2023-07-14T11:53:40.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.316+0000 7f32e1e03700 1 -- start start 2023-07-14T11:53:40.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.316+0000 7f32e1e03700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f32d4036320 0x7f32d40e4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:40.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.316+0000 7f32e1e03700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f32d40514c0 0x7f32d4039d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:40.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.316+0000 7f32e1e03700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d40e5260 con 0x7f32d4038980 2023-07-14T11:53:40.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.316+0000 7f32e1e03700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d4053500 con 0x7f32d40514c0 2023-07-14T11:53:40.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.316+0000 7f32e1e03700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d4051950 con 0x7f32d4036320 2023-07-14T11:53:40.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.316+0000 7f32dbfff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f32d40514c0 0x7f32d4039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:40.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.316+0000 7f32dbfff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f32d40514c0 0x7f32d4039d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42760/0 (socket says 172.21.15.104:42760) 2023-07-14T11:53:40.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.316+0000 7f32dbfff700 1 -- 172.21.15.104:0/3059008245 learned_addr learned my addr 172.21.15.104:0/3059008245 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:40.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.316+0000 7f32e0e01700 1 --2- 172.21.15.104:0/3059008245 >> v2:172.21.15.121:3300/0 conn(0x7f32d4036320 0x7f32d40e4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:40.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32dbfff700 1 -- 172.21.15.104:0/3059008245 >> v2:172.21.15.121:3300/0 conn(0x7f32d4036320 msgr2=0x7f32d40e4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:40.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32dbfff700 1 --2- 172.21.15.104:0/3059008245 >> v2:172.21.15.121:3300/0 conn(0x7f32d4036320 0x7f32d40e4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32dbfff700 1 -- 172.21.15.104:0/3059008245 >> v1:172.21.15.121:6789/0 conn(0x7f32d4038980 legacy=0x7f32d4055650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:40.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32dbfff700 1 -- 172.21.15.104:0/3059008245 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d4054b70 con 0x7f32d40514c0 2023-07-14T11:53:40.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32dbfff700 1 --2- 172.21.15.104:0/3059008245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32d40514c0 0x7f32d4039d50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f32d400ae80 tx=0x7f32c8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc94b3a73af5fa6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:40.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32db7fe700 1 -- 172.21.15.104:0/3059008245 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c800b8a0 con 0x7f32d40514c0 2023-07-14T11:53:40.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32db7fe700 1 -- 172.21.15.104:0/3059008245 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c800ba00 con 0x7f32d40514c0 2023-07-14T11:53:40.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32db7fe700 1 -- 172.21.15.104:0/3059008245 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c800bd90 con 0x7f32d40514c0 2023-07-14T11:53:40.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32e1e03700 1 -- 172.21.15.104:0/3059008245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32d40514c0 msgr2=0x7f32d4039d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:40.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32e1e03700 1 --2- 172.21.15.104:0/3059008245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32d40514c0 0x7f32d4039d50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f32d400ae80 tx=0x7f32c8009bf0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32e1e03700 1 -- 172.21.15.104:0/3059008245 shutdown_connections 2023-07-14T11:53:40.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32e1e03700 1 --2- 172.21.15.104:0/3059008245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32d40514c0 0x7f32d4039d50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32e1e03700 1 --2- 172.21.15.104:0/3059008245 >> v2:172.21.15.121:3300/0 conn(0x7f32d4036320 0x7f32d40e4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32e1e03700 1 -- 172.21.15.104:0/3059008245 >> 172.21.15.104:0/3059008245 conn(0x7f32d4057b90 msgr2=0x7f32d4053980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:40.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32e1e03700 1 -- 172.21.15.104:0/3059008245 shutdown_connections 2023-07-14T11:53:40.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.320+0000 7f32e1e03700 1 -- 172.21.15.104:0/3059008245 wait complete. 2023-07-14T11:53:40.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.324+0000 7f32e1e03700 1 Processor -- start 2023-07-14T11:53:40.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.324+0000 7f32e1e03700 1 -- start start 2023-07-14T11:53:40.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.324+0000 7f32e1e03700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d4036320 0x7f32d40de4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:40.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.324+0000 7f32e1e03700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32d4038980 0x7f32d40e3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:40.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.324+0000 7f32e1e03700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32d40514c0 0x7f32d403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:40.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.324+0000 7f32e1e03700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d40de9e0 con 0x7f32d40514c0 2023-07-14T11:53:40.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.324+0000 7f32e1e03700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d40e3d40 con 0x7f32d4036320 2023-07-14T11:53:40.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.324+0000 7f32e1e03700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d4032f00 con 0x7f32d4038980 2023-07-14T11:53:40.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32e1602700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32d40514c0 0x7f32d403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:40.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32e1602700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32d40514c0 0x7f32d403ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:37986/0 (socket says 172.21.15.104:37986) 2023-07-14T11:53:40.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32e0e01700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d4036320 0x7f32d40de4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:40.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32e1602700 1 -- 172.21.15.104:0/534357489 learned_addr learned my addr 172.21.15.104:0/534357489 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:40.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32dbfff700 1 --2- 172.21.15.104:0/534357489 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32d4038980 0x7f32d40e3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:40.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32e1602700 1 -- 172.21.15.104:0/534357489 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32d4038980 msgr2=0x7f32d40e3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:40.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32e1602700 1 --2- 172.21.15.104:0/534357489 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32d4038980 0x7f32d40e3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32e1602700 1 -- 172.21.15.104:0/534357489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d4036320 msgr2=0x7f32d40de4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:40.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32e1602700 1 --2- 172.21.15.104:0/534357489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d4036320 0x7f32d40de4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32e1602700 1 -- 172.21.15.104:0/534357489 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c8009570 con 0x7f32d40514c0 2023-07-14T11:53:40.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32e1602700 1 --2- 172.21.15.104:0/534357489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32d40514c0 0x7f32d403ad80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f32dc063960 tx=0x7f32dc070dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:40.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32d97fa700 1 -- 172.21.15.104:0/534357489 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc06e540 con 0x7f32d40514c0 2023-07-14T11:53:40.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32d97fa700 1 -- 172.21.15.104:0/534357489 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc066d70 con 0x7f32d40514c0 2023-07-14T11:53:40.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32e1e03700 1 -- 172.21.15.104:0/534357489 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d40331e0 con 0x7f32d40514c0 2023-07-14T11:53:40.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32d97fa700 1 -- 172.21.15.104:0/534357489 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc072f70 con 0x7f32d40514c0 2023-07-14T11:53:40.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32e1e03700 1 -- 172.21.15.104:0/534357489 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d40337a0 con 0x7f32d40514c0 2023-07-14T11:53:40.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.328+0000 7f32e1e03700 1 -- 172.21.15.104:0/534357489 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d40059c0 con 0x7f32d40514c0 2023-07-14T11:53:40.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.332+0000 7f32d97fa700 1 -- 172.21.15.104:0/534357489 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f32dc0730d0 con 0x7f32d40514c0 2023-07-14T11:53:40.317 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.332+0000 7f32d97fa700 1 --2- 172.21.15.104:0/534357489 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f32c0066320 0x7f32c00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:40.317 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.332+0000 7f32d97fa700 1 -- 172.21.15.104:0/534357489 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f32dc0fbbb0 con 0x7f32d40514c0 2023-07-14T11:53:40.318 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.332+0000 7f32e0e01700 1 --2- 172.21.15.104:0/534357489 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f32c0066320 0x7f32c00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:40.318 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.336+0000 7f32e0e01700 1 --2- 172.21.15.104:0/534357489 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f32c0066320 0x7f32c00687d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009a00 tx=0x7f32d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:40.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.340+0000 7f32d97fa700 1 -- 172.21.15.104:0/534357489 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f32dc0c8d20 con 0x7f32d40514c0 2023-07-14T11:53:40.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.548+0000 7f32e1e03700 1 -- 172.21.15.104:0/534357489 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f32d40e5220 con 0x7f32d40514c0 2023-07-14T11:53:40.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.784+0000 7f32d97fa700 1 -- 172.21.15.104:0/534357489 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f32dc0cc340 con 0x7f32d40514c0 2023-07-14T11:53:40.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.788+0000 7f32e1e03700 1 -- 172.21.15.104:0/534357489 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f32c0066320 msgr2=0x7f32c00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:40.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.788+0000 7f32e1e03700 1 --2- 172.21.15.104:0/534357489 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f32c0066320 0x7f32c00687d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009a00 tx=0x7f32d0008040 comp rx=0 tx=0).stop 2023-07-14T11:53:40.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.788+0000 7f32e1e03700 1 -- 172.21.15.104:0/534357489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32d40514c0 msgr2=0x7f32d403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:40.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.788+0000 7f32e1e03700 1 --2- 172.21.15.104:0/534357489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32d40514c0 0x7f32d403ad80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f32dc063960 tx=0x7f32dc070dd0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.792+0000 7f32e1e03700 1 -- 172.21.15.104:0/534357489 shutdown_connections 2023-07-14T11:53:40.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.792+0000 7f32e1e03700 1 --2- 172.21.15.104:0/534357489 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f32c0066320 0x7f32c00687d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.792+0000 7f32e1e03700 1 --2- 172.21.15.104:0/534357489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32d40514c0 0x7f32d403ad80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.792+0000 7f32e1e03700 1 --2- 172.21.15.104:0/534357489 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32d4038980 0x7f32d40e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.792+0000 7f32e1e03700 1 --2- 172.21.15.104:0/534357489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d4036320 0x7f32d40de4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.792+0000 7f32e1e03700 1 -- 172.21.15.104:0/534357489 >> 172.21.15.104:0/534357489 conn(0x7f32d4057b90 msgr2=0x7f32d4038da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:40.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.792+0000 7f32e1e03700 1 -- 172.21.15.104:0/534357489 shutdown_connections 2023-07-14T11:53:40.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.792+0000 7f32e1e03700 1 -- 172.21.15.104:0/534357489 wait complete. 2023-07-14T11:53:40.800 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-07-14T11:53:40.800 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-14T11:53:40.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.992+0000 7fb80b3e8700 1 Processor -- start 2023-07-14T11:53:40.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.992+0000 7fb80b3e8700 1 -- start start 2023-07-14T11:53:40.980 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.996+0000 7fb80b3e8700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8040cd260 0x7fb8040d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:40.980 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.996+0000 7fb80b3e8700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8040d1a60 con 0x7fb8040d0610 2023-07-14T11:53:40.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.996+0000 7fb80b3e8700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8040d1ef0 con 0x7fb8040b0ee0 2023-07-14T11:53:40.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.996+0000 7fb80b3e8700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040d0e50 con 0x7fb8040cd260 2023-07-14T11:53:40.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.996+0000 7fb80abe7700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb8040d0610 0x7fb8040af0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42422/0 (socket says 172.21.15.104:42422) 2023-07-14T11:53:40.982 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.996+0000 7fb80abe7700 1 -- 172.21.15.104:0/2665913055 learned_addr learned my addr 172.21.15.104:0/2665913055 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:40.982 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.996+0000 7fb809be5700 1 --2- 172.21.15.104:0/2665913055 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8040cd260 0x7fb8040d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:40.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.996+0000 7fb8093e4700 1 -- 172.21.15.104:0/2665913055 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706979261 0 0) 0x7fb8040d1a60 con 0x7fb8040d0610 2023-07-14T11:53:40.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.996+0000 7fb8093e4700 1 -- 172.21.15.104:0/2665913055 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f00035d0 con 0x7fb8040d0610 2023-07-14T11:53:40.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.996+0000 7fb8093e4700 1 -- 172.21.15.104:0/2665913055 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239222913 0 0) 0x7fb8040d1ef0 con 0x7fb8040b0ee0 2023-07-14T11:53:40.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.996+0000 7fb8093e4700 1 -- 172.21.15.104:0/2665913055 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8040d1a60 con 0x7fb8040b0ee0 2023-07-14T11:53:40.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.996+0000 7fb809be5700 1 -- 172.21.15.104:0/2665913055 >> v1:172.21.15.121:6789/0 conn(0x7fb8040b0ee0 legacy=0x7fb8040ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:40.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.996+0000 7fb809be5700 1 -- 172.21.15.104:0/2665913055 >> v1:172.21.15.104:6789/0 conn(0x7fb8040d0610 legacy=0x7fb8040af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:40.984 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:40.996+0000 7fb809be5700 1 -- 172.21.15.104:0/2665913055 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8040d1130 con 0x7fb8040cd260 2023-07-14T11:53:40.984 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.000+0000 7fb809be5700 1 --2- 172.21.15.104:0/2665913055 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8040cd260 0x7fb8040d00d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009880 tx=0x7fb7f4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ca16496bd216c34 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:40.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.000+0000 7fb8093e4700 1 -- 172.21.15.104:0/2665913055 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4011830 con 0x7fb8040cd260 2023-07-14T11:53:40.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.000+0000 7fb8093e4700 1 -- 172.21.15.104:0/2665913055 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f4011990 con 0x7fb8040cd260 2023-07-14T11:53:40.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.000+0000 7fb8093e4700 1 -- 172.21.15.104:0/2665913055 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4011d20 con 0x7fb8040cd260 2023-07-14T11:53:40.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.000+0000 7fb80b3e8700 1 -- 172.21.15.104:0/2665913055 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8040cd260 msgr2=0x7fb8040d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:40.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.000+0000 7fb80b3e8700 1 --2- 172.21.15.104:0/2665913055 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8040cd260 0x7fb8040d00d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009880 tx=0x7fb7f4009b90 comp rx=0 tx=0).stop 2023-07-14T11:53:40.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.000+0000 7fb80b3e8700 1 -- 172.21.15.104:0/2665913055 shutdown_connections 2023-07-14T11:53:40.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.000+0000 7fb80b3e8700 1 --2- 172.21.15.104:0/2665913055 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8040cd260 0x7fb8040d00d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.000+0000 7fb80b3e8700 1 -- 172.21.15.104:0/2665913055 >> 172.21.15.104:0/2665913055 conn(0x7fb804060c40 msgr2=0x7fb80409b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:40.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.000+0000 7fb80b3e8700 1 -- 172.21.15.104:0/2665913055 shutdown_connections 2023-07-14T11:53:40.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.000+0000 7fb80b3e8700 1 -- 172.21.15.104:0/2665913055 wait complete. 2023-07-14T11:53:40.988 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.004+0000 7fb80b3e8700 1 Processor -- start 2023-07-14T11:53:40.988 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.004+0000 7fb80b3e8700 1 -- start start 2023-07-14T11:53:40.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.004+0000 7fb80b3e8700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8040b0ee0 0x7fb8041f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:40.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.004+0000 7fb80b3e8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8040cd260 0x7fb8040c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:40.990 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.004+0000 7fb80b3e8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8040d0610 0x7fb8040b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:40.990 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.004+0000 7fb80b3e8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040b4980 con 0x7fb8040d0610 2023-07-14T11:53:40.990 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.004+0000 7fb80b3e8700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8041f5840 con 0x7fb8040cd260 2023-07-14T11:53:40.990 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.004+0000 7fb80b3e8700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040c8d60 con 0x7fb8040b0ee0 2023-07-14T11:53:40.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.004+0000 7fb80abe7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8040d0610 0x7fb8040b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:40.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.004+0000 7fb80abe7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8040d0610 0x7fb8040b60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:37996/0 (socket says 172.21.15.104:37996) 2023-07-14T11:53:40.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.004+0000 7fb80abe7700 1 -- 172.21.15.104:0/3340543521 learned_addr learned my addr 172.21.15.104:0/3340543521 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:40.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.004+0000 7fb80a3e6700 1 --2- 172.21.15.104:0/3340543521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8040b0ee0 0x7fb8041f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:40.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.004+0000 7fb809be5700 1 --2- 172.21.15.104:0/3340543521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8040cd260 0x7fb8040c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:40.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.008+0000 7fb80a3e6700 1 -- 172.21.15.104:0/3340543521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8040cd260 msgr2=0x7fb8040c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:40.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.008+0000 7fb80a3e6700 1 --2- 172.21.15.104:0/3340543521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8040cd260 0x7fb8040c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.008+0000 7fb80a3e6700 1 -- 172.21.15.104:0/3340543521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8040d0610 msgr2=0x7fb8040b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:40.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.008+0000 7fb80a3e6700 1 --2- 172.21.15.104:0/3340543521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8040d0610 0x7fb8040b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:40.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.008+0000 7fb80a3e6700 1 -- 172.21.15.104:0/3340543521 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f40094b0 con 0x7fb8040b0ee0 2023-07-14T11:53:40.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.008+0000 7fb80abe7700 1 --2- 172.21.15.104:0/3340543521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8040d0610 0x7fb8040b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:40.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.008+0000 7fb80a3e6700 1 --2- 172.21.15.104:0/3340543521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8040b0ee0 0x7fb8041f2e30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb80000cb40 tx=0x7fb80000ce50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:40.993 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.008+0000 7fb7faffd700 1 -- 172.21.15.104:0/3340543521 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800003c20 con 0x7fb8040b0ee0 2023-07-14T11:53:40.993 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.008+0000 7fb7faffd700 1 -- 172.21.15.104:0/3340543521 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb800003d80 con 0x7fb8040b0ee0 2023-07-14T11:53:40.993 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.008+0000 7fb7faffd700 1 -- 172.21.15.104:0/3340543521 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80000aa00 con 0x7fb8040b0ee0 2023-07-14T11:53:40.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.008+0000 7fb80b3e8700 1 -- 172.21.15.104:0/3340543521 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8040c8fe0 con 0x7fb8040b0ee0 2023-07-14T11:53:40.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.008+0000 7fb80b3e8700 1 -- 172.21.15.104:0/3340543521 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8041f5da0 con 0x7fb8040b0ee0 2023-07-14T11:53:40.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.008+0000 7fb80b3e8700 1 -- 172.21.15.104:0/3340543521 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80404e500 con 0x7fb8040b0ee0 2023-07-14T11:53:40.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.012+0000 7fb7faffd700 1 -- 172.21.15.104:0/3340543521 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb80000ab60 con 0x7fb8040b0ee0 2023-07-14T11:53:40.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.012+0000 7fb7faffd700 1 --2- 172.21.15.104:0/3340543521 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb7e80662f0 0x7fb7e80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:40.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.012+0000 7fb7faffd700 1 -- 172.21.15.104:0/3340543521 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb800083fe0 con 0x7fb8040b0ee0 2023-07-14T11:53:40.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.012+0000 7fb809be5700 1 --2- 172.21.15.104:0/3340543521 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb7e80662f0 0x7fb7e80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:40.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.012+0000 7fb809be5700 1 --2- 172.21.15.104:0/3340543521 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb7e80662f0 0x7fb7e80687a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb8040cd650 tx=0x7fb7f401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:41.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.020+0000 7fb7faffd700 1 -- 172.21.15.104:0/3340543521 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb8000510d0 con 0x7fb8040b0ee0 2023-07-14T11:53:41.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.232+0000 7fb80b3e8700 1 -- 172.21.15.104:0/3340543521 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb8040b80d0 con 0x7fb8040b0ee0 2023-07-14T11:53:41.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.800+0000 7fb7faffd700 1 -- 172.21.15.104:0/3340543521 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb8000546f0 con 0x7fb8040b0ee0 2023-07-14T11:53:41.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.804+0000 7fb80b3e8700 1 -- 172.21.15.104:0/3340543521 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb7e80662f0 msgr2=0x7fb7e80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:41.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.804+0000 7fb80b3e8700 1 --2- 172.21.15.104:0/3340543521 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb7e80662f0 0x7fb7e80687a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb8040cd650 tx=0x7fb7f401c040 comp rx=0 tx=0).stop 2023-07-14T11:53:41.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.804+0000 7fb80b3e8700 1 -- 172.21.15.104:0/3340543521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8040b0ee0 msgr2=0x7fb8041f2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:41.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.804+0000 7fb80b3e8700 1 --2- 172.21.15.104:0/3340543521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8040b0ee0 0x7fb8041f2e30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb80000cb40 tx=0x7fb80000ce50 comp rx=0 tx=0).stop 2023-07-14T11:53:41.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.804+0000 7fb80b3e8700 1 -- 172.21.15.104:0/3340543521 shutdown_connections 2023-07-14T11:53:41.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.804+0000 7fb80b3e8700 1 --2- 172.21.15.104:0/3340543521 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb7e80662f0 0x7fb7e80687a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:41.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.804+0000 7fb80b3e8700 1 --2- 172.21.15.104:0/3340543521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8040d0610 0x7fb8040b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:41.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.804+0000 7fb80b3e8700 1 --2- 172.21.15.104:0/3340543521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8040cd260 0x7fb8040c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:41.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.804+0000 7fb80b3e8700 1 --2- 172.21.15.104:0/3340543521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8040b0ee0 0x7fb8041f2e30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:41.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.804+0000 7fb80b3e8700 1 -- 172.21.15.104:0/3340543521 >> 172.21.15.104:0/3340543521 conn(0x7fb804060c40 msgr2=0x7fb8040ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:41.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.804+0000 7fb80b3e8700 1 -- 172.21.15.104:0/3340543521 shutdown_connections 2023-07-14T11:53:41.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:41.804+0000 7fb80b3e8700 1 -- 172.21.15.104:0/3340543521 wait complete. 2023-07-14T11:53:41.812 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-14T11:53:41.813 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:53:41.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.012+0000 7fb444f9e700 1 Processor -- start 2023-07-14T11:53:41.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.012+0000 7fb444f9e700 1 -- start start 2023-07-14T11:53:41.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.012+0000 7fb444f9e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb4400cca70 0x7fb4400cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:41.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.012+0000 7fb444f9e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4400d0930 con 0x7fb4400b1a50 2023-07-14T11:53:41.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.012+0000 7fb444f9e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4400cf7a0 con 0x7fb4400d0530 2023-07-14T11:53:41.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.012+0000 7fb444f9e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4400ce4f0 con 0x7fb4400cca70 2023-07-14T11:53:41.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.012+0000 7fb43f7fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb4400b1a50 0x7fb4401600a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42436/0 (socket says 172.21.15.104:42436) 2023-07-14T11:53:41.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.012+0000 7fb43f7fe700 1 -- 172.21.15.104:0/1568742137 learned_addr learned my addr 172.21.15.104:0/1568742137 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:41.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43effd700 1 --2- 172.21.15.104:0/1568742137 >> v2:172.21.15.121:3300/0 conn(0x7fb4400cca70 0x7fb4400cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:41.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 -- 172.21.15.104:0/1568742137 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954715960 0 0) 0x7fb4400d0930 con 0x7fb4400b1a50 2023-07-14T11:53:41.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 -- 172.21.15.104:0/1568742137 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4280035d0 con 0x7fb4400b1a50 2023-07-14T11:53:42.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 -- 172.21.15.104:0/1568742137 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1195771771 0 0) 0x7fb4280035d0 con 0x7fb4400b1a50 2023-07-14T11:53:42.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 -- 172.21.15.104:0/1568742137 >> v2:172.21.15.121:3300/0 conn(0x7fb4400cca70 msgr2=0x7fb4400cfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:42.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 --2- 172.21.15.104:0/1568742137 >> v2:172.21.15.121:3300/0 conn(0x7fb4400cca70 0x7fb4400cfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 -- 172.21.15.104:0/1568742137 >> v1:172.21.15.121:6789/0 conn(0x7fb4400d0530 legacy=0x7fb4400d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 -- 172.21.15.104:0/1568742137 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4400ce700 con 0x7fb4400b1a50 2023-07-14T11:53:42.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 -- 172.21.15.104:0/1568742137 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7fb430001670 con 0x7fb4400b1a50 2023-07-14T11:53:42.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 -- 172.21.15.104:0/1568742137 >> v1:172.21.15.104:6789/0 conn(0x7fb4400b1a50 legacy=0x7fb4401600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 --2- 172.21.15.104:0/1568742137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb428003a20 0x7fb428003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 --2- 172.21.15.104:0/1568742137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb428004410 0x7fb4280048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 --2- 172.21.15.104:0/1568742137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4280050d0 0x7fb42800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 -- 172.21.15.104:0/1568742137 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4400ce4f0 con 0x7fb4280050d0 2023-07-14T11:53:42.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 -- 172.21.15.104:0/1568742137 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42800c020 con 0x7fb428004410 2023-07-14T11:53:42.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 -- 172.21.15.104:0/1568742137 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42800c1e0 con 0x7fb428003a20 2023-07-14T11:53:42.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43ffff700 1 --2- 172.21.15.104:0/1568742137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4280050d0 0x7fb42800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43effd700 1 --2- 172.21.15.104:0/1568742137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb428004410 0x7fb4280048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43f7fe700 1 --2- 172.21.15.104:0/1568742137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb428003a20 0x7fb428003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43ffff700 1 -- 172.21.15.104:0/1568742137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb428003a20 msgr2=0x7fb428003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:42.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43ffff700 1 --2- 172.21.15.104:0/1568742137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb428003a20 0x7fb428003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43ffff700 1 -- 172.21.15.104:0/1568742137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb428004410 msgr2=0x7fb4280048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:42.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43ffff700 1 --2- 172.21.15.104:0/1568742137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb428004410 0x7fb4280048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43ffff700 1 -- 172.21.15.104:0/1568742137 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4280035d0 con 0x7fb4280050d0 2023-07-14T11:53:42.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43f7fe700 1 --2- 172.21.15.104:0/1568742137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb428003a20 0x7fb428003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:42.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43effd700 1 --2- 172.21.15.104:0/1568742137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb428004410 0x7fb4280048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:42.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43ffff700 1 -- 172.21.15.104:0/1568742137 reap_dead start 2023-07-14T11:53:42.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43ffff700 1 --2- 172.21.15.104:0/1568742137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4280050d0 0x7fb42800ba30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb43800c370 tx=0x7fb4280058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439532cd6220e22d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:42.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 -- 172.21.15.104:0/1568742137 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb438016070 con 0x7fb4280050d0 2023-07-14T11:53:42.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 -- 172.21.15.104:0/1568742137 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43800cc90 con 0x7fb4280050d0 2023-07-14T11:53:42.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.016+0000 7fb43e7fc700 1 -- 172.21.15.104:0/1568742137 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb438001f20 con 0x7fb4280050d0 2023-07-14T11:53:42.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.020+0000 7fb444f9e700 1 -- 172.21.15.104:0/1568742137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4280050d0 msgr2=0x7fb42800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.020+0000 7fb444f9e700 1 --2- 172.21.15.104:0/1568742137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4280050d0 0x7fb42800ba30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb43800c370 tx=0x7fb4280058a0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.020+0000 7fb444f9e700 1 -- 172.21.15.104:0/1568742137 shutdown_connections 2023-07-14T11:53:42.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.020+0000 7fb444f9e700 1 --2- 172.21.15.104:0/1568742137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4280050d0 0x7fb42800ba30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.020+0000 7fb444f9e700 1 -- 172.21.15.104:0/1568742137 >> 172.21.15.104:0/1568742137 conn(0x7fb440060320 msgr2=0x7fb4400606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:42.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.020+0000 7fb444f9e700 1 -- 172.21.15.104:0/1568742137 shutdown_connections 2023-07-14T11:53:42.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.020+0000 7fb444f9e700 1 -- 172.21.15.104:0/1568742137 wait complete. 2023-07-14T11:53:42.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.020+0000 7fb444f9e700 1 Processor -- start 2023-07-14T11:53:42.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.020+0000 7fb444f9e700 1 -- start start 2023-07-14T11:53:42.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.020+0000 7fb444f9e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4280050d0 0x7fb4400cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.020+0000 7fb444f9e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4400d2820 0x7fb4400d2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb444f9e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb440152c10 0x7fb440153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb444f9e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb440151410 con 0x7fb4400d2820 2023-07-14T11:53:42.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb444f9e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4400d31d0 con 0x7fb440152c10 2023-07-14T11:53:42.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb444f9e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb440153860 con 0x7fb4280050d0 2023-07-14T11:53:42.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb43effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4400d2820 0x7fb4400d2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb43effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4400d2820 0x7fb4400d2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38018/0 (socket says 172.21.15.104:38018) 2023-07-14T11:53:42.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb43effd700 1 -- 172.21.15.104:0/1148324205 learned_addr learned my addr 172.21.15.104:0/1148324205 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:42.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb43f7fe700 1 --2- 172.21.15.104:0/1148324205 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4280050d0 0x7fb4400cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb43ffff700 1 --2- 172.21.15.104:0/1148324205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb440152c10 0x7fb440153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb43effd700 1 -- 172.21.15.104:0/1148324205 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4280050d0 msgr2=0x7fb4400cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb43effd700 1 --2- 172.21.15.104:0/1148324205 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4280050d0 0x7fb4400cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb43effd700 1 -- 172.21.15.104:0/1148324205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb440152c10 msgr2=0x7fb440153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb43effd700 1 --2- 172.21.15.104:0/1148324205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb440152c10 0x7fb440153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb43effd700 1 -- 172.21.15.104:0/1148324205 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4401535c0 con 0x7fb4400d2820 2023-07-14T11:53:42.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb43effd700 1 --2- 172.21.15.104:0/1148324205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4400d2820 0x7fb4400d2c90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb434010990 tx=0x7fb434010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:42.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb427fff700 1 -- 172.21.15.104:0/1148324205 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb434004220 con 0x7fb4400d2820 2023-07-14T11:53:42.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb427fff700 1 -- 172.21.15.104:0/1148324205 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb434004380 con 0x7fb4400d2820 2023-07-14T11:53:42.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb444f9e700 1 -- 172.21.15.104:0/1148324205 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4400c9690 con 0x7fb4400d2820 2023-07-14T11:53:42.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb427fff700 1 -- 172.21.15.104:0/1148324205 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4340036c0 con 0x7fb4400d2820 2023-07-14T11:53:42.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb444f9e700 1 -- 172.21.15.104:0/1148324205 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4400c9c50 con 0x7fb4400d2820 2023-07-14T11:53:42.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.024+0000 7fb444f9e700 1 -- 172.21.15.104:0/1148324205 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44004e500 con 0x7fb4400d2820 2023-07-14T11:53:42.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.028+0000 7fb427fff700 1 -- 172.21.15.104:0/1148324205 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb434003820 con 0x7fb4400d2820 2023-07-14T11:53:42.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.028+0000 7fb427fff700 1 --2- 172.21.15.104:0/1148324205 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb42c066320 0x7fb42c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.028+0000 7fb427fff700 1 -- 172.21.15.104:0/1148324205 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb4340841b0 con 0x7fb4400d2820 2023-07-14T11:53:42.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.028+0000 7fb43f7fe700 1 --2- 172.21.15.104:0/1148324205 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb42c066320 0x7fb42c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.028+0000 7fb43f7fe700 1 --2- 172.21.15.104:0/1148324205 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb42c066320 0x7fb42c0687d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb4300017a0 tx=0x7fb430009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:42.020 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.036+0000 7fb427fff700 1 -- 172.21.15.104:0/1148324205 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb434051220 con 0x7fb4400d2820 2023-07-14T11:53:42.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.240+0000 7fb444f9e700 1 -- 172.21.15.104:0/1148324205 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4400b4740 con 0x7fb4400d2820 2023-07-14T11:53:42.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.240+0000 7fb427fff700 1 -- 172.21.15.104:0/1148324205 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4717 (secure 0 0 0) 0x7fb434054840 con 0x7fb4400d2820 2023-07-14T11:53:42.226 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:42.226 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2103358277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2103358277},{"type":"v1","addr":"172.21.15.121:6833","nonce":2103358277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1841065287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1841065287},{"type":"v1","addr":"172.21.15.121:6835","nonce":1841065287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3342053518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3342053518},{"type":"v1","addr":"172.21.15.104:6835","nonce":3342053518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:53:15.029045+0000","modified":"2023-07-14T11:53:41.785898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/2666329866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2666329866},{"type":"v1","addr":"172.21.15.104:6837","nonce":2666329866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:53:42.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.244+0000 7fb444f9e700 1 -- 172.21.15.104:0/1148324205 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb42c066320 msgr2=0x7fb42c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.244+0000 7fb444f9e700 1 --2- 172.21.15.104:0/1148324205 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb42c066320 0x7fb42c0687d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb4300017a0 tx=0x7fb430009040 comp rx=0 tx=0).stop 2023-07-14T11:53:42.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.244+0000 7fb444f9e700 1 -- 172.21.15.104:0/1148324205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4400d2820 msgr2=0x7fb4400d2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.244+0000 7fb444f9e700 1 --2- 172.21.15.104:0/1148324205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4400d2820 0x7fb4400d2c90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb434010990 tx=0x7fb434010ca0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.244+0000 7fb444f9e700 1 -- 172.21.15.104:0/1148324205 shutdown_connections 2023-07-14T11:53:42.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.244+0000 7fb444f9e700 1 --2- 172.21.15.104:0/1148324205 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb42c066320 0x7fb42c0687d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.244+0000 7fb444f9e700 1 --2- 172.21.15.104:0/1148324205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4400d2820 0x7fb4400d2c90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.244+0000 7fb444f9e700 1 --2- 172.21.15.104:0/1148324205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb440152c10 0x7fb440153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.244+0000 7fb444f9e700 1 --2- 172.21.15.104:0/1148324205 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4280050d0 0x7fb4400cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.244+0000 7fb444f9e700 1 -- 172.21.15.104:0/1148324205 >> 172.21.15.104:0/1148324205 conn(0x7fb440060320 msgr2=0x7fb4400b7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:42.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.244+0000 7fb444f9e700 1 -- 172.21.15.104:0/1148324205 shutdown_connections 2023-07-14T11:53:42.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.244+0000 7fb444f9e700 1 -- 172.21.15.104:0/1148324205 wait complete. 2023-07-14T11:53:42.232 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 9 2023-07-14T11:53:42.251 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:42.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.436+0000 7f6cf2648700 1 Processor -- start 2023-07-14T11:53:42.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.436+0000 7f6cf2648700 1 -- start start 2023-07-14T11:53:42.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6cf2648700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6cec15bff0 0x7f6cec0ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6cf2648700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6cec0cf1c0 0x7f6cec0afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6cf2648700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cec0af0a0 con 0x7f6cec0cd070 2023-07-14T11:53:42.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6cf2648700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cec0cd4f0 con 0x7f6cec15bff0 2023-07-14T11:53:42.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6cf2648700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0cf720 con 0x7f6cec0cf1c0 2023-07-14T11:53:42.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6cf1646700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6cec15bff0 0x7f6cec0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6cf1646700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6cec15bff0 0x7f6cec0ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38032/0 (socket says 172.21.15.104:38032) 2023-07-14T11:53:42.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6cf1646700 1 -- 172.21.15.104:0/507634551 learned_addr learned my addr 172.21.15.104:0/507634551 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:42.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6cf1e47700 1 --2- 172.21.15.104:0/507634551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6cec0cf1c0 0x7f6cec0afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6cf1646700 1 -- 172.21.15.104:0/507634551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6cec0cf1c0 msgr2=0x7f6cec0afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:42.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6cf1646700 1 --2- 172.21.15.104:0/507634551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6cec0cf1c0 0x7f6cec0afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6cf1646700 1 -- 172.21.15.104:0/507634551 >> v1:172.21.15.121:6789/0 conn(0x7f6cec0cd070 legacy=0x7f6cec0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6cf1646700 1 -- 172.21.15.104:0/507634551 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec0af2f0 con 0x7f6cec15bff0 2023-07-14T11:53:42.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6cf1646700 1 --2- 172.21.15.104:0/507634551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cec15bff0 0x7f6cec0ccb30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009910 tx=0x7f6ce8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4131f056e66d05f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:42.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6ce3fff700 1 -- 172.21.15.104:0/507634551 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce800ba60 con 0x7f6cec15bff0 2023-07-14T11:53:42.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6ce3fff700 1 -- 172.21.15.104:0/507634551 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce800bbc0 con 0x7f6cec15bff0 2023-07-14T11:53:42.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.440+0000 7f6ce3fff700 1 -- 172.21.15.104:0/507634551 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8009e60 con 0x7f6cec15bff0 2023-07-14T11:53:42.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.444+0000 7f6cf2648700 1 -- 172.21.15.104:0/507634551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cec15bff0 msgr2=0x7f6cec0ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.444+0000 7f6cf2648700 1 --2- 172.21.15.104:0/507634551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cec15bff0 0x7f6cec0ccb30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009910 tx=0x7f6ce8009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:42.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.444+0000 7f6cf2648700 1 -- 172.21.15.104:0/507634551 shutdown_connections 2023-07-14T11:53:42.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.444+0000 7f6cf2648700 1 --2- 172.21.15.104:0/507634551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6cec0cf1c0 0x7f6cec0afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.444+0000 7f6cf2648700 1 --2- 172.21.15.104:0/507634551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cec15bff0 0x7f6cec0ccb30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.444+0000 7f6cf2648700 1 -- 172.21.15.104:0/507634551 >> 172.21.15.104:0/507634551 conn(0x7f6cec060c40 msgr2=0x7f6cec09b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:42.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.444+0000 7f6cf2648700 1 -- 172.21.15.104:0/507634551 shutdown_connections 2023-07-14T11:53:42.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.444+0000 7f6cf2648700 1 -- 172.21.15.104:0/507634551 wait complete. 2023-07-14T11:53:42.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf2648700 1 Processor -- start 2023-07-14T11:53:42.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf2648700 1 -- start start 2023-07-14T11:53:42.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf2648700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6cec0cd070 0x7f6cec0b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf2648700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cec0cf1c0 0x7f6cec156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf2648700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cec15bff0 0x7f6cec0b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf2648700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec15d560 con 0x7f6cec15bff0 2023-07-14T11:53:42.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf2648700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec15d9f0 con 0x7f6cec0cf1c0 2023-07-14T11:53:42.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf2648700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec15c950 con 0x7f6cec0cd070 2023-07-14T11:53:42.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf1e47700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cec15bff0 0x7f6cec0b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf1e47700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cec15bff0 0x7f6cec0b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38048/0 (socket says 172.21.15.104:38048) 2023-07-14T11:53:42.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf1e47700 1 -- 172.21.15.104:0/2049720286 learned_addr learned my addr 172.21.15.104:0/2049720286 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:42.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf1646700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6cec0cd070 0x7f6cec0b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf1e47700 1 -- 172.21.15.104:0/2049720286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6cec0cd070 msgr2=0x7f6cec0b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf1e47700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6cec0cd070 0x7f6cec0b7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf1e47700 1 -- 172.21.15.104:0/2049720286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cec0cf1c0 msgr2=0x7f6cec156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf0e45700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cec0cf1c0 0x7f6cec156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf1e47700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cec0cf1c0 0x7f6cec156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf1e47700 1 -- 172.21.15.104:0/2049720286 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce8009570 con 0x7f6cec15bff0 2023-07-14T11:53:42.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf1646700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6cec0cd070 0x7f6cec0b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:42.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf0e45700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cec0cf1c0 0x7f6cec156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:42.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.448+0000 7f6cf1e47700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cec15bff0 0x7f6cec0b5d00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4007e60 tx=0x7f6ce400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:42.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.452+0000 7f6ce1ffb700 1 -- 172.21.15.104:0/2049720286 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce400ede0 con 0x7f6cec15bff0 2023-07-14T11:53:42.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.452+0000 7f6ce1ffb700 1 -- 172.21.15.104:0/2049720286 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce4014420 con 0x7f6cec15bff0 2023-07-14T11:53:42.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.452+0000 7f6cf2648700 1 -- 172.21.15.104:0/2049720286 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec15cc30 con 0x7f6cec15bff0 2023-07-14T11:53:42.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.452+0000 7f6ce1ffb700 1 -- 172.21.15.104:0/2049720286 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40147d0 con 0x7f6cec15bff0 2023-07-14T11:53:42.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.452+0000 7f6cf2648700 1 -- 172.21.15.104:0/2049720286 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec15d1f0 con 0x7f6cec15bff0 2023-07-14T11:53:42.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.452+0000 7f6cf2648700 1 -- 172.21.15.104:0/2049720286 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cec15d6d0 con 0x7f6cec15bff0 2023-07-14T11:53:42.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.456+0000 7f6ce1ffb700 1 -- 172.21.15.104:0/2049720286 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6ce4014930 con 0x7f6cec15bff0 2023-07-14T11:53:42.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.456+0000 7f6ce1ffb700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6cd00662f0 0x7f6cd00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.456+0000 7f6ce1ffb700 1 -- 172.21.15.104:0/2049720286 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6ce4084810 con 0x7f6cec15bff0 2023-07-14T11:53:42.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.456+0000 7f6cf1646700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6cd00662f0 0x7f6cd00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.456+0000 7f6cf1646700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6cd00662f0 0x7f6cd00687a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8000c00 tx=0x7f6ce8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:42.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.460+0000 7f6ce1ffb700 1 -- 172.21.15.104:0/2049720286 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f6ce4055420 con 0x7f6cec15bff0 2023-07-14T11:53:42.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.636+0000 7f6cf2648700 1 -- 172.21.15.104:0/2049720286 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cec04e500 con 0x7f6cec15bff0 2023-07-14T11:53:42.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.640+0000 7f6ce1ffb700 1 -- 172.21.15.104:0/2049720286 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20156 (secure 0 0 0) 0x7f6ce4054fb0 con 0x7f6cec15bff0 2023-07-14T11:53:42.624 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:42.624 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":28,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:20.091434+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:42.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.640+0000 7f6cf2648700 1 -- 172.21.15.104:0/2049720286 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6cd00662f0 msgr2=0x7f6cd00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.640+0000 7f6cf2648700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6cd00662f0 0x7f6cd00687a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8000c00 tx=0x7f6ce8008030 comp rx=0 tx=0).stop 2023-07-14T11:53:42.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.640+0000 7f6cf2648700 1 -- 172.21.15.104:0/2049720286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cec15bff0 msgr2=0x7f6cec0b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.640+0000 7f6cf2648700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cec15bff0 0x7f6cec0b5d00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4007e60 tx=0x7f6ce400e3c0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.644+0000 7f6cf2648700 1 -- 172.21.15.104:0/2049720286 shutdown_connections 2023-07-14T11:53:42.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.644+0000 7f6cf2648700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6cd00662f0 0x7f6cd00687a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.644+0000 7f6cf2648700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cec15bff0 0x7f6cec0b5d00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.644+0000 7f6cf2648700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cec0cf1c0 0x7f6cec156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.644+0000 7f6cf2648700 1 --2- 172.21.15.104:0/2049720286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6cec0cd070 0x7f6cec0b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.644+0000 7f6cf2648700 1 -- 172.21.15.104:0/2049720286 >> 172.21.15.104:0/2049720286 conn(0x7f6cec060c40 msgr2=0x7f6cec0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:42.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.644+0000 7f6cf2648700 1 -- 172.21.15.104:0/2049720286 shutdown_connections 2023-07-14T11:53:42.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.644+0000 7f6cf2648700 1 -- 172.21.15.104:0/2049720286 wait complete. 2023-07-14T11:53:42.648 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-14T11:53:42.649 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:53:42.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.828+0000 7fe50359e700 1 Processor -- start 2023-07-14T11:53:42.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.828+0000 7fe50359e700 1 -- start start 2023-07-14T11:53:42.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.828+0000 7fe50359e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe4fc0e12a0 0x7fe4fc037b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.828+0000 7fe50359e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe4fc0e1670 0x7fe4fc0e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.828+0000 7fe50359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0e4f50 0x7fe4fc036770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.828+0000 7fe50359e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4fc0e45f0 con 0x7fe4fc0e1670 2023-07-14T11:53:42.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.828+0000 7fe50359e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4fc038080 con 0x7fe4fc0e12a0 2023-07-14T11:53:42.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.828+0000 7fe50359e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0e0af0 con 0x7fe4fc0e4f50 2023-07-14T11:53:42.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50259c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe4fc0e12a0 0x7fe4fc037b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe502d9d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0e4f50 0x7fe4fc036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50259c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe4fc0e12a0 0x7fe4fc037b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57074/0 (socket says 172.21.15.104:57074) 2023-07-14T11:53:42.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50259c700 1 -- 172.21.15.104:0/3798926508 learned_addr learned my addr 172.21.15.104:0/3798926508 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:42.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe501d9b700 1 --2- 172.21.15.104:0/3798926508 >> v2:172.21.15.104:3300/0 conn(0x7fe4fc0e1670 0x7fe4fc0e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe502d9d700 1 -- 172.21.15.104:0/3798926508 >> v2:172.21.15.121:3300/0 conn(0x7fe4fc0e12a0 msgr2=0x7fe4fc037b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:42.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe502d9d700 1 --2- 172.21.15.104:0/3798926508 >> v2:172.21.15.121:3300/0 conn(0x7fe4fc0e12a0 0x7fe4fc037b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe502d9d700 1 -- 172.21.15.104:0/3798926508 >> v2:172.21.15.104:3300/0 conn(0x7fe4fc0e1670 msgr2=0x7fe4fc0e4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:42.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe502d9d700 1 --2- 172.21.15.104:0/3798926508 >> v2:172.21.15.104:3300/0 conn(0x7fe4fc0e1670 0x7fe4fc0e4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe502d9d700 1 -- 172.21.15.104:0/3798926508 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc050f00 con 0x7fe4fc0e4f50 2023-07-14T11:53:42.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50259c700 1 --2- 172.21.15.104:0/3798926508 >> v2:172.21.15.121:3300/0 conn(0x7fe4fc0e12a0 0x7fe4fc037b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:42.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe502d9d700 1 --2- 172.21.15.104:0/3798926508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0e4f50 0x7fe4fc036770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe50402c910 tx=0x7fe504073c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=964ae3d1443c74c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:42.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50159a700 1 -- 172.21.15.104:0/3798926508 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50406fae0 con 0x7fe4fc0e4f50 2023-07-14T11:53:42.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50159a700 1 -- 172.21.15.104:0/3798926508 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50406fc40 con 0x7fe4fc0e4f50 2023-07-14T11:53:42.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50159a700 1 -- 172.21.15.104:0/3798926508 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50406e130 con 0x7fe4fc0e4f50 2023-07-14T11:53:42.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50359e700 1 -- 172.21.15.104:0/3798926508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0e4f50 msgr2=0x7fe4fc036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50359e700 1 --2- 172.21.15.104:0/3798926508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0e4f50 0x7fe4fc036770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe50402c910 tx=0x7fe504073c10 comp rx=0 tx=0).stop 2023-07-14T11:53:42.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50359e700 1 -- 172.21.15.104:0/3798926508 shutdown_connections 2023-07-14T11:53:42.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50359e700 1 --2- 172.21.15.104:0/3798926508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0e4f50 0x7fe4fc036770 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50359e700 1 --2- 172.21.15.104:0/3798926508 >> v2:172.21.15.104:3300/0 conn(0x7fe4fc0e1670 0x7fe4fc0e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50359e700 1 --2- 172.21.15.104:0/3798926508 >> v2:172.21.15.121:3300/0 conn(0x7fe4fc0e12a0 0x7fe4fc037b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50359e700 1 -- 172.21.15.104:0/3798926508 >> 172.21.15.104:0/3798926508 conn(0x7fe4fc051770 msgr2=0x7fe4fc051b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:42.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50359e700 1 -- 172.21.15.104:0/3798926508 shutdown_connections 2023-07-14T11:53:42.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.832+0000 7fe50359e700 1 -- 172.21.15.104:0/3798926508 wait complete. 2023-07-14T11:53:42.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.836+0000 7fe50359e700 1 Processor -- start 2023-07-14T11:53:42.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.836+0000 7fe50359e700 1 -- start start 2023-07-14T11:53:42.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe50359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc0e12a0 0x7fe4fc0dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe50359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0e1670 0x7fe4fc0ddfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe50359e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4fc0e4f50 0x7fe4fc0de4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe50359e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc053390 con 0x7fe4fc0e4f50 2023-07-14T11:53:42.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe50359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0548f0 con 0x7fe4fc0e12a0 2023-07-14T11:53:42.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe50359e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0529f0 con 0x7fe4fc0e1670 2023-07-14T11:53:42.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe502d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4fc0e4f50 0x7fe4fc0de4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe502d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4fc0e4f50 0x7fe4fc0de4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38072/0 (socket says 172.21.15.104:38072) 2023-07-14T11:53:42.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe502d9d700 1 -- 172.21.15.104:0/3581895416 learned_addr learned my addr 172.21.15.104:0/3581895416 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:42.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe50259c700 1 --2- 172.21.15.104:0/3581895416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc0e12a0 0x7fe4fc0dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe501d9b700 1 --2- 172.21.15.104:0/3581895416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0e1670 0x7fe4fc0ddfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe502d9d700 1 -- 172.21.15.104:0/3581895416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0e1670 msgr2=0x7fe4fc0ddfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe502d9d700 1 --2- 172.21.15.104:0/3581895416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0e1670 0x7fe4fc0ddfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe502d9d700 1 -- 172.21.15.104:0/3581895416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc0e12a0 msgr2=0x7fe4fc0dbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:42.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe502d9d700 1 --2- 172.21.15.104:0/3581895416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc0e12a0 0x7fe4fc0dbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:42.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe502d9d700 1 -- 172.21.15.104:0/3581895416 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f4009570 con 0x7fe4fc0e4f50 2023-07-14T11:53:42.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe502d9d700 1 --2- 172.21.15.104:0/3581895416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4fc0e4f50 0x7fe4fc0de4e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe504067550 tx=0x7fe5040696c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:42.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe4faffd700 1 -- 172.21.15.104:0/3581895416 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504066830 con 0x7fe4fc0e4f50 2023-07-14T11:53:42.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe4faffd700 1 -- 172.21.15.104:0/3581895416 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe504066990 con 0x7fe4fc0e4f50 2023-07-14T11:53:42.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe4faffd700 1 -- 172.21.15.104:0/3581895416 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504066d00 con 0x7fe4fc0e4f50 2023-07-14T11:53:42.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe50359e700 1 -- 172.21.15.104:0/3581895416 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe504073710 con 0x7fe4fc0e4f50 2023-07-14T11:53:42.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.840+0000 7fe50359e700 1 -- 172.21.15.104:0/3581895416 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc052fb0 con 0x7fe4fc0e4f50 2023-07-14T11:53:42.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.844+0000 7fe50359e700 1 -- 172.21.15.104:0/3581895416 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4fc0534d0 con 0x7fe4fc0e4f50 2023-07-14T11:53:42.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.844+0000 7fe4faffd700 1 -- 172.21.15.104:0/3581895416 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe504072d30 con 0x7fe4fc0e4f50 2023-07-14T11:53:42.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.844+0000 7fe4faffd700 1 --2- 172.21.15.104:0/3581895416 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe4e40662f0 0x7fe4e40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:42.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.844+0000 7fe4faffd700 1 -- 172.21.15.104:0/3581895416 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe5040fe0b0 con 0x7fe4fc0e4f50 2023-07-14T11:53:42.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.844+0000 7fe50259c700 1 --2- 172.21.15.104:0/3581895416 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe4e40662f0 0x7fe4e40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:42.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.848+0000 7fe50259c700 1 --2- 172.21.15.104:0/3581895416 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe4e40662f0 0x7fe4e40687a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009c70 tx=0x7fe4f40092e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:42.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:42.852+0000 7fe4faffd700 1 -- 172.21.15.104:0/3581895416 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe4fc0534d0 con 0x7fe4fc0e4f50 2023-07-14T11:53:43.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.024+0000 7fe50359e700 1 -- 172.21.15.104:0/3581895416 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4fc051430 con 0x7fe4e40662f0 2023-07-14T11:53:43.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.036+0000 7fe4faffd700 1 -- 172.21.15.104:0/3581895416 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367160 (secure 0 0 0) 0x7fe4f4009db0 con 0x7fe4e40662f0 2023-07-14T11:53:43.027 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:43.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.044+0000 7fe50359e700 1 -- 172.21.15.104:0/3581895416 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe4e40662f0 msgr2=0x7fe4e40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.044+0000 7fe50359e700 1 --2- 172.21.15.104:0/3581895416 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe4e40662f0 0x7fe4e40687a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009c70 tx=0x7fe4f40092e0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.044+0000 7fe50359e700 1 -- 172.21.15.104:0/3581895416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4fc0e4f50 msgr2=0x7fe4fc0de4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.044+0000 7fe50359e700 1 --2- 172.21.15.104:0/3581895416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4fc0e4f50 0x7fe4fc0de4e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe504067550 tx=0x7fe5040696c0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.044+0000 7fe50359e700 1 -- 172.21.15.104:0/3581895416 shutdown_connections 2023-07-14T11:53:43.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.044+0000 7fe50359e700 1 --2- 172.21.15.104:0/3581895416 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe4e40662f0 0x7fe4e40687a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.044+0000 7fe50359e700 1 --2- 172.21.15.104:0/3581895416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4fc0e4f50 0x7fe4fc0de4e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.044+0000 7fe50359e700 1 --2- 172.21.15.104:0/3581895416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4fc0e1670 0x7fe4fc0ddfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.044+0000 7fe50359e700 1 --2- 172.21.15.104:0/3581895416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4fc0e12a0 0x7fe4fc0dbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.044+0000 7fe50359e700 1 -- 172.21.15.104:0/3581895416 >> 172.21.15.104:0/3581895416 conn(0x7fe4fc051770 msgr2=0x7fe4fc048aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:43.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.044+0000 7fe50359e700 1 -- 172.21.15.104:0/3581895416 shutdown_connections 2023-07-14T11:53:43.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.044+0000 7fe50359e700 1 -- 172.21.15.104:0/3581895416 wait complete. 2023-07-14T11:53:43.031 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:53:43.049 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2023-07-14T11:53:41.894235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594959,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219208,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755512,"statfs":{"total":773094113280,"available":772869644288,"internally_reserved":0,"allocated":3678208,"data_stored":2186494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004542"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877499+0000","last_change":"2023-07-14T11:53:19.666668+0000","last_active":"2023-07-14T11:53:20.877499+0000","last_peered":"2023-07-14T11:53:20.877499+0000","last_clean":"2023-07-14T11:53:20.877499+0000","last_became_active":"2023-07-14T11:53:19.665905+0000","last_became_peered":"2023-07-14T11:53:19.665905+0000","last_unstale":"2023-07-14T11:53:20.877499+0000","last_undegraded":"2023-07-14T11:53:20.877499+0000","last_fullsized":"2023-07-14T11:53:20.877499+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:59.851971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568528+0000","last_change":"2023-07-14T11:53:14.600663+0000","last_active":"2023-07-14T11:53:21.568528+0000","last_peered":"2023-07-14T11:53:21.568528+0000","last_clean":"2023-07-14T11:53:21.568528+0000","last_became_active":"2023-07-14T11:53:14.600396+0000","last_became_peered":"2023-07-14T11:53:14.600396+0000","last_unstale":"2023-07-14T11:53:21.568528+0000","last_undegraded":"2023-07-14T11:53:21.568528+0000","last_fullsized":"2023-07-14T11:53:21.568528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:38.720680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662714+0000","last_change":"2023-07-14T11:53:15.580781+0000","last_active":"2023-07-14T11:53:20.662714+0000","last_peered":"2023-07-14T11:53:20.662714+0000","last_clean":"2023-07-14T11:53:20.662714+0000","last_became_active":"2023-07-14T11:53:15.580452+0000","last_became_peered":"2023-07-14T11:53:15.580452+0000","last_unstale":"2023-07-14T11:53:20.662714+0000","last_undegraded":"2023-07-14T11:53:20.662714+0000","last_fullsized":"2023-07-14T11:53:20.662714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:20:09.506259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662575+0000","last_change":"2023-07-14T11:53:19.666546+0000","last_active":"2023-07-14T11:53:20.662575+0000","last_peered":"2023-07-14T11:53:20.662575+0000","last_clean":"2023-07-14T11:53:20.662575+0000","last_became_active":"2023-07-14T11:53:19.666168+0000","last_became_peered":"2023-07-14T11:53:19.666168+0000","last_unstale":"2023-07-14T11:53:20.662575+0000","last_undegraded":"2023-07-14T11:53:20.662575+0000","last_fullsized":"2023-07-14T11:53:20.662575+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:14.566966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569423+0000","last_change":"2023-07-14T11:53:14.586141+0000","last_active":"2023-07-14T11:53:21.569423+0000","last_peered":"2023-07-14T11:53:21.569423+0000","last_clean":"2023-07-14T11:53:21.569423+0000","last_became_active":"2023-07-14T11:53:14.585836+0000","last_became_peered":"2023-07-14T11:53:14.585836+0000","last_unstale":"2023-07-14T11:53:21.569423+0000","last_undegraded":"2023-07-14T11:53:21.569423+0000","last_fullsized":"2023-07-14T11:53:21.569423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:42:48.188371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568268+0000","last_change":"2023-07-14T11:53:15.582115+0000","last_active":"2023-07-14T11:53:21.568268+0000","last_peered":"2023-07-14T11:53:21.568268+0000","last_clean":"2023-07-14T11:53:21.568268+0000","last_became_active":"2023-07-14T11:53:15.581864+0000","last_became_peered":"2023-07-14T11:53:15.581864+0000","last_unstale":"2023-07-14T11:53:21.568268+0000","last_undegraded":"2023-07-14T11:53:21.568268+0000","last_fullsized":"2023-07-14T11:53:21.568268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:31.536389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566924+0000","last_change":"2023-07-14T11:53:19.680406+0000","last_active":"2023-07-14T11:53:21.566924+0000","last_peered":"2023-07-14T11:53:21.566924+0000","last_clean":"2023-07-14T11:53:21.566924+0000","last_became_active":"2023-07-14T11:53:19.680059+0000","last_became_peered":"2023-07-14T11:53:19.680059+0000","last_unstale":"2023-07-14T11:53:21.566924+0000","last_undegraded":"2023-07-14T11:53:21.566924+0000","last_fullsized":"2023-07-14T11:53:21.566924+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:43.799463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567583+0000","last_change":"2023-07-14T11:53:14.596827+0000","last_active":"2023-07-14T11:53:21.567583+0000","last_peered":"2023-07-14T11:53:21.567583+0000","last_clean":"2023-07-14T11:53:21.567583+0000","last_became_active":"2023-07-14T11:53:14.596221+0000","last_became_peered":"2023-07-14T11:53:14.596221+0000","last_unstale":"2023-07-14T11:53:21.567583+0000","last_undegraded":"2023-07-14T11:53:21.567583+0000","last_fullsized":"2023-07-14T11:53:21.567583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:54:59.284344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663572+0000","last_change":"2023-07-14T11:53:15.607538+0000","last_active":"2023-07-14T11:53:20.663572+0000","last_peered":"2023-07-14T11:53:20.663572+0000","last_clean":"2023-07-14T11:53:20.663572+0000","last_became_active":"2023-07-14T11:53:15.592890+0000","last_became_peered":"2023-07-14T11:53:15.592890+0000","last_unstale":"2023-07-14T11:53:20.663572+0000","last_undegraded":"2023-07-14T11:53:20.663572+0000","last_fullsized":"2023-07-14T11:53:20.663572+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:30:22.821980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568105+0000","last_change":"2023-07-14T11:53:19.662893+0000","last_active":"2023-07-14T11:53:21.568105+0000","last_peered":"2023-07-14T11:53:21.568105+0000","last_clean":"2023-07-14T11:53:21.568105+0000","last_became_active":"2023-07-14T11:53:19.662484+0000","last_became_peered":"2023-07-14T11:53:19.662484+0000","last_unstale":"2023-07-14T11:53:21.568105+0000","last_undegraded":"2023-07-14T11:53:21.568105+0000","last_fullsized":"2023-07-14T11:53:21.568105+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:43.799463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566672+0000","last_change":"2023-07-14T11:53:14.590222+0000","last_active":"2023-07-14T11:53:21.566672+0000","last_peered":"2023-07-14T11:53:21.566672+0000","last_clean":"2023-07-14T11:53:21.566672+0000","last_became_active":"2023-07-14T11:53:14.589684+0000","last_became_peered":"2023-07-14T11:53:14.589684+0000","last_unstale":"2023-07-14T11:53:21.566672+0000","last_undegraded":"2023-07-14T11:53:21.566672+0000","last_fullsized":"2023-07-14T11:53:21.566672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:43.338727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663068+0000","last_change":"2023-07-14T11:53:15.592933+0000","last_active":"2023-07-14T11:53:20.663068+0000","last_peered":"2023-07-14T11:53:20.663068+0000","last_clean":"2023-07-14T11:53:20.663068+0000","last_became_active":"2023-07-14T11:53:15.592619+0000","last_became_peered":"2023-07-14T11:53:15.592619+0000","last_unstale":"2023-07-14T11:53:20.663068+0000","last_undegraded":"2023-07-14T11:53:20.663068+0000","last_fullsized":"2023-07-14T11:53:20.663068+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:41:38.469614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878123+0000","last_change":"2023-07-14T11:53:19.683165+0000","last_active":"2023-07-14T11:53:20.878123+0000","last_peered":"2023-07-14T11:53:20.878123+0000","last_clean":"2023-07-14T11:53:20.878123+0000","last_became_active":"2023-07-14T11:53:19.682676+0000","last_became_peered":"2023-07-14T11:53:19.682676+0000","last_unstale":"2023-07-14T11:53:20.878123+0000","last_undegraded":"2023-07-14T11:53:20.878123+0000","last_fullsized":"2023-07-14T11:53:20.878123+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:36:19.425078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569431+0000","last_change":"2023-07-14T11:53:14.592834+0000","last_active":"2023-07-14T11:53:21.569431+0000","last_peered":"2023-07-14T11:53:21.569431+0000","last_clean":"2023-07-14T11:53:21.569431+0000","last_became_active":"2023-07-14T11:53:14.592535+0000","last_became_peered":"2023-07-14T11:53:14.592535+0000","last_unstale":"2023-07-14T11:53:21.569431+0000","last_undegraded":"2023-07-14T11:53:21.569431+0000","last_fullsized":"2023-07-14T11:53:21.569431+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:39.643364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661401+0000","last_change":"2023-07-14T11:53:15.593173+0000","last_active":"2023-07-14T11:53:20.661401+0000","last_peered":"2023-07-14T11:53:20.661401+0000","last_clean":"2023-07-14T11:53:20.661401+0000","last_became_active":"2023-07-14T11:53:15.592873+0000","last_became_peered":"2023-07-14T11:53:15.592873+0000","last_unstale":"2023-07-14T11:53:20.661401+0000","last_undegraded":"2023-07-14T11:53:20.661401+0000","last_fullsized":"2023-07-14T11:53:20.661401+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:59:03.301440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663003+0000","last_change":"2023-07-14T11:53:19.672048+0000","last_active":"2023-07-14T11:53:20.663003+0000","last_peered":"2023-07-14T11:53:20.663003+0000","last_clean":"2023-07-14T11:53:20.663003+0000","last_became_active":"2023-07-14T11:53:19.671493+0000","last_became_peered":"2023-07-14T11:53:19.671493+0000","last_unstale":"2023-07-14T11:53:20.663003+0000","last_undegraded":"2023-07-14T11:53:20.663003+0000","last_fullsized":"2023-07-14T11:53:20.663003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:34:10.732809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664092+0000","last_change":"2023-07-14T11:53:14.591651+0000","last_active":"2023-07-14T11:53:20.664092+0000","last_peered":"2023-07-14T11:53:20.664092+0000","last_clean":"2023-07-14T11:53:20.664092+0000","last_became_active":"2023-07-14T11:53:14.591356+0000","last_became_peered":"2023-07-14T11:53:14.591356+0000","last_unstale":"2023-07-14T11:53:20.664092+0000","last_undegraded":"2023-07-14T11:53:20.664092+0000","last_fullsized":"2023-07-14T11:53:20.664092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:38:54.408913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568174+0000","last_change":"2023-07-14T11:53:15.596998+0000","last_active":"2023-07-14T11:53:21.568174+0000","last_peered":"2023-07-14T11:53:21.568174+0000","last_clean":"2023-07-14T11:53:21.568174+0000","last_became_active":"2023-07-14T11:53:15.596278+0000","last_became_peered":"2023-07-14T11:53:15.596278+0000","last_unstale":"2023-07-14T11:53:21.568174+0000","last_undegraded":"2023-07-14T11:53:21.568174+0000","last_fullsized":"2023-07-14T11:53:21.568174+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:39.727537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665493+0000","last_change":"2023-07-14T11:53:19.664365+0000","last_active":"2023-07-14T11:53:20.665493+0000","last_peered":"2023-07-14T11:53:20.665493+0000","last_clean":"2023-07-14T11:53:20.665493+0000","last_became_active":"2023-07-14T11:53:19.663967+0000","last_became_peered":"2023-07-14T11:53:19.663967+0000","last_unstale":"2023-07-14T11:53:20.665493+0000","last_undegraded":"2023-07-14T11:53:20.665493+0000","last_fullsized":"2023-07-14T11:53:20.665493+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:37:34.816983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877811+0000","last_change":"2023-07-14T11:53:14.590756+0000","last_active":"2023-07-14T11:53:20.877811+0000","last_peered":"2023-07-14T11:53:20.877811+0000","last_clean":"2023-07-14T11:53:20.877811+0000","last_became_active":"2023-07-14T11:53:14.590511+0000","last_became_peered":"2023-07-14T11:53:14.590511+0000","last_unstale":"2023-07-14T11:53:20.877811+0000","last_undegraded":"2023-07-14T11:53:20.877811+0000","last_fullsized":"2023-07-14T11:53:20.877811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:18:16.526406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568118+0000","last_change":"2023-07-14T11:53:15.612239+0000","last_active":"2023-07-14T11:53:21.568118+0000","last_peered":"2023-07-14T11:53:21.568118+0000","last_clean":"2023-07-14T11:53:21.568118+0000","last_became_active":"2023-07-14T11:53:15.611948+0000","last_became_peered":"2023-07-14T11:53:15.611948+0000","last_unstale":"2023-07-14T11:53:21.568118+0000","last_undegraded":"2023-07-14T11:53:21.568118+0000","last_fullsized":"2023-07-14T11:53:21.568118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:44.345584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567809+0000","last_change":"2023-07-14T11:53:19.680386+0000","last_active":"2023-07-14T11:53:21.567809+0000","last_peered":"2023-07-14T11:53:21.567809+0000","last_clean":"2023-07-14T11:53:21.567809+0000","last_became_active":"2023-07-14T11:53:19.674870+0000","last_became_peered":"2023-07-14T11:53:19.674870+0000","last_unstale":"2023-07-14T11:53:21.567809+0000","last_undegraded":"2023-07-14T11:53:21.567809+0000","last_fullsized":"2023-07-14T11:53:21.567809+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:00.015845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568390+0000","last_change":"2023-07-14T11:53:14.592538+0000","last_active":"2023-07-14T11:53:21.568390+0000","last_peered":"2023-07-14T11:53:21.568390+0000","last_clean":"2023-07-14T11:53:21.568390+0000","last_became_active":"2023-07-14T11:53:14.592269+0000","last_became_peered":"2023-07-14T11:53:14.592269+0000","last_unstale":"2023-07-14T11:53:21.568390+0000","last_undegraded":"2023-07-14T11:53:21.568390+0000","last_fullsized":"2023-07-14T11:53:21.568390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:17.757069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665555+0000","last_change":"2023-07-14T11:53:15.587863+0000","last_active":"2023-07-14T11:53:20.665555+0000","last_peered":"2023-07-14T11:53:20.665555+0000","last_clean":"2023-07-14T11:53:20.665555+0000","last_became_active":"2023-07-14T11:53:15.587560+0000","last_became_peered":"2023-07-14T11:53:15.587560+0000","last_unstale":"2023-07-14T11:53:20.665555+0000","last_undegraded":"2023-07-14T11:53:20.665555+0000","last_fullsized":"2023-07-14T11:53:20.665555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:01:34.117398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568069+0000","last_change":"2023-07-14T11:53:15.596816+0000","last_active":"2023-07-14T11:53:21.568069+0000","last_peered":"2023-07-14T11:53:21.568069+0000","last_clean":"2023-07-14T11:53:21.568069+0000","last_became_active":"2023-07-14T11:53:15.596544+0000","last_became_peered":"2023-07-14T11:53:15.596544+0000","last_unstale":"2023-07-14T11:53:21.568069+0000","last_undegraded":"2023-07-14T11:53:21.568069+0000","last_fullsized":"2023-07-14T11:53:21.568069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:13.267093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568721+0000","last_change":"2023-07-14T11:53:14.600709+0000","last_active":"2023-07-14T11:53:21.568721+0000","last_peered":"2023-07-14T11:53:21.568721+0000","last_clean":"2023-07-14T11:53:21.568721+0000","last_became_active":"2023-07-14T11:53:14.600430+0000","last_became_peered":"2023-07-14T11:53:14.600430+0000","last_unstale":"2023-07-14T11:53:21.568721+0000","last_undegraded":"2023-07-14T11:53:21.568721+0000","last_fullsized":"2023-07-14T11:53:21.568721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:30.529533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664703+0000","last_change":"2023-07-14T11:53:19.664675+0000","last_active":"2023-07-14T11:53:20.664703+0000","last_peered":"2023-07-14T11:53:20.664703+0000","last_clean":"2023-07-14T11:53:20.664703+0000","last_became_active":"2023-07-14T11:53:19.663889+0000","last_became_peered":"2023-07-14T11:53:19.663889+0000","last_unstale":"2023-07-14T11:53:20.664703+0000","last_undegraded":"2023-07-14T11:53:20.664703+0000","last_fullsized":"2023-07-14T11:53:20.664703+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:46.192512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568024+0000","last_change":"2023-07-14T11:53:15.606972+0000","last_active":"2023-07-14T11:53:21.568024+0000","last_peered":"2023-07-14T11:53:21.568024+0000","last_clean":"2023-07-14T11:53:21.568024+0000","last_became_active":"2023-07-14T11:53:15.606615+0000","last_became_peered":"2023-07-14T11:53:15.606615+0000","last_unstale":"2023-07-14T11:53:21.568024+0000","last_undegraded":"2023-07-14T11:53:21.568024+0000","last_fullsized":"2023-07-14T11:53:21.568024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:29.641887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569420+0000","last_change":"2023-07-14T11:53:14.595852+0000","last_active":"2023-07-14T11:53:21.569420+0000","last_peered":"2023-07-14T11:53:21.569420+0000","last_clean":"2023-07-14T11:53:21.569420+0000","last_became_active":"2023-07-14T11:53:14.589714+0000","last_became_peered":"2023-07-14T11:53:14.589714+0000","last_unstale":"2023-07-14T11:53:21.569420+0000","last_undegraded":"2023-07-14T11:53:21.569420+0000","last_fullsized":"2023-07-14T11:53:21.569420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:30:20.713856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567265+0000","last_change":"2023-07-14T11:53:19.675250+0000","last_active":"2023-07-14T11:53:21.567265+0000","last_peered":"2023-07-14T11:53:21.567265+0000","last_clean":"2023-07-14T11:53:21.567265+0000","last_became_active":"2023-07-14T11:53:19.674805+0000","last_became_peered":"2023-07-14T11:53:19.674805+0000","last_unstale":"2023-07-14T11:53:21.567265+0000","last_undegraded":"2023-07-14T11:53:21.567265+0000","last_fullsized":"2023-07-14T11:53:21.567265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:44:55.840426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568563+0000","last_change":"2023-07-14T11:53:14.587048+0000","last_active":"2023-07-14T11:53:21.568563+0000","last_peered":"2023-07-14T11:53:21.568563+0000","last_clean":"2023-07-14T11:53:21.568563+0000","last_became_active":"2023-07-14T11:53:14.586722+0000","last_became_peered":"2023-07-14T11:53:14.586722+0000","last_unstale":"2023-07-14T11:53:21.568563+0000","last_undegraded":"2023-07-14T11:53:21.568563+0000","last_fullsized":"2023-07-14T11:53:21.568563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:17.757069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878086+0000","last_change":"2023-07-14T11:53:15.598564+0000","last_active":"2023-07-14T11:53:20.878086+0000","last_peered":"2023-07-14T11:53:20.878086+0000","last_clean":"2023-07-14T11:53:20.878086+0000","last_became_active":"2023-07-14T11:53:15.596542+0000","last_became_peered":"2023-07-14T11:53:15.596542+0000","last_unstale":"2023-07-14T11:53:20.878086+0000","last_undegraded":"2023-07-14T11:53:20.878086+0000","last_fullsized":"2023-07-14T11:53:20.878086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:44:51.768501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568782+0000","last_change":"2023-07-14T11:53:19.670353+0000","last_active":"2023-07-14T11:53:21.568782+0000","last_peered":"2023-07-14T11:53:21.568782+0000","last_clean":"2023-07-14T11:53:21.568782+0000","last_became_active":"2023-07-14T11:53:19.669974+0000","last_became_peered":"2023-07-14T11:53:19.669974+0000","last_unstale":"2023-07-14T11:53:21.568782+0000","last_undegraded":"2023-07-14T11:53:21.568782+0000","last_fullsized":"2023-07-14T11:53:21.568782+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:43.632890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:33.130929+0000","last_change":"2023-07-14T11:53:14.586133+0000","last_active":"2023-07-14T11:53:33.130929+0000","last_peered":"2023-07-14T11:53:33.130929+0000","last_clean":"2023-07-14T11:53:33.130929+0000","last_became_active":"2023-07-14T11:53:14.585705+0000","last_became_peered":"2023-07-14T11:53:14.585705+0000","last_unstale":"2023-07-14T11:53:33.130929+0000","last_undegraded":"2023-07-14T11:53:33.130929+0000","last_fullsized":"2023-07-14T11:53:33.130929+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:48:36.912719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3793,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663286+0000","last_change":"2023-07-14T11:53:15.581733+0000","last_active":"2023-07-14T11:53:20.663286+0000","last_peered":"2023-07-14T11:53:20.663286+0000","last_clean":"2023-07-14T11:53:20.663286+0000","last_became_active":"2023-07-14T11:53:15.581354+0000","last_became_peered":"2023-07-14T11:53:15.581354+0000","last_unstale":"2023-07-14T11:53:20.663286+0000","last_undegraded":"2023-07-14T11:53:20.663286+0000","last_fullsized":"2023-07-14T11:53:20.663286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:24:39.278456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877743+0000","last_change":"2023-07-14T11:53:19.678063+0000","last_active":"2023-07-14T11:53:20.877743+0000","last_peered":"2023-07-14T11:53:20.877743+0000","last_clean":"2023-07-14T11:53:20.877743+0000","last_became_active":"2023-07-14T11:53:19.677638+0000","last_became_peered":"2023-07-14T11:53:19.677638+0000","last_unstale":"2023-07-14T11:53:20.877743+0000","last_undegraded":"2023-07-14T11:53:20.877743+0000","last_fullsized":"2023-07-14T11:53:20.877743+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:32:43.273428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663397+0000","last_change":"2023-07-14T11:53:15.590752+0000","last_active":"2023-07-14T11:53:20.663397+0000","last_peered":"2023-07-14T11:53:20.663397+0000","last_clean":"2023-07-14T11:53:20.663397+0000","last_became_active":"2023-07-14T11:53:15.589947+0000","last_became_peered":"2023-07-14T11:53:15.589947+0000","last_unstale":"2023-07-14T11:53:20.663397+0000","last_undegraded":"2023-07-14T11:53:20.663397+0000","last_fullsized":"2023-07-14T11:53:20.663397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:13:58.842089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665649+0000","last_change":"2023-07-14T11:53:09.145201+0000","last_active":"2023-07-14T11:53:20.665649+0000","last_peered":"2023-07-14T11:53:20.665649+0000","last_clean":"2023-07-14T11:53:20.665649+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:53:20.665649+0000","last_undegraded":"2023-07-14T11:53:20.665649+0000","last_fullsized":"2023-07-14T11:53:20.665649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:54.319358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568612+0000","last_change":"2023-07-14T11:53:14.592544+0000","last_active":"2023-07-14T11:53:21.568612+0000","last_peered":"2023-07-14T11:53:21.568612+0000","last_clean":"2023-07-14T11:53:21.568612+0000","last_became_active":"2023-07-14T11:53:14.592204+0000","last_became_peered":"2023-07-14T11:53:14.592204+0000","last_unstale":"2023-07-14T11:53:21.568612+0000","last_undegraded":"2023-07-14T11:53:21.568612+0000","last_fullsized":"2023-07-14T11:53:21.568612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:12.881399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567791+0000","last_change":"2023-07-14T11:53:19.676141+0000","last_active":"2023-07-14T11:53:21.567791+0000","last_peered":"2023-07-14T11:53:21.567791+0000","last_clean":"2023-07-14T11:53:21.567791+0000","last_became_active":"2023-07-14T11:53:19.675720+0000","last_became_peered":"2023-07-14T11:53:19.675720+0000","last_unstale":"2023-07-14T11:53:21.567791+0000","last_undegraded":"2023-07-14T11:53:21.567791+0000","last_fullsized":"2023-07-14T11:53:21.567791+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:48.417509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568246+0000","last_change":"2023-07-14T11:53:15.607026+0000","last_active":"2023-07-14T11:53:21.568246+0000","last_peered":"2023-07-14T11:53:21.568246+0000","last_clean":"2023-07-14T11:53:21.568246+0000","last_became_active":"2023-07-14T11:53:15.606605+0000","last_became_peered":"2023-07-14T11:53:15.606605+0000","last_unstale":"2023-07-14T11:53:21.568246+0000","last_undegraded":"2023-07-14T11:53:21.568246+0000","last_fullsized":"2023-07-14T11:53:21.568246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:13.888256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878901+0000","last_change":"2023-07-14T11:53:09.146150+0000","last_active":"2023-07-14T11:53:20.878901+0000","last_peered":"2023-07-14T11:53:20.878901+0000","last_clean":"2023-07-14T11:53:20.878901+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:53:20.878901+0000","last_undegraded":"2023-07-14T11:53:20.878901+0000","last_fullsized":"2023-07-14T11:53:20.878901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:07.460041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569870+0000","last_change":"2023-07-14T11:53:14.586533+0000","last_active":"2023-07-14T11:53:21.569870+0000","last_peered":"2023-07-14T11:53:21.569870+0000","last_clean":"2023-07-14T11:53:21.569870+0000","last_became_active":"2023-07-14T11:53:14.586222+0000","last_became_peered":"2023-07-14T11:53:14.586222+0000","last_unstale":"2023-07-14T11:53:21.569870+0000","last_undegraded":"2023-07-14T11:53:21.569870+0000","last_fullsized":"2023-07-14T11:53:21.569870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:38:03.789491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662421+0000","last_change":"2023-07-14T11:53:19.675677+0000","last_active":"2023-07-14T11:53:20.662421+0000","last_peered":"2023-07-14T11:53:20.662421+0000","last_clean":"2023-07-14T11:53:20.662421+0000","last_became_active":"2023-07-14T11:53:19.675259+0000","last_became_peered":"2023-07-14T11:53:19.675259+0000","last_unstale":"2023-07-14T11:53:20.662421+0000","last_undegraded":"2023-07-14T11:53:20.662421+0000","last_fullsized":"2023-07-14T11:53:20.662421+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:05.121760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567406+0000","last_change":"2023-07-14T11:53:15.600612+0000","last_active":"2023-07-14T11:53:21.567406+0000","last_peered":"2023-07-14T11:53:21.567406+0000","last_clean":"2023-07-14T11:53:21.567406+0000","last_became_active":"2023-07-14T11:53:15.600320+0000","last_became_peered":"2023-07-14T11:53:15.600320+0000","last_unstale":"2023-07-14T11:53:21.567406+0000","last_undegraded":"2023-07-14T11:53:21.567406+0000","last_fullsized":"2023-07-14T11:53:21.567406+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:55:00.291201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568702+0000","last_change":"2023-07-14T11:53:09.435876+0000","last_active":"2023-07-14T11:53:21.568702+0000","last_peered":"2023-07-14T11:53:21.568702+0000","last_clean":"2023-07-14T11:53:21.568702+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:53:21.568702+0000","last_undegraded":"2023-07-14T11:53:21.568702+0000","last_fullsized":"2023-07-14T11:53:21.568702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:34.222006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878739+0000","last_change":"2023-07-14T11:53:14.594216+0000","last_active":"2023-07-14T11:53:20.878739+0000","last_peered":"2023-07-14T11:53:20.878739+0000","last_clean":"2023-07-14T11:53:20.878739+0000","last_became_active":"2023-07-14T11:53:14.593954+0000","last_became_peered":"2023-07-14T11:53:14.593954+0000","last_unstale":"2023-07-14T11:53:20.878739+0000","last_undegraded":"2023-07-14T11:53:20.878739+0000","last_fullsized":"2023-07-14T11:53:20.878739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:12.260236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665060+0000","last_change":"2023-07-14T11:53:19.667039+0000","last_active":"2023-07-14T11:53:20.665060+0000","last_peered":"2023-07-14T11:53:20.665060+0000","last_clean":"2023-07-14T11:53:20.665060+0000","last_became_active":"2023-07-14T11:53:19.666360+0000","last_became_peered":"2023-07-14T11:53:19.666360+0000","last_unstale":"2023-07-14T11:53:20.665060+0000","last_undegraded":"2023-07-14T11:53:20.665060+0000","last_fullsized":"2023-07-14T11:53:20.665060+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:37:11.705483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878456+0000","last_change":"2023-07-14T11:53:15.598379+0000","last_active":"2023-07-14T11:53:20.878456+0000","last_peered":"2023-07-14T11:53:20.878456+0000","last_clean":"2023-07-14T11:53:20.878456+0000","last_became_active":"2023-07-14T11:53:15.596661+0000","last_became_peered":"2023-07-14T11:53:15.596661+0000","last_unstale":"2023-07-14T11:53:20.878456+0000","last_undegraded":"2023-07-14T11:53:20.878456+0000","last_fullsized":"2023-07-14T11:53:20.878456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:44.345584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878968+0000","last_change":"2023-07-14T11:53:09.434255+0000","last_active":"2023-07-14T11:53:20.878968+0000","last_peered":"2023-07-14T11:53:20.878968+0000","last_clean":"2023-07-14T11:53:20.878968+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:53:20.878968+0000","last_undegraded":"2023-07-14T11:53:20.878968+0000","last_fullsized":"2023-07-14T11:53:20.878968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:12.335710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567656+0000","last_change":"2023-07-14T11:53:14.589144+0000","last_active":"2023-07-14T11:53:21.567656+0000","last_peered":"2023-07-14T11:53:21.567656+0000","last_clean":"2023-07-14T11:53:21.567656+0000","last_became_active":"2023-07-14T11:53:14.588849+0000","last_became_peered":"2023-07-14T11:53:14.588849+0000","last_unstale":"2023-07-14T11:53:21.567656+0000","last_undegraded":"2023-07-14T11:53:21.567656+0000","last_fullsized":"2023-07-14T11:53:21.567656+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:38.554107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567120+0000","last_change":"2023-07-14T11:53:19.672406+0000","last_active":"2023-07-14T11:53:21.567120+0000","last_peered":"2023-07-14T11:53:21.567120+0000","last_clean":"2023-07-14T11:53:21.567120+0000","last_became_active":"2023-07-14T11:53:19.672009+0000","last_became_peered":"2023-07-14T11:53:19.672009+0000","last_unstale":"2023-07-14T11:53:21.567120+0000","last_undegraded":"2023-07-14T11:53:21.567120+0000","last_fullsized":"2023-07-14T11:53:21.567120+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:36:19.425078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878726+0000","last_change":"2023-07-14T11:53:15.604123+0000","last_active":"2023-07-14T11:53:20.878726+0000","last_peered":"2023-07-14T11:53:20.878726+0000","last_clean":"2023-07-14T11:53:20.878726+0000","last_became_active":"2023-07-14T11:53:15.603815+0000","last_became_peered":"2023-07-14T11:53:15.603815+0000","last_unstale":"2023-07-14T11:53:20.878726+0000","last_undegraded":"2023-07-14T11:53:20.878726+0000","last_fullsized":"2023-07-14T11:53:20.878726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:29.641887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663321+0000","last_change":"2023-07-14T11:53:14.591218+0000","last_active":"2023-07-14T11:53:20.663321+0000","last_peered":"2023-07-14T11:53:20.663321+0000","last_clean":"2023-07-14T11:53:20.663321+0000","last_became_active":"2023-07-14T11:53:14.589122+0000","last_became_peered":"2023-07-14T11:53:14.589122+0000","last_unstale":"2023-07-14T11:53:20.663321+0000","last_undegraded":"2023-07-14T11:53:20.663321+0000","last_fullsized":"2023-07-14T11:53:20.663321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:51:10.380803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566815+0000","last_change":"2023-07-14T11:53:09.147040+0000","last_active":"2023-07-14T11:53:21.566815+0000","last_peered":"2023-07-14T11:53:21.566815+0000","last_clean":"2023-07-14T11:53:21.566815+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:53:21.566815+0000","last_undegraded":"2023-07-14T11:53:21.566815+0000","last_fullsized":"2023-07-14T11:53:21.566815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:23.213672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567493+0000","last_change":"2023-07-14T11:53:19.680353+0000","last_active":"2023-07-14T11:53:21.567493+0000","last_peered":"2023-07-14T11:53:21.567493+0000","last_clean":"2023-07-14T11:53:21.567493+0000","last_became_active":"2023-07-14T11:53:19.674999+0000","last_became_peered":"2023-07-14T11:53:19.674999+0000","last_unstale":"2023-07-14T11:53:21.567493+0000","last_undegraded":"2023-07-14T11:53:21.567493+0000","last_fullsized":"2023-07-14T11:53:21.567493+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:08:07.808470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878795+0000","last_change":"2023-07-14T11:53:15.597158+0000","last_active":"2023-07-14T11:53:20.878795+0000","last_peered":"2023-07-14T11:53:20.878795+0000","last_clean":"2023-07-14T11:53:20.878795+0000","last_became_active":"2023-07-14T11:53:15.595897+0000","last_became_peered":"2023-07-14T11:53:15.595897+0000","last_unstale":"2023-07-14T11:53:20.878795+0000","last_undegraded":"2023-07-14T11:53:20.878795+0000","last_fullsized":"2023-07-14T11:53:20.878795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:39.727537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877308+0000","last_change":"2023-07-14T11:53:14.589940+0000","last_active":"2023-07-14T11:53:20.877308+0000","last_peered":"2023-07-14T11:53:20.877308+0000","last_clean":"2023-07-14T11:53:20.877308+0000","last_became_active":"2023-07-14T11:53:14.589607+0000","last_became_peered":"2023-07-14T11:53:14.589607+0000","last_unstale":"2023-07-14T11:53:20.877308+0000","last_undegraded":"2023-07-14T11:53:20.877308+0000","last_fullsized":"2023-07-14T11:53:20.877308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:28:09.703297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660746+0000","last_change":"2023-07-14T11:53:09.619294+0000","last_active":"2023-07-14T11:53:20.660746+0000","last_peered":"2023-07-14T11:53:20.660746+0000","last_clean":"2023-07-14T11:53:20.660746+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:53:20.660746+0000","last_undegraded":"2023-07-14T11:53:20.660746+0000","last_fullsized":"2023-07-14T11:53:20.660746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:19:36.961857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661632+0000","last_change":"2023-07-14T11:53:19.666668+0000","last_active":"2023-07-14T11:53:20.661632+0000","last_peered":"2023-07-14T11:53:20.661632+0000","last_clean":"2023-07-14T11:53:20.661632+0000","last_became_active":"2023-07-14T11:53:19.666170+0000","last_became_peered":"2023-07-14T11:53:19.666170+0000","last_unstale":"2023-07-14T11:53:20.661632+0000","last_undegraded":"2023-07-14T11:53:20.661632+0000","last_fullsized":"2023-07-14T11:53:20.661632+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:11:15.273201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568313+0000","last_change":"2023-07-14T11:53:15.611544+0000","last_active":"2023-07-14T11:53:21.568313+0000","last_peered":"2023-07-14T11:53:21.568313+0000","last_clean":"2023-07-14T11:53:21.568313+0000","last_became_active":"2023-07-14T11:53:15.606262+0000","last_became_peered":"2023-07-14T11:53:15.606262+0000","last_unstale":"2023-07-14T11:53:21.568313+0000","last_undegraded":"2023-07-14T11:53:21.568313+0000","last_fullsized":"2023-07-14T11:53:21.568313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566653+0000","last_change":"2023-07-14T11:53:14.588920+0000","last_active":"2023-07-14T11:53:21.566653+0000","last_peered":"2023-07-14T11:53:21.566653+0000","last_clean":"2023-07-14T11:53:21.566653+0000","last_became_active":"2023-07-14T11:53:14.588605+0000","last_became_peered":"2023-07-14T11:53:14.588605+0000","last_unstale":"2023-07-14T11:53:21.566653+0000","last_undegraded":"2023-07-14T11:53:21.566653+0000","last_fullsized":"2023-07-14T11:53:21.566653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:54.773189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567666+0000","last_change":"2023-07-14T11:53:09.146156+0000","last_active":"2023-07-14T11:53:21.567666+0000","last_peered":"2023-07-14T11:53:21.567666+0000","last_clean":"2023-07-14T11:53:21.567666+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:53:21.567666+0000","last_undegraded":"2023-07-14T11:53:21.567666+0000","last_fullsized":"2023-07-14T11:53:21.567666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:37.917368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569099+0000","last_change":"2023-07-14T11:53:19.680478+0000","last_active":"2023-07-14T11:53:21.569099+0000","last_peered":"2023-07-14T11:53:21.569099+0000","last_clean":"2023-07-14T11:53:21.569099+0000","last_became_active":"2023-07-14T11:53:19.680082+0000","last_became_peered":"2023-07-14T11:53:19.680082+0000","last_unstale":"2023-07-14T11:53:21.569099+0000","last_undegraded":"2023-07-14T11:53:21.569099+0000","last_fullsized":"2023-07-14T11:53:21.569099+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:42:53.267154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665451+0000","last_change":"2023-07-14T11:53:15.587902+0000","last_active":"2023-07-14T11:53:20.665451+0000","last_peered":"2023-07-14T11:53:20.665451+0000","last_clean":"2023-07-14T11:53:20.665451+0000","last_became_active":"2023-07-14T11:53:15.587578+0000","last_became_peered":"2023-07-14T11:53:15.587578+0000","last_unstale":"2023-07-14T11:53:20.665451+0000","last_undegraded":"2023-07-14T11:53:20.665451+0000","last_fullsized":"2023-07-14T11:53:20.665451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:43:21.367420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567625+0000","last_change":"2023-07-14T11:53:14.600123+0000","last_active":"2023-07-14T11:53:21.567625+0000","last_peered":"2023-07-14T11:53:21.567625+0000","last_clean":"2023-07-14T11:53:21.567625+0000","last_became_active":"2023-07-14T11:53:14.599830+0000","last_became_peered":"2023-07-14T11:53:14.599830+0000","last_unstale":"2023-07-14T11:53:21.567625+0000","last_undegraded":"2023-07-14T11:53:21.567625+0000","last_fullsized":"2023-07-14T11:53:21.567625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:54.937063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567091+0000","last_change":"2023-07-14T11:53:09.146556+0000","last_active":"2023-07-14T11:53:21.567091+0000","last_peered":"2023-07-14T11:53:21.567091+0000","last_clean":"2023-07-14T11:53:21.567091+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:53:21.567091+0000","last_undegraded":"2023-07-14T11:53:21.567091+0000","last_fullsized":"2023-07-14T11:53:21.567091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:07.460041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567928+0000","last_change":"2023-07-14T11:53:03.159818+0000","last_active":"2023-07-14T11:53:21.567928+0000","last_peered":"2023-07-14T11:53:21.567928+0000","last_clean":"2023-07-14T11:53:21.567928+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:53:21.567928+0000","last_undegraded":"2023-07-14T11:53:21.567928+0000","last_fullsized":"2023-07-14T11:53:21.567928+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_clean_scrub_stamp":"2023-07-14T11:53:02.054348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:17.151944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567578+0000","last_change":"2023-07-14T11:53:19.658486+0000","last_active":"2023-07-14T11:53:21.567578+0000","last_peered":"2023-07-14T11:53:21.567578+0000","last_clean":"2023-07-14T11:53:21.567578+0000","last_became_active":"2023-07-14T11:53:19.658083+0000","last_became_peered":"2023-07-14T11:53:19.658083+0000","last_unstale":"2023-07-14T11:53:21.567578+0000","last_undegraded":"2023-07-14T11:53:21.567578+0000","last_fullsized":"2023-07-14T11:53:21.567578+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:08:07.808470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878421+0000","last_change":"2023-07-14T11:53:15.596089+0000","last_active":"2023-07-14T11:53:20.878421+0000","last_peered":"2023-07-14T11:53:20.878421+0000","last_clean":"2023-07-14T11:53:20.878421+0000","last_became_active":"2023-07-14T11:53:15.594769+0000","last_became_peered":"2023-07-14T11:53:15.594769+0000","last_unstale":"2023-07-14T11:53:20.878421+0000","last_undegraded":"2023-07-14T11:53:20.878421+0000","last_fullsized":"2023-07-14T11:53:20.878421+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:55.943920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568475+0000","last_change":"2023-07-14T11:53:14.591999+0000","last_active":"2023-07-14T11:53:21.568475+0000","last_peered":"2023-07-14T11:53:21.568475+0000","last_clean":"2023-07-14T11:53:21.568475+0000","last_became_active":"2023-07-14T11:53:14.591754+0000","last_became_peered":"2023-07-14T11:53:14.591754+0000","last_unstale":"2023-07-14T11:53:21.568475+0000","last_undegraded":"2023-07-14T11:53:21.568475+0000","last_fullsized":"2023-07-14T11:53:21.568475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:39.643364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664358+0000","last_change":"2023-07-14T11:53:19.674798+0000","last_active":"2023-07-14T11:53:20.664358+0000","last_peered":"2023-07-14T11:53:20.664358+0000","last_clean":"2023-07-14T11:53:20.664358+0000","last_became_active":"2023-07-14T11:53:19.674362+0000","last_became_peered":"2023-07-14T11:53:19.674362+0000","last_unstale":"2023-07-14T11:53:20.664358+0000","last_undegraded":"2023-07-14T11:53:20.664358+0000","last_fullsized":"2023-07-14T11:53:20.664358+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:52:07.441376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664411+0000","last_change":"2023-07-14T11:53:15.582299+0000","last_active":"2023-07-14T11:53:20.664411+0000","last_peered":"2023-07-14T11:53:20.664411+0000","last_clean":"2023-07-14T11:53:20.664411+0000","last_became_active":"2023-07-14T11:53:15.582045+0000","last_became_peered":"2023-07-14T11:53:15.582045+0000","last_unstale":"2023-07-14T11:53:20.664411+0000","last_undegraded":"2023-07-14T11:53:20.664411+0000","last_fullsized":"2023-07-14T11:53:20.664411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:14:24.652057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663574+0000","last_change":"2023-07-14T11:53:14.574223+0000","last_active":"2023-07-14T11:53:20.663574+0000","last_peered":"2023-07-14T11:53:20.663574+0000","last_clean":"2023-07-14T11:53:20.663574+0000","last_became_active":"2023-07-14T11:53:14.573601+0000","last_became_peered":"2023-07-14T11:53:14.573601+0000","last_unstale":"2023-07-14T11:53:20.663574+0000","last_undegraded":"2023-07-14T11:53:20.663574+0000","last_fullsized":"2023-07-14T11:53:20.663574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:44:27.636525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568891+0000","last_change":"2023-07-14T11:53:19.671897+0000","last_active":"2023-07-14T11:53:21.568891+0000","last_peered":"2023-07-14T11:53:21.568891+0000","last_clean":"2023-07-14T11:53:21.568891+0000","last_became_active":"2023-07-14T11:53:19.671406+0000","last_became_peered":"2023-07-14T11:53:19.671406+0000","last_unstale":"2023-07-14T11:53:21.568891+0000","last_undegraded":"2023-07-14T11:53:21.568891+0000","last_fullsized":"2023-07-14T11:53:21.568891+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:37:01.963700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664358+0000","last_change":"2023-07-14T11:53:15.583755+0000","last_active":"2023-07-14T11:53:20.664358+0000","last_peered":"2023-07-14T11:53:20.664358+0000","last_clean":"2023-07-14T11:53:20.664358+0000","last_became_active":"2023-07-14T11:53:15.583491+0000","last_became_peered":"2023-07-14T11:53:15.583491+0000","last_unstale":"2023-07-14T11:53:20.664358+0000","last_undegraded":"2023-07-14T11:53:20.664358+0000","last_fullsized":"2023-07-14T11:53:20.664358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:30.105172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665591+0000","last_change":"2023-07-14T11:53:14.594378+0000","last_active":"2023-07-14T11:53:20.665591+0000","last_peered":"2023-07-14T11:53:20.665591+0000","last_clean":"2023-07-14T11:53:20.665591+0000","last_became_active":"2023-07-14T11:53:14.593863+0000","last_became_peered":"2023-07-14T11:53:14.593863+0000","last_unstale":"2023-07-14T11:53:20.665591+0000","last_undegraded":"2023-07-14T11:53:20.665591+0000","last_fullsized":"2023-07-14T11:53:20.665591+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:05:26.104433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878321+0000","last_change":"2023-07-14T11:53:19.674408+0000","last_active":"2023-07-14T11:53:20.878321+0000","last_peered":"2023-07-14T11:53:20.878321+0000","last_clean":"2023-07-14T11:53:20.878321+0000","last_became_active":"2023-07-14T11:53:19.673467+0000","last_became_peered":"2023-07-14T11:53:19.673467+0000","last_unstale":"2023-07-14T11:53:20.878321+0000","last_undegraded":"2023-07-14T11:53:20.878321+0000","last_fullsized":"2023-07-14T11:53:20.878321+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:08:07.808470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663737+0000","last_change":"2023-07-14T11:53:15.592271+0000","last_active":"2023-07-14T11:53:20.663737+0000","last_peered":"2023-07-14T11:53:20.663737+0000","last_clean":"2023-07-14T11:53:20.663737+0000","last_became_active":"2023-07-14T11:53:15.591906+0000","last_became_peered":"2023-07-14T11:53:15.591906+0000","last_unstale":"2023-07-14T11:53:20.663737+0000","last_undegraded":"2023-07-14T11:53:20.663737+0000","last_fullsized":"2023-07-14T11:53:20.663737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:17:30.232523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569314+0000","last_change":"2023-07-14T11:53:14.595162+0000","last_active":"2023-07-14T11:53:21.569314+0000","last_peered":"2023-07-14T11:53:21.569314+0000","last_clean":"2023-07-14T11:53:21.569314+0000","last_became_active":"2023-07-14T11:53:14.594754+0000","last_became_peered":"2023-07-14T11:53:14.594754+0000","last_unstale":"2023-07-14T11:53:21.569314+0000","last_undegraded":"2023-07-14T11:53:21.569314+0000","last_fullsized":"2023-07-14T11:53:21.569314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:19:28.596554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661153+0000","last_change":"2023-07-14T11:53:19.666034+0000","last_active":"2023-07-14T11:53:20.661153+0000","last_peered":"2023-07-14T11:53:20.661153+0000","last_clean":"2023-07-14T11:53:20.661153+0000","last_became_active":"2023-07-14T11:53:19.665570+0000","last_became_peered":"2023-07-14T11:53:19.665570+0000","last_unstale":"2023-07-14T11:53:20.661153+0000","last_undegraded":"2023-07-14T11:53:20.661153+0000","last_fullsized":"2023-07-14T11:53:20.661153+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:48:08.806526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567270+0000","last_change":"2023-07-14T11:53:15.612021+0000","last_active":"2023-07-14T11:53:21.567270+0000","last_peered":"2023-07-14T11:53:21.567270+0000","last_clean":"2023-07-14T11:53:21.567270+0000","last_became_active":"2023-07-14T11:53:15.611653+0000","last_became_peered":"2023-07-14T11:53:15.611653+0000","last_unstale":"2023-07-14T11:53:21.567270+0000","last_undegraded":"2023-07-14T11:53:21.567270+0000","last_fullsized":"2023-07-14T11:53:21.567270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:44:51.768501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566941+0000","last_change":"2023-07-14T11:53:14.590015+0000","last_active":"2023-07-14T11:53:21.566941+0000","last_peered":"2023-07-14T11:53:21.566941+0000","last_clean":"2023-07-14T11:53:21.566941+0000","last_became_active":"2023-07-14T11:53:14.589288+0000","last_became_peered":"2023-07-14T11:53:14.589288+0000","last_unstale":"2023-07-14T11:53:21.566941+0000","last_undegraded":"2023-07-14T11:53:21.566941+0000","last_fullsized":"2023-07-14T11:53:21.566941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:12.260236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660851+0000","last_change":"2023-07-14T11:53:19.664514+0000","last_active":"2023-07-14T11:53:20.660851+0000","last_peered":"2023-07-14T11:53:20.660851+0000","last_clean":"2023-07-14T11:53:20.660851+0000","last_became_active":"2023-07-14T11:53:19.663999+0000","last_became_peered":"2023-07-14T11:53:19.663999+0000","last_unstale":"2023-07-14T11:53:20.660851+0000","last_undegraded":"2023-07-14T11:53:20.660851+0000","last_fullsized":"2023-07-14T11:53:20.660851+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:23:47.861604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664958+0000","last_change":"2023-07-14T11:53:15.597946+0000","last_active":"2023-07-14T11:53:20.664958+0000","last_peered":"2023-07-14T11:53:20.664958+0000","last_clean":"2023-07-14T11:53:20.664958+0000","last_became_active":"2023-07-14T11:53:15.597589+0000","last_became_peered":"2023-07-14T11:53:15.597589+0000","last_unstale":"2023-07-14T11:53:20.664958+0000","last_undegraded":"2023-07-14T11:53:20.664958+0000","last_fullsized":"2023-07-14T11:53:20.664958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:51.380702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568455+0000","last_change":"2023-07-14T11:53:14.599636+0000","last_active":"2023-07-14T11:53:21.568455+0000","last_peered":"2023-07-14T11:53:21.568455+0000","last_clean":"2023-07-14T11:53:21.568455+0000","last_became_active":"2023-07-14T11:53:14.599183+0000","last_became_peered":"2023-07-14T11:53:14.599183+0000","last_unstale":"2023-07-14T11:53:21.568455+0000","last_undegraded":"2023-07-14T11:53:21.568455+0000","last_fullsized":"2023-07-14T11:53:21.568455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:30.529533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661123+0000","last_change":"2023-07-14T11:53:19.681580+0000","last_active":"2023-07-14T11:53:20.661123+0000","last_peered":"2023-07-14T11:53:20.661123+0000","last_clean":"2023-07-14T11:53:20.661123+0000","last_became_active":"2023-07-14T11:53:19.680891+0000","last_became_peered":"2023-07-14T11:53:19.680891+0000","last_unstale":"2023-07-14T11:53:20.661123+0000","last_undegraded":"2023-07-14T11:53:20.661123+0000","last_fullsized":"2023-07-14T11:53:20.661123+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:20:03.759130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663189+0000","last_change":"2023-07-14T11:53:15.610606+0000","last_active":"2023-07-14T11:53:20.663189+0000","last_peered":"2023-07-14T11:53:20.663189+0000","last_clean":"2023-07-14T11:53:20.663189+0000","last_became_active":"2023-07-14T11:53:15.610180+0000","last_became_peered":"2023-07-14T11:53:15.610180+0000","last_unstale":"2023-07-14T11:53:20.663189+0000","last_undegraded":"2023-07-14T11:53:20.663189+0000","last_fullsized":"2023-07-14T11:53:20.663189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:09.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568785+0000","last_change":"2023-07-14T11:53:14.591805+0000","last_active":"2023-07-14T11:53:21.568785+0000","last_peered":"2023-07-14T11:53:21.568785+0000","last_clean":"2023-07-14T11:53:21.568785+0000","last_became_active":"2023-07-14T11:53:14.591439+0000","last_became_peered":"2023-07-14T11:53:14.591439+0000","last_unstale":"2023-07-14T11:53:21.568785+0000","last_undegraded":"2023-07-14T11:53:21.568785+0000","last_fullsized":"2023-07-14T11:53:21.568785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:54:59.284344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660363+0000","last_change":"2023-07-14T11:53:19.675759+0000","last_active":"2023-07-14T11:53:20.660363+0000","last_peered":"2023-07-14T11:53:20.660363+0000","last_clean":"2023-07-14T11:53:20.660363+0000","last_became_active":"2023-07-14T11:53:19.675353+0000","last_became_peered":"2023-07-14T11:53:19.675353+0000","last_unstale":"2023-07-14T11:53:20.660363+0000","last_undegraded":"2023-07-14T11:53:20.660363+0000","last_fullsized":"2023-07-14T11:53:20.660363+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:35:00.376887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566299+0000","last_change":"2023-07-14T11:53:15.603352+0000","last_active":"2023-07-14T11:53:21.566299+0000","last_peered":"2023-07-14T11:53:21.566299+0000","last_clean":"2023-07-14T11:53:21.566299+0000","last_became_active":"2023-07-14T11:53:15.602777+0000","last_became_peered":"2023-07-14T11:53:15.602777+0000","last_unstale":"2023-07-14T11:53:21.566299+0000","last_undegraded":"2023-07-14T11:53:21.566299+0000","last_fullsized":"2023-07-14T11:53:21.566299+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568843+0000","last_change":"2023-07-14T11:53:14.590504+0000","last_active":"2023-07-14T11:53:21.568843+0000","last_peered":"2023-07-14T11:53:21.568843+0000","last_clean":"2023-07-14T11:53:21.568843+0000","last_became_active":"2023-07-14T11:53:14.589844+0000","last_became_peered":"2023-07-14T11:53:14.589844+0000","last_unstale":"2023-07-14T11:53:21.568843+0000","last_undegraded":"2023-07-14T11:53:21.568843+0000","last_fullsized":"2023-07-14T11:53:21.568843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:38.554107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567609+0000","last_change":"2023-07-14T11:53:19.668417+0000","last_active":"2023-07-14T11:53:21.567609+0000","last_peered":"2023-07-14T11:53:21.567609+0000","last_clean":"2023-07-14T11:53:21.567609+0000","last_became_active":"2023-07-14T11:53:19.667924+0000","last_became_peered":"2023-07-14T11:53:19.667924+0000","last_unstale":"2023-07-14T11:53:21.567609+0000","last_undegraded":"2023-07-14T11:53:21.567609+0000","last_fullsized":"2023-07-14T11:53:21.567609+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:00.015845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568675+0000","last_change":"2023-07-14T11:53:19.668219+0000","last_active":"2023-07-14T11:53:21.568675+0000","last_peered":"2023-07-14T11:53:21.568675+0000","last_clean":"2023-07-14T11:53:21.568675+0000","last_became_active":"2023-07-14T11:53:19.667862+0000","last_became_peered":"2023-07-14T11:53:19.667862+0000","last_unstale":"2023-07-14T11:53:21.568675+0000","last_undegraded":"2023-07-14T11:53:21.568675+0000","last_fullsized":"2023-07-14T11:53:21.568675+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:53:26.423183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665093+0000","last_change":"2023-07-14T11:53:14.588139+0000","last_active":"2023-07-14T11:53:20.665093+0000","last_peered":"2023-07-14T11:53:20.665093+0000","last_clean":"2023-07-14T11:53:20.665093+0000","last_became_active":"2023-07-14T11:53:14.587730+0000","last_became_peered":"2023-07-14T11:53:14.587730+0000","last_unstale":"2023-07-14T11:53:20.665093+0000","last_undegraded":"2023-07-14T11:53:20.665093+0000","last_fullsized":"2023-07-14T11:53:20.665093+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:02.335678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569769+0000","last_change":"2023-07-14T11:53:15.606880+0000","last_active":"2023-07-14T11:53:21.569769+0000","last_peered":"2023-07-14T11:53:21.569769+0000","last_clean":"2023-07-14T11:53:21.569769+0000","last_became_active":"2023-07-14T11:53:15.598546+0000","last_became_peered":"2023-07-14T11:53:15.598546+0000","last_unstale":"2023-07-14T11:53:21.569769+0000","last_undegraded":"2023-07-14T11:53:21.569769+0000","last_fullsized":"2023-07-14T11:53:21.569769+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:30:21.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569600+0000","last_change":"2023-07-14T11:53:19.676159+0000","last_active":"2023-07-14T11:53:21.569600+0000","last_peered":"2023-07-14T11:53:21.569600+0000","last_clean":"2023-07-14T11:53:21.569600+0000","last_became_active":"2023-07-14T11:53:19.675729+0000","last_became_peered":"2023-07-14T11:53:19.675729+0000","last_unstale":"2023-07-14T11:53:21.569600+0000","last_undegraded":"2023-07-14T11:53:21.569600+0000","last_fullsized":"2023-07-14T11:53:21.569600+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:49.754582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569051+0000","last_change":"2023-07-14T11:53:14.592455+0000","last_active":"2023-07-14T11:53:21.569051+0000","last_peered":"2023-07-14T11:53:21.569051+0000","last_clean":"2023-07-14T11:53:21.569051+0000","last_became_active":"2023-07-14T11:53:14.592040+0000","last_became_peered":"2023-07-14T11:53:14.592040+0000","last_unstale":"2023-07-14T11:53:21.569051+0000","last_undegraded":"2023-07-14T11:53:21.569051+0000","last_fullsized":"2023-07-14T11:53:21.569051+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:36:56.884917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568516+0000","last_change":"2023-07-14T11:53:15.593198+0000","last_active":"2023-07-14T11:53:21.568516+0000","last_peered":"2023-07-14T11:53:21.568516+0000","last_clean":"2023-07-14T11:53:21.568516+0000","last_became_active":"2023-07-14T11:53:15.592898+0000","last_became_peered":"2023-07-14T11:53:15.592898+0000","last_unstale":"2023-07-14T11:53:21.568516+0000","last_undegraded":"2023-07-14T11:53:21.568516+0000","last_fullsized":"2023-07-14T11:53:21.568516+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:13.267093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566870+0000","last_change":"2023-07-14T11:53:14.588962+0000","last_active":"2023-07-14T11:53:21.566870+0000","last_peered":"2023-07-14T11:53:21.566870+0000","last_clean":"2023-07-14T11:53:21.566870+0000","last_became_active":"2023-07-14T11:53:14.588674+0000","last_became_peered":"2023-07-14T11:53:14.588674+0000","last_unstale":"2023-07-14T11:53:21.566870+0000","last_undegraded":"2023-07-14T11:53:21.566870+0000","last_fullsized":"2023-07-14T11:53:21.566870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:32:38.194645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567389+0000","last_change":"2023-07-14T11:53:15.603164+0000","last_active":"2023-07-14T11:53:21.567389+0000","last_peered":"2023-07-14T11:53:21.567389+0000","last_clean":"2023-07-14T11:53:21.567389+0000","last_became_active":"2023-07-14T11:53:15.602752+0000","last_became_peered":"2023-07-14T11:53:15.602752+0000","last_unstale":"2023-07-14T11:53:21.567389+0000","last_undegraded":"2023-07-14T11:53:21.567389+0000","last_fullsized":"2023-07-14T11:53:21.567389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:40.650221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568701+0000","last_change":"2023-07-14T11:53:19.664284+0000","last_active":"2023-07-14T11:53:21.568701+0000","last_peered":"2023-07-14T11:53:21.568701+0000","last_clean":"2023-07-14T11:53:21.568701+0000","last_became_active":"2023-07-14T11:53:19.663903+0000","last_became_peered":"2023-07-14T11:53:19.663903+0000","last_unstale":"2023-07-14T11:53:21.568701+0000","last_undegraded":"2023-07-14T11:53:21.568701+0000","last_fullsized":"2023-07-14T11:53:21.568701+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:55:04.363127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568411+0000","last_change":"2023-07-14T11:53:14.591199+0000","last_active":"2023-07-14T11:53:21.568411+0000","last_peered":"2023-07-14T11:53:21.568411+0000","last_clean":"2023-07-14T11:53:21.568411+0000","last_became_active":"2023-07-14T11:53:14.590897+0000","last_became_peered":"2023-07-14T11:53:14.590897+0000","last_unstale":"2023-07-14T11:53:21.568411+0000","last_undegraded":"2023-07-14T11:53:21.568411+0000","last_fullsized":"2023-07-14T11:53:21.568411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:53:21.344401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568918+0000","last_change":"2023-07-14T11:53:15.594450+0000","last_active":"2023-07-14T11:53:21.568918+0000","last_peered":"2023-07-14T11:53:21.568918+0000","last_clean":"2023-07-14T11:53:21.568918+0000","last_became_active":"2023-07-14T11:53:15.594170+0000","last_became_peered":"2023-07-14T11:53:15.594170+0000","last_unstale":"2023-07-14T11:53:21.568918+0000","last_undegraded":"2023-07-14T11:53:21.568918+0000","last_fullsized":"2023-07-14T11:53:21.568918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:53:22.351258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567104+0000","last_change":"2023-07-14T11:53:19.682025+0000","last_active":"2023-07-14T11:53:21.567104+0000","last_peered":"2023-07-14T11:53:21.567104+0000","last_clean":"2023-07-14T11:53:21.567104+0000","last_became_active":"2023-07-14T11:53:19.681631+0000","last_became_peered":"2023-07-14T11:53:19.681631+0000","last_unstale":"2023-07-14T11:53:21.567104+0000","last_undegraded":"2023-07-14T11:53:21.567104+0000","last_fullsized":"2023-07-14T11:53:21.567104+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:35.608315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664672+0000","last_change":"2023-07-14T11:53:15.581069+0000","last_active":"2023-07-14T11:53:20.664672+0000","last_peered":"2023-07-14T11:53:20.664672+0000","last_clean":"2023-07-14T11:53:20.664672+0000","last_became_active":"2023-07-14T11:53:15.580609+0000","last_became_peered":"2023-07-14T11:53:15.580609+0000","last_unstale":"2023-07-14T11:53:20.664672+0000","last_undegraded":"2023-07-14T11:53:20.664672+0000","last_fullsized":"2023-07-14T11:53:20.664672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:11:11.201276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567881+0000","last_change":"2023-07-14T11:53:15.616860+0000","last_active":"2023-07-14T11:53:21.567881+0000","last_peered":"2023-07-14T11:53:21.567881+0000","last_clean":"2023-07-14T11:53:21.567881+0000","last_became_active":"2023-07-14T11:53:15.616511+0000","last_became_peered":"2023-07-14T11:53:15.616511+0000","last_unstale":"2023-07-14T11:53:21.567881+0000","last_undegraded":"2023-07-14T11:53:21.567881+0000","last_fullsized":"2023-07-14T11:53:21.567881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663897+0000","last_change":"2023-07-14T11:53:15.580935+0000","last_active":"2023-07-14T11:53:20.663897+0000","last_peered":"2023-07-14T11:53:20.663897+0000","last_clean":"2023-07-14T11:53:20.663897+0000","last_became_active":"2023-07-14T11:53:15.580624+0000","last_became_peered":"2023-07-14T11:53:15.580624+0000","last_unstale":"2023-07-14T11:53:20.663897+0000","last_undegraded":"2023-07-14T11:53:20.663897+0000","last_fullsized":"2023-07-14T11:53:20.663897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:54:27.366163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569054+0000","last_change":"2023-07-14T11:53:15.608124+0000","last_active":"2023-07-14T11:53:21.569054+0000","last_peered":"2023-07-14T11:53:21.569054+0000","last_clean":"2023-07-14T11:53:21.569054+0000","last_became_active":"2023-07-14T11:53:15.606540+0000","last_became_peered":"2023-07-14T11:53:15.606540+0000","last_unstale":"2023-07-14T11:53:21.569054+0000","last_undegraded":"2023-07-14T11:53:21.569054+0000","last_fullsized":"2023-07-14T11:53:21.569054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:36:57.891774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878952+0000","last_change":"2023-07-14T11:53:15.607644+0000","last_active":"2023-07-14T11:53:20.878952+0000","last_peered":"2023-07-14T11:53:20.878952+0000","last_clean":"2023-07-14T11:53:20.878952+0000","last_became_active":"2023-07-14T11:53:15.607318+0000","last_became_peered":"2023-07-14T11:53:15.607318+0000","last_unstale":"2023-07-14T11:53:20.878952+0000","last_undegraded":"2023-07-14T11:53:20.878952+0000","last_fullsized":"2023-07-14T11:53:20.878952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:31.536389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567297+0000","last_change":"2023-07-14T11:53:15.603168+0000","last_active":"2023-07-14T11:53:21.567297+0000","last_peered":"2023-07-14T11:53:21.567297+0000","last_clean":"2023-07-14T11:53:21.567297+0000","last_became_active":"2023-07-14T11:53:15.602791+0000","last_became_peered":"2023-07-14T11:53:15.602791+0000","last_unstale":"2023-07-14T11:53:21.567297+0000","last_undegraded":"2023-07-14T11:53:21.567297+0000","last_fullsized":"2023-07-14T11:53:21.567297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:18.763925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878622+0000","last_change":"2023-07-14T11:53:15.600466+0000","last_active":"2023-07-14T11:53:20.878622+0000","last_peered":"2023-07-14T11:53:20.878622+0000","last_clean":"2023-07-14T11:53:20.878622+0000","last_became_active":"2023-07-14T11:53:15.600232+0000","last_became_peered":"2023-07-14T11:53:15.600232+0000","last_unstale":"2023-07-14T11:53:20.878622+0000","last_undegraded":"2023-07-14T11:53:20.878622+0000","last_fullsized":"2023-07-14T11:53:20.878622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878439+0000","last_change":"2023-07-14T11:53:15.605971+0000","last_active":"2023-07-14T11:53:20.878439+0000","last_peered":"2023-07-14T11:53:20.878439+0000","last_clean":"2023-07-14T11:53:20.878439+0000","last_became_active":"2023-07-14T11:53:15.605661+0000","last_became_peered":"2023-07-14T11:53:15.605661+0000","last_unstale":"2023-07-14T11:53:20.878439+0000","last_undegraded":"2023-07-14T11:53:20.878439+0000","last_fullsized":"2023-07-14T11:53:20.878439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567617+0000","last_change":"2023-07-14T11:53:15.606414+0000","last_active":"2023-07-14T11:53:21.567617+0000","last_peered":"2023-07-14T11:53:21.567617+0000","last_clean":"2023-07-14T11:53:21.567617+0000","last_became_active":"2023-07-14T11:53:15.605906+0000","last_became_peered":"2023-07-14T11:53:15.605906+0000","last_unstale":"2023-07-14T11:53:21.567617+0000","last_undegraded":"2023-07-14T11:53:21.567617+0000","last_fullsized":"2023-07-14T11:53:21.567617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567728+0000","last_change":"2023-07-14T11:53:15.598006+0000","last_active":"2023-07-14T11:53:21.567728+0000","last_peered":"2023-07-14T11:53:21.567728+0000","last_clean":"2023-07-14T11:53:21.567728+0000","last_became_active":"2023-07-14T11:53:15.597749+0000","last_became_peered":"2023-07-14T11:53:15.597749+0000","last_unstale":"2023-07-14T11:53:21.567728+0000","last_undegraded":"2023-07-14T11:53:21.567728+0000","last_fullsized":"2023-07-14T11:53:21.567728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567052+0000","last_change":"2023-07-14T11:53:15.600730+0000","last_active":"2023-07-14T11:53:21.567052+0000","last_peered":"2023-07-14T11:53:21.567052+0000","last_clean":"2023-07-14T11:53:21.567052+0000","last_became_active":"2023-07-14T11:53:15.600299+0000","last_became_peered":"2023-07-14T11:53:15.600299+0000","last_unstale":"2023-07-14T11:53:21.567052+0000","last_undegraded":"2023-07-14T11:53:21.567052+0000","last_fullsized":"2023-07-14T11:53:21.567052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:18:17.533263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568245+0000","last_change":"2023-07-14T11:53:15.619732+0000","last_active":"2023-07-14T11:53:21.568245+0000","last_peered":"2023-07-14T11:53:21.568245+0000","last_clean":"2023-07-14T11:53:21.568245+0000","last_became_active":"2023-07-14T11:53:15.619425+0000","last_became_peered":"2023-07-14T11:53:15.619425+0000","last_unstale":"2023-07-14T11:53:21.568245+0000","last_undegraded":"2023-07-14T11:53:21.568245+0000","last_fullsized":"2023-07-14T11:53:21.568245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:13.888256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663540+0000","last_change":"2023-07-14T11:53:15.586169+0000","last_active":"2023-07-14T11:53:20.663540+0000","last_peered":"2023-07-14T11:53:20.663540+0000","last_clean":"2023-07-14T11:53:20.663540+0000","last_became_active":"2023-07-14T11:53:15.585876+0000","last_became_peered":"2023-07-14T11:53:15.585876+0000","last_unstale":"2023-07-14T11:53:20.663540+0000","last_undegraded":"2023-07-14T11:53:20.663540+0000","last_fullsized":"2023-07-14T11:53:20.663540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:47:20.678085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661229+0000","last_change":"2023-07-14T11:53:15.584035+0000","last_active":"2023-07-14T11:53:20.661229+0000","last_peered":"2023-07-14T11:53:20.661229+0000","last_clean":"2023-07-14T11:53:20.661229+0000","last_became_active":"2023-07-14T11:53:15.582303+0000","last_became_peered":"2023-07-14T11:53:15.582303+0000","last_unstale":"2023-07-14T11:53:20.661229+0000","last_undegraded":"2023-07-14T11:53:20.661229+0000","last_fullsized":"2023-07-14T11:53:20.661229+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:48:16.619167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664187+0000","last_change":"2023-07-14T11:53:15.578919+0000","last_active":"2023-07-14T11:53:20.664187+0000","last_peered":"2023-07-14T11:53:20.664187+0000","last_clean":"2023-07-14T11:53:20.664187+0000","last_became_active":"2023-07-14T11:53:15.578590+0000","last_became_peered":"2023-07-14T11:53:15.578590+0000","last_unstale":"2023-07-14T11:53:20.664187+0000","last_undegraded":"2023-07-14T11:53:20.664187+0000","last_fullsized":"2023-07-14T11:53:20.664187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:03:59.635384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568314+0000","last_change":"2023-07-14T11:53:15.613102+0000","last_active":"2023-07-14T11:53:21.568314+0000","last_peered":"2023-07-14T11:53:21.568314+0000","last_clean":"2023-07-14T11:53:21.568314+0000","last_became_active":"2023-07-14T11:53:15.612540+0000","last_became_peered":"2023-07-14T11:53:15.612540+0000","last_unstale":"2023-07-14T11:53:21.568314+0000","last_undegraded":"2023-07-14T11:53:21.568314+0000","last_fullsized":"2023-07-14T11:53:21.568314+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:29.641887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662627+0000","last_change":"2023-07-14T11:53:15.592089+0000","last_active":"2023-07-14T11:53:20.662627+0000","last_peered":"2023-07-14T11:53:20.662627+0000","last_clean":"2023-07-14T11:53:20.662627+0000","last_became_active":"2023-07-14T11:53:15.591826+0000","last_became_peered":"2023-07-14T11:53:15.591826+0000","last_unstale":"2023-07-14T11:53:20.662627+0000","last_undegraded":"2023-07-14T11:53:20.662627+0000","last_fullsized":"2023-07-14T11:53:20.662627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:06:57.233859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568086+0000","last_change":"2023-07-14T11:53:15.606636+0000","last_active":"2023-07-14T11:53:21.568086+0000","last_peered":"2023-07-14T11:53:21.568086+0000","last_clean":"2023-07-14T11:53:21.568086+0000","last_became_active":"2023-07-14T11:53:15.606067+0000","last_became_peered":"2023-07-14T11:53:15.606067+0000","last_unstale":"2023-07-14T11:53:21.568086+0000","last_undegraded":"2023-07-14T11:53:21.568086+0000","last_fullsized":"2023-07-14T11:53:21.568086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:13.267093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566709+0000","last_change":"2023-07-14T11:53:15.599253+0000","last_active":"2023-07-14T11:53:21.566709+0000","last_peered":"2023-07-14T11:53:21.566709+0000","last_clean":"2023-07-14T11:53:21.566709+0000","last_became_active":"2023-07-14T11:53:15.598955+0000","last_became_peered":"2023-07-14T11:53:15.598955+0000","last_unstale":"2023-07-14T11:53:21.566709+0000","last_undegraded":"2023-07-14T11:53:21.566709+0000","last_fullsized":"2023-07-14T11:53:21.566709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660486+0000","last_change":"2023-07-14T11:53:15.592119+0000","last_active":"2023-07-14T11:53:20.660486+0000","last_peered":"2023-07-14T11:53:20.660486+0000","last_clean":"2023-07-14T11:53:20.660486+0000","last_became_active":"2023-07-14T11:53:15.591822+0000","last_became_peered":"2023-07-14T11:53:15.591822+0000","last_unstale":"2023-07-14T11:53:20.660486+0000","last_undegraded":"2023-07-14T11:53:20.660486+0000","last_fullsized":"2023-07-14T11:53:20.660486+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:04:29.483383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661402+0000","last_change":"2023-07-14T11:53:15.590820+0000","last_active":"2023-07-14T11:53:20.661402+0000","last_peered":"2023-07-14T11:53:20.661402+0000","last_clean":"2023-07-14T11:53:20.661402+0000","last_became_active":"2023-07-14T11:53:15.590485+0000","last_became_peered":"2023-07-14T11:53:15.590485+0000","last_unstale":"2023-07-14T11:53:20.661402+0000","last_undegraded":"2023-07-14T11:53:20.661402+0000","last_fullsized":"2023-07-14T11:53:20.661402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:42.120586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664155+0000","last_change":"2023-07-14T11:53:15.589398+0000","last_active":"2023-07-14T11:53:20.664155+0000","last_peered":"2023-07-14T11:53:20.664155+0000","last_clean":"2023-07-14T11:53:20.664155+0000","last_became_active":"2023-07-14T11:53:15.589111+0000","last_became_peered":"2023-07-14T11:53:15.589111+0000","last_unstale":"2023-07-14T11:53:20.664155+0000","last_undegraded":"2023-07-14T11:53:20.664155+0000","last_fullsized":"2023-07-14T11:53:20.664155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:40:44.358262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569248+0000","last_change":"2023-07-14T11:53:15.599081+0000","last_active":"2023-07-14T11:53:21.569248+0000","last_peered":"2023-07-14T11:53:21.569248+0000","last_clean":"2023-07-14T11:53:21.569248+0000","last_became_active":"2023-07-14T11:53:15.598786+0000","last_became_peered":"2023-07-14T11:53:15.598786+0000","last_unstale":"2023-07-14T11:53:21.569248+0000","last_undegraded":"2023-07-14T11:53:21.569248+0000","last_fullsized":"2023-07-14T11:53:21.569248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:45.682657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567984+0000","last_change":"2023-07-14T11:53:15.613338+0000","last_active":"2023-07-14T11:53:21.567984+0000","last_peered":"2023-07-14T11:53:21.567984+0000","last_clean":"2023-07-14T11:53:21.567984+0000","last_became_active":"2023-07-14T11:53:15.612876+0000","last_became_peered":"2023-07-14T11:53:15.612876+0000","last_unstale":"2023-07-14T11:53:21.567984+0000","last_undegraded":"2023-07-14T11:53:21.567984+0000","last_fullsized":"2023-07-14T11:53:21.567984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566164+0000","last_change":"2023-07-14T11:53:15.608445+0000","last_active":"2023-07-14T11:53:21.566164+0000","last_peered":"2023-07-14T11:53:21.566164+0000","last_clean":"2023-07-14T11:53:21.566164+0000","last_became_active":"2023-07-14T11:53:15.608146+0000","last_became_peered":"2023-07-14T11:53:15.608146+0000","last_unstale":"2023-07-14T11:53:21.566164+0000","last_undegraded":"2023-07-14T11:53:21.566164+0000","last_fullsized":"2023-07-14T11:53:21.566164+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:55.943920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567855+0000","last_change":"2023-07-14T11:53:15.606963+0000","last_active":"2023-07-14T11:53:21.567855+0000","last_peered":"2023-07-14T11:53:21.567855+0000","last_clean":"2023-07-14T11:53:21.567855+0000","last_became_active":"2023-07-14T11:53:15.606397+0000","last_became_peered":"2023-07-14T11:53:15.606397+0000","last_unstale":"2023-07-14T11:53:21.567855+0000","last_undegraded":"2023-07-14T11:53:21.567855+0000","last_fullsized":"2023-07-14T11:53:21.567855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663074+0000","last_change":"2023-07-14T11:53:15.607539+0000","last_active":"2023-07-14T11:53:20.663074+0000","last_peered":"2023-07-14T11:53:20.663074+0000","last_clean":"2023-07-14T11:53:20.663074+0000","last_became_active":"2023-07-14T11:53:15.593279+0000","last_became_peered":"2023-07-14T11:53:15.593279+0000","last_unstale":"2023-07-14T11:53:20.663074+0000","last_undegraded":"2023-07-14T11:53:20.663074+0000","last_fullsized":"2023-07-14T11:53:20.663074+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:34.163470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569328+0000","last_change":"2023-07-14T11:53:15.592332+0000","last_active":"2023-07-14T11:53:21.569328+0000","last_peered":"2023-07-14T11:53:21.569328+0000","last_clean":"2023-07-14T11:53:21.569328+0000","last_became_active":"2023-07-14T11:53:15.591983+0000","last_became_peered":"2023-07-14T11:53:15.591983+0000","last_unstale":"2023-07-14T11:53:21.569328+0000","last_undegraded":"2023-07-14T11:53:21.569328+0000","last_fullsized":"2023-07-14T11:53:21.569328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:18.763925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665081+0000","last_change":"2023-07-14T11:53:15.594122+0000","last_active":"2023-07-14T11:53:20.665081+0000","last_peered":"2023-07-14T11:53:20.665081+0000","last_clean":"2023-07-14T11:53:20.665081+0000","last_became_active":"2023-07-14T11:53:15.593773+0000","last_became_peered":"2023-07-14T11:53:15.593773+0000","last_unstale":"2023-07-14T11:53:20.665081+0000","last_undegraded":"2023-07-14T11:53:20.665081+0000","last_fullsized":"2023-07-14T11:53:20.665081+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:59:21.935826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664107+0000","last_change":"2023-07-14T11:53:15.593314+0000","last_active":"2023-07-14T11:53:20.664107+0000","last_peered":"2023-07-14T11:53:20.664107+0000","last_clean":"2023-07-14T11:53:20.664107+0000","last_became_active":"2023-07-14T11:53:15.592778+0000","last_became_peered":"2023-07-14T11:53:15.592778+0000","last_unstale":"2023-07-14T11:53:20.664107+0000","last_undegraded":"2023-07-14T11:53:20.664107+0000","last_fullsized":"2023-07-14T11:53:20.664107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:37.263541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568937+0000","last_change":"2023-07-14T11:53:15.599579+0000","last_active":"2023-07-14T11:53:21.568937+0000","last_peered":"2023-07-14T11:53:21.568937+0000","last_clean":"2023-07-14T11:53:21.568937+0000","last_became_active":"2023-07-14T11:53:15.599380+0000","last_became_peered":"2023-07-14T11:53:15.599380+0000","last_unstale":"2023-07-14T11:53:21.568937+0000","last_undegraded":"2023-07-14T11:53:21.568937+0000","last_fullsized":"2023-07-14T11:53:21.568937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:19:29.603411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661719+0000","last_change":"2023-07-14T11:53:15.594232+0000","last_active":"2023-07-14T11:53:20.661719+0000","last_peered":"2023-07-14T11:53:20.661719+0000","last_clean":"2023-07-14T11:53:20.661719+0000","last_became_active":"2023-07-14T11:53:15.594049+0000","last_became_peered":"2023-07-14T11:53:15.594049+0000","last_unstale":"2023-07-14T11:53:20.661719+0000","last_undegraded":"2023-07-14T11:53:20.661719+0000","last_fullsized":"2023-07-14T11:53:20.661719+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:51.380702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4591,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":925696,"data_stored":718949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6220000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":311296,"data_stored":125224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4750000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":323584,"data_stored":128421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.609}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3580000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4670000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":901120,"data_stored":715522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:53:43.058 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:53:43.273 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.288+0000 7f41b5693700 1 Processor -- start 2023-07-14T11:53:43.273 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.288+0000 7f41b5693700 1 -- start start 2023-07-14T11:53:43.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41b5693700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41b015bff0 0x7f41b00ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:43.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41b5693700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f41b00cf1c0 0x7f41b00afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:43.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41b5693700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b00cd4c0 con 0x7f41b00cd070 2023-07-14T11:53:43.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41b5693700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b00af0a0 con 0x7f41b00cf1c0 2023-07-14T11:53:43.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41b5693700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b00cf720 con 0x7f41b015bff0 2023-07-14T11:53:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41b4e92700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f41b00cf1c0 0x7f41b00afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41b4e92700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f41b00cf1c0 0x7f41b00afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38088/0 (socket says 172.21.15.104:38088) 2023-07-14T11:53:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41b4e92700 1 -- 172.21.15.104:0/1984991679 learned_addr learned my addr 172.21.15.104:0/1984991679 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41affff700 1 --2- 172.21.15.104:0/1984991679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41b015bff0 0x7f41b00ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41b4e92700 1 -- 172.21.15.104:0/1984991679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41b015bff0 msgr2=0x7f41b00ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41af7fe700 1 -- 172.21.15.104:0/1984991679 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486253391 0 0) 0x7f4198001110 con 0x7f41b00cd070 2023-07-14T11:53:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41b4e92700 1 --2- 172.21.15.104:0/1984991679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41b015bff0 0x7f41b00ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41b4e92700 1 -- 172.21.15.104:0/1984991679 >> v1:172.21.15.121:6789/0 conn(0x7f41b00cd070 legacy=0x7f41b00ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41b4e92700 1 -- 172.21.15.104:0/1984991679 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b00af2f0 con 0x7f41b00cf1c0 2023-07-14T11:53:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41b4e92700 1 --2- 172.21.15.104:0/1984991679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41b00cf1c0 0x7f41b00afdd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009e50 tx=0x7f41a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b062b697fffb520e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41af7fe700 1 -- 172.21.15.104:0/1984991679 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a800e070 con 0x7f41b00cf1c0 2023-07-14T11:53:43.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41af7fe700 1 -- 172.21.15.104:0/1984991679 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a800b910 con 0x7f41b00cf1c0 2023-07-14T11:53:43.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.292+0000 7f41af7fe700 1 -- 172.21.15.104:0/1984991679 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a800bcc0 con 0x7f41b00cf1c0 2023-07-14T11:53:43.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.296+0000 7f41b5693700 1 -- 172.21.15.104:0/1984991679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41b00cf1c0 msgr2=0x7f41b00afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.296+0000 7f41b5693700 1 --2- 172.21.15.104:0/1984991679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41b00cf1c0 0x7f41b00afdd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009e50 tx=0x7f41a800b330 comp rx=0 tx=0).stop 2023-07-14T11:53:43.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.296+0000 7f41b5693700 1 -- 172.21.15.104:0/1984991679 shutdown_connections 2023-07-14T11:53:43.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.296+0000 7f41b5693700 1 --2- 172.21.15.104:0/1984991679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41b00cf1c0 0x7f41b00afdd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.296+0000 7f41b5693700 1 --2- 172.21.15.104:0/1984991679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41b015bff0 0x7f41b00ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.296+0000 7f41b5693700 1 -- 172.21.15.104:0/1984991679 >> 172.21.15.104:0/1984991679 conn(0x7f41b0060c40 msgr2=0x7f41b009b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.296+0000 7f41b5693700 1 -- 172.21.15.104:0/1984991679 shutdown_connections 2023-07-14T11:53:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.296+0000 7f41b5693700 1 -- 172.21.15.104:0/1984991679 wait complete. 2023-07-14T11:53:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.296+0000 7f41b5693700 1 Processor -- start 2023-07-14T11:53:43.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.296+0000 7f41b5693700 1 -- start start 2023-07-14T11:53:43.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41b5693700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41b00cd070 0x7f41b01f0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:43.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41b5693700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41b00cf1c0 0x7f41b00b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:43.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41b5693700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41b015bff0 0x7f41b00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:43.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41b5693700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b01f5300 con 0x7f41b00cf1c0 2023-07-14T11:53:43.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41b5693700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b01f4960 con 0x7f41b00cd070 2023-07-14T11:53:43.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41b5693700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b01f4c40 con 0x7f41b015bff0 2023-07-14T11:53:43.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41a7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41b00cf1c0 0x7f41b00b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:43.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41a7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41b00cf1c0 0x7f41b00b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38100/0 (socket says 172.21.15.104:38100) 2023-07-14T11:53:43.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41a7fff700 1 -- 172.21.15.104:0/529427276 learned_addr learned my addr 172.21.15.104:0/529427276 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:43.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41affff700 1 --2- 172.21.15.104:0/529427276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41b00cd070 0x7f41b01f0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:43.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41b4e92700 1 --2- 172.21.15.104:0/529427276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41b015bff0 0x7f41b00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:43.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41a7fff700 1 -- 172.21.15.104:0/529427276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41b015bff0 msgr2=0x7f41b00710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41a7fff700 1 --2- 172.21.15.104:0/529427276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41b015bff0 0x7f41b00710c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41a7fff700 1 -- 172.21.15.104:0/529427276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41b00cd070 msgr2=0x7f41b01f0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41a7fff700 1 --2- 172.21.15.104:0/529427276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41b00cd070 0x7f41b01f0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41a7fff700 1 -- 172.21.15.104:0/529427276 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a8009b30 con 0x7f41b00cf1c0 2023-07-14T11:53:43.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41a7fff700 1 --2- 172.21.15.104:0/529427276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41b00cf1c0 0x7f41b00b18b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f419800c520 tx=0x7f419800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:43.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41ad7fa700 1 -- 172.21.15.104:0/529427276 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41980054e0 con 0x7f41b00cf1c0 2023-07-14T11:53:43.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41ad7fa700 1 -- 172.21.15.104:0/529427276 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41980036a0 con 0x7f41b00cf1c0 2023-07-14T11:53:43.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41b5693700 1 -- 172.21.15.104:0/529427276 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b01f4f20 con 0x7f41b00cf1c0 2023-07-14T11:53:43.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41ad7fa700 1 -- 172.21.15.104:0/529427276 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198003a50 con 0x7f41b00cf1c0 2023-07-14T11:53:43.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.300+0000 7f41b5693700 1 -- 172.21.15.104:0/529427276 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b015d620 con 0x7f41b00cf1c0 2023-07-14T11:53:43.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.304+0000 7f41b5693700 1 -- 172.21.15.104:0/529427276 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b01f5440 con 0x7f41b00cf1c0 2023-07-14T11:53:43.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.304+0000 7f41ad7fa700 1 -- 172.21.15.104:0/529427276 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f4198003bb0 con 0x7f41b00cf1c0 2023-07-14T11:53:43.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.304+0000 7f41ad7fa700 1 --2- 172.21.15.104:0/529427276 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f41940662f0 0x7f41940687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:43.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.304+0000 7f41ad7fa700 1 -- 172.21.15.104:0/529427276 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f41980833e0 con 0x7f41b00cf1c0 2023-07-14T11:53:43.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.304+0000 7f41affff700 1 --2- 172.21.15.104:0/529427276 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f41940662f0 0x7f41940687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:43.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.308+0000 7f41affff700 1 --2- 172.21.15.104:0/529427276 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f41940662f0 0x7f41940687a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009ae0 tx=0x7f41a0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:43.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.312+0000 7f41ad7fa700 1 -- 172.21.15.104:0/529427276 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f41b01f5440 con 0x7f41b00cf1c0 2023-07-14T11:53:43.466 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.480+0000 7f41b5693700 1 -- 172.21.15.104:0/529427276 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41b00af8b0 con 0x7f41940662f0 2023-07-14T11:53:43.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.496+0000 7f41ad7fa700 1 -- 172.21.15.104:0/529427276 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367160 (secure 0 0 0) 0x7f41a0011750 con 0x7f41940662f0 2023-07-14T11:53:43.485 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:43.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.508+0000 7f41b5693700 1 -- 172.21.15.104:0/529427276 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f41940662f0 msgr2=0x7f41940687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.508+0000 7f41b5693700 1 --2- 172.21.15.104:0/529427276 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f41940662f0 0x7f41940687a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009ae0 tx=0x7f41a0008040 comp rx=0 tx=0).stop 2023-07-14T11:53:43.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.508+0000 7f41b5693700 1 -- 172.21.15.104:0/529427276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41b00cf1c0 msgr2=0x7f41b00b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.508+0000 7f41b5693700 1 --2- 172.21.15.104:0/529427276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41b00cf1c0 0x7f41b00b18b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f419800c520 tx=0x7f419800c8e0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.508+0000 7f41b5693700 1 -- 172.21.15.104:0/529427276 shutdown_connections 2023-07-14T11:53:43.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.508+0000 7f41b5693700 1 --2- 172.21.15.104:0/529427276 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f41940662f0 0x7f41940687a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.508+0000 7f41b5693700 1 --2- 172.21.15.104:0/529427276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41b00cf1c0 0x7f41b00b18b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.508+0000 7f41b5693700 1 --2- 172.21.15.104:0/529427276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41b015bff0 0x7f41b00710c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.508+0000 7f41b5693700 1 --2- 172.21.15.104:0/529427276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41b00cd070 0x7f41b01f0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.508+0000 7f41b5693700 1 -- 172.21.15.104:0/529427276 >> 172.21.15.104:0/529427276 conn(0x7f41b0060c40 msgr2=0x7f41b00d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:43.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.508+0000 7f41b5693700 1 -- 172.21.15.104:0/529427276 shutdown_connections 2023-07-14T11:53:43.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.508+0000 7f41b5693700 1 -- 172.21.15.104:0/529427276 wait complete. 2023-07-14T11:53:43.495 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:53:43.514 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2023-07-14T11:53:41.894235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594959,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219208,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755512,"statfs":{"total":773094113280,"available":772869644288,"internally_reserved":0,"allocated":3678208,"data_stored":2186494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004542"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877499+0000","last_change":"2023-07-14T11:53:19.666668+0000","last_active":"2023-07-14T11:53:20.877499+0000","last_peered":"2023-07-14T11:53:20.877499+0000","last_clean":"2023-07-14T11:53:20.877499+0000","last_became_active":"2023-07-14T11:53:19.665905+0000","last_became_peered":"2023-07-14T11:53:19.665905+0000","last_unstale":"2023-07-14T11:53:20.877499+0000","last_undegraded":"2023-07-14T11:53:20.877499+0000","last_fullsized":"2023-07-14T11:53:20.877499+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:59.851971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568528+0000","last_change":"2023-07-14T11:53:14.600663+0000","last_active":"2023-07-14T11:53:21.568528+0000","last_peered":"2023-07-14T11:53:21.568528+0000","last_clean":"2023-07-14T11:53:21.568528+0000","last_became_active":"2023-07-14T11:53:14.600396+0000","last_became_peered":"2023-07-14T11:53:14.600396+0000","last_unstale":"2023-07-14T11:53:21.568528+0000","last_undegraded":"2023-07-14T11:53:21.568528+0000","last_fullsized":"2023-07-14T11:53:21.568528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:38.720680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662714+0000","last_change":"2023-07-14T11:53:15.580781+0000","last_active":"2023-07-14T11:53:20.662714+0000","last_peered":"2023-07-14T11:53:20.662714+0000","last_clean":"2023-07-14T11:53:20.662714+0000","last_became_active":"2023-07-14T11:53:15.580452+0000","last_became_peered":"2023-07-14T11:53:15.580452+0000","last_unstale":"2023-07-14T11:53:20.662714+0000","last_undegraded":"2023-07-14T11:53:20.662714+0000","last_fullsized":"2023-07-14T11:53:20.662714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:20:09.506259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662575+0000","last_change":"2023-07-14T11:53:19.666546+0000","last_active":"2023-07-14T11:53:20.662575+0000","last_peered":"2023-07-14T11:53:20.662575+0000","last_clean":"2023-07-14T11:53:20.662575+0000","last_became_active":"2023-07-14T11:53:19.666168+0000","last_became_peered":"2023-07-14T11:53:19.666168+0000","last_unstale":"2023-07-14T11:53:20.662575+0000","last_undegraded":"2023-07-14T11:53:20.662575+0000","last_fullsized":"2023-07-14T11:53:20.662575+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:14.566966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569423+0000","last_change":"2023-07-14T11:53:14.586141+0000","last_active":"2023-07-14T11:53:21.569423+0000","last_peered":"2023-07-14T11:53:21.569423+0000","last_clean":"2023-07-14T11:53:21.569423+0000","last_became_active":"2023-07-14T11:53:14.585836+0000","last_became_peered":"2023-07-14T11:53:14.585836+0000","last_unstale":"2023-07-14T11:53:21.569423+0000","last_undegraded":"2023-07-14T11:53:21.569423+0000","last_fullsized":"2023-07-14T11:53:21.569423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:42:48.188371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568268+0000","last_change":"2023-07-14T11:53:15.582115+0000","last_active":"2023-07-14T11:53:21.568268+0000","last_peered":"2023-07-14T11:53:21.568268+0000","last_clean":"2023-07-14T11:53:21.568268+0000","last_became_active":"2023-07-14T11:53:15.581864+0000","last_became_peered":"2023-07-14T11:53:15.581864+0000","last_unstale":"2023-07-14T11:53:21.568268+0000","last_undegraded":"2023-07-14T11:53:21.568268+0000","last_fullsized":"2023-07-14T11:53:21.568268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:31.536389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566924+0000","last_change":"2023-07-14T11:53:19.680406+0000","last_active":"2023-07-14T11:53:21.566924+0000","last_peered":"2023-07-14T11:53:21.566924+0000","last_clean":"2023-07-14T11:53:21.566924+0000","last_became_active":"2023-07-14T11:53:19.680059+0000","last_became_peered":"2023-07-14T11:53:19.680059+0000","last_unstale":"2023-07-14T11:53:21.566924+0000","last_undegraded":"2023-07-14T11:53:21.566924+0000","last_fullsized":"2023-07-14T11:53:21.566924+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:43.799463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567583+0000","last_change":"2023-07-14T11:53:14.596827+0000","last_active":"2023-07-14T11:53:21.567583+0000","last_peered":"2023-07-14T11:53:21.567583+0000","last_clean":"2023-07-14T11:53:21.567583+0000","last_became_active":"2023-07-14T11:53:14.596221+0000","last_became_peered":"2023-07-14T11:53:14.596221+0000","last_unstale":"2023-07-14T11:53:21.567583+0000","last_undegraded":"2023-07-14T11:53:21.567583+0000","last_fullsized":"2023-07-14T11:53:21.567583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:54:59.284344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663572+0000","last_change":"2023-07-14T11:53:15.607538+0000","last_active":"2023-07-14T11:53:20.663572+0000","last_peered":"2023-07-14T11:53:20.663572+0000","last_clean":"2023-07-14T11:53:20.663572+0000","last_became_active":"2023-07-14T11:53:15.592890+0000","last_became_peered":"2023-07-14T11:53:15.592890+0000","last_unstale":"2023-07-14T11:53:20.663572+0000","last_undegraded":"2023-07-14T11:53:20.663572+0000","last_fullsized":"2023-07-14T11:53:20.663572+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:30:22.821980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568105+0000","last_change":"2023-07-14T11:53:19.662893+0000","last_active":"2023-07-14T11:53:21.568105+0000","last_peered":"2023-07-14T11:53:21.568105+0000","last_clean":"2023-07-14T11:53:21.568105+0000","last_became_active":"2023-07-14T11:53:19.662484+0000","last_became_peered":"2023-07-14T11:53:19.662484+0000","last_unstale":"2023-07-14T11:53:21.568105+0000","last_undegraded":"2023-07-14T11:53:21.568105+0000","last_fullsized":"2023-07-14T11:53:21.568105+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:43.799463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566672+0000","last_change":"2023-07-14T11:53:14.590222+0000","last_active":"2023-07-14T11:53:21.566672+0000","last_peered":"2023-07-14T11:53:21.566672+0000","last_clean":"2023-07-14T11:53:21.566672+0000","last_became_active":"2023-07-14T11:53:14.589684+0000","last_became_peered":"2023-07-14T11:53:14.589684+0000","last_unstale":"2023-07-14T11:53:21.566672+0000","last_undegraded":"2023-07-14T11:53:21.566672+0000","last_fullsized":"2023-07-14T11:53:21.566672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:43.338727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663068+0000","last_change":"2023-07-14T11:53:15.592933+0000","last_active":"2023-07-14T11:53:20.663068+0000","last_peered":"2023-07-14T11:53:20.663068+0000","last_clean":"2023-07-14T11:53:20.663068+0000","last_became_active":"2023-07-14T11:53:15.592619+0000","last_became_peered":"2023-07-14T11:53:15.592619+0000","last_unstale":"2023-07-14T11:53:20.663068+0000","last_undegraded":"2023-07-14T11:53:20.663068+0000","last_fullsized":"2023-07-14T11:53:20.663068+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:41:38.469614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878123+0000","last_change":"2023-07-14T11:53:19.683165+0000","last_active":"2023-07-14T11:53:20.878123+0000","last_peered":"2023-07-14T11:53:20.878123+0000","last_clean":"2023-07-14T11:53:20.878123+0000","last_became_active":"2023-07-14T11:53:19.682676+0000","last_became_peered":"2023-07-14T11:53:19.682676+0000","last_unstale":"2023-07-14T11:53:20.878123+0000","last_undegraded":"2023-07-14T11:53:20.878123+0000","last_fullsized":"2023-07-14T11:53:20.878123+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:36:19.425078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569431+0000","last_change":"2023-07-14T11:53:14.592834+0000","last_active":"2023-07-14T11:53:21.569431+0000","last_peered":"2023-07-14T11:53:21.569431+0000","last_clean":"2023-07-14T11:53:21.569431+0000","last_became_active":"2023-07-14T11:53:14.592535+0000","last_became_peered":"2023-07-14T11:53:14.592535+0000","last_unstale":"2023-07-14T11:53:21.569431+0000","last_undegraded":"2023-07-14T11:53:21.569431+0000","last_fullsized":"2023-07-14T11:53:21.569431+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:39.643364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661401+0000","last_change":"2023-07-14T11:53:15.593173+0000","last_active":"2023-07-14T11:53:20.661401+0000","last_peered":"2023-07-14T11:53:20.661401+0000","last_clean":"2023-07-14T11:53:20.661401+0000","last_became_active":"2023-07-14T11:53:15.592873+0000","last_became_peered":"2023-07-14T11:53:15.592873+0000","last_unstale":"2023-07-14T11:53:20.661401+0000","last_undegraded":"2023-07-14T11:53:20.661401+0000","last_fullsized":"2023-07-14T11:53:20.661401+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:59:03.301440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663003+0000","last_change":"2023-07-14T11:53:19.672048+0000","last_active":"2023-07-14T11:53:20.663003+0000","last_peered":"2023-07-14T11:53:20.663003+0000","last_clean":"2023-07-14T11:53:20.663003+0000","last_became_active":"2023-07-14T11:53:19.671493+0000","last_became_peered":"2023-07-14T11:53:19.671493+0000","last_unstale":"2023-07-14T11:53:20.663003+0000","last_undegraded":"2023-07-14T11:53:20.663003+0000","last_fullsized":"2023-07-14T11:53:20.663003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:34:10.732809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664092+0000","last_change":"2023-07-14T11:53:14.591651+0000","last_active":"2023-07-14T11:53:20.664092+0000","last_peered":"2023-07-14T11:53:20.664092+0000","last_clean":"2023-07-14T11:53:20.664092+0000","last_became_active":"2023-07-14T11:53:14.591356+0000","last_became_peered":"2023-07-14T11:53:14.591356+0000","last_unstale":"2023-07-14T11:53:20.664092+0000","last_undegraded":"2023-07-14T11:53:20.664092+0000","last_fullsized":"2023-07-14T11:53:20.664092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:38:54.408913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568174+0000","last_change":"2023-07-14T11:53:15.596998+0000","last_active":"2023-07-14T11:53:21.568174+0000","last_peered":"2023-07-14T11:53:21.568174+0000","last_clean":"2023-07-14T11:53:21.568174+0000","last_became_active":"2023-07-14T11:53:15.596278+0000","last_became_peered":"2023-07-14T11:53:15.596278+0000","last_unstale":"2023-07-14T11:53:21.568174+0000","last_undegraded":"2023-07-14T11:53:21.568174+0000","last_fullsized":"2023-07-14T11:53:21.568174+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:39.727537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665493+0000","last_change":"2023-07-14T11:53:19.664365+0000","last_active":"2023-07-14T11:53:20.665493+0000","last_peered":"2023-07-14T11:53:20.665493+0000","last_clean":"2023-07-14T11:53:20.665493+0000","last_became_active":"2023-07-14T11:53:19.663967+0000","last_became_peered":"2023-07-14T11:53:19.663967+0000","last_unstale":"2023-07-14T11:53:20.665493+0000","last_undegraded":"2023-07-14T11:53:20.665493+0000","last_fullsized":"2023-07-14T11:53:20.665493+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:37:34.816983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877811+0000","last_change":"2023-07-14T11:53:14.590756+0000","last_active":"2023-07-14T11:53:20.877811+0000","last_peered":"2023-07-14T11:53:20.877811+0000","last_clean":"2023-07-14T11:53:20.877811+0000","last_became_active":"2023-07-14T11:53:14.590511+0000","last_became_peered":"2023-07-14T11:53:14.590511+0000","last_unstale":"2023-07-14T11:53:20.877811+0000","last_undegraded":"2023-07-14T11:53:20.877811+0000","last_fullsized":"2023-07-14T11:53:20.877811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:18:16.526406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568118+0000","last_change":"2023-07-14T11:53:15.612239+0000","last_active":"2023-07-14T11:53:21.568118+0000","last_peered":"2023-07-14T11:53:21.568118+0000","last_clean":"2023-07-14T11:53:21.568118+0000","last_became_active":"2023-07-14T11:53:15.611948+0000","last_became_peered":"2023-07-14T11:53:15.611948+0000","last_unstale":"2023-07-14T11:53:21.568118+0000","last_undegraded":"2023-07-14T11:53:21.568118+0000","last_fullsized":"2023-07-14T11:53:21.568118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:44.345584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567809+0000","last_change":"2023-07-14T11:53:19.680386+0000","last_active":"2023-07-14T11:53:21.567809+0000","last_peered":"2023-07-14T11:53:21.567809+0000","last_clean":"2023-07-14T11:53:21.567809+0000","last_became_active":"2023-07-14T11:53:19.674870+0000","last_became_peered":"2023-07-14T11:53:19.674870+0000","last_unstale":"2023-07-14T11:53:21.567809+0000","last_undegraded":"2023-07-14T11:53:21.567809+0000","last_fullsized":"2023-07-14T11:53:21.567809+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:00.015845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568390+0000","last_change":"2023-07-14T11:53:14.592538+0000","last_active":"2023-07-14T11:53:21.568390+0000","last_peered":"2023-07-14T11:53:21.568390+0000","last_clean":"2023-07-14T11:53:21.568390+0000","last_became_active":"2023-07-14T11:53:14.592269+0000","last_became_peered":"2023-07-14T11:53:14.592269+0000","last_unstale":"2023-07-14T11:53:21.568390+0000","last_undegraded":"2023-07-14T11:53:21.568390+0000","last_fullsized":"2023-07-14T11:53:21.568390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:17.757069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665555+0000","last_change":"2023-07-14T11:53:15.587863+0000","last_active":"2023-07-14T11:53:20.665555+0000","last_peered":"2023-07-14T11:53:20.665555+0000","last_clean":"2023-07-14T11:53:20.665555+0000","last_became_active":"2023-07-14T11:53:15.587560+0000","last_became_peered":"2023-07-14T11:53:15.587560+0000","last_unstale":"2023-07-14T11:53:20.665555+0000","last_undegraded":"2023-07-14T11:53:20.665555+0000","last_fullsized":"2023-07-14T11:53:20.665555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:01:34.117398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568069+0000","last_change":"2023-07-14T11:53:15.596816+0000","last_active":"2023-07-14T11:53:21.568069+0000","last_peered":"2023-07-14T11:53:21.568069+0000","last_clean":"2023-07-14T11:53:21.568069+0000","last_became_active":"2023-07-14T11:53:15.596544+0000","last_became_peered":"2023-07-14T11:53:15.596544+0000","last_unstale":"2023-07-14T11:53:21.568069+0000","last_undegraded":"2023-07-14T11:53:21.568069+0000","last_fullsized":"2023-07-14T11:53:21.568069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:13.267093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568721+0000","last_change":"2023-07-14T11:53:14.600709+0000","last_active":"2023-07-14T11:53:21.568721+0000","last_peered":"2023-07-14T11:53:21.568721+0000","last_clean":"2023-07-14T11:53:21.568721+0000","last_became_active":"2023-07-14T11:53:14.600430+0000","last_became_peered":"2023-07-14T11:53:14.600430+0000","last_unstale":"2023-07-14T11:53:21.568721+0000","last_undegraded":"2023-07-14T11:53:21.568721+0000","last_fullsized":"2023-07-14T11:53:21.568721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:30.529533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664703+0000","last_change":"2023-07-14T11:53:19.664675+0000","last_active":"2023-07-14T11:53:20.664703+0000","last_peered":"2023-07-14T11:53:20.664703+0000","last_clean":"2023-07-14T11:53:20.664703+0000","last_became_active":"2023-07-14T11:53:19.663889+0000","last_became_peered":"2023-07-14T11:53:19.663889+0000","last_unstale":"2023-07-14T11:53:20.664703+0000","last_undegraded":"2023-07-14T11:53:20.664703+0000","last_fullsized":"2023-07-14T11:53:20.664703+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:46.192512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568024+0000","last_change":"2023-07-14T11:53:15.606972+0000","last_active":"2023-07-14T11:53:21.568024+0000","last_peered":"2023-07-14T11:53:21.568024+0000","last_clean":"2023-07-14T11:53:21.568024+0000","last_became_active":"2023-07-14T11:53:15.606615+0000","last_became_peered":"2023-07-14T11:53:15.606615+0000","last_unstale":"2023-07-14T11:53:21.568024+0000","last_undegraded":"2023-07-14T11:53:21.568024+0000","last_fullsized":"2023-07-14T11:53:21.568024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:29.641887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569420+0000","last_change":"2023-07-14T11:53:14.595852+0000","last_active":"2023-07-14T11:53:21.569420+0000","last_peered":"2023-07-14T11:53:21.569420+0000","last_clean":"2023-07-14T11:53:21.569420+0000","last_became_active":"2023-07-14T11:53:14.589714+0000","last_became_peered":"2023-07-14T11:53:14.589714+0000","last_unstale":"2023-07-14T11:53:21.569420+0000","last_undegraded":"2023-07-14T11:53:21.569420+0000","last_fullsized":"2023-07-14T11:53:21.569420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:30:20.713856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567265+0000","last_change":"2023-07-14T11:53:19.675250+0000","last_active":"2023-07-14T11:53:21.567265+0000","last_peered":"2023-07-14T11:53:21.567265+0000","last_clean":"2023-07-14T11:53:21.567265+0000","last_became_active":"2023-07-14T11:53:19.674805+0000","last_became_peered":"2023-07-14T11:53:19.674805+0000","last_unstale":"2023-07-14T11:53:21.567265+0000","last_undegraded":"2023-07-14T11:53:21.567265+0000","last_fullsized":"2023-07-14T11:53:21.567265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:44:55.840426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568563+0000","last_change":"2023-07-14T11:53:14.587048+0000","last_active":"2023-07-14T11:53:21.568563+0000","last_peered":"2023-07-14T11:53:21.568563+0000","last_clean":"2023-07-14T11:53:21.568563+0000","last_became_active":"2023-07-14T11:53:14.586722+0000","last_became_peered":"2023-07-14T11:53:14.586722+0000","last_unstale":"2023-07-14T11:53:21.568563+0000","last_undegraded":"2023-07-14T11:53:21.568563+0000","last_fullsized":"2023-07-14T11:53:21.568563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:17.757069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878086+0000","last_change":"2023-07-14T11:53:15.598564+0000","last_active":"2023-07-14T11:53:20.878086+0000","last_peered":"2023-07-14T11:53:20.878086+0000","last_clean":"2023-07-14T11:53:20.878086+0000","last_became_active":"2023-07-14T11:53:15.596542+0000","last_became_peered":"2023-07-14T11:53:15.596542+0000","last_unstale":"2023-07-14T11:53:20.878086+0000","last_undegraded":"2023-07-14T11:53:20.878086+0000","last_fullsized":"2023-07-14T11:53:20.878086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:44:51.768501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568782+0000","last_change":"2023-07-14T11:53:19.670353+0000","last_active":"2023-07-14T11:53:21.568782+0000","last_peered":"2023-07-14T11:53:21.568782+0000","last_clean":"2023-07-14T11:53:21.568782+0000","last_became_active":"2023-07-14T11:53:19.669974+0000","last_became_peered":"2023-07-14T11:53:19.669974+0000","last_unstale":"2023-07-14T11:53:21.568782+0000","last_undegraded":"2023-07-14T11:53:21.568782+0000","last_fullsized":"2023-07-14T11:53:21.568782+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:43.632890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:33.130929+0000","last_change":"2023-07-14T11:53:14.586133+0000","last_active":"2023-07-14T11:53:33.130929+0000","last_peered":"2023-07-14T11:53:33.130929+0000","last_clean":"2023-07-14T11:53:33.130929+0000","last_became_active":"2023-07-14T11:53:14.585705+0000","last_became_peered":"2023-07-14T11:53:14.585705+0000","last_unstale":"2023-07-14T11:53:33.130929+0000","last_undegraded":"2023-07-14T11:53:33.130929+0000","last_fullsized":"2023-07-14T11:53:33.130929+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:48:36.912719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3793,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663286+0000","last_change":"2023-07-14T11:53:15.581733+0000","last_active":"2023-07-14T11:53:20.663286+0000","last_peered":"2023-07-14T11:53:20.663286+0000","last_clean":"2023-07-14T11:53:20.663286+0000","last_became_active":"2023-07-14T11:53:15.581354+0000","last_became_peered":"2023-07-14T11:53:15.581354+0000","last_unstale":"2023-07-14T11:53:20.663286+0000","last_undegraded":"2023-07-14T11:53:20.663286+0000","last_fullsized":"2023-07-14T11:53:20.663286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:24:39.278456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877743+0000","last_change":"2023-07-14T11:53:19.678063+0000","last_active":"2023-07-14T11:53:20.877743+0000","last_peered":"2023-07-14T11:53:20.877743+0000","last_clean":"2023-07-14T11:53:20.877743+0000","last_became_active":"2023-07-14T11:53:19.677638+0000","last_became_peered":"2023-07-14T11:53:19.677638+0000","last_unstale":"2023-07-14T11:53:20.877743+0000","last_undegraded":"2023-07-14T11:53:20.877743+0000","last_fullsized":"2023-07-14T11:53:20.877743+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:32:43.273428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663397+0000","last_change":"2023-07-14T11:53:15.590752+0000","last_active":"2023-07-14T11:53:20.663397+0000","last_peered":"2023-07-14T11:53:20.663397+0000","last_clean":"2023-07-14T11:53:20.663397+0000","last_became_active":"2023-07-14T11:53:15.589947+0000","last_became_peered":"2023-07-14T11:53:15.589947+0000","last_unstale":"2023-07-14T11:53:20.663397+0000","last_undegraded":"2023-07-14T11:53:20.663397+0000","last_fullsized":"2023-07-14T11:53:20.663397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:13:58.842089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665649+0000","last_change":"2023-07-14T11:53:09.145201+0000","last_active":"2023-07-14T11:53:20.665649+0000","last_peered":"2023-07-14T11:53:20.665649+0000","last_clean":"2023-07-14T11:53:20.665649+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:53:20.665649+0000","last_undegraded":"2023-07-14T11:53:20.665649+0000","last_fullsized":"2023-07-14T11:53:20.665649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:54.319358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568612+0000","last_change":"2023-07-14T11:53:14.592544+0000","last_active":"2023-07-14T11:53:21.568612+0000","last_peered":"2023-07-14T11:53:21.568612+0000","last_clean":"2023-07-14T11:53:21.568612+0000","last_became_active":"2023-07-14T11:53:14.592204+0000","last_became_peered":"2023-07-14T11:53:14.592204+0000","last_unstale":"2023-07-14T11:53:21.568612+0000","last_undegraded":"2023-07-14T11:53:21.568612+0000","last_fullsized":"2023-07-14T11:53:21.568612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:12.881399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567791+0000","last_change":"2023-07-14T11:53:19.676141+0000","last_active":"2023-07-14T11:53:21.567791+0000","last_peered":"2023-07-14T11:53:21.567791+0000","last_clean":"2023-07-14T11:53:21.567791+0000","last_became_active":"2023-07-14T11:53:19.675720+0000","last_became_peered":"2023-07-14T11:53:19.675720+0000","last_unstale":"2023-07-14T11:53:21.567791+0000","last_undegraded":"2023-07-14T11:53:21.567791+0000","last_fullsized":"2023-07-14T11:53:21.567791+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:48.417509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568246+0000","last_change":"2023-07-14T11:53:15.607026+0000","last_active":"2023-07-14T11:53:21.568246+0000","last_peered":"2023-07-14T11:53:21.568246+0000","last_clean":"2023-07-14T11:53:21.568246+0000","last_became_active":"2023-07-14T11:53:15.606605+0000","last_became_peered":"2023-07-14T11:53:15.606605+0000","last_unstale":"2023-07-14T11:53:21.568246+0000","last_undegraded":"2023-07-14T11:53:21.568246+0000","last_fullsized":"2023-07-14T11:53:21.568246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:13.888256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878901+0000","last_change":"2023-07-14T11:53:09.146150+0000","last_active":"2023-07-14T11:53:20.878901+0000","last_peered":"2023-07-14T11:53:20.878901+0000","last_clean":"2023-07-14T11:53:20.878901+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:53:20.878901+0000","last_undegraded":"2023-07-14T11:53:20.878901+0000","last_fullsized":"2023-07-14T11:53:20.878901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:07.460041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569870+0000","last_change":"2023-07-14T11:53:14.586533+0000","last_active":"2023-07-14T11:53:21.569870+0000","last_peered":"2023-07-14T11:53:21.569870+0000","last_clean":"2023-07-14T11:53:21.569870+0000","last_became_active":"2023-07-14T11:53:14.586222+0000","last_became_peered":"2023-07-14T11:53:14.586222+0000","last_unstale":"2023-07-14T11:53:21.569870+0000","last_undegraded":"2023-07-14T11:53:21.569870+0000","last_fullsized":"2023-07-14T11:53:21.569870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:38:03.789491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662421+0000","last_change":"2023-07-14T11:53:19.675677+0000","last_active":"2023-07-14T11:53:20.662421+0000","last_peered":"2023-07-14T11:53:20.662421+0000","last_clean":"2023-07-14T11:53:20.662421+0000","last_became_active":"2023-07-14T11:53:19.675259+0000","last_became_peered":"2023-07-14T11:53:19.675259+0000","last_unstale":"2023-07-14T11:53:20.662421+0000","last_undegraded":"2023-07-14T11:53:20.662421+0000","last_fullsized":"2023-07-14T11:53:20.662421+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:05.121760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567406+0000","last_change":"2023-07-14T11:53:15.600612+0000","last_active":"2023-07-14T11:53:21.567406+0000","last_peered":"2023-07-14T11:53:21.567406+0000","last_clean":"2023-07-14T11:53:21.567406+0000","last_became_active":"2023-07-14T11:53:15.600320+0000","last_became_peered":"2023-07-14T11:53:15.600320+0000","last_unstale":"2023-07-14T11:53:21.567406+0000","last_undegraded":"2023-07-14T11:53:21.567406+0000","last_fullsized":"2023-07-14T11:53:21.567406+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:55:00.291201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568702+0000","last_change":"2023-07-14T11:53:09.435876+0000","last_active":"2023-07-14T11:53:21.568702+0000","last_peered":"2023-07-14T11:53:21.568702+0000","last_clean":"2023-07-14T11:53:21.568702+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:53:21.568702+0000","last_undegraded":"2023-07-14T11:53:21.568702+0000","last_fullsized":"2023-07-14T11:53:21.568702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:34.222006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878739+0000","last_change":"2023-07-14T11:53:14.594216+0000","last_active":"2023-07-14T11:53:20.878739+0000","last_peered":"2023-07-14T11:53:20.878739+0000","last_clean":"2023-07-14T11:53:20.878739+0000","last_became_active":"2023-07-14T11:53:14.593954+0000","last_became_peered":"2023-07-14T11:53:14.593954+0000","last_unstale":"2023-07-14T11:53:20.878739+0000","last_undegraded":"2023-07-14T11:53:20.878739+0000","last_fullsized":"2023-07-14T11:53:20.878739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:12.260236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665060+0000","last_change":"2023-07-14T11:53:19.667039+0000","last_active":"2023-07-14T11:53:20.665060+0000","last_peered":"2023-07-14T11:53:20.665060+0000","last_clean":"2023-07-14T11:53:20.665060+0000","last_became_active":"2023-07-14T11:53:19.666360+0000","last_became_peered":"2023-07-14T11:53:19.666360+0000","last_unstale":"2023-07-14T11:53:20.665060+0000","last_undegraded":"2023-07-14T11:53:20.665060+0000","last_fullsized":"2023-07-14T11:53:20.665060+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:37:11.705483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878456+0000","last_change":"2023-07-14T11:53:15.598379+0000","last_active":"2023-07-14T11:53:20.878456+0000","last_peered":"2023-07-14T11:53:20.878456+0000","last_clean":"2023-07-14T11:53:20.878456+0000","last_became_active":"2023-07-14T11:53:15.596661+0000","last_became_peered":"2023-07-14T11:53:15.596661+0000","last_unstale":"2023-07-14T11:53:20.878456+0000","last_undegraded":"2023-07-14T11:53:20.878456+0000","last_fullsized":"2023-07-14T11:53:20.878456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:44.345584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878968+0000","last_change":"2023-07-14T11:53:09.434255+0000","last_active":"2023-07-14T11:53:20.878968+0000","last_peered":"2023-07-14T11:53:20.878968+0000","last_clean":"2023-07-14T11:53:20.878968+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:53:20.878968+0000","last_undegraded":"2023-07-14T11:53:20.878968+0000","last_fullsized":"2023-07-14T11:53:20.878968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:12.335710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567656+0000","last_change":"2023-07-14T11:53:14.589144+0000","last_active":"2023-07-14T11:53:21.567656+0000","last_peered":"2023-07-14T11:53:21.567656+0000","last_clean":"2023-07-14T11:53:21.567656+0000","last_became_active":"2023-07-14T11:53:14.588849+0000","last_became_peered":"2023-07-14T11:53:14.588849+0000","last_unstale":"2023-07-14T11:53:21.567656+0000","last_undegraded":"2023-07-14T11:53:21.567656+0000","last_fullsized":"2023-07-14T11:53:21.567656+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:38.554107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567120+0000","last_change":"2023-07-14T11:53:19.672406+0000","last_active":"2023-07-14T11:53:21.567120+0000","last_peered":"2023-07-14T11:53:21.567120+0000","last_clean":"2023-07-14T11:53:21.567120+0000","last_became_active":"2023-07-14T11:53:19.672009+0000","last_became_peered":"2023-07-14T11:53:19.672009+0000","last_unstale":"2023-07-14T11:53:21.567120+0000","last_undegraded":"2023-07-14T11:53:21.567120+0000","last_fullsized":"2023-07-14T11:53:21.567120+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:36:19.425078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878726+0000","last_change":"2023-07-14T11:53:15.604123+0000","last_active":"2023-07-14T11:53:20.878726+0000","last_peered":"2023-07-14T11:53:20.878726+0000","last_clean":"2023-07-14T11:53:20.878726+0000","last_became_active":"2023-07-14T11:53:15.603815+0000","last_became_peered":"2023-07-14T11:53:15.603815+0000","last_unstale":"2023-07-14T11:53:20.878726+0000","last_undegraded":"2023-07-14T11:53:20.878726+0000","last_fullsized":"2023-07-14T11:53:20.878726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:29.641887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663321+0000","last_change":"2023-07-14T11:53:14.591218+0000","last_active":"2023-07-14T11:53:20.663321+0000","last_peered":"2023-07-14T11:53:20.663321+0000","last_clean":"2023-07-14T11:53:20.663321+0000","last_became_active":"2023-07-14T11:53:14.589122+0000","last_became_peered":"2023-07-14T11:53:14.589122+0000","last_unstale":"2023-07-14T11:53:20.663321+0000","last_undegraded":"2023-07-14T11:53:20.663321+0000","last_fullsized":"2023-07-14T11:53:20.663321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:51:10.380803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566815+0000","last_change":"2023-07-14T11:53:09.147040+0000","last_active":"2023-07-14T11:53:21.566815+0000","last_peered":"2023-07-14T11:53:21.566815+0000","last_clean":"2023-07-14T11:53:21.566815+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:53:21.566815+0000","last_undegraded":"2023-07-14T11:53:21.566815+0000","last_fullsized":"2023-07-14T11:53:21.566815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:23.213672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567493+0000","last_change":"2023-07-14T11:53:19.680353+0000","last_active":"2023-07-14T11:53:21.567493+0000","last_peered":"2023-07-14T11:53:21.567493+0000","last_clean":"2023-07-14T11:53:21.567493+0000","last_became_active":"2023-07-14T11:53:19.674999+0000","last_became_peered":"2023-07-14T11:53:19.674999+0000","last_unstale":"2023-07-14T11:53:21.567493+0000","last_undegraded":"2023-07-14T11:53:21.567493+0000","last_fullsized":"2023-07-14T11:53:21.567493+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:08:07.808470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878795+0000","last_change":"2023-07-14T11:53:15.597158+0000","last_active":"2023-07-14T11:53:20.878795+0000","last_peered":"2023-07-14T11:53:20.878795+0000","last_clean":"2023-07-14T11:53:20.878795+0000","last_became_active":"2023-07-14T11:53:15.595897+0000","last_became_peered":"2023-07-14T11:53:15.595897+0000","last_unstale":"2023-07-14T11:53:20.878795+0000","last_undegraded":"2023-07-14T11:53:20.878795+0000","last_fullsized":"2023-07-14T11:53:20.878795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:39.727537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877308+0000","last_change":"2023-07-14T11:53:14.589940+0000","last_active":"2023-07-14T11:53:20.877308+0000","last_peered":"2023-07-14T11:53:20.877308+0000","last_clean":"2023-07-14T11:53:20.877308+0000","last_became_active":"2023-07-14T11:53:14.589607+0000","last_became_peered":"2023-07-14T11:53:14.589607+0000","last_unstale":"2023-07-14T11:53:20.877308+0000","last_undegraded":"2023-07-14T11:53:20.877308+0000","last_fullsized":"2023-07-14T11:53:20.877308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:28:09.703297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660746+0000","last_change":"2023-07-14T11:53:09.619294+0000","last_active":"2023-07-14T11:53:20.660746+0000","last_peered":"2023-07-14T11:53:20.660746+0000","last_clean":"2023-07-14T11:53:20.660746+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:53:20.660746+0000","last_undegraded":"2023-07-14T11:53:20.660746+0000","last_fullsized":"2023-07-14T11:53:20.660746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:19:36.961857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661632+0000","last_change":"2023-07-14T11:53:19.666668+0000","last_active":"2023-07-14T11:53:20.661632+0000","last_peered":"2023-07-14T11:53:20.661632+0000","last_clean":"2023-07-14T11:53:20.661632+0000","last_became_active":"2023-07-14T11:53:19.666170+0000","last_became_peered":"2023-07-14T11:53:19.666170+0000","last_unstale":"2023-07-14T11:53:20.661632+0000","last_undegraded":"2023-07-14T11:53:20.661632+0000","last_fullsized":"2023-07-14T11:53:20.661632+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:11:15.273201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568313+0000","last_change":"2023-07-14T11:53:15.611544+0000","last_active":"2023-07-14T11:53:21.568313+0000","last_peered":"2023-07-14T11:53:21.568313+0000","last_clean":"2023-07-14T11:53:21.568313+0000","last_became_active":"2023-07-14T11:53:15.606262+0000","last_became_peered":"2023-07-14T11:53:15.606262+0000","last_unstale":"2023-07-14T11:53:21.568313+0000","last_undegraded":"2023-07-14T11:53:21.568313+0000","last_fullsized":"2023-07-14T11:53:21.568313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566653+0000","last_change":"2023-07-14T11:53:14.588920+0000","last_active":"2023-07-14T11:53:21.566653+0000","last_peered":"2023-07-14T11:53:21.566653+0000","last_clean":"2023-07-14T11:53:21.566653+0000","last_became_active":"2023-07-14T11:53:14.588605+0000","last_became_peered":"2023-07-14T11:53:14.588605+0000","last_unstale":"2023-07-14T11:53:21.566653+0000","last_undegraded":"2023-07-14T11:53:21.566653+0000","last_fullsized":"2023-07-14T11:53:21.566653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:54.773189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567666+0000","last_change":"2023-07-14T11:53:09.146156+0000","last_active":"2023-07-14T11:53:21.567666+0000","last_peered":"2023-07-14T11:53:21.567666+0000","last_clean":"2023-07-14T11:53:21.567666+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:53:21.567666+0000","last_undegraded":"2023-07-14T11:53:21.567666+0000","last_fullsized":"2023-07-14T11:53:21.567666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:37.917368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569099+0000","last_change":"2023-07-14T11:53:19.680478+0000","last_active":"2023-07-14T11:53:21.569099+0000","last_peered":"2023-07-14T11:53:21.569099+0000","last_clean":"2023-07-14T11:53:21.569099+0000","last_became_active":"2023-07-14T11:53:19.680082+0000","last_became_peered":"2023-07-14T11:53:19.680082+0000","last_unstale":"2023-07-14T11:53:21.569099+0000","last_undegraded":"2023-07-14T11:53:21.569099+0000","last_fullsized":"2023-07-14T11:53:21.569099+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:42:53.267154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665451+0000","last_change":"2023-07-14T11:53:15.587902+0000","last_active":"2023-07-14T11:53:20.665451+0000","last_peered":"2023-07-14T11:53:20.665451+0000","last_clean":"2023-07-14T11:53:20.665451+0000","last_became_active":"2023-07-14T11:53:15.587578+0000","last_became_peered":"2023-07-14T11:53:15.587578+0000","last_unstale":"2023-07-14T11:53:20.665451+0000","last_undegraded":"2023-07-14T11:53:20.665451+0000","last_fullsized":"2023-07-14T11:53:20.665451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:43:21.367420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567625+0000","last_change":"2023-07-14T11:53:14.600123+0000","last_active":"2023-07-14T11:53:21.567625+0000","last_peered":"2023-07-14T11:53:21.567625+0000","last_clean":"2023-07-14T11:53:21.567625+0000","last_became_active":"2023-07-14T11:53:14.599830+0000","last_became_peered":"2023-07-14T11:53:14.599830+0000","last_unstale":"2023-07-14T11:53:21.567625+0000","last_undegraded":"2023-07-14T11:53:21.567625+0000","last_fullsized":"2023-07-14T11:53:21.567625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:54.937063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567091+0000","last_change":"2023-07-14T11:53:09.146556+0000","last_active":"2023-07-14T11:53:21.567091+0000","last_peered":"2023-07-14T11:53:21.567091+0000","last_clean":"2023-07-14T11:53:21.567091+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:53:21.567091+0000","last_undegraded":"2023-07-14T11:53:21.567091+0000","last_fullsized":"2023-07-14T11:53:21.567091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:07.460041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567928+0000","last_change":"2023-07-14T11:53:03.159818+0000","last_active":"2023-07-14T11:53:21.567928+0000","last_peered":"2023-07-14T11:53:21.567928+0000","last_clean":"2023-07-14T11:53:21.567928+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:53:21.567928+0000","last_undegraded":"2023-07-14T11:53:21.567928+0000","last_fullsized":"2023-07-14T11:53:21.567928+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_clean_scrub_stamp":"2023-07-14T11:53:02.054348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:17.151944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567578+0000","last_change":"2023-07-14T11:53:19.658486+0000","last_active":"2023-07-14T11:53:21.567578+0000","last_peered":"2023-07-14T11:53:21.567578+0000","last_clean":"2023-07-14T11:53:21.567578+0000","last_became_active":"2023-07-14T11:53:19.658083+0000","last_became_peered":"2023-07-14T11:53:19.658083+0000","last_unstale":"2023-07-14T11:53:21.567578+0000","last_undegraded":"2023-07-14T11:53:21.567578+0000","last_fullsized":"2023-07-14T11:53:21.567578+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:08:07.808470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878421+0000","last_change":"2023-07-14T11:53:15.596089+0000","last_active":"2023-07-14T11:53:20.878421+0000","last_peered":"2023-07-14T11:53:20.878421+0000","last_clean":"2023-07-14T11:53:20.878421+0000","last_became_active":"2023-07-14T11:53:15.594769+0000","last_became_peered":"2023-07-14T11:53:15.594769+0000","last_unstale":"2023-07-14T11:53:20.878421+0000","last_undegraded":"2023-07-14T11:53:20.878421+0000","last_fullsized":"2023-07-14T11:53:20.878421+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:55.943920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568475+0000","last_change":"2023-07-14T11:53:14.591999+0000","last_active":"2023-07-14T11:53:21.568475+0000","last_peered":"2023-07-14T11:53:21.568475+0000","last_clean":"2023-07-14T11:53:21.568475+0000","last_became_active":"2023-07-14T11:53:14.591754+0000","last_became_peered":"2023-07-14T11:53:14.591754+0000","last_unstale":"2023-07-14T11:53:21.568475+0000","last_undegraded":"2023-07-14T11:53:21.568475+0000","last_fullsized":"2023-07-14T11:53:21.568475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:39.643364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664358+0000","last_change":"2023-07-14T11:53:19.674798+0000","last_active":"2023-07-14T11:53:20.664358+0000","last_peered":"2023-07-14T11:53:20.664358+0000","last_clean":"2023-07-14T11:53:20.664358+0000","last_became_active":"2023-07-14T11:53:19.674362+0000","last_became_peered":"2023-07-14T11:53:19.674362+0000","last_unstale":"2023-07-14T11:53:20.664358+0000","last_undegraded":"2023-07-14T11:53:20.664358+0000","last_fullsized":"2023-07-14T11:53:20.664358+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:52:07.441376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664411+0000","last_change":"2023-07-14T11:53:15.582299+0000","last_active":"2023-07-14T11:53:20.664411+0000","last_peered":"2023-07-14T11:53:20.664411+0000","last_clean":"2023-07-14T11:53:20.664411+0000","last_became_active":"2023-07-14T11:53:15.582045+0000","last_became_peered":"2023-07-14T11:53:15.582045+0000","last_unstale":"2023-07-14T11:53:20.664411+0000","last_undegraded":"2023-07-14T11:53:20.664411+0000","last_fullsized":"2023-07-14T11:53:20.664411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:14:24.652057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663574+0000","last_change":"2023-07-14T11:53:14.574223+0000","last_active":"2023-07-14T11:53:20.663574+0000","last_peered":"2023-07-14T11:53:20.663574+0000","last_clean":"2023-07-14T11:53:20.663574+0000","last_became_active":"2023-07-14T11:53:14.573601+0000","last_became_peered":"2023-07-14T11:53:14.573601+0000","last_unstale":"2023-07-14T11:53:20.663574+0000","last_undegraded":"2023-07-14T11:53:20.663574+0000","last_fullsized":"2023-07-14T11:53:20.663574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:44:27.636525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568891+0000","last_change":"2023-07-14T11:53:19.671897+0000","last_active":"2023-07-14T11:53:21.568891+0000","last_peered":"2023-07-14T11:53:21.568891+0000","last_clean":"2023-07-14T11:53:21.568891+0000","last_became_active":"2023-07-14T11:53:19.671406+0000","last_became_peered":"2023-07-14T11:53:19.671406+0000","last_unstale":"2023-07-14T11:53:21.568891+0000","last_undegraded":"2023-07-14T11:53:21.568891+0000","last_fullsized":"2023-07-14T11:53:21.568891+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:37:01.963700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664358+0000","last_change":"2023-07-14T11:53:15.583755+0000","last_active":"2023-07-14T11:53:20.664358+0000","last_peered":"2023-07-14T11:53:20.664358+0000","last_clean":"2023-07-14T11:53:20.664358+0000","last_became_active":"2023-07-14T11:53:15.583491+0000","last_became_peered":"2023-07-14T11:53:15.583491+0000","last_unstale":"2023-07-14T11:53:20.664358+0000","last_undegraded":"2023-07-14T11:53:20.664358+0000","last_fullsized":"2023-07-14T11:53:20.664358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:30.105172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665591+0000","last_change":"2023-07-14T11:53:14.594378+0000","last_active":"2023-07-14T11:53:20.665591+0000","last_peered":"2023-07-14T11:53:20.665591+0000","last_clean":"2023-07-14T11:53:20.665591+0000","last_became_active":"2023-07-14T11:53:14.593863+0000","last_became_peered":"2023-07-14T11:53:14.593863+0000","last_unstale":"2023-07-14T11:53:20.665591+0000","last_undegraded":"2023-07-14T11:53:20.665591+0000","last_fullsized":"2023-07-14T11:53:20.665591+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:05:26.104433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878321+0000","last_change":"2023-07-14T11:53:19.674408+0000","last_active":"2023-07-14T11:53:20.878321+0000","last_peered":"2023-07-14T11:53:20.878321+0000","last_clean":"2023-07-14T11:53:20.878321+0000","last_became_active":"2023-07-14T11:53:19.673467+0000","last_became_peered":"2023-07-14T11:53:19.673467+0000","last_unstale":"2023-07-14T11:53:20.878321+0000","last_undegraded":"2023-07-14T11:53:20.878321+0000","last_fullsized":"2023-07-14T11:53:20.878321+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:08:07.808470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663737+0000","last_change":"2023-07-14T11:53:15.592271+0000","last_active":"2023-07-14T11:53:20.663737+0000","last_peered":"2023-07-14T11:53:20.663737+0000","last_clean":"2023-07-14T11:53:20.663737+0000","last_became_active":"2023-07-14T11:53:15.591906+0000","last_became_peered":"2023-07-14T11:53:15.591906+0000","last_unstale":"2023-07-14T11:53:20.663737+0000","last_undegraded":"2023-07-14T11:53:20.663737+0000","last_fullsized":"2023-07-14T11:53:20.663737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:17:30.232523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569314+0000","last_change":"2023-07-14T11:53:14.595162+0000","last_active":"2023-07-14T11:53:21.569314+0000","last_peered":"2023-07-14T11:53:21.569314+0000","last_clean":"2023-07-14T11:53:21.569314+0000","last_became_active":"2023-07-14T11:53:14.594754+0000","last_became_peered":"2023-07-14T11:53:14.594754+0000","last_unstale":"2023-07-14T11:53:21.569314+0000","last_undegraded":"2023-07-14T11:53:21.569314+0000","last_fullsized":"2023-07-14T11:53:21.569314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:19:28.596554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661153+0000","last_change":"2023-07-14T11:53:19.666034+0000","last_active":"2023-07-14T11:53:20.661153+0000","last_peered":"2023-07-14T11:53:20.661153+0000","last_clean":"2023-07-14T11:53:20.661153+0000","last_became_active":"2023-07-14T11:53:19.665570+0000","last_became_peered":"2023-07-14T11:53:19.665570+0000","last_unstale":"2023-07-14T11:53:20.661153+0000","last_undegraded":"2023-07-14T11:53:20.661153+0000","last_fullsized":"2023-07-14T11:53:20.661153+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:48:08.806526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567270+0000","last_change":"2023-07-14T11:53:15.612021+0000","last_active":"2023-07-14T11:53:21.567270+0000","last_peered":"2023-07-14T11:53:21.567270+0000","last_clean":"2023-07-14T11:53:21.567270+0000","last_became_active":"2023-07-14T11:53:15.611653+0000","last_became_peered":"2023-07-14T11:53:15.611653+0000","last_unstale":"2023-07-14T11:53:21.567270+0000","last_undegraded":"2023-07-14T11:53:21.567270+0000","last_fullsized":"2023-07-14T11:53:21.567270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:44:51.768501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566941+0000","last_change":"2023-07-14T11:53:14.590015+0000","last_active":"2023-07-14T11:53:21.566941+0000","last_peered":"2023-07-14T11:53:21.566941+0000","last_clean":"2023-07-14T11:53:21.566941+0000","last_became_active":"2023-07-14T11:53:14.589288+0000","last_became_peered":"2023-07-14T11:53:14.589288+0000","last_unstale":"2023-07-14T11:53:21.566941+0000","last_undegraded":"2023-07-14T11:53:21.566941+0000","last_fullsized":"2023-07-14T11:53:21.566941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:12.260236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660851+0000","last_change":"2023-07-14T11:53:19.664514+0000","last_active":"2023-07-14T11:53:20.660851+0000","last_peered":"2023-07-14T11:53:20.660851+0000","last_clean":"2023-07-14T11:53:20.660851+0000","last_became_active":"2023-07-14T11:53:19.663999+0000","last_became_peered":"2023-07-14T11:53:19.663999+0000","last_unstale":"2023-07-14T11:53:20.660851+0000","last_undegraded":"2023-07-14T11:53:20.660851+0000","last_fullsized":"2023-07-14T11:53:20.660851+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:23:47.861604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664958+0000","last_change":"2023-07-14T11:53:15.597946+0000","last_active":"2023-07-14T11:53:20.664958+0000","last_peered":"2023-07-14T11:53:20.664958+0000","last_clean":"2023-07-14T11:53:20.664958+0000","last_became_active":"2023-07-14T11:53:15.597589+0000","last_became_peered":"2023-07-14T11:53:15.597589+0000","last_unstale":"2023-07-14T11:53:20.664958+0000","last_undegraded":"2023-07-14T11:53:20.664958+0000","last_fullsized":"2023-07-14T11:53:20.664958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:51.380702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568455+0000","last_change":"2023-07-14T11:53:14.599636+0000","last_active":"2023-07-14T11:53:21.568455+0000","last_peered":"2023-07-14T11:53:21.568455+0000","last_clean":"2023-07-14T11:53:21.568455+0000","last_became_active":"2023-07-14T11:53:14.599183+0000","last_became_peered":"2023-07-14T11:53:14.599183+0000","last_unstale":"2023-07-14T11:53:21.568455+0000","last_undegraded":"2023-07-14T11:53:21.568455+0000","last_fullsized":"2023-07-14T11:53:21.568455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:30.529533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661123+0000","last_change":"2023-07-14T11:53:19.681580+0000","last_active":"2023-07-14T11:53:20.661123+0000","last_peered":"2023-07-14T11:53:20.661123+0000","last_clean":"2023-07-14T11:53:20.661123+0000","last_became_active":"2023-07-14T11:53:19.680891+0000","last_became_peered":"2023-07-14T11:53:19.680891+0000","last_unstale":"2023-07-14T11:53:20.661123+0000","last_undegraded":"2023-07-14T11:53:20.661123+0000","last_fullsized":"2023-07-14T11:53:20.661123+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:20:03.759130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663189+0000","last_change":"2023-07-14T11:53:15.610606+0000","last_active":"2023-07-14T11:53:20.663189+0000","last_peered":"2023-07-14T11:53:20.663189+0000","last_clean":"2023-07-14T11:53:20.663189+0000","last_became_active":"2023-07-14T11:53:15.610180+0000","last_became_peered":"2023-07-14T11:53:15.610180+0000","last_unstale":"2023-07-14T11:53:20.663189+0000","last_undegraded":"2023-07-14T11:53:20.663189+0000","last_fullsized":"2023-07-14T11:53:20.663189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:09.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568785+0000","last_change":"2023-07-14T11:53:14.591805+0000","last_active":"2023-07-14T11:53:21.568785+0000","last_peered":"2023-07-14T11:53:21.568785+0000","last_clean":"2023-07-14T11:53:21.568785+0000","last_became_active":"2023-07-14T11:53:14.591439+0000","last_became_peered":"2023-07-14T11:53:14.591439+0000","last_unstale":"2023-07-14T11:53:21.568785+0000","last_undegraded":"2023-07-14T11:53:21.568785+0000","last_fullsized":"2023-07-14T11:53:21.568785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:54:59.284344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660363+0000","last_change":"2023-07-14T11:53:19.675759+0000","last_active":"2023-07-14T11:53:20.660363+0000","last_peered":"2023-07-14T11:53:20.660363+0000","last_clean":"2023-07-14T11:53:20.660363+0000","last_became_active":"2023-07-14T11:53:19.675353+0000","last_became_peered":"2023-07-14T11:53:19.675353+0000","last_unstale":"2023-07-14T11:53:20.660363+0000","last_undegraded":"2023-07-14T11:53:20.660363+0000","last_fullsized":"2023-07-14T11:53:20.660363+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:35:00.376887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566299+0000","last_change":"2023-07-14T11:53:15.603352+0000","last_active":"2023-07-14T11:53:21.566299+0000","last_peered":"2023-07-14T11:53:21.566299+0000","last_clean":"2023-07-14T11:53:21.566299+0000","last_became_active":"2023-07-14T11:53:15.602777+0000","last_became_peered":"2023-07-14T11:53:15.602777+0000","last_unstale":"2023-07-14T11:53:21.566299+0000","last_undegraded":"2023-07-14T11:53:21.566299+0000","last_fullsized":"2023-07-14T11:53:21.566299+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568843+0000","last_change":"2023-07-14T11:53:14.590504+0000","last_active":"2023-07-14T11:53:21.568843+0000","last_peered":"2023-07-14T11:53:21.568843+0000","last_clean":"2023-07-14T11:53:21.568843+0000","last_became_active":"2023-07-14T11:53:14.589844+0000","last_became_peered":"2023-07-14T11:53:14.589844+0000","last_unstale":"2023-07-14T11:53:21.568843+0000","last_undegraded":"2023-07-14T11:53:21.568843+0000","last_fullsized":"2023-07-14T11:53:21.568843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:38.554107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567609+0000","last_change":"2023-07-14T11:53:19.668417+0000","last_active":"2023-07-14T11:53:21.567609+0000","last_peered":"2023-07-14T11:53:21.567609+0000","last_clean":"2023-07-14T11:53:21.567609+0000","last_became_active":"2023-07-14T11:53:19.667924+0000","last_became_peered":"2023-07-14T11:53:19.667924+0000","last_unstale":"2023-07-14T11:53:21.567609+0000","last_undegraded":"2023-07-14T11:53:21.567609+0000","last_fullsized":"2023-07-14T11:53:21.567609+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:00.015845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568675+0000","last_change":"2023-07-14T11:53:19.668219+0000","last_active":"2023-07-14T11:53:21.568675+0000","last_peered":"2023-07-14T11:53:21.568675+0000","last_clean":"2023-07-14T11:53:21.568675+0000","last_became_active":"2023-07-14T11:53:19.667862+0000","last_became_peered":"2023-07-14T11:53:19.667862+0000","last_unstale":"2023-07-14T11:53:21.568675+0000","last_undegraded":"2023-07-14T11:53:21.568675+0000","last_fullsized":"2023-07-14T11:53:21.568675+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:53:26.423183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665093+0000","last_change":"2023-07-14T11:53:14.588139+0000","last_active":"2023-07-14T11:53:20.665093+0000","last_peered":"2023-07-14T11:53:20.665093+0000","last_clean":"2023-07-14T11:53:20.665093+0000","last_became_active":"2023-07-14T11:53:14.587730+0000","last_became_peered":"2023-07-14T11:53:14.587730+0000","last_unstale":"2023-07-14T11:53:20.665093+0000","last_undegraded":"2023-07-14T11:53:20.665093+0000","last_fullsized":"2023-07-14T11:53:20.665093+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:02.335678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569769+0000","last_change":"2023-07-14T11:53:15.606880+0000","last_active":"2023-07-14T11:53:21.569769+0000","last_peered":"2023-07-14T11:53:21.569769+0000","last_clean":"2023-07-14T11:53:21.569769+0000","last_became_active":"2023-07-14T11:53:15.598546+0000","last_became_peered":"2023-07-14T11:53:15.598546+0000","last_unstale":"2023-07-14T11:53:21.569769+0000","last_undegraded":"2023-07-14T11:53:21.569769+0000","last_fullsized":"2023-07-14T11:53:21.569769+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:30:21.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569600+0000","last_change":"2023-07-14T11:53:19.676159+0000","last_active":"2023-07-14T11:53:21.569600+0000","last_peered":"2023-07-14T11:53:21.569600+0000","last_clean":"2023-07-14T11:53:21.569600+0000","last_became_active":"2023-07-14T11:53:19.675729+0000","last_became_peered":"2023-07-14T11:53:19.675729+0000","last_unstale":"2023-07-14T11:53:21.569600+0000","last_undegraded":"2023-07-14T11:53:21.569600+0000","last_fullsized":"2023-07-14T11:53:21.569600+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:49.754582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569051+0000","last_change":"2023-07-14T11:53:14.592455+0000","last_active":"2023-07-14T11:53:21.569051+0000","last_peered":"2023-07-14T11:53:21.569051+0000","last_clean":"2023-07-14T11:53:21.569051+0000","last_became_active":"2023-07-14T11:53:14.592040+0000","last_became_peered":"2023-07-14T11:53:14.592040+0000","last_unstale":"2023-07-14T11:53:21.569051+0000","last_undegraded":"2023-07-14T11:53:21.569051+0000","last_fullsized":"2023-07-14T11:53:21.569051+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:36:56.884917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568516+0000","last_change":"2023-07-14T11:53:15.593198+0000","last_active":"2023-07-14T11:53:21.568516+0000","last_peered":"2023-07-14T11:53:21.568516+0000","last_clean":"2023-07-14T11:53:21.568516+0000","last_became_active":"2023-07-14T11:53:15.592898+0000","last_became_peered":"2023-07-14T11:53:15.592898+0000","last_unstale":"2023-07-14T11:53:21.568516+0000","last_undegraded":"2023-07-14T11:53:21.568516+0000","last_fullsized":"2023-07-14T11:53:21.568516+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:13.267093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566870+0000","last_change":"2023-07-14T11:53:14.588962+0000","last_active":"2023-07-14T11:53:21.566870+0000","last_peered":"2023-07-14T11:53:21.566870+0000","last_clean":"2023-07-14T11:53:21.566870+0000","last_became_active":"2023-07-14T11:53:14.588674+0000","last_became_peered":"2023-07-14T11:53:14.588674+0000","last_unstale":"2023-07-14T11:53:21.566870+0000","last_undegraded":"2023-07-14T11:53:21.566870+0000","last_fullsized":"2023-07-14T11:53:21.566870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:32:38.194645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567389+0000","last_change":"2023-07-14T11:53:15.603164+0000","last_active":"2023-07-14T11:53:21.567389+0000","last_peered":"2023-07-14T11:53:21.567389+0000","last_clean":"2023-07-14T11:53:21.567389+0000","last_became_active":"2023-07-14T11:53:15.602752+0000","last_became_peered":"2023-07-14T11:53:15.602752+0000","last_unstale":"2023-07-14T11:53:21.567389+0000","last_undegraded":"2023-07-14T11:53:21.567389+0000","last_fullsized":"2023-07-14T11:53:21.567389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:40.650221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568701+0000","last_change":"2023-07-14T11:53:19.664284+0000","last_active":"2023-07-14T11:53:21.568701+0000","last_peered":"2023-07-14T11:53:21.568701+0000","last_clean":"2023-07-14T11:53:21.568701+0000","last_became_active":"2023-07-14T11:53:19.663903+0000","last_became_peered":"2023-07-14T11:53:19.663903+0000","last_unstale":"2023-07-14T11:53:21.568701+0000","last_undegraded":"2023-07-14T11:53:21.568701+0000","last_fullsized":"2023-07-14T11:53:21.568701+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:55:04.363127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568411+0000","last_change":"2023-07-14T11:53:14.591199+0000","last_active":"2023-07-14T11:53:21.568411+0000","last_peered":"2023-07-14T11:53:21.568411+0000","last_clean":"2023-07-14T11:53:21.568411+0000","last_became_active":"2023-07-14T11:53:14.590897+0000","last_became_peered":"2023-07-14T11:53:14.590897+0000","last_unstale":"2023-07-14T11:53:21.568411+0000","last_undegraded":"2023-07-14T11:53:21.568411+0000","last_fullsized":"2023-07-14T11:53:21.568411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:53:21.344401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568918+0000","last_change":"2023-07-14T11:53:15.594450+0000","last_active":"2023-07-14T11:53:21.568918+0000","last_peered":"2023-07-14T11:53:21.568918+0000","last_clean":"2023-07-14T11:53:21.568918+0000","last_became_active":"2023-07-14T11:53:15.594170+0000","last_became_peered":"2023-07-14T11:53:15.594170+0000","last_unstale":"2023-07-14T11:53:21.568918+0000","last_undegraded":"2023-07-14T11:53:21.568918+0000","last_fullsized":"2023-07-14T11:53:21.568918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:53:22.351258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567104+0000","last_change":"2023-07-14T11:53:19.682025+0000","last_active":"2023-07-14T11:53:21.567104+0000","last_peered":"2023-07-14T11:53:21.567104+0000","last_clean":"2023-07-14T11:53:21.567104+0000","last_became_active":"2023-07-14T11:53:19.681631+0000","last_became_peered":"2023-07-14T11:53:19.681631+0000","last_unstale":"2023-07-14T11:53:21.567104+0000","last_undegraded":"2023-07-14T11:53:21.567104+0000","last_fullsized":"2023-07-14T11:53:21.567104+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:35.608315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664672+0000","last_change":"2023-07-14T11:53:15.581069+0000","last_active":"2023-07-14T11:53:20.664672+0000","last_peered":"2023-07-14T11:53:20.664672+0000","last_clean":"2023-07-14T11:53:20.664672+0000","last_became_active":"2023-07-14T11:53:15.580609+0000","last_became_peered":"2023-07-14T11:53:15.580609+0000","last_unstale":"2023-07-14T11:53:20.664672+0000","last_undegraded":"2023-07-14T11:53:20.664672+0000","last_fullsized":"2023-07-14T11:53:20.664672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:11:11.201276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567881+0000","last_change":"2023-07-14T11:53:15.616860+0000","last_active":"2023-07-14T11:53:21.567881+0000","last_peered":"2023-07-14T11:53:21.567881+0000","last_clean":"2023-07-14T11:53:21.567881+0000","last_became_active":"2023-07-14T11:53:15.616511+0000","last_became_peered":"2023-07-14T11:53:15.616511+0000","last_unstale":"2023-07-14T11:53:21.567881+0000","last_undegraded":"2023-07-14T11:53:21.567881+0000","last_fullsized":"2023-07-14T11:53:21.567881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663897+0000","last_change":"2023-07-14T11:53:15.580935+0000","last_active":"2023-07-14T11:53:20.663897+0000","last_peered":"2023-07-14T11:53:20.663897+0000","last_clean":"2023-07-14T11:53:20.663897+0000","last_became_active":"2023-07-14T11:53:15.580624+0000","last_became_peered":"2023-07-14T11:53:15.580624+0000","last_unstale":"2023-07-14T11:53:20.663897+0000","last_undegraded":"2023-07-14T11:53:20.663897+0000","last_fullsized":"2023-07-14T11:53:20.663897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:54:27.366163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569054+0000","last_change":"2023-07-14T11:53:15.608124+0000","last_active":"2023-07-14T11:53:21.569054+0000","last_peered":"2023-07-14T11:53:21.569054+0000","last_clean":"2023-07-14T11:53:21.569054+0000","last_became_active":"2023-07-14T11:53:15.606540+0000","last_became_peered":"2023-07-14T11:53:15.606540+0000","last_unstale":"2023-07-14T11:53:21.569054+0000","last_undegraded":"2023-07-14T11:53:21.569054+0000","last_fullsized":"2023-07-14T11:53:21.569054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:36:57.891774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878952+0000","last_change":"2023-07-14T11:53:15.607644+0000","last_active":"2023-07-14T11:53:20.878952+0000","last_peered":"2023-07-14T11:53:20.878952+0000","last_clean":"2023-07-14T11:53:20.878952+0000","last_became_active":"2023-07-14T11:53:15.607318+0000","last_became_peered":"2023-07-14T11:53:15.607318+0000","last_unstale":"2023-07-14T11:53:20.878952+0000","last_undegraded":"2023-07-14T11:53:20.878952+0000","last_fullsized":"2023-07-14T11:53:20.878952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:31.536389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567297+0000","last_change":"2023-07-14T11:53:15.603168+0000","last_active":"2023-07-14T11:53:21.567297+0000","last_peered":"2023-07-14T11:53:21.567297+0000","last_clean":"2023-07-14T11:53:21.567297+0000","last_became_active":"2023-07-14T11:53:15.602791+0000","last_became_peered":"2023-07-14T11:53:15.602791+0000","last_unstale":"2023-07-14T11:53:21.567297+0000","last_undegraded":"2023-07-14T11:53:21.567297+0000","last_fullsized":"2023-07-14T11:53:21.567297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:18.763925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878622+0000","last_change":"2023-07-14T11:53:15.600466+0000","last_active":"2023-07-14T11:53:20.878622+0000","last_peered":"2023-07-14T11:53:20.878622+0000","last_clean":"2023-07-14T11:53:20.878622+0000","last_became_active":"2023-07-14T11:53:15.600232+0000","last_became_peered":"2023-07-14T11:53:15.600232+0000","last_unstale":"2023-07-14T11:53:20.878622+0000","last_undegraded":"2023-07-14T11:53:20.878622+0000","last_fullsized":"2023-07-14T11:53:20.878622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878439+0000","last_change":"2023-07-14T11:53:15.605971+0000","last_active":"2023-07-14T11:53:20.878439+0000","last_peered":"2023-07-14T11:53:20.878439+0000","last_clean":"2023-07-14T11:53:20.878439+0000","last_became_active":"2023-07-14T11:53:15.605661+0000","last_became_peered":"2023-07-14T11:53:15.605661+0000","last_unstale":"2023-07-14T11:53:20.878439+0000","last_undegraded":"2023-07-14T11:53:20.878439+0000","last_fullsized":"2023-07-14T11:53:20.878439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567617+0000","last_change":"2023-07-14T11:53:15.606414+0000","last_active":"2023-07-14T11:53:21.567617+0000","last_peered":"2023-07-14T11:53:21.567617+0000","last_clean":"2023-07-14T11:53:21.567617+0000","last_became_active":"2023-07-14T11:53:15.605906+0000","last_became_peered":"2023-07-14T11:53:15.605906+0000","last_unstale":"2023-07-14T11:53:21.567617+0000","last_undegraded":"2023-07-14T11:53:21.567617+0000","last_fullsized":"2023-07-14T11:53:21.567617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567728+0000","last_change":"2023-07-14T11:53:15.598006+0000","last_active":"2023-07-14T11:53:21.567728+0000","last_peered":"2023-07-14T11:53:21.567728+0000","last_clean":"2023-07-14T11:53:21.567728+0000","last_became_active":"2023-07-14T11:53:15.597749+0000","last_became_peered":"2023-07-14T11:53:15.597749+0000","last_unstale":"2023-07-14T11:53:21.567728+0000","last_undegraded":"2023-07-14T11:53:21.567728+0000","last_fullsized":"2023-07-14T11:53:21.567728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567052+0000","last_change":"2023-07-14T11:53:15.600730+0000","last_active":"2023-07-14T11:53:21.567052+0000","last_peered":"2023-07-14T11:53:21.567052+0000","last_clean":"2023-07-14T11:53:21.567052+0000","last_became_active":"2023-07-14T11:53:15.600299+0000","last_became_peered":"2023-07-14T11:53:15.600299+0000","last_unstale":"2023-07-14T11:53:21.567052+0000","last_undegraded":"2023-07-14T11:53:21.567052+0000","last_fullsized":"2023-07-14T11:53:21.567052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:18:17.533263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568245+0000","last_change":"2023-07-14T11:53:15.619732+0000","last_active":"2023-07-14T11:53:21.568245+0000","last_peered":"2023-07-14T11:53:21.568245+0000","last_clean":"2023-07-14T11:53:21.568245+0000","last_became_active":"2023-07-14T11:53:15.619425+0000","last_became_peered":"2023-07-14T11:53:15.619425+0000","last_unstale":"2023-07-14T11:53:21.568245+0000","last_undegraded":"2023-07-14T11:53:21.568245+0000","last_fullsized":"2023-07-14T11:53:21.568245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:13.888256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663540+0000","last_change":"2023-07-14T11:53:15.586169+0000","last_active":"2023-07-14T11:53:20.663540+0000","last_peered":"2023-07-14T11:53:20.663540+0000","last_clean":"2023-07-14T11:53:20.663540+0000","last_became_active":"2023-07-14T11:53:15.585876+0000","last_became_peered":"2023-07-14T11:53:15.585876+0000","last_unstale":"2023-07-14T11:53:20.663540+0000","last_undegraded":"2023-07-14T11:53:20.663540+0000","last_fullsized":"2023-07-14T11:53:20.663540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:47:20.678085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661229+0000","last_change":"2023-07-14T11:53:15.584035+0000","last_active":"2023-07-14T11:53:20.661229+0000","last_peered":"2023-07-14T11:53:20.661229+0000","last_clean":"2023-07-14T11:53:20.661229+0000","last_became_active":"2023-07-14T11:53:15.582303+0000","last_became_peered":"2023-07-14T11:53:15.582303+0000","last_unstale":"2023-07-14T11:53:20.661229+0000","last_undegraded":"2023-07-14T11:53:20.661229+0000","last_fullsized":"2023-07-14T11:53:20.661229+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:48:16.619167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664187+0000","last_change":"2023-07-14T11:53:15.578919+0000","last_active":"2023-07-14T11:53:20.664187+0000","last_peered":"2023-07-14T11:53:20.664187+0000","last_clean":"2023-07-14T11:53:20.664187+0000","last_became_active":"2023-07-14T11:53:15.578590+0000","last_became_peered":"2023-07-14T11:53:15.578590+0000","last_unstale":"2023-07-14T11:53:20.664187+0000","last_undegraded":"2023-07-14T11:53:20.664187+0000","last_fullsized":"2023-07-14T11:53:20.664187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:03:59.635384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568314+0000","last_change":"2023-07-14T11:53:15.613102+0000","last_active":"2023-07-14T11:53:21.568314+0000","last_peered":"2023-07-14T11:53:21.568314+0000","last_clean":"2023-07-14T11:53:21.568314+0000","last_became_active":"2023-07-14T11:53:15.612540+0000","last_became_peered":"2023-07-14T11:53:15.612540+0000","last_unstale":"2023-07-14T11:53:21.568314+0000","last_undegraded":"2023-07-14T11:53:21.568314+0000","last_fullsized":"2023-07-14T11:53:21.568314+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:29.641887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662627+0000","last_change":"2023-07-14T11:53:15.592089+0000","last_active":"2023-07-14T11:53:20.662627+0000","last_peered":"2023-07-14T11:53:20.662627+0000","last_clean":"2023-07-14T11:53:20.662627+0000","last_became_active":"2023-07-14T11:53:15.591826+0000","last_became_peered":"2023-07-14T11:53:15.591826+0000","last_unstale":"2023-07-14T11:53:20.662627+0000","last_undegraded":"2023-07-14T11:53:20.662627+0000","last_fullsized":"2023-07-14T11:53:20.662627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:06:57.233859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568086+0000","last_change":"2023-07-14T11:53:15.606636+0000","last_active":"2023-07-14T11:53:21.568086+0000","last_peered":"2023-07-14T11:53:21.568086+0000","last_clean":"2023-07-14T11:53:21.568086+0000","last_became_active":"2023-07-14T11:53:15.606067+0000","last_became_peered":"2023-07-14T11:53:15.606067+0000","last_unstale":"2023-07-14T11:53:21.568086+0000","last_undegraded":"2023-07-14T11:53:21.568086+0000","last_fullsized":"2023-07-14T11:53:21.568086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:13.267093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566709+0000","last_change":"2023-07-14T11:53:15.599253+0000","last_active":"2023-07-14T11:53:21.566709+0000","last_peered":"2023-07-14T11:53:21.566709+0000","last_clean":"2023-07-14T11:53:21.566709+0000","last_became_active":"2023-07-14T11:53:15.598955+0000","last_became_peered":"2023-07-14T11:53:15.598955+0000","last_unstale":"2023-07-14T11:53:21.566709+0000","last_undegraded":"2023-07-14T11:53:21.566709+0000","last_fullsized":"2023-07-14T11:53:21.566709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660486+0000","last_change":"2023-07-14T11:53:15.592119+0000","last_active":"2023-07-14T11:53:20.660486+0000","last_peered":"2023-07-14T11:53:20.660486+0000","last_clean":"2023-07-14T11:53:20.660486+0000","last_became_active":"2023-07-14T11:53:15.591822+0000","last_became_peered":"2023-07-14T11:53:15.591822+0000","last_unstale":"2023-07-14T11:53:20.660486+0000","last_undegraded":"2023-07-14T11:53:20.660486+0000","last_fullsized":"2023-07-14T11:53:20.660486+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:04:29.483383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661402+0000","last_change":"2023-07-14T11:53:15.590820+0000","last_active":"2023-07-14T11:53:20.661402+0000","last_peered":"2023-07-14T11:53:20.661402+0000","last_clean":"2023-07-14T11:53:20.661402+0000","last_became_active":"2023-07-14T11:53:15.590485+0000","last_became_peered":"2023-07-14T11:53:15.590485+0000","last_unstale":"2023-07-14T11:53:20.661402+0000","last_undegraded":"2023-07-14T11:53:20.661402+0000","last_fullsized":"2023-07-14T11:53:20.661402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:42.120586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664155+0000","last_change":"2023-07-14T11:53:15.589398+0000","last_active":"2023-07-14T11:53:20.664155+0000","last_peered":"2023-07-14T11:53:20.664155+0000","last_clean":"2023-07-14T11:53:20.664155+0000","last_became_active":"2023-07-14T11:53:15.589111+0000","last_became_peered":"2023-07-14T11:53:15.589111+0000","last_unstale":"2023-07-14T11:53:20.664155+0000","last_undegraded":"2023-07-14T11:53:20.664155+0000","last_fullsized":"2023-07-14T11:53:20.664155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:40:44.358262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569248+0000","last_change":"2023-07-14T11:53:15.599081+0000","last_active":"2023-07-14T11:53:21.569248+0000","last_peered":"2023-07-14T11:53:21.569248+0000","last_clean":"2023-07-14T11:53:21.569248+0000","last_became_active":"2023-07-14T11:53:15.598786+0000","last_became_peered":"2023-07-14T11:53:15.598786+0000","last_unstale":"2023-07-14T11:53:21.569248+0000","last_undegraded":"2023-07-14T11:53:21.569248+0000","last_fullsized":"2023-07-14T11:53:21.569248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:45.682657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567984+0000","last_change":"2023-07-14T11:53:15.613338+0000","last_active":"2023-07-14T11:53:21.567984+0000","last_peered":"2023-07-14T11:53:21.567984+0000","last_clean":"2023-07-14T11:53:21.567984+0000","last_became_active":"2023-07-14T11:53:15.612876+0000","last_became_peered":"2023-07-14T11:53:15.612876+0000","last_unstale":"2023-07-14T11:53:21.567984+0000","last_undegraded":"2023-07-14T11:53:21.567984+0000","last_fullsized":"2023-07-14T11:53:21.567984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566164+0000","last_change":"2023-07-14T11:53:15.608445+0000","last_active":"2023-07-14T11:53:21.566164+0000","last_peered":"2023-07-14T11:53:21.566164+0000","last_clean":"2023-07-14T11:53:21.566164+0000","last_became_active":"2023-07-14T11:53:15.608146+0000","last_became_peered":"2023-07-14T11:53:15.608146+0000","last_unstale":"2023-07-14T11:53:21.566164+0000","last_undegraded":"2023-07-14T11:53:21.566164+0000","last_fullsized":"2023-07-14T11:53:21.566164+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:55.943920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567855+0000","last_change":"2023-07-14T11:53:15.606963+0000","last_active":"2023-07-14T11:53:21.567855+0000","last_peered":"2023-07-14T11:53:21.567855+0000","last_clean":"2023-07-14T11:53:21.567855+0000","last_became_active":"2023-07-14T11:53:15.606397+0000","last_became_peered":"2023-07-14T11:53:15.606397+0000","last_unstale":"2023-07-14T11:53:21.567855+0000","last_undegraded":"2023-07-14T11:53:21.567855+0000","last_fullsized":"2023-07-14T11:53:21.567855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663074+0000","last_change":"2023-07-14T11:53:15.607539+0000","last_active":"2023-07-14T11:53:20.663074+0000","last_peered":"2023-07-14T11:53:20.663074+0000","last_clean":"2023-07-14T11:53:20.663074+0000","last_became_active":"2023-07-14T11:53:15.593279+0000","last_became_peered":"2023-07-14T11:53:15.593279+0000","last_unstale":"2023-07-14T11:53:20.663074+0000","last_undegraded":"2023-07-14T11:53:20.663074+0000","last_fullsized":"2023-07-14T11:53:20.663074+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:34.163470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569328+0000","last_change":"2023-07-14T11:53:15.592332+0000","last_active":"2023-07-14T11:53:21.569328+0000","last_peered":"2023-07-14T11:53:21.569328+0000","last_clean":"2023-07-14T11:53:21.569328+0000","last_became_active":"2023-07-14T11:53:15.591983+0000","last_became_peered":"2023-07-14T11:53:15.591983+0000","last_unstale":"2023-07-14T11:53:21.569328+0000","last_undegraded":"2023-07-14T11:53:21.569328+0000","last_fullsized":"2023-07-14T11:53:21.569328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:18.763925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665081+0000","last_change":"2023-07-14T11:53:15.594122+0000","last_active":"2023-07-14T11:53:20.665081+0000","last_peered":"2023-07-14T11:53:20.665081+0000","last_clean":"2023-07-14T11:53:20.665081+0000","last_became_active":"2023-07-14T11:53:15.593773+0000","last_became_peered":"2023-07-14T11:53:15.593773+0000","last_unstale":"2023-07-14T11:53:20.665081+0000","last_undegraded":"2023-07-14T11:53:20.665081+0000","last_fullsized":"2023-07-14T11:53:20.665081+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:59:21.935826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664107+0000","last_change":"2023-07-14T11:53:15.593314+0000","last_active":"2023-07-14T11:53:20.664107+0000","last_peered":"2023-07-14T11:53:20.664107+0000","last_clean":"2023-07-14T11:53:20.664107+0000","last_became_active":"2023-07-14T11:53:15.592778+0000","last_became_peered":"2023-07-14T11:53:15.592778+0000","last_unstale":"2023-07-14T11:53:20.664107+0000","last_undegraded":"2023-07-14T11:53:20.664107+0000","last_fullsized":"2023-07-14T11:53:20.664107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:37.263541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568937+0000","last_change":"2023-07-14T11:53:15.599579+0000","last_active":"2023-07-14T11:53:21.568937+0000","last_peered":"2023-07-14T11:53:21.568937+0000","last_clean":"2023-07-14T11:53:21.568937+0000","last_became_active":"2023-07-14T11:53:15.599380+0000","last_became_peered":"2023-07-14T11:53:15.599380+0000","last_unstale":"2023-07-14T11:53:21.568937+0000","last_undegraded":"2023-07-14T11:53:21.568937+0000","last_fullsized":"2023-07-14T11:53:21.568937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:19:29.603411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661719+0000","last_change":"2023-07-14T11:53:15.594232+0000","last_active":"2023-07-14T11:53:20.661719+0000","last_peered":"2023-07-14T11:53:20.661719+0000","last_clean":"2023-07-14T11:53:20.661719+0000","last_became_active":"2023-07-14T11:53:15.594049+0000","last_became_peered":"2023-07-14T11:53:15.594049+0000","last_unstale":"2023-07-14T11:53:20.661719+0000","last_undegraded":"2023-07-14T11:53:20.661719+0000","last_fullsized":"2023-07-14T11:53:20.661719+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:51.380702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4591,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":925696,"data_stored":718949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6220000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":311296,"data_stored":125224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4750000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":323584,"data_stored":128421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.609}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3580000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4670000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":901120,"data_stored":715522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:53:43.524 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-14T11:53:43.524 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-14T11:53:43.525 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-07-14T11:53:43.525 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-07-14T11:53:43.526 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-07-14T11:53:43.730 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.744+0000 7fe56a14d700 1 Processor -- start 2023-07-14T11:53:43.731 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe56a14d700 1 -- start start 2023-07-14T11:53:43.732 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe56a14d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe56406be70 0x7fe5640cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:43.733 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe56a14d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe5640c1e00 0x7fe5640cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:43.733 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe56a14d700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5640cc1b0 con 0x7fe56406d860 2023-07-14T11:53:43.733 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe56a14d700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5640cd690 con 0x7fe5640c1e00 2023-07-14T11:53:43.733 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe56a14d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5640cd970 con 0x7fe56406be70 2023-07-14T11:53:43.734 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe5637fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fe56406d860 0x7fe56406dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42438/0 (socket says 172.21.15.104:42438) 2023-07-14T11:53:43.734 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe5637fe700 1 -- 172.21.15.104:0/828436218 learned_addr learned my addr 172.21.15.104:0/828436218 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:43.734 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe563fff700 1 --2- 172.21.15.104:0/828436218 >> v2:172.21.15.104:3300/0 conn(0x7fe5640c1e00 0x7fe5640cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:43.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe55bfff700 1 --2- 172.21.15.104:0/828436218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe56406be70 0x7fe5640cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:43.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe562ffd700 1 -- 172.21.15.104:0/828436218 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805752014 0 0) 0x7fe5640cc1b0 con 0x7fe56406d860 2023-07-14T11:53:43.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe563fff700 1 -- 172.21.15.104:0/828436218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe56406be70 msgr2=0x7fe5640cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:43.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe563fff700 1 --2- 172.21.15.104:0/828436218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe56406be70 0x7fe5640cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe563fff700 1 -- 172.21.15.104:0/828436218 >> v1:172.21.15.104:6789/0 conn(0x7fe56406d860 legacy=0x7fe56406dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe563fff700 1 -- 172.21.15.104:0/828436218 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5640cdc50 con 0x7fe5640c1e00 2023-07-14T11:53:43.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.748+0000 7fe55bfff700 1 --2- 172.21.15.104:0/828436218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe56406be70 0x7fe5640cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:43.736 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.752+0000 7fe563fff700 1 --2- 172.21.15.104:0/828436218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5640c1e00 0x7fe5640cc640 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009dc0 tx=0x7fe55c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d70a49986eeda7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:43.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.752+0000 7fe562ffd700 1 -- 172.21.15.104:0/828436218 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c00e070 con 0x7fe5640c1e00 2023-07-14T11:53:43.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.752+0000 7fe562ffd700 1 -- 172.21.15.104:0/828436218 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55c00bae0 con 0x7fe5640c1e00 2023-07-14T11:53:43.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.752+0000 7fe562ffd700 1 -- 172.21.15.104:0/828436218 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c00be90 con 0x7fe5640c1e00 2023-07-14T11:53:43.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.752+0000 7fe56a14d700 1 -- 172.21.15.104:0/828436218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5640c1e00 msgr2=0x7fe5640cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.752+0000 7fe56a14d700 1 --2- 172.21.15.104:0/828436218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5640c1e00 0x7fe5640cc640 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009dc0 tx=0x7fe55c00b330 comp rx=0 tx=0).stop 2023-07-14T11:53:43.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.752+0000 7fe56a14d700 1 -- 172.21.15.104:0/828436218 shutdown_connections 2023-07-14T11:53:43.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.752+0000 7fe56a14d700 1 --2- 172.21.15.104:0/828436218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5640c1e00 0x7fe5640cc640 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.752+0000 7fe56a14d700 1 --2- 172.21.15.104:0/828436218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe56406be70 0x7fe5640cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.752+0000 7fe56a14d700 1 -- 172.21.15.104:0/828436218 >> 172.21.15.104:0/828436218 conn(0x7fe5640cad30 msgr2=0x7fe5640cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:43.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.752+0000 7fe56a14d700 1 -- 172.21.15.104:0/828436218 shutdown_connections 2023-07-14T11:53:43.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.752+0000 7fe56a14d700 1 -- 172.21.15.104:0/828436218 wait complete. 2023-07-14T11:53:43.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe56a14d700 1 Processor -- start 2023-07-14T11:53:43.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe56a14d700 1 -- start start 2023-07-14T11:53:43.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe56a14d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe56406be70 0x7fe564150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:43.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe56a14d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56406d860 0x7fe564070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:43.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe56a14d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5640c1e00 0x7fe564070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:43.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe56a14d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5640afc80 con 0x7fe5640c1e00 2023-07-14T11:53:43.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe56a14d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe564150ed0 con 0x7fe56406d860 2023-07-14T11:53:43.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe56a14d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5640af070 con 0x7fe56406be70 2023-07-14T11:53:43.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe563fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5640c1e00 0x7fe564070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:43.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe563fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5640c1e00 0x7fe564070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38118/0 (socket says 172.21.15.104:38118) 2023-07-14T11:53:43.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe563fff700 1 -- 172.21.15.104:0/1951448763 learned_addr learned my addr 172.21.15.104:0/1951448763 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:43.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe55bfff700 1 --2- 172.21.15.104:0/1951448763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56406d860 0x7fe564070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:43.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe5637fe700 1 --2- 172.21.15.104:0/1951448763 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe56406be70 0x7fe564150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:43.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe563fff700 1 -- 172.21.15.104:0/1951448763 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe56406be70 msgr2=0x7fe564150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe563fff700 1 --2- 172.21.15.104:0/1951448763 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe56406be70 0x7fe564150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe563fff700 1 -- 172.21.15.104:0/1951448763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56406d860 msgr2=0x7fe564070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe563fff700 1 --2- 172.21.15.104:0/1951448763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56406d860 0x7fe564070640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe563fff700 1 -- 172.21.15.104:0/1951448763 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c0094b0 con 0x7fe5640c1e00 2023-07-14T11:53:43.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe55bfff700 1 --2- 172.21.15.104:0/1951448763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56406d860 0x7fe564070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:43.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.756+0000 7fe563fff700 1 --2- 172.21.15.104:0/1951448763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5640c1e00 0x7fe564070b80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe55c014040 tx=0x7fe55c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:43.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.760+0000 7fe560ff9700 1 -- 172.21.15.104:0/1951448763 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c004740 con 0x7fe5640c1e00 2023-07-14T11:53:43.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.760+0000 7fe560ff9700 1 -- 172.21.15.104:0/1951448763 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55c020070 con 0x7fe5640c1e00 2023-07-14T11:53:43.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.760+0000 7fe56a14d700 1 -- 172.21.15.104:0/1951448763 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55c009a70 con 0x7fe5640c1e00 2023-07-14T11:53:43.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.760+0000 7fe560ff9700 1 -- 172.21.15.104:0/1951448763 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c027480 con 0x7fe5640c1e00 2023-07-14T11:53:43.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.760+0000 7fe56a14d700 1 -- 172.21.15.104:0/1951448763 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5640af630 con 0x7fe5640c1e00 2023-07-14T11:53:43.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.760+0000 7fe56a14d700 1 -- 172.21.15.104:0/1951448763 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5640afdf0 con 0x7fe5640c1e00 2023-07-14T11:53:43.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.760+0000 7fe560ff9700 1 -- 172.21.15.104:0/1951448763 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe55c027600 con 0x7fe5640c1e00 2023-07-14T11:53:43.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.764+0000 7fe560ff9700 1 --2- 172.21.15.104:0/1951448763 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe5440662f0 0x7fe5440687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:43.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.764+0000 7fe560ff9700 1 -- 172.21.15.104:0/1951448763 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe55c022070 con 0x7fe5640c1e00 2023-07-14T11:53:43.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.764+0000 7fe5637fe700 1 --2- 172.21.15.104:0/1951448763 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe5440662f0 0x7fe5440687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:43.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.764+0000 7fe5637fe700 1 --2- 172.21.15.104:0/1951448763 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe5440662f0 0x7fe5440687a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe554000f60 tx=0x7fe554007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:43.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.768+0000 7fe560ff9700 1 -- 172.21.15.104:0/1951448763 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe55c0531f0 con 0x7fe5640c1e00 2023-07-14T11:53:43.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.984+0000 7fe56a14d700 1 -- 172.21.15.104:0/1951448763 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe56404dcb0 con 0x7fe5640c1e00 2023-07-14T11:53:43.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.988+0000 7fe560ff9700 1 -- 172.21.15.104:0/1951448763 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164245 (secure 0 0 0) 0x7fe55c02c090 con 0x7fe5640c1e00 2023-07-14T11:53:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:43.980 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.996+0000 7fe56a14d700 1 -- 172.21.15.104:0/1951448763 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe5440662f0 msgr2=0x7fe5440687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.980 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.996+0000 7fe56a14d700 1 --2- 172.21.15.104:0/1951448763 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe5440662f0 0x7fe5440687a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe554000f60 tx=0x7fe554007c00 comp rx=0 tx=0).stop 2023-07-14T11:53:43.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.996+0000 7fe56a14d700 1 -- 172.21.15.104:0/1951448763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5640c1e00 msgr2=0x7fe564070b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.996+0000 7fe56a14d700 1 --2- 172.21.15.104:0/1951448763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5640c1e00 0x7fe564070b80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe55c014040 tx=0x7fe55c004af0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.996+0000 7fe56a14d700 1 -- 172.21.15.104:0/1951448763 shutdown_connections 2023-07-14T11:53:43.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.996+0000 7fe56a14d700 1 --2- 172.21.15.104:0/1951448763 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe5440662f0 0x7fe5440687a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.982 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.996+0000 7fe56a14d700 1 --2- 172.21.15.104:0/1951448763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5640c1e00 0x7fe564070b80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.982 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.996+0000 7fe56a14d700 1 --2- 172.21.15.104:0/1951448763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56406d860 0x7fe564070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.982 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.996+0000 7fe56a14d700 1 --2- 172.21.15.104:0/1951448763 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe56406be70 0x7fe564150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.996+0000 7fe56a14d700 1 -- 172.21.15.104:0/1951448763 >> 172.21.15.104:0/1951448763 conn(0x7fe5640cad30 msgr2=0x7fe564157540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:43.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.996+0000 7fe56a14d700 1 -- 172.21.15.104:0/1951448763 shutdown_connections 2023-07-14T11:53:43.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:43.996+0000 7fe56a14d700 1 -- 172.21.15.104:0/1951448763 wait complete. 2023-07-14T11:53:44.005 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":5,"active_gid":4108,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":608103909},{"type":"v1","addr":"172.21.15.104:6833","nonce":608103909}]},"active_addr":"172.21.15.104:6833/608103909","active_change":"2023-07-14T11:52:53.857768+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4101,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.104:0","nonce":1027171977}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.104:0","nonce":111975155}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.104:0","nonce":2107309340}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.104:0","nonce":4263940863}]}]} 2023-07-14T11:53:44.009 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-07-14T11:53:44.010 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-07-14T11:53:44.010 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:44.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.204+0000 7fe1af59e700 1 Processor -- start 2023-07-14T11:53:44.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.204+0000 7fe1af59e700 1 -- start start 2023-07-14T11:53:44.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.204+0000 7fe1af59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe1b0159f20 0x7fe1b00caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:44.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.204+0000 7fe1af59e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b00b0ef0 con 0x7fe1b00caf70 2023-07-14T11:53:44.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.204+0000 7fe1af59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b00ce9c0 con 0x7fe1b00c9d90 2023-07-14T11:53:44.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.204+0000 7fe1af59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b00cee50 con 0x7fe1b0159f20 2023-07-14T11:53:44.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.204+0000 7fe1aed9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fe1b00caf70 0x7fe1b015d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42454/0 (socket says 172.21.15.104:42454) 2023-07-14T11:53:44.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.204+0000 7fe1aed9d700 1 -- 172.21.15.104:0/3792822783 learned_addr learned my addr 172.21.15.104:0/3792822783 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:44.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.204+0000 7fe1a7fff700 1 --2- 172.21.15.104:0/3792822783 >> v2:172.21.15.121:3300/0 conn(0x7fe1b0159f20 0x7fe1b00caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:44.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.204+0000 7fe1add9b700 1 -- 172.21.15.104:0/3792822783 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412752333 0 0) 0x7fe1b00b0ef0 con 0x7fe1b00caf70 2023-07-14T11:53:44.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.204+0000 7fe1add9b700 1 -- 172.21.15.104:0/3792822783 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe19c0035d0 con 0x7fe1b00caf70 2023-07-14T11:53:44.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 -- 172.21.15.104:0/3792822783 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 49017401 0 0) 0x7fe19c0035d0 con 0x7fe1b00caf70 2023-07-14T11:53:44.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 -- 172.21.15.104:0/3792822783 >> v2:172.21.15.121:3300/0 conn(0x7fe1b0159f20 msgr2=0x7fe1b00caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:44.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 --2- 172.21.15.104:0/3792822783 >> v2:172.21.15.121:3300/0 conn(0x7fe1b0159f20 0x7fe1b00caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 -- 172.21.15.104:0/3792822783 >> v1:172.21.15.121:6789/0 conn(0x7fe1b00c9d90 legacy=0x7fe1b00ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:44.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 -- 172.21.15.104:0/3792822783 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b00cb4f0 con 0x7fe1b00caf70 2023-07-14T11:53:44.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 -- 172.21.15.104:0/3792822783 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7fe1a80019f0 con 0x7fe1b00caf70 2023-07-14T11:53:44.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 -- 172.21.15.104:0/3792822783 >> v1:172.21.15.104:6789/0 conn(0x7fe1b00caf70 legacy=0x7fe1b015d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:44.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 --2- 172.21.15.104:0/3792822783 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe19c003a20 0x7fe19c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:44.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 --2- 172.21.15.104:0/3792822783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c004870 0x7fe19c004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:44.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 --2- 172.21.15.104:0/3792822783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe19c005220 0x7fe19c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:44.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 -- 172.21.15.104:0/3792822783 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b00cee50 con 0x7fe19c005220 2023-07-14T11:53:44.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 -- 172.21.15.104:0/3792822783 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c00c1b0 con 0x7fe19c004870 2023-07-14T11:53:44.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 -- 172.21.15.104:0/3792822783 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c00c370 con 0x7fe19c003a20 2023-07-14T11:53:44.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1a7fff700 1 --2- 172.21.15.104:0/3792822783 >> v2:172.21.15.121:3300/0 conn(0x7fe1b0159f20 0x7fe1b00caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:44.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1aed9d700 1 --2- 172.21.15.104:0/3792822783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe19c005220 0x7fe19c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:44.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1a7fff700 1 --2- 172.21.15.104:0/3792822783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c004870 0x7fe19c004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:44.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1aed9d700 1 -- 172.21.15.104:0/3792822783 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe19c003a20 msgr2=0x7fe19c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:44.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1aed9d700 1 --2- 172.21.15.104:0/3792822783 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe19c003a20 0x7fe19c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1aed9d700 1 -- 172.21.15.104:0/3792822783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c004870 msgr2=0x7fe19c004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:44.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1aed9d700 1 --2- 172.21.15.104:0/3792822783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c004870 0x7fe19c004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1aed9d700 1 -- 172.21.15.104:0/3792822783 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19c0035d0 con 0x7fe19c005220 2023-07-14T11:53:44.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1aed9d700 1 -- 172.21.15.104:0/3792822783 reap_dead start 2023-07-14T11:53:44.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1aed9d700 1 --2- 172.21.15.104:0/3792822783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe19c005220 0x7fe19c00ba30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800a800 tx=0x7fe19c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d429debac12e0e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:44.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 -- 172.21.15.104:0/3792822783 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a800a2f0 con 0x7fe19c005220 2023-07-14T11:53:44.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 -- 172.21.15.104:0/3792822783 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a80072f0 con 0x7fe19c005220 2023-07-14T11:53:44.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1add9b700 1 -- 172.21.15.104:0/3792822783 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a80076a0 con 0x7fe19c005220 2023-07-14T11:53:44.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1af59e700 1 -- 172.21.15.104:0/3792822783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe19c005220 msgr2=0x7fe19c00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:44.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1af59e700 1 --2- 172.21.15.104:0/3792822783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe19c005220 0x7fe19c00ba30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800a800 tx=0x7fe19c0058a0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1af59e700 1 -- 172.21.15.104:0/3792822783 shutdown_connections 2023-07-14T11:53:44.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1af59e700 1 --2- 172.21.15.104:0/3792822783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe19c005220 0x7fe19c00ba30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1af59e700 1 -- 172.21.15.104:0/3792822783 >> 172.21.15.104:0/3792822783 conn(0x7fe1b006e960 msgr2=0x7fe1b006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:44.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1af59e700 1 -- 172.21.15.104:0/3792822783 shutdown_connections 2023-07-14T11:53:44.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.208+0000 7fe1af59e700 1 -- 172.21.15.104:0/3792822783 wait complete. 2023-07-14T11:53:44.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.212+0000 7fe1af59e700 1 Processor -- start 2023-07-14T11:53:44.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.212+0000 7fe1af59e700 1 -- start start 2023-07-14T11:53:44.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.212+0000 7fe1af59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c005220 0x7fe1b01589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:44.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.212+0000 7fe1af59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1b00c6e10 0x7fe1b00c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:44.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.212+0000 7fe1af59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1b00c5670 0x7fe1b01473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:44.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.212+0000 7fe1af59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b0145650 con 0x7fe1b00c5670 2023-07-14T11:53:44.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.212+0000 7fe1af59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b0145930 con 0x7fe19c005220 2023-07-14T11:53:44.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.212+0000 7fe1af59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b0145c10 con 0x7fe1b00c6e10 2023-07-14T11:53:44.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.212+0000 7fe1aed9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1b00c5670 0x7fe1b01473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:44.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.212+0000 7fe1aed9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1b00c5670 0x7fe1b01473b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38136/0 (socket says 172.21.15.104:38136) 2023-07-14T11:53:44.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.212+0000 7fe1aed9d700 1 -- 172.21.15.104:0/2050753050 learned_addr learned my addr 172.21.15.104:0/2050753050 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:44.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.212+0000 7fe1ae59c700 1 --2- 172.21.15.104:0/2050753050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c005220 0x7fe1b01589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:44.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.212+0000 7fe1a7fff700 1 --2- 172.21.15.104:0/2050753050 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1b00c6e10 0x7fe1b00c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:44.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.216+0000 7fe1aed9d700 1 -- 172.21.15.104:0/2050753050 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1b00c6e10 msgr2=0x7fe1b00c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:44.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.216+0000 7fe1aed9d700 1 --2- 172.21.15.104:0/2050753050 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1b00c6e10 0x7fe1b00c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.216+0000 7fe1aed9d700 1 -- 172.21.15.104:0/2050753050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c005220 msgr2=0x7fe1b01589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:44.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.216+0000 7fe1aed9d700 1 --2- 172.21.15.104:0/2050753050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c005220 0x7fe1b01589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.216+0000 7fe1aed9d700 1 -- 172.21.15.104:0/2050753050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b0145ef0 con 0x7fe1b00c5670 2023-07-14T11:53:44.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.216+0000 7fe1aed9d700 1 --2- 172.21.15.104:0/2050753050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1b00c5670 0x7fe1b01473b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00ca870 tx=0x7fe1a8001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:44.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.216+0000 7fe1a6ffd700 1 -- 172.21.15.104:0/2050753050 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8024030 con 0x7fe1b00c5670 2023-07-14T11:53:44.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.216+0000 7fe1a6ffd700 1 -- 172.21.15.104:0/2050753050 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a8007870 con 0x7fe1b00c5670 2023-07-14T11:53:44.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.216+0000 7fe1a6ffd700 1 -- 172.21.15.104:0/2050753050 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8001250 con 0x7fe1b00c5670 2023-07-14T11:53:44.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.216+0000 7fe1af59e700 1 -- 172.21.15.104:0/2050753050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b01461d0 con 0x7fe1b00c5670 2023-07-14T11:53:44.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.216+0000 7fe1af59e700 1 -- 172.21.15.104:0/2050753050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b0158ee0 con 0x7fe1b00c5670 2023-07-14T11:53:44.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.216+0000 7fe1af59e700 1 -- 172.21.15.104:0/2050753050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b01463c0 con 0x7fe1b00c5670 2023-07-14T11:53:44.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.216+0000 7fe1a6ffd700 1 -- 172.21.15.104:0/2050753050 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe1a8027070 con 0x7fe1b00c5670 2023-07-14T11:53:44.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.220+0000 7fe1a6ffd700 1 --2- 172.21.15.104:0/2050753050 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe1900662f0 0x7fe1900687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:44.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.220+0000 7fe1a6ffd700 1 -- 172.21.15.104:0/2050753050 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe1a8025070 con 0x7fe1b00c5670 2023-07-14T11:53:44.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.220+0000 7fe1ae59c700 1 --2- 172.21.15.104:0/2050753050 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe1900662f0 0x7fe1900687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:44.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.220+0000 7fe1ae59c700 1 --2- 172.21.15.104:0/2050753050 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe1900662f0 0x7fe1900687a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0007fd0 tx=0x7fe1a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:44.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.224+0000 7fe1a6ffd700 1 -- 172.21.15.104:0/2050753050 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe1a80523b0 con 0x7fe1b00c5670 2023-07-14T11:53:44.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.396+0000 7fe1af59e700 1 -- 172.21.15.104:0/2050753050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1b004e500 con 0x7fe1b00c5670 2023-07-14T11:53:44.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.400+0000 7fe1a6ffd700 1 -- 172.21.15.104:0/2050753050 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20156 (secure 0 0 0) 0x7fe1a80559d0 con 0x7fe1b00c5670 2023-07-14T11:53:44.385 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:44.386 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":28,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:20.091434+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:44.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.404+0000 7fe1af59e700 1 -- 172.21.15.104:0/2050753050 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe1900662f0 msgr2=0x7fe1900687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:44.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.404+0000 7fe1af59e700 1 --2- 172.21.15.104:0/2050753050 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe1900662f0 0x7fe1900687a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0007fd0 tx=0x7fe1a0009040 comp rx=0 tx=0).stop 2023-07-14T11:53:44.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.404+0000 7fe1af59e700 1 -- 172.21.15.104:0/2050753050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1b00c5670 msgr2=0x7fe1b01473b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:44.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.404+0000 7fe1af59e700 1 --2- 172.21.15.104:0/2050753050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1b00c5670 0x7fe1b01473b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00ca870 tx=0x7fe1a8001d80 comp rx=0 tx=0).stop 2023-07-14T11:53:44.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.404+0000 7fe1af59e700 1 -- 172.21.15.104:0/2050753050 shutdown_connections 2023-07-14T11:53:44.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.404+0000 7fe1af59e700 1 --2- 172.21.15.104:0/2050753050 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe1900662f0 0x7fe1900687a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.404+0000 7fe1af59e700 1 --2- 172.21.15.104:0/2050753050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1b00c5670 0x7fe1b01473b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.404+0000 7fe1af59e700 1 --2- 172.21.15.104:0/2050753050 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1b00c6e10 0x7fe1b00c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.404+0000 7fe1af59e700 1 --2- 172.21.15.104:0/2050753050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c005220 0x7fe1b01589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.404+0000 7fe1af59e700 1 -- 172.21.15.104:0/2050753050 >> 172.21.15.104:0/2050753050 conn(0x7fe1b006e960 msgr2=0x7fe1b00b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:44.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.404+0000 7fe1af59e700 1 -- 172.21.15.104:0/2050753050 shutdown_connections 2023-07-14T11:53:44.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.404+0000 7fe1af59e700 1 -- 172.21.15.104:0/2050753050 wait complete. 2023-07-14T11:53:44.410 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-07-14T11:53:44.410 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:44.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.600+0000 7f3334850700 1 Processor -- start 2023-07-14T11:53:44.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.600+0000 7f3334850700 1 -- start start 2023-07-14T11:53:44.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.600+0000 7f3334850700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f333006dc20 0x7f33300cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:44.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.600+0000 7f3334850700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f33301596f0 0x7f33300ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:44.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.600+0000 7f3334850700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3330159d60 con 0x7f33300c9820 2023-07-14T11:53:44.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.600+0000 7f3334850700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33300cd250 con 0x7f333006dc20 2023-07-14T11:53:44.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.600+0000 7f3334850700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33300c8a60 con 0x7f33301596f0 2023-07-14T11:53:44.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.600+0000 7f332ed9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f333006dc20 0x7f33300cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:44.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.600+0000 7f332e59c700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f33300c9820 0x7f3330158fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42466/0 (socket says 172.21.15.104:42466) 2023-07-14T11:53:44.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.600+0000 7f332e59c700 1 -- 172.21.15.104:0/2541345941 learned_addr learned my addr 172.21.15.104:0/2541345941 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:44.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.600+0000 7f332ed9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f333006dc20 0x7f33300cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38152/0 (socket says 172.21.15.104:38152) 2023-07-14T11:53:44.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f332f59e700 1 --2- 172.21.15.104:0/2541345941 >> v2:172.21.15.121:3300/0 conn(0x7f33301596f0 0x7f33300ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:44.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f332ed9d700 1 -- 172.21.15.104:0/2541345941 >> v2:172.21.15.121:3300/0 conn(0x7f33301596f0 msgr2=0x7f33300ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:44.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f332ed9d700 1 --2- 172.21.15.104:0/2541345941 >> v2:172.21.15.121:3300/0 conn(0x7f33301596f0 0x7f33300ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f332dd9b700 1 -- 172.21.15.104:0/2541345941 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1072288809 0 0) 0x7f3330159d60 con 0x7f33300c9820 2023-07-14T11:53:44.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f332ed9d700 1 -- 172.21.15.104:0/2541345941 >> v1:172.21.15.104:6789/0 conn(0x7f33300c9820 legacy=0x7f3330158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:44.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f332ed9d700 1 -- 172.21.15.104:0/2541345941 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33300c8d40 con 0x7f333006dc20 2023-07-14T11:53:44.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f332f59e700 1 --2- 172.21.15.104:0/2541345941 >> v2:172.21.15.121:3300/0 conn(0x7f33301596f0 0x7f33300ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:44.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f332ed9d700 1 --2- 172.21.15.104:0/2541345941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f333006dc20 0x7f33300cded0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f333006ba10 tx=0x7f3320009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8761b62e22af1bb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:44.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f332dd9b700 1 -- 172.21.15.104:0/2541345941 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3320011830 con 0x7f333006dc20 2023-07-14T11:53:44.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f332dd9b700 1 -- 172.21.15.104:0/2541345941 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3320011990 con 0x7f333006dc20 2023-07-14T11:53:44.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f332dd9b700 1 -- 172.21.15.104:0/2541345941 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3320011d40 con 0x7f333006dc20 2023-07-14T11:53:44.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f3334850700 1 -- 172.21.15.104:0/2541345941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f333006dc20 msgr2=0x7f33300cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:44.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f3334850700 1 --2- 172.21.15.104:0/2541345941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f333006dc20 0x7f33300cded0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f333006ba10 tx=0x7f3320009b30 comp rx=0 tx=0).stop 2023-07-14T11:53:44.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f3334850700 1 -- 172.21.15.104:0/2541345941 shutdown_connections 2023-07-14T11:53:44.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f3334850700 1 --2- 172.21.15.104:0/2541345941 >> v2:172.21.15.121:3300/0 conn(0x7f33301596f0 0x7f33300ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f3334850700 1 --2- 172.21.15.104:0/2541345941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f333006dc20 0x7f33300cded0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f3334850700 1 -- 172.21.15.104:0/2541345941 >> 172.21.15.104:0/2541345941 conn(0x7f333006e960 msgr2=0x7f333006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:44.591 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f3334850700 1 -- 172.21.15.104:0/2541345941 shutdown_connections 2023-07-14T11:53:44.591 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.604+0000 7f3334850700 1 -- 172.21.15.104:0/2541345941 wait complete. 2023-07-14T11:53:44.591 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f3334850700 1 Processor -- start 2023-07-14T11:53:44.591 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f3334850700 1 -- start start 2023-07-14T11:53:44.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f3334850700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33300c9820 0x7f3330148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:44.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f3334850700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33301596f0 0x7f33300aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:44.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f3334850700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3330148480 0x7f33300b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:44.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f3334850700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33300c60a0 con 0x7f3330148480 2023-07-14T11:53:44.594 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f3334850700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33300c6380 con 0x7f33301596f0 2023-07-14T11:53:44.594 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f3334850700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33300c6660 con 0x7f33300c9820 2023-07-14T11:53:44.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f332f59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3330148480 0x7f33300b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:44.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f332f59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3330148480 0x7f33300b03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38158/0 (socket says 172.21.15.104:38158) 2023-07-14T11:53:44.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f332f59e700 1 -- 172.21.15.104:0/2200554346 learned_addr learned my addr 172.21.15.104:0/2200554346 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:44.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f332ed9d700 1 --2- 172.21.15.104:0/2200554346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33300c9820 0x7f3330148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:44.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f332e59c700 1 --2- 172.21.15.104:0/2200554346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33301596f0 0x7f33300aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:44.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f332f59e700 1 -- 172.21.15.104:0/2200554346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33300c9820 msgr2=0x7f3330148c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:44.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f332f59e700 1 --2- 172.21.15.104:0/2200554346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33300c9820 0x7f3330148c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f332f59e700 1 -- 172.21.15.104:0/2200554346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33301596f0 msgr2=0x7f33300aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:44.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f332f59e700 1 --2- 172.21.15.104:0/2200554346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33301596f0 0x7f33300aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f332f59e700 1 -- 172.21.15.104:0/2200554346 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33200094b0 con 0x7f3330148480 2023-07-14T11:53:44.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.608+0000 7f332f59e700 1 --2- 172.21.15.104:0/2200554346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3330148480 0x7f33300b03c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f332800cd50 tx=0x7f332800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:44.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.612+0000 7f33177fe700 1 -- 172.21.15.104:0/2200554346 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332800cea0 con 0x7f3330148480 2023-07-14T11:53:44.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.612+0000 7f33177fe700 1 -- 172.21.15.104:0/2200554346 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332800f480 con 0x7f3330148480 2023-07-14T11:53:44.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.612+0000 7f33177fe700 1 -- 172.21.15.104:0/2200554346 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332800f810 con 0x7f3330148480 2023-07-14T11:53:44.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.612+0000 7f3334850700 1 -- 172.21.15.104:0/2200554346 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33300c6940 con 0x7f3330148480 2023-07-14T11:53:44.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.612+0000 7f3334850700 1 -- 172.21.15.104:0/2200554346 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33300b0900 con 0x7f3330148480 2023-07-14T11:53:44.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.612+0000 7f3334850700 1 -- 172.21.15.104:0/2200554346 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33300c6e60 con 0x7f3330148480 2023-07-14T11:53:44.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.612+0000 7f33177fe700 1 -- 172.21.15.104:0/2200554346 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f332800b040 con 0x7f3330148480 2023-07-14T11:53:44.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.616+0000 7f33177fe700 1 --2- 172.21.15.104:0/2200554346 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f331c0662f0 0x7f331c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:44.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.616+0000 7f33177fe700 1 -- 172.21.15.104:0/2200554346 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3328084760 con 0x7f3330148480 2023-07-14T11:53:44.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.616+0000 7f332ed9d700 1 --2- 172.21.15.104:0/2200554346 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f331c0662f0 0x7f331c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:44.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.616+0000 7f332ed9d700 1 --2- 172.21.15.104:0/2200554346 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f331c0662f0 0x7f331c0687a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f333006b710 tx=0x7f3320008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:44.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.620+0000 7f33177fe700 1 -- 172.21.15.104:0/2200554346 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f33300c6e60 con 0x7f3330148480 2023-07-14T11:53:44.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.796+0000 7f3334850700 1 -- 172.21.15.104:0/2200554346 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33300c6e60 con 0x7f3330148480 2023-07-14T11:53:44.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.800+0000 7f33177fe700 1 -- 172.21.15.104:0/2200554346 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20156 (secure 0 0 0) 0x7f33300c6e60 con 0x7f3330148480 2023-07-14T11:53:44.783 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:44.783 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":28,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:20.091434+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:44.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.800+0000 7f3334850700 1 -- 172.21.15.104:0/2200554346 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f331c0662f0 msgr2=0x7f331c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:44.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.800+0000 7f3334850700 1 --2- 172.21.15.104:0/2200554346 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f331c0662f0 0x7f331c0687a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f333006b710 tx=0x7f3320008030 comp rx=0 tx=0).stop 2023-07-14T11:53:44.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.800+0000 7f3334850700 1 -- 172.21.15.104:0/2200554346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3330148480 msgr2=0x7f33300b03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:44.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.800+0000 7f3334850700 1 --2- 172.21.15.104:0/2200554346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3330148480 0x7f33300b03c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f332800cd50 tx=0x7f332800a560 comp rx=0 tx=0).stop 2023-07-14T11:53:44.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.800+0000 7f3334850700 1 -- 172.21.15.104:0/2200554346 shutdown_connections 2023-07-14T11:53:44.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.800+0000 7f3334850700 1 --2- 172.21.15.104:0/2200554346 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f331c0662f0 0x7f331c0687a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.800+0000 7f3334850700 1 --2- 172.21.15.104:0/2200554346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3330148480 0x7f33300b03c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.800+0000 7f3334850700 1 --2- 172.21.15.104:0/2200554346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33301596f0 0x7f33300aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.800+0000 7f3334850700 1 --2- 172.21.15.104:0/2200554346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33300c9820 0x7f3330148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:44.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.800+0000 7f3334850700 1 -- 172.21.15.104:0/2200554346 >> 172.21.15.104:0/2200554346 conn(0x7f333006e960 msgr2=0x7f333015b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:44.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.800+0000 7f3334850700 1 -- 172.21.15.104:0/2200554346 shutdown_connections 2023-07-14T11:53:44.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:44.800+0000 7f3334850700 1 -- 172.21.15.104:0/2200554346 wait complete. 2023-07-14T11:53:44.809 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-07-14T11:53:44.810 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-07-14T11:53:44.810 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-07-14T11:53:44.811 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-07-14T11:53:44.811 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-07-14T11:53:44.812 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-07-14T11:53:44.812 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-07-14T11:53:44.813 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-07-14T11:53:45.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.060+0000 7fa611c71700 1 Processor -- start 2023-07-14T11:53:45.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.060+0000 7fa611c71700 1 -- start start 2023-07-14T11:53:45.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.060+0000 7fa611c71700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa60c15f1c0 0x7fa60c0714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.060+0000 7fa611c71700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa60c0c86b0 0x7fa60c0cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.060+0000 7fa611c71700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60c0cbf30 con 0x7fa60c0c9ca0 2023-07-14T11:53:45.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.060+0000 7fa611c71700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60c0c8c00 con 0x7fa60c15f1c0 2023-07-14T11:53:45.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.060+0000 7fa611c71700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60c0c8ee0 con 0x7fa60c0c86b0 2023-07-14T11:53:45.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa60b7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa60c15f1c0 0x7fa60c0714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa60b7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa60c15f1c0 0x7fa60c0714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38170/0 (socket says 172.21.15.104:38170) 2023-07-14T11:53:45.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa60b7fe700 1 -- 172.21.15.104:0/2864457592 learned_addr learned my addr 172.21.15.104:0/2864457592 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa60affd700 1 --2- 172.21.15.104:0/2864457592 >> v2:172.21.15.121:3300/0 conn(0x7fa60c0c86b0 0x7fa60c0cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa60b7fe700 1 -- 172.21.15.104:0/2864457592 >> v2:172.21.15.121:3300/0 conn(0x7fa60c0c86b0 msgr2=0x7fa60c0cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa60b7fe700 1 --2- 172.21.15.104:0/2864457592 >> v2:172.21.15.121:3300/0 conn(0x7fa60c0c86b0 0x7fa60c0cdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa60b7fe700 1 -- 172.21.15.104:0/2864457592 >> v1:172.21.15.121:6789/0 conn(0x7fa60c0c9ca0 legacy=0x7fa60c0ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa60b7fe700 1 -- 172.21.15.104:0/2864457592 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c0c91c0 con 0x7fa60c15f1c0 2023-07-14T11:53:45.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fe67d186700 1 Processor -- start 2023-07-14T11:53:45.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.060+0000 7fd684546700 1 Processor -- start 2023-07-14T11:53:45.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.060+0000 7fd684546700 1 -- start start 2023-07-14T11:53:45.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa60b7fe700 1 --2- 172.21.15.104:0/2864457592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa60c15f1c0 0x7fa60c0714a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0713c0 tx=0x7fa60400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05846130563b2c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd684546700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fd67c15fd30 0x7fd67c160100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd684546700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd67c0cdf70 0x7fd67c0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd684546700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67c0ce8e0 con 0x7fd67c0cad80 2023-07-14T11:53:45.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd684546700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67c0cb8f0 con 0x7fd67c15fd30 2023-07-14T11:53:45.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd684546700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67c0cbd40 con 0x7fd67c0cdf70 2023-07-14T11:53:45.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd682ae3700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fd67c0cad80 0x7fd67c0cb1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42470/0 (socket says 172.21.15.104:42470) 2023-07-14T11:53:45.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd682ae3700 1 -- 172.21.15.104:0/879168013 learned_addr learned my addr 172.21.15.104:0/879168013 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:45.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd681ae1700 1 --2- 172.21.15.104:0/879168013 >> v2:172.21.15.121:3300/0 conn(0x7fd67c0cdf70 0x7fd67c0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd681ae1700 1 -- 172.21.15.104:0/879168013 >> v2:172.21.15.104:3300/0 conn(0x7fd67c15fd30 msgr2=0x7fd67c160100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:53:45.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd6812e0700 1 -- 172.21.15.104:0/879168013 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180188750 0 0) 0x7fd67c0ce8e0 con 0x7fd67c0cad80 2023-07-14T11:53:45.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa60a7fc700 1 -- 172.21.15.104:0/2864457592 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60400e070 con 0x7fa60c15f1c0 2023-07-14T11:53:45.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd6822e2700 1 --2- 172.21.15.104:0/879168013 >> v2:172.21.15.104:3300/0 conn(0x7fd67c15fd30 0x7fd67c160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa60a7fc700 1 -- 172.21.15.104:0/2864457592 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa604003a40 con 0x7fa60c15f1c0 2023-07-14T11:53:45.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa60a7fc700 1 -- 172.21.15.104:0/2864457592 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604003df0 con 0x7fa60c15f1c0 2023-07-14T11:53:45.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd681ae1700 1 --2- 172.21.15.104:0/879168013 >> v2:172.21.15.104:3300/0 conn(0x7fd67c15fd30 0x7fd67c160100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd681ae1700 1 -- 172.21.15.104:0/879168013 >> v1:172.21.15.104:6789/0 conn(0x7fd67c0cad80 legacy=0x7fd67c0cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd681ae1700 1 -- 172.21.15.104:0/879168013 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67c0cc020 con 0x7fd67c0cdf70 2023-07-14T11:53:45.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd6822e2700 1 --2- 172.21.15.104:0/879168013 >> v2:172.21.15.104:3300/0 conn(0x7fd67c15fd30 0x7fd67c160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:45.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa611c71700 1 -- 172.21.15.104:0/2864457592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa60c15f1c0 msgr2=0x7fa60c0714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa611c71700 1 --2- 172.21.15.104:0/2864457592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa60c15f1c0 0x7fa60c0714a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0713c0 tx=0x7fa60400b330 comp rx=0 tx=0).stop 2023-07-14T11:53:45.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fe67d186700 1 -- start start 2023-07-14T11:53:45.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fd681ae1700 1 --2- 172.21.15.104:0/879168013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd67c0cdf70 0x7fd67c0ce3a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0b03f0 tx=0x7fd678009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cf3d5a004cefc60 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fe67d186700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6780afbf0 0x7fe67806bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fe67d186700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6780cdd50 con 0x7fe6780caa70 2023-07-14T11:53:45.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fe67d186700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6780cda20 con 0x7fe6780c86b0 2023-07-14T11:53:45.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fe67d186700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6780ca440 con 0x7fe6780afbf0 2023-07-14T11:53:45.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa611c71700 1 -- 172.21.15.104:0/2864457592 shutdown_connections 2023-07-14T11:53:45.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa611c71700 1 --2- 172.21.15.104:0/2864457592 >> v2:172.21.15.121:3300/0 conn(0x7fa60c0c86b0 0x7fa60c0cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa611c71700 1 --2- 172.21.15.104:0/2864457592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa60c15f1c0 0x7fa60c0714a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fa611c71700 1 -- 172.21.15.104:0/2864457592 >> 172.21.15.104:0/2864457592 conn(0x7fa60c060320 msgr2=0x7fa60c0606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fe676d9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fe6780caa70 0x7fe678158c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42474/0 (socket says 172.21.15.104:42474) 2023-07-14T11:53:45.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.064+0000 7fe676d9d700 1 -- 172.21.15.104:0/2881028065 learned_addr learned my addr 172.21.15.104:0/2881028065 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:45.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fd6812e0700 1 -- 172.21.15.104:0/879168013 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678011830 con 0x7fd67c0cdf70 2023-07-14T11:53:45.062 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fa611c71700 1 -- 172.21.15.104:0/2864457592 shutdown_connections 2023-07-14T11:53:45.062 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fa611c71700 1 -- 172.21.15.104:0/2864457592 wait complete. 2023-07-14T11:53:45.063 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe675d9b700 1 -- 172.21.15.104:0/2881028065 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4159030982 0 0) 0x7fe6780cdd50 con 0x7fe6780caa70 2023-07-14T11:53:45.063 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe675d9b700 1 -- 172.21.15.104:0/2881028065 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6600035d0 con 0x7fe6780caa70 2023-07-14T11:53:45.063 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe67759e700 1 --2- 172.21.15.104:0/2881028065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6780afbf0 0x7fe67806bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.063 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe675d9b700 1 -- 172.21.15.104:0/2881028065 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 280387600 0 0) 0x7fe6600035d0 con 0x7fe6780caa70 2023-07-14T11:53:45.064 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fd6812e0700 1 -- 172.21.15.104:0/879168013 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd678011990 con 0x7fd67c0cdf70 2023-07-14T11:53:45.064 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fd6812e0700 1 -- 172.21.15.104:0/879168013 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678011d40 con 0x7fd67c0cdf70 2023-07-14T11:53:45.065 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fd684546700 1 -- 172.21.15.104:0/879168013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd67c0cdf70 msgr2=0x7fd67c0ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.065 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fd684546700 1 --2- 172.21.15.104:0/879168013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd67c0cdf70 0x7fd67c0ce3a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0b03f0 tx=0x7fd678009b30 comp rx=0 tx=0).stop 2023-07-14T11:53:45.066 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe675d9b700 1 -- 172.21.15.104:0/2881028065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6780afbf0 msgr2=0x7fe67806bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.066 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe675d9b700 1 --2- 172.21.15.104:0/2881028065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6780afbf0 0x7fe67806bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.066 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe675d9b700 1 -- 172.21.15.104:0/2881028065 >> v1:172.21.15.121:6789/0 conn(0x7fe6780c86b0 legacy=0x7fe6780cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.066 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe675d9b700 1 -- 172.21.15.104:0/2881028065 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6780ca720 con 0x7fe6780caa70 2023-07-14T11:53:45.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe675d9b700 1 -- 172.21.15.104:0/2881028065 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7fe668001670 con 0x7fe6780caa70 2023-07-14T11:53:45.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fd684546700 1 -- 172.21.15.104:0/879168013 shutdown_connections 2023-07-14T11:53:45.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fd684546700 1 --2- 172.21.15.104:0/879168013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd67c0cdf70 0x7fd67c0ce3a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fd684546700 1 --2- 172.21.15.104:0/879168013 >> v2:172.21.15.104:3300/0 conn(0x7fd67c15fd30 0x7fd67c160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fd684546700 1 -- 172.21.15.104:0/879168013 >> 172.21.15.104:0/879168013 conn(0x7fd67c060320 msgr2=0x7fd67c0606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe675d9b700 1 -- 172.21.15.104:0/2881028065 >> v1:172.21.15.104:6789/0 conn(0x7fe6780caa70 legacy=0x7fe678158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.069 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe675d9b700 1 --2- 172.21.15.104:0/2881028065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe660003a50 0x7fe660003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.069 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe675d9b700 1 --2- 172.21.15.104:0/2881028065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6780afbf0 0x7fe660004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.069 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe675d9b700 1 --2- 172.21.15.104:0/2881028065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe660004db0 0x7fe660005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.069 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe675d9b700 1 -- 172.21.15.104:0/2881028065 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6780ca440 con 0x7fe660003a50 2023-07-14T11:53:45.069 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe675d9b700 1 -- 172.21.15.104:0/2881028065 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe660005c30 con 0x7fe660004db0 2023-07-14T11:53:45.069 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe675d9b700 1 -- 172.21.15.104:0/2881028065 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe660005df0 con 0x7fe6780afbf0 2023-07-14T11:53:45.070 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe676d9d700 1 --2- 172.21.15.104:0/2881028065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe660003a50 0x7fe660003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.070 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe67659c700 1 --2- 172.21.15.104:0/2881028065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6780afbf0 0x7fe660004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.070 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fd684546700 1 -- 172.21.15.104:0/879168013 shutdown_connections 2023-07-14T11:53:45.070 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fd684546700 1 -- 172.21.15.104:0/879168013 wait complete. 2023-07-14T11:53:45.071 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fa611c71700 1 Processor -- start 2023-07-14T11:53:45.071 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe67759e700 1 --2- 172.21.15.104:0/2881028065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe660004db0 0x7fe660005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.072 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe67659c700 1 -- 172.21.15.104:0/2881028065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe660004db0 msgr2=0x7fe660005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.072 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe67659c700 1 --2- 172.21.15.104:0/2881028065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe660004db0 0x7fe660005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.072 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe67659c700 1 -- 172.21.15.104:0/2881028065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe660003a50 msgr2=0x7fe660003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.072 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe67659c700 1 --2- 172.21.15.104:0/2881028065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe660003a50 0x7fe660003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe67659c700 1 -- 172.21.15.104:0/2881028065 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6600035d0 con 0x7fe6780afbf0 2023-07-14T11:53:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fe67659c700 1 --2- 172.21.15.104:0/2881028065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6780afbf0 0x7fe660004870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0017e0 tx=0x7fe6600058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecfbe07964b0d4d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fe67759e700 1 --2- 172.21.15.104:0/2881028065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe660004db0 0x7fe660005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fe675d9b700 1 -- 172.21.15.104:0/2881028065 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c0052d0 con 0x7fe6780afbf0 2023-07-14T11:53:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fe675d9b700 1 -- 172.21.15.104:0/2881028065 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66c005430 con 0x7fe6780afbf0 2023-07-14T11:53:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fe675d9b700 1 -- 172.21.15.104:0/2881028065 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c0055a0 con 0x7fe6780afbf0 2023-07-14T11:53:45.074 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.068+0000 7fa611c71700 1 -- start start 2023-07-14T11:53:45.074 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa611c71700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa60c0c86b0 0x7fa60c155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.074 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7f7c3759e700 1 Processor -- start 2023-07-14T11:53:45.075 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa611c71700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa60c0c9ca0 0x7fa60c0b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.075 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fd684546700 1 Processor -- start 2023-07-14T11:53:45.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa611c71700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa60c15f1c0 0x7fa60c159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fe67d186700 1 -- 172.21.15.104:0/2881028065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6780afbf0 msgr2=0x7fe660004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fe67d186700 1 --2- 172.21.15.104:0/2881028065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6780afbf0 0x7fe660004870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0017e0 tx=0x7fe6600058a0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fe67759e700 1 -- 172.21.15.104:0/2881028065 reap_dead start 2023-07-14T11:53:45.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fe67d186700 1 -- 172.21.15.104:0/2881028065 shutdown_connections 2023-07-14T11:53:45.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fe67d186700 1 -- 172.21.15.104:0/2881028065 >> 172.21.15.104:0/2881028065 conn(0x7fe678060320 msgr2=0x7fe6780606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa611c71700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c156770 con 0x7fa60c0c86b0 2023-07-14T11:53:45.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa60b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa60c0c86b0 0x7fa60c155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa611c71700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c0b16e0 con 0x7fa60c0c9ca0 2023-07-14T11:53:45.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa611c71700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c0b3fc0 con 0x7fa60c15f1c0 2023-07-14T11:53:45.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa60affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa60c0c9ca0 0x7fa60c0b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa60b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa60c0c86b0 0x7fa60c155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38198/0 (socket says 172.21.15.104:38198) 2023-07-14T11:53:45.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa60affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa60c0c9ca0 0x7fa60c0b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57178/0 (socket says 172.21.15.104:57178) 2023-07-14T11:53:45.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa60b7fe700 1 -- 172.21.15.104:0/3957996206 learned_addr learned my addr 172.21.15.104:0/3957996206 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fe67d186700 1 -- 172.21.15.104:0/2881028065 shutdown_connections 2023-07-14T11:53:45.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fe67d186700 1 -- 172.21.15.104:0/2881028065 wait complete. 2023-07-14T11:53:45.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fd684546700 1 -- start start 2023-07-14T11:53:45.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa60bfff700 1 --2- 172.21.15.104:0/3957996206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa60c15f1c0 0x7fa60c159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7f7c3759e700 1 -- start start 2023-07-14T11:53:45.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fd684546700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd67c0cad80 0x7fd67c14aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fd684546700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c0cdf70 0x7fd67c152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa60b7fe700 1 -- 172.21.15.104:0/3957996206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa60c15f1c0 msgr2=0x7fa60c159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa60b7fe700 1 --2- 172.21.15.104:0/3957996206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa60c15f1c0 0x7fa60c159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa60b7fe700 1 -- 172.21.15.104:0/3957996206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa60c0c9ca0 msgr2=0x7fa60c0b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa60b7fe700 1 --2- 172.21.15.104:0/3957996206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa60c0c9ca0 0x7fa60c0b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fa60b7fe700 1 -- 172.21.15.104:0/3957996206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa604009b30 con 0x7fa60c0c86b0 2023-07-14T11:53:45.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7f7c3759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c380caa70 0x7f7c380c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fd6822e2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd67c0cad80 0x7fd67c14aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fd6822e2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd67c0cad80 0x7fd67c14aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57186/0 (socket says 172.21.15.104:57186) 2023-07-14T11:53:45.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7fd6822e2700 1 -- 172.21.15.104:0/2251453549 learned_addr learned my addr 172.21.15.104:0/2251453549 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7f7c3759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c38158c70 0x7f7c380cae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fa60b7fe700 1 --2- 172.21.15.104:0/3957996206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa60c0c86b0 0x7fa60c155e00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa60400b730 tx=0x7fa604004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7f7c3759e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7c380ce1e0 0x7f7c3806bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fd681ae1700 1 --2- 172.21.15.104:0/2251453549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c0cdf70 0x7fd67c152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fd684546700 1 --2- 172.21.15.104:0/2251453549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd67c15fd30 0x7fd67c14d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f7c3659c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c380caa70 0x7f7c380c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f7c35d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c38158c70 0x7f7c380cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f7c35d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c38158c70 0x7f7c380cae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57196/0 (socket says 172.21.15.104:57196) 2023-07-14T11:53:45.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f7c3659c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c380caa70 0x7f7c380c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:51466/0 (socket says 172.21.15.104:51466) 2023-07-14T11:53:45.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fa5f3fff700 1 -- 172.21.15.104:0/3957996206 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604004440 con 0x7fa60c0c86b0 2023-07-14T11:53:45.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fa5f3fff700 1 -- 172.21.15.104:0/3957996206 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6040045a0 con 0x7fa60c0c86b0 2023-07-14T11:53:45.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f7c35d9b700 1 -- 172.21.15.104:0/2522309715 learned_addr learned my addr 172.21.15.104:0/2522309715 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fd682ae3700 1 --2- 172.21.15.104:0/2251453549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd67c15fd30 0x7fd67c14d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f7c36d9d700 1 --2- 172.21.15.104:0/2522309715 >> v2:172.21.15.104:3300/0 conn(0x7f7c380ce1e0 0x7f7c3806bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f7c3759e700 1 -- 172.21.15.104:0/2522309715 --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c380cdd50 con 0x7f7c380ce1e0 2023-07-14T11:53:45.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fd684546700 1 -- 172.21.15.104:0/2251453549 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c1500b0 con 0x7fd67c15fd30 2023-07-14T11:53:45.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fd684546700 1 -- 172.21.15.104:0/2251453549 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c0b30c0 con 0x7fd67c0cad80 2023-07-14T11:53:45.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fd684546700 1 -- 172.21.15.104:0/2251453549 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c14afc0 con 0x7fd67c0cdf70 2023-07-14T11:53:45.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f7c3759e700 1 -- 172.21.15.104:0/2522309715 --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c380cda20 con 0x7f7c38158c70 2023-07-14T11:53:45.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f7c3759e700 1 -- 172.21.15.104:0/2522309715 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c380cb380 con 0x7f7c380caa70 2023-07-14T11:53:45.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fe67d186700 1 Processor -- start 2023-07-14T11:53:45.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fa5f3fff700 1 -- 172.21.15.104:0/3957996206 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604004950 con 0x7fa60c0c86b0 2023-07-14T11:53:45.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fe67d186700 1 -- start start 2023-07-14T11:53:45.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fc92b651700 1 Processor -- start 2023-07-14T11:53:45.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f87fdf08700 1 Processor -- start 2023-07-14T11:53:45.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fd681ae1700 1 -- 172.21.15.104:0/2251453549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd67c0cad80 msgr2=0x7fd67c14aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fd681ae1700 1 --2- 172.21.15.104:0/2251453549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd67c0cad80 0x7fd67c14aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fd681ae1700 1 -- 172.21.15.104:0/2251453549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd67c15fd30 msgr2=0x7fd67c14d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fd681ae1700 1 --2- 172.21.15.104:0/2251453549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd67c15fd30 0x7fd67c14d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fd681ae1700 1 -- 172.21.15.104:0/2251453549 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6780094b0 con 0x7fd67c0cdf70 2023-07-14T11:53:45.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f7c36d9d700 1 -- 172.21.15.104:0/2522309715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c380caa70 msgr2=0x7f7c380c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f7c36d9d700 1 --2- 172.21.15.104:0/2522309715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c380caa70 0x7f7c380c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f7c36d9d700 1 -- 172.21.15.104:0/2522309715 >> v2:172.21.15.121:3300/0 conn(0x7f7c38158c70 msgr2=0x7f7c380cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f7c36d9d700 1 --2- 172.21.15.104:0/2522309715 >> v2:172.21.15.121:3300/0 conn(0x7f7c38158c70 0x7f7c380cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fa611c71700 1 -- 172.21.15.104:0/3957996206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa60c159b70 con 0x7fa60c0c86b0 2023-07-14T11:53:45.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f7c36d9d700 1 -- 172.21.15.104:0/2522309715 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c380ca440 con 0x7f7c380ce1e0 2023-07-14T11:53:45.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fa611c71700 1 -- 172.21.15.104:0/3957996206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60c0b1090 con 0x7fa60c0c86b0 2023-07-14T11:53:45.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fa611c71700 1 -- 172.21.15.104:0/3957996206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa60c0c9780 con 0x7fa60c0c86b0 2023-07-14T11:53:45.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f87fdf08700 1 -- start start 2023-07-14T11:53:45.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe67d186700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6780afbf0 0x7fe6781546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe67d186700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6780c5130 0x7fe6781592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fd681ae1700 1 --2- 172.21.15.104:0/2251453549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c0cdf70 0x7fd67c152a00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0cf120 tx=0x7fd678004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f7c36d9d700 1 --2- 172.21.15.104:0/2522309715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c380ce1e0 0x7f7c3806bc60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7c380cbba0 tx=0x7f7c3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790d89bf6b689445 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe676d9d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6780afbf0 0x7fe6781546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe676d9d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6780afbf0 0x7fe6781546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:51488/0 (socket says 172.21.15.104:51488) 2023-07-14T11:53:45.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe676d9d700 1 -- 172.21.15.104:0/4237368131 learned_addr learned my addr 172.21.15.104:0/4237368131 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe67d186700 1 --2- 172.21.15.104:0/4237368131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6781597f0 0x7fe67806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe67659c700 1 --2- 172.21.15.104:0/4237368131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6780c5130 0x7fe6781592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe67759e700 1 --2- 172.21.15.104:0/4237368131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6781597f0 0x7fe67806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe67d186700 1 -- 172.21.15.104:0/4237368131 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe678159c70 con 0x7fe6781597f0 2023-07-14T11:53:45.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe67d186700 1 -- 172.21.15.104:0/4237368131 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67815abf0 con 0x7fe6780c5130 2023-07-14T11:53:45.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe67d186700 1 -- 172.21.15.104:0/4237368131 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6780c2b30 con 0x7fe6780afbf0 2023-07-14T11:53:45.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fd672ffd700 1 -- 172.21.15.104:0/2251453549 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67801aa10 con 0x7fd67c0cdf70 2023-07-14T11:53:45.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fd672ffd700 1 -- 172.21.15.104:0/2251453549 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67801f070 con 0x7fd67c0cdf70 2023-07-14T11:53:45.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fd672ffd700 1 -- 172.21.15.104:0/2251453549 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67801adb0 con 0x7fd67c0cdf70 2023-07-14T11:53:45.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f87fdf08700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87f0037b40 0x7f87f00e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f87fdf08700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f87f00e4f50 0x7f87f0036770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f87fdf08700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f00e0af0 con 0x7f87f00e12a0 2023-07-14T11:53:45.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f87fdf08700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f00e45f0 con 0x7f87f00e4f50 2023-07-14T11:53:45.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f87fdf08700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0050f00 con 0x7f87f0037b40 2023-07-14T11:53:45.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fa5f3fff700 1 -- 172.21.15.104:0/3957996206 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa604019030 con 0x7fa60c0c86b0 2023-07-14T11:53:45.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe67759e700 1 -- 172.21.15.104:0/4237368131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6780afbf0 msgr2=0x7fe6781546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe67759e700 1 --2- 172.21.15.104:0/4237368131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6780afbf0 0x7fe6781546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe67759e700 1 -- 172.21.15.104:0/4237368131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6780c5130 msgr2=0x7fe6781592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe67759e700 1 --2- 172.21.15.104:0/4237368131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6780c5130 0x7fe6781592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe67759e700 1 -- 172.21.15.104:0/4237368131 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe678159e70 con 0x7fe6781597f0 2023-07-14T11:53:45.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f7c3559a700 1 -- 172.21.15.104:0/2522309715 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3000e070 con 0x7f7c380ce1e0 2023-07-14T11:53:45.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f7c3559a700 1 -- 172.21.15.104:0/2522309715 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3000f250 con 0x7f7c380ce1e0 2023-07-14T11:53:45.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7fc92b651700 1 -- start start 2023-07-14T11:53:45.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fc92b651700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc92415ffa0 0x7fc92406d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fc92b651700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9240ca8a0 con 0x7fc9240cb180 2023-07-14T11:53:45.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fc92b651700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9240cbcd0 con 0x7fc92406dc50 2023-07-14T11:53:45.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fc92b651700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9240cbf70 con 0x7fc92415ffa0 2023-07-14T11:53:45.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7f85fccc6700 1 Processor -- start 2023-07-14T11:53:45.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.072+0000 7f85fccc6700 1 -- start start 2023-07-14T11:53:45.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f85fccc6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f85f00de730 0x7f85f0036630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f85f659c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f85f00de730 0x7f85f0036630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f85f659c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f85f00de730 0x7f85f0036630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38218/0 (socket says 172.21.15.104:38218) 2023-07-14T11:53:45.110 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fd684546700 1 -- 172.21.15.104:0/2251453549 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd67c0b35f0 con 0x7fd67c0cdf70 2023-07-14T11:53:45.110 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fd684546700 1 -- 172.21.15.104:0/2251453549 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67c151ae0 con 0x7fd67c0cdf70 2023-07-14T11:53:45.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f87f77fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f87f00e12a0 0x7f87f00e1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:56546/0 (socket says 172.21.15.104:56546) 2023-07-14T11:53:45.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f87f77fe700 1 -- 172.21.15.104:0/765736916 learned_addr learned my addr 172.21.15.104:0/765736916 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:45.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fc929bee700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fc9240cb180 0x7fc9240ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42484/0 (socket says 172.21.15.104:42484) 2023-07-14T11:53:45.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fc928bec700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc92406dc50 0x7fc9240caa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:56552/0 (socket says 172.21.15.104:56552) 2023-07-14T11:53:45.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fc929bee700 1 -- 172.21.15.104:0/3034103285 learned_addr learned my addr 172.21.15.104:0/3034103285 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:45.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fc9293ed700 1 --2- 172.21.15.104:0/3034103285 >> v2:172.21.15.121:3300/0 conn(0x7fc92415ffa0 0x7fc92406d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f7c3559a700 1 -- 172.21.15.104:0/2522309715 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3000bd40 con 0x7f7c380ce1e0 2023-07-14T11:53:45.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f85f659c700 1 -- 172.21.15.104:0/1516181377 learned_addr learned my addr 172.21.15.104:0/1516181377 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f85fccc6700 1 --2- 172.21.15.104:0/1516181377 >> v2:172.21.15.121:3300/0 conn(0x7f85f00deb00 0x7f85f0036b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f85fccc6700 1 -- 172.21.15.104:0/1516181377 --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f0034460 con 0x7f85f004f200 2023-07-14T11:53:45.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f85fccc6700 1 -- 172.21.15.104:0/1516181377 --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f00e3300 con 0x7f85f00de730 2023-07-14T11:53:45.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f85fccc6700 1 -- 172.21.15.104:0/1516181377 --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f00df090 con 0x7f85f00deb00 2023-07-14T11:53:45.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f87f6ffd700 1 --2- 172.21.15.104:0/765736916 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87f0037b40 0x7f87f00e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f85f5d9b700 1 --2- 172.21.15.104:0/1516181377 >> v2:172.21.15.121:3300/0 conn(0x7f85f00deb00 0x7f85f0036b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f85f5d9b700 1 -- 172.21.15.104:0/1516181377 >> v2:172.21.15.104:3300/0 conn(0x7f85f00de730 msgr2=0x7f85f0036630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f85f5d9b700 1 --2- 172.21.15.104:0/1516181377 >> v2:172.21.15.104:3300/0 conn(0x7f85f00de730 0x7f85f0036630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f85f5d9b700 1 -- 172.21.15.104:0/1516181377 >> v1:172.21.15.104:6789/0 conn(0x7f85f004f200 legacy=0x7f85f004f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.076+0000 7f85f5d9b700 1 -- 172.21.15.104:0/1516181377 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f00df370 con 0x7f85f00deb00 2023-07-14T11:53:45.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f85f5d9b700 1 --2- 172.21.15.104:0/1516181377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85f00deb00 0x7f85f0036b70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f85f0054b80 tx=0x7f85e8009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26a0396d27b40b6d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f85f559a700 1 -- 172.21.15.104:0/1516181377 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8004230 con 0x7f85f00deb00 2023-07-14T11:53:45.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f85f559a700 1 -- 172.21.15.104:0/1516181377 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e8004390 con 0x7f85f00deb00 2023-07-14T11:53:45.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f85f559a700 1 -- 172.21.15.104:0/1516181377 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e80036c0 con 0x7f85f00deb00 2023-07-14T11:53:45.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f85fccc6700 1 -- 172.21.15.104:0/1516181377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85f00deb00 msgr2=0x7f85f0036b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f85fccc6700 1 --2- 172.21.15.104:0/1516181377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85f00deb00 0x7f85f0036b70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f85f0054b80 tx=0x7f85e8009b30 comp rx=0 tx=0).stop 2023-07-14T11:53:45.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f87f7fff700 1 --2- 172.21.15.104:0/765736916 >> v2:172.21.15.121:3300/0 conn(0x7f87f00e4f50 0x7f87f0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f7c3759e700 1 -- 172.21.15.104:0/2522309715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c380ce1e0 msgr2=0x7f7c3806bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f7c3759e700 1 --2- 172.21.15.104:0/2522309715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c380ce1e0 0x7f7c3806bc60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7c380cbba0 tx=0x7f7c3000b330 comp rx=0 tx=0).stop 2023-07-14T11:53:45.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fe67759e700 1 --2- 172.21.15.104:0/4237368131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6781597f0 0x7fe67806c870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe6780cbba0 tx=0x7fe670004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f7c3759e700 1 -- 172.21.15.104:0/2522309715 shutdown_connections 2023-07-14T11:53:45.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f7c3759e700 1 --2- 172.21.15.104:0/2522309715 >> v2:172.21.15.121:3300/0 conn(0x7f7c38158c70 0x7f7c380cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f7c3759e700 1 --2- 172.21.15.104:0/2522309715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c380ce1e0 0x7f7c3806bc60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f7c3759e700 1 --2- 172.21.15.104:0/2522309715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c380caa70 0x7f7c380c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f7c3759e700 1 -- 172.21.15.104:0/2522309715 >> 172.21.15.104:0/2522309715 conn(0x7f7c38060320 msgr2=0x7f7c380606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f85fccc6700 1 -- 172.21.15.104:0/1516181377 shutdown_connections 2023-07-14T11:53:45.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f85fccc6700 1 --2- 172.21.15.104:0/1516181377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85f00deb00 0x7f85f0036b70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f85fccc6700 1 --2- 172.21.15.104:0/1516181377 >> v2:172.21.15.104:3300/0 conn(0x7f85f00de730 0x7f85f0036630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f85fccc6700 1 -- 172.21.15.104:0/1516181377 >> 172.21.15.104:0/1516181377 conn(0x7f85f0051630 msgr2=0x7f85f0051a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f85fccc6700 1 -- 172.21.15.104:0/1516181377 shutdown_connections 2023-07-14T11:53:45.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f85fccc6700 1 -- 172.21.15.104:0/1516181377 wait complete. 2023-07-14T11:53:45.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f87f67fc700 1 -- 172.21.15.104:0/765736916 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052649678 0 0) 0x7f87f8073710 con 0x7f87f00e12a0 2023-07-14T11:53:45.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7f87f67fc700 1 -- 172.21.15.104:0/765736916 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87e80035d0 con 0x7f87f00e12a0 2023-07-14T11:53:45.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fc91bfff700 1 -- 172.21.15.104:0/3034103285 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079590876 0 0) 0x7fc9240cbcd0 con 0x7fc92406dc50 2023-07-14T11:53:45.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.080+0000 7fc91bfff700 1 -- 172.21.15.104:0/3034103285 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9100035d0 con 0x7fc92406dc50 2023-07-14T11:53:45.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f7c3759e700 1 -- 172.21.15.104:0/2522309715 shutdown_connections 2023-07-14T11:53:45.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fc91bfff700 1 -- 172.21.15.104:0/3034103285 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030031277 0 0) 0x7fc91c001490 con 0x7fc9240cb180 2023-07-14T11:53:45.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fc91bfff700 1 -- 172.21.15.104:0/3034103285 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9240cbcd0 con 0x7fc9240cb180 2023-07-14T11:53:45.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f67fc700 1 -- 172.21.15.104:0/765736916 <== mon.1 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1551631777 0 0) 0x7f87e80035d0 con 0x7f87f00e12a0 2023-07-14T11:53:45.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f67fc700 1 -- 172.21.15.104:0/765736916 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87f0037b40 msgr2=0x7f87f00e4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f67fc700 1 --2- 172.21.15.104:0/765736916 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87f0037b40 0x7f87f00e4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f67fc700 1 -- 172.21.15.104:0/765736916 >> v2:172.21.15.121:3300/0 conn(0x7f87f00e4f50 msgr2=0x7f87f0036770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f67fc700 1 --2- 172.21.15.104:0/765736916 >> v2:172.21.15.121:3300/0 conn(0x7f87f00e4f50 0x7f87f0036770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f67fc700 1 -- 172.21.15.104:0/765736916 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f0051150 con 0x7f87f00e12a0 2023-07-14T11:53:45.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f67fc700 1 -- 172.21.15.104:0/765736916 <== mon.1 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f87f804e500 con 0x7f87f00e12a0 2023-07-14T11:53:45.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f7fff700 1 --2- 172.21.15.104:0/765736916 >> v2:172.21.15.121:3300/0 conn(0x7f87f00e4f50 0x7f87f0036770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:45.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f67fc700 1 -- 172.21.15.104:0/765736916 >> v1:172.21.15.121:6789/0 conn(0x7f87f00e12a0 legacy=0x7f87f00e1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f67fc700 1 --2- 172.21.15.104:0/765736916 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e8003a20 0x7f87e8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f67fc700 1 --2- 172.21.15.104:0/765736916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87e80043b0 0x7f87e8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f67fc700 1 --2- 172.21.15.104:0/765736916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87e8004ee0 0x7f87e800ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f67fc700 1 -- 172.21.15.104:0/765736916 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f00e45f0 con 0x7f87e8004ee0 2023-07-14T11:53:45.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fc91bfff700 1 -- 172.21.15.104:0/3034103285 <== mon.1 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4105111311 0 0) 0x7fc9100035d0 con 0x7fc92406dc50 2023-07-14T11:53:45.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fc9293ed700 1 -- 172.21.15.104:0/3034103285 >> v1:172.21.15.121:6789/0 conn(0x7fc92406dc50 legacy=0x7fc9240caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fc9293ed700 1 -- 172.21.15.104:0/3034103285 >> v1:172.21.15.104:6789/0 conn(0x7fc9240cb180 legacy=0x7fc9240ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fc9293ed700 1 -- 172.21.15.104:0/3034103285 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9240cc250 con 0x7fc92415ffa0 2023-07-14T11:53:45.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fa5f3fff700 1 --2- 172.21.15.104:0/3957996206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa5f80663c0 0x7fa5f8068870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f67fc700 1 -- 172.21.15.104:0/765736916 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0050f00 con 0x7f87e80043b0 2023-07-14T11:53:45.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f67fc700 1 -- 172.21.15.104:0/765736916 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e800bf60 con 0x7f87e8003a20 2023-07-14T11:53:45.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fa5f3fff700 1 -- 172.21.15.104:0/3957996206 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa6040851e0 con 0x7fa60c0c86b0 2023-07-14T11:53:45.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f7fff700 1 --2- 172.21.15.104:0/765736916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87e8004ee0 0x7f87e800ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fc9293ed700 1 --2- 172.21.15.104:0/3034103285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc92415ffa0 0x7fc92406d710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc9200097d0 tx=0x7fc920009ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6686ac858d37e080 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fa60affd700 1 --2- 172.21.15.104:0/3957996206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa5f80663c0 0x7fa5f8068870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f6ffd700 1 --2- 172.21.15.104:0/765736916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87e80043b0 0x7f87e8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fc91bfff700 1 -- 172.21.15.104:0/3034103285 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914001670 con 0x7fc92415ffa0 2023-07-14T11:53:45.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fc91bfff700 1 -- 172.21.15.104:0/3034103285 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc920004030 con 0x7fc92415ffa0 2023-07-14T11:53:45.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fc91bfff700 1 -- 172.21.15.104:0/3034103285 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9200043c0 con 0x7fc92415ffa0 2023-07-14T11:53:45.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f77fe700 1 --2- 172.21.15.104:0/765736916 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e8003a20 0x7f87e8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fa5f3fff700 1 --2- 172.21.15.104:0/3957996206 >> [v2:172.21.15.104:6824/2317846209,v1:172.21.15.104:6825/2317846209] conn(0x7fa5f806ff20 0x7fa5f8072350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fa5f3fff700 1 -- 172.21.15.104:0/3957996206 --> [v2:172.21.15.104:6824/2317846209,v1:172.21.15.104:6825/2317846209] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5f8072a20 con 0x7fa5f806ff20 2023-07-14T11:53:45.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fa60affd700 1 --2- 172.21.15.104:0/3957996206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa5f80663c0 0x7fa5f8068870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa60c1f4720 tx=0x7fa5fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f7fff700 1 -- 172.21.15.104:0/765736916 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e8003a20 msgr2=0x7f87e8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f7fff700 1 --2- 172.21.15.104:0/765736916 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e8003a20 0x7f87e8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f7fff700 1 -- 172.21.15.104:0/765736916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87e80043b0 msgr2=0x7f87e8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f7fff700 1 --2- 172.21.15.104:0/765736916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87e80043b0 0x7f87e8004870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f7fff700 1 -- 172.21.15.104:0/765736916 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f0051150 con 0x7f87e8004ee0 2023-07-14T11:53:45.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fe65f7fe700 1 -- 172.21.15.104:0/4237368131 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700079f0 con 0x7fe6781597f0 2023-07-14T11:53:45.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fe65f7fe700 1 -- 172.21.15.104:0/4237368131 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe670007b70 con 0x7fe6781597f0 2023-07-14T11:53:45.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fe65f7fe700 1 -- 172.21.15.104:0/4237368131 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe670014400 con 0x7fe6781597f0 2023-07-14T11:53:45.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fe67d186700 1 -- 172.21.15.104:0/4237368131 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67815a150 con 0x7fe6781597f0 2023-07-14T11:53:45.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fc92b651700 1 -- 172.21.15.104:0/3034103285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc92415ffa0 msgr2=0x7fc92406d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fc92b651700 1 --2- 172.21.15.104:0/3034103285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc92415ffa0 0x7fc92406d710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc9200097d0 tx=0x7fc920009ae0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fa5f3fff700 1 -- 172.21.15.104:0/3957996206 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6040877a0 con 0x7fa60c0c86b0 2023-07-14T11:53:45.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f77fe700 1 --2- 172.21.15.104:0/765736916 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e8003a20 0x7f87e8003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:45.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f87f7fff700 1 --2- 172.21.15.104:0/765736916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87e8004ee0 0x7f87e800ba20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009d10 tx=0x7f87e80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499399a44c4aa779 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f85fccc6700 1 Processor -- start 2023-07-14T11:53:45.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fe67d186700 1 -- 172.21.15.104:0/4237368131 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67815a430 con 0x7fe6781597f0 2023-07-14T11:53:45.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fa60bfff700 1 --2- 172.21.15.104:0/3957996206 >> [v2:172.21.15.104:6824/2317846209,v1:172.21.15.104:6825/2317846209] conn(0x7fa5f806ff20 0x7fa5f8072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fc92b651700 1 -- 172.21.15.104:0/3034103285 shutdown_connections 2023-07-14T11:53:45.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fc92b651700 1 --2- 172.21.15.104:0/3034103285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc92415ffa0 0x7fc92406d710 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc9200097d0 tx=0x7fc920009ae0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fc92b651700 1 -- 172.21.15.104:0/3034103285 >> 172.21.15.104:0/3034103285 conn(0x7fc92406e960 msgr2=0x7fc92406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7fd684546700 1 -- 172.21.15.104:0/2251453549 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd67c0cc5e0 con 0x7fd67c0cdf70 2023-07-14T11:53:45.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f7c3759e700 1 -- 172.21.15.104:0/2522309715 wait complete. 2023-07-14T11:53:45.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f7c3759e700 1 Processor -- start 2023-07-14T11:53:45.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fa60bfff700 1 --2- 172.21.15.104:0/3957996206 >> [v2:172.21.15.104:6824/2317846209,v1:172.21.15.104:6825/2317846209] conn(0x7fa5f806ff20 0x7fa5f8072350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fe67d186700 1 -- 172.21.15.104:0/4237368131 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe6780ca0f0 con 0x7fe6781597f0 2023-07-14T11:53:45.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f85fccc6700 1 -- start start 2023-07-14T11:53:45.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f87f67fc700 1 -- 172.21.15.104:0/765736916 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4009e60 con 0x7f87e8004ee0 2023-07-14T11:53:45.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f87f67fc700 1 -- 172.21.15.104:0/765736916 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e400e9a0 con 0x7f87e8004ee0 2023-07-14T11:53:45.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f87f67fc700 1 -- 172.21.15.104:0/765736916 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e400ed10 con 0x7f87e8004ee0 2023-07-14T11:53:45.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fdb10f13700 1 Processor -- start 2023-07-14T11:53:45.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85fccc6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85f004f200 0x7f85f00e2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fc92b651700 1 -- 172.21.15.104:0/3034103285 shutdown_connections 2023-07-14T11:53:45.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fe65f7fe700 1 -- 172.21.15.104:0/4237368131 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe670014560 con 0x7fe6781597f0 2023-07-14T11:53:45.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.084+0000 7f7c3759e700 1 -- start start 2023-07-14T11:53:45.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85fccc6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85f00de730 0x7f85f004ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fa5f3fff700 1 -- 172.21.15.104:0/3957996206 <== osd.1 v2:172.21.15.104:6824/2317846209 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa5f8072a20 con 0x7fa5f806ff20 2023-07-14T11:53:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85fccc6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85f00deb00 0x7f85f0034610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fd672ffd700 1 -- 172.21.15.104:0/2251453549 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd67802e450 con 0x7fd67c0cdf70 2023-07-14T11:53:45.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fd672ffd700 1 --2- 172.21.15.104:0/2251453549 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd660066340 0x7fd6600687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fd672ffd700 1 -- 172.21.15.104:0/2251453549 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd678084ff0 con 0x7fd67c0cdf70 2023-07-14T11:53:45.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fd672ffd700 1 --2- 172.21.15.104:0/2251453549 >> [v2:172.21.15.104:6803/3734215753,v1:172.21.15.104:6807/3734215753] conn(0x7fd66006ff90 0x7fd6600723c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fd672ffd700 1 -- 172.21.15.104:0/2251453549 --> [v2:172.21.15.104:6803/3734215753,v1:172.21.15.104:6807/3734215753] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd660072a90 con 0x7fd66006ff90 2023-07-14T11:53:45.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fd6822e2700 1 --2- 172.21.15.104:0/2251453549 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd660066340 0x7fd6600687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fd682ae3700 1 --2- 172.21.15.104:0/2251453549 >> [v2:172.21.15.104:6803/3734215753,v1:172.21.15.104:6807/3734215753] conn(0x7fd66006ff90 0x7fd6600723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fd682ae3700 1 --2- 172.21.15.104:0/2251453549 >> [v2:172.21.15.104:6803/3734215753,v1:172.21.15.104:6807/3734215753] conn(0x7fd66006ff90 0x7fd6600723c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fd6822e2700 1 --2- 172.21.15.104:0/2251453549 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd660066340 0x7fd6600687f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd66c006fd0 tx=0x7fd66c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fd672ffd700 1 -- 172.21.15.104:0/2251453549 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6780522f0 con 0x7fd67c0cdf70 2023-07-14T11:53:45.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fd672ffd700 1 -- 172.21.15.104:0/2251453549 <== osd.0 v2:172.21.15.104:6803/3734215753 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd660072a90 con 0x7fd66006ff90 2023-07-14T11:53:45.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fe65f7fe700 1 --2- 172.21.15.104:0/4237368131 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe664066340 0x7fe6640687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fe65f7fe700 1 -- 172.21.15.104:0/4237368131 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe670083720 con 0x7fe6781597f0 2023-07-14T11:53:45.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fe65f7fe700 1 --2- 172.21.15.104:0/4237368131 >> [v2:172.21.15.104:6800/3827055184,v1:172.21.15.104:6801/3827055184] conn(0x7fe664070010 0x7fe664072440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fe65f7fe700 1 -- 172.21.15.104:0/4237368131 --> [v2:172.21.15.104:6800/3827055184,v1:172.21.15.104:6801/3827055184] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe664072b10 con 0x7fe664070010 2023-07-14T11:53:45.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fe65f7fe700 1 -- 172.21.15.104:0/4237368131 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe670083aa0 con 0x7fe6781597f0 2023-07-14T11:53:45.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fe676d9d700 1 --2- 172.21.15.104:0/4237368131 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe664066340 0x7fe6640687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fe67659c700 1 --2- 172.21.15.104:0/4237368131 >> [v2:172.21.15.104:6800/3827055184,v1:172.21.15.104:6801/3827055184] conn(0x7fe664070010 0x7fe664072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fe676d9d700 1 --2- 172.21.15.104:0/4237368131 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe664066340 0x7fe6640687f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe6680017a0 tx=0x7fe668007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fe67659c700 1 --2- 172.21.15.104:0/4237368131 >> [v2:172.21.15.104:6800/3827055184,v1:172.21.15.104:6801/3827055184] conn(0x7fe664070010 0x7fe664072440 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fe65f7fe700 1 -- 172.21.15.104:0/4237368131 <== osd.2 v2:172.21.15.104:6800/3827055184 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe664072b10 con 0x7fe664070010 2023-07-14T11:53:45.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7f7c3759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c380caa70 0x7f7c38153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7f7c3759e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c380ce1e0 0x7f7c380b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c3659c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c380caa70 0x7f7c38153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c3659c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c380caa70 0x7f7c38153420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:51526/0 (socket says 172.21.15.104:51526) 2023-07-14T11:53:45.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c3659c700 1 -- 172.21.15.104:0/2650317142 learned_addr learned my addr 172.21.15.104:0/2650317142 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c3759e700 1 --2- 172.21.15.104:0/2650317142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c38158c70 0x7f7c3815a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c3759e700 1 -- 172.21.15.104:0/2650317142 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c38154370 con 0x7f7c380ce1e0 2023-07-14T11:53:45.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c3759e700 1 -- 172.21.15.104:0/2650317142 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c381556f0 con 0x7f7c38158c70 2023-07-14T11:53:45.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c3759e700 1 -- 172.21.15.104:0/2650317142 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c380c2760 con 0x7f7c380caa70 2023-07-14T11:53:45.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c35d9b700 1 --2- 172.21.15.104:0/2650317142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c380ce1e0 0x7f7c380b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c36d9d700 1 --2- 172.21.15.104:0/2650317142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c38158c70 0x7f7c3815a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c35d9b700 1 -- 172.21.15.104:0/2650317142 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c380caa70 msgr2=0x7f7c38153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c35d9b700 1 --2- 172.21.15.104:0/2650317142 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c380caa70 0x7f7c38153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c35d9b700 1 -- 172.21.15.104:0/2650317142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c38158c70 msgr2=0x7f7c3815a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c35d9b700 1 --2- 172.21.15.104:0/2650317142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c38158c70 0x7f7c3815a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c35d9b700 1 -- 172.21.15.104:0/2650317142 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c30009a70 con 0x7f7c380ce1e0 2023-07-14T11:53:45.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c35d9b700 1 --2- 172.21.15.104:0/2650317142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c380ce1e0 0x7f7c380b0ef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00e760 tx=0x7f7c2c00ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85fccc6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f00e44d0 con 0x7f85f004f200 2023-07-14T11:53:45.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85fccc6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f00e4900 con 0x7f85f00de730 2023-07-14T11:53:45.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85fccc6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f00e4be0 con 0x7f85f00deb00 2023-07-14T11:53:45.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85f659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85f004f200 0x7f85f00e2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85f659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85f004f200 0x7f85f00e2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38242/0 (socket says 172.21.15.104:38242) 2023-07-14T11:53:45.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85f659c700 1 -- 172.21.15.104:0/389397467 learned_addr learned my addr 172.21.15.104:0/389397467 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85f6d9d700 1 --2- 172.21.15.104:0/389397467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85f00deb00 0x7f85f0034610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85f5d9b700 1 --2- 172.21.15.104:0/389397467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85f00de730 0x7f85f004ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85f6d9d700 1 -- 172.21.15.104:0/389397467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85f00de730 msgr2=0x7f85f004ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85f6d9d700 1 --2- 172.21.15.104:0/389397467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85f00de730 0x7f85f004ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85f6d9d700 1 -- 172.21.15.104:0/389397467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85f004f200 msgr2=0x7f85f00e2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85f6d9d700 1 --2- 172.21.15.104:0/389397467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85f004f200 0x7f85f00e2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f85f6d9d700 1 -- 172.21.15.104:0/389397467 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e80094b0 con 0x7f85f00deb00 2023-07-14T11:53:45.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7f85f6d9d700 1 --2- 172.21.15.104:0/389397467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85f00deb00 0x7f85f0034610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f85f8068720 tx=0x7f85f80674e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7f85e6ffd700 1 -- 172.21.15.104:0/389397467 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f807c070 con 0x7f85f00deb00 2023-07-14T11:53:45.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7f85e6ffd700 1 -- 172.21.15.104:0/389397467 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f804e220 con 0x7f85f00deb00 2023-07-14T11:53:45.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7f85e6ffd700 1 -- 172.21.15.104:0/389397467 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f806e5a0 con 0x7f85f00deb00 2023-07-14T11:53:45.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7f85fccc6700 1 -- 172.21.15.104:0/389397467 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f00e5270 con 0x7f85f00deb00 2023-07-14T11:53:45.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7f85fccc6700 1 -- 172.21.15.104:0/389397467 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f00d9570 con 0x7f85f00deb00 2023-07-14T11:53:45.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7f85fccc6700 1 -- 172.21.15.104:0/389397467 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f85f00df930 con 0x7f85f00deb00 2023-07-14T11:53:45.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7f85e6ffd700 1 -- 172.21.15.104:0/389397467 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f85f8072d30 con 0x7f85f00deb00 2023-07-14T11:53:45.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f85e6ffd700 1 --2- 172.21.15.104:0/389397467 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f85d80663f0 0x7f85d80688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f85e6ffd700 1 -- 172.21.15.104:0/389397467 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f85f80ebc30 con 0x7f85f00deb00 2023-07-14T11:53:45.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f85e6ffd700 1 --2- 172.21.15.104:0/389397467 >> [v2:172.21.15.121:6822/2394615815,v1:172.21.15.121:6825/2394615815] conn(0x7f85d80700c0 0x7f85d80724f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f85e6ffd700 1 -- 172.21.15.104:0/389397467 --> [v2:172.21.15.121:6822/2394615815,v1:172.21.15.121:6825/2394615815] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f85d8072bc0 con 0x7f85d80700c0 2023-07-14T11:53:45.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f85e6ffd700 1 -- 172.21.15.104:0/389397467 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85f80ebfe0 con 0x7f85f00deb00 2023-07-14T11:53:45.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f85f659c700 1 --2- 172.21.15.104:0/389397467 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f85d80663f0 0x7f85d80688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f85f5d9b700 1 --2- 172.21.15.104:0/389397467 >> [v2:172.21.15.121:6822/2394615815,v1:172.21.15.121:6825/2394615815] conn(0x7f85d80700c0 0x7f85d80724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f85f5d9b700 1 --2- 172.21.15.104:0/389397467 >> [v2:172.21.15.121:6822/2394615815,v1:172.21.15.121:6825/2394615815] conn(0x7f85d80700c0 0x7f85d80724f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f85f659c700 1 --2- 172.21.15.104:0/389397467 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f85d80663f0 0x7f85d80688a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009ad0 tx=0x7f85ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f87fdf08700 1 -- 172.21.15.104:0/765736916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87e8004ee0 msgr2=0x7f87e800ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f87fdf08700 1 --2- 172.21.15.104:0/765736916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87e8004ee0 0x7f87e800ba20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009d10 tx=0x7f87e80058a0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f87f7fff700 1 -- 172.21.15.104:0/765736916 reap_dead start 2023-07-14T11:53:45.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f87fdf08700 1 -- 172.21.15.104:0/765736916 shutdown_connections 2023-07-14T11:53:45.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f87fdf08700 1 -- 172.21.15.104:0/765736916 >> 172.21.15.104:0/765736916 conn(0x7f87f0051770 msgr2=0x7f87f0051b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f87fdf08700 1 -- 172.21.15.104:0/765736916 shutdown_connections 2023-07-14T11:53:45.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7f87fdf08700 1 -- 172.21.15.104:0/765736916 wait complete. 2023-07-14T11:53:45.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7f87fdf08700 1 Processor -- start 2023-07-14T11:53:45.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7f87fdf08700 1 -- start start 2023-07-14T11:53:45.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f87fdf08700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e8004ee0 0x7f87f0053c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f87f77fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e8004ee0 0x7f87f0053c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f87f77fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e8004ee0 0x7f87f0053c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:51544/0 (socket says 172.21.15.104:51544) 2023-07-14T11:53:45.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fdb10f13700 1 -- start start 2023-07-14T11:53:45.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fdb10f13700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb0c0caa70 0x7fdb0c0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fdb10f13700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdb0c158c70 0x7fdb0c0cae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fdb0a59c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb0c0caa70 0x7fdb0c0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fdb0a59c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb0c0caa70 0x7fdb0c0c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:51536/0 (socket says 172.21.15.104:51536) 2023-07-14T11:53:45.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fdb0ad9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fdb0c0ce1e0 0x7fdb0c06bc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42498/0 (socket says 172.21.15.104:42498) 2023-07-14T11:53:45.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.088+0000 7fc92b651700 1 -- 172.21.15.104:0/3034103285 wait complete. 2023-07-14T11:53:45.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fc92b651700 1 Processor -- start 2023-07-14T11:53:45.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fc92b651700 1 -- start start 2023-07-14T11:53:45.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fc92b651700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc92406dc50 0x7fc9241f3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fc92b651700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9240cb180 0x7fc9241634c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fc92b651700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc924160370 0x7fc924151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fc9293ed700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc92406dc50 0x7fc9241f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fc928bec700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9240cb180 0x7fc9241634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fc928bec700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9240cb180 0x7fc9241634c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57254/0 (socket says 172.21.15.104:57254) 2023-07-14T11:53:45.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fc928bec700 1 -- 172.21.15.104:0/1507107882 learned_addr learned my addr 172.21.15.104:0/1507107882 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fc92b651700 1 -- 172.21.15.104:0/1507107882 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9241615e0 con 0x7fc924160370 2023-07-14T11:53:45.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fc92b651700 1 -- 172.21.15.104:0/1507107882 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9241f3690 con 0x7fc9240cb180 2023-07-14T11:53:45.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fc92b651700 1 -- 172.21.15.104:0/1507107882 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc924160d60 con 0x7fc92406dc50 2023-07-14T11:53:45.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.092+0000 7fc929bee700 1 --2- 172.21.15.104:0/1507107882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc924160370 0x7fc924151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fc928bec700 1 -- 172.21.15.104:0/1507107882 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc92406dc50 msgr2=0x7fc9241f3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fc928bec700 1 --2- 172.21.15.104:0/1507107882 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc92406dc50 0x7fc9241f3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fc928bec700 1 -- 172.21.15.104:0/1507107882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc924160370 msgr2=0x7fc924151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fc928bec700 1 --2- 172.21.15.104:0/1507107882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc924160370 0x7fc924151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fc928bec700 1 -- 172.21.15.104:0/1507107882 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9200094b0 con 0x7fc9240cb180 2023-07-14T11:53:45.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fc928bec700 1 --2- 172.21.15.104:0/1507107882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9240cb180 0x7fc9241634c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc91400c6b0 tx=0x7fc91400c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fc919ffb700 1 -- 172.21.15.104:0/1507107882 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914004ca0 con 0x7fc9240cb180 2023-07-14T11:53:45.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fc919ffb700 1 -- 172.21.15.104:0/1507107882 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc914004e00 con 0x7fc9240cb180 2023-07-14T11:53:45.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fc919ffb700 1 -- 172.21.15.104:0/1507107882 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9140072d0 con 0x7fc9240cb180 2023-07-14T11:53:45.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fc92b651700 1 -- 172.21.15.104:0/1507107882 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc924163a00 con 0x7fc9240cb180 2023-07-14T11:53:45.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fc92b651700 1 -- 172.21.15.104:0/1507107882 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc924160870 con 0x7fc9240cb180 2023-07-14T11:53:45.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f87f77fe700 1 -- 172.21.15.104:0/1853478016 learned_addr learned my addr 172.21.15.104:0/1853478016 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f87fdf08700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87f004f6b0 0x7f87f004fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fc92b651700 1 -- 172.21.15.104:0/1507107882 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc9240cc810 con 0x7fc9240cb180 2023-07-14T11:53:45.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c26ffd700 1 -- 172.21.15.104:0/2650317142 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c014890 con 0x7f7c380ce1e0 2023-07-14T11:53:45.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c26ffd700 1 -- 172.21.15.104:0/2650317142 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2c00f040 con 0x7f7c380ce1e0 2023-07-14T11:53:45.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c26ffd700 1 -- 172.21.15.104:0/2650317142 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c014c30 con 0x7f7c380ce1e0 2023-07-14T11:53:45.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f87fdf08700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87f0035210 0x7f87f0035680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fdb0a59c700 1 -- 172.21.15.104:0/771710386 learned_addr learned my addr 172.21.15.104:0/771710386 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fdb10f13700 1 -- 172.21.15.104:0/771710386 --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0c0cdd50 con 0x7fdb0c0ce1e0 2023-07-14T11:53:45.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fdb10f13700 1 -- 172.21.15.104:0/771710386 --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0c0cda20 con 0x7fdb0c158c70 2023-07-14T11:53:45.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7fdb10f13700 1 -- 172.21.15.104:0/771710386 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0cb380 con 0x7fdb0c0caa70 2023-07-14T11:53:45.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c3759e700 1 -- 172.21.15.104:0/2650317142 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c380c2a40 con 0x7f7c380ce1e0 2023-07-14T11:53:45.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f7c3759e700 1 -- 172.21.15.104:0/2650317142 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c38153960 con 0x7f7c380ce1e0 2023-07-14T11:53:45.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.096+0000 7f87f6ffd700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87f004f6b0 0x7f87f004fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7f87f7fff700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87f0035210 0x7f87f0035680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7f7c3759e700 1 -- 172.21.15.104:0/2650317142 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7c38153e80 con 0x7f7c380ce1e0 2023-07-14T11:53:45.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7f85e6ffd700 1 -- 172.21.15.104:0/389397467 <== osd.4 v2:172.21.15.121:6822/2394615815 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f85d8072bc0 con 0x7f85d80700c0 2023-07-14T11:53:45.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7fdb09d9b700 1 -- 172.21.15.104:0/771710386 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517381066 0 0) 0x7fdb0c0cdd50 con 0x7fdb0c0ce1e0 2023-07-14T11:53:45.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7fdb09d9b700 1 -- 172.21.15.104:0/771710386 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaf80057e0 con 0x7fdb0c0ce1e0 2023-07-14T11:53:45.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7fdb0a59c700 1 -- 172.21.15.104:0/771710386 >> v2:172.21.15.104:3300/0 conn(0x7fdb0c158c70 msgr2=0x7fdb0c0cae40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:53:45.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7fdb0a59c700 1 --2- 172.21.15.104:0/771710386 >> v2:172.21.15.104:3300/0 conn(0x7fdb0c158c70 0x7fdb0c0cae40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7fdb0a59c700 1 -- 172.21.15.104:0/771710386 >> v1:172.21.15.104:6789/0 conn(0x7fdb0c0ce1e0 legacy=0x7fdb0c06bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7fdb0a59c700 1 -- 172.21.15.104:0/771710386 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c0ca440 con 0x7fdb0c0caa70 2023-07-14T11:53:45.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7f87fdf08700 1 -- 172.21.15.104:0/1853478016 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f00d3200 con 0x7f87f0035210 2023-07-14T11:53:45.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7f87fdf08700 1 -- 172.21.15.104:0/1853478016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f00d34e0 con 0x7f87f004f6b0 2023-07-14T11:53:45.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7f87fdf08700 1 -- 172.21.15.104:0/1853478016 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f00d37c0 con 0x7f87e8004ee0 2023-07-14T11:53:45.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7f7c26ffd700 1 -- 172.21.15.104:0/2650317142 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7c2c01d400 con 0x7f7c380ce1e0 2023-07-14T11:53:45.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7f7c26ffd700 1 --2- 172.21.15.104:0/2650317142 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c180663c0 0x7f7c18068870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7f7c26ffd700 1 -- 172.21.15.104:0/2650317142 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7c2c083e40 con 0x7f7c380ce1e0 2023-07-14T11:53:45.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f7c26ffd700 1 --2- 172.21.15.104:0/2650317142 >> [v2:172.21.15.104:6816/831948081,v1:172.21.15.104:6817/831948081] conn(0x7f7c18070010 0x7f7c18072440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f7c26ffd700 1 -- 172.21.15.104:0/2650317142 --> [v2:172.21.15.104:6816/831948081,v1:172.21.15.104:6817/831948081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c18072b10 con 0x7f7c18070010 2023-07-14T11:53:45.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fdb0a59c700 1 --2- 172.21.15.104:0/771710386 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb0c0caa70 0x7fdb0c0c86b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00c370 tx=0x7fdafc00c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64b573dd76bf8613 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f7c3659c700 1 --2- 172.21.15.104:0/2650317142 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c180663c0 0x7f7c18068870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f7c36d9d700 1 --2- 172.21.15.104:0/2650317142 >> [v2:172.21.15.104:6816/831948081,v1:172.21.15.104:6817/831948081] conn(0x7f7c18070010 0x7f7c18072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f7c26ffd700 1 -- 172.21.15.104:0/2650317142 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c2c051040 con 0x7f7c380ce1e0 2023-07-14T11:53:45.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f7c36d9d700 1 --2- 172.21.15.104:0/2650317142 >> [v2:172.21.15.104:6816/831948081,v1:172.21.15.104:6817/831948081] conn(0x7f7c18070010 0x7f7c18072440 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f7c3659c700 1 --2- 172.21.15.104:0/2650317142 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c180663c0 0x7f7c18068870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7c28007b10 tx=0x7f7c28008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f7c26ffd700 1 -- 172.21.15.104:0/2650317142 <== osd.3 v2:172.21.15.104:6816/831948081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7c18072b10 con 0x7f7c18070010 2023-07-14T11:53:45.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f87f6ffd700 1 -- 172.21.15.104:0/1853478016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e8004ee0 msgr2=0x7f87f0053c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f87f6ffd700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87e8004ee0 0x7f87f0053c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f87f6ffd700 1 -- 172.21.15.104:0/1853478016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87f0035210 msgr2=0x7f87f0035680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f87f6ffd700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87f0035210 0x7f87f0035680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f87f6ffd700 1 -- 172.21.15.104:0/1853478016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f00d3aa0 con 0x7f87f004f6b0 2023-07-14T11:53:45.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f87f6ffd700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87f004f6b0 0x7f87f004fb20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f87f000bf50 tx=0x7f87ec010a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f87e3fff700 1 -- 172.21.15.104:0/1853478016 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec014800 con 0x7f87f004f6b0 2023-07-14T11:53:45.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f87e3fff700 1 -- 172.21.15.104:0/1853478016 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ec014960 con 0x7f87f004f6b0 2023-07-14T11:53:45.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f87e3fff700 1 -- 172.21.15.104:0/1853478016 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec014d10 con 0x7f87f004f6b0 2023-07-14T11:53:45.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f87fdf08700 1 -- 172.21.15.104:0/1853478016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f00d3d80 con 0x7f87f004f6b0 2023-07-14T11:53:45.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f87fdf08700 1 -- 172.21.15.104:0/1853478016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f0050060 con 0x7f87f004f6b0 2023-07-14T11:53:45.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fdb09d9b700 1 -- 172.21.15.104:0/771710386 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc004030 con 0x7fdb0c0caa70 2023-07-14T11:53:45.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fdb09d9b700 1 -- 172.21.15.104:0/771710386 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc004190 con 0x7fdb0c0caa70 2023-07-14T11:53:45.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fdb09d9b700 1 -- 172.21.15.104:0/771710386 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc004520 con 0x7fdb0c0caa70 2023-07-14T11:53:45.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fdb10f13700 1 -- 172.21.15.104:0/771710386 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb0c0caa70 msgr2=0x7fdb0c0c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fdb10f13700 1 --2- 172.21.15.104:0/771710386 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb0c0caa70 0x7fdb0c0c86b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00c370 tx=0x7fdafc00c680 comp rx=0 tx=0).stop 2023-07-14T11:53:45.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.100+0000 7fc919ffb700 1 -- 172.21.15.104:0/1507107882 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc914007430 con 0x7fc9240cb180 2023-07-14T11:53:45.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fc919ffb700 1 --2- 172.21.15.104:0/1507107882 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc908066370 0x7fc908068820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fc919ffb700 1 -- 172.21.15.104:0/1507107882 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc9140843a0 con 0x7fc9240cb180 2023-07-14T11:53:45.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fc919ffb700 1 --2- 172.21.15.104:0/1507107882 >> [v2:172.21.15.121:6808/2610729951,v1:172.21.15.121:6809/2610729951] conn(0x7fc908070040 0x7fc908072470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fc919ffb700 1 -- 172.21.15.104:0/1507107882 --> [v2:172.21.15.121:6808/2610729951,v1:172.21.15.121:6809/2610729951] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc908072b40 con 0x7fc908070040 2023-07-14T11:53:45.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fc919ffb700 1 -- 172.21.15.104:0/1507107882 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc914084710 con 0x7fc9240cb180 2023-07-14T11:53:45.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fc929bee700 1 --2- 172.21.15.104:0/1507107882 >> [v2:172.21.15.121:6808/2610729951,v1:172.21.15.121:6809/2610729951] conn(0x7fc908070040 0x7fc908072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fc9293ed700 1 --2- 172.21.15.104:0/1507107882 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc908066370 0x7fc908068820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fc929bee700 1 --2- 172.21.15.104:0/1507107882 >> [v2:172.21.15.121:6808/2610729951,v1:172.21.15.121:6809/2610729951] conn(0x7fc908070040 0x7fc908072470 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fc9293ed700 1 --2- 172.21.15.104:0/1507107882 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc908066370 0x7fc908068820 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc9240cf100 tx=0x7fc920008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.176 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fdb10f13700 1 -- 172.21.15.104:0/771710386 shutdown_connections 2023-07-14T11:53:45.176 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fdb10f13700 1 --2- 172.21.15.104:0/771710386 >> v2:172.21.15.104:3300/0 conn(0x7fdb0c158c70 0x7fdb0c0cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fdb10f13700 1 --2- 172.21.15.104:0/771710386 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb0c0caa70 0x7fdb0c0c86b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fdb10f13700 1 -- 172.21.15.104:0/771710386 >> 172.21.15.104:0/771710386 conn(0x7fdb0c060320 msgr2=0x7fdb0c0606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7fdb10f13700 1 -- 172.21.15.104:0/771710386 shutdown_connections 2023-07-14T11:53:45.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.104+0000 7f87fdf08700 1 -- 172.21.15.104:0/1853478016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f87f00d3650 con 0x7f87f004f6b0 2023-07-14T11:53:45.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.108+0000 7fdb10f13700 1 -- 172.21.15.104:0/771710386 wait complete. 2023-07-14T11:53:45.179 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.108+0000 7fc919ffb700 1 -- 172.21.15.104:0/1507107882 <== osd.7 v2:172.21.15.121:6808/2610729951 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc908072b40 con 0x7fc908070040 2023-07-14T11:53:45.179 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.108+0000 7fdb10f13700 1 Processor -- start 2023-07-14T11:53:45.179 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.108+0000 7fdb10f13700 1 -- start start 2023-07-14T11:53:45.180 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.108+0000 7f87e3fff700 1 -- 172.21.15.104:0/1853478016 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f87ec004020 con 0x7f87f004f6b0 2023-07-14T11:53:45.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.112+0000 7f87e3fff700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87dc0662f0 0x7f87dc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.112+0000 7f87e3fff700 1 -- 172.21.15.104:0/1853478016 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f87ec084690 con 0x7f87f004f6b0 2023-07-14T11:53:45.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.112+0000 7f87e3fff700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.121:6800/1185793020,v1:172.21.15.121:6801/1185793020] conn(0x7f87dc06ffc0 0x7f87dc0723f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.112+0000 7f87e3fff700 1 -- 172.21.15.104:0/1853478016 --> [v2:172.21.15.121:6800/1185793020,v1:172.21.15.121:6801/1185793020] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87dc072ac0 con 0x7f87dc06ffc0 2023-07-14T11:53:45.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.112+0000 7f87e3fff700 1 -- 172.21.15.104:0/1853478016 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87ec084a40 con 0x7f87f004f6b0 2023-07-14T11:53:45.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.112+0000 7f87f77fe700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87dc0662f0 0x7f87dc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.112+0000 7f87f7fff700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.121:6800/1185793020,v1:172.21.15.121:6801/1185793020] conn(0x7f87dc06ffc0 0x7f87dc0723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.112+0000 7f87f77fe700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87dc0662f0 0x7f87dc0687a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f87f8073600 tx=0x7f87f806d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.112+0000 7f87f7fff700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.121:6800/1185793020,v1:172.21.15.121:6801/1185793020] conn(0x7f87dc06ffc0 0x7f87dc0723f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb10f13700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb0c0caa70 0x7fdb0c0c2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb10f13700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb0c0ce1e0 0x7fdb0c148430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb10f13700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb0c158c70 0x7fdb0c153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb10f13700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c152a90 con 0x7fdb0c0caa70 2023-07-14T11:53:45.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb10f13700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0c2c40 con 0x7fdb0c158c70 2023-07-14T11:53:45.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb10f13700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c06df70 con 0x7fdb0c0ce1e0 2023-07-14T11:53:45.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb0a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb0c0caa70 0x7fdb0c0c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb0a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb0c0caa70 0x7fdb0c0c2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38282/0 (socket says 172.21.15.104:38282) 2023-07-14T11:53:45.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb0a59c700 1 -- 172.21.15.104:0/2312317733 learned_addr learned my addr 172.21.15.104:0/2312317733 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb0ad9d700 1 --2- 172.21.15.104:0/2312317733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb0c158c70 0x7fdb0c153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb0a59c700 1 -- 172.21.15.104:0/2312317733 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb0c0ce1e0 msgr2=0x7fdb0c148430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:53:45.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb0a59c700 1 --2- 172.21.15.104:0/2312317733 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb0c0ce1e0 0x7fdb0c148430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb0a59c700 1 -- 172.21.15.104:0/2312317733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb0c158c70 msgr2=0x7fdb0c153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.186 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb0a59c700 1 --2- 172.21.15.104:0/2312317733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb0c158c70 0x7fdb0c153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.186 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb0a59c700 1 -- 172.21.15.104:0/2312317733 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc00c050 con 0x7fdb0c0caa70 2023-07-14T11:53:45.186 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb0ad9d700 1 --2- 172.21.15.104:0/2312317733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb0c158c70 0x7fdb0c153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:45.186 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7f87e3fff700 1 -- 172.21.15.104:0/1853478016 <== osd.5 v2:172.21.15.121:6800/1185793020 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f87dc072ac0 con 0x7f87dc06ffc0 2023-07-14T11:53:45.187 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb0a59c700 1 --2- 172.21.15.104:0/2312317733 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb0c0caa70 0x7fdb0c0c2700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0cb4c0 tx=0x7fdafc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb037fe700 1 -- 172.21.15.104:0/2312317733 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0038c0 con 0x7fdb0c0caa70 2023-07-14T11:53:45.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb037fe700 1 -- 172.21.15.104:0/2312317733 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc003a40 con 0x7fdb0c0caa70 2023-07-14T11:53:45.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb037fe700 1 -- 172.21.15.104:0/2312317733 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc003df0 con 0x7fdb0c0caa70 2023-07-14T11:53:45.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.116+0000 7fdb10f13700 1 -- 172.21.15.104:0/2312317733 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c06dc90 con 0x7fdb0c0caa70 2023-07-14T11:53:45.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.120+0000 7fdb10f13700 1 -- 172.21.15.104:0/2312317733 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c0c1f00 con 0x7fdb0c0caa70 2023-07-14T11:53:45.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.120+0000 7fdb10f13700 1 -- 172.21.15.104:0/2312317733 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdb0c0ca0f0 con 0x7fdb0c0caa70 2023-07-14T11:53:45.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.120+0000 7fdb037fe700 1 -- 172.21.15.104:0/2312317733 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fdafc011070 con 0x7fdb0c0caa70 2023-07-14T11:53:45.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fe67d186700 1 -- 172.21.15.104:0/4237368131 --> [v2:172.21.15.104:6800/3827055184,v1:172.21.15.104:6801/3827055184] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe6780c3010 con 0x7fe664070010 2023-07-14T11:53:45.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fd684546700 1 -- 172.21.15.104:0/2251453549 --> [v2:172.21.15.104:6803/3734215753,v1:172.21.15.104:6807/3734215753] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd67c0cc300 con 0x7fd66006ff90 2023-07-14T11:53:45.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fd672ffd700 1 -- 172.21.15.104:0/2251453549 <== osd.0 v2:172.21.15.104:6803/3734215753 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd674001e80 con 0x7fd66006ff90 2023-07-14T11:53:45.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fe65f7fe700 1 -- 172.21.15.104:0/4237368131 <== osd.2 v2:172.21.15.104:6800/3827055184 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe6780c3010 con 0x7fe664070010 2023-07-14T11:53:45.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fdb037fe700 1 --2- 172.21.15.104:0/2312317733 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdaec066340 0x7fdaec0687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fdb037fe700 1 -- 172.21.15.104:0/2312317733 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdafc085a80 con 0x7fdb0c0caa70 2023-07-14T11:53:45.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fdb037fe700 1 --2- 172.21.15.104:0/2312317733 >> [v2:172.21.15.121:6816/754390529,v1:172.21.15.121:6817/754390529] conn(0x7fdaec070010 0x7fdaec072440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fdb037fe700 1 -- 172.21.15.104:0/2312317733 --> [v2:172.21.15.121:6816/754390529,v1:172.21.15.121:6817/754390529] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdaec072b10 con 0x7fdaec070010 2023-07-14T11:53:45.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fdb037fe700 1 -- 172.21.15.104:0/2312317733 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdafc085f20 con 0x7fdb0c0caa70 2023-07-14T11:53:45.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fdb01d9b700 1 --2- 172.21.15.104:0/2312317733 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdaec066340 0x7fdaec0687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fe67d186700 1 -- 172.21.15.104:0/4237368131 >> [v2:172.21.15.104:6800/3827055184,v1:172.21.15.104:6801/3827055184] conn(0x7fe664070010 msgr2=0x7fe664072440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fe67d186700 1 --2- 172.21.15.104:0/4237368131 >> [v2:172.21.15.104:6800/3827055184,v1:172.21.15.104:6801/3827055184] conn(0x7fe664070010 0x7fe664072440 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fa611c71700 1 -- 172.21.15.104:0/3957996206 --> [v2:172.21.15.104:6824/2317846209,v1:172.21.15.104:6825/2317846209] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa60c0c94a0 con 0x7fa5f806ff20 2023-07-14T11:53:45.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fdb0ad9d700 1 --2- 172.21.15.104:0/2312317733 >> [v2:172.21.15.121:6816/754390529,v1:172.21.15.121:6817/754390529] conn(0x7fdaec070010 0x7fdaec072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fd684546700 1 -- 172.21.15.104:0/2251453549 >> [v2:172.21.15.104:6803/3734215753,v1:172.21.15.104:6807/3734215753] conn(0x7fd66006ff90 msgr2=0x7fd6600723c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fd684546700 1 --2- 172.21.15.104:0/2251453549 >> [v2:172.21.15.104:6803/3734215753,v1:172.21.15.104:6807/3734215753] conn(0x7fd66006ff90 0x7fd6600723c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fe67d186700 1 -- 172.21.15.104:0/4237368131 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe664066340 msgr2=0x7fe6640687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fd684546700 1 -- 172.21.15.104:0/2251453549 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd660066340 msgr2=0x7fd6600687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fd684546700 1 --2- 172.21.15.104:0/2251453549 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd660066340 0x7fd6600687f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd66c006fd0 tx=0x7fd66c008040 comp rx=0 tx=0).stop 2023-07-14T11:53:45.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fe67d186700 1 --2- 172.21.15.104:0/4237368131 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe664066340 0x7fe6640687f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe6680017a0 tx=0x7fe668007040 comp rx=0 tx=0).stop 2023-07-14T11:53:45.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fd684546700 1 -- 172.21.15.104:0/2251453549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c0cdf70 msgr2=0x7fd67c152a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fd684546700 1 --2- 172.21.15.104:0/2251453549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd67c0cdf70 0x7fd67c152a00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0cf120 tx=0x7fd678004770 comp rx=0 tx=0).stop 2023-07-14T11:53:45.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fa5f3fff700 1 -- 172.21.15.104:0/3957996206 <== osd.1 v2:172.21.15.104:6824/2317846209 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa600005140 con 0x7fa5f806ff20 2023-07-14T11:53:45.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fd682ae3700 1 -- 172.21.15.104:0/2251453549 reap_dead start 2023-07-14T11:53:45.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fe67d186700 1 -- 172.21.15.104:0/4237368131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6781597f0 msgr2=0x7fe67806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fd684546700 1 -- 172.21.15.104:0/2251453549 shutdown_connections 2023-07-14T11:53:45.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fd684546700 1 -- 172.21.15.104:0/2251453549 >> 172.21.15.104:0/2251453549 conn(0x7fd67c060320 msgr2=0x7fd67c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fe67d186700 1 --2- 172.21.15.104:0/4237368131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6781597f0 0x7fe67806c870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe6780cbba0 tx=0x7fe670004940 comp rx=0 tx=0).stop 2023-07-14T11:53:45.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fd684546700 1 -- 172.21.15.104:0/2251453549 shutdown_connections 2023-07-14T11:53:45.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fe67759e700 1 -- 172.21.15.104:0/4237368131 reap_dead start 2023-07-14T11:53:45.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fdb0ad9d700 1 --2- 172.21.15.104:0/2312317733 >> [v2:172.21.15.121:6816/754390529,v1:172.21.15.121:6817/754390529] conn(0x7fdaec070010 0x7fdaec072440 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fdb01d9b700 1 --2- 172.21.15.104:0/2312317733 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdaec066340 0x7fdaec0687f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c1f1180 tx=0x7fdaf4006be0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fd684546700 1 -- 172.21.15.104:0/2251453549 wait complete. 2023-07-14T11:53:45.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fe67d186700 1 -- 172.21.15.104:0/4237368131 shutdown_connections 2023-07-14T11:53:45.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fe67d186700 1 -- 172.21.15.104:0/4237368131 >> 172.21.15.104:0/4237368131 conn(0x7fe678060320 msgr2=0x7fe6780b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fa5f1ffb700 1 -- 172.21.15.104:0/3957996206 >> [v2:172.21.15.104:6824/2317846209,v1:172.21.15.104:6825/2317846209] conn(0x7fa5f806ff20 msgr2=0x7fa5f8072350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fa5f1ffb700 1 --2- 172.21.15.104:0/3957996206 >> [v2:172.21.15.104:6824/2317846209,v1:172.21.15.104:6825/2317846209] conn(0x7fa5f806ff20 0x7fa5f8072350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.124+0000 7fe67d186700 1 -- 172.21.15.104:0/4237368131 shutdown_connections 2023-07-14T11:53:45.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.128+0000 7fe67d186700 1 -- 172.21.15.104:0/4237368131 wait complete. 2023-07-14T11:53:45.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.128+0000 7fa5f1ffb700 1 -- 172.21.15.104:0/3957996206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa5f80663c0 msgr2=0x7fa5f8068870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.128+0000 7fa5f1ffb700 1 --2- 172.21.15.104:0/3957996206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa5f80663c0 0x7fa5f8068870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa60c1f4720 tx=0x7fa5fc008040 comp rx=0 tx=0).stop 2023-07-14T11:53:45.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.128+0000 7fa5f1ffb700 1 -- 172.21.15.104:0/3957996206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa60c0c86b0 msgr2=0x7fa60c155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.128+0000 7fa5f1ffb700 1 --2- 172.21.15.104:0/3957996206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa60c0c86b0 0x7fa60c155e00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa60400b730 tx=0x7fa604004af0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.128+0000 7fa60bfff700 1 -- 172.21.15.104:0/3957996206 reap_dead start 2023-07-14T11:53:45.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.128+0000 7fa5f1ffb700 1 -- 172.21.15.104:0/3957996206 shutdown_connections 2023-07-14T11:53:45.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.128+0000 7fa5f1ffb700 1 -- 172.21.15.104:0/3957996206 >> 172.21.15.104:0/3957996206 conn(0x7fa60c060320 msgr2=0x7fa60c0cd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.128+0000 7fa5f1ffb700 1 -- 172.21.15.104:0/3957996206 shutdown_connections 2023-07-14T11:53:45.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.128+0000 7fa5f1ffb700 1 -- 172.21.15.104:0/3957996206 wait complete. 2023-07-14T11:53:45.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.128+0000 7fdb037fe700 1 -- 172.21.15.104:0/2312317733 <== osd.6 v2:172.21.15.121:6816/754390529 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdaec072b10 con 0x7fdaec070010 2023-07-14T11:53:45.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.132+0000 7f85fccc6700 1 -- 172.21.15.104:0/389397467 --> [v2:172.21.15.121:6822/2394615815,v1:172.21.15.121:6825/2394615815] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f85f00df650 con 0x7f85d80700c0 2023-07-14T11:53:45.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.132+0000 7f85e6ffd700 1 -- 172.21.15.104:0/389397467 <== osd.4 v2:172.21.15.121:6822/2394615815 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f85e8006c80 con 0x7f85d80700c0 2023-07-14T11:53:45.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.136+0000 7f7c3759e700 1 -- 172.21.15.104:0/2650317142 --> [v2:172.21.15.104:6816/831948081,v1:172.21.15.104:6817/831948081] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7c380ca0f0 con 0x7f7c18070010 2023-07-14T11:53:45.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.136+0000 7f85fccc6700 1 -- 172.21.15.104:0/389397467 >> [v2:172.21.15.121:6822/2394615815,v1:172.21.15.121:6825/2394615815] conn(0x7f85d80700c0 msgr2=0x7f85d80724f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.136+0000 7f7c26ffd700 1 -- 172.21.15.104:0/2650317142 <== osd.3 v2:172.21.15.104:6816/831948081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7c3000cdb0 con 0x7f7c18070010 2023-07-14T11:53:45.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.136+0000 7f85fccc6700 1 --2- 172.21.15.104:0/389397467 >> [v2:172.21.15.121:6822/2394615815,v1:172.21.15.121:6825/2394615815] conn(0x7f85d80700c0 0x7f85d80724f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.136+0000 7f85fccc6700 1 -- 172.21.15.104:0/389397467 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f85d80663f0 msgr2=0x7f85d80688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.136+0000 7f85fccc6700 1 --2- 172.21.15.104:0/389397467 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f85d80663f0 0x7f85d80688a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009ad0 tx=0x7f85ec008040 comp rx=0 tx=0).stop 2023-07-14T11:53:45.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.136+0000 7fc92b651700 1 -- 172.21.15.104:0/1507107882 --> [v2:172.21.15.121:6808/2610729951,v1:172.21.15.121:6809/2610729951] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc9240cc530 con 0x7fc908070040 2023-07-14T11:53:45.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f85fccc6700 1 -- 172.21.15.104:0/389397467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85f00deb00 msgr2=0x7f85f0034610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f85fccc6700 1 --2- 172.21.15.104:0/389397467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85f00deb00 0x7f85f0034610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f85f8068720 tx=0x7f85f80674e0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f85f6d9d700 1 -- 172.21.15.104:0/389397467 reap_dead start 2023-07-14T11:53:45.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f85fccc6700 1 -- 172.21.15.104:0/389397467 shutdown_connections 2023-07-14T11:53:45.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f85fccc6700 1 -- 172.21.15.104:0/389397467 >> 172.21.15.104:0/389397467 conn(0x7f85f0051630 msgr2=0x7f85f0006760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7fc919ffb700 1 -- 172.21.15.104:0/1507107882 <== osd.7 v2:172.21.15.121:6808/2610729951 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc91c001f30 con 0x7fc908070040 2023-07-14T11:53:45.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f85fccc6700 1 -- 172.21.15.104:0/389397467 shutdown_connections 2023-07-14T11:53:45.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f85fccc6700 1 -- 172.21.15.104:0/389397467 wait complete. 2023-07-14T11:53:45.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7fc92b651700 1 -- 172.21.15.104:0/1507107882 >> [v2:172.21.15.121:6808/2610729951,v1:172.21.15.121:6809/2610729951] conn(0x7fc908070040 msgr2=0x7fc908072470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7fc92b651700 1 --2- 172.21.15.104:0/1507107882 >> [v2:172.21.15.121:6808/2610729951,v1:172.21.15.121:6809/2610729951] conn(0x7fc908070040 0x7fc908072470 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f7c24ff9700 1 -- 172.21.15.104:0/2650317142 >> [v2:172.21.15.104:6816/831948081,v1:172.21.15.104:6817/831948081] conn(0x7f7c18070010 msgr2=0x7f7c18072440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f7c24ff9700 1 --2- 172.21.15.104:0/2650317142 >> [v2:172.21.15.104:6816/831948081,v1:172.21.15.104:6817/831948081] conn(0x7f7c18070010 0x7f7c18072440 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7fc92b651700 1 -- 172.21.15.104:0/1507107882 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc908066370 msgr2=0x7fc908068820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7fc92b651700 1 --2- 172.21.15.104:0/1507107882 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc908066370 0x7fc908068820 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc9240cf100 tx=0x7fc920008030 comp rx=0 tx=0).stop 2023-07-14T11:53:45.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7fc92b651700 1 -- 172.21.15.104:0/1507107882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9240cb180 msgr2=0x7fc9241634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7fc92b651700 1 --2- 172.21.15.104:0/1507107882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9240cb180 0x7fc9241634c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc91400c6b0 tx=0x7fc91400c9c0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7fc929bee700 1 -- 172.21.15.104:0/1507107882 reap_dead start 2023-07-14T11:53:45.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7fc92b651700 1 -- 172.21.15.104:0/1507107882 shutdown_connections 2023-07-14T11:53:45.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7fc92b651700 1 -- 172.21.15.104:0/1507107882 >> 172.21.15.104:0/1507107882 conn(0x7fc92406e960 msgr2=0x7fc9240adb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f7c24ff9700 1 -- 172.21.15.104:0/2650317142 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c180663c0 msgr2=0x7f7c18068870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f7c24ff9700 1 --2- 172.21.15.104:0/2650317142 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c180663c0 0x7f7c18068870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7c28007b10 tx=0x7f7c28008040 comp rx=0 tx=0).stop 2023-07-14T11:53:45.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f7c24ff9700 1 -- 172.21.15.104:0/2650317142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c380ce1e0 msgr2=0x7f7c380b0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f7c24ff9700 1 --2- 172.21.15.104:0/2650317142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c380ce1e0 0x7f7c380b0ef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00e760 tx=0x7f7c2c00ea70 comp rx=0 tx=0).stop 2023-07-14T11:53:45.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f7c36d9d700 1 -- 172.21.15.104:0/2650317142 reap_dead start 2023-07-14T11:53:45.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7fc92b651700 1 -- 172.21.15.104:0/1507107882 shutdown_connections 2023-07-14T11:53:45.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f7c24ff9700 1 -- 172.21.15.104:0/2650317142 shutdown_connections 2023-07-14T11:53:45.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7fc92b651700 1 -- 172.21.15.104:0/1507107882 wait complete. 2023-07-14T11:53:45.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f7c24ff9700 1 -- 172.21.15.104:0/2650317142 >> 172.21.15.104:0/2650317142 conn(0x7f7c38060320 msgr2=0x7f7c38155e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f7c24ff9700 1 -- 172.21.15.104:0/2650317142 shutdown_connections 2023-07-14T11:53:45.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.140+0000 7f7c24ff9700 1 -- 172.21.15.104:0/2650317142 wait complete. 2023-07-14T11:53:45.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.148+0000 7f87fdf08700 1 -- 172.21.15.104:0/1853478016 --> [v2:172.21.15.121:6800/1185793020,v1:172.21.15.121:6801/1185793020] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f87f00d3930 con 0x7f87dc06ffc0 2023-07-14T11:53:45.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.148+0000 7f87e3fff700 1 -- 172.21.15.104:0/1853478016 <== osd.5 v2:172.21.15.121:6800/1185793020 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f87f00d3930 con 0x7f87dc06ffc0 2023-07-14T11:53:45.216 INFO:teuthology.orchestra.run.smithi104.stdout:55834574859 2023-07-14T11:53:45.216 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-14T11:53:45.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7f87e1ffb700 1 -- 172.21.15.104:0/1853478016 >> [v2:172.21.15.121:6800/1185793020,v1:172.21.15.121:6801/1185793020] conn(0x7f87dc06ffc0 msgr2=0x7f87dc0723f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7f87e1ffb700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.121:6800/1185793020,v1:172.21.15.121:6801/1185793020] conn(0x7f87dc06ffc0 0x7f87dc0723f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7fdb10f13700 1 -- 172.21.15.104:0/2312317733 --> [v2:172.21.15.121:6816/754390529,v1:172.21.15.121:6817/754390529] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdb0c0ca720 con 0x7fdaec070010 2023-07-14T11:53:45.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7f87e1ffb700 1 -- 172.21.15.104:0/1853478016 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87dc0662f0 msgr2=0x7f87dc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7f87e1ffb700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87dc0662f0 0x7f87dc0687a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f87f8073600 tx=0x7f87f806d040 comp rx=0 tx=0).stop 2023-07-14T11:53:45.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7f87e1ffb700 1 -- 172.21.15.104:0/1853478016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87f004f6b0 msgr2=0x7f87f004fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7f87e1ffb700 1 --2- 172.21.15.104:0/1853478016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87f004f6b0 0x7f87f004fb20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f87f000bf50 tx=0x7f87ec010a70 comp rx=0 tx=0).stop 2023-07-14T11:53:45.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7f87f7fff700 1 -- 172.21.15.104:0/1853478016 reap_dead start 2023-07-14T11:53:45.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7f87e1ffb700 1 -- 172.21.15.104:0/1853478016 shutdown_connections 2023-07-14T11:53:45.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7f87e1ffb700 1 -- 172.21.15.104:0/1853478016 >> 172.21.15.104:0/1853478016 conn(0x7f87f0051770 msgr2=0x7f87f00396a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7f87e1ffb700 1 -- 172.21.15.104:0/1853478016 shutdown_connections 2023-07-14T11:53:45.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7f87e1ffb700 1 -- 172.21.15.104:0/1853478016 wait complete. 2023-07-14T11:53:45.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7fdb037fe700 1 -- 172.21.15.104:0/2312317733 <== osd.6 v2:172.21.15.121:6816/754390529 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdb04001f90 con 0x7fdaec070010 2023-07-14T11:53:45.220 INFO:teuthology.orchestra.run.smithi104.stdout:55834574859 2023-07-14T11:53:45.221 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-14T11:53:45.221 INFO:teuthology.orchestra.run.smithi104.stdout:55834574859 2023-07-14T11:53:45.222 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-14T11:53:45.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7fdb10f13700 1 -- 172.21.15.104:0/2312317733 >> [v2:172.21.15.121:6816/754390529,v1:172.21.15.121:6817/754390529] conn(0x7fdaec070010 msgr2=0x7fdaec072440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7fdb10f13700 1 --2- 172.21.15.104:0/2312317733 >> [v2:172.21.15.121:6816/754390529,v1:172.21.15.121:6817/754390529] conn(0x7fdaec070010 0x7fdaec072440 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7fdb10f13700 1 -- 172.21.15.104:0/2312317733 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdaec066340 msgr2=0x7fdaec0687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7fdb10f13700 1 --2- 172.21.15.104:0/2312317733 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdaec066340 0x7fdaec0687f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c1f1180 tx=0x7fdaf4006be0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7fdb10f13700 1 -- 172.21.15.104:0/2312317733 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb0c0caa70 msgr2=0x7fdb0c0c2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7fdb10f13700 1 --2- 172.21.15.104:0/2312317733 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb0c0caa70 0x7fdb0c0c2700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0cb4c0 tx=0x7fdafc004770 comp rx=0 tx=0).stop 2023-07-14T11:53:45.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7fdb0ad9d700 1 -- 172.21.15.104:0/2312317733 reap_dead start 2023-07-14T11:53:45.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7fdb10f13700 1 -- 172.21.15.104:0/2312317733 shutdown_connections 2023-07-14T11:53:45.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7fdb10f13700 1 -- 172.21.15.104:0/2312317733 >> 172.21.15.104:0/2312317733 conn(0x7fdb0c060320 msgr2=0x7fdb0c15af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7fdb10f13700 1 -- 172.21.15.104:0/2312317733 shutdown_connections 2023-07-14T11:53:45.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.156+0000 7fdb10f13700 1 -- 172.21.15.104:0/2312317733 wait complete. 2023-07-14T11:53:45.225 INFO:teuthology.orchestra.run.smithi104.stdout:55834574859 2023-07-14T11:53:45.225 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-14T11:53:45.226 INFO:teuthology.orchestra.run.smithi104.stdout:55834574860 2023-07-14T11:53:45.226 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-14T11:53:45.227 INFO:teuthology.orchestra.run.smithi104.stdout:55834574859 2023-07-14T11:53:45.227 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-14T11:53:45.228 INFO:teuthology.orchestra.run.smithi104.stdout:55834574858 2023-07-14T11:53:45.229 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-14T11:53:45.230 INFO:teuthology.orchestra.run.smithi104.stdout:55834574859 2023-07-14T11:53:45.230 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-14T11:53:45.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.504+0000 7f7d04aee700 1 Processor -- start 2023-07-14T11:53:45.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.504+0000 7ff7a5a71700 1 Processor -- start 2023-07-14T11:53:45.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.504+0000 7ff7a5a71700 1 -- start start 2023-07-14T11:53:45.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.504+0000 7f7d04aee700 1 -- start start 2023-07-14T11:53:45.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff7a5a71700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff7a015fd30 0x7ff7a0160100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff7a5a71700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7a00cb490 0x7ff7a00ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff7a5a71700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a00ce8e0 con 0x7ff7a00cdf70 2023-07-14T11:53:45.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff7a5a71700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a00cb910 con 0x7ff7a015fd30 2023-07-14T11:53:45.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff7a5a71700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a00cbd40 con 0x7ff7a00cb490 2023-07-14T11:53:45.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7d04aee700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7d000afbf0 0x7f7d000c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7d04aee700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d00158c70 0x7f7d000ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7d04aee700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7d000ce720 0x7f7d000caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7d04aee700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d000cdd50 con 0x7f7d000afbf0 2023-07-14T11:53:45.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7d04aee700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d000cda20 con 0x7f7d000ce720 2023-07-14T11:53:45.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7d04aee700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d000ca440 con 0x7f7d00158c70 2023-07-14T11:53:45.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff79f7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff7a015fd30 0x7ff7a0160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff79ffff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7a00cb490 0x7ff7a00ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff79f7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff7a015fd30 0x7ff7a0160100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38284/0 (socket says 172.21.15.104:38284) 2023-07-14T11:53:45.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7cfe59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7d000afbf0 0x7f7d000c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7cfed9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7d000ce720 0x7f7d000caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f63d9c2c700 1 Processor -- start 2023-07-14T11:53:45.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7cfed9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7d000ce720 0x7f7d000caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57300/0 (socket says 172.21.15.104:57300) 2023-07-14T11:53:45.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7cfed9d700 1 -- 172.21.15.104:0/1585446445 learned_addr learned my addr 172.21.15.104:0/1585446445 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff79f7fe700 1 -- 172.21.15.104:0/1192572083 learned_addr learned my addr 172.21.15.104:0/1192572083 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff79ffff700 1 -- 172.21.15.104:0/1192572083 >> v2:172.21.15.104:3300/0 conn(0x7ff7a015fd30 msgr2=0x7ff7a0160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f63d9c2c700 1 -- start start 2023-07-14T11:53:45.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff79ffff700 1 --2- 172.21.15.104:0/1192572083 >> v2:172.21.15.104:3300/0 conn(0x7ff7a015fd30 0x7ff7a0160100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff79ffff700 1 -- 172.21.15.104:0/1192572083 >> v1:172.21.15.121:6789/0 conn(0x7ff7a00cdf70 legacy=0x7ff7a00cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff79ffff700 1 -- 172.21.15.104:0/1192572083 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a00cc020 con 0x7ff7a00cb490 2023-07-14T11:53:45.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff79f7fe700 1 --2- 172.21.15.104:0/1192572083 >> v2:172.21.15.104:3300/0 conn(0x7ff7a015fd30 0x7ff7a0160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:45.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f6cb8126700 1 Processor -- start 2023-07-14T11:53:45.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff79ffff700 1 --2- 172.21.15.104:0/1192572083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7a00cb490 0x7ff7a00ce3a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff790009850 tx=0x7ff790009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e2f80f708ba4ea9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7cfed9d700 1 -- 172.21.15.104:0/1585446445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d00158c70 msgr2=0x7f7d000ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff79e7fc700 1 -- 172.21.15.104:0/1192572083 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790011830 con 0x7ff7a00cb490 2023-07-14T11:53:45.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f6cb8126700 1 -- start start 2023-07-14T11:53:45.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7cfed9d700 1 --2- 172.21.15.104:0/1585446445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d00158c70 0x7f7d000ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7cfed9d700 1 -- 172.21.15.104:0/1585446445 >> v2:172.21.15.104:3300/0 conn(0x7f7d000afbf0 msgr2=0x7f7d000c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7cfed9d700 1 --2- 172.21.15.104:0/1585446445 >> v2:172.21.15.104:3300/0 conn(0x7f7d000afbf0 0x7f7d000c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7cfed9d700 1 -- 172.21.15.104:0/1585446445 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d000ca720 con 0x7f7d000ce720 2023-07-14T11:53:45.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f7cfe59c700 1 --2- 172.21.15.104:0/1585446445 >> v2:172.21.15.104:3300/0 conn(0x7f7d000afbf0 0x7f7d000c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:45.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f63d9c2c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f63d40c86b0 0x7f63d40cae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f63d9c2c700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d40cdd50 con 0x7f63d40caa70 2023-07-14T11:53:45.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f63d9c2c700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d40cda20 con 0x7f63d40ce1e0 2023-07-14T11:53:45.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f63d9c2c700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d40cb380 con 0x7f63d40c86b0 2023-07-14T11:53:45.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff79e7fc700 1 -- 172.21.15.104:0/1192572083 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff790011990 con 0x7ff7a00cb490 2023-07-14T11:53:45.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f63d2ffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f63d40c86b0 0x7f63d40cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7f63d37fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f63d40caa70 0x7f63d4158c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42506/0 (socket says 172.21.15.104:42506) 2023-07-14T11:53:45.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f63d37fe700 1 -- 172.21.15.104:0/2488441442 learned_addr learned my addr 172.21.15.104:0/2488441442 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:45.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f7cfed9d700 1 --2- 172.21.15.104:0/1585446445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d000ce720 0x7f7d000caa70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d000c1ae0 tx=0x7f7cf0009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9221be8b827bb91 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f63d2ffd700 1 -- 172.21.15.104:0/2488441442 >> v1:172.21.15.121:6789/0 conn(0x7f63d40ce1e0 legacy=0x7f63d406bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff79e7fc700 1 -- 172.21.15.104:0/1192572083 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790011d40 con 0x7ff7a00cb490 2023-07-14T11:53:45.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff7a5a71700 1 -- 172.21.15.104:0/1192572083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7a00cb490 msgr2=0x7ff7a00ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.508+0000 7ff7a5a71700 1 --2- 172.21.15.104:0/1192572083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7a00cb490 0x7ff7a00ce3a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff790009850 tx=0x7ff790009b60 comp rx=0 tx=0).stop 2023-07-14T11:53:45.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f2d00cb3700 1 Processor -- start 2023-07-14T11:53:45.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f63d2ffd700 1 -- 172.21.15.104:0/2488441442 >> v1:172.21.15.104:6789/0 conn(0x7f63d40caa70 legacy=0x7f63d4158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f63d27fc700 1 -- 172.21.15.104:0/2488441442 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190437469 0 0) 0x7f63d40cdd50 con 0x7f63d40caa70 2023-07-14T11:53:45.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7ff7a5a71700 1 -- 172.21.15.104:0/1192572083 shutdown_connections 2023-07-14T11:53:45.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7ff7a5a71700 1 --2- 172.21.15.104:0/1192572083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7a00cb490 0x7ff7a00ce3a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7ff7a5a71700 1 --2- 172.21.15.104:0/1192572083 >> v2:172.21.15.104:3300/0 conn(0x7ff7a015fd30 0x7ff7a0160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f6cb5ec2700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f6cb00caa70 0x7f6cb0158c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42510/0 (socket says 172.21.15.104:42510) 2023-07-14T11:53:45.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f7cfd59a700 1 -- 172.21.15.104:0/1585446445 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0011830 con 0x7f7d000ce720 2023-07-14T11:53:45.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f7cfd59a700 1 -- 172.21.15.104:0/1585446445 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf0011990 con 0x7f7d000ce720 2023-07-14T11:53:45.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f7cfd59a700 1 -- 172.21.15.104:0/1585446445 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0011d00 con 0x7f7d000ce720 2023-07-14T11:53:45.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f7d04aee700 1 -- 172.21.15.104:0/1585446445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d000ce720 msgr2=0x7f7d000caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f7d04aee700 1 --2- 172.21.15.104:0/1585446445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d000ce720 0x7f7d000caa70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d000c1ae0 tx=0x7f7cf0009bf0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7ff7a5a71700 1 -- 172.21.15.104:0/1192572083 >> 172.21.15.104:0/1192572083 conn(0x7ff7a0060320 msgr2=0x7ff7a00606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7ff7a5a71700 1 -- 172.21.15.104:0/1192572083 shutdown_connections 2023-07-14T11:53:45.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7ff7a5a71700 1 -- 172.21.15.104:0/1192572083 wait complete. 2023-07-14T11:53:45.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f6cb5ec2700 1 -- 172.21.15.104:0/3320130010 learned_addr learned my addr 172.21.15.104:0/3320130010 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:45.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f6cb8126700 1 --2- 172.21.15.104:0/3320130010 >> v2:172.21.15.121:3300/0 conn(0x7f6cb00ce1e0 0x7f6cb006bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f6cb8126700 1 -- 172.21.15.104:0/3320130010 --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb00cdd50 con 0x7f6cb00caa70 2023-07-14T11:53:45.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f63d2ffd700 1 -- 172.21.15.104:0/2488441442 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d40ca440 con 0x7f63d40c86b0 2023-07-14T11:53:45.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f63d2ffd700 1 --2- 172.21.15.104:0/2488441442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63d40c86b0 0x7f63d40cae40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009850 tx=0x7f63c8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b402524f393936 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f6cb66c3700 1 --2- 172.21.15.104:0/3320130010 >> v2:172.21.15.121:3300/0 conn(0x7f6cb00ce1e0 0x7f6cb006bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f6cb4ec0700 1 -- 172.21.15.104:0/3320130010 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3067201753 0 0) 0x7f6cb00cdd50 con 0x7f6cb00caa70 2023-07-14T11:53:45.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f6cb8126700 1 -- 172.21.15.104:0/3320130010 --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb00cda20 con 0x7f6cb00c86b0 2023-07-14T11:53:45.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7ff7a5a71700 1 Processor -- start 2023-07-14T11:53:45.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f6cb8126700 1 -- 172.21.15.104:0/3320130010 --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb00ca440 con 0x7f6cb00ce1e0 2023-07-14T11:53:45.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f7d04aee700 1 -- 172.21.15.104:0/1585446445 shutdown_connections 2023-07-14T11:53:45.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f7d04aee700 1 --2- 172.21.15.104:0/1585446445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d000ce720 0x7f7d000caa70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f7d04aee700 1 --2- 172.21.15.104:0/1585446445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d00158c70 0x7f7d000ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f7d04aee700 1 --2- 172.21.15.104:0/1585446445 >> v2:172.21.15.104:3300/0 conn(0x7f7d000afbf0 0x7f7d000c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f7d04aee700 1 -- 172.21.15.104:0/1585446445 >> 172.21.15.104:0/1585446445 conn(0x7f7d00060320 msgr2=0x7f7d000606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f7d04aee700 1 -- 172.21.15.104:0/1585446445 shutdown_connections 2023-07-14T11:53:45.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f7d04aee700 1 -- 172.21.15.104:0/1585446445 wait complete. 2023-07-14T11:53:45.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f7d04aee700 1 Processor -- start 2023-07-14T11:53:45.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7ff7a5a71700 1 -- start start 2023-07-14T11:53:45.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff7a5a71700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7a00cb490 0x7ff7a00c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff7a5a71700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7a00cdf70 0x7ff7a01621e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff7a5a71700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7a015fd30 0x7ff7a0162720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff7a5a71700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a01f9da0 con 0x7ff7a00cb490 2023-07-14T11:53:45.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff7a5a71700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a00c52d0 con 0x7ff7a015fd30 2023-07-14T11:53:45.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff7a5a71700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a00c55b0 con 0x7ff7a00cdf70 2023-07-14T11:53:45.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff79ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7a015fd30 0x7ff7a0162720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff79ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7a015fd30 0x7ff7a0162720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57326/0 (socket says 172.21.15.104:57326) 2023-07-14T11:53:45.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff79ffff700 1 -- 172.21.15.104:0/939615154 learned_addr learned my addr 172.21.15.104:0/939615154 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff79f7fe700 1 --2- 172.21.15.104:0/939615154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7a00cb490 0x7ff7a00c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff79effd700 1 --2- 172.21.15.104:0/939615154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7a00cdf70 0x7ff7a01621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f63d27fc700 1 -- 172.21.15.104:0/2488441442 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8004230 con 0x7f63d40c86b0 2023-07-14T11:53:45.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f63d27fc700 1 -- 172.21.15.104:0/2488441442 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c8004390 con 0x7f63d40c86b0 2023-07-14T11:53:45.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f63d27fc700 1 -- 172.21.15.104:0/2488441442 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c80036c0 con 0x7f63d40c86b0 2023-07-14T11:53:45.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f63d9c2c700 1 -- 172.21.15.104:0/2488441442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63d40c86b0 msgr2=0x7f63d40cae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f63d9c2c700 1 --2- 172.21.15.104:0/2488441442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63d40c86b0 0x7f63d40cae40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009850 tx=0x7f63c8009b60 comp rx=0 tx=0).stop 2023-07-14T11:53:45.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f63d9c2c700 1 -- 172.21.15.104:0/2488441442 shutdown_connections 2023-07-14T11:53:45.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f63d9c2c700 1 --2- 172.21.15.104:0/2488441442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63d40c86b0 0x7f63d40cae40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f63d9c2c700 1 -- 172.21.15.104:0/2488441442 >> 172.21.15.104:0/2488441442 conn(0x7f63d4060320 msgr2=0x7f63d40606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f63d9c2c700 1 -- 172.21.15.104:0/2488441442 shutdown_connections 2023-07-14T11:53:45.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7fa2fa43e700 1 Processor -- start 2023-07-14T11:53:45.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f2d00cb3700 1 -- start start 2023-07-14T11:53:45.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f2d00cb3700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2cfc15ffa0 0x7f2cfc06d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f2d00cb3700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2cfc06dc50 0x7f2cfc0caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f2d00cb3700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cfc0ca8a0 con 0x7f2cfc160370 2023-07-14T11:53:45.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f2d00cb3700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cfc0cbcd0 con 0x7f2cfc06dc50 2023-07-14T11:53:45.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f2d00cb3700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cfc0cbf70 con 0x7f2cfc15ffa0 2023-07-14T11:53:45.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f2cfa59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2cfc15ffa0 0x7f2cfc06d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f2cfa59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2cfc15ffa0 0x7f2cfc06d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57316/0 (socket says 172.21.15.104:57316) 2023-07-14T11:53:45.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f2cfa59c700 1 -- 172.21.15.104:0/159071053 learned_addr learned my addr 172.21.15.104:0/159071053 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f2cf9d9b700 1 --2- 172.21.15.104:0/159071053 >> v2:172.21.15.104:3300/0 conn(0x7f2cfc06dc50 0x7f2cfc0caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f2cfa59c700 1 -- 172.21.15.104:0/159071053 >> v2:172.21.15.104:3300/0 conn(0x7f2cfc06dc50 msgr2=0x7f2cfc0caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f2cfa59c700 1 --2- 172.21.15.104:0/159071053 >> v2:172.21.15.104:3300/0 conn(0x7f2cfc06dc50 0x7f2cfc0caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f2cfa59c700 1 -- 172.21.15.104:0/159071053 >> v1:172.21.15.104:6789/0 conn(0x7f2cfc160370 legacy=0x7f2cfc0ce340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T11:53:45.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7fa2fa43e700 1 -- start start 2023-07-14T11:53:45.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7fa2fa43e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa2f415ffa0 0x7fa2f406d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7fa2fa43e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f40ca8a0 con 0x7fa2f40cb180 2023-07-14T11:53:45.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7fa2fa43e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f40cbcd0 con 0x7fa2f406dc50 2023-07-14T11:53:45.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7fa2fa43e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f40cbf70 con 0x7fa2f415ffa0 2023-07-14T11:53:45.521 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff79ffff700 1 -- 172.21.15.104:0/939615154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7a00cdf70 msgr2=0x7ff7a01621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.521 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff79ffff700 1 --2- 172.21.15.104:0/939615154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7a00cdf70 0x7ff7a01621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.521 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff79ffff700 1 -- 172.21.15.104:0/939615154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7a00cb490 msgr2=0x7ff7a00c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff79ffff700 1 --2- 172.21.15.104:0/939615154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7a00cb490 0x7ff7a00c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff79ffff700 1 -- 172.21.15.104:0/939615154 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff798009b30 con 0x7ff7a015fd30 2023-07-14T11:53:45.523 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f63d9c2c700 1 -- 172.21.15.104:0/2488441442 wait complete. 2023-07-14T11:53:45.523 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7fa2f37fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa2f406dc50 0x7fa2f40caa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:56588/0 (socket says 172.21.15.104:56588) 2023-07-14T11:53:45.524 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7fa2f37fe700 1 -- 172.21.15.104:0/754146621 learned_addr learned my addr 172.21.15.104:0/754146621 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:45.524 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f2cfa59c700 1 -- 172.21.15.104:0/159071053 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc0cc250 con 0x7f2cfc15ffa0 2023-07-14T11:53:45.524 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f63d9c2c700 1 Processor -- start 2023-07-14T11:53:45.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f7d04aee700 1 -- start start 2023-07-14T11:53:45.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7ff79ffff700 1 --2- 172.21.15.104:0/939615154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7a015fd30 0x7ff7a0162720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff7900090b0 tx=0x7ff790004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7d04aee700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d000afbf0 0x7f7d00168c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.512+0000 7f6cb4ec0700 1 -- 172.21.15.104:0/3320130010 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c9c0035b0 con 0x7f6cb00caa70 2023-07-14T11:53:45.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb4ec0700 1 -- 172.21.15.104:0/3320130010 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3926283069 0 0) 0x7f6c9c0035b0 con 0x7f6cb00caa70 2023-07-14T11:53:45.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb4ec0700 1 -- 172.21.15.104:0/3320130010 >> v2:172.21.15.121:3300/0 conn(0x7f6cb00ce1e0 msgr2=0x7f6cb006bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb4ec0700 1 --2- 172.21.15.104:0/3320130010 >> v2:172.21.15.121:3300/0 conn(0x7f6cb00ce1e0 0x7f6cb006bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb4ec0700 1 -- 172.21.15.104:0/3320130010 >> v1:172.21.15.121:6789/0 conn(0x7f6cb00c86b0 legacy=0x7f6cb00cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb4ec0700 1 -- 172.21.15.104:0/3320130010 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb00ca720 con 0x7f6cb00caa70 2023-07-14T11:53:45.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb4ec0700 1 -- 172.21.15.104:0/3320130010 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f6cac001670 con 0x7f6cb00caa70 2023-07-14T11:53:45.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb4ec0700 1 -- 172.21.15.104:0/3320130010 >> v1:172.21.15.104:6789/0 conn(0x7f6cb00caa70 legacy=0x7f6cb0158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb4ec0700 1 --2- 172.21.15.104:0/3320130010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c9c003d80 0x7f6c9c0061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb4ec0700 1 --2- 172.21.15.104:0/3320130010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c9c0066e0 0x7f6c9c008b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb4ec0700 1 --2- 172.21.15.104:0/3320130010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c9c009400 0x7f6c9c00b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb4ec0700 1 -- 172.21.15.104:0/3320130010 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb00ca440 con 0x7f6c9c009400 2023-07-14T11:53:45.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb4ec0700 1 -- 172.21.15.104:0/3320130010 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c00c190 con 0x7f6c9c0066e0 2023-07-14T11:53:45.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb4ec0700 1 -- 172.21.15.104:0/3320130010 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c00c350 con 0x7f6c9c003d80 2023-07-14T11:53:45.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb66c3700 1 --2- 172.21.15.104:0/3320130010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c9c009400 0x7f6c9c00b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb66c3700 1 -- 172.21.15.104:0/3320130010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c9c003d80 msgr2=0x7f6c9c0061a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T11:53:45.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb66c3700 1 --2- 172.21.15.104:0/3320130010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c9c003d80 0x7f6c9c0061a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb66c3700 1 -- 172.21.15.104:0/3320130010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c9c0066e0 msgr2=0x7f6c9c008b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T11:53:45.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb66c3700 1 --2- 172.21.15.104:0/3320130010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c9c0066e0 0x7f6c9c008b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb66c3700 1 -- 172.21.15.104:0/3320130010 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb00ca720 con 0x7f6c9c009400 2023-07-14T11:53:45.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb66c3700 1 -- 172.21.15.104:0/3320130010 reap_dead start 2023-07-14T11:53:45.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.516+0000 7f6cb66c3700 1 --2- 172.21.15.104:0/3320130010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c9c009400 0x7f6c9c00b8b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800ac30 tx=0x7f6c9c003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8c0e2fe1c27e09 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f6cb4ec0700 1 -- 172.21.15.104:0/3320130010 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca800f040 con 0x7f6c9c009400 2023-07-14T11:53:45.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f6cb4ec0700 1 -- 172.21.15.104:0/3320130010 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca8008bf0 con 0x7f6c9c009400 2023-07-14T11:53:45.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f6cb4ec0700 1 -- 172.21.15.104:0/3320130010 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8037410 con 0x7f6c9c009400 2023-07-14T11:53:45.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7fa2f3fff700 1 --2- 172.21.15.104:0/754146621 >> v2:172.21.15.104:3300/0 conn(0x7fa2f415ffa0 0x7fa2f406d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f2cfa59c700 1 --2- 172.21.15.104:0/159071053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc15ffa0 0x7f2cfc06d710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0caa40 tx=0x7f2cf400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34705ddaa882a9bf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7d04aee700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d000ce720 0x7f7d00169140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.532 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7ff787fff700 1 -- 172.21.15.104:0/939615154 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790004610 con 0x7ff7a015fd30 2023-07-14T11:53:45.532 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7ff787fff700 1 -- 172.21.15.104:0/939615154 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79001e070 con 0x7ff7a015fd30 2023-07-14T11:53:45.532 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7ff787fff700 1 -- 172.21.15.104:0/939615154 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7900038e0 con 0x7ff7a015fd30 2023-07-14T11:53:45.532 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7ff7a5a71700 1 -- 172.21.15.104:0/939615154 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7900094b0 con 0x7ff7a015fd30 2023-07-14T11:53:45.532 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7ff7a5a71700 1 -- 172.21.15.104:0/939615154 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a01f90e0 con 0x7ff7a015fd30 2023-07-14T11:53:45.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7fa2f3fff700 1 -- 172.21.15.104:0/754146621 >> v1:172.21.15.121:6789/0 conn(0x7fa2f406dc50 legacy=0x7fa2f40caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7fa2f3fff700 1 -- 172.21.15.104:0/754146621 >> v1:172.21.15.104:6789/0 conn(0x7fa2f40cb180 legacy=0x7fa2f40ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7fa2f3fff700 1 -- 172.21.15.104:0/754146621 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f40cc250 con 0x7fa2f415ffa0 2023-07-14T11:53:45.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7cfe59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d000afbf0 0x7f7d00168c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7cfe59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d000afbf0 0x7f7d00168c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57332/0 (socket says 172.21.15.104:57332) 2023-07-14T11:53:45.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7cfdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d000ce720 0x7f7d00169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7cfdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d000ce720 0x7f7d00169140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38370/0 (socket says 172.21.15.104:38370) 2023-07-14T11:53:45.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f2cf959a700 1 -- 172.21.15.104:0/159071053 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400e070 con 0x7f2cfc15ffa0 2023-07-14T11:53:45.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f2cf959a700 1 -- 172.21.15.104:0/159071053 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf4003a40 con 0x7f2cfc15ffa0 2023-07-14T11:53:45.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f2cf959a700 1 -- 172.21.15.104:0/159071053 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4003df0 con 0x7f2cfc15ffa0 2023-07-14T11:53:45.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7d04aee700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d00158c70 0x7f7d00144a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7d04aee700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0014aac0 con 0x7f7d000ce720 2023-07-14T11:53:45.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7d04aee700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d000c4ab0 con 0x7f7d000afbf0 2023-07-14T11:53:45.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7d04aee700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d000c47d0 con 0x7f7d00158c70 2023-07-14T11:53:45.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7cfe59c700 1 -- 172.21.15.104:0/2976378799 learned_addr learned my addr 172.21.15.104:0/2976378799 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f63d9c2c700 1 -- start start 2023-07-14T11:53:45.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f63d9c2c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63d40c86b0 0x7f63d40c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f63d9c2c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63d40ce1e0 0x7f63d414a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f6cb8126700 1 -- 172.21.15.104:0/3320130010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c9c009400 msgr2=0x7f6c9c00b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f6cb8126700 1 --2- 172.21.15.104:0/3320130010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c9c009400 0x7f6c9c00b8b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800ac30 tx=0x7f6c9c003860 comp rx=0 tx=0).stop 2023-07-14T11:53:45.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7fa2f3fff700 1 --2- 172.21.15.104:0/754146621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f415ffa0 0x7fa2f406d710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40097d0 tx=0x7fa2e4009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902880b0e5922594 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7cfed9d700 1 --2- 172.21.15.104:0/2976378799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d00158c70 0x7f7d00144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7ff7a5a71700 1 -- 172.21.15.104:0/939615154 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff780005310 con 0x7ff7a015fd30 2023-07-14T11:53:45.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f63d37fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63d40c86b0 0x7f63d40c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f63d37fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63d40c86b0 0x7f63d40c43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:51620/0 (socket says 172.21.15.104:51620) 2023-07-14T11:53:45.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f63d9c2c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63d414ab20 0x7f63d40c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7cfdd9b700 1 -- 172.21.15.104:0/2976378799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d00158c70 msgr2=0x7f7d00144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7cfdd9b700 1 --2- 172.21.15.104:0/2976378799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d00158c70 0x7f7d00144a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7cfdd9b700 1 -- 172.21.15.104:0/2976378799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d000afbf0 msgr2=0x7f7d00168c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7cfdd9b700 1 --2- 172.21.15.104:0/2976378799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d000afbf0 0x7f7d00168c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7cfdd9b700 1 -- 172.21.15.104:0/2976378799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf40094b0 con 0x7f7d000ce720 2023-07-14T11:53:45.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f63d37fe700 1 -- 172.21.15.104:0/4224977765 learned_addr learned my addr 172.21.15.104:0/4224977765 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f63d2ffd700 1 --2- 172.21.15.104:0/4224977765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63d40ce1e0 0x7f63d414a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f2d00cb3700 1 -- 172.21.15.104:0/159071053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc15ffa0 msgr2=0x7f2cfc06d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f2d00cb3700 1 --2- 172.21.15.104:0/159071053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc15ffa0 0x7f2cfc06d710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0caa40 tx=0x7f2cf400b330 comp rx=0 tx=0).stop 2023-07-14T11:53:45.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f63d9c2c700 1 -- 172.21.15.104:0/4224977765 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d414afc0 con 0x7f63d414ab20 2023-07-14T11:53:45.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f63d3fff700 1 --2- 172.21.15.104:0/4224977765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63d414ab20 0x7f63d40c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f63d9c2c700 1 -- 172.21.15.104:0/4224977765 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d40c4920 con 0x7f63d40ce1e0 2023-07-14T11:53:45.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f63d9c2c700 1 -- 172.21.15.104:0/4224977765 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d40c27d0 con 0x7f63d40c86b0 2023-07-14T11:53:45.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f2d00cb3700 1 -- 172.21.15.104:0/159071053 shutdown_connections 2023-07-14T11:53:45.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f2d00cb3700 1 --2- 172.21.15.104:0/159071053 >> v2:172.21.15.104:3300/0 conn(0x7f2cfc06dc50 0x7f2cfc0caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f2d00cb3700 1 --2- 172.21.15.104:0/159071053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc15ffa0 0x7f2cfc06d710 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f2d00cb3700 1 -- 172.21.15.104:0/159071053 >> 172.21.15.104:0/159071053 conn(0x7f2cfc06e960 msgr2=0x7f2cfc06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f7cfe59c700 1 --2- 172.21.15.104:0/2976378799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d000afbf0 0x7f7d00168c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:45.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f6cb8126700 1 -- 172.21.15.104:0/3320130010 shutdown_connections 2023-07-14T11:53:45.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f6cb8126700 1 --2- 172.21.15.104:0/3320130010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c9c009400 0x7f6c9c00b8b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f6cb8126700 1 -- 172.21.15.104:0/3320130010 >> 172.21.15.104:0/3320130010 conn(0x7f6cb0060320 msgr2=0x7f6cb00606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f2d00cb3700 1 -- 172.21.15.104:0/159071053 shutdown_connections 2023-07-14T11:53:45.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f6cb8126700 1 -- 172.21.15.104:0/3320130010 shutdown_connections 2023-07-14T11:53:45.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f7cfdd9b700 1 --2- 172.21.15.104:0/2976378799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d000ce720 0x7f7d00169140 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800bda0 tx=0x7f7cf800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f63d37fe700 1 -- 172.21.15.104:0/4224977765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63d40ce1e0 msgr2=0x7f63d414a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f63d37fe700 1 --2- 172.21.15.104:0/4224977765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63d40ce1e0 0x7f63d414a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f63d37fe700 1 -- 172.21.15.104:0/4224977765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63d414ab20 msgr2=0x7f63d40c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f63d37fe700 1 --2- 172.21.15.104:0/4224977765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63d414ab20 0x7f63d40c1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f63d37fe700 1 -- 172.21.15.104:0/4224977765 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c80094b0 con 0x7f63d40c86b0 2023-07-14T11:53:45.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7fa2f2ffd700 1 -- 172.21.15.104:0/754146621 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4004230 con 0x7fa2f415ffa0 2023-07-14T11:53:45.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7fa2f2ffd700 1 -- 172.21.15.104:0/754146621 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e4004390 con 0x7fa2f415ffa0 2023-07-14T11:53:45.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7fa2f2ffd700 1 -- 172.21.15.104:0/754146621 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e40036c0 con 0x7fa2f415ffa0 2023-07-14T11:53:45.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7fa2fa43e700 1 -- 172.21.15.104:0/754146621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f415ffa0 msgr2=0x7fa2f406d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7fa2fa43e700 1 --2- 172.21.15.104:0/754146621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f415ffa0 0x7fa2f406d710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40097d0 tx=0x7fa2e4009ae0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f7ceeffd700 1 -- 172.21.15.104:0/2976378799 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf800cc90 con 0x7f7d000ce720 2023-07-14T11:53:45.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7fa2fa43e700 1 -- 172.21.15.104:0/754146621 shutdown_connections 2023-07-14T11:53:45.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7fa2fa43e700 1 --2- 172.21.15.104:0/754146621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f415ffa0 0x7fa2f406d710 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7fa2fa43e700 1 -- 172.21.15.104:0/754146621 >> 172.21.15.104:0/754146621 conn(0x7fa2f406e960 msgr2=0x7fa2f406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f63d37fe700 1 --2- 172.21.15.104:0/4224977765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63d40c86b0 0x7f63d40c43e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f63c400c520 tx=0x7f63c400c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f6cb8126700 1 -- 172.21.15.104:0/3320130010 wait complete. 2023-07-14T11:53:45.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7ff787fff700 1 -- 172.21.15.104:0/939615154 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff790003a40 con 0x7ff7a015fd30 2023-07-14T11:53:45.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f7d04aee700 1 -- 172.21.15.104:0/2976378799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf0009570 con 0x7f7d000ce720 2023-07-14T11:53:45.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f7d04aee700 1 -- 172.21.15.104:0/2976378799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0014a750 con 0x7f7d000ce720 2023-07-14T11:53:45.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f7ceeffd700 1 -- 172.21.15.104:0/2976378799 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf800ce10 con 0x7f7d000ce720 2023-07-14T11:53:45.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f7ceeffd700 1 -- 172.21.15.104:0/2976378799 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8014680 con 0x7f7d000ce720 2023-07-14T11:53:45.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f63bbfff700 1 -- 172.21.15.104:0/4224977765 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4004ca0 con 0x7f63d40c86b0 2023-07-14T11:53:45.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f63bbfff700 1 -- 172.21.15.104:0/4224977765 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c4004e00 con 0x7f63d40c86b0 2023-07-14T11:53:45.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f63bbfff700 1 -- 172.21.15.104:0/4224977765 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4007310 con 0x7f63d40c86b0 2023-07-14T11:53:45.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f63d9c2c700 1 -- 172.21.15.104:0/4224977765 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d40b1aa0 con 0x7f63d40c86b0 2023-07-14T11:53:45.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f63d9c2c700 1 -- 172.21.15.104:0/4224977765 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d40b2060 con 0x7f63d40c86b0 2023-07-14T11:53:45.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.520+0000 7f2d00cb3700 1 -- 172.21.15.104:0/159071053 wait complete. 2023-07-14T11:53:45.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f2d00cb3700 1 Processor -- start 2023-07-14T11:53:45.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7ff787fff700 1 --2- 172.21.15.104:0/939615154 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff788066470 0x7ff788068920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f6cb8126700 1 Processor -- start 2023-07-14T11:53:45.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7ff787fff700 1 -- 172.21.15.104:0/939615154 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff790020070 con 0x7ff7a015fd30 2023-07-14T11:53:45.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7fa2fa43e700 1 -- 172.21.15.104:0/754146621 shutdown_connections 2023-07-14T11:53:45.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7f6cb8126700 1 -- start start 2023-07-14T11:53:45.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7ff79f7fe700 1 --2- 172.21.15.104:0/939615154 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff788066470 0x7ff788068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f63d9c2c700 1 -- 172.21.15.104:0/4224977765 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b4005310 con 0x7f63d40c86b0 2023-07-14T11:53:45.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f7d04aee700 1 -- 172.21.15.104:0/2976378799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce4005310 con 0x7f7d000ce720 2023-07-14T11:53:45.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f77f125a700 1 Processor -- start 2023-07-14T11:53:45.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f77f125a700 1 -- start start 2023-07-14T11:53:45.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb8126700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c9c009400 0x7f6cb00c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb8126700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb00c2e90 0x7f6cb014c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb8126700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cb0159390 0x7f6cb00cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb8126700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb00c4a30 con 0x7f6cb0159390 2023-07-14T11:53:45.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb8126700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb014c8f0 con 0x7f6cb00c2e90 2023-07-14T11:53:45.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb8126700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb00c3450 con 0x7f6c9c009400 2023-07-14T11:53:45.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7ff79f7fe700 1 --2- 172.21.15.104:0/939615154 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff788066470 0x7ff788068920 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff7a01e8800 tx=0x7ff79800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f450904a700 1 Processor -- start 2023-07-14T11:53:45.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f450904a700 1 -- start start 2023-07-14T11:53:45.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.524+0000 7fa2fa43e700 1 -- 172.21.15.104:0/754146621 wait complete. 2023-07-14T11:53:45.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb5ec2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c9c009400 0x7f6cb00c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb5ec2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c9c009400 0x7f6cb00c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:51622/0 (socket says 172.21.15.104:51622) 2023-07-14T11:53:45.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb5ec2700 1 -- 172.21.15.104:0/1960711854 learned_addr learned my addr 172.21.15.104:0/1960711854 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb56c1700 1 --2- 172.21.15.104:0/1960711854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb00c2e90 0x7f6cb014c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f2d00cb3700 1 -- start start 2023-07-14T11:53:45.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7fa2fa43e700 1 Processor -- start 2023-07-14T11:53:45.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f450904a700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f450415fd30 0x7f4504160100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb66c3700 1 --2- 172.21.15.104:0/1960711854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cb0159390 0x7f6cb00cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb56c1700 1 -- 172.21.15.104:0/1960711854 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c9c009400 msgr2=0x7f6cb00c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb56c1700 1 --2- 172.21.15.104:0/1960711854 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c9c009400 0x7f6cb00c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb56c1700 1 -- 172.21.15.104:0/1960711854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cb0159390 msgr2=0x7f6cb00cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb56c1700 1 --2- 172.21.15.104:0/1960711854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cb0159390 0x7f6cb00cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb56c1700 1 -- 172.21.15.104:0/1960711854 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb00c3730 con 0x7f6cb00c2e90 2023-07-14T11:53:45.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7fa2fa43e700 1 -- start start 2023-07-14T11:53:45.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f450904a700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45040cb490 0x7f45040ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f77f125a700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f77e40e12a0 0x7f77e4037b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f77f125a700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77e40e1670 0x7f77e40e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f77f125a700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e40e0af0 con 0x7f77e40e4f50 2023-07-14T11:53:45.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f77f125a700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e40e45f0 con 0x7f77e40e12a0 2023-07-14T11:53:45.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f77f125a700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4038080 con 0x7f77e40e1670 2023-07-14T11:53:45.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f450904a700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45040ce8e0 con 0x7f45040cdf70 2023-07-14T11:53:45.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f450904a700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45040cb910 con 0x7f450415fd30 2023-07-14T11:53:45.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f450904a700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45040cbd40 con 0x7f45040cb490 2023-07-14T11:53:45.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f450359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45040cb490 0x7f45040ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f2d00cb3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc06dc50 0x7f2cfc1489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f2d00cb3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cfc15ffa0 0x7f2cfc149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f2d00cb3700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2cfc160370 0x7f2cfc149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f2d00cb3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc148f30 con 0x7f2cfc15ffa0 2023-07-14T11:53:45.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f2d00cb3700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc0b1900 con 0x7f2cfc06dc50 2023-07-14T11:53:45.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f2d00cb3700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc0b1be0 con 0x7f2cfc160370 2023-07-14T11:53:45.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f450359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45040cb490 0x7f45040ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:51638/0 (socket says 172.21.15.104:51638) 2023-07-14T11:53:45.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f450359e700 1 -- 172.21.15.104:0/3817120311 learned_addr learned my addr 172.21.15.104:0/3817120311 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f7ceeffd700 1 -- 172.21.15.104:0/2976378799 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7cf801c360 con 0x7f7d000ce720 2023-07-14T11:53:45.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f7ceeffd700 1 --2- 172.21.15.104:0/2976378799 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7ce00663c0 0x7f7ce0068870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f7ceeffd700 1 -- 172.21.15.104:0/2976378799 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7cf8083e30 con 0x7f7d000ce720 2023-07-14T11:53:45.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f63bbfff700 1 -- 172.21.15.104:0/4224977765 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f63c4007470 con 0x7f63d40c86b0 2023-07-14T11:53:45.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f63bbfff700 1 --2- 172.21.15.104:0/4224977765 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f63c00663f0 0x7f63c00688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f63bbfff700 1 -- 172.21.15.104:0/4224977765 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f63c4083b60 con 0x7f63d40c86b0 2023-07-14T11:53:45.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f4502d9d700 1 --2- 172.21.15.104:0/3817120311 >> v2:172.21.15.121:3300/0 conn(0x7f450415fd30 0x7f4504160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f2cfa59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc06dc50 0x7f2cfc1489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f2cfa59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc06dc50 0x7f2cfc1489f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57382/0 (socket says 172.21.15.104:57382) 2023-07-14T11:53:45.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f2cfa59c700 1 -- 172.21.15.104:0/1595059729 learned_addr learned my addr 172.21.15.104:0/1595059729 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f6cb56c1700 1 --2- 172.21.15.104:0/1960711854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb00c2e90 0x7f6cb014c3b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cb004f580 tx=0x7f6ca000c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7ff787fff700 1 -- 172.21.15.104:0/939615154 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff790053180 con 0x7ff7a015fd30 2023-07-14T11:53:45.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f2cfad9d700 1 --2- 172.21.15.104:0/1595059729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2cfc160370 0x7f2cfc149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f450359e700 1 -- 172.21.15.104:0/3817120311 >> v2:172.21.15.121:3300/0 conn(0x7f450415fd30 msgr2=0x7f4504160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f450359e700 1 --2- 172.21.15.104:0/3817120311 >> v2:172.21.15.121:3300/0 conn(0x7f450415fd30 0x7f4504160100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f77f0a59700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f77e40e4f50 0x7f77e4036770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42536/0 (socket says 172.21.15.104:42536) 2023-07-14T11:53:45.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.528+0000 7f77f0a59700 1 -- 172.21.15.104:0/1704840984 learned_addr learned my addr 172.21.15.104:0/1704840984 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:45.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f450359e700 1 -- 172.21.15.104:0/3817120311 >> v1:172.21.15.104:6789/0 conn(0x7f45040cdf70 legacy=0x7f45040cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f63d2ffd700 1 --2- 172.21.15.104:0/4224977765 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f63c00663f0 0x7f63c00688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f77ebfff700 1 --2- 172.21.15.104:0/1704840984 >> v2:172.21.15.121:3300/0 conn(0x7f77e40e12a0 0x7f77e4037b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2fa43e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f406dc50 0x7fa2f41fde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2fa43e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f40cb180 0x7fa2f41f2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2fa43e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f415ffa0 0x7fa2f42022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2fa43e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f4202810 con 0x7fa2f40cb180 2023-07-14T11:53:45.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2fa43e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f41f2740 con 0x7fa2f406dc50 2023-07-14T11:53:45.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2fa43e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f41f2a20 con 0x7fa2f415ffa0 2023-07-14T11:53:45.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f450359e700 1 -- 172.21.15.104:0/3817120311 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45040cc020 con 0x7f45040cb490 2023-07-14T11:53:45.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f77eb7fe700 1 --2- 172.21.15.104:0/1704840984 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77e40e1670 0x7f77e40e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f2cf9d9b700 1 --2- 172.21.15.104:0/1595059729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cfc15ffa0 0x7f2cfc149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f2cfa59c700 1 -- 172.21.15.104:0/1595059729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2cfc160370 msgr2=0x7f2cfc149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f2cfa59c700 1 --2- 172.21.15.104:0/1595059729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2cfc160370 0x7f2cfc149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f2cfa59c700 1 -- 172.21.15.104:0/1595059729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cfc15ffa0 msgr2=0x7f2cfc149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f2cfa59c700 1 --2- 172.21.15.104:0/1595059729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cfc15ffa0 0x7f2cfc149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f2cfa59c700 1 -- 172.21.15.104:0/1595059729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf4009b30 con 0x7f2cfc06dc50 2023-07-14T11:53:45.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f6ca67fc700 1 -- 172.21.15.104:0/1960711854 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca00052d0 con 0x7f6cb00c2e90 2023-07-14T11:53:45.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2f3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f406dc50 0x7fa2f41fde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2f3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f406dc50 0x7fa2f41fde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57388/0 (socket says 172.21.15.104:57388) 2023-07-14T11:53:45.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2f3fff700 1 -- 172.21.15.104:0/4266937034 learned_addr learned my addr 172.21.15.104:0/4266937034 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2f37fe700 1 --2- 172.21.15.104:0/4266937034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f40cb180 0x7fa2f41f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f6cb8126700 1 -- 172.21.15.104:0/1960711854 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb00c3a10 con 0x7f6cb00c2e90 2023-07-14T11:53:45.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f2cfa59c700 1 --2- 172.21.15.104:0/1595059729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc06dc50 0x7f2cfc1489f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0cc0b0 tx=0x7f2cf4004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2f89db700 1 --2- 172.21.15.104:0/4266937034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f415ffa0 0x7fa2f42022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f450359e700 1 --2- 172.21.15.104:0/3817120311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45040cb490 0x7f45040ce3a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009e50 tx=0x7f44fc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0c490762ec0e69e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f77ebfff700 1 -- 172.21.15.104:0/1704840984 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77e40e1670 msgr2=0x7f77e40e4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:45.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f77ebfff700 1 --2- 172.21.15.104:0/1704840984 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77e40e1670 0x7f77e40e4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f77ebfff700 1 -- 172.21.15.104:0/1704840984 >> v1:172.21.15.104:6789/0 conn(0x7f77e40e4f50 legacy=0x7f77e4036770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f6cb8126700 1 -- 172.21.15.104:0/1960711854 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb00c3fd0 con 0x7f6cb00c2e90 2023-07-14T11:53:45.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f77ebfff700 1 -- 172.21.15.104:0/1704840984 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e4050f00 con 0x7f77e40e12a0 2023-07-14T11:53:45.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f7cfe59c700 1 --2- 172.21.15.104:0/2976378799 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7ce00663c0 0x7f7ce0068870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2f37fe700 1 -- 172.21.15.104:0/4266937034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f415ffa0 msgr2=0x7fa2f42022d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2f37fe700 1 --2- 172.21.15.104:0/4266937034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f415ffa0 0x7fa2f42022d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2f37fe700 1 -- 172.21.15.104:0/4266937034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f406dc50 msgr2=0x7fa2f41fde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2f37fe700 1 --2- 172.21.15.104:0/4266937034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f406dc50 0x7fa2f41fde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2f37fe700 1 -- 172.21.15.104:0/4266937034 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e40094b0 con 0x7fa2f40cb180 2023-07-14T11:53:45.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7fa2f37fe700 1 --2- 172.21.15.104:0/4266937034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f40cb180 0x7fa2f41f2110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00cbf0 tx=0x7fa2ec00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f77ebfff700 1 --2- 172.21.15.104:0/1704840984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e40e12a0 0x7f77e4037b40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009850 tx=0x7f77d8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cf70f7aac58850c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f6ca67fc700 1 -- 172.21.15.104:0/1960711854 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca0005430 con 0x7f6cb00c2e90 2023-07-14T11:53:45.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f6ca67fc700 1 -- 172.21.15.104:0/1960711854 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca00055a0 con 0x7f6cb00c2e90 2023-07-14T11:53:45.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f63d2ffd700 1 --2- 172.21.15.104:0/4224977765 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f63c00663f0 0x7f63c00688a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009850 tx=0x7f63c8008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f77eaffd700 1 -- 172.21.15.104:0/1704840984 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d8011830 con 0x7f77e40e12a0 2023-07-14T11:53:45.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f77eaffd700 1 -- 172.21.15.104:0/1704840984 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d8011990 con 0x7f77e40e12a0 2023-07-14T11:53:45.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f77eaffd700 1 -- 172.21.15.104:0/1704840984 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d8011d40 con 0x7f77e40e12a0 2023-07-14T11:53:45.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f2cf2ffd700 1 -- 172.21.15.104:0/1595059729 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4004440 con 0x7f2cfc06dc50 2023-07-14T11:53:45.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f2cf2ffd700 1 -- 172.21.15.104:0/1595059729 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf40092e0 con 0x7f2cfc06dc50 2023-07-14T11:53:45.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f2cf2ffd700 1 -- 172.21.15.104:0/1595059729 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf40047e0 con 0x7f2cfc06dc50 2023-07-14T11:53:45.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f2d00cb3700 1 -- 172.21.15.104:0/1595059729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cfc0b1ec0 con 0x7f2cfc06dc50 2023-07-14T11:53:45.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f7cfe59c700 1 --2- 172.21.15.104:0/2976378799 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7ce00663c0 0x7f7ce0068870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40114b0 tx=0x7f7cf4009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f2d00cb3700 1 -- 172.21.15.104:0/1595059729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cfc0b2480 con 0x7f2cfc06dc50 2023-07-14T11:53:45.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f6cb8126700 1 -- 172.21.15.104:0/1960711854 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c98005310 con 0x7f6cb00c2e90 2023-07-14T11:53:45.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f77f125a700 1 -- 172.21.15.104:0/1704840984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e40e12a0 msgr2=0x7f77e4037b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.532+0000 7f77f125a700 1 --2- 172.21.15.104:0/1704840984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e40e12a0 0x7f77e4037b40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009850 tx=0x7f77d8009b60 comp rx=0 tx=0).stop 2023-07-14T11:53:45.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f77f125a700 1 -- 172.21.15.104:0/1704840984 shutdown_connections 2023-07-14T11:53:45.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f77f125a700 1 --2- 172.21.15.104:0/1704840984 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77e40e1670 0x7f77e40e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f77f125a700 1 --2- 172.21.15.104:0/1704840984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e40e12a0 0x7f77e4037b40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f77f125a700 1 -- 172.21.15.104:0/1704840984 >> 172.21.15.104:0/1704840984 conn(0x7f77e4051770 msgr2=0x7f77e4051b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f77f125a700 1 -- 172.21.15.104:0/1704840984 shutdown_connections 2023-07-14T11:53:45.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f77f125a700 1 -- 172.21.15.104:0/1704840984 wait complete. 2023-07-14T11:53:45.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f2d00cb3700 1 -- 172.21.15.104:0/1595059729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cfc06d9e0 con 0x7f2cfc06dc50 2023-07-14T11:53:45.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f6ca67fc700 1 -- 172.21.15.104:0/1960711854 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6ca0006a80 con 0x7f6cb00c2e90 2023-07-14T11:53:45.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f2cf2ffd700 1 -- 172.21.15.104:0/1595059729 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2cf4019030 con 0x7f2cfc06dc50 2023-07-14T11:53:45.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f6ca67fc700 1 --2- 172.21.15.104:0/1960711854 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c94066370 0x7f6c94068820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f6ca67fc700 1 -- 172.21.15.104:0/1960711854 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6ca00834f0 con 0x7f6cb00c2e90 2023-07-14T11:53:45.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f6cb5ec2700 1 --2- 172.21.15.104:0/1960711854 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c94066370 0x7f6c94068820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f2cf2ffd700 1 --2- 172.21.15.104:0/1595059729 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2ce00662f0 0x7f2ce00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f2cf2ffd700 1 -- 172.21.15.104:0/1595059729 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2cf4084e60 con 0x7f2cfc06dc50 2023-07-14T11:53:45.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f6cb5ec2700 1 --2- 172.21.15.104:0/1960711854 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c94066370 0x7f6c94068820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6cac004cd0 tx=0x7f6cac001b30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f2cf9d9b700 1 --2- 172.21.15.104:0/1595059729 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2ce00662f0 0x7f2ce00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f77f125a700 1 Processor -- start 2023-07-14T11:53:45.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f77f125a700 1 -- start start 2023-07-14T11:53:45.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.536+0000 7f2cf9d9b700 1 --2- 172.21.15.104:0/1595059729 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2ce00662f0 0x7f2ce00687a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc150710 tx=0x7f2ce4009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f4501d9b700 1 -- 172.21.15.104:0/3817120311 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc00e070 con 0x7f45040cb490 2023-07-14T11:53:45.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f4501d9b700 1 -- 172.21.15.104:0/3817120311 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc003a40 con 0x7f45040cb490 2023-07-14T11:53:45.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f4501d9b700 1 -- 172.21.15.104:0/3817120311 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc003df0 con 0x7f45040cb490 2023-07-14T11:53:45.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f63bbfff700 1 -- 172.21.15.104:0/4224977765 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f63c4050cd0 con 0x7f63d40c86b0 2023-07-14T11:53:45.591 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7fa2f0ff9700 1 -- 172.21.15.104:0/4266937034 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec00ab50 con 0x7fa2f40cb180 2023-07-14T11:53:45.591 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7fa2f0ff9700 1 -- 172.21.15.104:0/4266937034 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ec00acb0 con 0x7fa2f40cb180 2023-07-14T11:53:45.591 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7fa2f0ff9700 1 -- 172.21.15.104:0/4266937034 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0072d0 con 0x7fa2f40cb180 2023-07-14T11:53:45.591 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7fa2fa43e700 1 -- 172.21.15.104:0/4266937034 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f41f2d00 con 0x7fa2f40cb180 2023-07-14T11:53:45.592 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f77f125a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e40e12a0 0x7f77e40d2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.592 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f7ceeffd700 1 -- 172.21.15.104:0/2976378799 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f7cf8050fa0 con 0x7f7d000ce720 2023-07-14T11:53:45.592 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7fa2fa43e700 1 -- 172.21.15.104:0/4266937034 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f40c6370 con 0x7fa2f40cb180 2023-07-14T11:53:45.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f450904a700 1 -- 172.21.15.104:0/3817120311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45040cb490 msgr2=0x7f45040ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f450904a700 1 --2- 172.21.15.104:0/3817120311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45040cb490 0x7f45040ce3a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009e50 tx=0x7f44fc00b330 comp rx=0 tx=0).stop 2023-07-14T11:53:45.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f77f125a700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77e40e1670 0x7f77e4034bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.594 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f77ebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e40e12a0 0x7f77e40d2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.594 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f77f125a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77e40e4f50 0x7f77e40e2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.594 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f6ca67fc700 1 -- 172.21.15.104:0/1960711854 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f6ca0050660 con 0x7f6cb00c2e90 2023-07-14T11:53:45.594 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f450904a700 1 -- 172.21.15.104:0/3817120311 shutdown_connections 2023-07-14T11:53:45.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f450904a700 1 --2- 172.21.15.104:0/3817120311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45040cb490 0x7f45040ce3a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f450904a700 1 --2- 172.21.15.104:0/3817120311 >> v2:172.21.15.121:3300/0 conn(0x7f450415fd30 0x7f4504160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f450904a700 1 -- 172.21.15.104:0/3817120311 >> 172.21.15.104:0/3817120311 conn(0x7f4504060320 msgr2=0x7f45040606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f77eb7fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77e40e1670 0x7f77e4034bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f77eb7fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77e40e1670 0x7f77e4034bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:51680/0 (socket says 172.21.15.104:51680) 2023-07-14T11:53:45.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f77f0a59700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77e40e4f50 0x7f77e40e2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f77f0a59700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77e40e4f50 0x7f77e40e2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38404/0 (socket says 172.21.15.104:38404) 2023-07-14T11:53:45.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f77eb7fe700 1 -- 172.21.15.104:0/3163212405 learned_addr learned my addr 172.21.15.104:0/3163212405 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f77f125a700 1 -- 172.21.15.104:0/3163212405 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4035110 con 0x7f77e40e4f50 2023-07-14T11:53:45.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f77f125a700 1 -- 172.21.15.104:0/3163212405 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4048bc0 con 0x7f77e40e12a0 2023-07-14T11:53:45.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7f77f125a700 1 -- 172.21.15.104:0/3163212405 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4048ea0 con 0x7f77e40e1670 2023-07-14T11:53:45.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.544+0000 7f450904a700 1 -- 172.21.15.104:0/3817120311 shutdown_connections 2023-07-14T11:53:45.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.544+0000 7f450904a700 1 -- 172.21.15.104:0/3817120311 wait complete. 2023-07-14T11:53:45.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.544+0000 7f450904a700 1 Processor -- start 2023-07-14T11:53:45.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.540+0000 7fa2fa43e700 1 -- 172.21.15.104:0/4266937034 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f40c6590 con 0x7fa2f40cb180 2023-07-14T11:53:45.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f450904a700 1 -- start start 2023-07-14T11:53:45.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.544+0000 7f77eb7fe700 1 -- 172.21.15.104:0/3163212405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e40e12a0 msgr2=0x7f77e40d2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.544+0000 7f77eb7fe700 1 --2- 172.21.15.104:0/3163212405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e40e12a0 0x7f77e40d2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.544+0000 7f77eb7fe700 1 -- 172.21.15.104:0/3163212405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77e40e4f50 msgr2=0x7f77e40e2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.544+0000 7f77eb7fe700 1 --2- 172.21.15.104:0/3163212405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77e40e4f50 0x7f77e40e2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.544+0000 7f77eb7fe700 1 -- 172.21.15.104:0/3163212405 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d80094b0 con 0x7f77e40e1670 2023-07-14T11:53:45.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.544+0000 7f77eb7fe700 1 --2- 172.21.15.104:0/3163212405 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77e40e1670 0x7f77e4034bd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0693f0 tx=0x7f77ec06fdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.544+0000 7f77e8ff9700 1 -- 172.21.15.104:0/3163212405 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec066a30 con 0x7f77e40e1670 2023-07-14T11:53:45.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.544+0000 7f77e8ff9700 1 -- 172.21.15.104:0/3163212405 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec066b90 con 0x7f77e40e1670 2023-07-14T11:53:45.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.544+0000 7f77e8ff9700 1 -- 172.21.15.104:0/3163212405 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec072d30 con 0x7f77e40e1670 2023-07-14T11:53:45.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.544+0000 7f77f125a700 1 -- 172.21.15.104:0/3163212405 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e4049180 con 0x7f77e40e1670 2023-07-14T11:53:45.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.544+0000 7f77f125a700 1 -- 172.21.15.104:0/3163212405 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e4049740 con 0x7f77e40e1670 2023-07-14T11:53:45.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.544+0000 7f77d67fc700 1 -- 172.21.15.104:0/3163212405 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c4005310 con 0x7f77e40e1670 2023-07-14T11:53:45.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f2cf2ffd700 1 -- 172.21.15.104:0/1595059729 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f2cf4055a70 con 0x7f2cfc06dc50 2023-07-14T11:53:45.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f77e8ff9700 1 -- 172.21.15.104:0/3163212405 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f77ec0730a0 con 0x7f77e40e1670 2023-07-14T11:53:45.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f450904a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45040cb490 0x7f45041536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7fa2f0ff9700 1 -- 172.21.15.104:0/4266937034 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa2ec007430 con 0x7fa2f40cb180 2023-07-14T11:53:45.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f450904a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45040cdf70 0x7f45040b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f77e8ff9700 1 --2- 172.21.15.104:0/3163212405 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f77d00663c0 0x7f77d0068870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f450904a700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f450415fd30 0x7f45040b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f77e8ff9700 1 -- 172.21.15.104:0/3163212405 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f77ec08f070 con 0x7f77e40e1670 2023-07-14T11:53:45.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f4502d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45040cb490 0x7f45041536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f4502d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45040cb490 0x7f45041536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57416/0 (socket says 172.21.15.104:57416) 2023-07-14T11:53:45.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f4502d9d700 1 -- 172.21.15.104:0/822693622 learned_addr learned my addr 172.21.15.104:0/822693622 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:45.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f450904a700 1 -- 172.21.15.104:0/822693622 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45040c4350 con 0x7f45040cdf70 2023-07-14T11:53:45.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f450904a700 1 -- 172.21.15.104:0/822693622 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45040c36d0 con 0x7f45040cb490 2023-07-14T11:53:45.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f450359e700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f450415fd30 0x7f45040b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f450904a700 1 -- 172.21.15.104:0/822693622 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45040c39b0 con 0x7f450415fd30 2023-07-14T11:53:45.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7f450259c700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45040cdf70 0x7f45040b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7fa2f0ff9700 1 --2- 172.21.15.104:0/4266937034 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa2e0066370 0x7fa2e0068820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.548+0000 7fa2f0ff9700 1 -- 172.21.15.104:0/4266937034 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa2ec083c40 con 0x7fa2f40cb180 2023-07-14T11:53:45.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.552+0000 7fa2f3fff700 1 --2- 172.21.15.104:0/4266937034 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa2e0066370 0x7fa2e0068820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.552+0000 7f450259c700 1 -- 172.21.15.104:0/822693622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f450415fd30 msgr2=0x7f45040b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.552+0000 7f450259c700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f450415fd30 0x7f45040b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.552+0000 7f450259c700 1 -- 172.21.15.104:0/822693622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45040cb490 msgr2=0x7f45041536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.552+0000 7f450259c700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45040cb490 0x7f45041536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.552+0000 7fa2f3fff700 1 --2- 172.21.15.104:0/4266937034 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa2e0066370 0x7fa2e0068820 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009b20 tx=0x7fa2e4010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.552+0000 7f450259c700 1 -- 172.21.15.104:0/822693622 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc009b30 con 0x7f45040cdf70 2023-07-14T11:53:45.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.552+0000 7f450259c700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45040cdf70 0x7f45040b2f00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f44f800c800 tx=0x7f44f800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.552+0000 7f77ebfff700 1 --2- 172.21.15.104:0/3163212405 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f77d00663c0 0x7f77d0068870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.552+0000 7f4502d9d700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45040cb490 0x7f45041536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:45.608 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.552+0000 7f450359e700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f450415fd30 0x7f45040b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:45.608 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.556+0000 7f77ebfff700 1 --2- 172.21.15.104:0/3163212405 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f77d00663c0 0x7f77d0068870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f77d80090b0 tx=0x7f77d8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.608 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.556+0000 7f77e8ff9700 1 -- 172.21.15.104:0/3163212405 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f77ec0c9f00 con 0x7f77e40e1670 2023-07-14T11:53:45.609 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.560+0000 7fa2f0ff9700 1 -- 172.21.15.104:0/4266937034 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fa2ec054a80 con 0x7fa2f40cb180 2023-07-14T11:53:45.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.560+0000 7f44eb7fe700 1 -- 172.21.15.104:0/822693622 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f8012620 con 0x7f45040cdf70 2023-07-14T11:53:45.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.560+0000 7f450904a700 1 -- 172.21.15.104:0/822693622 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45040c3c90 con 0x7f45040cdf70 2023-07-14T11:53:45.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.560+0000 7f450904a700 1 -- 172.21.15.104:0/822693622 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45040c7cd0 con 0x7f45040cdf70 2023-07-14T11:53:45.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.560+0000 7f44eb7fe700 1 -- 172.21.15.104:0/822693622 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f8012780 con 0x7f45040cdf70 2023-07-14T11:53:45.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.560+0000 7f44eb7fe700 1 -- 172.21.15.104:0/822693622 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f8012b30 con 0x7f45040cdf70 2023-07-14T11:53:45.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.564+0000 7f44eb7fe700 1 -- 172.21.15.104:0/822693622 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f44f8012c90 con 0x7f45040cdf70 2023-07-14T11:53:45.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.564+0000 7f44eb7fe700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f44f00662f0 0x7f44f00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.564+0000 7f44eb7fe700 1 -- 172.21.15.104:0/822693622 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f44f8084020 con 0x7f45040cdf70 2023-07-14T11:53:45.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.564+0000 7f4502d9d700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f44f00662f0 0x7f44f00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.568+0000 7f4502d9d700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f44f00662f0 0x7f44f00687a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009c30 tx=0x7f44f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.572+0000 7f450904a700 1 -- 172.21.15.104:0/822693622 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e4005310 con 0x7f45040cdf70 2023-07-14T11:53:45.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.588+0000 7f44eb7fe700 1 -- 172.21.15.104:0/822693622 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f44f8016080 con 0x7f45040cdf70 2023-07-14T11:53:45.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.828+0000 7f63d9c2c700 1 -- 172.21.15.104:0/4224977765 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f63b40055c0 con 0x7f63d40c86b0 2023-07-14T11:53:45.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.828+0000 7f63bbfff700 1 -- 172.21.15.104:0/4224977765 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f63c4007720 con 0x7f63d40c86b0 2023-07-14T11:53:45.817 INFO:teuthology.orchestra.run.smithi104.stdout:55834574858 2023-07-14T11:53:45.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.832+0000 7f63d9c2c700 1 -- 172.21.15.104:0/4224977765 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f63c00663f0 msgr2=0x7f63c00688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.832+0000 7f63d9c2c700 1 --2- 172.21.15.104:0/4224977765 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f63c00663f0 0x7f63c00688a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009850 tx=0x7f63c8008070 comp rx=0 tx=0).stop 2023-07-14T11:53:45.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.832+0000 7f63d9c2c700 1 -- 172.21.15.104:0/4224977765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63d40c86b0 msgr2=0x7f63d40c43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.832+0000 7f63d9c2c700 1 --2- 172.21.15.104:0/4224977765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63d40c86b0 0x7f63d40c43e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f63c400c520 tx=0x7f63c400c830 comp rx=0 tx=0).stop 2023-07-14T11:53:45.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.832+0000 7f63d9c2c700 1 -- 172.21.15.104:0/4224977765 shutdown_connections 2023-07-14T11:53:45.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.832+0000 7f63d9c2c700 1 --2- 172.21.15.104:0/4224977765 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f63c00663f0 0x7f63c00688a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.832+0000 7f63d9c2c700 1 --2- 172.21.15.104:0/4224977765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63d414ab20 0x7f63d40c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.832+0000 7f63d9c2c700 1 --2- 172.21.15.104:0/4224977765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63d40ce1e0 0x7f63d414a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.832+0000 7f63d9c2c700 1 --2- 172.21.15.104:0/4224977765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63d40c86b0 0x7f63d40c43e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.836+0000 7f63d9c2c700 1 -- 172.21.15.104:0/4224977765 >> 172.21.15.104:0/4224977765 conn(0x7f63d4060320 msgr2=0x7f63d415b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.836+0000 7f63d9c2c700 1 -- 172.21.15.104:0/4224977765 shutdown_connections 2023-07-14T11:53:45.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.836+0000 7f63d9c2c700 1 -- 172.21.15.104:0/4224977765 wait complete. 2023-07-14T11:53:45.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.7 2023-07-14T11:53:45.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.860+0000 7f6cb8126700 1 -- 172.21.15.104:0/1960711854 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6c98005890 con 0x7f6cb00c2e90 2023-07-14T11:53:45.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.860+0000 7f6ca67fc700 1 -- 172.21.15.104:0/1960711854 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6ca0053c80 con 0x7f6cb00c2e90 2023-07-14T11:53:45.846 INFO:teuthology.orchestra.run.smithi104.stdout:55834574858 2023-07-14T11:53:45.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.864+0000 7ff7a5a71700 1 -- 172.21.15.104:0/939615154 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff780005890 con 0x7ff7a015fd30 2023-07-14T11:53:45.847 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.864+0000 7ff787fff700 1 -- 172.21.15.104:0/939615154 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff79002a090 con 0x7ff7a015fd30 2023-07-14T11:53:45.847 INFO:teuthology.orchestra.run.smithi104.stdout:55834574858 2023-07-14T11:53:45.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.864+0000 7f6cb8126700 1 -- 172.21.15.104:0/1960711854 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c94066370 msgr2=0x7f6c94068820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.864+0000 7f6cb8126700 1 --2- 172.21.15.104:0/1960711854 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c94066370 0x7f6c94068820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6cac004cd0 tx=0x7f6cac001b30 comp rx=0 tx=0).stop 2023-07-14T11:53:45.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.864+0000 7f6cb8126700 1 -- 172.21.15.104:0/1960711854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb00c2e90 msgr2=0x7f6cb014c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.864+0000 7f6cb8126700 1 --2- 172.21.15.104:0/1960711854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb00c2e90 0x7f6cb014c3b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cb004f580 tx=0x7f6ca000c670 comp rx=0 tx=0).stop 2023-07-14T11:53:45.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.864+0000 7f6cb8126700 1 -- 172.21.15.104:0/1960711854 shutdown_connections 2023-07-14T11:53:45.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.864+0000 7f6cb8126700 1 --2- 172.21.15.104:0/1960711854 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c94066370 0x7f6c94068820 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.864+0000 7f6cb8126700 1 --2- 172.21.15.104:0/1960711854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cb0159390 0x7f6cb00cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.864+0000 7f6cb8126700 1 --2- 172.21.15.104:0/1960711854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb00c2e90 0x7f6cb014c3b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.864+0000 7f6cb8126700 1 --2- 172.21.15.104:0/1960711854 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c9c009400 0x7f6cb00c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.864+0000 7f6cb8126700 1 -- 172.21.15.104:0/1960711854 >> 172.21.15.104:0/1960711854 conn(0x7f6cb0060320 msgr2=0x7f6cb00b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7f6cb8126700 1 -- 172.21.15.104:0/1960711854 shutdown_connections 2023-07-14T11:53:45.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7ff7a5a71700 1 -- 172.21.15.104:0/939615154 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff788066470 msgr2=0x7ff788068920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7ff7a5a71700 1 --2- 172.21.15.104:0/939615154 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff788066470 0x7ff788068920 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff7a01e8800 tx=0x7ff79800b040 comp rx=0 tx=0).stop 2023-07-14T11:53:45.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7ff7a5a71700 1 -- 172.21.15.104:0/939615154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7a015fd30 msgr2=0x7ff7a0162720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7ff7a5a71700 1 --2- 172.21.15.104:0/939615154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7a015fd30 0x7ff7a0162720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff7900090b0 tx=0x7ff790004770 comp rx=0 tx=0).stop 2023-07-14T11:53:45.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7f6cb8126700 1 -- 172.21.15.104:0/1960711854 wait complete. 2023-07-14T11:53:45.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7ff7a5a71700 1 -- 172.21.15.104:0/939615154 shutdown_connections 2023-07-14T11:53:45.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7ff7a5a71700 1 --2- 172.21.15.104:0/939615154 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff788066470 0x7ff788068920 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7ff7a5a71700 1 --2- 172.21.15.104:0/939615154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7a015fd30 0x7ff7a0162720 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7ff7a5a71700 1 --2- 172.21.15.104:0/939615154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7a00cdf70 0x7ff7a01621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7ff7a5a71700 1 --2- 172.21.15.104:0/939615154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7a00cb490 0x7ff7a00c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7ff7a5a71700 1 -- 172.21.15.104:0/939615154 >> 172.21.15.104:0/939615154 conn(0x7ff7a0060320 msgr2=0x7ff7a00c1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7ff7a5a71700 1 -- 172.21.15.104:0/939615154 shutdown_connections 2023-07-14T11:53:45.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7ff7a5a71700 1 -- 172.21.15.104:0/939615154 wait complete. 2023-07-14T11:53:45.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7f7d04aee700 1 -- 172.21.15.104:0/2976378799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7ce40055c0 con 0x7f7d000ce720 2023-07-14T11:53:45.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.868+0000 7f7ceeffd700 1 -- 172.21.15.104:0/2976378799 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7cf80545c0 con 0x7f7d000ce720 2023-07-14T11:53:45.856 INFO:teuthology.orchestra.run.smithi104.stdout:55834574858 2023-07-14T11:53:45.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.872+0000 7f2d00cb3700 1 -- 172.21.15.104:0/1595059729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2cfc0ca8a0 con 0x7f2cfc06dc50 2023-07-14T11:53:45.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.872+0000 7f2cf2ffd700 1 -- 172.21.15.104:0/1595059729 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2cfc0ca8a0 con 0x7f2cfc06dc50 2023-07-14T11:53:45.857 INFO:teuthology.orchestra.run.smithi104.stdout:55834574858 2023-07-14T11:53:45.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.872+0000 7f7d04aee700 1 -- 172.21.15.104:0/2976378799 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7ce00663c0 msgr2=0x7f7ce0068870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.872+0000 7f7d04aee700 1 --2- 172.21.15.104:0/2976378799 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7ce00663c0 0x7f7ce0068870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40114b0 tx=0x7f7cf4009240 comp rx=0 tx=0).stop 2023-07-14T11:53:45.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.872+0000 7f7d04aee700 1 -- 172.21.15.104:0/2976378799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d000ce720 msgr2=0x7f7d00169140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.872+0000 7f7d04aee700 1 --2- 172.21.15.104:0/2976378799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d000ce720 0x7f7d00169140 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800bda0 tx=0x7f7cf800bdd0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.872+0000 7f7d04aee700 1 -- 172.21.15.104:0/2976378799 shutdown_connections 2023-07-14T11:53:45.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.872+0000 7f7d04aee700 1 --2- 172.21.15.104:0/2976378799 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7ce00663c0 0x7f7ce0068870 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.872+0000 7f7d04aee700 1 --2- 172.21.15.104:0/2976378799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d000ce720 0x7f7d00169140 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.872+0000 7f7d04aee700 1 --2- 172.21.15.104:0/2976378799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d00158c70 0x7f7d00144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.872+0000 7f7d04aee700 1 --2- 172.21.15.104:0/2976378799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d000afbf0 0x7f7d00168c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.872+0000 7f7d04aee700 1 -- 172.21.15.104:0/2976378799 >> 172.21.15.104:0/2976378799 conn(0x7f7d00060320 msgr2=0x7f7d00159240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.872+0000 7f7d04aee700 1 -- 172.21.15.104:0/2976378799 shutdown_connections 2023-07-14T11:53:45.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.872+0000 7f7d04aee700 1 -- 172.21.15.104:0/2976378799 wait complete. 2023-07-14T11:53:45.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7f2d00cb3700 1 -- 172.21.15.104:0/1595059729 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2ce00662f0 msgr2=0x7f2ce00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7f2d00cb3700 1 --2- 172.21.15.104:0/1595059729 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2ce00662f0 0x7f2ce00687a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc150710 tx=0x7f2ce4009240 comp rx=0 tx=0).stop 2023-07-14T11:53:45.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7f2d00cb3700 1 -- 172.21.15.104:0/1595059729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc06dc50 msgr2=0x7f2cfc1489f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7f2d00cb3700 1 --2- 172.21.15.104:0/1595059729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc06dc50 0x7f2cfc1489f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0cc0b0 tx=0x7f2cf4004af0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7f2d00cb3700 1 -- 172.21.15.104:0/1595059729 shutdown_connections 2023-07-14T11:53:45.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7f2d00cb3700 1 --2- 172.21.15.104:0/1595059729 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2ce00662f0 0x7f2ce00687a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7f2d00cb3700 1 --2- 172.21.15.104:0/1595059729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cfc15ffa0 0x7f2cfc149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7f2d00cb3700 1 --2- 172.21.15.104:0/1595059729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2cfc160370 0x7f2cfc149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7f2d00cb3700 1 --2- 172.21.15.104:0/1595059729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc06dc50 0x7f2cfc1489f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7f2d00cb3700 1 -- 172.21.15.104:0/1595059729 >> 172.21.15.104:0/1595059729 conn(0x7f2cfc06e960 msgr2=0x7f2cfc0adb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7fa2fa43e700 1 -- 172.21.15.104:0/4266937034 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa2f4071620 con 0x7fa2f40cb180 2023-07-14T11:53:45.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7f2d00cb3700 1 -- 172.21.15.104:0/1595059729 shutdown_connections 2023-07-14T11:53:45.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7f2d00cb3700 1 -- 172.21.15.104:0/1595059729 wait complete. 2023-07-14T11:53:45.863 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7f77d67fc700 1 -- 172.21.15.104:0/3163212405 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f77c40055c0 con 0x7f77e40e1670 2023-07-14T11:53:45.863 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7fa2f0ff9700 1 -- 172.21.15.104:0/4266937034 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa2ec054610 con 0x7fa2f40cb180 2023-07-14T11:53:45.863 INFO:teuthology.orchestra.run.smithi104.stdout:55834574859 2023-07-14T11:53:45.864 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.876+0000 7f77e8ff9700 1 -- 172.21.15.104:0/3163212405 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f77ec0cd520 con 0x7f77e40e1670 2023-07-14T11:53:45.864 INFO:teuthology.orchestra.run.smithi104.stdout:55834574858 2023-07-14T11:53:45.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7fa2fa43e700 1 -- 172.21.15.104:0/4266937034 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa2e0066370 msgr2=0x7fa2e0068820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7fa2fa43e700 1 --2- 172.21.15.104:0/4266937034 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa2e0066370 0x7fa2e0068820 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009b20 tx=0x7fa2e4010000 comp rx=0 tx=0).stop 2023-07-14T11:53:45.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7fa2fa43e700 1 -- 172.21.15.104:0/4266937034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f40cb180 msgr2=0x7fa2f41f2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7fa2fa43e700 1 --2- 172.21.15.104:0/4266937034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f40cb180 0x7fa2f41f2110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00cbf0 tx=0x7fa2ec00cf00 comp rx=0 tx=0).stop 2023-07-14T11:53:45.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7fa2fa43e700 1 -- 172.21.15.104:0/4266937034 shutdown_connections 2023-07-14T11:53:45.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7fa2fa43e700 1 --2- 172.21.15.104:0/4266937034 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa2e0066370 0x7fa2e0068820 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7fa2fa43e700 1 --2- 172.21.15.104:0/4266937034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f40cb180 0x7fa2f41f2110 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7fa2fa43e700 1 --2- 172.21.15.104:0/4266937034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2f415ffa0 0x7fa2f42022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7fa2fa43e700 1 --2- 172.21.15.104:0/4266937034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f406dc50 0x7fa2f41fde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7fa2fa43e700 1 -- 172.21.15.104:0/4266937034 >> 172.21.15.104:0/4266937034 conn(0x7fa2f406e960 msgr2=0x7fa2f40ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7fa2fa43e700 1 -- 172.21.15.104:0/4266937034 shutdown_connections 2023-07-14T11:53:45.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7fa2fa43e700 1 -- 172.21.15.104:0/4266937034 wait complete. 2023-07-14T11:53:45.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7f77d67fc700 1 -- 172.21.15.104:0/3163212405 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f77d00663c0 msgr2=0x7f77d0068870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7f77d67fc700 1 --2- 172.21.15.104:0/3163212405 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f77d00663c0 0x7f77d0068870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f77d80090b0 tx=0x7f77d8008030 comp rx=0 tx=0).stop 2023-07-14T11:53:45.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7f77d67fc700 1 -- 172.21.15.104:0/3163212405 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77e40e1670 msgr2=0x7f77e4034bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7f77d67fc700 1 --2- 172.21.15.104:0/3163212405 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77e40e1670 0x7f77e4034bd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0693f0 tx=0x7f77ec06fdc0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7f77d67fc700 1 -- 172.21.15.104:0/3163212405 shutdown_connections 2023-07-14T11:53:45.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7f77d67fc700 1 --2- 172.21.15.104:0/3163212405 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f77d00663c0 0x7f77d0068870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7f77d67fc700 1 --2- 172.21.15.104:0/3163212405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77e40e4f50 0x7f77e40e2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7f77d67fc700 1 --2- 172.21.15.104:0/3163212405 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77e40e1670 0x7f77e4034bd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7f77d67fc700 1 --2- 172.21.15.104:0/3163212405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e40e12a0 0x7f77e40d2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.884+0000 7f77d67fc700 1 -- 172.21.15.104:0/3163212405 >> 172.21.15.104:0/3163212405 conn(0x7f77e4051770 msgr2=0x7f77e40528a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.888+0000 7f77d67fc700 1 -- 172.21.15.104:0/3163212405 shutdown_connections 2023-07-14T11:53:45.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.888+0000 7f77d67fc700 1 -- 172.21.15.104:0/3163212405 wait complete. 2023-07-14T11:53:45.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.0 2023-07-14T11:53:45.879 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.2 2023-07-14T11:53:45.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.1 2023-07-14T11:53:45.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.900+0000 7f450904a700 1 -- 172.21.15.104:0/822693622 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f44e40055c0 con 0x7f45040cdf70 2023-07-14T11:53:45.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.900+0000 7f44eb7fe700 1 -- 172.21.15.104:0/822693622 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f44f8054c30 con 0x7f45040cdf70 2023-07-14T11:53:45.886 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2023-07-14T11:53:45.887 INFO:teuthology.orchestra.run.smithi104.stdout:55834574857 2023-07-14T11:53:45.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.904+0000 7f450904a700 1 -- 172.21.15.104:0/822693622 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f44f00662f0 msgr2=0x7f44f00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.904+0000 7f450904a700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f44f00662f0 0x7f44f00687a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009c30 tx=0x7f44f4008040 comp rx=0 tx=0).stop 2023-07-14T11:53:45.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.904+0000 7f450904a700 1 -- 172.21.15.104:0/822693622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45040cdf70 msgr2=0x7f45040b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.904+0000 7f450904a700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45040cdf70 0x7f45040b2f00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f44f800c800 tx=0x7f44f800cbc0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.904+0000 7f450904a700 1 -- 172.21.15.104:0/822693622 shutdown_connections 2023-07-14T11:53:45.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.904+0000 7f450904a700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f44f00662f0 0x7f44f00687a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.904+0000 7f450904a700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45040cdf70 0x7f45040b2f00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.904+0000 7f450904a700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f450415fd30 0x7f45040b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.904+0000 7f450904a700 1 --2- 172.21.15.104:0/822693622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45040cb490 0x7f45041536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.904+0000 7f450904a700 1 -- 172.21.15.104:0/822693622 >> 172.21.15.104:0/822693622 conn(0x7f4504060320 msgr2=0x7f45041607f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.904+0000 7f450904a700 1 -- 172.21.15.104:0/822693622 shutdown_connections 2023-07-14T11:53:45.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:45.904+0000 7f450904a700 1 -- 172.21.15.104:0/822693622 wait complete. 2023-07-14T11:53:45.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.4 2023-07-14T11:53:45.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2023-07-14T11:53:45.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2023-07-14T11:53:46.845 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-14T11:53:46.878 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-14T11:53:46.880 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-14T11:53:46.883 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-14T11:53:46.887 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-14T11:53:46.893 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-14T11:53:46.898 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-14T11:53:46.909 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-14T11:53:47.065 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.080+0000 7f938392d700 1 Processor -- start 2023-07-14T11:53:47.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.080+0000 7f938392d700 1 -- start start 2023-07-14T11:53:47.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f938392d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f937c0caa70 0x7f937c0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f938392d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f937c0ce1e0 0x7f937c06bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.069 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f938392d700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f937c0cdd50 con 0x7f937c158c70 2023-07-14T11:53:47.069 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f938392d700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f937c0cda20 con 0x7f937c0caa70 2023-07-14T11:53:47.069 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f938392d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937c0ca440 con 0x7f937c0ce1e0 2023-07-14T11:53:47.071 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f9380ec8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f937c158c70 0x7f937c0cae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:56592/0 (socket says 172.21.15.104:56592) 2023-07-14T11:53:47.071 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f9380ec8700 1 -- 172.21.15.104:0/3866764976 learned_addr learned my addr 172.21.15.104:0/3866764976 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:47.071 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f9381eca700 1 --2- 172.21.15.104:0/3866764976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f937c0ce1e0 0x7f937c06bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.071 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f93816c9700 1 --2- 172.21.15.104:0/3866764976 >> v2:172.21.15.121:3300/0 conn(0x7f937c0caa70 0x7f937c0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.072 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f9373fff700 1 -- 172.21.15.104:0/3866764976 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 617541395 0 0) 0x7f937c0cdd50 con 0x7f937c158c70 2023-07-14T11:53:47.072 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f9373fff700 1 -- 172.21.15.104:0/3866764976 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93680057e0 con 0x7f937c158c70 2023-07-14T11:53:47.072 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f9381eca700 1 -- 172.21.15.104:0/3866764976 >> v2:172.21.15.121:3300/0 conn(0x7f937c0caa70 msgr2=0x7f937c0c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f9381eca700 1 --2- 172.21.15.104:0/3866764976 >> v2:172.21.15.121:3300/0 conn(0x7f937c0caa70 0x7f937c0c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f9381eca700 1 -- 172.21.15.104:0/3866764976 >> v1:172.21.15.121:6789/0 conn(0x7f937c158c70 legacy=0x7f937c0cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f9381eca700 1 -- 172.21.15.104:0/3866764976 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c0ca720 con 0x7f937c0ce1e0 2023-07-14T11:53:47.074 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.084+0000 7f9381eca700 1 --2- 172.21.15.104:0/3866764976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f937c0ce1e0 0x7f937c06bc60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f936c009850 tx=0x7f936c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b62353f8ba3302f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.074 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.088+0000 7f9373fff700 1 -- 172.21.15.104:0/3866764976 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c011830 con 0x7f937c0ce1e0 2023-07-14T11:53:47.074 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.088+0000 7f9373fff700 1 -- 172.21.15.104:0/3866764976 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936c011990 con 0x7f937c0ce1e0 2023-07-14T11:53:47.074 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.088+0000 7f9373fff700 1 -- 172.21.15.104:0/3866764976 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c011d40 con 0x7f937c0ce1e0 2023-07-14T11:53:47.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.088+0000 7f938392d700 1 -- 172.21.15.104:0/3866764976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f937c0ce1e0 msgr2=0x7f937c06bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.088+0000 7f938392d700 1 --2- 172.21.15.104:0/3866764976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f937c0ce1e0 0x7f937c06bc60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f936c009850 tx=0x7f936c009b60 comp rx=0 tx=0).stop 2023-07-14T11:53:47.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.088+0000 7f938392d700 1 -- 172.21.15.104:0/3866764976 shutdown_connections 2023-07-14T11:53:47.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.088+0000 7f938392d700 1 --2- 172.21.15.104:0/3866764976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f937c0ce1e0 0x7f937c06bc60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.088+0000 7f938392d700 1 --2- 172.21.15.104:0/3866764976 >> v2:172.21.15.121:3300/0 conn(0x7f937c0caa70 0x7f937c0c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.088+0000 7f938392d700 1 -- 172.21.15.104:0/3866764976 >> 172.21.15.104:0/3866764976 conn(0x7f937c060320 msgr2=0x7f937c0606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.088+0000 7f938392d700 1 -- 172.21.15.104:0/3866764976 shutdown_connections 2023-07-14T11:53:47.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.088+0000 7f938392d700 1 -- 172.21.15.104:0/3866764976 wait complete. 2023-07-14T11:53:47.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.092+0000 7f938392d700 1 Processor -- start 2023-07-14T11:53:47.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.092+0000 7f938392d700 1 -- start start 2023-07-14T11:53:47.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.092+0000 7f938392d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f937c0caa70 0x7f937c14b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.092+0000 7f938392d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f937c0ce1e0 0x7f937c0c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.092+0000 7f938392d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f937c158c70 0x7f937c0c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.092+0000 7f93816c9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f937c0caa70 0x7f937c14b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.092+0000 7f93816c9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f937c0caa70 0x7f937c14b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57436/0 (socket says 172.21.15.104:57436) 2023-07-14T11:53:47.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.092+0000 7f9380ec8700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f937c0ce1e0 0x7f937c0c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.092+0000 7f9380ec8700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f937c0ce1e0 0x7f937c0c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:51708/0 (socket says 172.21.15.104:51708) 2023-07-14T11:53:47.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.092+0000 7f9381eca700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f937c158c70 0x7f937c0c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.092+0000 7f9381eca700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f937c158c70 0x7f937c0c74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38422/0 (socket says 172.21.15.104:38422) 2023-07-14T11:53:47.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.092+0000 7f93816c9700 1 -- 172.21.15.104:0/2167493890 learned_addr learned my addr 172.21.15.104:0/2167493890 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:47.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.092+0000 7f938392d700 1 -- 172.21.15.104:0/2167493890 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c0b1e20 con 0x7f937c158c70 2023-07-14T11:53:47.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.092+0000 7f938392d700 1 -- 172.21.15.104:0/2167493890 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c0b2100 con 0x7f937c0caa70 2023-07-14T11:53:47.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.096+0000 7f938392d700 1 -- 172.21.15.104:0/2167493890 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937c0b23e0 con 0x7f937c0ce1e0 2023-07-14T11:53:47.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.096+0000 7f9381eca700 1 -- 172.21.15.104:0/2167493890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f937c0ce1e0 msgr2=0x7f937c0c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.096+0000 7f9381eca700 1 --2- 172.21.15.104:0/2167493890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f937c0ce1e0 0x7f937c0c6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.096+0000 7f9381eca700 1 -- 172.21.15.104:0/2167493890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f937c0caa70 msgr2=0x7f937c14b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.096+0000 7f9381eca700 1 --2- 172.21.15.104:0/2167493890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f937c0caa70 0x7f937c14b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.096+0000 7f9381eca700 1 -- 172.21.15.104:0/2167493890 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c0094b0 con 0x7f937c158c70 2023-07-14T11:53:47.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.096+0000 7f9381eca700 1 --2- 172.21.15.104:0/2167493890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f937c158c70 0x7f937c0c74e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f936c016000 tx=0x7f936c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.096+0000 7f9371ffb700 1 -- 172.21.15.104:0/2167493890 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c004510 con 0x7f937c158c70 2023-07-14T11:53:47.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.096+0000 7f938392d700 1 -- 172.21.15.104:0/2167493890 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937c0b26c0 con 0x7f937c158c70 2023-07-14T11:53:47.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.096+0000 7f938392d700 1 -- 172.21.15.104:0/2167493890 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937c14b880 con 0x7f937c158c70 2023-07-14T11:53:47.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.100+0000 7f9371ffb700 1 -- 172.21.15.104:0/2167493890 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936c01f840 con 0x7f937c158c70 2023-07-14T11:53:47.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.100+0000 7f9371ffb700 1 -- 172.21.15.104:0/2167493890 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c01fbb0 con 0x7f937c158c70 2023-07-14T11:53:47.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.100+0000 7f9371ffb700 1 -- 172.21.15.104:0/2167493890 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f936c01fd50 con 0x7f937c158c70 2023-07-14T11:53:47.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.100+0000 7f938392d700 1 -- 172.21.15.104:0/2167493890 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935c005310 con 0x7f937c158c70 2023-07-14T11:53:47.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.108+0000 7f9371ffb700 1 --2- 172.21.15.104:0/2167493890 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9360066340 0x7f93600687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.108+0000 7f9371ffb700 1 -- 172.21.15.104:0/2167493890 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f936c025030 con 0x7f937c158c70 2023-07-14T11:53:47.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.108+0000 7f93816c9700 1 --2- 172.21.15.104:0/2167493890 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9360066340 0x7f93600687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.108+0000 7f93816c9700 1 --2- 172.21.15.104:0/2167493890 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9360066340 0x7f93600687f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f937800adb0 tx=0x7f937800a350 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.112+0000 7f9371ffb700 1 -- 172.21.15.104:0/2167493890 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f936c056b90 con 0x7f937c158c70 2023-07-14T11:53:47.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.116+0000 7f9dc0e89700 1 Processor -- start 2023-07-14T11:53:47.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.116+0000 7f9dc0e89700 1 -- start start 2023-07-14T11:53:47.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.120+0000 7f9dc0e89700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9dbc0caa70 0x7f9dbc0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.120+0000 7f9dc0e89700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dbc0ce1e0 0x7f9dbc06bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.120+0000 7f9dc0e89700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dbc0cdd50 con 0x7f9dbc158c70 2023-07-14T11:53:47.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.120+0000 7f9dc0e89700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dbc0cda20 con 0x7f9dbc0caa70 2023-07-14T11:53:47.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.120+0000 7f9dc0e89700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc0ca440 con 0x7f9dbc0ce1e0 2023-07-14T11:53:47.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.120+0000 7f9dba59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9dbc0caa70 0x7f9dbc0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.120+0000 7f9dba59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9dbc0caa70 0x7f9dbc0c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57442/0 (socket says 172.21.15.104:57442) 2023-07-14T11:53:47.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.120+0000 7f9dba59c700 1 -- 172.21.15.104:0/1035509672 learned_addr learned my addr 172.21.15.104:0/1035509672 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:47.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.120+0000 7f9dba59c700 1 -- 172.21.15.104:0/1035509672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dbc0ce1e0 msgr2=0x7f9dbc06bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.120+0000 7f9dba59c700 1 --2- 172.21.15.104:0/1035509672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dbc0ce1e0 0x7f9dbc06bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.120+0000 7f9dba59c700 1 -- 172.21.15.104:0/1035509672 >> v1:172.21.15.104:6789/0 conn(0x7f9dbc158c70 legacy=0x7f9dbc0cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.120+0000 7f9dba59c700 1 -- 172.21.15.104:0/1035509672 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc0ca720 con 0x7f9dbc0caa70 2023-07-14T11:53:47.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.124+0000 7f9dba59c700 1 --2- 172.21.15.104:0/1035509672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dbc0caa70 0x7f9dbc0c86b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009bd0 tx=0x7f9db400d3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fc13a372ea5ad2e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.124+0000 7f9db959a700 1 -- 172.21.15.104:0/1035509672 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db4003c40 con 0x7f9dbc0caa70 2023-07-14T11:53:47.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.124+0000 7f9db959a700 1 -- 172.21.15.104:0/1035509672 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db4003da0 con 0x7f9dbc0caa70 2023-07-14T11:53:47.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.124+0000 7f9dc0e89700 1 -- 172.21.15.104:0/1035509672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dbc0caa70 msgr2=0x7f9dbc0c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.124+0000 7f9dc0e89700 1 --2- 172.21.15.104:0/1035509672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dbc0caa70 0x7f9dbc0c86b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009bd0 tx=0x7f9db400d3c0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.124+0000 7f9dc0e89700 1 -- 172.21.15.104:0/1035509672 shutdown_connections 2023-07-14T11:53:47.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.124+0000 7f9dc0e89700 1 --2- 172.21.15.104:0/1035509672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dbc0ce1e0 0x7f9dbc06bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.124+0000 7f9dc0e89700 1 --2- 172.21.15.104:0/1035509672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dbc0caa70 0x7f9dbc0c86b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.124+0000 7f9dc0e89700 1 -- 172.21.15.104:0/1035509672 >> 172.21.15.104:0/1035509672 conn(0x7f9dbc060320 msgr2=0x7f9dbc0606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.110 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.124+0000 7f9dc0e89700 1 -- 172.21.15.104:0/1035509672 shutdown_connections 2023-07-14T11:53:47.110 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.124+0000 7f9dc0e89700 1 -- 172.21.15.104:0/1035509672 wait complete. 2023-07-14T11:53:47.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.128+0000 7f9dc0e89700 1 Processor -- start 2023-07-14T11:53:47.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.128+0000 7f9dc0e89700 1 -- start start 2023-07-14T11:53:47.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.128+0000 7f9dc0e89700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dbc0caa70 0x7f9dbc0b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.128+0000 7f9dc0e89700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dbc0ce1e0 0x7f9dbc200be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.128+0000 7f9dc0e89700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dbc158c70 0x7f9dbc201120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.128+0000 7f9dc0e89700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc201660 con 0x7f9dbc0ce1e0 2023-07-14T11:53:47.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.128+0000 7f9dc0e89700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc0c3590 con 0x7f9dbc158c70 2023-07-14T11:53:47.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.128+0000 7f9dc0e89700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc0c3830 con 0x7f9dbc0caa70 2023-07-14T11:53:47.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.132+0000 7f9db9d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dbc0ce1e0 0x7f9dbc200be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.132+0000 7f9db9d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dbc0ce1e0 0x7f9dbc200be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38428/0 (socket says 172.21.15.104:38428) 2023-07-14T11:53:47.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.132+0000 7f9db9d9b700 1 -- 172.21.15.104:0/3799239317 learned_addr learned my addr 172.21.15.104:0/3799239317 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:47.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.132+0000 7f9dba59c700 1 --2- 172.21.15.104:0/3799239317 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dbc0caa70 0x7f9dbc0b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.132+0000 7f9dbad9d700 1 --2- 172.21.15.104:0/3799239317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dbc158c70 0x7f9dbc201120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.132+0000 7f9db9d9b700 1 -- 172.21.15.104:0/3799239317 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dbc0caa70 msgr2=0x7f9dbc0b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.132+0000 7f9db9d9b700 1 --2- 172.21.15.104:0/3799239317 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dbc0caa70 0x7f9dbc0b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.132+0000 7f9db9d9b700 1 -- 172.21.15.104:0/3799239317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dbc158c70 msgr2=0x7f9dbc201120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.132+0000 7f9db9d9b700 1 --2- 172.21.15.104:0/3799239317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dbc158c70 0x7f9dbc201120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.132+0000 7f9db9d9b700 1 -- 172.21.15.104:0/3799239317 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db4009830 con 0x7f9dbc0ce1e0 2023-07-14T11:53:47.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.132+0000 7f9db9d9b700 1 --2- 172.21.15.104:0/3799239317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dbc0ce1e0 0x7f9dbc200be0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00c780 tx=0x7f9dac00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.132+0000 7f9daaffd700 1 -- 172.21.15.104:0/3799239317 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac0054e0 con 0x7f9dbc0ce1e0 2023-07-14T11:53:47.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.132+0000 7f9dc0e89700 1 -- 172.21.15.104:0/3799239317 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dbc0c3b10 con 0x7f9dbc0ce1e0 2023-07-14T11:53:47.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.136+0000 7f140d95b700 1 Processor -- start 2023-07-14T11:53:47.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.136+0000 7f140d95b700 1 -- start start 2023-07-14T11:53:47.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.136+0000 7f9daaffd700 1 -- 172.21.15.104:0/3799239317 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac0036a0 con 0x7f9dbc0ce1e0 2023-07-14T11:53:47.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.136+0000 7f9daaffd700 1 -- 172.21.15.104:0/3799239317 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac003a50 con 0x7f9dbc0ce1e0 2023-07-14T11:53:47.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.136+0000 7f9dc0e89700 1 -- 172.21.15.104:0/3799239317 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dbc0c40d0 con 0x7f9dbc0ce1e0 2023-07-14T11:53:47.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.136+0000 7f9daaffd700 1 -- 172.21.15.104:0/3799239317 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9dac003c30 con 0x7f9dbc0ce1e0 2023-07-14T11:53:47.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.136+0000 7f140d95b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f14080cebf0 0x7f140806dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.136+0000 7f140d95b700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14081605a0 con 0x7f14081601d0 2023-07-14T11:53:47.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.136+0000 7f140d95b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14080ca630 con 0x7f14080afdd0 2023-07-14T11:53:47.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.136+0000 7f140d95b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14080ca410 con 0x7f14080cebf0 2023-07-14T11:53:47.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.136+0000 7fca163bb700 1 Processor -- start 2023-07-14T11:53:47.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.136+0000 7f14067fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f14080cebf0 0x7f140806dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.136+0000 7f14067fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f14080cebf0 0x7f140806dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57462/0 (socket says 172.21.15.104:57462) 2023-07-14T11:53:47.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f14077fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f14080afdd0 0x7f14080c0c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:56602/0 (socket says 172.21.15.104:56602) 2023-07-14T11:53:47.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f14077fe700 1 -- 172.21.15.104:0/2903624883 learned_addr learned my addr 172.21.15.104:0/2903624883 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:47.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f9daaffd700 1 --2- 172.21.15.104:0/3799239317 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9d9c066340 0x7f9d9c0687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f9daaffd700 1 -- 172.21.15.104:0/3799239317 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9dac0830e0 con 0x7f9dbc0ce1e0 2023-07-14T11:53:47.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f9dba59c700 1 --2- 172.21.15.104:0/3799239317 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9d9c066340 0x7f9d9c0687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.136+0000 7f9dc0e89700 1 -- 172.21.15.104:0/3799239317 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dbc04e500 con 0x7f9dbc0ce1e0 2023-07-14T11:53:47.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1405ffb700 1 -- 172.21.15.104:0/2903624883 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311532492 0 0) 0x7f14080ca630 con 0x7f14080afdd0 2023-07-14T11:53:47.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1405ffb700 1 -- 172.21.15.104:0/2903624883 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13f40035d0 con 0x7f14080afdd0 2023-07-14T11:53:47.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f9dba59c700 1 --2- 172.21.15.104:0/3799239317 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9d9c066340 0x7f9d9c0687f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009b20 tx=0x7f9db4012000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1405ffb700 1 -- 172.21.15.104:0/2903624883 <== mon.1 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3859514269 0 0) 0x7f13f40035d0 con 0x7f14080afdd0 2023-07-14T11:53:47.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1405ffb700 1 -- 172.21.15.104:0/2903624883 >> v2:172.21.15.121:3300/0 conn(0x7f14080cebf0 msgr2=0x7f140806dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1405ffb700 1 --2- 172.21.15.104:0/2903624883 >> v2:172.21.15.121:3300/0 conn(0x7f14080cebf0 0x7f140806dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1405ffb700 1 -- 172.21.15.104:0/2903624883 >> v1:172.21.15.104:6789/0 conn(0x7f14081601d0 legacy=0x7f14080c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1405ffb700 1 -- 172.21.15.104:0/2903624883 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14080ce110 con 0x7f14080afdd0 2023-07-14T11:53:47.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1405ffb700 1 -- 172.21.15.104:0/2903624883 <== mon.1 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f13f8001670 con 0x7f14080afdd0 2023-07-14T11:53:47.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1405ffb700 1 -- 172.21.15.104:0/2903624883 >> v1:172.21.15.121:6789/0 conn(0x7f14080afdd0 legacy=0x7f14080c0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1405ffb700 1 --2- 172.21.15.104:0/2903624883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13f4003a20 0x7f13f4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1405ffb700 1 --2- 172.21.15.104:0/2903624883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13f4004410 0x7f13f40048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1405ffb700 1 --2- 172.21.15.104:0/2903624883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13f40050d0 0x7f13f400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1405ffb700 1 -- 172.21.15.104:0/2903624883 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14080ca410 con 0x7f13f4003a20 2023-07-14T11:53:47.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1405ffb700 1 -- 172.21.15.104:0/2903624883 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f400c020 con 0x7f13f4004410 2023-07-14T11:53:47.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1405ffb700 1 -- 172.21.15.104:0/2903624883 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f400c1e0 con 0x7f13f40050d0 2023-07-14T11:53:47.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7fca163bb700 1 -- start start 2023-07-14T11:53:47.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7fca163bb700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fca08006ec0 0x7fca080538a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7fca163bb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fca08053de0 0x7fca0804f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7fca163bb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca08052b70 con 0x7fca080342d0 2023-07-14T11:53:47.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7fca163bb700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca08036f20 con 0x7fca08006ec0 2023-07-14T11:53:47.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7fca163bb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca080deb70 con 0x7fca08053de0 2023-07-14T11:53:47.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f14077fe700 1 --2- 172.21.15.104:0/2903624883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13f40050d0 0x7f13f400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f14067fc700 1 --2- 172.21.15.104:0/2903624883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13f4004410 0x7f13f40048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f1406ffd700 1 --2- 172.21.15.104:0/2903624883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13f4003a20 0x7f13f4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7fca1761d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fca08053de0 0x7fca0804f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7fca1761d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fca08053de0 0x7fca0804f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57480/0 (socket says 172.21.15.104:57480) 2023-07-14T11:53:47.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7fca1761d700 1 -- 172.21.15.104:0/1008051039 learned_addr learned my addr 172.21.15.104:0/1008051039 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:47.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f325d6d6700 1 Processor -- start 2023-07-14T11:53:47.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f14077fe700 1 -- 172.21.15.104:0/2903624883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13f4004410 msgr2=0x7f13f40048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f14077fe700 1 --2- 172.21.15.104:0/2903624883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13f4004410 0x7f13f40048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f14077fe700 1 -- 172.21.15.104:0/2903624883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13f4003a20 msgr2=0x7f13f4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f14077fe700 1 --2- 172.21.15.104:0/2903624883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13f4003a20 0x7f13f4003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f14077fe700 1 -- 172.21.15.104:0/2903624883 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14080ce110 con 0x7f13f40050d0 2023-07-14T11:53:47.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.140+0000 7f14077fe700 1 -- 172.21.15.104:0/2903624883 reap_dead start 2023-07-14T11:53:47.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca1761d700 1 -- 172.21.15.104:0/1008051039 >> v2:172.21.15.104:3300/0 conn(0x7fca08006ec0 msgr2=0x7fca080538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca1761d700 1 --2- 172.21.15.104:0/1008051039 >> v2:172.21.15.104:3300/0 conn(0x7fca08006ec0 0x7fca080538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca1761d700 1 -- 172.21.15.104:0/1008051039 >> v1:172.21.15.121:6789/0 conn(0x7fca080342d0 legacy=0x7fca08036810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca07fff700 1 -- 172.21.15.104:0/1008051039 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702208386 0 0) 0x7fca08052b70 con 0x7fca080342d0 2023-07-14T11:53:47.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca1761d700 1 -- 172.21.15.104:0/1008051039 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca08037140 con 0x7fca08053de0 2023-07-14T11:53:47.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7f14077fe700 1 --2- 172.21.15.104:0/2903624883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13f40050d0 0x7f13f400ba30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f13f800c370 tx=0x7f13f40058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9595bfb25e642614 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca1761d700 1 --2- 172.21.15.104:0/1008051039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca08053de0 0x7fca0804f3e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fca1004df80 tx=0x7fca10073c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86434412bb497a1c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7f325d6d6700 1 -- start start 2023-07-14T11:53:47.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca07fff700 1 -- 172.21.15.104:0/1008051039 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca10066220 con 0x7fca08053de0 2023-07-14T11:53:47.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7f1405ffb700 1 -- 172.21.15.104:0/2903624883 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8007c30 con 0x7f13f40050d0 2023-07-14T11:53:47.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7f1405ffb700 1 -- 172.21.15.104:0/2903624883 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f8005980 con 0x7f13f40050d0 2023-07-14T11:53:47.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7f1405ffb700 1 -- 172.21.15.104:0/2903624883 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8005d30 con 0x7f13f40050d0 2023-07-14T11:53:47.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca07fff700 1 -- 172.21.15.104:0/1008051039 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca10066380 con 0x7fca08053de0 2023-07-14T11:53:47.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca07fff700 1 -- 172.21.15.104:0/1008051039 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1006fce0 con 0x7fca08053de0 2023-07-14T11:53:47.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7f140d95b700 1 -- 172.21.15.104:0/2903624883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13f40050d0 msgr2=0x7f13f400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7f140d95b700 1 --2- 172.21.15.104:0/2903624883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13f40050d0 0x7f13f400ba30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f13f800c370 tx=0x7f13f40058a0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca163bb700 1 -- 172.21.15.104:0/1008051039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca08053de0 msgr2=0x7fca0804f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca163bb700 1 --2- 172.21.15.104:0/1008051039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca08053de0 0x7fca0804f3e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fca1004df80 tx=0x7fca10073c10 comp rx=0 tx=0).stop 2023-07-14T11:53:47.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7f140d95b700 1 -- 172.21.15.104:0/2903624883 shutdown_connections 2023-07-14T11:53:47.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7f140d95b700 1 --2- 172.21.15.104:0/2903624883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13f40050d0 0x7f13f400ba30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7f140d95b700 1 -- 172.21.15.104:0/2903624883 >> 172.21.15.104:0/2903624883 conn(0x7f1408060320 msgr2=0x7f14080606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca163bb700 1 -- 172.21.15.104:0/1008051039 shutdown_connections 2023-07-14T11:53:47.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca163bb700 1 --2- 172.21.15.104:0/1008051039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca08053de0 0x7fca0804f3e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca163bb700 1 --2- 172.21.15.104:0/1008051039 >> v2:172.21.15.104:3300/0 conn(0x7fca08006ec0 0x7fca080538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca163bb700 1 -- 172.21.15.104:0/1008051039 >> 172.21.15.104:0/1008051039 conn(0x7fca08051810 msgr2=0x7fca08051c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7f140d95b700 1 -- 172.21.15.104:0/2903624883 shutdown_connections 2023-07-14T11:53:47.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7f140d95b700 1 -- 172.21.15.104:0/2903624883 wait complete. 2023-07-14T11:53:47.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca163bb700 1 -- 172.21.15.104:0/1008051039 shutdown_connections 2023-07-14T11:53:47.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.144+0000 7fca163bb700 1 -- 172.21.15.104:0/1008051039 wait complete. 2023-07-14T11:53:47.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7fca163bb700 1 Processor -- start 2023-07-14T11:53:47.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f140d95b700 1 Processor -- start 2023-07-14T11:53:47.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7fca163bb700 1 -- start start 2023-07-14T11:53:47.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f9daaffd700 1 -- 172.21.15.104:0/3799239317 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f9dac088060 con 0x7f9dbc0ce1e0 2023-07-14T11:53:47.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f325d6d6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f32580c86b0 0x7f3258157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f325d6d6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32580ce1e0 0x7f32580caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f325d6d6700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32580cdd50 con 0x7f32580afbf0 2023-07-14T11:53:47.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f325d6d6700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32580ca440 con 0x7f32580c86b0 2023-07-14T11:53:47.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f325d6d6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32580ca720 con 0x7f32580ce1e0 2023-07-14T11:53:47.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f324e7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f32580c86b0 0x7f3258157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f324e7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f32580c86b0 0x7f3258157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57494/0 (socket says 172.21.15.104:57494) 2023-07-14T11:53:47.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f324e7fc700 1 -- 172.21.15.104:0/2687135931 learned_addr learned my addr 172.21.15.104:0/2687135931 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:47.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f32577fe700 1 --2- 172.21.15.104:0/2687135931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32580ce1e0 0x7f32580caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7fca163bb700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca08006ec0 0x7fca0804b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7fca163bb700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca080342d0 0x7fca080d6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f324e7fc700 1 -- 172.21.15.104:0/2687135931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32580ce1e0 msgr2=0x7f32580caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f324e7fc700 1 --2- 172.21.15.104:0/2687135931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32580ce1e0 0x7f32580caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f324e7fc700 1 -- 172.21.15.104:0/2687135931 >> v1:172.21.15.121:6789/0 conn(0x7f32580afbf0 legacy=0x7f3258158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f324e7fc700 1 -- 172.21.15.104:0/2687135931 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32580ca0f0 con 0x7f32580c86b0 2023-07-14T11:53:47.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f140d95b700 1 -- start start 2023-07-14T11:53:47.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f324e7fc700 1 --2- 172.21.15.104:0/2687135931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32580c86b0 0x7f3258157cb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3240009910 tx=0x7f3240009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53b714736264605b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7fca14bb8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca080342d0 0x7fca080d6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7fca14bb8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca080342d0 0x7fca080d6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38472/0 (socket says 172.21.15.104:38472) 2023-07-14T11:53:47.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7fca153b9700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca08006ec0 0x7fca0804b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7fca153b9700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca08006ec0 0x7fca0804b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:51728/0 (socket says 172.21.15.104:51728) 2023-07-14T11:53:47.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7fca14bb8700 1 -- 172.21.15.104:0/1243395340 learned_addr learned my addr 172.21.15.104:0/1243395340 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:47.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7fca163bb700 1 --2- 172.21.15.104:0/1243395340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca08053de0 0x7fca080dc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7fca163bb700 1 -- 172.21.15.104:0/1243395340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0817c710 con 0x7fca080342d0 2023-07-14T11:53:47.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7fca163bb700 1 -- 172.21.15.104:0/1243395340 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0804b660 con 0x7fca08053de0 2023-07-14T11:53:47.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7fca163bb700 1 -- 172.21.15.104:0/1243395340 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0817bb90 con 0x7fca08006ec0 2023-07-14T11:53:47.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7fca14bb8700 1 -- 172.21.15.104:0/1243395340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca08006ec0 msgr2=0x7fca0804b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7fca14bb8700 1 --2- 172.21.15.104:0/1243395340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca08006ec0 0x7fca0804b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7fca14bb8700 1 -- 172.21.15.104:0/1243395340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca08053de0 msgr2=0x7fca080dc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7fca14bb8700 1 --2- 172.21.15.104:0/1243395340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca08053de0 0x7fca080dc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7fca14bb8700 1 -- 172.21.15.104:0/1243395340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca10073710 con 0x7fca080342d0 2023-07-14T11:53:47.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7fca153b9700 1 --2- 172.21.15.104:0/1243395340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca08006ec0 0x7fca0804b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:47.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.148+0000 7f32567fc700 1 -- 172.21.15.104:0/2687135931 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240011830 con 0x7f32580c86b0 2023-07-14T11:53:47.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f32567fc700 1 -- 172.21.15.104:0/2687135931 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3240011990 con 0x7f32580c86b0 2023-07-14T11:53:47.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f32567fc700 1 -- 172.21.15.104:0/2687135931 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240011d40 con 0x7f32580c86b0 2023-07-14T11:53:47.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f325d6d6700 1 -- 172.21.15.104:0/2687135931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32580c86b0 msgr2=0x7f3258157cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f325d6d6700 1 --2- 172.21.15.104:0/2687135931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32580c86b0 0x7f3258157cb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3240009910 tx=0x7f3240009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:47.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f325d6d6700 1 -- 172.21.15.104:0/2687135931 shutdown_connections 2023-07-14T11:53:47.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f325d6d6700 1 --2- 172.21.15.104:0/2687135931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32580ce1e0 0x7f32580caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f325d6d6700 1 --2- 172.21.15.104:0/2687135931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32580c86b0 0x7f3258157cb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f325d6d6700 1 -- 172.21.15.104:0/2687135931 >> 172.21.15.104:0/2687135931 conn(0x7f3258060320 msgr2=0x7f32580606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f140d95b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13f40050d0 0x7f1408157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7fca14bb8700 1 --2- 172.21.15.104:0/1243395340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca080342d0 0x7fca080d6bd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fca0000b480 tx=0x7fca0000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f325d6d6700 1 -- 172.21.15.104:0/2687135931 shutdown_connections 2023-07-14T11:53:47.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f325d6d6700 1 -- 172.21.15.104:0/2687135931 wait complete. 2023-07-14T11:53:47.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7fca05ffb700 1 -- 172.21.15.104:0/1243395340 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00011830 con 0x7fca080342d0 2023-07-14T11:53:47.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7fca163bb700 1 -- 172.21.15.104:0/1243395340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0817be70 con 0x7fca080342d0 2023-07-14T11:53:47.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f140d95b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14081f6280 0x7f14080c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7fca05ffb700 1 -- 172.21.15.104:0/1243395340 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca00011990 con 0x7fca080342d0 2023-07-14T11:53:47.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f140d95b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14080c6f10 0x7f14080c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f1406ffd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13f40050d0 0x7f1408157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f1406ffd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13f40050d0 0x7f1408157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:51730/0 (socket says 172.21.15.104:51730) 2023-07-14T11:53:47.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7fca163bb700 1 -- 172.21.15.104:0/1243395340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0817c430 con 0x7fca080342d0 2023-07-14T11:53:47.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7fca05ffb700 1 -- 172.21.15.104:0/1243395340 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00011d40 con 0x7fca080342d0 2023-07-14T11:53:47.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f1406ffd700 1 -- 172.21.15.104:0/446436257 learned_addr learned my addr 172.21.15.104:0/446436257 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:47.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f140d95b700 1 -- 172.21.15.104:0/446436257 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14080c66b0 con 0x7f14080c6f10 2023-07-14T11:53:47.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f140d95b700 1 -- 172.21.15.104:0/446436257 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14080c78c0 con 0x7f14081f6280 2023-07-14T11:53:47.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f140d95b700 1 -- 172.21.15.104:0/446436257 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14080c5a30 con 0x7f13f40050d0 2023-07-14T11:53:47.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f14077fe700 1 --2- 172.21.15.104:0/446436257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14080c6f10 0x7f14080c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.152+0000 7f14067fc700 1 --2- 172.21.15.104:0/446436257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14081f6280 0x7f14080c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f1406ffd700 1 -- 172.21.15.104:0/446436257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14081f6280 msgr2=0x7f14080c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f1406ffd700 1 --2- 172.21.15.104:0/446436257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14081f6280 0x7f14080c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f1406ffd700 1 -- 172.21.15.104:0/446436257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14080c6f10 msgr2=0x7f14080c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f1406ffd700 1 --2- 172.21.15.104:0/446436257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14080c6f10 0x7f14080c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f1406ffd700 1 -- 172.21.15.104:0/446436257 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14080c5d10 con 0x7f13f40050d0 2023-07-14T11:53:47.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f1406ffd700 1 --2- 172.21.15.104:0/446436257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13f40050d0 0x7f1408157600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f13f000e800 tx=0x7f13f000eb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f325d6d6700 1 Processor -- start 2023-07-14T11:53:47.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7fca163bb700 1 -- 172.21.15.104:0/1243395340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca08038d40 con 0x7fca080342d0 2023-07-14T11:53:47.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f325d6d6700 1 -- start start 2023-07-14T11:53:47.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f13fffff700 1 -- 172.21.15.104:0/446436257 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f00052e0 con 0x7f13f40050d0 2023-07-14T11:53:47.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f13fffff700 1 -- 172.21.15.104:0/446436257 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f0005440 con 0x7f13f40050d0 2023-07-14T11:53:47.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f13fffff700 1 -- 172.21.15.104:0/446436257 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f00038e0 con 0x7f13f40050d0 2023-07-14T11:53:47.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f325d6d6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32580afbf0 0x7f32580b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f325d6d6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32580c86b0 0x7f325815b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f325d6d6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32580ce1e0 0x7f325815bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7fca05ffb700 1 -- 172.21.15.104:0/1243395340 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fca0001a400 con 0x7fca080342d0 2023-07-14T11:53:47.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7fca05ffb700 1 --2- 172.21.15.104:0/1243395340 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc9f4066320 0x7fc9f40687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f3256ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32580afbf0 0x7f32580b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f3256ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32580afbf0 0x7f32580b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38500/0 (socket says 172.21.15.104:38500) 2023-07-14T11:53:47.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f3256ffd700 1 -- 172.21.15.104:0/863014431 learned_addr learned my addr 172.21.15.104:0/863014431 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:47.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f325d6d6700 1 -- 172.21.15.104:0/863014431 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3258159b10 con 0x7f32580afbf0 2023-07-14T11:53:47.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7f325d6d6700 1 -- 172.21.15.104:0/863014431 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32580b1020 con 0x7f32580ce1e0 2023-07-14T11:53:47.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f32577fe700 1 --2- 172.21.15.104:0/863014431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32580ce1e0 0x7f325815bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f325d6d6700 1 -- 172.21.15.104:0/863014431 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32580b1300 con 0x7f32580c86b0 2023-07-14T11:53:47.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.156+0000 7fca05ffb700 1 -- 172.21.15.104:0/1243395340 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fca00084160 con 0x7fca080342d0 2023-07-14T11:53:47.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7fca153b9700 1 --2- 172.21.15.104:0/1243395340 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc9f4066320 0x7fc9f40687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f140d95b700 1 -- 172.21.15.104:0/446436257 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14080c5ff0 con 0x7f13f40050d0 2023-07-14T11:53:47.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f140d95b700 1 -- 172.21.15.104:0/446436257 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14080c6ad0 con 0x7f13f40050d0 2023-07-14T11:53:47.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f32577fe700 1 -- 172.21.15.104:0/863014431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32580c86b0 msgr2=0x7f325815b770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:53:47.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f32577fe700 1 --2- 172.21.15.104:0/863014431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32580c86b0 0x7f325815b770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f32577fe700 1 -- 172.21.15.104:0/863014431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32580afbf0 msgr2=0x7f32580b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f32577fe700 1 --2- 172.21.15.104:0/863014431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32580afbf0 0x7f32580b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f32577fe700 1 -- 172.21.15.104:0/863014431 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3240009570 con 0x7f32580ce1e0 2023-07-14T11:53:47.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7fca153b9700 1 --2- 172.21.15.104:0/1243395340 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc9f4066320 0x7fc9f40687d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fca0c000f60 tx=0x7fca0c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f32577fe700 1 --2- 172.21.15.104:0/863014431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32580ce1e0 0x7f325815bcb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f325000eba0 tx=0x7f325000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f62f3503700 1 Processor -- start 2023-07-14T11:53:47.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f62f3503700 1 -- start start 2023-07-14T11:53:47.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f324ffff700 1 -- 172.21.15.104:0/863014431 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325000ca30 con 0x7f32580ce1e0 2023-07-14T11:53:47.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f324ffff700 1 -- 172.21.15.104:0/863014431 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325000f040 con 0x7f32580ce1e0 2023-07-14T11:53:47.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f324ffff700 1 -- 172.21.15.104:0/863014431 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325000cdb0 con 0x7f32580ce1e0 2023-07-14T11:53:47.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f325d6d6700 1 -- 172.21.15.104:0/863014431 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32580b15e0 con 0x7f32580ce1e0 2023-07-14T11:53:47.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f325d6d6700 1 -- 172.21.15.104:0/863014431 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3258159770 con 0x7f32580ce1e0 2023-07-14T11:53:47.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f3503700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f62e40e2e20 0x7f62e40542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f3503700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f62e4054810 0x7f62e4052eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f3503700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e40064e0 con 0x7f62e4036770 2023-07-14T11:53:47.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f3503700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e40067c0 con 0x7f62e4054810 2023-07-14T11:53:47.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f3503700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e4006aa0 con 0x7f62e40e2e20 2023-07-14T11:53:47.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f325d6d6700 1 -- 172.21.15.104:0/863014431 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325804e500 con 0x7f32580ce1e0 2023-07-14T11:53:47.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f1d00700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f62e40e2e20 0x7f62e40542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f1d00700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f62e40e2e20 0x7f62e40542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:57544/0 (socket says 172.21.15.104:57544) 2023-07-14T11:53:47.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f1d00700 1 -- 172.21.15.104:0/2709801525 learned_addr learned my addr 172.21.15.104:0/2709801525 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:47.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f2d02700 1 --2- 172.21.15.104:0/2709801525 >> v2:172.21.15.104:3300/0 conn(0x7f62e4054810 0x7f62e4052eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f14ff700 1 -- 172.21.15.104:0/2709801525 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 503372873 0 0) 0x7f62e40064e0 con 0x7f62e4036770 2023-07-14T11:53:47.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f14ff700 1 -- 172.21.15.104:0/2709801525 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62d80035d0 con 0x7f62e4036770 2023-07-14T11:53:47.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f2d02700 1 -- 172.21.15.104:0/2709801525 >> v2:172.21.15.121:3300/0 conn(0x7f62e40e2e20 msgr2=0x7f62e40542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f2d02700 1 --2- 172.21.15.104:0/2709801525 >> v2:172.21.15.121:3300/0 conn(0x7f62e40e2e20 0x7f62e40542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f2d02700 1 -- 172.21.15.104:0/2709801525 >> v1:172.21.15.121:6789/0 conn(0x7f62e4036770 legacy=0x7f62e4036b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f2d02700 1 -- 172.21.15.104:0/2709801525 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e4006d80 con 0x7f62e4054810 2023-07-14T11:53:47.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f62f2d02700 1 --2- 172.21.15.104:0/2709801525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62e4054810 0x7f62e4052eb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f62ec02c910 tx=0x7f62ec073c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7064be510a8431f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7fca05ffb700 1 -- 172.21.15.104:0/1243395340 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fca00051250 con 0x7fca080342d0 2023-07-14T11:53:47.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.160+0000 7f140d95b700 1 -- 172.21.15.104:0/446436257 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140804e500 con 0x7f13f40050d0 2023-07-14T11:53:47.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f13fffff700 1 -- 172.21.15.104:0/446436257 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f13f0003a40 con 0x7f13f40050d0 2023-07-14T11:53:47.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f13fffff700 1 --2- 172.21.15.104:0/446436257 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f13e80663c0 0x7f13e8068870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f13fffff700 1 -- 172.21.15.104:0/446436257 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f13f0083860 con 0x7f13f40050d0 2023-07-14T11:53:47.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.164+0000 7f14067fc700 1 --2- 172.21.15.104:0/446436257 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f13e80663c0 0x7f13e8068870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.168+0000 7f62f14ff700 1 -- 172.21.15.104:0/2709801525 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec06fae0 con 0x7f62e4054810 2023-07-14T11:53:47.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.168+0000 7f62f14ff700 1 -- 172.21.15.104:0/2709801525 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ec06fc40 con 0x7f62e4054810 2023-07-14T11:53:47.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.168+0000 7f62f3503700 1 -- 172.21.15.104:0/2709801525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62e4054810 msgr2=0x7f62e4052eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.168+0000 7f62f3503700 1 --2- 172.21.15.104:0/2709801525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62e4054810 0x7f62e4052eb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f62ec02c910 tx=0x7f62ec073c10 comp rx=0 tx=0).stop 2023-07-14T11:53:47.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.168+0000 7f62f3503700 1 -- 172.21.15.104:0/2709801525 shutdown_connections 2023-07-14T11:53:47.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.168+0000 7f62f3503700 1 --2- 172.21.15.104:0/2709801525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62e4054810 0x7f62e4052eb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.168+0000 7f62f3503700 1 --2- 172.21.15.104:0/2709801525 >> v2:172.21.15.121:3300/0 conn(0x7f62e40e2e20 0x7f62e40542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.168+0000 7f62f3503700 1 -- 172.21.15.104:0/2709801525 >> 172.21.15.104:0/2709801525 conn(0x7f62e4051770 msgr2=0x7f62e4051b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.168+0000 7f324ffff700 1 -- 172.21.15.104:0/863014431 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3250014480 con 0x7f32580ce1e0 2023-07-14T11:53:47.176 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.168+0000 7f14067fc700 1 --2- 172.21.15.104:0/446436257 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f13e80663c0 0x7f13e8068870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f140815a900 tx=0x7f140000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.176 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.168+0000 7f62f3503700 1 -- 172.21.15.104:0/2709801525 shutdown_connections 2023-07-14T11:53:47.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.168+0000 7f62f3503700 1 -- 172.21.15.104:0/2709801525 wait complete. 2023-07-14T11:53:47.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.168+0000 7f13fffff700 1 -- 172.21.15.104:0/446436257 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f13f0050a60 con 0x7f13f40050d0 2023-07-14T11:53:47.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.168+0000 7f324ffff700 1 --2- 172.21.15.104:0/863014431 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3238066340 0x7f32380687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.168+0000 7f324ffff700 1 -- 172.21.15.104:0/863014431 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3250084f40 con 0x7f32580ce1e0 2023-07-14T11:53:47.179 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.172+0000 7f3256ffd700 1 --2- 172.21.15.104:0/863014431 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3238066340 0x7f32380687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.179 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.172+0000 7f3256ffd700 1 --2- 172.21.15.104:0/863014431 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3238066340 0x7f32380687f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3248001940 tx=0x7f3248009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.180 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.172+0000 7f62f3503700 1 Processor -- start 2023-07-14T11:53:47.180 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.172+0000 7f62f3503700 1 -- start start 2023-07-14T11:53:47.180 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.172+0000 7f324ffff700 1 -- 172.21.15.104:0/863014431 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f3250052140 con 0x7f32580ce1e0 2023-07-14T11:53:47.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f3503700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62e4036770 0x7f62e404ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f2501700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62e4036770 0x7f62e404ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f2501700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62e4036770 0x7f62e404ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38506/0 (socket says 172.21.15.104:38506) 2023-07-14T11:53:47.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f2501700 1 -- 172.21.15.104:0/2871990539 learned_addr learned my addr 172.21.15.104:0/2871990539 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:47.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f3503700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e4054810 0x7f62e4054f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f3503700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62e40e2e20 0x7f62e40d5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f3503700 1 -- 172.21.15.104:0/2871990539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e40e49d0 con 0x7f62e4036770 2023-07-14T11:53:47.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f3503700 1 -- 172.21.15.104:0/2871990539 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e40e4cb0 con 0x7f62e4054810 2023-07-14T11:53:47.186 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f3503700 1 -- 172.21.15.104:0/2871990539 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e40e4f90 con 0x7f62e40e2e20 2023-07-14T11:53:47.186 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f1d00700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e4054810 0x7f62e4054f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.186 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f2d02700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62e40e2e20 0x7f62e40d5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.187 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f2501700 1 -- 172.21.15.104:0/2871990539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62e40e2e20 msgr2=0x7f62e40d5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.187 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f2501700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62e40e2e20 0x7f62e40d5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.187 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f2501700 1 -- 172.21.15.104:0/2871990539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e4054810 msgr2=0x7f62e4054f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.187 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f2501700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e4054810 0x7f62e4054f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.176+0000 7f62f2501700 1 -- 172.21.15.104:0/2871990539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec073710 con 0x7f62e4036770 2023-07-14T11:53:47.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.180+0000 7f62f2501700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62e4036770 0x7f62e404ccb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f62e800c730 tx=0x7f62e800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.180+0000 7f62e2ffd700 1 -- 172.21.15.104:0/2871990539 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e80052d0 con 0x7f62e4036770 2023-07-14T11:53:47.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.180+0000 7f62e2ffd700 1 -- 172.21.15.104:0/2871990539 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e8005450 con 0x7f62e4036770 2023-07-14T11:53:47.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.180+0000 7f62e2ffd700 1 -- 172.21.15.104:0/2871990539 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e80038e0 con 0x7f62e4036770 2023-07-14T11:53:47.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.180+0000 7f62f3503700 1 -- 172.21.15.104:0/2871990539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e40e5270 con 0x7f62e4036770 2023-07-14T11:53:47.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.180+0000 7f62f3503700 1 -- 172.21.15.104:0/2871990539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e40554a0 con 0x7f62e4036770 2023-07-14T11:53:47.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.184+0000 7f62e2ffd700 1 -- 172.21.15.104:0/2871990539 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f62e8003a40 con 0x7f62e4036770 2023-07-14T11:53:47.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.184+0000 7f62e2ffd700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f62d00663c0 0x7f62d0068870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.184+0000 7f62e2ffd700 1 -- 172.21.15.104:0/2871990539 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f62e80834b0 con 0x7f62e4036770 2023-07-14T11:53:47.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.184+0000 7f62f1d00700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f62d00663c0 0x7f62d0068870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.188+0000 7f62f1d00700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f62d00663c0 0x7f62d0068870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f62e417b1c0 tx=0x7f62dc0092e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.188+0000 7f62f3503700 1 -- 172.21.15.104:0/2871990539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e4005530 con 0x7f62e4036770 2023-07-14T11:53:47.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.192+0000 7f62e2ffd700 1 -- 172.21.15.104:0/2871990539 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f62e80506f0 con 0x7f62e4036770 2023-07-14T11:53:47.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.196+0000 7f8de172d700 1 Processor -- start 2023-07-14T11:53:47.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.196+0000 7f8de172d700 1 -- start start 2023-07-14T11:53:47.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.196+0000 7f8de172d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ddc0afbf0 0x7f8ddc0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.196+0000 7f8de172d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8ddc0caa70 0x7f8ddc0caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.196+0000 7f8de172d700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc0cdd50 con 0x7f8ddc158c70 2023-07-14T11:53:47.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.196+0000 7f8de172d700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ddc0cb430 con 0x7f8ddc0caa70 2023-07-14T11:53:47.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.196+0000 7f8de172d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc0cda20 con 0x7f8ddc0afbf0 2023-07-14T11:53:47.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.200+0000 7f8ddb7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8ddc0caa70 0x7f8ddc0caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.200+0000 7f8dda7fc700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f8ddc158c70 0x7f8ddc0ce1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:56624/0 (socket says 172.21.15.104:56624) 2023-07-14T11:53:47.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.200+0000 7f8dda7fc700 1 -- 172.21.15.104:0/242259308 learned_addr learned my addr 172.21.15.104:0/242259308 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:47.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.200+0000 7f8ddaffd700 1 --2- 172.21.15.104:0/242259308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ddc0afbf0 0x7f8ddc0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.200+0000 7f8dd9ffb700 1 -- 172.21.15.104:0/242259308 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1081738283 0 0) 0x7f8ddc0cdd50 con 0x7f8ddc158c70 2023-07-14T11:53:47.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.200+0000 7f8dd9ffb700 1 -- 172.21.15.104:0/242259308 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dc40035d0 con 0x7f8ddc158c70 2023-07-14T11:53:47.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.200+0000 7f8ddb7fe700 1 -- 172.21.15.104:0/242259308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ddc0afbf0 msgr2=0x7f8ddc0c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.200+0000 7f8ddb7fe700 1 --2- 172.21.15.104:0/242259308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ddc0afbf0 0x7f8ddc0c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.200+0000 7f8ddb7fe700 1 -- 172.21.15.104:0/242259308 >> v1:172.21.15.121:6789/0 conn(0x7f8ddc158c70 legacy=0x7f8ddc0ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.200+0000 7f8ddb7fe700 1 -- 172.21.15.104:0/242259308 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc0ce8f0 con 0x7f8ddc0caa70 2023-07-14T11:53:47.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.200+0000 7f8ddaffd700 1 --2- 172.21.15.104:0/242259308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ddc0afbf0 0x7f8ddc0c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:47.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.200+0000 7f8ddb7fe700 1 --2- 172.21.15.104:0/242259308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ddc0caa70 0x7f8ddc0caef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4009d90 tx=0x7f8dd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72658edb0978623 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.200+0000 7f8dd9ffb700 1 -- 172.21.15.104:0/242259308 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd400e070 con 0x7f8ddc0caa70 2023-07-14T11:53:47.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.200+0000 7f8dd9ffb700 1 -- 172.21.15.104:0/242259308 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd400bb20 con 0x7f8ddc0caa70 2023-07-14T11:53:47.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.200+0000 7f8dd9ffb700 1 -- 172.21.15.104:0/242259308 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4012430 con 0x7f8ddc0caa70 2023-07-14T11:53:47.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.204+0000 7f8de172d700 1 -- 172.21.15.104:0/242259308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ddc0caa70 msgr2=0x7f8ddc0caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.204+0000 7f8de172d700 1 --2- 172.21.15.104:0/242259308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ddc0caa70 0x7f8ddc0caef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4009d90 tx=0x7f8dd400b330 comp rx=0 tx=0).stop 2023-07-14T11:53:47.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.204+0000 7f8de172d700 1 -- 172.21.15.104:0/242259308 shutdown_connections 2023-07-14T11:53:47.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.204+0000 7f8de172d700 1 --2- 172.21.15.104:0/242259308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ddc0caa70 0x7f8ddc0caef0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.204+0000 7f8de172d700 1 --2- 172.21.15.104:0/242259308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ddc0afbf0 0x7f8ddc0c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.204+0000 7f8de172d700 1 -- 172.21.15.104:0/242259308 >> 172.21.15.104:0/242259308 conn(0x7f8ddc060320 msgr2=0x7f8ddc0606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.204+0000 7f8de172d700 1 -- 172.21.15.104:0/242259308 shutdown_connections 2023-07-14T11:53:47.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.204+0000 7f8de172d700 1 -- 172.21.15.104:0/242259308 wait complete. 2023-07-14T11:53:47.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.208+0000 7f8de172d700 1 Processor -- start 2023-07-14T11:53:47.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.208+0000 7f8de172d700 1 -- start start 2023-07-14T11:53:47.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.208+0000 7f8de172d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ddc0afbf0 0x7f8ddc1fddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.208+0000 7f8de172d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ddc0caa70 0x7f8ddc0c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.208+0000 7f8ddaffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ddc0afbf0 0x7f8ddc1fddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.208+0000 7f8ddaffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ddc0afbf0 0x7f8ddc1fddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38514/0 (socket says 172.21.15.104:38514) 2023-07-14T11:53:47.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.208+0000 7f8ddaffd700 1 -- 172.21.15.104:0/3598404985 learned_addr learned my addr 172.21.15.104:0/3598404985 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:47.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.208+0000 7f8de172d700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ddc158c70 0x7f8ddc0c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.208+0000 7f8de172d700 1 -- 172.21.15.104:0/3598404985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc157ae0 con 0x7f8ddc0afbf0 2023-07-14T11:53:47.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.208+0000 7f8de172d700 1 -- 172.21.15.104:0/3598404985 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc157520 con 0x7f8ddc0caa70 2023-07-14T11:53:47.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.208+0000 7f8dda7fc700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ddc0caa70 0x7f8ddc0c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.208+0000 7f8de172d700 1 -- 172.21.15.104:0/3598404985 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc157800 con 0x7f8ddc158c70 2023-07-14T11:53:47.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.208+0000 7f8ddb7fe700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ddc158c70 0x7f8ddc0c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.212+0000 7f8ddaffd700 1 -- 172.21.15.104:0/3598404985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ddc158c70 msgr2=0x7f8ddc0c3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.212+0000 7f8ddaffd700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ddc158c70 0x7f8ddc0c3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.212+0000 7f8ddaffd700 1 -- 172.21.15.104:0/3598404985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ddc0caa70 msgr2=0x7f8ddc0c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.212+0000 7f8ddaffd700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ddc0caa70 0x7f8ddc0c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.212+0000 7f8ddaffd700 1 -- 172.21.15.104:0/3598404985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd00094b0 con 0x7f8ddc0afbf0 2023-07-14T11:53:47.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.212+0000 7f8dda7fc700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ddc0caa70 0x7f8ddc0c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:47.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.212+0000 7f8ddaffd700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ddc0afbf0 0x7f8ddc1fddd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000ebe0 tx=0x7f8dd0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.212+0000 7f8dcb7fe700 1 -- 172.21.15.104:0/3598404985 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd000c8b0 con 0x7f8ddc0afbf0 2023-07-14T11:53:47.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.212+0000 7f8dcb7fe700 1 -- 172.21.15.104:0/3598404985 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd000ca10 con 0x7f8ddc0afbf0 2023-07-14T11:53:47.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.212+0000 7f8dcb7fe700 1 -- 172.21.15.104:0/3598404985 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd000cda0 con 0x7f8ddc0afbf0 2023-07-14T11:53:47.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.212+0000 7f8de172d700 1 -- 172.21.15.104:0/3598404985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd4009a70 con 0x7f8ddc0afbf0 2023-07-14T11:53:47.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.212+0000 7f8de172d700 1 -- 172.21.15.104:0/3598404985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc0b1300 con 0x7f8ddc0afbf0 2023-07-14T11:53:47.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.216+0000 7fa7cf59e700 1 Processor -- start 2023-07-14T11:53:47.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.216+0000 7f8dc97fa700 1 -- 172.21.15.104:0/3598404985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ddc04e500 con 0x7f8ddc0afbf0 2023-07-14T11:53:47.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.216+0000 7fa7cf59e700 1 -- start start 2023-07-14T11:53:47.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.216+0000 7fa7cf59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7d015bff0 0x7fa7d00ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7cf59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa7d00cd070 0x7fa7d00ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7cf59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa7d00ceff0 0x7fa7d00d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7cf59e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d00cd4c0 con 0x7fa7d00cd070 2023-07-14T11:53:47.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7cf59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d00af0a0 con 0x7fa7d00ceff0 2023-07-14T11:53:47.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7cf59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00cf470 con 0x7fa7d015bff0 2023-07-14T11:53:47.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7cdd9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa7d00cd070 0x7fa7d00ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7cdd9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa7d00cd070 0x7fa7d00ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38524/0 (socket says 172.21.15.104:38524) 2023-07-14T11:53:47.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7cdd9b700 1 -- 172.21.15.104:0/2522198435 learned_addr learned my addr 172.21.15.104:0/2522198435 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:47.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7ced9d700 1 --2- 172.21.15.104:0/2522198435 >> v2:172.21.15.121:3300/0 conn(0x7fa7d00ceff0 0x7fa7d00d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7ce59c700 1 --2- 172.21.15.104:0/2522198435 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7d015bff0 0x7fa7d00ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7ced9d700 1 -- 172.21.15.104:0/2522198435 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7d015bff0 msgr2=0x7fa7d00ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7ced9d700 1 --2- 172.21.15.104:0/2522198435 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7d015bff0 0x7fa7d00ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7ced9d700 1 -- 172.21.15.104:0/2522198435 >> v2:172.21.15.104:3300/0 conn(0x7fa7d00cd070 msgr2=0x7fa7d00ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:47.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7ced9d700 1 --2- 172.21.15.104:0/2522198435 >> v2:172.21.15.104:3300/0 conn(0x7fa7d00cd070 0x7fa7d00ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7ced9d700 1 -- 172.21.15.104:0/2522198435 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d00cf720 con 0x7fa7d00ceff0 2023-07-14T11:53:47.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7cdd9b700 1 --2- 172.21.15.104:0/2522198435 >> v2:172.21.15.104:3300/0 conn(0x7fa7d00cd070 0x7fa7d00ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:47.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7ced9d700 1 --2- 172.21.15.104:0/2522198435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7d00ceff0 0x7fa7d00d0020 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa7d008b700 tx=0x7fa7c0009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5520e73771d59a76 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7cd59a700 1 -- 172.21.15.104:0/2522198435 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0011830 con 0x7fa7d00ceff0 2023-07-14T11:53:47.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7cd59a700 1 -- 172.21.15.104:0/2522198435 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c0011990 con 0x7fa7d00ceff0 2023-07-14T11:53:47.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7fa7cd59a700 1 -- 172.21.15.104:0/2522198435 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0011d20 con 0x7fa7d00ceff0 2023-07-14T11:53:47.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7f8dcb7fe700 1 -- 172.21.15.104:0/3598404985 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8dd0013470 con 0x7f8ddc0afbf0 2023-07-14T11:53:47.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.220+0000 7f8dcb7fe700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8dc0066340 0x7f8dc00687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.224+0000 7fa7cf59e700 1 -- 172.21.15.104:0/2522198435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7d00ceff0 msgr2=0x7fa7d00d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.224+0000 7fa7cf59e700 1 --2- 172.21.15.104:0/2522198435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7d00ceff0 0x7fa7d00d0020 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa7d008b700 tx=0x7fa7c0009b30 comp rx=0 tx=0).stop 2023-07-14T11:53:47.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.224+0000 7fa7cf59e700 1 -- 172.21.15.104:0/2522198435 shutdown_connections 2023-07-14T11:53:47.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.224+0000 7fa7cf59e700 1 --2- 172.21.15.104:0/2522198435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7d00ceff0 0x7fa7d00d0020 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.224+0000 7fa7cf59e700 1 --2- 172.21.15.104:0/2522198435 >> v2:172.21.15.104:3300/0 conn(0x7fa7d00cd070 0x7fa7d00ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.224+0000 7fa7cf59e700 1 --2- 172.21.15.104:0/2522198435 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7d015bff0 0x7fa7d00ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.224+0000 7fa7cf59e700 1 -- 172.21.15.104:0/2522198435 >> 172.21.15.104:0/2522198435 conn(0x7fa7d0060c40 msgr2=0x7fa7d009b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.224+0000 7f8dda7fc700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8dc0066340 0x7f8dc00687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.224+0000 7f8dcb7fe700 1 -- 172.21.15.104:0/3598404985 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8dd00586f0 con 0x7f8ddc0afbf0 2023-07-14T11:53:47.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.224+0000 7f8dda7fc700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8dc0066340 0x7f8dc00687f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc201a10 tx=0x7f8dcc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.228+0000 7fa7cf59e700 1 -- 172.21.15.104:0/2522198435 shutdown_connections 2023-07-14T11:53:47.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.228+0000 7fa7cf59e700 1 -- 172.21.15.104:0/2522198435 wait complete. 2023-07-14T11:53:47.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.228+0000 7fa7cf59e700 1 Processor -- start 2023-07-14T11:53:47.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.228+0000 7fa7cf59e700 1 -- start start 2023-07-14T11:53:47.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.228+0000 7f8dcb7fe700 1 -- 172.21.15.104:0/3598404985 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f8dd0050a70 con 0x7f8ddc0afbf0 2023-07-14T11:53:47.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.232+0000 7fa7cf59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7d00cd070 0x7fa7d00ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.232+0000 7fa7cf59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7d00ceff0 0x7fa7d01580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.232+0000 7fa7cf59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7d015bff0 0x7fa7d0158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.232+0000 7fa7cf59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d01f57c0 con 0x7fa7d00ceff0 2023-07-14T11:53:47.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.232+0000 7fa7ce59c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7d00cd070 0x7fa7d00ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.232+0000 7fa7cdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7d00ceff0 0x7fa7d01580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.232+0000 7fa7cdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7d00ceff0 0x7fa7d01580f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38540/0 (socket says 172.21.15.104:38540) 2023-07-14T11:53:47.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.232+0000 7fa7cdd9b700 1 -- 172.21.15.104:0/1202176484 learned_addr learned my addr 172.21.15.104:0/1202176484 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:47.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.232+0000 7fa7cf59e700 1 -- 172.21.15.104:0/1202176484 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00ca220 con 0x7fa7d015bff0 2023-07-14T11:53:47.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.232+0000 7fa7ced9d700 1 --2- 172.21.15.104:0/1202176484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7d015bff0 0x7fa7d0158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.232+0000 7fa7cf59e700 1 -- 172.21.15.104:0/1202176484 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00c9650 con 0x7fa7d00cd070 2023-07-14T11:53:47.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.236+0000 7fa7cdd9b700 1 -- 172.21.15.104:0/1202176484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7d00cd070 msgr2=0x7fa7d00ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.236+0000 7fa7cdd9b700 1 --2- 172.21.15.104:0/1202176484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7d00cd070 0x7fa7d00ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.236+0000 7fa7cdd9b700 1 -- 172.21.15.104:0/1202176484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7d015bff0 msgr2=0x7fa7d0158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.236+0000 7fa7cdd9b700 1 --2- 172.21.15.104:0/1202176484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7d015bff0 0x7fa7d0158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.236+0000 7fa7cdd9b700 1 -- 172.21.15.104:0/1202176484 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c4009570 con 0x7fa7d00ceff0 2023-07-14T11:53:47.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.236+0000 7fa7ced9d700 1 --2- 172.21.15.104:0/1202176484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7d015bff0 0x7fa7d0158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:47.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.236+0000 7fa7cdd9b700 1 --2- 172.21.15.104:0/1202176484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7d00ceff0 0x7fa7d01580f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009fd0 tx=0x7fa7c400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.236+0000 7fa7beffd700 1 -- 172.21.15.104:0/1202176484 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4004030 con 0x7fa7d00ceff0 2023-07-14T11:53:47.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.236+0000 7fa7cf59e700 1 -- 172.21.15.104:0/1202176484 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c00094b0 con 0x7fa7d00ceff0 2023-07-14T11:53:47.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.236+0000 7fa7cf59e700 1 -- 172.21.15.104:0/1202176484 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d00c9bd0 con 0x7fa7d00ceff0 2023-07-14T11:53:47.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.240+0000 7fa7beffd700 1 -- 172.21.15.104:0/1202176484 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c4004190 con 0x7fa7d00ceff0 2023-07-14T11:53:47.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.240+0000 7fa7beffd700 1 -- 172.21.15.104:0/1202176484 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4004520 con 0x7fa7d00ceff0 2023-07-14T11:53:47.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.240+0000 7fa7beffd700 1 -- 172.21.15.104:0/1202176484 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa7c40036c0 con 0x7fa7d00ceff0 2023-07-14T11:53:47.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.244+0000 7fa7beffd700 1 --2- 172.21.15.104:0/1202176484 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7b0066440 0x7fa7b00688f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.244+0000 7fa7beffd700 1 -- 172.21.15.104:0/1202176484 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa7c4083e10 con 0x7fa7d00ceff0 2023-07-14T11:53:47.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.244+0000 7fa7ce59c700 1 --2- 172.21.15.104:0/1202176484 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7b0066440 0x7fa7b00688f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.244+0000 7fa7ce59c700 1 --2- 172.21.15.104:0/1202176484 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7b0066440 0x7fa7b00688f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00cc790 tx=0x7fa7c800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.248+0000 7fa7cf59e700 1 -- 172.21.15.104:0/1202176484 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b4005310 con 0x7fa7d00ceff0 2023-07-14T11:53:47.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.256+0000 7fa7beffd700 1 -- 172.21.15.104:0/1202176484 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fa7c4050f80 con 0x7fa7d00ceff0 2023-07-14T11:53:47.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.432+0000 7f938392d700 1 -- 172.21.15.104:0/2167493890 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f935c005890 con 0x7f937c158c70 2023-07-14T11:53:47.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.432+0000 7f9371ffb700 1 -- 172.21.15.104:0/2167493890 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f936c05a1b0 con 0x7f937c158c70 2023-07-14T11:53:47.418 INFO:teuthology.orchestra.run.smithi104.stdout:55834574859 2023-07-14T11:53:47.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.436+0000 7f938392d700 1 -- 172.21.15.104:0/2167493890 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9360066340 msgr2=0x7f93600687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.436+0000 7f938392d700 1 --2- 172.21.15.104:0/2167493890 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9360066340 0x7f93600687f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f937800adb0 tx=0x7f937800a350 comp rx=0 tx=0).stop 2023-07-14T11:53:47.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.436+0000 7f938392d700 1 -- 172.21.15.104:0/2167493890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f937c158c70 msgr2=0x7f937c0c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.436+0000 7f938392d700 1 --2- 172.21.15.104:0/2167493890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f937c158c70 0x7f937c0c74e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f936c016000 tx=0x7f936c004770 comp rx=0 tx=0).stop 2023-07-14T11:53:47.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.436+0000 7f938392d700 1 -- 172.21.15.104:0/2167493890 shutdown_connections 2023-07-14T11:53:47.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.436+0000 7f938392d700 1 --2- 172.21.15.104:0/2167493890 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9360066340 0x7f93600687f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.436+0000 7f938392d700 1 --2- 172.21.15.104:0/2167493890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f937c158c70 0x7f937c0c74e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.436+0000 7f938392d700 1 --2- 172.21.15.104:0/2167493890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f937c0ce1e0 0x7f937c0c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.436+0000 7f938392d700 1 --2- 172.21.15.104:0/2167493890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f937c0caa70 0x7f937c14b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.436+0000 7f938392d700 1 -- 172.21.15.104:0/2167493890 >> 172.21.15.104:0/2167493890 conn(0x7f937c060320 msgr2=0x7f937c0cc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.436+0000 7f938392d700 1 -- 172.21.15.104:0/2167493890 shutdown_connections 2023-07-14T11:53:47.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.436+0000 7f938392d700 1 -- 172.21.15.104:0/2167493890 wait complete. 2023-07-14T11:53:47.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2023-07-14T11:53:47.445 DEBUG:teuthology.parallel:result is None 2023-07-14T11:53:47.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.472+0000 7f140d95b700 1 -- 172.21.15.104:0/446436257 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1408160d70 con 0x7f13f40050d0 2023-07-14T11:53:47.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.472+0000 7f13fffff700 1 -- 172.21.15.104:0/446436257 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1408160d70 con 0x7f13f40050d0 2023-07-14T11:53:47.456 INFO:teuthology.orchestra.run.smithi104.stdout:55834574859 2023-07-14T11:53:47.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.472+0000 7f9dc0e89700 1 -- 172.21.15.104:0/3799239317 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9dbc0711b0 con 0x7f9dbc0ce1e0 2023-07-14T11:53:47.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.472+0000 7f9daaffd700 1 -- 172.21.15.104:0/3799239317 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9dac017020 con 0x7f9dbc0ce1e0 2023-07-14T11:53:47.457 INFO:teuthology.orchestra.run.smithi104.stdout:55834574859 2023-07-14T11:53:47.459 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f140d95b700 1 -- 172.21.15.104:0/446436257 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f13e80663c0 msgr2=0x7f13e8068870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.459 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f140d95b700 1 --2- 172.21.15.104:0/446436257 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f13e80663c0 0x7f13e8068870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f140815a900 tx=0x7f140000b040 comp rx=0 tx=0).stop 2023-07-14T11:53:47.459 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f140d95b700 1 -- 172.21.15.104:0/446436257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13f40050d0 msgr2=0x7f1408157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.459 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f140d95b700 1 --2- 172.21.15.104:0/446436257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13f40050d0 0x7f1408157600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f13f000e800 tx=0x7f13f000eb10 comp rx=0 tx=0).stop 2023-07-14T11:53:47.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f140d95b700 1 -- 172.21.15.104:0/446436257 shutdown_connections 2023-07-14T11:53:47.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f140d95b700 1 --2- 172.21.15.104:0/446436257 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f13e80663c0 0x7f13e8068870 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f140d95b700 1 --2- 172.21.15.104:0/446436257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14080c6f10 0x7f14080c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f140d95b700 1 --2- 172.21.15.104:0/446436257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14081f6280 0x7f14080c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f140d95b700 1 --2- 172.21.15.104:0/446436257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13f40050d0 0x7f1408157600 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f140d95b700 1 -- 172.21.15.104:0/446436257 >> 172.21.15.104:0/446436257 conn(0x7f1408060320 msgr2=0x7f14080cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f140d95b700 1 -- 172.21.15.104:0/446436257 shutdown_connections 2023-07-14T11:53:47.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f140d95b700 1 -- 172.21.15.104:0/446436257 wait complete. 2023-07-14T11:53:47.462 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f9dc0e89700 1 -- 172.21.15.104:0/3799239317 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9d9c066340 msgr2=0x7f9d9c0687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.462 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f9dc0e89700 1 --2- 172.21.15.104:0/3799239317 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9d9c066340 0x7f9d9c0687f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009b20 tx=0x7f9db4012000 comp rx=0 tx=0).stop 2023-07-14T11:53:47.462 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f9dc0e89700 1 -- 172.21.15.104:0/3799239317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dbc0ce1e0 msgr2=0x7f9dbc200be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.462 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f9dc0e89700 1 --2- 172.21.15.104:0/3799239317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dbc0ce1e0 0x7f9dbc200be0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00c780 tx=0x7f9dac00ca90 comp rx=0 tx=0).stop 2023-07-14T11:53:47.462 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f9dc0e89700 1 -- 172.21.15.104:0/3799239317 shutdown_connections 2023-07-14T11:53:47.462 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f9dc0e89700 1 --2- 172.21.15.104:0/3799239317 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9d9c066340 0x7f9d9c0687f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.463 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f9dc0e89700 1 --2- 172.21.15.104:0/3799239317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dbc0ce1e0 0x7f9dbc200be0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.463 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f9dc0e89700 1 --2- 172.21.15.104:0/3799239317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dbc158c70 0x7f9dbc201120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.463 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f9dc0e89700 1 --2- 172.21.15.104:0/3799239317 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dbc0caa70 0x7f9dbc0b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.463 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f9dc0e89700 1 -- 172.21.15.104:0/3799239317 >> 172.21.15.104:0/3799239317 conn(0x7f9dbc060320 msgr2=0x7f9dbc15b770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.463 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f9dc0e89700 1 -- 172.21.15.104:0/3799239317 shutdown_connections 2023-07-14T11:53:47.463 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.476+0000 7f9dc0e89700 1 -- 172.21.15.104:0/3799239317 wait complete. 2023-07-14T11:53:47.470 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.484+0000 7fca163bb700 1 -- 172.21.15.104:0/1243395340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fca080055d0 con 0x7fca080342d0 2023-07-14T11:53:47.470 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.488+0000 7fca05ffb700 1 -- 172.21.15.104:0/1243395340 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fca00054870 con 0x7fca080342d0 2023-07-14T11:53:47.471 INFO:teuthology.orchestra.run.smithi104.stdout:55834574859 2023-07-14T11:53:47.474 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.488+0000 7fca163bb700 1 -- 172.21.15.104:0/1243395340 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc9f4066320 msgr2=0x7fc9f40687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.474 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.488+0000 7fca163bb700 1 --2- 172.21.15.104:0/1243395340 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc9f4066320 0x7fc9f40687d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fca0c000f60 tx=0x7fca0c009040 comp rx=0 tx=0).stop 2023-07-14T11:53:47.475 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.492+0000 7fca163bb700 1 -- 172.21.15.104:0/1243395340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca080342d0 msgr2=0x7fca080d6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.475 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.492+0000 7fca163bb700 1 --2- 172.21.15.104:0/1243395340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca080342d0 0x7fca080d6bd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fca0000b480 tx=0x7fca0000b840 comp rx=0 tx=0).stop 2023-07-14T11:53:47.475 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.492+0000 7fca163bb700 1 -- 172.21.15.104:0/1243395340 shutdown_connections 2023-07-14T11:53:47.475 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.492+0000 7fca163bb700 1 --2- 172.21.15.104:0/1243395340 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc9f4066320 0x7fc9f40687d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.475 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.492+0000 7fca163bb700 1 --2- 172.21.15.104:0/1243395340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca080342d0 0x7fca080d6bd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.492+0000 7fca163bb700 1 --2- 172.21.15.104:0/1243395340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca08053de0 0x7fca080dc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.492+0000 7fca163bb700 1 --2- 172.21.15.104:0/1243395340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca08006ec0 0x7fca0804b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.492+0000 7fca163bb700 1 -- 172.21.15.104:0/1243395340 >> 172.21.15.104:0/1243395340 conn(0x7fca08051810 msgr2=0x7fca080e0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.477 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.492+0000 7fca163bb700 1 -- 172.21.15.104:0/1243395340 shutdown_connections 2023-07-14T11:53:47.477 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.492+0000 7fca163bb700 1 -- 172.21.15.104:0/1243395340 wait complete. 2023-07-14T11:53:47.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.492+0000 7f325d6d6700 1 -- 172.21.15.104:0/863014431 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3258071920 con 0x7f32580ce1e0 2023-07-14T11:53:47.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.496+0000 7f324ffff700 1 -- 172.21.15.104:0/863014431 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3250018020 con 0x7f32580ce1e0 2023-07-14T11:53:47.482 INFO:teuthology.orchestra.run.smithi104.stdout:55834574859 2023-07-14T11:53:47.487 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2023-07-14T11:53:47.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.500+0000 7f325d6d6700 1 -- 172.21.15.104:0/863014431 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3238066340 msgr2=0x7f32380687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.500+0000 7f325d6d6700 1 --2- 172.21.15.104:0/863014431 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3238066340 0x7f32380687f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3248001940 tx=0x7f3248009040 comp rx=0 tx=0).stop 2023-07-14T11:53:47.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.500+0000 7f325d6d6700 1 -- 172.21.15.104:0/863014431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32580ce1e0 msgr2=0x7f325815bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.500+0000 7f325d6d6700 1 --2- 172.21.15.104:0/863014431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32580ce1e0 0x7f325815bcb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f325000eba0 tx=0x7f325000eeb0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.504+0000 7f325d6d6700 1 -- 172.21.15.104:0/863014431 shutdown_connections 2023-07-14T11:53:47.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.504+0000 7f325d6d6700 1 --2- 172.21.15.104:0/863014431 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3238066340 0x7f32380687f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.488 DEBUG:teuthology.parallel:result is None 2023-07-14T11:53:47.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.504+0000 7f325d6d6700 1 --2- 172.21.15.104:0/863014431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32580ce1e0 0x7f325815bcb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.504+0000 7f325d6d6700 1 --2- 172.21.15.104:0/863014431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32580c86b0 0x7f325815b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.504+0000 7f325d6d6700 1 --2- 172.21.15.104:0/863014431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32580afbf0 0x7f32580b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.504+0000 7f325d6d6700 1 -- 172.21.15.104:0/863014431 >> 172.21.15.104:0/863014431 conn(0x7f3258060320 msgr2=0x7f325806bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.490 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.0 2023-07-14T11:53:47.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.504+0000 7f325d6d6700 1 -- 172.21.15.104:0/863014431 shutdown_connections 2023-07-14T11:53:47.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.504+0000 7f325d6d6700 1 -- 172.21.15.104:0/863014431 wait complete. 2023-07-14T11:53:47.490 DEBUG:teuthology.parallel:result is None 2023-07-14T11:53:47.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.512+0000 7f62f3503700 1 -- 172.21.15.104:0/2871990539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f62e4007670 con 0x7f62e4036770 2023-07-14T11:53:47.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.512+0000 7f62e2ffd700 1 -- 172.21.15.104:0/2871990539 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f62e8017020 con 0x7f62e4036770 2023-07-14T11:53:47.497 INFO:teuthology.orchestra.run.smithi104.stdout:55834574860 2023-07-14T11:53:47.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2023-07-14T11:53:47.500 DEBUG:teuthology.parallel:result is None 2023-07-14T11:53:47.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.516+0000 7f62f3503700 1 -- 172.21.15.104:0/2871990539 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f62d00663c0 msgr2=0x7f62d0068870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.516+0000 7f62f3503700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f62d00663c0 0x7f62d0068870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f62e417b1c0 tx=0x7f62dc0092e0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.516+0000 7f62f3503700 1 -- 172.21.15.104:0/2871990539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62e4036770 msgr2=0x7f62e404ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.516+0000 7f62f3503700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62e4036770 0x7f62e404ccb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f62e800c730 tx=0x7f62e800ca40 comp rx=0 tx=0).stop 2023-07-14T11:53:47.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.516+0000 7f62f3503700 1 -- 172.21.15.104:0/2871990539 shutdown_connections 2023-07-14T11:53:47.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.516+0000 7f62f3503700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f62d00663c0 0x7f62d0068870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.516+0000 7f62f3503700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62e40e2e20 0x7f62e40d5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.516+0000 7f62f3503700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e4054810 0x7f62e4054f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.516+0000 7f62f3503700 1 --2- 172.21.15.104:0/2871990539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62e4036770 0x7f62e404ccb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.516+0000 7f62f3503700 1 -- 172.21.15.104:0/2871990539 >> 172.21.15.104:0/2871990539 conn(0x7f62e4051770 msgr2=0x7f62e40d7030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.516+0000 7f62f3503700 1 -- 172.21.15.104:0/2871990539 shutdown_connections 2023-07-14T11:53:47.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.516+0000 7f62f3503700 1 -- 172.21.15.104:0/2871990539 wait complete. 2023-07-14T11:53:47.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.520+0000 7f8dc97fa700 1 -- 172.21.15.104:0/3598404985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f8ddc06ddb0 con 0x7f8ddc0afbf0 2023-07-14T11:53:47.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.520+0000 7f8dcb7fe700 1 -- 172.21.15.104:0/3598404985 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8dd0017020 con 0x7f8ddc0afbf0 2023-07-14T11:53:47.504 INFO:teuthology.orchestra.run.smithi104.stdout:55834574858 2023-07-14T11:53:47.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.520+0000 7f8dc97fa700 1 -- 172.21.15.104:0/3598404985 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8dc0066340 msgr2=0x7f8dc00687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.520+0000 7f8dc97fa700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8dc0066340 0x7f8dc00687f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc201a10 tx=0x7f8dcc009040 comp rx=0 tx=0).stop 2023-07-14T11:53:47.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.520+0000 7f8dc97fa700 1 -- 172.21.15.104:0/3598404985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ddc0afbf0 msgr2=0x7f8ddc1fddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.520+0000 7f8dc97fa700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ddc0afbf0 0x7f8ddc1fddd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000ebe0 tx=0x7f8dd0009e30 comp rx=0 tx=0).stop 2023-07-14T11:53:47.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.520+0000 7f8dc97fa700 1 -- 172.21.15.104:0/3598404985 shutdown_connections 2023-07-14T11:53:47.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.520+0000 7f8dc97fa700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8dc0066340 0x7f8dc00687f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.520+0000 7f8dc97fa700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ddc158c70 0x7f8ddc0c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.520+0000 7f8dc97fa700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ddc0caa70 0x7f8ddc0c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.520+0000 7f8dc97fa700 1 --2- 172.21.15.104:0/3598404985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ddc0afbf0 0x7f8ddc1fddd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.520+0000 7f8dc97fa700 1 -- 172.21.15.104:0/3598404985 >> 172.21.15.104:0/3598404985 conn(0x7f8ddc060320 msgr2=0x7f8ddc0aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.520+0000 7f8dc97fa700 1 -- 172.21.15.104:0/3598404985 shutdown_connections 2023-07-14T11:53:47.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.524+0000 7f8dc97fa700 1 -- 172.21.15.104:0/3598404985 wait complete. 2023-07-14T11:53:47.510 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2023-07-14T11:53:47.510 DEBUG:teuthology.parallel:result is None 2023-07-14T11:53:47.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.4 2023-07-14T11:53:47.518 DEBUG:teuthology.parallel:result is None 2023-07-14T11:53:47.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2023-07-14T11:53:47.524 DEBUG:teuthology.parallel:result is None 2023-07-14T11:53:47.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.544+0000 7fa7cf59e700 1 -- 172.21.15.104:0/1202176484 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa7b40055c0 con 0x7fa7d00ceff0 2023-07-14T11:53:47.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.544+0000 7fa7beffd700 1 -- 172.21.15.104:0/1202176484 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa7c4020020 con 0x7fa7d00ceff0 2023-07-14T11:53:47.528 INFO:teuthology.orchestra.run.smithi104.stdout:55834574859 2023-07-14T11:53:47.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.544+0000 7fa7cf59e700 1 -- 172.21.15.104:0/1202176484 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7b0066440 msgr2=0x7fa7b00688f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.544+0000 7fa7cf59e700 1 --2- 172.21.15.104:0/1202176484 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7b0066440 0x7fa7b00688f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00cc790 tx=0x7fa7c800b040 comp rx=0 tx=0).stop 2023-07-14T11:53:47.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.544+0000 7fa7cf59e700 1 -- 172.21.15.104:0/1202176484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7d00ceff0 msgr2=0x7fa7d01580f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.544+0000 7fa7cf59e700 1 --2- 172.21.15.104:0/1202176484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7d00ceff0 0x7fa7d01580f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009fd0 tx=0x7fa7c400ec60 comp rx=0 tx=0).stop 2023-07-14T11:53:47.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.544+0000 7fa7cf59e700 1 -- 172.21.15.104:0/1202176484 shutdown_connections 2023-07-14T11:53:47.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.544+0000 7fa7cf59e700 1 --2- 172.21.15.104:0/1202176484 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7b0066440 0x7fa7b00688f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.544+0000 7fa7cf59e700 1 --2- 172.21.15.104:0/1202176484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7d00ceff0 0x7fa7d01580f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.544+0000 7fa7cf59e700 1 --2- 172.21.15.104:0/1202176484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7d015bff0 0x7fa7d0158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.544+0000 7fa7cf59e700 1 --2- 172.21.15.104:0/1202176484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7d00cd070 0x7fa7d00ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.544+0000 7fa7cf59e700 1 -- 172.21.15.104:0/1202176484 >> 172.21.15.104:0/1202176484 conn(0x7fa7d0060c40 msgr2=0x7fa7d0070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.544+0000 7fa7cf59e700 1 -- 172.21.15.104:0/1202176484 shutdown_connections 2023-07-14T11:53:47.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.544+0000 7fa7cf59e700 1 -- 172.21.15.104:0/1202176484 wait complete. 2023-07-14T11:53:47.550 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2023-07-14T11:53:47.551 DEBUG:teuthology.parallel:result is None 2023-07-14T11:53:47.551 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-14T11:53:47.551 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:53:47.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.776+0000 7f14be513700 1 Processor -- start 2023-07-14T11:53:47.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.776+0000 7f14be513700 1 -- start start 2023-07-14T11:53:47.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.780+0000 7f14be513700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f14b8161d00 0x7f14b81640e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.780+0000 7f14be513700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b80c97f0 con 0x7f14b80c9c80 2023-07-14T11:53:47.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.780+0000 7f14be513700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b80c8900 con 0x7f14b80afd30 2023-07-14T11:53:47.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.780+0000 7f14be513700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b80c8be0 con 0x7f14b8161d00 2023-07-14T11:53:47.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.780+0000 7f14affff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f14b80c9c80 0x7f14b806d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42578/0 (socket says 172.21.15.104:42578) 2023-07-14T11:53:47.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.780+0000 7f14affff700 1 -- 172.21.15.104:0/1622541254 learned_addr learned my addr 172.21.15.104:0/1622541254 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:47.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.780+0000 7f14bcab0700 1 --2- 172.21.15.104:0/1622541254 >> v2:172.21.15.121:3300/0 conn(0x7f14b8161d00 0x7f14b81640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.780+0000 7f14b77fe700 1 -- 172.21.15.104:0/1622541254 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962983816 0 0) 0x7f14b80c97f0 con 0x7f14b80c9c80 2023-07-14T11:53:47.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.780+0000 7f14b77fe700 1 -- 172.21.15.104:0/1622541254 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14a40035d0 con 0x7f14b80c9c80 2023-07-14T11:53:47.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.780+0000 7f14bcab0700 1 -- 172.21.15.104:0/1622541254 >> v1:172.21.15.121:6789/0 conn(0x7f14b80afd30 legacy=0x7f14b80b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.780+0000 7f14bcab0700 1 -- 172.21.15.104:0/1622541254 >> v1:172.21.15.104:6789/0 conn(0x7f14b80c9c80 legacy=0x7f14b806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.780+0000 7f14bcab0700 1 -- 172.21.15.104:0/1622541254 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b80c8ec0 con 0x7f14b8161d00 2023-07-14T11:53:47.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.784+0000 7f14bcab0700 1 --2- 172.21.15.104:0/1622541254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b8161d00 0x7f14b81640e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009d90 tx=0x7f14b000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d2a7c9ccc0e149b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.784+0000 7f14b77fe700 1 -- 172.21.15.104:0/1622541254 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b000e070 con 0x7f14b8161d00 2023-07-14T11:53:47.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.784+0000 7f14b77fe700 1 -- 172.21.15.104:0/1622541254 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b000bb20 con 0x7f14b8161d00 2023-07-14T11:53:47.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.784+0000 7f14b77fe700 1 -- 172.21.15.104:0/1622541254 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b000be90 con 0x7f14b8161d00 2023-07-14T11:53:47.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.784+0000 7f14be513700 1 -- 172.21.15.104:0/1622541254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b8161d00 msgr2=0x7f14b81640e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.784+0000 7f14be513700 1 --2- 172.21.15.104:0/1622541254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b8161d00 0x7f14b81640e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009d90 tx=0x7f14b000b330 comp rx=0 tx=0).stop 2023-07-14T11:53:47.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.784+0000 7f14be513700 1 -- 172.21.15.104:0/1622541254 shutdown_connections 2023-07-14T11:53:47.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.784+0000 7f14be513700 1 --2- 172.21.15.104:0/1622541254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b8161d00 0x7f14b81640e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.784+0000 7f14be513700 1 -- 172.21.15.104:0/1622541254 >> 172.21.15.104:0/1622541254 conn(0x7f14b80cad30 msgr2=0x7f14b80cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.784+0000 7f14be513700 1 -- 172.21.15.104:0/1622541254 shutdown_connections 2023-07-14T11:53:47.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.784+0000 7f14be513700 1 -- 172.21.15.104:0/1622541254 wait complete. 2023-07-14T11:53:47.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.784+0000 7f14be513700 1 Processor -- start 2023-07-14T11:53:47.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.784+0000 7f14be513700 1 -- start start 2023-07-14T11:53:47.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14be513700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14b80afd30 0x7f14b814c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14be513700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b80c9c80 0x7f14b80b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14be513700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14b8161d00 0x7f14b814d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14be513700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80b2560 con 0x7f14b80afd30 2023-07-14T11:53:47.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14be513700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80b2840 con 0x7f14b80c9c80 2023-07-14T11:53:47.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14be513700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80b2b20 con 0x7f14b8161d00 2023-07-14T11:53:47.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14b7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14b80afd30 0x7f14b814c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14b7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14b80afd30 0x7f14b814c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38556/0 (socket says 172.21.15.104:38556) 2023-07-14T11:53:47.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14b7fff700 1 -- 172.21.15.104:0/2476444621 learned_addr learned my addr 172.21.15.104:0/2476444621 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:47.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14affff700 1 --2- 172.21.15.104:0/2476444621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b80c9c80 0x7f14b80b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14bcab0700 1 --2- 172.21.15.104:0/2476444621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14b8161d00 0x7f14b814d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14b7fff700 1 -- 172.21.15.104:0/2476444621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14b8161d00 msgr2=0x7f14b814d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14b7fff700 1 --2- 172.21.15.104:0/2476444621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14b8161d00 0x7f14b814d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14b7fff700 1 -- 172.21.15.104:0/2476444621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b80c9c80 msgr2=0x7f14b80b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14b7fff700 1 --2- 172.21.15.104:0/2476444621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b80c9c80 0x7f14b80b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14b7fff700 1 -- 172.21.15.104:0/2476444621 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b0009a70 con 0x7f14b80afd30 2023-07-14T11:53:47.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14bcab0700 1 --2- 172.21.15.104:0/2476444621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14b8161d00 0x7f14b814d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:47.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14b7fff700 1 --2- 172.21.15.104:0/2476444621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14b80afd30 0x7f14b814c0b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f14a800c500 tx=0x7f14a800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14b57fa700 1 -- 172.21.15.104:0/2476444621 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80052d0 con 0x7f14b80afd30 2023-07-14T11:53:47.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14b57fa700 1 -- 172.21.15.104:0/2476444621 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a8005430 con 0x7f14b80afd30 2023-07-14T11:53:47.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14b57fa700 1 -- 172.21.15.104:0/2476444621 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80055a0 con 0x7f14b80afd30 2023-07-14T11:53:47.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14be513700 1 -- 172.21.15.104:0/2476444621 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b80b2e00 con 0x7f14b80afd30 2023-07-14T11:53:47.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.788+0000 7f14be513700 1 -- 172.21.15.104:0/2476444621 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b814c850 con 0x7f14b80afd30 2023-07-14T11:53:47.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.792+0000 7f14be513700 1 -- 172.21.15.104:0/2476444621 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b80ce0b0 con 0x7f14b80afd30 2023-07-14T11:53:47.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.792+0000 7f14b57fa700 1 -- 172.21.15.104:0/2476444621 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f14a80038e0 con 0x7f14b80afd30 2023-07-14T11:53:47.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.792+0000 7f14b57fa700 1 --2- 172.21.15.104:0/2476444621 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f14980662f0 0x7f14980687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:47.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.792+0000 7f14b57fa700 1 -- 172.21.15.104:0/2476444621 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f14a8083570 con 0x7f14b80afd30 2023-07-14T11:53:47.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.796+0000 7f14affff700 1 --2- 172.21.15.104:0/2476444621 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f14980662f0 0x7f14980687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:47.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.796+0000 7f14affff700 1 --2- 172.21.15.104:0/2476444621 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f14980662f0 0x7f14980687a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f14b81f4e60 tx=0x7f14a0007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:47.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.800+0000 7f14b57fa700 1 -- 172.21.15.104:0/2476444621 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f14a80505d0 con 0x7f14b80afd30 2023-07-14T11:53:47.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.964+0000 7f14be513700 1 -- 172.21.15.104:0/2476444621 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f14b80c9480 con 0x7f14980662f0 2023-07-14T11:53:47.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.972+0000 7f14b57fa700 1 -- 172.21.15.104:0/2476444621 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367165 (secure 0 0 0) 0x7f14a0001390 con 0x7f14980662f0 2023-07-14T11:53:47.959 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:47.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.984+0000 7f14be513700 1 -- 172.21.15.104:0/2476444621 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f14980662f0 msgr2=0x7f14980687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.984+0000 7f14be513700 1 --2- 172.21.15.104:0/2476444621 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f14980662f0 0x7f14980687a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f14b81f4e60 tx=0x7f14a0007de0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.984+0000 7f14be513700 1 -- 172.21.15.104:0/2476444621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14b80afd30 msgr2=0x7f14b814c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:47.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.984+0000 7f14be513700 1 --2- 172.21.15.104:0/2476444621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14b80afd30 0x7f14b814c0b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f14a800c500 tx=0x7f14a800c810 comp rx=0 tx=0).stop 2023-07-14T11:53:47.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.984+0000 7f14be513700 1 -- 172.21.15.104:0/2476444621 shutdown_connections 2023-07-14T11:53:47.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.984+0000 7f14be513700 1 --2- 172.21.15.104:0/2476444621 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f14980662f0 0x7f14980687a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.984+0000 7f14be513700 1 --2- 172.21.15.104:0/2476444621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14b8161d00 0x7f14b814d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.984+0000 7f14be513700 1 --2- 172.21.15.104:0/2476444621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b80c9c80 0x7f14b80b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.984+0000 7f14be513700 1 --2- 172.21.15.104:0/2476444621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14b80afd30 0x7f14b814c0b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:47.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.984+0000 7f14be513700 1 -- 172.21.15.104:0/2476444621 >> 172.21.15.104:0/2476444621 conn(0x7f14b80cad30 msgr2=0x7f14b8160d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:47.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.984+0000 7f14be513700 1 -- 172.21.15.104:0/2476444621 shutdown_connections 2023-07-14T11:53:47.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:47.984+0000 7f14be513700 1 -- 172.21.15.104:0/2476444621 wait complete. 2023-07-14T11:53:47.970 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:53:47.988 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2023-07-14T11:53:47.896676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594959,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219208,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755512,"statfs":{"total":773094113280,"available":772869644288,"internally_reserved":0,"allocated":3678208,"data_stored":2186494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004716"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877499+0000","last_change":"2023-07-14T11:53:19.666668+0000","last_active":"2023-07-14T11:53:20.877499+0000","last_peered":"2023-07-14T11:53:20.877499+0000","last_clean":"2023-07-14T11:53:20.877499+0000","last_became_active":"2023-07-14T11:53:19.665905+0000","last_became_peered":"2023-07-14T11:53:19.665905+0000","last_unstale":"2023-07-14T11:53:20.877499+0000","last_undegraded":"2023-07-14T11:53:20.877499+0000","last_fullsized":"2023-07-14T11:53:20.877499+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:59.851971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568528+0000","last_change":"2023-07-14T11:53:14.600663+0000","last_active":"2023-07-14T11:53:21.568528+0000","last_peered":"2023-07-14T11:53:21.568528+0000","last_clean":"2023-07-14T11:53:21.568528+0000","last_became_active":"2023-07-14T11:53:14.600396+0000","last_became_peered":"2023-07-14T11:53:14.600396+0000","last_unstale":"2023-07-14T11:53:21.568528+0000","last_undegraded":"2023-07-14T11:53:21.568528+0000","last_fullsized":"2023-07-14T11:53:21.568528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:38.720680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662714+0000","last_change":"2023-07-14T11:53:15.580781+0000","last_active":"2023-07-14T11:53:20.662714+0000","last_peered":"2023-07-14T11:53:20.662714+0000","last_clean":"2023-07-14T11:53:20.662714+0000","last_became_active":"2023-07-14T11:53:15.580452+0000","last_became_peered":"2023-07-14T11:53:15.580452+0000","last_unstale":"2023-07-14T11:53:20.662714+0000","last_undegraded":"2023-07-14T11:53:20.662714+0000","last_fullsized":"2023-07-14T11:53:20.662714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:20:09.506259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662575+0000","last_change":"2023-07-14T11:53:19.666546+0000","last_active":"2023-07-14T11:53:20.662575+0000","last_peered":"2023-07-14T11:53:20.662575+0000","last_clean":"2023-07-14T11:53:20.662575+0000","last_became_active":"2023-07-14T11:53:19.666168+0000","last_became_peered":"2023-07-14T11:53:19.666168+0000","last_unstale":"2023-07-14T11:53:20.662575+0000","last_undegraded":"2023-07-14T11:53:20.662575+0000","last_fullsized":"2023-07-14T11:53:20.662575+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:14.566966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569423+0000","last_change":"2023-07-14T11:53:14.586141+0000","last_active":"2023-07-14T11:53:21.569423+0000","last_peered":"2023-07-14T11:53:21.569423+0000","last_clean":"2023-07-14T11:53:21.569423+0000","last_became_active":"2023-07-14T11:53:14.585836+0000","last_became_peered":"2023-07-14T11:53:14.585836+0000","last_unstale":"2023-07-14T11:53:21.569423+0000","last_undegraded":"2023-07-14T11:53:21.569423+0000","last_fullsized":"2023-07-14T11:53:21.569423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:42:48.188371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568268+0000","last_change":"2023-07-14T11:53:15.582115+0000","last_active":"2023-07-14T11:53:21.568268+0000","last_peered":"2023-07-14T11:53:21.568268+0000","last_clean":"2023-07-14T11:53:21.568268+0000","last_became_active":"2023-07-14T11:53:15.581864+0000","last_became_peered":"2023-07-14T11:53:15.581864+0000","last_unstale":"2023-07-14T11:53:21.568268+0000","last_undegraded":"2023-07-14T11:53:21.568268+0000","last_fullsized":"2023-07-14T11:53:21.568268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:31.536389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566924+0000","last_change":"2023-07-14T11:53:19.680406+0000","last_active":"2023-07-14T11:53:21.566924+0000","last_peered":"2023-07-14T11:53:21.566924+0000","last_clean":"2023-07-14T11:53:21.566924+0000","last_became_active":"2023-07-14T11:53:19.680059+0000","last_became_peered":"2023-07-14T11:53:19.680059+0000","last_unstale":"2023-07-14T11:53:21.566924+0000","last_undegraded":"2023-07-14T11:53:21.566924+0000","last_fullsized":"2023-07-14T11:53:21.566924+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:43.799463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567583+0000","last_change":"2023-07-14T11:53:14.596827+0000","last_active":"2023-07-14T11:53:21.567583+0000","last_peered":"2023-07-14T11:53:21.567583+0000","last_clean":"2023-07-14T11:53:21.567583+0000","last_became_active":"2023-07-14T11:53:14.596221+0000","last_became_peered":"2023-07-14T11:53:14.596221+0000","last_unstale":"2023-07-14T11:53:21.567583+0000","last_undegraded":"2023-07-14T11:53:21.567583+0000","last_fullsized":"2023-07-14T11:53:21.567583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:54:59.284344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663572+0000","last_change":"2023-07-14T11:53:15.607538+0000","last_active":"2023-07-14T11:53:20.663572+0000","last_peered":"2023-07-14T11:53:20.663572+0000","last_clean":"2023-07-14T11:53:20.663572+0000","last_became_active":"2023-07-14T11:53:15.592890+0000","last_became_peered":"2023-07-14T11:53:15.592890+0000","last_unstale":"2023-07-14T11:53:20.663572+0000","last_undegraded":"2023-07-14T11:53:20.663572+0000","last_fullsized":"2023-07-14T11:53:20.663572+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:30:22.821980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568105+0000","last_change":"2023-07-14T11:53:19.662893+0000","last_active":"2023-07-14T11:53:21.568105+0000","last_peered":"2023-07-14T11:53:21.568105+0000","last_clean":"2023-07-14T11:53:21.568105+0000","last_became_active":"2023-07-14T11:53:19.662484+0000","last_became_peered":"2023-07-14T11:53:19.662484+0000","last_unstale":"2023-07-14T11:53:21.568105+0000","last_undegraded":"2023-07-14T11:53:21.568105+0000","last_fullsized":"2023-07-14T11:53:21.568105+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:43.799463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566672+0000","last_change":"2023-07-14T11:53:14.590222+0000","last_active":"2023-07-14T11:53:21.566672+0000","last_peered":"2023-07-14T11:53:21.566672+0000","last_clean":"2023-07-14T11:53:21.566672+0000","last_became_active":"2023-07-14T11:53:14.589684+0000","last_became_peered":"2023-07-14T11:53:14.589684+0000","last_unstale":"2023-07-14T11:53:21.566672+0000","last_undegraded":"2023-07-14T11:53:21.566672+0000","last_fullsized":"2023-07-14T11:53:21.566672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:43.338727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663068+0000","last_change":"2023-07-14T11:53:15.592933+0000","last_active":"2023-07-14T11:53:20.663068+0000","last_peered":"2023-07-14T11:53:20.663068+0000","last_clean":"2023-07-14T11:53:20.663068+0000","last_became_active":"2023-07-14T11:53:15.592619+0000","last_became_peered":"2023-07-14T11:53:15.592619+0000","last_unstale":"2023-07-14T11:53:20.663068+0000","last_undegraded":"2023-07-14T11:53:20.663068+0000","last_fullsized":"2023-07-14T11:53:20.663068+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:41:38.469614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878123+0000","last_change":"2023-07-14T11:53:19.683165+0000","last_active":"2023-07-14T11:53:20.878123+0000","last_peered":"2023-07-14T11:53:20.878123+0000","last_clean":"2023-07-14T11:53:20.878123+0000","last_became_active":"2023-07-14T11:53:19.682676+0000","last_became_peered":"2023-07-14T11:53:19.682676+0000","last_unstale":"2023-07-14T11:53:20.878123+0000","last_undegraded":"2023-07-14T11:53:20.878123+0000","last_fullsized":"2023-07-14T11:53:20.878123+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:36:19.425078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569431+0000","last_change":"2023-07-14T11:53:14.592834+0000","last_active":"2023-07-14T11:53:21.569431+0000","last_peered":"2023-07-14T11:53:21.569431+0000","last_clean":"2023-07-14T11:53:21.569431+0000","last_became_active":"2023-07-14T11:53:14.592535+0000","last_became_peered":"2023-07-14T11:53:14.592535+0000","last_unstale":"2023-07-14T11:53:21.569431+0000","last_undegraded":"2023-07-14T11:53:21.569431+0000","last_fullsized":"2023-07-14T11:53:21.569431+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:39.643364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661401+0000","last_change":"2023-07-14T11:53:15.593173+0000","last_active":"2023-07-14T11:53:20.661401+0000","last_peered":"2023-07-14T11:53:20.661401+0000","last_clean":"2023-07-14T11:53:20.661401+0000","last_became_active":"2023-07-14T11:53:15.592873+0000","last_became_peered":"2023-07-14T11:53:15.592873+0000","last_unstale":"2023-07-14T11:53:20.661401+0000","last_undegraded":"2023-07-14T11:53:20.661401+0000","last_fullsized":"2023-07-14T11:53:20.661401+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:59:03.301440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663003+0000","last_change":"2023-07-14T11:53:19.672048+0000","last_active":"2023-07-14T11:53:20.663003+0000","last_peered":"2023-07-14T11:53:20.663003+0000","last_clean":"2023-07-14T11:53:20.663003+0000","last_became_active":"2023-07-14T11:53:19.671493+0000","last_became_peered":"2023-07-14T11:53:19.671493+0000","last_unstale":"2023-07-14T11:53:20.663003+0000","last_undegraded":"2023-07-14T11:53:20.663003+0000","last_fullsized":"2023-07-14T11:53:20.663003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:34:10.732809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664092+0000","last_change":"2023-07-14T11:53:14.591651+0000","last_active":"2023-07-14T11:53:20.664092+0000","last_peered":"2023-07-14T11:53:20.664092+0000","last_clean":"2023-07-14T11:53:20.664092+0000","last_became_active":"2023-07-14T11:53:14.591356+0000","last_became_peered":"2023-07-14T11:53:14.591356+0000","last_unstale":"2023-07-14T11:53:20.664092+0000","last_undegraded":"2023-07-14T11:53:20.664092+0000","last_fullsized":"2023-07-14T11:53:20.664092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:38:54.408913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568174+0000","last_change":"2023-07-14T11:53:15.596998+0000","last_active":"2023-07-14T11:53:21.568174+0000","last_peered":"2023-07-14T11:53:21.568174+0000","last_clean":"2023-07-14T11:53:21.568174+0000","last_became_active":"2023-07-14T11:53:15.596278+0000","last_became_peered":"2023-07-14T11:53:15.596278+0000","last_unstale":"2023-07-14T11:53:21.568174+0000","last_undegraded":"2023-07-14T11:53:21.568174+0000","last_fullsized":"2023-07-14T11:53:21.568174+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:39.727537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665493+0000","last_change":"2023-07-14T11:53:19.664365+0000","last_active":"2023-07-14T11:53:20.665493+0000","last_peered":"2023-07-14T11:53:20.665493+0000","last_clean":"2023-07-14T11:53:20.665493+0000","last_became_active":"2023-07-14T11:53:19.663967+0000","last_became_peered":"2023-07-14T11:53:19.663967+0000","last_unstale":"2023-07-14T11:53:20.665493+0000","last_undegraded":"2023-07-14T11:53:20.665493+0000","last_fullsized":"2023-07-14T11:53:20.665493+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:37:34.816983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877811+0000","last_change":"2023-07-14T11:53:14.590756+0000","last_active":"2023-07-14T11:53:20.877811+0000","last_peered":"2023-07-14T11:53:20.877811+0000","last_clean":"2023-07-14T11:53:20.877811+0000","last_became_active":"2023-07-14T11:53:14.590511+0000","last_became_peered":"2023-07-14T11:53:14.590511+0000","last_unstale":"2023-07-14T11:53:20.877811+0000","last_undegraded":"2023-07-14T11:53:20.877811+0000","last_fullsized":"2023-07-14T11:53:20.877811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:18:16.526406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568118+0000","last_change":"2023-07-14T11:53:15.612239+0000","last_active":"2023-07-14T11:53:21.568118+0000","last_peered":"2023-07-14T11:53:21.568118+0000","last_clean":"2023-07-14T11:53:21.568118+0000","last_became_active":"2023-07-14T11:53:15.611948+0000","last_became_peered":"2023-07-14T11:53:15.611948+0000","last_unstale":"2023-07-14T11:53:21.568118+0000","last_undegraded":"2023-07-14T11:53:21.568118+0000","last_fullsized":"2023-07-14T11:53:21.568118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:44.345584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567809+0000","last_change":"2023-07-14T11:53:19.680386+0000","last_active":"2023-07-14T11:53:21.567809+0000","last_peered":"2023-07-14T11:53:21.567809+0000","last_clean":"2023-07-14T11:53:21.567809+0000","last_became_active":"2023-07-14T11:53:19.674870+0000","last_became_peered":"2023-07-14T11:53:19.674870+0000","last_unstale":"2023-07-14T11:53:21.567809+0000","last_undegraded":"2023-07-14T11:53:21.567809+0000","last_fullsized":"2023-07-14T11:53:21.567809+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:00.015845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568390+0000","last_change":"2023-07-14T11:53:14.592538+0000","last_active":"2023-07-14T11:53:21.568390+0000","last_peered":"2023-07-14T11:53:21.568390+0000","last_clean":"2023-07-14T11:53:21.568390+0000","last_became_active":"2023-07-14T11:53:14.592269+0000","last_became_peered":"2023-07-14T11:53:14.592269+0000","last_unstale":"2023-07-14T11:53:21.568390+0000","last_undegraded":"2023-07-14T11:53:21.568390+0000","last_fullsized":"2023-07-14T11:53:21.568390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:17.757069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665555+0000","last_change":"2023-07-14T11:53:15.587863+0000","last_active":"2023-07-14T11:53:20.665555+0000","last_peered":"2023-07-14T11:53:20.665555+0000","last_clean":"2023-07-14T11:53:20.665555+0000","last_became_active":"2023-07-14T11:53:15.587560+0000","last_became_peered":"2023-07-14T11:53:15.587560+0000","last_unstale":"2023-07-14T11:53:20.665555+0000","last_undegraded":"2023-07-14T11:53:20.665555+0000","last_fullsized":"2023-07-14T11:53:20.665555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:01:34.117398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568069+0000","last_change":"2023-07-14T11:53:15.596816+0000","last_active":"2023-07-14T11:53:21.568069+0000","last_peered":"2023-07-14T11:53:21.568069+0000","last_clean":"2023-07-14T11:53:21.568069+0000","last_became_active":"2023-07-14T11:53:15.596544+0000","last_became_peered":"2023-07-14T11:53:15.596544+0000","last_unstale":"2023-07-14T11:53:21.568069+0000","last_undegraded":"2023-07-14T11:53:21.568069+0000","last_fullsized":"2023-07-14T11:53:21.568069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:13.267093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568721+0000","last_change":"2023-07-14T11:53:14.600709+0000","last_active":"2023-07-14T11:53:21.568721+0000","last_peered":"2023-07-14T11:53:21.568721+0000","last_clean":"2023-07-14T11:53:21.568721+0000","last_became_active":"2023-07-14T11:53:14.600430+0000","last_became_peered":"2023-07-14T11:53:14.600430+0000","last_unstale":"2023-07-14T11:53:21.568721+0000","last_undegraded":"2023-07-14T11:53:21.568721+0000","last_fullsized":"2023-07-14T11:53:21.568721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:30.529533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664703+0000","last_change":"2023-07-14T11:53:19.664675+0000","last_active":"2023-07-14T11:53:20.664703+0000","last_peered":"2023-07-14T11:53:20.664703+0000","last_clean":"2023-07-14T11:53:20.664703+0000","last_became_active":"2023-07-14T11:53:19.663889+0000","last_became_peered":"2023-07-14T11:53:19.663889+0000","last_unstale":"2023-07-14T11:53:20.664703+0000","last_undegraded":"2023-07-14T11:53:20.664703+0000","last_fullsized":"2023-07-14T11:53:20.664703+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:46.192512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568024+0000","last_change":"2023-07-14T11:53:15.606972+0000","last_active":"2023-07-14T11:53:21.568024+0000","last_peered":"2023-07-14T11:53:21.568024+0000","last_clean":"2023-07-14T11:53:21.568024+0000","last_became_active":"2023-07-14T11:53:15.606615+0000","last_became_peered":"2023-07-14T11:53:15.606615+0000","last_unstale":"2023-07-14T11:53:21.568024+0000","last_undegraded":"2023-07-14T11:53:21.568024+0000","last_fullsized":"2023-07-14T11:53:21.568024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:29.641887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569420+0000","last_change":"2023-07-14T11:53:14.595852+0000","last_active":"2023-07-14T11:53:21.569420+0000","last_peered":"2023-07-14T11:53:21.569420+0000","last_clean":"2023-07-14T11:53:21.569420+0000","last_became_active":"2023-07-14T11:53:14.589714+0000","last_became_peered":"2023-07-14T11:53:14.589714+0000","last_unstale":"2023-07-14T11:53:21.569420+0000","last_undegraded":"2023-07-14T11:53:21.569420+0000","last_fullsized":"2023-07-14T11:53:21.569420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:30:20.713856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567265+0000","last_change":"2023-07-14T11:53:19.675250+0000","last_active":"2023-07-14T11:53:21.567265+0000","last_peered":"2023-07-14T11:53:21.567265+0000","last_clean":"2023-07-14T11:53:21.567265+0000","last_became_active":"2023-07-14T11:53:19.674805+0000","last_became_peered":"2023-07-14T11:53:19.674805+0000","last_unstale":"2023-07-14T11:53:21.567265+0000","last_undegraded":"2023-07-14T11:53:21.567265+0000","last_fullsized":"2023-07-14T11:53:21.567265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:44:55.840426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568563+0000","last_change":"2023-07-14T11:53:14.587048+0000","last_active":"2023-07-14T11:53:21.568563+0000","last_peered":"2023-07-14T11:53:21.568563+0000","last_clean":"2023-07-14T11:53:21.568563+0000","last_became_active":"2023-07-14T11:53:14.586722+0000","last_became_peered":"2023-07-14T11:53:14.586722+0000","last_unstale":"2023-07-14T11:53:21.568563+0000","last_undegraded":"2023-07-14T11:53:21.568563+0000","last_fullsized":"2023-07-14T11:53:21.568563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:17.757069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878086+0000","last_change":"2023-07-14T11:53:15.598564+0000","last_active":"2023-07-14T11:53:20.878086+0000","last_peered":"2023-07-14T11:53:20.878086+0000","last_clean":"2023-07-14T11:53:20.878086+0000","last_became_active":"2023-07-14T11:53:15.596542+0000","last_became_peered":"2023-07-14T11:53:15.596542+0000","last_unstale":"2023-07-14T11:53:20.878086+0000","last_undegraded":"2023-07-14T11:53:20.878086+0000","last_fullsized":"2023-07-14T11:53:20.878086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:44:51.768501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568782+0000","last_change":"2023-07-14T11:53:19.670353+0000","last_active":"2023-07-14T11:53:21.568782+0000","last_peered":"2023-07-14T11:53:21.568782+0000","last_clean":"2023-07-14T11:53:21.568782+0000","last_became_active":"2023-07-14T11:53:19.669974+0000","last_became_peered":"2023-07-14T11:53:19.669974+0000","last_unstale":"2023-07-14T11:53:21.568782+0000","last_undegraded":"2023-07-14T11:53:21.568782+0000","last_fullsized":"2023-07-14T11:53:21.568782+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:43.632890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:33.130929+0000","last_change":"2023-07-14T11:53:14.586133+0000","last_active":"2023-07-14T11:53:33.130929+0000","last_peered":"2023-07-14T11:53:33.130929+0000","last_clean":"2023-07-14T11:53:33.130929+0000","last_became_active":"2023-07-14T11:53:14.585705+0000","last_became_peered":"2023-07-14T11:53:14.585705+0000","last_unstale":"2023-07-14T11:53:33.130929+0000","last_undegraded":"2023-07-14T11:53:33.130929+0000","last_fullsized":"2023-07-14T11:53:33.130929+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:48:36.912719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3793,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663286+0000","last_change":"2023-07-14T11:53:15.581733+0000","last_active":"2023-07-14T11:53:20.663286+0000","last_peered":"2023-07-14T11:53:20.663286+0000","last_clean":"2023-07-14T11:53:20.663286+0000","last_became_active":"2023-07-14T11:53:15.581354+0000","last_became_peered":"2023-07-14T11:53:15.581354+0000","last_unstale":"2023-07-14T11:53:20.663286+0000","last_undegraded":"2023-07-14T11:53:20.663286+0000","last_fullsized":"2023-07-14T11:53:20.663286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:24:39.278456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877743+0000","last_change":"2023-07-14T11:53:19.678063+0000","last_active":"2023-07-14T11:53:20.877743+0000","last_peered":"2023-07-14T11:53:20.877743+0000","last_clean":"2023-07-14T11:53:20.877743+0000","last_became_active":"2023-07-14T11:53:19.677638+0000","last_became_peered":"2023-07-14T11:53:19.677638+0000","last_unstale":"2023-07-14T11:53:20.877743+0000","last_undegraded":"2023-07-14T11:53:20.877743+0000","last_fullsized":"2023-07-14T11:53:20.877743+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:32:43.273428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663397+0000","last_change":"2023-07-14T11:53:15.590752+0000","last_active":"2023-07-14T11:53:20.663397+0000","last_peered":"2023-07-14T11:53:20.663397+0000","last_clean":"2023-07-14T11:53:20.663397+0000","last_became_active":"2023-07-14T11:53:15.589947+0000","last_became_peered":"2023-07-14T11:53:15.589947+0000","last_unstale":"2023-07-14T11:53:20.663397+0000","last_undegraded":"2023-07-14T11:53:20.663397+0000","last_fullsized":"2023-07-14T11:53:20.663397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:13:58.842089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665649+0000","last_change":"2023-07-14T11:53:09.145201+0000","last_active":"2023-07-14T11:53:20.665649+0000","last_peered":"2023-07-14T11:53:20.665649+0000","last_clean":"2023-07-14T11:53:20.665649+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:53:20.665649+0000","last_undegraded":"2023-07-14T11:53:20.665649+0000","last_fullsized":"2023-07-14T11:53:20.665649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:54.319358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568612+0000","last_change":"2023-07-14T11:53:14.592544+0000","last_active":"2023-07-14T11:53:21.568612+0000","last_peered":"2023-07-14T11:53:21.568612+0000","last_clean":"2023-07-14T11:53:21.568612+0000","last_became_active":"2023-07-14T11:53:14.592204+0000","last_became_peered":"2023-07-14T11:53:14.592204+0000","last_unstale":"2023-07-14T11:53:21.568612+0000","last_undegraded":"2023-07-14T11:53:21.568612+0000","last_fullsized":"2023-07-14T11:53:21.568612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:12.881399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567791+0000","last_change":"2023-07-14T11:53:19.676141+0000","last_active":"2023-07-14T11:53:21.567791+0000","last_peered":"2023-07-14T11:53:21.567791+0000","last_clean":"2023-07-14T11:53:21.567791+0000","last_became_active":"2023-07-14T11:53:19.675720+0000","last_became_peered":"2023-07-14T11:53:19.675720+0000","last_unstale":"2023-07-14T11:53:21.567791+0000","last_undegraded":"2023-07-14T11:53:21.567791+0000","last_fullsized":"2023-07-14T11:53:21.567791+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:48.417509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568246+0000","last_change":"2023-07-14T11:53:15.607026+0000","last_active":"2023-07-14T11:53:21.568246+0000","last_peered":"2023-07-14T11:53:21.568246+0000","last_clean":"2023-07-14T11:53:21.568246+0000","last_became_active":"2023-07-14T11:53:15.606605+0000","last_became_peered":"2023-07-14T11:53:15.606605+0000","last_unstale":"2023-07-14T11:53:21.568246+0000","last_undegraded":"2023-07-14T11:53:21.568246+0000","last_fullsized":"2023-07-14T11:53:21.568246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:13.888256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878901+0000","last_change":"2023-07-14T11:53:09.146150+0000","last_active":"2023-07-14T11:53:20.878901+0000","last_peered":"2023-07-14T11:53:20.878901+0000","last_clean":"2023-07-14T11:53:20.878901+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:53:20.878901+0000","last_undegraded":"2023-07-14T11:53:20.878901+0000","last_fullsized":"2023-07-14T11:53:20.878901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:07.460041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569870+0000","last_change":"2023-07-14T11:53:14.586533+0000","last_active":"2023-07-14T11:53:21.569870+0000","last_peered":"2023-07-14T11:53:21.569870+0000","last_clean":"2023-07-14T11:53:21.569870+0000","last_became_active":"2023-07-14T11:53:14.586222+0000","last_became_peered":"2023-07-14T11:53:14.586222+0000","last_unstale":"2023-07-14T11:53:21.569870+0000","last_undegraded":"2023-07-14T11:53:21.569870+0000","last_fullsized":"2023-07-14T11:53:21.569870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:38:03.789491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662421+0000","last_change":"2023-07-14T11:53:19.675677+0000","last_active":"2023-07-14T11:53:20.662421+0000","last_peered":"2023-07-14T11:53:20.662421+0000","last_clean":"2023-07-14T11:53:20.662421+0000","last_became_active":"2023-07-14T11:53:19.675259+0000","last_became_peered":"2023-07-14T11:53:19.675259+0000","last_unstale":"2023-07-14T11:53:20.662421+0000","last_undegraded":"2023-07-14T11:53:20.662421+0000","last_fullsized":"2023-07-14T11:53:20.662421+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:05.121760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567406+0000","last_change":"2023-07-14T11:53:15.600612+0000","last_active":"2023-07-14T11:53:21.567406+0000","last_peered":"2023-07-14T11:53:21.567406+0000","last_clean":"2023-07-14T11:53:21.567406+0000","last_became_active":"2023-07-14T11:53:15.600320+0000","last_became_peered":"2023-07-14T11:53:15.600320+0000","last_unstale":"2023-07-14T11:53:21.567406+0000","last_undegraded":"2023-07-14T11:53:21.567406+0000","last_fullsized":"2023-07-14T11:53:21.567406+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:55:00.291201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568702+0000","last_change":"2023-07-14T11:53:09.435876+0000","last_active":"2023-07-14T11:53:21.568702+0000","last_peered":"2023-07-14T11:53:21.568702+0000","last_clean":"2023-07-14T11:53:21.568702+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:53:21.568702+0000","last_undegraded":"2023-07-14T11:53:21.568702+0000","last_fullsized":"2023-07-14T11:53:21.568702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:34.222006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878739+0000","last_change":"2023-07-14T11:53:14.594216+0000","last_active":"2023-07-14T11:53:20.878739+0000","last_peered":"2023-07-14T11:53:20.878739+0000","last_clean":"2023-07-14T11:53:20.878739+0000","last_became_active":"2023-07-14T11:53:14.593954+0000","last_became_peered":"2023-07-14T11:53:14.593954+0000","last_unstale":"2023-07-14T11:53:20.878739+0000","last_undegraded":"2023-07-14T11:53:20.878739+0000","last_fullsized":"2023-07-14T11:53:20.878739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:12.260236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665060+0000","last_change":"2023-07-14T11:53:19.667039+0000","last_active":"2023-07-14T11:53:20.665060+0000","last_peered":"2023-07-14T11:53:20.665060+0000","last_clean":"2023-07-14T11:53:20.665060+0000","last_became_active":"2023-07-14T11:53:19.666360+0000","last_became_peered":"2023-07-14T11:53:19.666360+0000","last_unstale":"2023-07-14T11:53:20.665060+0000","last_undegraded":"2023-07-14T11:53:20.665060+0000","last_fullsized":"2023-07-14T11:53:20.665060+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:37:11.705483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878456+0000","last_change":"2023-07-14T11:53:15.598379+0000","last_active":"2023-07-14T11:53:20.878456+0000","last_peered":"2023-07-14T11:53:20.878456+0000","last_clean":"2023-07-14T11:53:20.878456+0000","last_became_active":"2023-07-14T11:53:15.596661+0000","last_became_peered":"2023-07-14T11:53:15.596661+0000","last_unstale":"2023-07-14T11:53:20.878456+0000","last_undegraded":"2023-07-14T11:53:20.878456+0000","last_fullsized":"2023-07-14T11:53:20.878456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:44.345584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878968+0000","last_change":"2023-07-14T11:53:09.434255+0000","last_active":"2023-07-14T11:53:20.878968+0000","last_peered":"2023-07-14T11:53:20.878968+0000","last_clean":"2023-07-14T11:53:20.878968+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:53:20.878968+0000","last_undegraded":"2023-07-14T11:53:20.878968+0000","last_fullsized":"2023-07-14T11:53:20.878968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:12.335710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567656+0000","last_change":"2023-07-14T11:53:14.589144+0000","last_active":"2023-07-14T11:53:21.567656+0000","last_peered":"2023-07-14T11:53:21.567656+0000","last_clean":"2023-07-14T11:53:21.567656+0000","last_became_active":"2023-07-14T11:53:14.588849+0000","last_became_peered":"2023-07-14T11:53:14.588849+0000","last_unstale":"2023-07-14T11:53:21.567656+0000","last_undegraded":"2023-07-14T11:53:21.567656+0000","last_fullsized":"2023-07-14T11:53:21.567656+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:38.554107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567120+0000","last_change":"2023-07-14T11:53:19.672406+0000","last_active":"2023-07-14T11:53:21.567120+0000","last_peered":"2023-07-14T11:53:21.567120+0000","last_clean":"2023-07-14T11:53:21.567120+0000","last_became_active":"2023-07-14T11:53:19.672009+0000","last_became_peered":"2023-07-14T11:53:19.672009+0000","last_unstale":"2023-07-14T11:53:21.567120+0000","last_undegraded":"2023-07-14T11:53:21.567120+0000","last_fullsized":"2023-07-14T11:53:21.567120+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:36:19.425078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878726+0000","last_change":"2023-07-14T11:53:15.604123+0000","last_active":"2023-07-14T11:53:20.878726+0000","last_peered":"2023-07-14T11:53:20.878726+0000","last_clean":"2023-07-14T11:53:20.878726+0000","last_became_active":"2023-07-14T11:53:15.603815+0000","last_became_peered":"2023-07-14T11:53:15.603815+0000","last_unstale":"2023-07-14T11:53:20.878726+0000","last_undegraded":"2023-07-14T11:53:20.878726+0000","last_fullsized":"2023-07-14T11:53:20.878726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:29.641887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663321+0000","last_change":"2023-07-14T11:53:14.591218+0000","last_active":"2023-07-14T11:53:20.663321+0000","last_peered":"2023-07-14T11:53:20.663321+0000","last_clean":"2023-07-14T11:53:20.663321+0000","last_became_active":"2023-07-14T11:53:14.589122+0000","last_became_peered":"2023-07-14T11:53:14.589122+0000","last_unstale":"2023-07-14T11:53:20.663321+0000","last_undegraded":"2023-07-14T11:53:20.663321+0000","last_fullsized":"2023-07-14T11:53:20.663321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:51:10.380803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566815+0000","last_change":"2023-07-14T11:53:09.147040+0000","last_active":"2023-07-14T11:53:21.566815+0000","last_peered":"2023-07-14T11:53:21.566815+0000","last_clean":"2023-07-14T11:53:21.566815+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:53:21.566815+0000","last_undegraded":"2023-07-14T11:53:21.566815+0000","last_fullsized":"2023-07-14T11:53:21.566815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:23.213672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567493+0000","last_change":"2023-07-14T11:53:19.680353+0000","last_active":"2023-07-14T11:53:21.567493+0000","last_peered":"2023-07-14T11:53:21.567493+0000","last_clean":"2023-07-14T11:53:21.567493+0000","last_became_active":"2023-07-14T11:53:19.674999+0000","last_became_peered":"2023-07-14T11:53:19.674999+0000","last_unstale":"2023-07-14T11:53:21.567493+0000","last_undegraded":"2023-07-14T11:53:21.567493+0000","last_fullsized":"2023-07-14T11:53:21.567493+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:08:07.808470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878795+0000","last_change":"2023-07-14T11:53:15.597158+0000","last_active":"2023-07-14T11:53:20.878795+0000","last_peered":"2023-07-14T11:53:20.878795+0000","last_clean":"2023-07-14T11:53:20.878795+0000","last_became_active":"2023-07-14T11:53:15.595897+0000","last_became_peered":"2023-07-14T11:53:15.595897+0000","last_unstale":"2023-07-14T11:53:20.878795+0000","last_undegraded":"2023-07-14T11:53:20.878795+0000","last_fullsized":"2023-07-14T11:53:20.878795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:39.727537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877308+0000","last_change":"2023-07-14T11:53:14.589940+0000","last_active":"2023-07-14T11:53:20.877308+0000","last_peered":"2023-07-14T11:53:20.877308+0000","last_clean":"2023-07-14T11:53:20.877308+0000","last_became_active":"2023-07-14T11:53:14.589607+0000","last_became_peered":"2023-07-14T11:53:14.589607+0000","last_unstale":"2023-07-14T11:53:20.877308+0000","last_undegraded":"2023-07-14T11:53:20.877308+0000","last_fullsized":"2023-07-14T11:53:20.877308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:28:09.703297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660746+0000","last_change":"2023-07-14T11:53:09.619294+0000","last_active":"2023-07-14T11:53:20.660746+0000","last_peered":"2023-07-14T11:53:20.660746+0000","last_clean":"2023-07-14T11:53:20.660746+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:53:20.660746+0000","last_undegraded":"2023-07-14T11:53:20.660746+0000","last_fullsized":"2023-07-14T11:53:20.660746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:19:36.961857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661632+0000","last_change":"2023-07-14T11:53:19.666668+0000","last_active":"2023-07-14T11:53:20.661632+0000","last_peered":"2023-07-14T11:53:20.661632+0000","last_clean":"2023-07-14T11:53:20.661632+0000","last_became_active":"2023-07-14T11:53:19.666170+0000","last_became_peered":"2023-07-14T11:53:19.666170+0000","last_unstale":"2023-07-14T11:53:20.661632+0000","last_undegraded":"2023-07-14T11:53:20.661632+0000","last_fullsized":"2023-07-14T11:53:20.661632+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:11:15.273201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568313+0000","last_change":"2023-07-14T11:53:15.611544+0000","last_active":"2023-07-14T11:53:21.568313+0000","last_peered":"2023-07-14T11:53:21.568313+0000","last_clean":"2023-07-14T11:53:21.568313+0000","last_became_active":"2023-07-14T11:53:15.606262+0000","last_became_peered":"2023-07-14T11:53:15.606262+0000","last_unstale":"2023-07-14T11:53:21.568313+0000","last_undegraded":"2023-07-14T11:53:21.568313+0000","last_fullsized":"2023-07-14T11:53:21.568313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566653+0000","last_change":"2023-07-14T11:53:14.588920+0000","last_active":"2023-07-14T11:53:21.566653+0000","last_peered":"2023-07-14T11:53:21.566653+0000","last_clean":"2023-07-14T11:53:21.566653+0000","last_became_active":"2023-07-14T11:53:14.588605+0000","last_became_peered":"2023-07-14T11:53:14.588605+0000","last_unstale":"2023-07-14T11:53:21.566653+0000","last_undegraded":"2023-07-14T11:53:21.566653+0000","last_fullsized":"2023-07-14T11:53:21.566653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:54.773189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567666+0000","last_change":"2023-07-14T11:53:09.146156+0000","last_active":"2023-07-14T11:53:21.567666+0000","last_peered":"2023-07-14T11:53:21.567666+0000","last_clean":"2023-07-14T11:53:21.567666+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:53:21.567666+0000","last_undegraded":"2023-07-14T11:53:21.567666+0000","last_fullsized":"2023-07-14T11:53:21.567666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:37.917368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569099+0000","last_change":"2023-07-14T11:53:19.680478+0000","last_active":"2023-07-14T11:53:21.569099+0000","last_peered":"2023-07-14T11:53:21.569099+0000","last_clean":"2023-07-14T11:53:21.569099+0000","last_became_active":"2023-07-14T11:53:19.680082+0000","last_became_peered":"2023-07-14T11:53:19.680082+0000","last_unstale":"2023-07-14T11:53:21.569099+0000","last_undegraded":"2023-07-14T11:53:21.569099+0000","last_fullsized":"2023-07-14T11:53:21.569099+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:42:53.267154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665451+0000","last_change":"2023-07-14T11:53:15.587902+0000","last_active":"2023-07-14T11:53:20.665451+0000","last_peered":"2023-07-14T11:53:20.665451+0000","last_clean":"2023-07-14T11:53:20.665451+0000","last_became_active":"2023-07-14T11:53:15.587578+0000","last_became_peered":"2023-07-14T11:53:15.587578+0000","last_unstale":"2023-07-14T11:53:20.665451+0000","last_undegraded":"2023-07-14T11:53:20.665451+0000","last_fullsized":"2023-07-14T11:53:20.665451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:43:21.367420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567625+0000","last_change":"2023-07-14T11:53:14.600123+0000","last_active":"2023-07-14T11:53:21.567625+0000","last_peered":"2023-07-14T11:53:21.567625+0000","last_clean":"2023-07-14T11:53:21.567625+0000","last_became_active":"2023-07-14T11:53:14.599830+0000","last_became_peered":"2023-07-14T11:53:14.599830+0000","last_unstale":"2023-07-14T11:53:21.567625+0000","last_undegraded":"2023-07-14T11:53:21.567625+0000","last_fullsized":"2023-07-14T11:53:21.567625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:54.937063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567091+0000","last_change":"2023-07-14T11:53:09.146556+0000","last_active":"2023-07-14T11:53:21.567091+0000","last_peered":"2023-07-14T11:53:21.567091+0000","last_clean":"2023-07-14T11:53:21.567091+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:53:21.567091+0000","last_undegraded":"2023-07-14T11:53:21.567091+0000","last_fullsized":"2023-07-14T11:53:21.567091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:07.460041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567928+0000","last_change":"2023-07-14T11:53:03.159818+0000","last_active":"2023-07-14T11:53:21.567928+0000","last_peered":"2023-07-14T11:53:21.567928+0000","last_clean":"2023-07-14T11:53:21.567928+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:53:21.567928+0000","last_undegraded":"2023-07-14T11:53:21.567928+0000","last_fullsized":"2023-07-14T11:53:21.567928+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_clean_scrub_stamp":"2023-07-14T11:53:02.054348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:17.151944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567578+0000","last_change":"2023-07-14T11:53:19.658486+0000","last_active":"2023-07-14T11:53:21.567578+0000","last_peered":"2023-07-14T11:53:21.567578+0000","last_clean":"2023-07-14T11:53:21.567578+0000","last_became_active":"2023-07-14T11:53:19.658083+0000","last_became_peered":"2023-07-14T11:53:19.658083+0000","last_unstale":"2023-07-14T11:53:21.567578+0000","last_undegraded":"2023-07-14T11:53:21.567578+0000","last_fullsized":"2023-07-14T11:53:21.567578+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:08:07.808470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878421+0000","last_change":"2023-07-14T11:53:15.596089+0000","last_active":"2023-07-14T11:53:20.878421+0000","last_peered":"2023-07-14T11:53:20.878421+0000","last_clean":"2023-07-14T11:53:20.878421+0000","last_became_active":"2023-07-14T11:53:15.594769+0000","last_became_peered":"2023-07-14T11:53:15.594769+0000","last_unstale":"2023-07-14T11:53:20.878421+0000","last_undegraded":"2023-07-14T11:53:20.878421+0000","last_fullsized":"2023-07-14T11:53:20.878421+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:55.943920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568475+0000","last_change":"2023-07-14T11:53:14.591999+0000","last_active":"2023-07-14T11:53:21.568475+0000","last_peered":"2023-07-14T11:53:21.568475+0000","last_clean":"2023-07-14T11:53:21.568475+0000","last_became_active":"2023-07-14T11:53:14.591754+0000","last_became_peered":"2023-07-14T11:53:14.591754+0000","last_unstale":"2023-07-14T11:53:21.568475+0000","last_undegraded":"2023-07-14T11:53:21.568475+0000","last_fullsized":"2023-07-14T11:53:21.568475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:39.643364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664358+0000","last_change":"2023-07-14T11:53:19.674798+0000","last_active":"2023-07-14T11:53:20.664358+0000","last_peered":"2023-07-14T11:53:20.664358+0000","last_clean":"2023-07-14T11:53:20.664358+0000","last_became_active":"2023-07-14T11:53:19.674362+0000","last_became_peered":"2023-07-14T11:53:19.674362+0000","last_unstale":"2023-07-14T11:53:20.664358+0000","last_undegraded":"2023-07-14T11:53:20.664358+0000","last_fullsized":"2023-07-14T11:53:20.664358+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:52:07.441376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664411+0000","last_change":"2023-07-14T11:53:15.582299+0000","last_active":"2023-07-14T11:53:20.664411+0000","last_peered":"2023-07-14T11:53:20.664411+0000","last_clean":"2023-07-14T11:53:20.664411+0000","last_became_active":"2023-07-14T11:53:15.582045+0000","last_became_peered":"2023-07-14T11:53:15.582045+0000","last_unstale":"2023-07-14T11:53:20.664411+0000","last_undegraded":"2023-07-14T11:53:20.664411+0000","last_fullsized":"2023-07-14T11:53:20.664411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:14:24.652057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663574+0000","last_change":"2023-07-14T11:53:14.574223+0000","last_active":"2023-07-14T11:53:20.663574+0000","last_peered":"2023-07-14T11:53:20.663574+0000","last_clean":"2023-07-14T11:53:20.663574+0000","last_became_active":"2023-07-14T11:53:14.573601+0000","last_became_peered":"2023-07-14T11:53:14.573601+0000","last_unstale":"2023-07-14T11:53:20.663574+0000","last_undegraded":"2023-07-14T11:53:20.663574+0000","last_fullsized":"2023-07-14T11:53:20.663574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:44:27.636525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568891+0000","last_change":"2023-07-14T11:53:19.671897+0000","last_active":"2023-07-14T11:53:21.568891+0000","last_peered":"2023-07-14T11:53:21.568891+0000","last_clean":"2023-07-14T11:53:21.568891+0000","last_became_active":"2023-07-14T11:53:19.671406+0000","last_became_peered":"2023-07-14T11:53:19.671406+0000","last_unstale":"2023-07-14T11:53:21.568891+0000","last_undegraded":"2023-07-14T11:53:21.568891+0000","last_fullsized":"2023-07-14T11:53:21.568891+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:37:01.963700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664358+0000","last_change":"2023-07-14T11:53:15.583755+0000","last_active":"2023-07-14T11:53:20.664358+0000","last_peered":"2023-07-14T11:53:20.664358+0000","last_clean":"2023-07-14T11:53:20.664358+0000","last_became_active":"2023-07-14T11:53:15.583491+0000","last_became_peered":"2023-07-14T11:53:15.583491+0000","last_unstale":"2023-07-14T11:53:20.664358+0000","last_undegraded":"2023-07-14T11:53:20.664358+0000","last_fullsized":"2023-07-14T11:53:20.664358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:30.105172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665591+0000","last_change":"2023-07-14T11:53:14.594378+0000","last_active":"2023-07-14T11:53:20.665591+0000","last_peered":"2023-07-14T11:53:20.665591+0000","last_clean":"2023-07-14T11:53:20.665591+0000","last_became_active":"2023-07-14T11:53:14.593863+0000","last_became_peered":"2023-07-14T11:53:14.593863+0000","last_unstale":"2023-07-14T11:53:20.665591+0000","last_undegraded":"2023-07-14T11:53:20.665591+0000","last_fullsized":"2023-07-14T11:53:20.665591+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:05:26.104433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878321+0000","last_change":"2023-07-14T11:53:19.674408+0000","last_active":"2023-07-14T11:53:20.878321+0000","last_peered":"2023-07-14T11:53:20.878321+0000","last_clean":"2023-07-14T11:53:20.878321+0000","last_became_active":"2023-07-14T11:53:19.673467+0000","last_became_peered":"2023-07-14T11:53:19.673467+0000","last_unstale":"2023-07-14T11:53:20.878321+0000","last_undegraded":"2023-07-14T11:53:20.878321+0000","last_fullsized":"2023-07-14T11:53:20.878321+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:08:07.808470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663737+0000","last_change":"2023-07-14T11:53:15.592271+0000","last_active":"2023-07-14T11:53:20.663737+0000","last_peered":"2023-07-14T11:53:20.663737+0000","last_clean":"2023-07-14T11:53:20.663737+0000","last_became_active":"2023-07-14T11:53:15.591906+0000","last_became_peered":"2023-07-14T11:53:15.591906+0000","last_unstale":"2023-07-14T11:53:20.663737+0000","last_undegraded":"2023-07-14T11:53:20.663737+0000","last_fullsized":"2023-07-14T11:53:20.663737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:17:30.232523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569314+0000","last_change":"2023-07-14T11:53:14.595162+0000","last_active":"2023-07-14T11:53:21.569314+0000","last_peered":"2023-07-14T11:53:21.569314+0000","last_clean":"2023-07-14T11:53:21.569314+0000","last_became_active":"2023-07-14T11:53:14.594754+0000","last_became_peered":"2023-07-14T11:53:14.594754+0000","last_unstale":"2023-07-14T11:53:21.569314+0000","last_undegraded":"2023-07-14T11:53:21.569314+0000","last_fullsized":"2023-07-14T11:53:21.569314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:19:28.596554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661153+0000","last_change":"2023-07-14T11:53:19.666034+0000","last_active":"2023-07-14T11:53:20.661153+0000","last_peered":"2023-07-14T11:53:20.661153+0000","last_clean":"2023-07-14T11:53:20.661153+0000","last_became_active":"2023-07-14T11:53:19.665570+0000","last_became_peered":"2023-07-14T11:53:19.665570+0000","last_unstale":"2023-07-14T11:53:20.661153+0000","last_undegraded":"2023-07-14T11:53:20.661153+0000","last_fullsized":"2023-07-14T11:53:20.661153+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:48:08.806526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567270+0000","last_change":"2023-07-14T11:53:15.612021+0000","last_active":"2023-07-14T11:53:21.567270+0000","last_peered":"2023-07-14T11:53:21.567270+0000","last_clean":"2023-07-14T11:53:21.567270+0000","last_became_active":"2023-07-14T11:53:15.611653+0000","last_became_peered":"2023-07-14T11:53:15.611653+0000","last_unstale":"2023-07-14T11:53:21.567270+0000","last_undegraded":"2023-07-14T11:53:21.567270+0000","last_fullsized":"2023-07-14T11:53:21.567270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:44:51.768501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566941+0000","last_change":"2023-07-14T11:53:14.590015+0000","last_active":"2023-07-14T11:53:21.566941+0000","last_peered":"2023-07-14T11:53:21.566941+0000","last_clean":"2023-07-14T11:53:21.566941+0000","last_became_active":"2023-07-14T11:53:14.589288+0000","last_became_peered":"2023-07-14T11:53:14.589288+0000","last_unstale":"2023-07-14T11:53:21.566941+0000","last_undegraded":"2023-07-14T11:53:21.566941+0000","last_fullsized":"2023-07-14T11:53:21.566941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:12.260236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660851+0000","last_change":"2023-07-14T11:53:19.664514+0000","last_active":"2023-07-14T11:53:20.660851+0000","last_peered":"2023-07-14T11:53:20.660851+0000","last_clean":"2023-07-14T11:53:20.660851+0000","last_became_active":"2023-07-14T11:53:19.663999+0000","last_became_peered":"2023-07-14T11:53:19.663999+0000","last_unstale":"2023-07-14T11:53:20.660851+0000","last_undegraded":"2023-07-14T11:53:20.660851+0000","last_fullsized":"2023-07-14T11:53:20.660851+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:23:47.861604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664958+0000","last_change":"2023-07-14T11:53:15.597946+0000","last_active":"2023-07-14T11:53:20.664958+0000","last_peered":"2023-07-14T11:53:20.664958+0000","last_clean":"2023-07-14T11:53:20.664958+0000","last_became_active":"2023-07-14T11:53:15.597589+0000","last_became_peered":"2023-07-14T11:53:15.597589+0000","last_unstale":"2023-07-14T11:53:20.664958+0000","last_undegraded":"2023-07-14T11:53:20.664958+0000","last_fullsized":"2023-07-14T11:53:20.664958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:51.380702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568455+0000","last_change":"2023-07-14T11:53:14.599636+0000","last_active":"2023-07-14T11:53:21.568455+0000","last_peered":"2023-07-14T11:53:21.568455+0000","last_clean":"2023-07-14T11:53:21.568455+0000","last_became_active":"2023-07-14T11:53:14.599183+0000","last_became_peered":"2023-07-14T11:53:14.599183+0000","last_unstale":"2023-07-14T11:53:21.568455+0000","last_undegraded":"2023-07-14T11:53:21.568455+0000","last_fullsized":"2023-07-14T11:53:21.568455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:30.529533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661123+0000","last_change":"2023-07-14T11:53:19.681580+0000","last_active":"2023-07-14T11:53:20.661123+0000","last_peered":"2023-07-14T11:53:20.661123+0000","last_clean":"2023-07-14T11:53:20.661123+0000","last_became_active":"2023-07-14T11:53:19.680891+0000","last_became_peered":"2023-07-14T11:53:19.680891+0000","last_unstale":"2023-07-14T11:53:20.661123+0000","last_undegraded":"2023-07-14T11:53:20.661123+0000","last_fullsized":"2023-07-14T11:53:20.661123+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:20:03.759130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663189+0000","last_change":"2023-07-14T11:53:15.610606+0000","last_active":"2023-07-14T11:53:20.663189+0000","last_peered":"2023-07-14T11:53:20.663189+0000","last_clean":"2023-07-14T11:53:20.663189+0000","last_became_active":"2023-07-14T11:53:15.610180+0000","last_became_peered":"2023-07-14T11:53:15.610180+0000","last_unstale":"2023-07-14T11:53:20.663189+0000","last_undegraded":"2023-07-14T11:53:20.663189+0000","last_fullsized":"2023-07-14T11:53:20.663189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:09.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568785+0000","last_change":"2023-07-14T11:53:14.591805+0000","last_active":"2023-07-14T11:53:21.568785+0000","last_peered":"2023-07-14T11:53:21.568785+0000","last_clean":"2023-07-14T11:53:21.568785+0000","last_became_active":"2023-07-14T11:53:14.591439+0000","last_became_peered":"2023-07-14T11:53:14.591439+0000","last_unstale":"2023-07-14T11:53:21.568785+0000","last_undegraded":"2023-07-14T11:53:21.568785+0000","last_fullsized":"2023-07-14T11:53:21.568785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:54:59.284344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660363+0000","last_change":"2023-07-14T11:53:19.675759+0000","last_active":"2023-07-14T11:53:20.660363+0000","last_peered":"2023-07-14T11:53:20.660363+0000","last_clean":"2023-07-14T11:53:20.660363+0000","last_became_active":"2023-07-14T11:53:19.675353+0000","last_became_peered":"2023-07-14T11:53:19.675353+0000","last_unstale":"2023-07-14T11:53:20.660363+0000","last_undegraded":"2023-07-14T11:53:20.660363+0000","last_fullsized":"2023-07-14T11:53:20.660363+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:35:00.376887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566299+0000","last_change":"2023-07-14T11:53:15.603352+0000","last_active":"2023-07-14T11:53:21.566299+0000","last_peered":"2023-07-14T11:53:21.566299+0000","last_clean":"2023-07-14T11:53:21.566299+0000","last_became_active":"2023-07-14T11:53:15.602777+0000","last_became_peered":"2023-07-14T11:53:15.602777+0000","last_unstale":"2023-07-14T11:53:21.566299+0000","last_undegraded":"2023-07-14T11:53:21.566299+0000","last_fullsized":"2023-07-14T11:53:21.566299+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568843+0000","last_change":"2023-07-14T11:53:14.590504+0000","last_active":"2023-07-14T11:53:21.568843+0000","last_peered":"2023-07-14T11:53:21.568843+0000","last_clean":"2023-07-14T11:53:21.568843+0000","last_became_active":"2023-07-14T11:53:14.589844+0000","last_became_peered":"2023-07-14T11:53:14.589844+0000","last_unstale":"2023-07-14T11:53:21.568843+0000","last_undegraded":"2023-07-14T11:53:21.568843+0000","last_fullsized":"2023-07-14T11:53:21.568843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:38.554107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567609+0000","last_change":"2023-07-14T11:53:19.668417+0000","last_active":"2023-07-14T11:53:21.567609+0000","last_peered":"2023-07-14T11:53:21.567609+0000","last_clean":"2023-07-14T11:53:21.567609+0000","last_became_active":"2023-07-14T11:53:19.667924+0000","last_became_peered":"2023-07-14T11:53:19.667924+0000","last_unstale":"2023-07-14T11:53:21.567609+0000","last_undegraded":"2023-07-14T11:53:21.567609+0000","last_fullsized":"2023-07-14T11:53:21.567609+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:00.015845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568675+0000","last_change":"2023-07-14T11:53:19.668219+0000","last_active":"2023-07-14T11:53:21.568675+0000","last_peered":"2023-07-14T11:53:21.568675+0000","last_clean":"2023-07-14T11:53:21.568675+0000","last_became_active":"2023-07-14T11:53:19.667862+0000","last_became_peered":"2023-07-14T11:53:19.667862+0000","last_unstale":"2023-07-14T11:53:21.568675+0000","last_undegraded":"2023-07-14T11:53:21.568675+0000","last_fullsized":"2023-07-14T11:53:21.568675+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:53:26.423183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665093+0000","last_change":"2023-07-14T11:53:14.588139+0000","last_active":"2023-07-14T11:53:20.665093+0000","last_peered":"2023-07-14T11:53:20.665093+0000","last_clean":"2023-07-14T11:53:20.665093+0000","last_became_active":"2023-07-14T11:53:14.587730+0000","last_became_peered":"2023-07-14T11:53:14.587730+0000","last_unstale":"2023-07-14T11:53:20.665093+0000","last_undegraded":"2023-07-14T11:53:20.665093+0000","last_fullsized":"2023-07-14T11:53:20.665093+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:02.335678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569769+0000","last_change":"2023-07-14T11:53:15.606880+0000","last_active":"2023-07-14T11:53:21.569769+0000","last_peered":"2023-07-14T11:53:21.569769+0000","last_clean":"2023-07-14T11:53:21.569769+0000","last_became_active":"2023-07-14T11:53:15.598546+0000","last_became_peered":"2023-07-14T11:53:15.598546+0000","last_unstale":"2023-07-14T11:53:21.569769+0000","last_undegraded":"2023-07-14T11:53:21.569769+0000","last_fullsized":"2023-07-14T11:53:21.569769+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:30:21.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569600+0000","last_change":"2023-07-14T11:53:19.676159+0000","last_active":"2023-07-14T11:53:21.569600+0000","last_peered":"2023-07-14T11:53:21.569600+0000","last_clean":"2023-07-14T11:53:21.569600+0000","last_became_active":"2023-07-14T11:53:19.675729+0000","last_became_peered":"2023-07-14T11:53:19.675729+0000","last_unstale":"2023-07-14T11:53:21.569600+0000","last_undegraded":"2023-07-14T11:53:21.569600+0000","last_fullsized":"2023-07-14T11:53:21.569600+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:49.754582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569051+0000","last_change":"2023-07-14T11:53:14.592455+0000","last_active":"2023-07-14T11:53:21.569051+0000","last_peered":"2023-07-14T11:53:21.569051+0000","last_clean":"2023-07-14T11:53:21.569051+0000","last_became_active":"2023-07-14T11:53:14.592040+0000","last_became_peered":"2023-07-14T11:53:14.592040+0000","last_unstale":"2023-07-14T11:53:21.569051+0000","last_undegraded":"2023-07-14T11:53:21.569051+0000","last_fullsized":"2023-07-14T11:53:21.569051+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:36:56.884917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568516+0000","last_change":"2023-07-14T11:53:15.593198+0000","last_active":"2023-07-14T11:53:21.568516+0000","last_peered":"2023-07-14T11:53:21.568516+0000","last_clean":"2023-07-14T11:53:21.568516+0000","last_became_active":"2023-07-14T11:53:15.592898+0000","last_became_peered":"2023-07-14T11:53:15.592898+0000","last_unstale":"2023-07-14T11:53:21.568516+0000","last_undegraded":"2023-07-14T11:53:21.568516+0000","last_fullsized":"2023-07-14T11:53:21.568516+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:13.267093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566870+0000","last_change":"2023-07-14T11:53:14.588962+0000","last_active":"2023-07-14T11:53:21.566870+0000","last_peered":"2023-07-14T11:53:21.566870+0000","last_clean":"2023-07-14T11:53:21.566870+0000","last_became_active":"2023-07-14T11:53:14.588674+0000","last_became_peered":"2023-07-14T11:53:14.588674+0000","last_unstale":"2023-07-14T11:53:21.566870+0000","last_undegraded":"2023-07-14T11:53:21.566870+0000","last_fullsized":"2023-07-14T11:53:21.566870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:32:38.194645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567389+0000","last_change":"2023-07-14T11:53:15.603164+0000","last_active":"2023-07-14T11:53:21.567389+0000","last_peered":"2023-07-14T11:53:21.567389+0000","last_clean":"2023-07-14T11:53:21.567389+0000","last_became_active":"2023-07-14T11:53:15.602752+0000","last_became_peered":"2023-07-14T11:53:15.602752+0000","last_unstale":"2023-07-14T11:53:21.567389+0000","last_undegraded":"2023-07-14T11:53:21.567389+0000","last_fullsized":"2023-07-14T11:53:21.567389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:40.650221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568701+0000","last_change":"2023-07-14T11:53:19.664284+0000","last_active":"2023-07-14T11:53:21.568701+0000","last_peered":"2023-07-14T11:53:21.568701+0000","last_clean":"2023-07-14T11:53:21.568701+0000","last_became_active":"2023-07-14T11:53:19.663903+0000","last_became_peered":"2023-07-14T11:53:19.663903+0000","last_unstale":"2023-07-14T11:53:21.568701+0000","last_undegraded":"2023-07-14T11:53:21.568701+0000","last_fullsized":"2023-07-14T11:53:21.568701+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:55:04.363127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568411+0000","last_change":"2023-07-14T11:53:14.591199+0000","last_active":"2023-07-14T11:53:21.568411+0000","last_peered":"2023-07-14T11:53:21.568411+0000","last_clean":"2023-07-14T11:53:21.568411+0000","last_became_active":"2023-07-14T11:53:14.590897+0000","last_became_peered":"2023-07-14T11:53:14.590897+0000","last_unstale":"2023-07-14T11:53:21.568411+0000","last_undegraded":"2023-07-14T11:53:21.568411+0000","last_fullsized":"2023-07-14T11:53:21.568411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:53:21.344401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568918+0000","last_change":"2023-07-14T11:53:15.594450+0000","last_active":"2023-07-14T11:53:21.568918+0000","last_peered":"2023-07-14T11:53:21.568918+0000","last_clean":"2023-07-14T11:53:21.568918+0000","last_became_active":"2023-07-14T11:53:15.594170+0000","last_became_peered":"2023-07-14T11:53:15.594170+0000","last_unstale":"2023-07-14T11:53:21.568918+0000","last_undegraded":"2023-07-14T11:53:21.568918+0000","last_fullsized":"2023-07-14T11:53:21.568918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:53:22.351258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567104+0000","last_change":"2023-07-14T11:53:19.682025+0000","last_active":"2023-07-14T11:53:21.567104+0000","last_peered":"2023-07-14T11:53:21.567104+0000","last_clean":"2023-07-14T11:53:21.567104+0000","last_became_active":"2023-07-14T11:53:19.681631+0000","last_became_peered":"2023-07-14T11:53:19.681631+0000","last_unstale":"2023-07-14T11:53:21.567104+0000","last_undegraded":"2023-07-14T11:53:21.567104+0000","last_fullsized":"2023-07-14T11:53:21.567104+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:35.608315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664672+0000","last_change":"2023-07-14T11:53:15.581069+0000","last_active":"2023-07-14T11:53:20.664672+0000","last_peered":"2023-07-14T11:53:20.664672+0000","last_clean":"2023-07-14T11:53:20.664672+0000","last_became_active":"2023-07-14T11:53:15.580609+0000","last_became_peered":"2023-07-14T11:53:15.580609+0000","last_unstale":"2023-07-14T11:53:20.664672+0000","last_undegraded":"2023-07-14T11:53:20.664672+0000","last_fullsized":"2023-07-14T11:53:20.664672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:11:11.201276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567881+0000","last_change":"2023-07-14T11:53:15.616860+0000","last_active":"2023-07-14T11:53:21.567881+0000","last_peered":"2023-07-14T11:53:21.567881+0000","last_clean":"2023-07-14T11:53:21.567881+0000","last_became_active":"2023-07-14T11:53:15.616511+0000","last_became_peered":"2023-07-14T11:53:15.616511+0000","last_unstale":"2023-07-14T11:53:21.567881+0000","last_undegraded":"2023-07-14T11:53:21.567881+0000","last_fullsized":"2023-07-14T11:53:21.567881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663897+0000","last_change":"2023-07-14T11:53:15.580935+0000","last_active":"2023-07-14T11:53:20.663897+0000","last_peered":"2023-07-14T11:53:20.663897+0000","last_clean":"2023-07-14T11:53:20.663897+0000","last_became_active":"2023-07-14T11:53:15.580624+0000","last_became_peered":"2023-07-14T11:53:15.580624+0000","last_unstale":"2023-07-14T11:53:20.663897+0000","last_undegraded":"2023-07-14T11:53:20.663897+0000","last_fullsized":"2023-07-14T11:53:20.663897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:54:27.366163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569054+0000","last_change":"2023-07-14T11:53:15.608124+0000","last_active":"2023-07-14T11:53:21.569054+0000","last_peered":"2023-07-14T11:53:21.569054+0000","last_clean":"2023-07-14T11:53:21.569054+0000","last_became_active":"2023-07-14T11:53:15.606540+0000","last_became_peered":"2023-07-14T11:53:15.606540+0000","last_unstale":"2023-07-14T11:53:21.569054+0000","last_undegraded":"2023-07-14T11:53:21.569054+0000","last_fullsized":"2023-07-14T11:53:21.569054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:36:57.891774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878952+0000","last_change":"2023-07-14T11:53:15.607644+0000","last_active":"2023-07-14T11:53:20.878952+0000","last_peered":"2023-07-14T11:53:20.878952+0000","last_clean":"2023-07-14T11:53:20.878952+0000","last_became_active":"2023-07-14T11:53:15.607318+0000","last_became_peered":"2023-07-14T11:53:15.607318+0000","last_unstale":"2023-07-14T11:53:20.878952+0000","last_undegraded":"2023-07-14T11:53:20.878952+0000","last_fullsized":"2023-07-14T11:53:20.878952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:31.536389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567297+0000","last_change":"2023-07-14T11:53:15.603168+0000","last_active":"2023-07-14T11:53:21.567297+0000","last_peered":"2023-07-14T11:53:21.567297+0000","last_clean":"2023-07-14T11:53:21.567297+0000","last_became_active":"2023-07-14T11:53:15.602791+0000","last_became_peered":"2023-07-14T11:53:15.602791+0000","last_unstale":"2023-07-14T11:53:21.567297+0000","last_undegraded":"2023-07-14T11:53:21.567297+0000","last_fullsized":"2023-07-14T11:53:21.567297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:18.763925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878622+0000","last_change":"2023-07-14T11:53:15.600466+0000","last_active":"2023-07-14T11:53:20.878622+0000","last_peered":"2023-07-14T11:53:20.878622+0000","last_clean":"2023-07-14T11:53:20.878622+0000","last_became_active":"2023-07-14T11:53:15.600232+0000","last_became_peered":"2023-07-14T11:53:15.600232+0000","last_unstale":"2023-07-14T11:53:20.878622+0000","last_undegraded":"2023-07-14T11:53:20.878622+0000","last_fullsized":"2023-07-14T11:53:20.878622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878439+0000","last_change":"2023-07-14T11:53:15.605971+0000","last_active":"2023-07-14T11:53:20.878439+0000","last_peered":"2023-07-14T11:53:20.878439+0000","last_clean":"2023-07-14T11:53:20.878439+0000","last_became_active":"2023-07-14T11:53:15.605661+0000","last_became_peered":"2023-07-14T11:53:15.605661+0000","last_unstale":"2023-07-14T11:53:20.878439+0000","last_undegraded":"2023-07-14T11:53:20.878439+0000","last_fullsized":"2023-07-14T11:53:20.878439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567617+0000","last_change":"2023-07-14T11:53:15.606414+0000","last_active":"2023-07-14T11:53:21.567617+0000","last_peered":"2023-07-14T11:53:21.567617+0000","last_clean":"2023-07-14T11:53:21.567617+0000","last_became_active":"2023-07-14T11:53:15.605906+0000","last_became_peered":"2023-07-14T11:53:15.605906+0000","last_unstale":"2023-07-14T11:53:21.567617+0000","last_undegraded":"2023-07-14T11:53:21.567617+0000","last_fullsized":"2023-07-14T11:53:21.567617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567728+0000","last_change":"2023-07-14T11:53:15.598006+0000","last_active":"2023-07-14T11:53:21.567728+0000","last_peered":"2023-07-14T11:53:21.567728+0000","last_clean":"2023-07-14T11:53:21.567728+0000","last_became_active":"2023-07-14T11:53:15.597749+0000","last_became_peered":"2023-07-14T11:53:15.597749+0000","last_unstale":"2023-07-14T11:53:21.567728+0000","last_undegraded":"2023-07-14T11:53:21.567728+0000","last_fullsized":"2023-07-14T11:53:21.567728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567052+0000","last_change":"2023-07-14T11:53:15.600730+0000","last_active":"2023-07-14T11:53:21.567052+0000","last_peered":"2023-07-14T11:53:21.567052+0000","last_clean":"2023-07-14T11:53:21.567052+0000","last_became_active":"2023-07-14T11:53:15.600299+0000","last_became_peered":"2023-07-14T11:53:15.600299+0000","last_unstale":"2023-07-14T11:53:21.567052+0000","last_undegraded":"2023-07-14T11:53:21.567052+0000","last_fullsized":"2023-07-14T11:53:21.567052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:18:17.533263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568245+0000","last_change":"2023-07-14T11:53:15.619732+0000","last_active":"2023-07-14T11:53:21.568245+0000","last_peered":"2023-07-14T11:53:21.568245+0000","last_clean":"2023-07-14T11:53:21.568245+0000","last_became_active":"2023-07-14T11:53:15.619425+0000","last_became_peered":"2023-07-14T11:53:15.619425+0000","last_unstale":"2023-07-14T11:53:21.568245+0000","last_undegraded":"2023-07-14T11:53:21.568245+0000","last_fullsized":"2023-07-14T11:53:21.568245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:13.888256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663540+0000","last_change":"2023-07-14T11:53:15.586169+0000","last_active":"2023-07-14T11:53:20.663540+0000","last_peered":"2023-07-14T11:53:20.663540+0000","last_clean":"2023-07-14T11:53:20.663540+0000","last_became_active":"2023-07-14T11:53:15.585876+0000","last_became_peered":"2023-07-14T11:53:15.585876+0000","last_unstale":"2023-07-14T11:53:20.663540+0000","last_undegraded":"2023-07-14T11:53:20.663540+0000","last_fullsized":"2023-07-14T11:53:20.663540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:47:20.678085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661229+0000","last_change":"2023-07-14T11:53:15.584035+0000","last_active":"2023-07-14T11:53:20.661229+0000","last_peered":"2023-07-14T11:53:20.661229+0000","last_clean":"2023-07-14T11:53:20.661229+0000","last_became_active":"2023-07-14T11:53:15.582303+0000","last_became_peered":"2023-07-14T11:53:15.582303+0000","last_unstale":"2023-07-14T11:53:20.661229+0000","last_undegraded":"2023-07-14T11:53:20.661229+0000","last_fullsized":"2023-07-14T11:53:20.661229+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:48:16.619167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664187+0000","last_change":"2023-07-14T11:53:15.578919+0000","last_active":"2023-07-14T11:53:20.664187+0000","last_peered":"2023-07-14T11:53:20.664187+0000","last_clean":"2023-07-14T11:53:20.664187+0000","last_became_active":"2023-07-14T11:53:15.578590+0000","last_became_peered":"2023-07-14T11:53:15.578590+0000","last_unstale":"2023-07-14T11:53:20.664187+0000","last_undegraded":"2023-07-14T11:53:20.664187+0000","last_fullsized":"2023-07-14T11:53:20.664187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:03:59.635384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568314+0000","last_change":"2023-07-14T11:53:15.613102+0000","last_active":"2023-07-14T11:53:21.568314+0000","last_peered":"2023-07-14T11:53:21.568314+0000","last_clean":"2023-07-14T11:53:21.568314+0000","last_became_active":"2023-07-14T11:53:15.612540+0000","last_became_peered":"2023-07-14T11:53:15.612540+0000","last_unstale":"2023-07-14T11:53:21.568314+0000","last_undegraded":"2023-07-14T11:53:21.568314+0000","last_fullsized":"2023-07-14T11:53:21.568314+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:29.641887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662627+0000","last_change":"2023-07-14T11:53:15.592089+0000","last_active":"2023-07-14T11:53:20.662627+0000","last_peered":"2023-07-14T11:53:20.662627+0000","last_clean":"2023-07-14T11:53:20.662627+0000","last_became_active":"2023-07-14T11:53:15.591826+0000","last_became_peered":"2023-07-14T11:53:15.591826+0000","last_unstale":"2023-07-14T11:53:20.662627+0000","last_undegraded":"2023-07-14T11:53:20.662627+0000","last_fullsized":"2023-07-14T11:53:20.662627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:06:57.233859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568086+0000","last_change":"2023-07-14T11:53:15.606636+0000","last_active":"2023-07-14T11:53:21.568086+0000","last_peered":"2023-07-14T11:53:21.568086+0000","last_clean":"2023-07-14T11:53:21.568086+0000","last_became_active":"2023-07-14T11:53:15.606067+0000","last_became_peered":"2023-07-14T11:53:15.606067+0000","last_unstale":"2023-07-14T11:53:21.568086+0000","last_undegraded":"2023-07-14T11:53:21.568086+0000","last_fullsized":"2023-07-14T11:53:21.568086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:13.267093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566709+0000","last_change":"2023-07-14T11:53:15.599253+0000","last_active":"2023-07-14T11:53:21.566709+0000","last_peered":"2023-07-14T11:53:21.566709+0000","last_clean":"2023-07-14T11:53:21.566709+0000","last_became_active":"2023-07-14T11:53:15.598955+0000","last_became_peered":"2023-07-14T11:53:15.598955+0000","last_unstale":"2023-07-14T11:53:21.566709+0000","last_undegraded":"2023-07-14T11:53:21.566709+0000","last_fullsized":"2023-07-14T11:53:21.566709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660486+0000","last_change":"2023-07-14T11:53:15.592119+0000","last_active":"2023-07-14T11:53:20.660486+0000","last_peered":"2023-07-14T11:53:20.660486+0000","last_clean":"2023-07-14T11:53:20.660486+0000","last_became_active":"2023-07-14T11:53:15.591822+0000","last_became_peered":"2023-07-14T11:53:15.591822+0000","last_unstale":"2023-07-14T11:53:20.660486+0000","last_undegraded":"2023-07-14T11:53:20.660486+0000","last_fullsized":"2023-07-14T11:53:20.660486+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:04:29.483383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661402+0000","last_change":"2023-07-14T11:53:15.590820+0000","last_active":"2023-07-14T11:53:20.661402+0000","last_peered":"2023-07-14T11:53:20.661402+0000","last_clean":"2023-07-14T11:53:20.661402+0000","last_became_active":"2023-07-14T11:53:15.590485+0000","last_became_peered":"2023-07-14T11:53:15.590485+0000","last_unstale":"2023-07-14T11:53:20.661402+0000","last_undegraded":"2023-07-14T11:53:20.661402+0000","last_fullsized":"2023-07-14T11:53:20.661402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:42.120586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664155+0000","last_change":"2023-07-14T11:53:15.589398+0000","last_active":"2023-07-14T11:53:20.664155+0000","last_peered":"2023-07-14T11:53:20.664155+0000","last_clean":"2023-07-14T11:53:20.664155+0000","last_became_active":"2023-07-14T11:53:15.589111+0000","last_became_peered":"2023-07-14T11:53:15.589111+0000","last_unstale":"2023-07-14T11:53:20.664155+0000","last_undegraded":"2023-07-14T11:53:20.664155+0000","last_fullsized":"2023-07-14T11:53:20.664155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:40:44.358262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569248+0000","last_change":"2023-07-14T11:53:15.599081+0000","last_active":"2023-07-14T11:53:21.569248+0000","last_peered":"2023-07-14T11:53:21.569248+0000","last_clean":"2023-07-14T11:53:21.569248+0000","last_became_active":"2023-07-14T11:53:15.598786+0000","last_became_peered":"2023-07-14T11:53:15.598786+0000","last_unstale":"2023-07-14T11:53:21.569248+0000","last_undegraded":"2023-07-14T11:53:21.569248+0000","last_fullsized":"2023-07-14T11:53:21.569248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:45.682657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567984+0000","last_change":"2023-07-14T11:53:15.613338+0000","last_active":"2023-07-14T11:53:21.567984+0000","last_peered":"2023-07-14T11:53:21.567984+0000","last_clean":"2023-07-14T11:53:21.567984+0000","last_became_active":"2023-07-14T11:53:15.612876+0000","last_became_peered":"2023-07-14T11:53:15.612876+0000","last_unstale":"2023-07-14T11:53:21.567984+0000","last_undegraded":"2023-07-14T11:53:21.567984+0000","last_fullsized":"2023-07-14T11:53:21.567984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566164+0000","last_change":"2023-07-14T11:53:15.608445+0000","last_active":"2023-07-14T11:53:21.566164+0000","last_peered":"2023-07-14T11:53:21.566164+0000","last_clean":"2023-07-14T11:53:21.566164+0000","last_became_active":"2023-07-14T11:53:15.608146+0000","last_became_peered":"2023-07-14T11:53:15.608146+0000","last_unstale":"2023-07-14T11:53:21.566164+0000","last_undegraded":"2023-07-14T11:53:21.566164+0000","last_fullsized":"2023-07-14T11:53:21.566164+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:55.943920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567855+0000","last_change":"2023-07-14T11:53:15.606963+0000","last_active":"2023-07-14T11:53:21.567855+0000","last_peered":"2023-07-14T11:53:21.567855+0000","last_clean":"2023-07-14T11:53:21.567855+0000","last_became_active":"2023-07-14T11:53:15.606397+0000","last_became_peered":"2023-07-14T11:53:15.606397+0000","last_unstale":"2023-07-14T11:53:21.567855+0000","last_undegraded":"2023-07-14T11:53:21.567855+0000","last_fullsized":"2023-07-14T11:53:21.567855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663074+0000","last_change":"2023-07-14T11:53:15.607539+0000","last_active":"2023-07-14T11:53:20.663074+0000","last_peered":"2023-07-14T11:53:20.663074+0000","last_clean":"2023-07-14T11:53:20.663074+0000","last_became_active":"2023-07-14T11:53:15.593279+0000","last_became_peered":"2023-07-14T11:53:15.593279+0000","last_unstale":"2023-07-14T11:53:20.663074+0000","last_undegraded":"2023-07-14T11:53:20.663074+0000","last_fullsized":"2023-07-14T11:53:20.663074+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:34.163470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569328+0000","last_change":"2023-07-14T11:53:15.592332+0000","last_active":"2023-07-14T11:53:21.569328+0000","last_peered":"2023-07-14T11:53:21.569328+0000","last_clean":"2023-07-14T11:53:21.569328+0000","last_became_active":"2023-07-14T11:53:15.591983+0000","last_became_peered":"2023-07-14T11:53:15.591983+0000","last_unstale":"2023-07-14T11:53:21.569328+0000","last_undegraded":"2023-07-14T11:53:21.569328+0000","last_fullsized":"2023-07-14T11:53:21.569328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:18.763925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665081+0000","last_change":"2023-07-14T11:53:15.594122+0000","last_active":"2023-07-14T11:53:20.665081+0000","last_peered":"2023-07-14T11:53:20.665081+0000","last_clean":"2023-07-14T11:53:20.665081+0000","last_became_active":"2023-07-14T11:53:15.593773+0000","last_became_peered":"2023-07-14T11:53:15.593773+0000","last_unstale":"2023-07-14T11:53:20.665081+0000","last_undegraded":"2023-07-14T11:53:20.665081+0000","last_fullsized":"2023-07-14T11:53:20.665081+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:59:21.935826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664107+0000","last_change":"2023-07-14T11:53:15.593314+0000","last_active":"2023-07-14T11:53:20.664107+0000","last_peered":"2023-07-14T11:53:20.664107+0000","last_clean":"2023-07-14T11:53:20.664107+0000","last_became_active":"2023-07-14T11:53:15.592778+0000","last_became_peered":"2023-07-14T11:53:15.592778+0000","last_unstale":"2023-07-14T11:53:20.664107+0000","last_undegraded":"2023-07-14T11:53:20.664107+0000","last_fullsized":"2023-07-14T11:53:20.664107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:37.263541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568937+0000","last_change":"2023-07-14T11:53:15.599579+0000","last_active":"2023-07-14T11:53:21.568937+0000","last_peered":"2023-07-14T11:53:21.568937+0000","last_clean":"2023-07-14T11:53:21.568937+0000","last_became_active":"2023-07-14T11:53:15.599380+0000","last_became_peered":"2023-07-14T11:53:15.599380+0000","last_unstale":"2023-07-14T11:53:21.568937+0000","last_undegraded":"2023-07-14T11:53:21.568937+0000","last_fullsized":"2023-07-14T11:53:21.568937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:19:29.603411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661719+0000","last_change":"2023-07-14T11:53:15.594232+0000","last_active":"2023-07-14T11:53:20.661719+0000","last_peered":"2023-07-14T11:53:20.661719+0000","last_clean":"2023-07-14T11:53:20.661719+0000","last_became_active":"2023-07-14T11:53:15.594049+0000","last_became_peered":"2023-07-14T11:53:15.594049+0000","last_unstale":"2023-07-14T11:53:20.661719+0000","last_undegraded":"2023-07-14T11:53:20.661719+0000","last_fullsized":"2023-07-14T11:53:20.661719+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:51.380702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4591,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":925696,"data_stored":718949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.46}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.502}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":311296,"data_stored":125224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.371}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.704}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":323584,"data_stored":128421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":901120,"data_stored":715522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:53:47.996 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:53:48.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.212+0000 7ffa26cf9700 1 Processor -- start 2023-07-14T11:53:48.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.216+0000 7ffa26cf9700 1 -- start start 2023-07-14T11:53:48.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.216+0000 7ffa26cf9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa200ce5e0 0x7ffa20158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.216+0000 7ffa26cf9700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ffa200c9820 0x7ffa20159520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.216+0000 7ffa26cf9700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa20159d60 con 0x7ffa2006dc20 2023-07-14T11:53:48.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.216+0000 7ffa26cf9700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa200cd250 con 0x7ffa200c9820 2023-07-14T11:53:48.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.216+0000 7ffa26cf9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa200c8a60 con 0x7ffa200ce5e0 2023-07-14T11:53:48.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.216+0000 7ffa25cf7700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ffa2006dc20 0x7ffa200cded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42580/0 (socket says 172.21.15.104:42580) 2023-07-14T11:53:48.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.216+0000 7ffa25cf7700 1 -- 172.21.15.104:0/735153402 learned_addr learned my addr 172.21.15.104:0/735153402 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:48.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.216+0000 7ffa264f8700 1 --2- 172.21.15.104:0/735153402 >> v2:172.21.15.104:3300/0 conn(0x7ffa200c9820 0x7ffa20159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.216+0000 7ffa254f6700 1 --2- 172.21.15.104:0/735153402 >> v2:172.21.15.121:3300/0 conn(0x7ffa200ce5e0 0x7ffa20158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.216+0000 7ffa24cf5700 1 -- 172.21.15.104:0/735153402 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600788313 0 0) 0x7ffa20159d60 con 0x7ffa2006dc20 2023-07-14T11:53:48.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.216+0000 7ffa264f8700 1 -- 172.21.15.104:0/735153402 >> v2:172.21.15.121:3300/0 conn(0x7ffa200ce5e0 msgr2=0x7ffa20158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:48.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.216+0000 7ffa264f8700 1 --2- 172.21.15.104:0/735153402 >> v2:172.21.15.121:3300/0 conn(0x7ffa200ce5e0 0x7ffa20158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.216+0000 7ffa264f8700 1 -- 172.21.15.104:0/735153402 >> v1:172.21.15.104:6789/0 conn(0x7ffa2006dc20 legacy=0x7ffa200cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.216+0000 7ffa264f8700 1 -- 172.21.15.104:0/735153402 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa200c8d40 con 0x7ffa200c9820 2023-07-14T11:53:48.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.220+0000 7ffa264f8700 1 --2- 172.21.15.104:0/735153402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa200c9820 0x7ffa20159520 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffa200cbd50 tx=0x7ffa18009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fc8f87607588a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:48.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.220+0000 7ffa24cf5700 1 -- 172.21.15.104:0/735153402 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18011830 con 0x7ffa200c9820 2023-07-14T11:53:48.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.220+0000 7ffa24cf5700 1 -- 172.21.15.104:0/735153402 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa18011990 con 0x7ffa200c9820 2023-07-14T11:53:48.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.220+0000 7ffa24cf5700 1 -- 172.21.15.104:0/735153402 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18011d20 con 0x7ffa200c9820 2023-07-14T11:53:48.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.220+0000 7ffa26cf9700 1 -- 172.21.15.104:0/735153402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa200c9820 msgr2=0x7ffa20159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.220+0000 7ffa26cf9700 1 --2- 172.21.15.104:0/735153402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa200c9820 0x7ffa20159520 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffa200cbd50 tx=0x7ffa18009b30 comp rx=0 tx=0).stop 2023-07-14T11:53:48.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.220+0000 7ffa26cf9700 1 -- 172.21.15.104:0/735153402 shutdown_connections 2023-07-14T11:53:48.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.220+0000 7ffa26cf9700 1 --2- 172.21.15.104:0/735153402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa200c9820 0x7ffa20159520 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.220+0000 7ffa26cf9700 1 --2- 172.21.15.104:0/735153402 >> v2:172.21.15.121:3300/0 conn(0x7ffa200ce5e0 0x7ffa20158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.220+0000 7ffa26cf9700 1 -- 172.21.15.104:0/735153402 >> 172.21.15.104:0/735153402 conn(0x7ffa2006e960 msgr2=0x7ffa2006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:48.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.220+0000 7ffa26cf9700 1 -- 172.21.15.104:0/735153402 shutdown_connections 2023-07-14T11:53:48.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.220+0000 7ffa26cf9700 1 -- 172.21.15.104:0/735153402 wait complete. 2023-07-14T11:53:48.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa26cf9700 1 Processor -- start 2023-07-14T11:53:48.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa26cf9700 1 -- start start 2023-07-14T11:53:48.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa26cf9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa2006dc20 0x7ffa200acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa26cf9700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa200c9820 0x7ffa2014f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa26cf9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa200ce5e0 0x7ffa200c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa26cf9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa200c78e0 con 0x7ffa200ce5e0 2023-07-14T11:53:48.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa26cf9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa200c7bc0 con 0x7ffa2006dc20 2023-07-14T11:53:48.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa26cf9700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa200c7ea0 con 0x7ffa200c9820 2023-07-14T11:53:48.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa264f8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa200ce5e0 0x7ffa200c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa25cf7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa2006dc20 0x7ffa200acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa264f8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa200ce5e0 0x7ffa200c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38562/0 (socket says 172.21.15.104:38562) 2023-07-14T11:53:48.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa264f8700 1 -- 172.21.15.104:0/1390179668 learned_addr learned my addr 172.21.15.104:0/1390179668 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:48.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa254f6700 1 --2- 172.21.15.104:0/1390179668 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa200c9820 0x7ffa2014f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa264f8700 1 -- 172.21.15.104:0/1390179668 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa200c9820 msgr2=0x7ffa2014f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa264f8700 1 --2- 172.21.15.104:0/1390179668 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa200c9820 0x7ffa2014f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa264f8700 1 -- 172.21.15.104:0/1390179668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa2006dc20 msgr2=0x7ffa200acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa264f8700 1 --2- 172.21.15.104:0/1390179668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa2006dc20 0x7ffa200acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa264f8700 1 -- 172.21.15.104:0/1390179668 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa180094b0 con 0x7ffa200ce5e0 2023-07-14T11:53:48.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa264f8700 1 --2- 172.21.15.104:0/1390179668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa200ce5e0 0x7ffa200c7040 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffa180090b0 tx=0x7ffa18004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:48.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa167fc700 1 -- 172.21.15.104:0/1390179668 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa180045d0 con 0x7ffa200ce5e0 2023-07-14T11:53:48.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa167fc700 1 -- 172.21.15.104:0/1390179668 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1801e070 con 0x7ffa200ce5e0 2023-07-14T11:53:48.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa26cf9700 1 -- 172.21.15.104:0/1390179668 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa200c8180 con 0x7ffa200ce5e0 2023-07-14T11:53:48.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa167fc700 1 -- 172.21.15.104:0/1390179668 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa180038c0 con 0x7ffa200ce5e0 2023-07-14T11:53:48.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.224+0000 7ffa26cf9700 1 -- 172.21.15.104:0/1390179668 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa200c7580 con 0x7ffa200ce5e0 2023-07-14T11:53:48.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.228+0000 7ffa26cf9700 1 -- 172.21.15.104:0/1390179668 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa2004e500 con 0x7ffa200ce5e0 2023-07-14T11:53:48.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.228+0000 7ffa167fc700 1 -- 172.21.15.104:0/1390179668 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ffa18003a20 con 0x7ffa200ce5e0 2023-07-14T11:53:48.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.232+0000 7ffa167fc700 1 --2- 172.21.15.104:0/1390179668 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa040662f0 0x7ffa040687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.232+0000 7ffa167fc700 1 -- 172.21.15.104:0/1390179668 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ffa18084ea0 con 0x7ffa200ce5e0 2023-07-14T11:53:48.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.232+0000 7ffa25cf7700 1 --2- 172.21.15.104:0/1390179668 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa040662f0 0x7ffa040687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.232+0000 7ffa25cf7700 1 --2- 172.21.15.104:0/1390179668 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa040662f0 0x7ffa040687a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0012e0 tx=0x7ffa1c001aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:48.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.236+0000 7ffa167fc700 1 -- 172.21.15.104:0/1390179668 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ffa18052020 con 0x7ffa200ce5e0 2023-07-14T11:53:48.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.392+0000 7ffa26cf9700 1 -- 172.21.15.104:0/1390179668 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffa200c7d30 con 0x7ffa040662f0 2023-07-14T11:53:48.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.400+0000 7ffa167fc700 1 -- 172.21.15.104:0/1390179668 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367165 (secure 0 0 0) 0x7ffa200c7d30 con 0x7ffa040662f0 2023-07-14T11:53:48.388 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:48.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.412+0000 7ffa26cf9700 1 -- 172.21.15.104:0/1390179668 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa040662f0 msgr2=0x7ffa040687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.412+0000 7ffa26cf9700 1 --2- 172.21.15.104:0/1390179668 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa040662f0 0x7ffa040687a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0012e0 tx=0x7ffa1c001aa0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.412+0000 7ffa26cf9700 1 -- 172.21.15.104:0/1390179668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa200ce5e0 msgr2=0x7ffa200c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.412+0000 7ffa26cf9700 1 --2- 172.21.15.104:0/1390179668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa200ce5e0 0x7ffa200c7040 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffa180090b0 tx=0x7ffa18004770 comp rx=0 tx=0).stop 2023-07-14T11:53:48.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.412+0000 7ffa26cf9700 1 -- 172.21.15.104:0/1390179668 shutdown_connections 2023-07-14T11:53:48.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.412+0000 7ffa26cf9700 1 --2- 172.21.15.104:0/1390179668 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa040662f0 0x7ffa040687a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.412+0000 7ffa26cf9700 1 --2- 172.21.15.104:0/1390179668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa200ce5e0 0x7ffa200c7040 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.412+0000 7ffa26cf9700 1 --2- 172.21.15.104:0/1390179668 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa200c9820 0x7ffa2014f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.412+0000 7ffa26cf9700 1 --2- 172.21.15.104:0/1390179668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa2006dc20 0x7ffa200acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.412+0000 7ffa26cf9700 1 -- 172.21.15.104:0/1390179668 >> 172.21.15.104:0/1390179668 conn(0x7ffa2006e960 msgr2=0x7ffa2006ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:48.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.412+0000 7ffa26cf9700 1 -- 172.21.15.104:0/1390179668 shutdown_connections 2023-07-14T11:53:48.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.412+0000 7ffa26cf9700 1 -- 172.21.15.104:0/1390179668 wait complete. 2023-07-14T11:53:48.397 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:53:48.416 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2023-07-14T11:53:47.896676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594959,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219208,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755512,"statfs":{"total":773094113280,"available":772869644288,"internally_reserved":0,"allocated":3678208,"data_stored":2186494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004716"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877499+0000","last_change":"2023-07-14T11:53:19.666668+0000","last_active":"2023-07-14T11:53:20.877499+0000","last_peered":"2023-07-14T11:53:20.877499+0000","last_clean":"2023-07-14T11:53:20.877499+0000","last_became_active":"2023-07-14T11:53:19.665905+0000","last_became_peered":"2023-07-14T11:53:19.665905+0000","last_unstale":"2023-07-14T11:53:20.877499+0000","last_undegraded":"2023-07-14T11:53:20.877499+0000","last_fullsized":"2023-07-14T11:53:20.877499+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:59.851971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568528+0000","last_change":"2023-07-14T11:53:14.600663+0000","last_active":"2023-07-14T11:53:21.568528+0000","last_peered":"2023-07-14T11:53:21.568528+0000","last_clean":"2023-07-14T11:53:21.568528+0000","last_became_active":"2023-07-14T11:53:14.600396+0000","last_became_peered":"2023-07-14T11:53:14.600396+0000","last_unstale":"2023-07-14T11:53:21.568528+0000","last_undegraded":"2023-07-14T11:53:21.568528+0000","last_fullsized":"2023-07-14T11:53:21.568528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:38.720680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662714+0000","last_change":"2023-07-14T11:53:15.580781+0000","last_active":"2023-07-14T11:53:20.662714+0000","last_peered":"2023-07-14T11:53:20.662714+0000","last_clean":"2023-07-14T11:53:20.662714+0000","last_became_active":"2023-07-14T11:53:15.580452+0000","last_became_peered":"2023-07-14T11:53:15.580452+0000","last_unstale":"2023-07-14T11:53:20.662714+0000","last_undegraded":"2023-07-14T11:53:20.662714+0000","last_fullsized":"2023-07-14T11:53:20.662714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:20:09.506259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662575+0000","last_change":"2023-07-14T11:53:19.666546+0000","last_active":"2023-07-14T11:53:20.662575+0000","last_peered":"2023-07-14T11:53:20.662575+0000","last_clean":"2023-07-14T11:53:20.662575+0000","last_became_active":"2023-07-14T11:53:19.666168+0000","last_became_peered":"2023-07-14T11:53:19.666168+0000","last_unstale":"2023-07-14T11:53:20.662575+0000","last_undegraded":"2023-07-14T11:53:20.662575+0000","last_fullsized":"2023-07-14T11:53:20.662575+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:14.566966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569423+0000","last_change":"2023-07-14T11:53:14.586141+0000","last_active":"2023-07-14T11:53:21.569423+0000","last_peered":"2023-07-14T11:53:21.569423+0000","last_clean":"2023-07-14T11:53:21.569423+0000","last_became_active":"2023-07-14T11:53:14.585836+0000","last_became_peered":"2023-07-14T11:53:14.585836+0000","last_unstale":"2023-07-14T11:53:21.569423+0000","last_undegraded":"2023-07-14T11:53:21.569423+0000","last_fullsized":"2023-07-14T11:53:21.569423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:42:48.188371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568268+0000","last_change":"2023-07-14T11:53:15.582115+0000","last_active":"2023-07-14T11:53:21.568268+0000","last_peered":"2023-07-14T11:53:21.568268+0000","last_clean":"2023-07-14T11:53:21.568268+0000","last_became_active":"2023-07-14T11:53:15.581864+0000","last_became_peered":"2023-07-14T11:53:15.581864+0000","last_unstale":"2023-07-14T11:53:21.568268+0000","last_undegraded":"2023-07-14T11:53:21.568268+0000","last_fullsized":"2023-07-14T11:53:21.568268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:31.536389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566924+0000","last_change":"2023-07-14T11:53:19.680406+0000","last_active":"2023-07-14T11:53:21.566924+0000","last_peered":"2023-07-14T11:53:21.566924+0000","last_clean":"2023-07-14T11:53:21.566924+0000","last_became_active":"2023-07-14T11:53:19.680059+0000","last_became_peered":"2023-07-14T11:53:19.680059+0000","last_unstale":"2023-07-14T11:53:21.566924+0000","last_undegraded":"2023-07-14T11:53:21.566924+0000","last_fullsized":"2023-07-14T11:53:21.566924+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:43.799463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567583+0000","last_change":"2023-07-14T11:53:14.596827+0000","last_active":"2023-07-14T11:53:21.567583+0000","last_peered":"2023-07-14T11:53:21.567583+0000","last_clean":"2023-07-14T11:53:21.567583+0000","last_became_active":"2023-07-14T11:53:14.596221+0000","last_became_peered":"2023-07-14T11:53:14.596221+0000","last_unstale":"2023-07-14T11:53:21.567583+0000","last_undegraded":"2023-07-14T11:53:21.567583+0000","last_fullsized":"2023-07-14T11:53:21.567583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:54:59.284344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663572+0000","last_change":"2023-07-14T11:53:15.607538+0000","last_active":"2023-07-14T11:53:20.663572+0000","last_peered":"2023-07-14T11:53:20.663572+0000","last_clean":"2023-07-14T11:53:20.663572+0000","last_became_active":"2023-07-14T11:53:15.592890+0000","last_became_peered":"2023-07-14T11:53:15.592890+0000","last_unstale":"2023-07-14T11:53:20.663572+0000","last_undegraded":"2023-07-14T11:53:20.663572+0000","last_fullsized":"2023-07-14T11:53:20.663572+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:30:22.821980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568105+0000","last_change":"2023-07-14T11:53:19.662893+0000","last_active":"2023-07-14T11:53:21.568105+0000","last_peered":"2023-07-14T11:53:21.568105+0000","last_clean":"2023-07-14T11:53:21.568105+0000","last_became_active":"2023-07-14T11:53:19.662484+0000","last_became_peered":"2023-07-14T11:53:19.662484+0000","last_unstale":"2023-07-14T11:53:21.568105+0000","last_undegraded":"2023-07-14T11:53:21.568105+0000","last_fullsized":"2023-07-14T11:53:21.568105+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:43.799463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566672+0000","last_change":"2023-07-14T11:53:14.590222+0000","last_active":"2023-07-14T11:53:21.566672+0000","last_peered":"2023-07-14T11:53:21.566672+0000","last_clean":"2023-07-14T11:53:21.566672+0000","last_became_active":"2023-07-14T11:53:14.589684+0000","last_became_peered":"2023-07-14T11:53:14.589684+0000","last_unstale":"2023-07-14T11:53:21.566672+0000","last_undegraded":"2023-07-14T11:53:21.566672+0000","last_fullsized":"2023-07-14T11:53:21.566672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:43.338727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663068+0000","last_change":"2023-07-14T11:53:15.592933+0000","last_active":"2023-07-14T11:53:20.663068+0000","last_peered":"2023-07-14T11:53:20.663068+0000","last_clean":"2023-07-14T11:53:20.663068+0000","last_became_active":"2023-07-14T11:53:15.592619+0000","last_became_peered":"2023-07-14T11:53:15.592619+0000","last_unstale":"2023-07-14T11:53:20.663068+0000","last_undegraded":"2023-07-14T11:53:20.663068+0000","last_fullsized":"2023-07-14T11:53:20.663068+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:41:38.469614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878123+0000","last_change":"2023-07-14T11:53:19.683165+0000","last_active":"2023-07-14T11:53:20.878123+0000","last_peered":"2023-07-14T11:53:20.878123+0000","last_clean":"2023-07-14T11:53:20.878123+0000","last_became_active":"2023-07-14T11:53:19.682676+0000","last_became_peered":"2023-07-14T11:53:19.682676+0000","last_unstale":"2023-07-14T11:53:20.878123+0000","last_undegraded":"2023-07-14T11:53:20.878123+0000","last_fullsized":"2023-07-14T11:53:20.878123+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:36:19.425078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569431+0000","last_change":"2023-07-14T11:53:14.592834+0000","last_active":"2023-07-14T11:53:21.569431+0000","last_peered":"2023-07-14T11:53:21.569431+0000","last_clean":"2023-07-14T11:53:21.569431+0000","last_became_active":"2023-07-14T11:53:14.592535+0000","last_became_peered":"2023-07-14T11:53:14.592535+0000","last_unstale":"2023-07-14T11:53:21.569431+0000","last_undegraded":"2023-07-14T11:53:21.569431+0000","last_fullsized":"2023-07-14T11:53:21.569431+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:39.643364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661401+0000","last_change":"2023-07-14T11:53:15.593173+0000","last_active":"2023-07-14T11:53:20.661401+0000","last_peered":"2023-07-14T11:53:20.661401+0000","last_clean":"2023-07-14T11:53:20.661401+0000","last_became_active":"2023-07-14T11:53:15.592873+0000","last_became_peered":"2023-07-14T11:53:15.592873+0000","last_unstale":"2023-07-14T11:53:20.661401+0000","last_undegraded":"2023-07-14T11:53:20.661401+0000","last_fullsized":"2023-07-14T11:53:20.661401+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:59:03.301440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663003+0000","last_change":"2023-07-14T11:53:19.672048+0000","last_active":"2023-07-14T11:53:20.663003+0000","last_peered":"2023-07-14T11:53:20.663003+0000","last_clean":"2023-07-14T11:53:20.663003+0000","last_became_active":"2023-07-14T11:53:19.671493+0000","last_became_peered":"2023-07-14T11:53:19.671493+0000","last_unstale":"2023-07-14T11:53:20.663003+0000","last_undegraded":"2023-07-14T11:53:20.663003+0000","last_fullsized":"2023-07-14T11:53:20.663003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:34:10.732809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664092+0000","last_change":"2023-07-14T11:53:14.591651+0000","last_active":"2023-07-14T11:53:20.664092+0000","last_peered":"2023-07-14T11:53:20.664092+0000","last_clean":"2023-07-14T11:53:20.664092+0000","last_became_active":"2023-07-14T11:53:14.591356+0000","last_became_peered":"2023-07-14T11:53:14.591356+0000","last_unstale":"2023-07-14T11:53:20.664092+0000","last_undegraded":"2023-07-14T11:53:20.664092+0000","last_fullsized":"2023-07-14T11:53:20.664092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:38:54.408913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568174+0000","last_change":"2023-07-14T11:53:15.596998+0000","last_active":"2023-07-14T11:53:21.568174+0000","last_peered":"2023-07-14T11:53:21.568174+0000","last_clean":"2023-07-14T11:53:21.568174+0000","last_became_active":"2023-07-14T11:53:15.596278+0000","last_became_peered":"2023-07-14T11:53:15.596278+0000","last_unstale":"2023-07-14T11:53:21.568174+0000","last_undegraded":"2023-07-14T11:53:21.568174+0000","last_fullsized":"2023-07-14T11:53:21.568174+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:39.727537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665493+0000","last_change":"2023-07-14T11:53:19.664365+0000","last_active":"2023-07-14T11:53:20.665493+0000","last_peered":"2023-07-14T11:53:20.665493+0000","last_clean":"2023-07-14T11:53:20.665493+0000","last_became_active":"2023-07-14T11:53:19.663967+0000","last_became_peered":"2023-07-14T11:53:19.663967+0000","last_unstale":"2023-07-14T11:53:20.665493+0000","last_undegraded":"2023-07-14T11:53:20.665493+0000","last_fullsized":"2023-07-14T11:53:20.665493+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:37:34.816983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877811+0000","last_change":"2023-07-14T11:53:14.590756+0000","last_active":"2023-07-14T11:53:20.877811+0000","last_peered":"2023-07-14T11:53:20.877811+0000","last_clean":"2023-07-14T11:53:20.877811+0000","last_became_active":"2023-07-14T11:53:14.590511+0000","last_became_peered":"2023-07-14T11:53:14.590511+0000","last_unstale":"2023-07-14T11:53:20.877811+0000","last_undegraded":"2023-07-14T11:53:20.877811+0000","last_fullsized":"2023-07-14T11:53:20.877811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:18:16.526406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568118+0000","last_change":"2023-07-14T11:53:15.612239+0000","last_active":"2023-07-14T11:53:21.568118+0000","last_peered":"2023-07-14T11:53:21.568118+0000","last_clean":"2023-07-14T11:53:21.568118+0000","last_became_active":"2023-07-14T11:53:15.611948+0000","last_became_peered":"2023-07-14T11:53:15.611948+0000","last_unstale":"2023-07-14T11:53:21.568118+0000","last_undegraded":"2023-07-14T11:53:21.568118+0000","last_fullsized":"2023-07-14T11:53:21.568118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:44.345584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567809+0000","last_change":"2023-07-14T11:53:19.680386+0000","last_active":"2023-07-14T11:53:21.567809+0000","last_peered":"2023-07-14T11:53:21.567809+0000","last_clean":"2023-07-14T11:53:21.567809+0000","last_became_active":"2023-07-14T11:53:19.674870+0000","last_became_peered":"2023-07-14T11:53:19.674870+0000","last_unstale":"2023-07-14T11:53:21.567809+0000","last_undegraded":"2023-07-14T11:53:21.567809+0000","last_fullsized":"2023-07-14T11:53:21.567809+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:00.015845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568390+0000","last_change":"2023-07-14T11:53:14.592538+0000","last_active":"2023-07-14T11:53:21.568390+0000","last_peered":"2023-07-14T11:53:21.568390+0000","last_clean":"2023-07-14T11:53:21.568390+0000","last_became_active":"2023-07-14T11:53:14.592269+0000","last_became_peered":"2023-07-14T11:53:14.592269+0000","last_unstale":"2023-07-14T11:53:21.568390+0000","last_undegraded":"2023-07-14T11:53:21.568390+0000","last_fullsized":"2023-07-14T11:53:21.568390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:17.757069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665555+0000","last_change":"2023-07-14T11:53:15.587863+0000","last_active":"2023-07-14T11:53:20.665555+0000","last_peered":"2023-07-14T11:53:20.665555+0000","last_clean":"2023-07-14T11:53:20.665555+0000","last_became_active":"2023-07-14T11:53:15.587560+0000","last_became_peered":"2023-07-14T11:53:15.587560+0000","last_unstale":"2023-07-14T11:53:20.665555+0000","last_undegraded":"2023-07-14T11:53:20.665555+0000","last_fullsized":"2023-07-14T11:53:20.665555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:01:34.117398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568069+0000","last_change":"2023-07-14T11:53:15.596816+0000","last_active":"2023-07-14T11:53:21.568069+0000","last_peered":"2023-07-14T11:53:21.568069+0000","last_clean":"2023-07-14T11:53:21.568069+0000","last_became_active":"2023-07-14T11:53:15.596544+0000","last_became_peered":"2023-07-14T11:53:15.596544+0000","last_unstale":"2023-07-14T11:53:21.568069+0000","last_undegraded":"2023-07-14T11:53:21.568069+0000","last_fullsized":"2023-07-14T11:53:21.568069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:13.267093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568721+0000","last_change":"2023-07-14T11:53:14.600709+0000","last_active":"2023-07-14T11:53:21.568721+0000","last_peered":"2023-07-14T11:53:21.568721+0000","last_clean":"2023-07-14T11:53:21.568721+0000","last_became_active":"2023-07-14T11:53:14.600430+0000","last_became_peered":"2023-07-14T11:53:14.600430+0000","last_unstale":"2023-07-14T11:53:21.568721+0000","last_undegraded":"2023-07-14T11:53:21.568721+0000","last_fullsized":"2023-07-14T11:53:21.568721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:30.529533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664703+0000","last_change":"2023-07-14T11:53:19.664675+0000","last_active":"2023-07-14T11:53:20.664703+0000","last_peered":"2023-07-14T11:53:20.664703+0000","last_clean":"2023-07-14T11:53:20.664703+0000","last_became_active":"2023-07-14T11:53:19.663889+0000","last_became_peered":"2023-07-14T11:53:19.663889+0000","last_unstale":"2023-07-14T11:53:20.664703+0000","last_undegraded":"2023-07-14T11:53:20.664703+0000","last_fullsized":"2023-07-14T11:53:20.664703+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:46.192512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568024+0000","last_change":"2023-07-14T11:53:15.606972+0000","last_active":"2023-07-14T11:53:21.568024+0000","last_peered":"2023-07-14T11:53:21.568024+0000","last_clean":"2023-07-14T11:53:21.568024+0000","last_became_active":"2023-07-14T11:53:15.606615+0000","last_became_peered":"2023-07-14T11:53:15.606615+0000","last_unstale":"2023-07-14T11:53:21.568024+0000","last_undegraded":"2023-07-14T11:53:21.568024+0000","last_fullsized":"2023-07-14T11:53:21.568024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:29.641887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569420+0000","last_change":"2023-07-14T11:53:14.595852+0000","last_active":"2023-07-14T11:53:21.569420+0000","last_peered":"2023-07-14T11:53:21.569420+0000","last_clean":"2023-07-14T11:53:21.569420+0000","last_became_active":"2023-07-14T11:53:14.589714+0000","last_became_peered":"2023-07-14T11:53:14.589714+0000","last_unstale":"2023-07-14T11:53:21.569420+0000","last_undegraded":"2023-07-14T11:53:21.569420+0000","last_fullsized":"2023-07-14T11:53:21.569420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:30:20.713856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567265+0000","last_change":"2023-07-14T11:53:19.675250+0000","last_active":"2023-07-14T11:53:21.567265+0000","last_peered":"2023-07-14T11:53:21.567265+0000","last_clean":"2023-07-14T11:53:21.567265+0000","last_became_active":"2023-07-14T11:53:19.674805+0000","last_became_peered":"2023-07-14T11:53:19.674805+0000","last_unstale":"2023-07-14T11:53:21.567265+0000","last_undegraded":"2023-07-14T11:53:21.567265+0000","last_fullsized":"2023-07-14T11:53:21.567265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:44:55.840426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568563+0000","last_change":"2023-07-14T11:53:14.587048+0000","last_active":"2023-07-14T11:53:21.568563+0000","last_peered":"2023-07-14T11:53:21.568563+0000","last_clean":"2023-07-14T11:53:21.568563+0000","last_became_active":"2023-07-14T11:53:14.586722+0000","last_became_peered":"2023-07-14T11:53:14.586722+0000","last_unstale":"2023-07-14T11:53:21.568563+0000","last_undegraded":"2023-07-14T11:53:21.568563+0000","last_fullsized":"2023-07-14T11:53:21.568563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:17.757069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878086+0000","last_change":"2023-07-14T11:53:15.598564+0000","last_active":"2023-07-14T11:53:20.878086+0000","last_peered":"2023-07-14T11:53:20.878086+0000","last_clean":"2023-07-14T11:53:20.878086+0000","last_became_active":"2023-07-14T11:53:15.596542+0000","last_became_peered":"2023-07-14T11:53:15.596542+0000","last_unstale":"2023-07-14T11:53:20.878086+0000","last_undegraded":"2023-07-14T11:53:20.878086+0000","last_fullsized":"2023-07-14T11:53:20.878086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:44:51.768501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568782+0000","last_change":"2023-07-14T11:53:19.670353+0000","last_active":"2023-07-14T11:53:21.568782+0000","last_peered":"2023-07-14T11:53:21.568782+0000","last_clean":"2023-07-14T11:53:21.568782+0000","last_became_active":"2023-07-14T11:53:19.669974+0000","last_became_peered":"2023-07-14T11:53:19.669974+0000","last_unstale":"2023-07-14T11:53:21.568782+0000","last_undegraded":"2023-07-14T11:53:21.568782+0000","last_fullsized":"2023-07-14T11:53:21.568782+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:43.632890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:33.130929+0000","last_change":"2023-07-14T11:53:14.586133+0000","last_active":"2023-07-14T11:53:33.130929+0000","last_peered":"2023-07-14T11:53:33.130929+0000","last_clean":"2023-07-14T11:53:33.130929+0000","last_became_active":"2023-07-14T11:53:14.585705+0000","last_became_peered":"2023-07-14T11:53:14.585705+0000","last_unstale":"2023-07-14T11:53:33.130929+0000","last_undegraded":"2023-07-14T11:53:33.130929+0000","last_fullsized":"2023-07-14T11:53:33.130929+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:48:36.912719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3793,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663286+0000","last_change":"2023-07-14T11:53:15.581733+0000","last_active":"2023-07-14T11:53:20.663286+0000","last_peered":"2023-07-14T11:53:20.663286+0000","last_clean":"2023-07-14T11:53:20.663286+0000","last_became_active":"2023-07-14T11:53:15.581354+0000","last_became_peered":"2023-07-14T11:53:15.581354+0000","last_unstale":"2023-07-14T11:53:20.663286+0000","last_undegraded":"2023-07-14T11:53:20.663286+0000","last_fullsized":"2023-07-14T11:53:20.663286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:24:39.278456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877743+0000","last_change":"2023-07-14T11:53:19.678063+0000","last_active":"2023-07-14T11:53:20.877743+0000","last_peered":"2023-07-14T11:53:20.877743+0000","last_clean":"2023-07-14T11:53:20.877743+0000","last_became_active":"2023-07-14T11:53:19.677638+0000","last_became_peered":"2023-07-14T11:53:19.677638+0000","last_unstale":"2023-07-14T11:53:20.877743+0000","last_undegraded":"2023-07-14T11:53:20.877743+0000","last_fullsized":"2023-07-14T11:53:20.877743+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:32:43.273428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663397+0000","last_change":"2023-07-14T11:53:15.590752+0000","last_active":"2023-07-14T11:53:20.663397+0000","last_peered":"2023-07-14T11:53:20.663397+0000","last_clean":"2023-07-14T11:53:20.663397+0000","last_became_active":"2023-07-14T11:53:15.589947+0000","last_became_peered":"2023-07-14T11:53:15.589947+0000","last_unstale":"2023-07-14T11:53:20.663397+0000","last_undegraded":"2023-07-14T11:53:20.663397+0000","last_fullsized":"2023-07-14T11:53:20.663397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:13:58.842089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665649+0000","last_change":"2023-07-14T11:53:09.145201+0000","last_active":"2023-07-14T11:53:20.665649+0000","last_peered":"2023-07-14T11:53:20.665649+0000","last_clean":"2023-07-14T11:53:20.665649+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:53:20.665649+0000","last_undegraded":"2023-07-14T11:53:20.665649+0000","last_fullsized":"2023-07-14T11:53:20.665649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:54.319358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568612+0000","last_change":"2023-07-14T11:53:14.592544+0000","last_active":"2023-07-14T11:53:21.568612+0000","last_peered":"2023-07-14T11:53:21.568612+0000","last_clean":"2023-07-14T11:53:21.568612+0000","last_became_active":"2023-07-14T11:53:14.592204+0000","last_became_peered":"2023-07-14T11:53:14.592204+0000","last_unstale":"2023-07-14T11:53:21.568612+0000","last_undegraded":"2023-07-14T11:53:21.568612+0000","last_fullsized":"2023-07-14T11:53:21.568612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:12.881399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567791+0000","last_change":"2023-07-14T11:53:19.676141+0000","last_active":"2023-07-14T11:53:21.567791+0000","last_peered":"2023-07-14T11:53:21.567791+0000","last_clean":"2023-07-14T11:53:21.567791+0000","last_became_active":"2023-07-14T11:53:19.675720+0000","last_became_peered":"2023-07-14T11:53:19.675720+0000","last_unstale":"2023-07-14T11:53:21.567791+0000","last_undegraded":"2023-07-14T11:53:21.567791+0000","last_fullsized":"2023-07-14T11:53:21.567791+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:48.417509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568246+0000","last_change":"2023-07-14T11:53:15.607026+0000","last_active":"2023-07-14T11:53:21.568246+0000","last_peered":"2023-07-14T11:53:21.568246+0000","last_clean":"2023-07-14T11:53:21.568246+0000","last_became_active":"2023-07-14T11:53:15.606605+0000","last_became_peered":"2023-07-14T11:53:15.606605+0000","last_unstale":"2023-07-14T11:53:21.568246+0000","last_undegraded":"2023-07-14T11:53:21.568246+0000","last_fullsized":"2023-07-14T11:53:21.568246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:13.888256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878901+0000","last_change":"2023-07-14T11:53:09.146150+0000","last_active":"2023-07-14T11:53:20.878901+0000","last_peered":"2023-07-14T11:53:20.878901+0000","last_clean":"2023-07-14T11:53:20.878901+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:53:20.878901+0000","last_undegraded":"2023-07-14T11:53:20.878901+0000","last_fullsized":"2023-07-14T11:53:20.878901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:07.460041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569870+0000","last_change":"2023-07-14T11:53:14.586533+0000","last_active":"2023-07-14T11:53:21.569870+0000","last_peered":"2023-07-14T11:53:21.569870+0000","last_clean":"2023-07-14T11:53:21.569870+0000","last_became_active":"2023-07-14T11:53:14.586222+0000","last_became_peered":"2023-07-14T11:53:14.586222+0000","last_unstale":"2023-07-14T11:53:21.569870+0000","last_undegraded":"2023-07-14T11:53:21.569870+0000","last_fullsized":"2023-07-14T11:53:21.569870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:38:03.789491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662421+0000","last_change":"2023-07-14T11:53:19.675677+0000","last_active":"2023-07-14T11:53:20.662421+0000","last_peered":"2023-07-14T11:53:20.662421+0000","last_clean":"2023-07-14T11:53:20.662421+0000","last_became_active":"2023-07-14T11:53:19.675259+0000","last_became_peered":"2023-07-14T11:53:19.675259+0000","last_unstale":"2023-07-14T11:53:20.662421+0000","last_undegraded":"2023-07-14T11:53:20.662421+0000","last_fullsized":"2023-07-14T11:53:20.662421+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:05.121760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567406+0000","last_change":"2023-07-14T11:53:15.600612+0000","last_active":"2023-07-14T11:53:21.567406+0000","last_peered":"2023-07-14T11:53:21.567406+0000","last_clean":"2023-07-14T11:53:21.567406+0000","last_became_active":"2023-07-14T11:53:15.600320+0000","last_became_peered":"2023-07-14T11:53:15.600320+0000","last_unstale":"2023-07-14T11:53:21.567406+0000","last_undegraded":"2023-07-14T11:53:21.567406+0000","last_fullsized":"2023-07-14T11:53:21.567406+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:55:00.291201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568702+0000","last_change":"2023-07-14T11:53:09.435876+0000","last_active":"2023-07-14T11:53:21.568702+0000","last_peered":"2023-07-14T11:53:21.568702+0000","last_clean":"2023-07-14T11:53:21.568702+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:53:21.568702+0000","last_undegraded":"2023-07-14T11:53:21.568702+0000","last_fullsized":"2023-07-14T11:53:21.568702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:34.222006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878739+0000","last_change":"2023-07-14T11:53:14.594216+0000","last_active":"2023-07-14T11:53:20.878739+0000","last_peered":"2023-07-14T11:53:20.878739+0000","last_clean":"2023-07-14T11:53:20.878739+0000","last_became_active":"2023-07-14T11:53:14.593954+0000","last_became_peered":"2023-07-14T11:53:14.593954+0000","last_unstale":"2023-07-14T11:53:20.878739+0000","last_undegraded":"2023-07-14T11:53:20.878739+0000","last_fullsized":"2023-07-14T11:53:20.878739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:12.260236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665060+0000","last_change":"2023-07-14T11:53:19.667039+0000","last_active":"2023-07-14T11:53:20.665060+0000","last_peered":"2023-07-14T11:53:20.665060+0000","last_clean":"2023-07-14T11:53:20.665060+0000","last_became_active":"2023-07-14T11:53:19.666360+0000","last_became_peered":"2023-07-14T11:53:19.666360+0000","last_unstale":"2023-07-14T11:53:20.665060+0000","last_undegraded":"2023-07-14T11:53:20.665060+0000","last_fullsized":"2023-07-14T11:53:20.665060+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:37:11.705483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878456+0000","last_change":"2023-07-14T11:53:15.598379+0000","last_active":"2023-07-14T11:53:20.878456+0000","last_peered":"2023-07-14T11:53:20.878456+0000","last_clean":"2023-07-14T11:53:20.878456+0000","last_became_active":"2023-07-14T11:53:15.596661+0000","last_became_peered":"2023-07-14T11:53:15.596661+0000","last_unstale":"2023-07-14T11:53:20.878456+0000","last_undegraded":"2023-07-14T11:53:20.878456+0000","last_fullsized":"2023-07-14T11:53:20.878456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:44.345584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878968+0000","last_change":"2023-07-14T11:53:09.434255+0000","last_active":"2023-07-14T11:53:20.878968+0000","last_peered":"2023-07-14T11:53:20.878968+0000","last_clean":"2023-07-14T11:53:20.878968+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:53:20.878968+0000","last_undegraded":"2023-07-14T11:53:20.878968+0000","last_fullsized":"2023-07-14T11:53:20.878968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:12.335710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567656+0000","last_change":"2023-07-14T11:53:14.589144+0000","last_active":"2023-07-14T11:53:21.567656+0000","last_peered":"2023-07-14T11:53:21.567656+0000","last_clean":"2023-07-14T11:53:21.567656+0000","last_became_active":"2023-07-14T11:53:14.588849+0000","last_became_peered":"2023-07-14T11:53:14.588849+0000","last_unstale":"2023-07-14T11:53:21.567656+0000","last_undegraded":"2023-07-14T11:53:21.567656+0000","last_fullsized":"2023-07-14T11:53:21.567656+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:38.554107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567120+0000","last_change":"2023-07-14T11:53:19.672406+0000","last_active":"2023-07-14T11:53:21.567120+0000","last_peered":"2023-07-14T11:53:21.567120+0000","last_clean":"2023-07-14T11:53:21.567120+0000","last_became_active":"2023-07-14T11:53:19.672009+0000","last_became_peered":"2023-07-14T11:53:19.672009+0000","last_unstale":"2023-07-14T11:53:21.567120+0000","last_undegraded":"2023-07-14T11:53:21.567120+0000","last_fullsized":"2023-07-14T11:53:21.567120+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:36:19.425078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878726+0000","last_change":"2023-07-14T11:53:15.604123+0000","last_active":"2023-07-14T11:53:20.878726+0000","last_peered":"2023-07-14T11:53:20.878726+0000","last_clean":"2023-07-14T11:53:20.878726+0000","last_became_active":"2023-07-14T11:53:15.603815+0000","last_became_peered":"2023-07-14T11:53:15.603815+0000","last_unstale":"2023-07-14T11:53:20.878726+0000","last_undegraded":"2023-07-14T11:53:20.878726+0000","last_fullsized":"2023-07-14T11:53:20.878726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:29.641887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663321+0000","last_change":"2023-07-14T11:53:14.591218+0000","last_active":"2023-07-14T11:53:20.663321+0000","last_peered":"2023-07-14T11:53:20.663321+0000","last_clean":"2023-07-14T11:53:20.663321+0000","last_became_active":"2023-07-14T11:53:14.589122+0000","last_became_peered":"2023-07-14T11:53:14.589122+0000","last_unstale":"2023-07-14T11:53:20.663321+0000","last_undegraded":"2023-07-14T11:53:20.663321+0000","last_fullsized":"2023-07-14T11:53:20.663321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:51:10.380803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566815+0000","last_change":"2023-07-14T11:53:09.147040+0000","last_active":"2023-07-14T11:53:21.566815+0000","last_peered":"2023-07-14T11:53:21.566815+0000","last_clean":"2023-07-14T11:53:21.566815+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:53:21.566815+0000","last_undegraded":"2023-07-14T11:53:21.566815+0000","last_fullsized":"2023-07-14T11:53:21.566815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:23.213672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567493+0000","last_change":"2023-07-14T11:53:19.680353+0000","last_active":"2023-07-14T11:53:21.567493+0000","last_peered":"2023-07-14T11:53:21.567493+0000","last_clean":"2023-07-14T11:53:21.567493+0000","last_became_active":"2023-07-14T11:53:19.674999+0000","last_became_peered":"2023-07-14T11:53:19.674999+0000","last_unstale":"2023-07-14T11:53:21.567493+0000","last_undegraded":"2023-07-14T11:53:21.567493+0000","last_fullsized":"2023-07-14T11:53:21.567493+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:08:07.808470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878795+0000","last_change":"2023-07-14T11:53:15.597158+0000","last_active":"2023-07-14T11:53:20.878795+0000","last_peered":"2023-07-14T11:53:20.878795+0000","last_clean":"2023-07-14T11:53:20.878795+0000","last_became_active":"2023-07-14T11:53:15.595897+0000","last_became_peered":"2023-07-14T11:53:15.595897+0000","last_unstale":"2023-07-14T11:53:20.878795+0000","last_undegraded":"2023-07-14T11:53:20.878795+0000","last_fullsized":"2023-07-14T11:53:20.878795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:39.727537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.877308+0000","last_change":"2023-07-14T11:53:14.589940+0000","last_active":"2023-07-14T11:53:20.877308+0000","last_peered":"2023-07-14T11:53:20.877308+0000","last_clean":"2023-07-14T11:53:20.877308+0000","last_became_active":"2023-07-14T11:53:14.589607+0000","last_became_peered":"2023-07-14T11:53:14.589607+0000","last_unstale":"2023-07-14T11:53:20.877308+0000","last_undegraded":"2023-07-14T11:53:20.877308+0000","last_fullsized":"2023-07-14T11:53:20.877308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:28:09.703297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660746+0000","last_change":"2023-07-14T11:53:09.619294+0000","last_active":"2023-07-14T11:53:20.660746+0000","last_peered":"2023-07-14T11:53:20.660746+0000","last_clean":"2023-07-14T11:53:20.660746+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:53:20.660746+0000","last_undegraded":"2023-07-14T11:53:20.660746+0000","last_fullsized":"2023-07-14T11:53:20.660746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:19:36.961857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661632+0000","last_change":"2023-07-14T11:53:19.666668+0000","last_active":"2023-07-14T11:53:20.661632+0000","last_peered":"2023-07-14T11:53:20.661632+0000","last_clean":"2023-07-14T11:53:20.661632+0000","last_became_active":"2023-07-14T11:53:19.666170+0000","last_became_peered":"2023-07-14T11:53:19.666170+0000","last_unstale":"2023-07-14T11:53:20.661632+0000","last_undegraded":"2023-07-14T11:53:20.661632+0000","last_fullsized":"2023-07-14T11:53:20.661632+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:11:15.273201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568313+0000","last_change":"2023-07-14T11:53:15.611544+0000","last_active":"2023-07-14T11:53:21.568313+0000","last_peered":"2023-07-14T11:53:21.568313+0000","last_clean":"2023-07-14T11:53:21.568313+0000","last_became_active":"2023-07-14T11:53:15.606262+0000","last_became_peered":"2023-07-14T11:53:15.606262+0000","last_unstale":"2023-07-14T11:53:21.568313+0000","last_undegraded":"2023-07-14T11:53:21.568313+0000","last_fullsized":"2023-07-14T11:53:21.568313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566653+0000","last_change":"2023-07-14T11:53:14.588920+0000","last_active":"2023-07-14T11:53:21.566653+0000","last_peered":"2023-07-14T11:53:21.566653+0000","last_clean":"2023-07-14T11:53:21.566653+0000","last_became_active":"2023-07-14T11:53:14.588605+0000","last_became_peered":"2023-07-14T11:53:14.588605+0000","last_unstale":"2023-07-14T11:53:21.566653+0000","last_undegraded":"2023-07-14T11:53:21.566653+0000","last_fullsized":"2023-07-14T11:53:21.566653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:54.773189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567666+0000","last_change":"2023-07-14T11:53:09.146156+0000","last_active":"2023-07-14T11:53:21.567666+0000","last_peered":"2023-07-14T11:53:21.567666+0000","last_clean":"2023-07-14T11:53:21.567666+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:53:21.567666+0000","last_undegraded":"2023-07-14T11:53:21.567666+0000","last_fullsized":"2023-07-14T11:53:21.567666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:37.917368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569099+0000","last_change":"2023-07-14T11:53:19.680478+0000","last_active":"2023-07-14T11:53:21.569099+0000","last_peered":"2023-07-14T11:53:21.569099+0000","last_clean":"2023-07-14T11:53:21.569099+0000","last_became_active":"2023-07-14T11:53:19.680082+0000","last_became_peered":"2023-07-14T11:53:19.680082+0000","last_unstale":"2023-07-14T11:53:21.569099+0000","last_undegraded":"2023-07-14T11:53:21.569099+0000","last_fullsized":"2023-07-14T11:53:21.569099+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:42:53.267154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665451+0000","last_change":"2023-07-14T11:53:15.587902+0000","last_active":"2023-07-14T11:53:20.665451+0000","last_peered":"2023-07-14T11:53:20.665451+0000","last_clean":"2023-07-14T11:53:20.665451+0000","last_became_active":"2023-07-14T11:53:15.587578+0000","last_became_peered":"2023-07-14T11:53:15.587578+0000","last_unstale":"2023-07-14T11:53:20.665451+0000","last_undegraded":"2023-07-14T11:53:20.665451+0000","last_fullsized":"2023-07-14T11:53:20.665451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:43:21.367420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567625+0000","last_change":"2023-07-14T11:53:14.600123+0000","last_active":"2023-07-14T11:53:21.567625+0000","last_peered":"2023-07-14T11:53:21.567625+0000","last_clean":"2023-07-14T11:53:21.567625+0000","last_became_active":"2023-07-14T11:53:14.599830+0000","last_became_peered":"2023-07-14T11:53:14.599830+0000","last_unstale":"2023-07-14T11:53:21.567625+0000","last_undegraded":"2023-07-14T11:53:21.567625+0000","last_fullsized":"2023-07-14T11:53:21.567625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:54.937063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567091+0000","last_change":"2023-07-14T11:53:09.146556+0000","last_active":"2023-07-14T11:53:21.567091+0000","last_peered":"2023-07-14T11:53:21.567091+0000","last_clean":"2023-07-14T11:53:21.567091+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:53:21.567091+0000","last_undegraded":"2023-07-14T11:53:21.567091+0000","last_fullsized":"2023-07-14T11:53:21.567091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:07.460041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567928+0000","last_change":"2023-07-14T11:53:03.159818+0000","last_active":"2023-07-14T11:53:21.567928+0000","last_peered":"2023-07-14T11:53:21.567928+0000","last_clean":"2023-07-14T11:53:21.567928+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:53:21.567928+0000","last_undegraded":"2023-07-14T11:53:21.567928+0000","last_fullsized":"2023-07-14T11:53:21.567928+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_clean_scrub_stamp":"2023-07-14T11:53:02.054348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:17.151944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567578+0000","last_change":"2023-07-14T11:53:19.658486+0000","last_active":"2023-07-14T11:53:21.567578+0000","last_peered":"2023-07-14T11:53:21.567578+0000","last_clean":"2023-07-14T11:53:21.567578+0000","last_became_active":"2023-07-14T11:53:19.658083+0000","last_became_peered":"2023-07-14T11:53:19.658083+0000","last_unstale":"2023-07-14T11:53:21.567578+0000","last_undegraded":"2023-07-14T11:53:21.567578+0000","last_fullsized":"2023-07-14T11:53:21.567578+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:08:07.808470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878421+0000","last_change":"2023-07-14T11:53:15.596089+0000","last_active":"2023-07-14T11:53:20.878421+0000","last_peered":"2023-07-14T11:53:20.878421+0000","last_clean":"2023-07-14T11:53:20.878421+0000","last_became_active":"2023-07-14T11:53:15.594769+0000","last_became_peered":"2023-07-14T11:53:15.594769+0000","last_unstale":"2023-07-14T11:53:20.878421+0000","last_undegraded":"2023-07-14T11:53:20.878421+0000","last_fullsized":"2023-07-14T11:53:20.878421+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:55.943920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568475+0000","last_change":"2023-07-14T11:53:14.591999+0000","last_active":"2023-07-14T11:53:21.568475+0000","last_peered":"2023-07-14T11:53:21.568475+0000","last_clean":"2023-07-14T11:53:21.568475+0000","last_became_active":"2023-07-14T11:53:14.591754+0000","last_became_peered":"2023-07-14T11:53:14.591754+0000","last_unstale":"2023-07-14T11:53:21.568475+0000","last_undegraded":"2023-07-14T11:53:21.568475+0000","last_fullsized":"2023-07-14T11:53:21.568475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:39.643364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664358+0000","last_change":"2023-07-14T11:53:19.674798+0000","last_active":"2023-07-14T11:53:20.664358+0000","last_peered":"2023-07-14T11:53:20.664358+0000","last_clean":"2023-07-14T11:53:20.664358+0000","last_became_active":"2023-07-14T11:53:19.674362+0000","last_became_peered":"2023-07-14T11:53:19.674362+0000","last_unstale":"2023-07-14T11:53:20.664358+0000","last_undegraded":"2023-07-14T11:53:20.664358+0000","last_fullsized":"2023-07-14T11:53:20.664358+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:52:07.441376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664411+0000","last_change":"2023-07-14T11:53:15.582299+0000","last_active":"2023-07-14T11:53:20.664411+0000","last_peered":"2023-07-14T11:53:20.664411+0000","last_clean":"2023-07-14T11:53:20.664411+0000","last_became_active":"2023-07-14T11:53:15.582045+0000","last_became_peered":"2023-07-14T11:53:15.582045+0000","last_unstale":"2023-07-14T11:53:20.664411+0000","last_undegraded":"2023-07-14T11:53:20.664411+0000","last_fullsized":"2023-07-14T11:53:20.664411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:14:24.652057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663574+0000","last_change":"2023-07-14T11:53:14.574223+0000","last_active":"2023-07-14T11:53:20.663574+0000","last_peered":"2023-07-14T11:53:20.663574+0000","last_clean":"2023-07-14T11:53:20.663574+0000","last_became_active":"2023-07-14T11:53:14.573601+0000","last_became_peered":"2023-07-14T11:53:14.573601+0000","last_unstale":"2023-07-14T11:53:20.663574+0000","last_undegraded":"2023-07-14T11:53:20.663574+0000","last_fullsized":"2023-07-14T11:53:20.663574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:44:27.636525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568891+0000","last_change":"2023-07-14T11:53:19.671897+0000","last_active":"2023-07-14T11:53:21.568891+0000","last_peered":"2023-07-14T11:53:21.568891+0000","last_clean":"2023-07-14T11:53:21.568891+0000","last_became_active":"2023-07-14T11:53:19.671406+0000","last_became_peered":"2023-07-14T11:53:19.671406+0000","last_unstale":"2023-07-14T11:53:21.568891+0000","last_undegraded":"2023-07-14T11:53:21.568891+0000","last_fullsized":"2023-07-14T11:53:21.568891+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:37:01.963700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664358+0000","last_change":"2023-07-14T11:53:15.583755+0000","last_active":"2023-07-14T11:53:20.664358+0000","last_peered":"2023-07-14T11:53:20.664358+0000","last_clean":"2023-07-14T11:53:20.664358+0000","last_became_active":"2023-07-14T11:53:15.583491+0000","last_became_peered":"2023-07-14T11:53:15.583491+0000","last_unstale":"2023-07-14T11:53:20.664358+0000","last_undegraded":"2023-07-14T11:53:20.664358+0000","last_fullsized":"2023-07-14T11:53:20.664358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:30.105172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665591+0000","last_change":"2023-07-14T11:53:14.594378+0000","last_active":"2023-07-14T11:53:20.665591+0000","last_peered":"2023-07-14T11:53:20.665591+0000","last_clean":"2023-07-14T11:53:20.665591+0000","last_became_active":"2023-07-14T11:53:14.593863+0000","last_became_peered":"2023-07-14T11:53:14.593863+0000","last_unstale":"2023-07-14T11:53:20.665591+0000","last_undegraded":"2023-07-14T11:53:20.665591+0000","last_fullsized":"2023-07-14T11:53:20.665591+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:05:26.104433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878321+0000","last_change":"2023-07-14T11:53:19.674408+0000","last_active":"2023-07-14T11:53:20.878321+0000","last_peered":"2023-07-14T11:53:20.878321+0000","last_clean":"2023-07-14T11:53:20.878321+0000","last_became_active":"2023-07-14T11:53:19.673467+0000","last_became_peered":"2023-07-14T11:53:19.673467+0000","last_unstale":"2023-07-14T11:53:20.878321+0000","last_undegraded":"2023-07-14T11:53:20.878321+0000","last_fullsized":"2023-07-14T11:53:20.878321+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:08:07.808470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663737+0000","last_change":"2023-07-14T11:53:15.592271+0000","last_active":"2023-07-14T11:53:20.663737+0000","last_peered":"2023-07-14T11:53:20.663737+0000","last_clean":"2023-07-14T11:53:20.663737+0000","last_became_active":"2023-07-14T11:53:15.591906+0000","last_became_peered":"2023-07-14T11:53:15.591906+0000","last_unstale":"2023-07-14T11:53:20.663737+0000","last_undegraded":"2023-07-14T11:53:20.663737+0000","last_fullsized":"2023-07-14T11:53:20.663737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:17:30.232523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569314+0000","last_change":"2023-07-14T11:53:14.595162+0000","last_active":"2023-07-14T11:53:21.569314+0000","last_peered":"2023-07-14T11:53:21.569314+0000","last_clean":"2023-07-14T11:53:21.569314+0000","last_became_active":"2023-07-14T11:53:14.594754+0000","last_became_peered":"2023-07-14T11:53:14.594754+0000","last_unstale":"2023-07-14T11:53:21.569314+0000","last_undegraded":"2023-07-14T11:53:21.569314+0000","last_fullsized":"2023-07-14T11:53:21.569314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:19:28.596554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661153+0000","last_change":"2023-07-14T11:53:19.666034+0000","last_active":"2023-07-14T11:53:20.661153+0000","last_peered":"2023-07-14T11:53:20.661153+0000","last_clean":"2023-07-14T11:53:20.661153+0000","last_became_active":"2023-07-14T11:53:19.665570+0000","last_became_peered":"2023-07-14T11:53:19.665570+0000","last_unstale":"2023-07-14T11:53:20.661153+0000","last_undegraded":"2023-07-14T11:53:20.661153+0000","last_fullsized":"2023-07-14T11:53:20.661153+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:48:08.806526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567270+0000","last_change":"2023-07-14T11:53:15.612021+0000","last_active":"2023-07-14T11:53:21.567270+0000","last_peered":"2023-07-14T11:53:21.567270+0000","last_clean":"2023-07-14T11:53:21.567270+0000","last_became_active":"2023-07-14T11:53:15.611653+0000","last_became_peered":"2023-07-14T11:53:15.611653+0000","last_unstale":"2023-07-14T11:53:21.567270+0000","last_undegraded":"2023-07-14T11:53:21.567270+0000","last_fullsized":"2023-07-14T11:53:21.567270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:44:51.768501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566941+0000","last_change":"2023-07-14T11:53:14.590015+0000","last_active":"2023-07-14T11:53:21.566941+0000","last_peered":"2023-07-14T11:53:21.566941+0000","last_clean":"2023-07-14T11:53:21.566941+0000","last_became_active":"2023-07-14T11:53:14.589288+0000","last_became_peered":"2023-07-14T11:53:14.589288+0000","last_unstale":"2023-07-14T11:53:21.566941+0000","last_undegraded":"2023-07-14T11:53:21.566941+0000","last_fullsized":"2023-07-14T11:53:21.566941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:12.260236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660851+0000","last_change":"2023-07-14T11:53:19.664514+0000","last_active":"2023-07-14T11:53:20.660851+0000","last_peered":"2023-07-14T11:53:20.660851+0000","last_clean":"2023-07-14T11:53:20.660851+0000","last_became_active":"2023-07-14T11:53:19.663999+0000","last_became_peered":"2023-07-14T11:53:19.663999+0000","last_unstale":"2023-07-14T11:53:20.660851+0000","last_undegraded":"2023-07-14T11:53:20.660851+0000","last_fullsized":"2023-07-14T11:53:20.660851+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:23:47.861604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664958+0000","last_change":"2023-07-14T11:53:15.597946+0000","last_active":"2023-07-14T11:53:20.664958+0000","last_peered":"2023-07-14T11:53:20.664958+0000","last_clean":"2023-07-14T11:53:20.664958+0000","last_became_active":"2023-07-14T11:53:15.597589+0000","last_became_peered":"2023-07-14T11:53:15.597589+0000","last_unstale":"2023-07-14T11:53:20.664958+0000","last_undegraded":"2023-07-14T11:53:20.664958+0000","last_fullsized":"2023-07-14T11:53:20.664958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:51.380702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568455+0000","last_change":"2023-07-14T11:53:14.599636+0000","last_active":"2023-07-14T11:53:21.568455+0000","last_peered":"2023-07-14T11:53:21.568455+0000","last_clean":"2023-07-14T11:53:21.568455+0000","last_became_active":"2023-07-14T11:53:14.599183+0000","last_became_peered":"2023-07-14T11:53:14.599183+0000","last_unstale":"2023-07-14T11:53:21.568455+0000","last_undegraded":"2023-07-14T11:53:21.568455+0000","last_fullsized":"2023-07-14T11:53:21.568455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:30.529533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661123+0000","last_change":"2023-07-14T11:53:19.681580+0000","last_active":"2023-07-14T11:53:20.661123+0000","last_peered":"2023-07-14T11:53:20.661123+0000","last_clean":"2023-07-14T11:53:20.661123+0000","last_became_active":"2023-07-14T11:53:19.680891+0000","last_became_peered":"2023-07-14T11:53:19.680891+0000","last_unstale":"2023-07-14T11:53:20.661123+0000","last_undegraded":"2023-07-14T11:53:20.661123+0000","last_fullsized":"2023-07-14T11:53:20.661123+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:20:03.759130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663189+0000","last_change":"2023-07-14T11:53:15.610606+0000","last_active":"2023-07-14T11:53:20.663189+0000","last_peered":"2023-07-14T11:53:20.663189+0000","last_clean":"2023-07-14T11:53:20.663189+0000","last_became_active":"2023-07-14T11:53:15.610180+0000","last_became_peered":"2023-07-14T11:53:15.610180+0000","last_unstale":"2023-07-14T11:53:20.663189+0000","last_undegraded":"2023-07-14T11:53:20.663189+0000","last_fullsized":"2023-07-14T11:53:20.663189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:09.649855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568785+0000","last_change":"2023-07-14T11:53:14.591805+0000","last_active":"2023-07-14T11:53:21.568785+0000","last_peered":"2023-07-14T11:53:21.568785+0000","last_clean":"2023-07-14T11:53:21.568785+0000","last_became_active":"2023-07-14T11:53:14.591439+0000","last_became_peered":"2023-07-14T11:53:14.591439+0000","last_unstale":"2023-07-14T11:53:21.568785+0000","last_undegraded":"2023-07-14T11:53:21.568785+0000","last_fullsized":"2023-07-14T11:53:21.568785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:54:59.284344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660363+0000","last_change":"2023-07-14T11:53:19.675759+0000","last_active":"2023-07-14T11:53:20.660363+0000","last_peered":"2023-07-14T11:53:20.660363+0000","last_clean":"2023-07-14T11:53:20.660363+0000","last_became_active":"2023-07-14T11:53:19.675353+0000","last_became_peered":"2023-07-14T11:53:19.675353+0000","last_unstale":"2023-07-14T11:53:20.660363+0000","last_undegraded":"2023-07-14T11:53:20.660363+0000","last_fullsized":"2023-07-14T11:53:20.660363+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:35:00.376887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566299+0000","last_change":"2023-07-14T11:53:15.603352+0000","last_active":"2023-07-14T11:53:21.566299+0000","last_peered":"2023-07-14T11:53:21.566299+0000","last_clean":"2023-07-14T11:53:21.566299+0000","last_became_active":"2023-07-14T11:53:15.602777+0000","last_became_peered":"2023-07-14T11:53:15.602777+0000","last_unstale":"2023-07-14T11:53:21.566299+0000","last_undegraded":"2023-07-14T11:53:21.566299+0000","last_fullsized":"2023-07-14T11:53:21.566299+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568843+0000","last_change":"2023-07-14T11:53:14.590504+0000","last_active":"2023-07-14T11:53:21.568843+0000","last_peered":"2023-07-14T11:53:21.568843+0000","last_clean":"2023-07-14T11:53:21.568843+0000","last_became_active":"2023-07-14T11:53:14.589844+0000","last_became_peered":"2023-07-14T11:53:14.589844+0000","last_unstale":"2023-07-14T11:53:21.568843+0000","last_undegraded":"2023-07-14T11:53:21.568843+0000","last_fullsized":"2023-07-14T11:53:21.568843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:38.554107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567609+0000","last_change":"2023-07-14T11:53:19.668417+0000","last_active":"2023-07-14T11:53:21.567609+0000","last_peered":"2023-07-14T11:53:21.567609+0000","last_clean":"2023-07-14T11:53:21.567609+0000","last_became_active":"2023-07-14T11:53:19.667924+0000","last_became_peered":"2023-07-14T11:53:19.667924+0000","last_unstale":"2023-07-14T11:53:21.567609+0000","last_undegraded":"2023-07-14T11:53:21.567609+0000","last_fullsized":"2023-07-14T11:53:21.567609+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:00.015845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568675+0000","last_change":"2023-07-14T11:53:19.668219+0000","last_active":"2023-07-14T11:53:21.568675+0000","last_peered":"2023-07-14T11:53:21.568675+0000","last_clean":"2023-07-14T11:53:21.568675+0000","last_became_active":"2023-07-14T11:53:19.667862+0000","last_became_peered":"2023-07-14T11:53:19.667862+0000","last_unstale":"2023-07-14T11:53:21.568675+0000","last_undegraded":"2023-07-14T11:53:21.568675+0000","last_fullsized":"2023-07-14T11:53:21.568675+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:53:26.423183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665093+0000","last_change":"2023-07-14T11:53:14.588139+0000","last_active":"2023-07-14T11:53:20.665093+0000","last_peered":"2023-07-14T11:53:20.665093+0000","last_clean":"2023-07-14T11:53:20.665093+0000","last_became_active":"2023-07-14T11:53:14.587730+0000","last_became_peered":"2023-07-14T11:53:14.587730+0000","last_unstale":"2023-07-14T11:53:20.665093+0000","last_undegraded":"2023-07-14T11:53:20.665093+0000","last_fullsized":"2023-07-14T11:53:20.665093+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:02.335678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569769+0000","last_change":"2023-07-14T11:53:15.606880+0000","last_active":"2023-07-14T11:53:21.569769+0000","last_peered":"2023-07-14T11:53:21.569769+0000","last_clean":"2023-07-14T11:53:21.569769+0000","last_became_active":"2023-07-14T11:53:15.598546+0000","last_became_peered":"2023-07-14T11:53:15.598546+0000","last_unstale":"2023-07-14T11:53:21.569769+0000","last_undegraded":"2023-07-14T11:53:21.569769+0000","last_fullsized":"2023-07-14T11:53:21.569769+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:30:21.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569600+0000","last_change":"2023-07-14T11:53:19.676159+0000","last_active":"2023-07-14T11:53:21.569600+0000","last_peered":"2023-07-14T11:53:21.569600+0000","last_clean":"2023-07-14T11:53:21.569600+0000","last_became_active":"2023-07-14T11:53:19.675729+0000","last_became_peered":"2023-07-14T11:53:19.675729+0000","last_unstale":"2023-07-14T11:53:21.569600+0000","last_undegraded":"2023-07-14T11:53:21.569600+0000","last_fullsized":"2023-07-14T11:53:21.569600+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:49.754582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569051+0000","last_change":"2023-07-14T11:53:14.592455+0000","last_active":"2023-07-14T11:53:21.569051+0000","last_peered":"2023-07-14T11:53:21.569051+0000","last_clean":"2023-07-14T11:53:21.569051+0000","last_became_active":"2023-07-14T11:53:14.592040+0000","last_became_peered":"2023-07-14T11:53:14.592040+0000","last_unstale":"2023-07-14T11:53:21.569051+0000","last_undegraded":"2023-07-14T11:53:21.569051+0000","last_fullsized":"2023-07-14T11:53:21.569051+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:36:56.884917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568516+0000","last_change":"2023-07-14T11:53:15.593198+0000","last_active":"2023-07-14T11:53:21.568516+0000","last_peered":"2023-07-14T11:53:21.568516+0000","last_clean":"2023-07-14T11:53:21.568516+0000","last_became_active":"2023-07-14T11:53:15.592898+0000","last_became_peered":"2023-07-14T11:53:15.592898+0000","last_unstale":"2023-07-14T11:53:21.568516+0000","last_undegraded":"2023-07-14T11:53:21.568516+0000","last_fullsized":"2023-07-14T11:53:21.568516+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:13.267093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566870+0000","last_change":"2023-07-14T11:53:14.588962+0000","last_active":"2023-07-14T11:53:21.566870+0000","last_peered":"2023-07-14T11:53:21.566870+0000","last_clean":"2023-07-14T11:53:21.566870+0000","last_became_active":"2023-07-14T11:53:14.588674+0000","last_became_peered":"2023-07-14T11:53:14.588674+0000","last_unstale":"2023-07-14T11:53:21.566870+0000","last_undegraded":"2023-07-14T11:53:21.566870+0000","last_fullsized":"2023-07-14T11:53:21.566870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:32:38.194645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567389+0000","last_change":"2023-07-14T11:53:15.603164+0000","last_active":"2023-07-14T11:53:21.567389+0000","last_peered":"2023-07-14T11:53:21.567389+0000","last_clean":"2023-07-14T11:53:21.567389+0000","last_became_active":"2023-07-14T11:53:15.602752+0000","last_became_peered":"2023-07-14T11:53:15.602752+0000","last_unstale":"2023-07-14T11:53:21.567389+0000","last_undegraded":"2023-07-14T11:53:21.567389+0000","last_fullsized":"2023-07-14T11:53:21.567389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:40.650221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568701+0000","last_change":"2023-07-14T11:53:19.664284+0000","last_active":"2023-07-14T11:53:21.568701+0000","last_peered":"2023-07-14T11:53:21.568701+0000","last_clean":"2023-07-14T11:53:21.568701+0000","last_became_active":"2023-07-14T11:53:19.663903+0000","last_became_peered":"2023-07-14T11:53:19.663903+0000","last_unstale":"2023-07-14T11:53:21.568701+0000","last_undegraded":"2023-07-14T11:53:21.568701+0000","last_fullsized":"2023-07-14T11:53:21.568701+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:55:04.363127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568411+0000","last_change":"2023-07-14T11:53:14.591199+0000","last_active":"2023-07-14T11:53:21.568411+0000","last_peered":"2023-07-14T11:53:21.568411+0000","last_clean":"2023-07-14T11:53:21.568411+0000","last_became_active":"2023-07-14T11:53:14.590897+0000","last_became_peered":"2023-07-14T11:53:14.590897+0000","last_unstale":"2023-07-14T11:53:21.568411+0000","last_undegraded":"2023-07-14T11:53:21.568411+0000","last_fullsized":"2023-07-14T11:53:21.568411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:13.537434+0000","last_clean_scrub_stamp":"2023-07-14T11:53:13.537434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:53:21.344401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568918+0000","last_change":"2023-07-14T11:53:15.594450+0000","last_active":"2023-07-14T11:53:21.568918+0000","last_peered":"2023-07-14T11:53:21.568918+0000","last_clean":"2023-07-14T11:53:21.568918+0000","last_became_active":"2023-07-14T11:53:15.594170+0000","last_became_peered":"2023-07-14T11:53:15.594170+0000","last_unstale":"2023-07-14T11:53:21.568918+0000","last_undegraded":"2023-07-14T11:53:21.568918+0000","last_fullsized":"2023-07-14T11:53:21.568918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:53:22.351258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567104+0000","last_change":"2023-07-14T11:53:19.682025+0000","last_active":"2023-07-14T11:53:21.567104+0000","last_peered":"2023-07-14T11:53:21.567104+0000","last_clean":"2023-07-14T11:53:21.567104+0000","last_became_active":"2023-07-14T11:53:19.681631+0000","last_became_peered":"2023-07-14T11:53:19.681631+0000","last_unstale":"2023-07-14T11:53:21.567104+0000","last_undegraded":"2023-07-14T11:53:21.567104+0000","last_fullsized":"2023-07-14T11:53:21.567104+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:18.616216+0000","last_clean_scrub_stamp":"2023-07-14T11:53:18.616216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:35.608315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664672+0000","last_change":"2023-07-14T11:53:15.581069+0000","last_active":"2023-07-14T11:53:20.664672+0000","last_peered":"2023-07-14T11:53:20.664672+0000","last_clean":"2023-07-14T11:53:20.664672+0000","last_became_active":"2023-07-14T11:53:15.580609+0000","last_became_peered":"2023-07-14T11:53:15.580609+0000","last_unstale":"2023-07-14T11:53:20.664672+0000","last_undegraded":"2023-07-14T11:53:20.664672+0000","last_fullsized":"2023-07-14T11:53:20.664672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:11:11.201276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567881+0000","last_change":"2023-07-14T11:53:15.616860+0000","last_active":"2023-07-14T11:53:21.567881+0000","last_peered":"2023-07-14T11:53:21.567881+0000","last_clean":"2023-07-14T11:53:21.567881+0000","last_became_active":"2023-07-14T11:53:15.616511+0000","last_became_peered":"2023-07-14T11:53:15.616511+0000","last_unstale":"2023-07-14T11:53:21.567881+0000","last_undegraded":"2023-07-14T11:53:21.567881+0000","last_fullsized":"2023-07-14T11:53:21.567881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663897+0000","last_change":"2023-07-14T11:53:15.580935+0000","last_active":"2023-07-14T11:53:20.663897+0000","last_peered":"2023-07-14T11:53:20.663897+0000","last_clean":"2023-07-14T11:53:20.663897+0000","last_became_active":"2023-07-14T11:53:15.580624+0000","last_became_peered":"2023-07-14T11:53:15.580624+0000","last_unstale":"2023-07-14T11:53:20.663897+0000","last_undegraded":"2023-07-14T11:53:20.663897+0000","last_fullsized":"2023-07-14T11:53:20.663897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:54:27.366163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569054+0000","last_change":"2023-07-14T11:53:15.608124+0000","last_active":"2023-07-14T11:53:21.569054+0000","last_peered":"2023-07-14T11:53:21.569054+0000","last_clean":"2023-07-14T11:53:21.569054+0000","last_became_active":"2023-07-14T11:53:15.606540+0000","last_became_peered":"2023-07-14T11:53:15.606540+0000","last_unstale":"2023-07-14T11:53:21.569054+0000","last_undegraded":"2023-07-14T11:53:21.569054+0000","last_fullsized":"2023-07-14T11:53:21.569054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:36:57.891774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878952+0000","last_change":"2023-07-14T11:53:15.607644+0000","last_active":"2023-07-14T11:53:20.878952+0000","last_peered":"2023-07-14T11:53:20.878952+0000","last_clean":"2023-07-14T11:53:20.878952+0000","last_became_active":"2023-07-14T11:53:15.607318+0000","last_became_peered":"2023-07-14T11:53:15.607318+0000","last_unstale":"2023-07-14T11:53:20.878952+0000","last_undegraded":"2023-07-14T11:53:20.878952+0000","last_fullsized":"2023-07-14T11:53:20.878952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:31.536389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567297+0000","last_change":"2023-07-14T11:53:15.603168+0000","last_active":"2023-07-14T11:53:21.567297+0000","last_peered":"2023-07-14T11:53:21.567297+0000","last_clean":"2023-07-14T11:53:21.567297+0000","last_became_active":"2023-07-14T11:53:15.602791+0000","last_became_peered":"2023-07-14T11:53:15.602791+0000","last_unstale":"2023-07-14T11:53:21.567297+0000","last_undegraded":"2023-07-14T11:53:21.567297+0000","last_fullsized":"2023-07-14T11:53:21.567297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:18.763925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878622+0000","last_change":"2023-07-14T11:53:15.600466+0000","last_active":"2023-07-14T11:53:20.878622+0000","last_peered":"2023-07-14T11:53:20.878622+0000","last_clean":"2023-07-14T11:53:20.878622+0000","last_became_active":"2023-07-14T11:53:15.600232+0000","last_became_peered":"2023-07-14T11:53:15.600232+0000","last_unstale":"2023-07-14T11:53:20.878622+0000","last_undegraded":"2023-07-14T11:53:20.878622+0000","last_fullsized":"2023-07-14T11:53:20.878622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.878439+0000","last_change":"2023-07-14T11:53:15.605971+0000","last_active":"2023-07-14T11:53:20.878439+0000","last_peered":"2023-07-14T11:53:20.878439+0000","last_clean":"2023-07-14T11:53:20.878439+0000","last_became_active":"2023-07-14T11:53:15.605661+0000","last_became_peered":"2023-07-14T11:53:15.605661+0000","last_unstale":"2023-07-14T11:53:20.878439+0000","last_undegraded":"2023-07-14T11:53:20.878439+0000","last_fullsized":"2023-07-14T11:53:20.878439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567617+0000","last_change":"2023-07-14T11:53:15.606414+0000","last_active":"2023-07-14T11:53:21.567617+0000","last_peered":"2023-07-14T11:53:21.567617+0000","last_clean":"2023-07-14T11:53:21.567617+0000","last_became_active":"2023-07-14T11:53:15.605906+0000","last_became_peered":"2023-07-14T11:53:15.605906+0000","last_unstale":"2023-07-14T11:53:21.567617+0000","last_undegraded":"2023-07-14T11:53:21.567617+0000","last_fullsized":"2023-07-14T11:53:21.567617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567728+0000","last_change":"2023-07-14T11:53:15.598006+0000","last_active":"2023-07-14T11:53:21.567728+0000","last_peered":"2023-07-14T11:53:21.567728+0000","last_clean":"2023-07-14T11:53:21.567728+0000","last_became_active":"2023-07-14T11:53:15.597749+0000","last_became_peered":"2023-07-14T11:53:15.597749+0000","last_unstale":"2023-07-14T11:53:21.567728+0000","last_undegraded":"2023-07-14T11:53:21.567728+0000","last_fullsized":"2023-07-14T11:53:21.567728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567052+0000","last_change":"2023-07-14T11:53:15.600730+0000","last_active":"2023-07-14T11:53:21.567052+0000","last_peered":"2023-07-14T11:53:21.567052+0000","last_clean":"2023-07-14T11:53:21.567052+0000","last_became_active":"2023-07-14T11:53:15.600299+0000","last_became_peered":"2023-07-14T11:53:15.600299+0000","last_unstale":"2023-07-14T11:53:21.567052+0000","last_undegraded":"2023-07-14T11:53:21.567052+0000","last_fullsized":"2023-07-14T11:53:21.567052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:18:17.533263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568245+0000","last_change":"2023-07-14T11:53:15.619732+0000","last_active":"2023-07-14T11:53:21.568245+0000","last_peered":"2023-07-14T11:53:21.568245+0000","last_clean":"2023-07-14T11:53:21.568245+0000","last_became_active":"2023-07-14T11:53:15.619425+0000","last_became_peered":"2023-07-14T11:53:15.619425+0000","last_unstale":"2023-07-14T11:53:21.568245+0000","last_undegraded":"2023-07-14T11:53:21.568245+0000","last_fullsized":"2023-07-14T11:53:21.568245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:13.888256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663540+0000","last_change":"2023-07-14T11:53:15.586169+0000","last_active":"2023-07-14T11:53:20.663540+0000","last_peered":"2023-07-14T11:53:20.663540+0000","last_clean":"2023-07-14T11:53:20.663540+0000","last_became_active":"2023-07-14T11:53:15.585876+0000","last_became_peered":"2023-07-14T11:53:15.585876+0000","last_unstale":"2023-07-14T11:53:20.663540+0000","last_undegraded":"2023-07-14T11:53:20.663540+0000","last_fullsized":"2023-07-14T11:53:20.663540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:47:20.678085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661229+0000","last_change":"2023-07-14T11:53:15.584035+0000","last_active":"2023-07-14T11:53:20.661229+0000","last_peered":"2023-07-14T11:53:20.661229+0000","last_clean":"2023-07-14T11:53:20.661229+0000","last_became_active":"2023-07-14T11:53:15.582303+0000","last_became_peered":"2023-07-14T11:53:15.582303+0000","last_unstale":"2023-07-14T11:53:20.661229+0000","last_undegraded":"2023-07-14T11:53:20.661229+0000","last_fullsized":"2023-07-14T11:53:20.661229+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:48:16.619167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664187+0000","last_change":"2023-07-14T11:53:15.578919+0000","last_active":"2023-07-14T11:53:20.664187+0000","last_peered":"2023-07-14T11:53:20.664187+0000","last_clean":"2023-07-14T11:53:20.664187+0000","last_became_active":"2023-07-14T11:53:15.578590+0000","last_became_peered":"2023-07-14T11:53:15.578590+0000","last_unstale":"2023-07-14T11:53:20.664187+0000","last_undegraded":"2023-07-14T11:53:20.664187+0000","last_fullsized":"2023-07-14T11:53:20.664187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:03:59.635384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568314+0000","last_change":"2023-07-14T11:53:15.613102+0000","last_active":"2023-07-14T11:53:21.568314+0000","last_peered":"2023-07-14T11:53:21.568314+0000","last_clean":"2023-07-14T11:53:21.568314+0000","last_became_active":"2023-07-14T11:53:15.612540+0000","last_became_peered":"2023-07-14T11:53:15.612540+0000","last_unstale":"2023-07-14T11:53:21.568314+0000","last_undegraded":"2023-07-14T11:53:21.568314+0000","last_fullsized":"2023-07-14T11:53:21.568314+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:11:29.641887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.662627+0000","last_change":"2023-07-14T11:53:15.592089+0000","last_active":"2023-07-14T11:53:20.662627+0000","last_peered":"2023-07-14T11:53:20.662627+0000","last_clean":"2023-07-14T11:53:20.662627+0000","last_became_active":"2023-07-14T11:53:15.591826+0000","last_became_peered":"2023-07-14T11:53:15.591826+0000","last_unstale":"2023-07-14T11:53:20.662627+0000","last_undegraded":"2023-07-14T11:53:20.662627+0000","last_fullsized":"2023-07-14T11:53:20.662627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:06:57.233859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568086+0000","last_change":"2023-07-14T11:53:15.606636+0000","last_active":"2023-07-14T11:53:21.568086+0000","last_peered":"2023-07-14T11:53:21.568086+0000","last_clean":"2023-07-14T11:53:21.568086+0000","last_became_active":"2023-07-14T11:53:15.606067+0000","last_became_peered":"2023-07-14T11:53:15.606067+0000","last_unstale":"2023-07-14T11:53:21.568086+0000","last_undegraded":"2023-07-14T11:53:21.568086+0000","last_fullsized":"2023-07-14T11:53:21.568086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:03:13.267093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566709+0000","last_change":"2023-07-14T11:53:15.599253+0000","last_active":"2023-07-14T11:53:21.566709+0000","last_peered":"2023-07-14T11:53:21.566709+0000","last_clean":"2023-07-14T11:53:21.566709+0000","last_became_active":"2023-07-14T11:53:15.598955+0000","last_became_peered":"2023-07-14T11:53:15.598955+0000","last_unstale":"2023-07-14T11:53:21.566709+0000","last_undegraded":"2023-07-14T11:53:21.566709+0000","last_fullsized":"2023-07-14T11:53:21.566709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.660486+0000","last_change":"2023-07-14T11:53:15.592119+0000","last_active":"2023-07-14T11:53:20.660486+0000","last_peered":"2023-07-14T11:53:20.660486+0000","last_clean":"2023-07-14T11:53:20.660486+0000","last_became_active":"2023-07-14T11:53:15.591822+0000","last_became_peered":"2023-07-14T11:53:15.591822+0000","last_unstale":"2023-07-14T11:53:20.660486+0000","last_undegraded":"2023-07-14T11:53:20.660486+0000","last_fullsized":"2023-07-14T11:53:20.660486+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:04:29.483383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661402+0000","last_change":"2023-07-14T11:53:15.590820+0000","last_active":"2023-07-14T11:53:20.661402+0000","last_peered":"2023-07-14T11:53:20.661402+0000","last_clean":"2023-07-14T11:53:20.661402+0000","last_became_active":"2023-07-14T11:53:15.590485+0000","last_became_peered":"2023-07-14T11:53:15.590485+0000","last_unstale":"2023-07-14T11:53:20.661402+0000","last_undegraded":"2023-07-14T11:53:20.661402+0000","last_fullsized":"2023-07-14T11:53:20.661402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:42.120586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664155+0000","last_change":"2023-07-14T11:53:15.589398+0000","last_active":"2023-07-14T11:53:20.664155+0000","last_peered":"2023-07-14T11:53:20.664155+0000","last_clean":"2023-07-14T11:53:20.664155+0000","last_became_active":"2023-07-14T11:53:15.589111+0000","last_became_peered":"2023-07-14T11:53:15.589111+0000","last_unstale":"2023-07-14T11:53:20.664155+0000","last_undegraded":"2023-07-14T11:53:20.664155+0000","last_fullsized":"2023-07-14T11:53:20.664155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:40:44.358262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569248+0000","last_change":"2023-07-14T11:53:15.599081+0000","last_active":"2023-07-14T11:53:21.569248+0000","last_peered":"2023-07-14T11:53:21.569248+0000","last_clean":"2023-07-14T11:53:21.569248+0000","last_became_active":"2023-07-14T11:53:15.598786+0000","last_became_peered":"2023-07-14T11:53:15.598786+0000","last_unstale":"2023-07-14T11:53:21.569248+0000","last_undegraded":"2023-07-14T11:53:21.569248+0000","last_fullsized":"2023-07-14T11:53:21.569248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:45.682657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567984+0000","last_change":"2023-07-14T11:53:15.613338+0000","last_active":"2023-07-14T11:53:21.567984+0000","last_peered":"2023-07-14T11:53:21.567984+0000","last_clean":"2023-07-14T11:53:21.567984+0000","last_became_active":"2023-07-14T11:53:15.612876+0000","last_became_peered":"2023-07-14T11:53:15.612876+0000","last_unstale":"2023-07-14T11:53:21.567984+0000","last_undegraded":"2023-07-14T11:53:21.567984+0000","last_fullsized":"2023-07-14T11:53:21.567984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:25.978611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.566164+0000","last_change":"2023-07-14T11:53:15.608445+0000","last_active":"2023-07-14T11:53:21.566164+0000","last_peered":"2023-07-14T11:53:21.566164+0000","last_clean":"2023-07-14T11:53:21.566164+0000","last_became_active":"2023-07-14T11:53:15.608146+0000","last_became_peered":"2023-07-14T11:53:15.608146+0000","last_unstale":"2023-07-14T11:53:21.566164+0000","last_undegraded":"2023-07-14T11:53:21.566164+0000","last_fullsized":"2023-07-14T11:53:21.566164+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:55.943920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.567855+0000","last_change":"2023-07-14T11:53:15.606963+0000","last_active":"2023-07-14T11:53:21.567855+0000","last_peered":"2023-07-14T11:53:21.567855+0000","last_clean":"2023-07-14T11:53:21.567855+0000","last_became_active":"2023-07-14T11:53:15.606397+0000","last_became_peered":"2023-07-14T11:53:15.606397+0000","last_unstale":"2023-07-14T11:53:21.567855+0000","last_undegraded":"2023-07-14T11:53:21.567855+0000","last_fullsized":"2023-07-14T11:53:21.567855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:24.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.663074+0000","last_change":"2023-07-14T11:53:15.607539+0000","last_active":"2023-07-14T11:53:20.663074+0000","last_peered":"2023-07-14T11:53:20.663074+0000","last_clean":"2023-07-14T11:53:20.663074+0000","last_became_active":"2023-07-14T11:53:15.593279+0000","last_became_peered":"2023-07-14T11:53:15.593279+0000","last_unstale":"2023-07-14T11:53:20.663074+0000","last_undegraded":"2023-07-14T11:53:20.663074+0000","last_fullsized":"2023-07-14T11:53:20.663074+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:34.163470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.569328+0000","last_change":"2023-07-14T11:53:15.592332+0000","last_active":"2023-07-14T11:53:21.569328+0000","last_peered":"2023-07-14T11:53:21.569328+0000","last_clean":"2023-07-14T11:53:21.569328+0000","last_became_active":"2023-07-14T11:53:15.591983+0000","last_became_peered":"2023-07-14T11:53:15.591983+0000","last_unstale":"2023-07-14T11:53:21.569328+0000","last_undegraded":"2023-07-14T11:53:21.569328+0000","last_fullsized":"2023-07-14T11:53:21.569328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:18.763925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.665081+0000","last_change":"2023-07-14T11:53:15.594122+0000","last_active":"2023-07-14T11:53:20.665081+0000","last_peered":"2023-07-14T11:53:20.665081+0000","last_clean":"2023-07-14T11:53:20.665081+0000","last_became_active":"2023-07-14T11:53:15.593773+0000","last_became_peered":"2023-07-14T11:53:15.593773+0000","last_unstale":"2023-07-14T11:53:20.665081+0000","last_undegraded":"2023-07-14T11:53:20.665081+0000","last_fullsized":"2023-07-14T11:53:20.665081+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:59:21.935826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.664107+0000","last_change":"2023-07-14T11:53:15.593314+0000","last_active":"2023-07-14T11:53:20.664107+0000","last_peered":"2023-07-14T11:53:20.664107+0000","last_clean":"2023-07-14T11:53:20.664107+0000","last_became_active":"2023-07-14T11:53:15.592778+0000","last_became_peered":"2023-07-14T11:53:15.592778+0000","last_unstale":"2023-07-14T11:53:20.664107+0000","last_undegraded":"2023-07-14T11:53:20.664107+0000","last_fullsized":"2023-07-14T11:53:20.664107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:37.263541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:21.568937+0000","last_change":"2023-07-14T11:53:15.599579+0000","last_active":"2023-07-14T11:53:21.568937+0000","last_peered":"2023-07-14T11:53:21.568937+0000","last_clean":"2023-07-14T11:53:21.568937+0000","last_became_active":"2023-07-14T11:53:15.599380+0000","last_became_peered":"2023-07-14T11:53:15.599380+0000","last_unstale":"2023-07-14T11:53:21.568937+0000","last_undegraded":"2023-07-14T11:53:21.568937+0000","last_fullsized":"2023-07-14T11:53:21.568937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:19:29.603411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-14T11:53:20.661719+0000","last_change":"2023-07-14T11:53:15.594232+0000","last_active":"2023-07-14T11:53:20.661719+0000","last_peered":"2023-07-14T11:53:20.661719+0000","last_clean":"2023-07-14T11:53:20.661719+0000","last_became_active":"2023-07-14T11:53:15.594049+0000","last_became_peered":"2023-07-14T11:53:15.594049+0000","last_unstale":"2023-07-14T11:53:20.661719+0000","last_undegraded":"2023-07-14T11:53:20.661719+0000","last_fullsized":"2023-07-14T11:53:20.661719+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:14.544291+0000","last_clean_scrub_stamp":"2023-07-14T11:53:14.544291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:51.380702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4591,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":925696,"data_stored":718949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.46}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.502}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":311296,"data_stored":125224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.371}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.704}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":323584,"data_stored":128421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":901120,"data_stored":715522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:53:48.424 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-14T11:53:48.424 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-14T11:53:48.424 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-07-14T11:53:48.425 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-14T11:53:48.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.636+0000 7f23c8a1f700 1 Processor -- start 2023-07-14T11:53:48.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.636+0000 7f23c8a1f700 1 -- start start 2023-07-14T11:53:48.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.636+0000 7f23c8a1f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f23c415f140 0x7f23c40c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.636+0000 7f23c8a1f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f23c41601a0 0x7f23c40caa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.636+0000 7f23c8a1f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23c4071770 0x7f23c40caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.636+0000 7f23c8a1f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c40ad290 con 0x7f23c415f140 2023-07-14T11:53:48.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.636+0000 7f23c8a1f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c415f510 con 0x7f23c41601a0 2023-07-14T11:53:48.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.636+0000 7f23c8a1f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c40ad500 con 0x7f23c4071770 2023-07-14T11:53:48.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.636+0000 7f23c2d9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f23c415f140 0x7f23c40c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.636+0000 7f23c2d9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f23c415f140 0x7f23c40c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38578/0 (socket says 172.21.15.104:38578) 2023-07-14T11:53:48.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.636+0000 7f23c2d9d700 1 -- 172.21.15.104:0/251314945 learned_addr learned my addr 172.21.15.104:0/251314945 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:48.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c359e700 1 --2- 172.21.15.104:0/251314945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23c4071770 0x7f23c40caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23bbfff700 1 --2- 172.21.15.104:0/251314945 >> v2:172.21.15.121:3300/0 conn(0x7f23c41601a0 0x7f23c40caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c2d9d700 1 -- 172.21.15.104:0/251314945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23c4071770 msgr2=0x7f23c40caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:48.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c2d9d700 1 --2- 172.21.15.104:0/251314945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23c4071770 0x7f23c40caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c2d9d700 1 -- 172.21.15.104:0/251314945 >> v2:172.21.15.121:3300/0 conn(0x7f23c41601a0 msgr2=0x7f23c40caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:48.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c2d9d700 1 --2- 172.21.15.104:0/251314945 >> v2:172.21.15.121:3300/0 conn(0x7f23c41601a0 0x7f23c40caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c2d9d700 1 -- 172.21.15.104:0/251314945 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c40ad7e0 con 0x7f23c415f140 2023-07-14T11:53:48.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23bbfff700 1 --2- 172.21.15.104:0/251314945 >> v2:172.21.15.121:3300/0 conn(0x7f23c41601a0 0x7f23c40caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:48.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c2d9d700 1 --2- 172.21.15.104:0/251314945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23c415f140 0x7f23c40c4670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f23c406aaa0 tx=0x7f23b4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6d0e8525f80618 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:48.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c259c700 1 -- 172.21.15.104:0/251314945 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b400b8a0 con 0x7f23c415f140 2023-07-14T11:53:48.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c259c700 1 -- 172.21.15.104:0/251314945 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b400ba00 con 0x7f23c415f140 2023-07-14T11:53:48.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c259c700 1 -- 172.21.15.104:0/251314945 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b400bd90 con 0x7f23c415f140 2023-07-14T11:53:48.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c8a1f700 1 -- 172.21.15.104:0/251314945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23c415f140 msgr2=0x7f23c40c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c8a1f700 1 --2- 172.21.15.104:0/251314945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23c415f140 0x7f23c40c4670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f23c406aaa0 tx=0x7f23b4009bf0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c8a1f700 1 -- 172.21.15.104:0/251314945 shutdown_connections 2023-07-14T11:53:48.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c8a1f700 1 --2- 172.21.15.104:0/251314945 >> v2:172.21.15.121:3300/0 conn(0x7f23c41601a0 0x7f23c40caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c8a1f700 1 --2- 172.21.15.104:0/251314945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23c4071770 0x7f23c40caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c8a1f700 1 --2- 172.21.15.104:0/251314945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23c415f140 0x7f23c40c4670 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c8a1f700 1 -- 172.21.15.104:0/251314945 >> 172.21.15.104:0/251314945 conn(0x7f23c406e960 msgr2=0x7f23c406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:48.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c8a1f700 1 -- 172.21.15.104:0/251314945 shutdown_connections 2023-07-14T11:53:48.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.640+0000 7f23c8a1f700 1 -- 172.21.15.104:0/251314945 wait complete. 2023-07-14T11:53:48.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c8a1f700 1 Processor -- start 2023-07-14T11:53:48.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c8a1f700 1 -- start start 2023-07-14T11:53:48.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c8a1f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23c4071770 0x7f23c40b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c8a1f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c415f140 0x7f23c40b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c8a1f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23c41601a0 0x7f23c4156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c8a1f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c41f11b0 con 0x7f23c4071770 2023-07-14T11:53:48.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c8a1f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c40b2f70 con 0x7f23c415f140 2023-07-14T11:53:48.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c8a1f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c40b3250 con 0x7f23c41601a0 2023-07-14T11:53:48.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c2d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23c4071770 0x7f23c40b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c2d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23c4071770 0x7f23c40b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38590/0 (socket says 172.21.15.104:38590) 2023-07-14T11:53:48.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c2d9d700 1 -- 172.21.15.104:0/2237822207 learned_addr learned my addr 172.21.15.104:0/2237822207 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:48.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c359e700 1 --2- 172.21.15.104:0/2237822207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23c41601a0 0x7f23c4156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23bbfff700 1 --2- 172.21.15.104:0/2237822207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c415f140 0x7f23c40b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c2d9d700 1 -- 172.21.15.104:0/2237822207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23c41601a0 msgr2=0x7f23c4156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c2d9d700 1 --2- 172.21.15.104:0/2237822207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23c41601a0 0x7f23c4156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c2d9d700 1 -- 172.21.15.104:0/2237822207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c415f140 msgr2=0x7f23c40b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c2d9d700 1 --2- 172.21.15.104:0/2237822207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c415f140 0x7f23c40b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23c2d9d700 1 -- 172.21.15.104:0/2237822207 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b4009570 con 0x7f23c4071770 2023-07-14T11:53:48.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.644+0000 7f23bbfff700 1 --2- 172.21.15.104:0/2237822207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c415f140 0x7f23c40b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:48.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.648+0000 7f23c2d9d700 1 --2- 172.21.15.104:0/2237822207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23c4071770 0x7f23c40b2990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f23b4000c00 tx=0x7f23b4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:48.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.648+0000 7f23bb7fe700 1 -- 172.21.15.104:0/2237822207 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b40038c0 con 0x7f23c4071770 2023-07-14T11:53:48.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.648+0000 7f23bb7fe700 1 -- 172.21.15.104:0/2237822207 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b401e070 con 0x7f23c4071770 2023-07-14T11:53:48.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.648+0000 7f23c8a1f700 1 -- 172.21.15.104:0/2237822207 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c40b3530 con 0x7f23c4071770 2023-07-14T11:53:48.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.648+0000 7f23bb7fe700 1 -- 172.21.15.104:0/2237822207 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4003c80 con 0x7f23c4071770 2023-07-14T11:53:48.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.648+0000 7f23c8a1f700 1 -- 172.21.15.104:0/2237822207 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c40b1fc0 con 0x7f23c4071770 2023-07-14T11:53:48.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.648+0000 7f23c8a1f700 1 -- 172.21.15.104:0/2237822207 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c40adbd0 con 0x7f23c4071770 2023-07-14T11:53:48.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.652+0000 7f23bb7fe700 1 -- 172.21.15.104:0/2237822207 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f23b4011420 con 0x7f23c4071770 2023-07-14T11:53:48.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.652+0000 7f23bb7fe700 1 --2- 172.21.15.104:0/2237822207 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f23a80662f0 0x7f23a80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.652+0000 7f23bb7fe700 1 -- 172.21.15.104:0/2237822207 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f23b4020070 con 0x7f23c4071770 2023-07-14T11:53:48.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.652+0000 7f23bbfff700 1 --2- 172.21.15.104:0/2237822207 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f23a80662f0 0x7f23a80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.652+0000 7f23bbfff700 1 --2- 172.21.15.104:0/2237822207 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f23a80662f0 0x7f23a80687a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f23c4070b80 tx=0x7f23ac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:48.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.656+0000 7f23bb7fe700 1 -- 172.21.15.104:0/2237822207 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f23c40adbd0 con 0x7f23c4071770 2023-07-14T11:53:48.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.868+0000 7f23c8a1f700 1 -- 172.21.15.104:0/2237822207 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f23c40adbd0 con 0x7f23c4071770 2023-07-14T11:53:48.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.872+0000 7f23bb7fe700 1 -- 172.21.15.104:0/2237822207 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f23b40529a0 con 0x7f23c4071770 2023-07-14T11:53:48.855 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:48.855 INFO:teuthology.orchestra.run.smithi104.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-14T11:53:48.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.872+0000 7f23c8a1f700 1 -- 172.21.15.104:0/2237822207 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f23a80662f0 msgr2=0x7f23a80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.872+0000 7f23c8a1f700 1 --2- 172.21.15.104:0/2237822207 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f23a80662f0 0x7f23a80687a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f23c4070b80 tx=0x7f23ac008040 comp rx=0 tx=0).stop 2023-07-14T11:53:48.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.872+0000 7f23c8a1f700 1 -- 172.21.15.104:0/2237822207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23c4071770 msgr2=0x7f23c40b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.872+0000 7f23c8a1f700 1 --2- 172.21.15.104:0/2237822207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23c4071770 0x7f23c40b2990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f23b4000c00 tx=0x7f23b4004770 comp rx=0 tx=0).stop 2023-07-14T11:53:48.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.872+0000 7f23c8a1f700 1 -- 172.21.15.104:0/2237822207 shutdown_connections 2023-07-14T11:53:48.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.872+0000 7f23c8a1f700 1 --2- 172.21.15.104:0/2237822207 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f23a80662f0 0x7f23a80687a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.872+0000 7f23c8a1f700 1 --2- 172.21.15.104:0/2237822207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23c41601a0 0x7f23c4156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.872+0000 7f23c8a1f700 1 --2- 172.21.15.104:0/2237822207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c415f140 0x7f23c40b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.872+0000 7f23c8a1f700 1 --2- 172.21.15.104:0/2237822207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23c4071770 0x7f23c40b2990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.872+0000 7f23c8a1f700 1 -- 172.21.15.104:0/2237822207 >> 172.21.15.104:0/2237822207 conn(0x7f23c406e960 msgr2=0x7f23c40c8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:48.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.872+0000 7f23c8a1f700 1 -- 172.21.15.104:0/2237822207 shutdown_connections 2023-07-14T11:53:48.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:48.872+0000 7f23c8a1f700 1 -- 172.21.15.104:0/2237822207 wait complete. 2023-07-14T11:53:48.881 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-07-14T11:53:48.881 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:49.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.060+0000 7f26137dc700 1 Processor -- start 2023-07-14T11:53:49.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.060+0000 7f26137dc700 1 -- start start 2023-07-14T11:53:49.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.064+0000 7f26137dc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f260c0c9c80 0x7f260c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.064+0000 7f26137dc700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f260c0c97f0 con 0x7f260c0afd30 2023-07-14T11:53:49.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.064+0000 7f26137dc700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f260c0c8900 con 0x7f260c0cc1b0 2023-07-14T11:53:49.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.064+0000 7f26137dc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260c0c8be0 con 0x7f260c0c9c80 2023-07-14T11:53:49.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.064+0000 7f2611578700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f260c0afd30 0x7f260c0b0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42582/0 (socket says 172.21.15.104:42582) 2023-07-14T11:53:49.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.064+0000 7f2611578700 1 -- 172.21.15.104:0/693622965 learned_addr learned my addr 172.21.15.104:0/693622965 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:49.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.064+0000 7f2610d77700 1 --2- 172.21.15.104:0/693622965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f260c0c9c80 0x7f260c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 -- 172.21.15.104:0/693622965 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290790981 0 0) 0x7f260c0c97f0 con 0x7f260c0afd30 2023-07-14T11:53:49.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 -- 172.21.15.104:0/693622965 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25f80035d0 con 0x7f260c0afd30 2023-07-14T11:53:49.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 -- 172.21.15.104:0/693622965 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 970332337 0 0) 0x7f25f80035d0 con 0x7f260c0afd30 2023-07-14T11:53:49.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 -- 172.21.15.104:0/693622965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f260c0c9c80 msgr2=0x7f260c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:49.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 --2- 172.21.15.104:0/693622965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f260c0c9c80 0x7f260c06d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 -- 172.21.15.104:0/693622965 >> v1:172.21.15.121:6789/0 conn(0x7f260c0cc1b0 legacy=0x7f260c0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 -- 172.21.15.104:0/693622965 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c0c8ec0 con 0x7f260c0afd30 2023-07-14T11:53:49.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 -- 172.21.15.104:0/693622965 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f2608001670 con 0x7f260c0afd30 2023-07-14T11:53:49.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 -- 172.21.15.104:0/693622965 >> v1:172.21.15.104:6789/0 conn(0x7f260c0afd30 legacy=0x7f260c0b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 --2- 172.21.15.104:0/693622965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25f8003a50 0x7f25f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 --2- 172.21.15.104:0/693622965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f260c0c9c80 0x7f25f8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 --2- 172.21.15.104:0/693622965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25f8004db0 0x7f25f8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 -- 172.21.15.104:0/693622965 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c0c8be0 con 0x7f25f8004db0 2023-07-14T11:53:49.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 -- 172.21.15.104:0/693622965 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f800dc40 con 0x7f25f8003a50 2023-07-14T11:53:49.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 -- 172.21.15.104:0/693622965 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f800ddd0 con 0x7f260c0c9c80 2023-07-14T11:53:49.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2611578700 1 --2- 172.21.15.104:0/693622965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25f8003a50 0x7f25f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2610d77700 1 --2- 172.21.15.104:0/693622965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f260c0c9c80 0x7f25f8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2611d79700 1 --2- 172.21.15.104:0/693622965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25f8004db0 0x7f25f8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2611578700 1 -- 172.21.15.104:0/693622965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f260c0c9c80 msgr2=0x7f25f8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:49.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2611578700 1 --2- 172.21.15.104:0/693622965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f260c0c9c80 0x7f25f8004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2611578700 1 -- 172.21.15.104:0/693622965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25f8004db0 msgr2=0x7f25f8005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:49.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2611578700 1 --2- 172.21.15.104:0/693622965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25f8004db0 0x7f25f8005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2611578700 1 -- 172.21.15.104:0/693622965 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f80035d0 con 0x7f25f8003a50 2023-07-14T11:53:49.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2611578700 1 --2- 172.21.15.104:0/693622965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25f8003a50 0x7f25f8003e20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f260800c2f0 tx=0x7f25f80058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56bdbadf2979c2ae server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:49.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 -- 172.21.15.104:0/693622965 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260800ce30 con 0x7f25f8003a50 2023-07-14T11:53:49.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 -- 172.21.15.104:0/693622965 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2608005980 con 0x7f25f8003a50 2023-07-14T11:53:49.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2603fff700 1 -- 172.21.15.104:0/693622965 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608005d10 con 0x7f25f8003a50 2023-07-14T11:53:49.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f26137dc700 1 -- 172.21.15.104:0/693622965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25f8003a50 msgr2=0x7f25f8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f26137dc700 1 --2- 172.21.15.104:0/693622965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25f8003a50 0x7f25f8003e20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f260800c2f0 tx=0x7f25f80058a0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f2611d79700 1 -- 172.21.15.104:0/693622965 reap_dead start 2023-07-14T11:53:49.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f26137dc700 1 -- 172.21.15.104:0/693622965 shutdown_connections 2023-07-14T11:53:49.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.068+0000 7f26137dc700 1 -- 172.21.15.104:0/693622965 >> 172.21.15.104:0/693622965 conn(0x7f260c0cad30 msgr2=0x7f260c0cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:49.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.072+0000 7f26137dc700 1 -- 172.21.15.104:0/693622965 shutdown_connections 2023-07-14T11:53:49.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.072+0000 7f26137dc700 1 -- 172.21.15.104:0/693622965 wait complete. 2023-07-14T11:53:49.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.072+0000 7f26137dc700 1 Processor -- start 2023-07-14T11:53:49.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.072+0000 7f26137dc700 1 -- start start 2023-07-14T11:53:49.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.072+0000 7f26137dc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25f8003a50 0x7f260c0c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.072+0000 7f26137dc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f260c1593b0 0x7f260c159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.072+0000 7f26137dc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f260c1575e0 0x7f260c157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.072+0000 7f26137dc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c157f90 con 0x7f260c1575e0 2023-07-14T11:53:49.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.072+0000 7f26137dc700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c15a6a0 con 0x7f260c1593b0 2023-07-14T11:53:49.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.072+0000 7f26137dc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260c159e00 con 0x7f25f8003a50 2023-07-14T11:53:49.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f2611d79700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f260c1575e0 0x7f260c157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f2611d79700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f260c1575e0 0x7f260c157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38610/0 (socket says 172.21.15.104:38610) 2023-07-14T11:53:49.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f2611d79700 1 -- 172.21.15.104:0/2470145986 learned_addr learned my addr 172.21.15.104:0/2470145986 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:49.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f2611578700 1 --2- 172.21.15.104:0/2470145986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25f8003a50 0x7f260c0c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f2610d77700 1 --2- 172.21.15.104:0/2470145986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f260c1593b0 0x7f260c159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f2611d79700 1 -- 172.21.15.104:0/2470145986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25f8003a50 msgr2=0x7f260c0c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f2611d79700 1 --2- 172.21.15.104:0/2470145986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25f8003a50 0x7f260c0c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f2611d79700 1 -- 172.21.15.104:0/2470145986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f260c1593b0 msgr2=0x7f260c159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f2611d79700 1 --2- 172.21.15.104:0/2470145986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f260c1593b0 0x7f260c159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f2611d79700 1 -- 172.21.15.104:0/2470145986 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c15a0e0 con 0x7f260c1575e0 2023-07-14T11:53:49.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f2611d79700 1 --2- 172.21.15.104:0/2470145986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f260c1575e0 0x7f260c157a50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f260c06be00 tx=0x7f260400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:49.062 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f2601ffb700 1 -- 172.21.15.104:0/2470145986 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400fe70 con 0x7f260c1575e0 2023-07-14T11:53:49.062 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f2601ffb700 1 -- 172.21.15.104:0/2470145986 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2604003c30 con 0x7f260c1575e0 2023-07-14T11:53:49.062 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f2601ffb700 1 -- 172.21.15.104:0/2470145986 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400d990 con 0x7f260c1575e0 2023-07-14T11:53:49.062 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f26137dc700 1 -- 172.21.15.104:0/2470145986 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260c15a3c0 con 0x7f260c1575e0 2023-07-14T11:53:49.062 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f26137dc700 1 -- 172.21.15.104:0/2470145986 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260c0b3c60 con 0x7f260c1575e0 2023-07-14T11:53:49.063 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.076+0000 7f26137dc700 1 -- 172.21.15.104:0/2470145986 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260c0b3df0 con 0x7f260c1575e0 2023-07-14T11:53:49.064 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.080+0000 7f2601ffb700 1 -- 172.21.15.104:0/2470145986 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f260400db10 con 0x7f260c1575e0 2023-07-14T11:53:49.065 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.080+0000 7f2601ffb700 1 --2- 172.21.15.104:0/2470145986 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f25f0066320 0x7f25f00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.065 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.080+0000 7f2601ffb700 1 -- 172.21.15.104:0/2470145986 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f26040842b0 con 0x7f260c1575e0 2023-07-14T11:53:49.065 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.080+0000 7f2611578700 1 --2- 172.21.15.104:0/2470145986 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f25f0066320 0x7f25f00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.066 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.084+0000 7f2611578700 1 --2- 172.21.15.104:0/2470145986 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f25f0066320 0x7f25f00687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2608008fd0 tx=0x7f2608007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:49.070 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.088+0000 7f2601ffb700 1 -- 172.21.15.104:0/2470145986 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f26040513c0 con 0x7f260c1575e0 2023-07-14T11:53:49.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.244+0000 7f26137dc700 1 -- 172.21.15.104:0/2470145986 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f260c0ce0b0 con 0x7f260c1575e0 2023-07-14T11:53:49.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.244+0000 7f2601ffb700 1 -- 172.21.15.104:0/2470145986 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20156 (secure 0 0 0) 0x7f26040549e0 con 0x7f260c1575e0 2023-07-14T11:53:49.231 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:49.232 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":28,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:20.091434+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:49.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.248+0000 7f26137dc700 1 -- 172.21.15.104:0/2470145986 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f25f0066320 msgr2=0x7f25f00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.248+0000 7f26137dc700 1 --2- 172.21.15.104:0/2470145986 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f25f0066320 0x7f25f00687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2608008fd0 tx=0x7f2608007000 comp rx=0 tx=0).stop 2023-07-14T11:53:49.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.248+0000 7f26137dc700 1 -- 172.21.15.104:0/2470145986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f260c1575e0 msgr2=0x7f260c157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.248+0000 7f26137dc700 1 --2- 172.21.15.104:0/2470145986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f260c1575e0 0x7f260c157a50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f260c06be00 tx=0x7f260400d430 comp rx=0 tx=0).stop 2023-07-14T11:53:49.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.248+0000 7f26137dc700 1 -- 172.21.15.104:0/2470145986 shutdown_connections 2023-07-14T11:53:49.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.248+0000 7f26137dc700 1 --2- 172.21.15.104:0/2470145986 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f25f0066320 0x7f25f00687d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.248+0000 7f26137dc700 1 --2- 172.21.15.104:0/2470145986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f260c1575e0 0x7f260c157a50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.248+0000 7f26137dc700 1 --2- 172.21.15.104:0/2470145986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f260c1593b0 0x7f260c159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.248+0000 7f26137dc700 1 --2- 172.21.15.104:0/2470145986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f25f8003a50 0x7f260c0c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.248+0000 7f26137dc700 1 -- 172.21.15.104:0/2470145986 >> 172.21.15.104:0/2470145986 conn(0x7f260c0cad30 msgr2=0x7f260c0ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:49.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.248+0000 7f26137dc700 1 -- 172.21.15.104:0/2470145986 shutdown_connections 2023-07-14T11:53:49.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.248+0000 7f26137dc700 1 -- 172.21.15.104:0/2470145986 wait complete. 2023-07-14T11:53:49.252 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:53:49.253 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:53:49.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.436+0000 7fd87b365700 1 Processor -- start 2023-07-14T11:53:49.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.436+0000 7fd87b365700 1 -- start start 2023-07-14T11:53:49.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.440+0000 7fd87b365700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd87415cff0 0x7fd874070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.440+0000 7fd87b365700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd874160720 con 0x7fd874160300 2023-07-14T11:53:49.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.440+0000 7fd87b365700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd874070110 con 0x7fd8740d0260 2023-07-14T11:53:49.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.440+0000 7fd87b365700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd874070a90 con 0x7fd87415cff0 2023-07-14T11:53:49.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.440+0000 7fd879902700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fd874160300 0x7fd8740d2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42598/0 (socket says 172.21.15.104:42598) 2023-07-14T11:53:49.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.440+0000 7fd879902700 1 -- 172.21.15.104:0/2969461294 learned_addr learned my addr 172.21.15.104:0/2969461294 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:49.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.440+0000 7fd879101700 1 --2- 172.21.15.104:0/2969461294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd87415cff0 0x7fd874070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.440+0000 7fd86bfff700 1 -- 172.21.15.104:0/2969461294 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 681968726 0 0) 0x7fd874160720 con 0x7fd874160300 2023-07-14T11:53:49.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.440+0000 7fd86bfff700 1 -- 172.21.15.104:0/2969461294 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8600035d0 con 0x7fd874160300 2023-07-14T11:53:49.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd86bfff700 1 -- 172.21.15.104:0/2969461294 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365037090 0 0) 0x7fd874070110 con 0x7fd8740d0260 2023-07-14T11:53:49.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd86bfff700 1 -- 172.21.15.104:0/2969461294 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd874160720 con 0x7fd8740d0260 2023-07-14T11:53:49.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd879101700 1 -- 172.21.15.104:0/2969461294 >> v1:172.21.15.121:6789/0 conn(0x7fd8740d0260 legacy=0x7fd87415fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd86bfff700 1 -- 172.21.15.104:0/2969461294 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1146550364 0 0) 0x7fd8600035d0 con 0x7fd874160300 2023-07-14T11:53:49.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd879101700 1 -- 172.21.15.104:0/2969461294 >> v1:172.21.15.104:6789/0 conn(0x7fd874160300 legacy=0x7fd8740d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd879101700 1 -- 172.21.15.104:0/2969461294 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd874160990 con 0x7fd87415cff0 2023-07-14T11:53:49.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd879101700 1 --2- 172.21.15.104:0/2969461294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd87415cff0 0x7fd874070eb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd870009880 tx=0x7fd870009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b61d6481af07f34 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:49.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd86bfff700 1 -- 172.21.15.104:0/2969461294 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0019f0 con 0x7fd87415cff0 2023-07-14T11:53:49.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd86bfff700 1 -- 172.21.15.104:0/2969461294 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd870004030 con 0x7fd87415cff0 2023-07-14T11:53:49.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd86bfff700 1 -- 172.21.15.104:0/2969461294 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8700043c0 con 0x7fd87415cff0 2023-07-14T11:53:49.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd87b365700 1 -- 172.21.15.104:0/2969461294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd87415cff0 msgr2=0x7fd874070eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd87b365700 1 --2- 172.21.15.104:0/2969461294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd87415cff0 0x7fd874070eb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd870009880 tx=0x7fd870009b90 comp rx=0 tx=0).stop 2023-07-14T11:53:49.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd87b365700 1 -- 172.21.15.104:0/2969461294 shutdown_connections 2023-07-14T11:53:49.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd87b365700 1 --2- 172.21.15.104:0/2969461294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd87415cff0 0x7fd874070eb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd87b365700 1 -- 172.21.15.104:0/2969461294 >> 172.21.15.104:0/2969461294 conn(0x7fd874060c40 msgr2=0x7fd87409ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:49.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd87b365700 1 -- 172.21.15.104:0/2969461294 shutdown_connections 2023-07-14T11:53:49.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.444+0000 7fd87b365700 1 -- 172.21.15.104:0/2969461294 wait complete. 2023-07-14T11:53:49.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd87b365700 1 Processor -- start 2023-07-14T11:53:49.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd87b365700 1 -- start start 2023-07-14T11:53:49.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd87b365700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8740d0260 0x7fd874149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd87b365700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd87415cff0 0x7fd8740b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd87b365700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd874160300 0x7fd8740b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd87b365700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87414a580 con 0x7fd87415cff0 2023-07-14T11:53:49.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd87b365700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8740b1970 con 0x7fd8740d0260 2023-07-14T11:53:49.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd87b365700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8740c8200 con 0x7fd874160300 2023-07-14T11:53:49.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd878900700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd87415cff0 0x7fd8740b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd878900700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd87415cff0 0x7fd8740b0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38626/0 (socket says 172.21.15.104:38626) 2023-07-14T11:53:49.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd878900700 1 -- 172.21.15.104:0/3669759930 learned_addr learned my addr 172.21.15.104:0/3669759930 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:49.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd879101700 1 --2- 172.21.15.104:0/3669759930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8740d0260 0x7fd874149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd879902700 1 --2- 172.21.15.104:0/3669759930 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd874160300 0x7fd8740b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd878900700 1 -- 172.21.15.104:0/3669759930 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd874160300 msgr2=0x7fd8740b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd878900700 1 --2- 172.21.15.104:0/3669759930 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd874160300 0x7fd8740b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd878900700 1 -- 172.21.15.104:0/3669759930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8740d0260 msgr2=0x7fd874149ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd878900700 1 --2- 172.21.15.104:0/3669759930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8740d0260 0x7fd874149ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd878900700 1 -- 172.21.15.104:0/3669759930 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8700094b0 con 0x7fd87415cff0 2023-07-14T11:53:49.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.448+0000 7fd879902700 1 --2- 172.21.15.104:0/3669759930 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd874160300 0x7fd8740b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:49.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.452+0000 7fd878900700 1 --2- 172.21.15.104:0/3669759930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd87415cff0 0x7fd8740b0c30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd86400c4c0 tx=0x7fd86400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:49.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.452+0000 7fd869ffb700 1 -- 172.21.15.104:0/3669759930 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8640052d0 con 0x7fd87415cff0 2023-07-14T11:53:49.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.452+0000 7fd869ffb700 1 -- 172.21.15.104:0/3669759930 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd864005450 con 0x7fd87415cff0 2023-07-14T11:53:49.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.452+0000 7fd87b365700 1 -- 172.21.15.104:0/3669759930 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd87414a7d0 con 0x7fd87415cff0 2023-07-14T11:53:49.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.452+0000 7fd869ffb700 1 -- 172.21.15.104:0/3669759930 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8640038e0 con 0x7fd87415cff0 2023-07-14T11:53:49.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.452+0000 7fd87b365700 1 -- 172.21.15.104:0/3669759930 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87414ad90 con 0x7fd87415cff0 2023-07-14T11:53:49.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.452+0000 7fd87b365700 1 -- 172.21.15.104:0/3669759930 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd87404dcb0 con 0x7fd87415cff0 2023-07-14T11:53:49.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.456+0000 7fd869ffb700 1 -- 172.21.15.104:0/3669759930 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd864003a40 con 0x7fd87415cff0 2023-07-14T11:53:49.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.456+0000 7fd869ffb700 1 --2- 172.21.15.104:0/3669759930 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8580662f0 0x7fd8580687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.456+0000 7fd869ffb700 1 -- 172.21.15.104:0/3669759930 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd864083430 con 0x7fd87415cff0 2023-07-14T11:53:49.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.456+0000 7fd879101700 1 --2- 172.21.15.104:0/3669759930 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8580662f0 0x7fd8580687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.460+0000 7fd879101700 1 --2- 172.21.15.104:0/3669759930 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8580662f0 0x7fd8580687a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd8700090b0 tx=0x7fd870010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:49.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.464+0000 7fd869ffb700 1 -- 172.21.15.104:0/3669759930 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fd864050670 con 0x7fd87415cff0 2023-07-14T11:53:49.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.624+0000 7fd87b365700 1 -- 172.21.15.104:0/3669759930 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd87408bf00 con 0x7fd87415cff0 2023-07-14T11:53:49.608 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.624+0000 7fd869ffb700 1 -- 172.21.15.104:0/3669759930 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd864053c90 con 0x7fd87415cff0 2023-07-14T11:53:49.608 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:53:49.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.624+0000 7fd87b365700 1 -- 172.21.15.104:0/3669759930 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8580662f0 msgr2=0x7fd8580687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.628+0000 7fd87b365700 1 --2- 172.21.15.104:0/3669759930 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8580662f0 0x7fd8580687a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd8700090b0 tx=0x7fd870010000 comp rx=0 tx=0).stop 2023-07-14T11:53:49.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.628+0000 7fd87b365700 1 -- 172.21.15.104:0/3669759930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd87415cff0 msgr2=0x7fd8740b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.628+0000 7fd87b365700 1 --2- 172.21.15.104:0/3669759930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd87415cff0 0x7fd8740b0c30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd86400c4c0 tx=0x7fd86400c7d0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.628+0000 7fd87b365700 1 -- 172.21.15.104:0/3669759930 shutdown_connections 2023-07-14T11:53:49.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.628+0000 7fd87b365700 1 --2- 172.21.15.104:0/3669759930 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8580662f0 0x7fd8580687a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.628+0000 7fd87b365700 1 --2- 172.21.15.104:0/3669759930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd87415cff0 0x7fd8740b0c30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.628+0000 7fd87b365700 1 --2- 172.21.15.104:0/3669759930 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd874160300 0x7fd8740b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.628+0000 7fd87b365700 1 --2- 172.21.15.104:0/3669759930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8740d0260 0x7fd874149ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.628+0000 7fd87b365700 1 -- 172.21.15.104:0/3669759930 >> 172.21.15.104:0/3669759930 conn(0x7fd874060c40 msgr2=0x7fd87415edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:49.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.628+0000 7fd87b365700 1 -- 172.21.15.104:0/3669759930 shutdown_connections 2023-07-14T11:53:49.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.628+0000 7fd87b365700 1 -- 172.21.15.104:0/3669759930 wait complete. 2023-07-14T11:53:49.628 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:53:49.801 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.816+0000 7fd70193e700 1 Processor -- start 2023-07-14T11:53:49.801 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.816+0000 7fd70193e700 1 -- start start 2023-07-14T11:53:49.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd70193e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd6fc06c220 0x7fd6fc0cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd70193e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fd6fc0cc260 0x7fd6fc06d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd70193e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6fc06d310 con 0x7fd6fc158a20 2023-07-14T11:53:49.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd70193e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc06c700 con 0x7fd6fc0cc260 2023-07-14T11:53:49.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd70193e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc06c9e0 con 0x7fd6fc06c220 2023-07-14T11:53:49.806 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd70093c700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fd6fc158a20 0x7fd6fc0cb2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42606/0 (socket says 172.21.15.104:42606) 2023-07-14T11:53:49.806 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd70093c700 1 -- 172.21.15.104:0/4204679635 learned_addr learned my addr 172.21.15.104:0/4204679635 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:49.806 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd70113d700 1 --2- 172.21.15.104:0/4204679635 >> v2:172.21.15.104:3300/0 conn(0x7fd6fc0cc260 0x7fd6fc06d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.806 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd6fbfff700 1 --2- 172.21.15.104:0/4204679635 >> v2:172.21.15.121:3300/0 conn(0x7fd6fc06c220 0x7fd6fc0cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.806 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd6fb7fe700 1 -- 172.21.15.104:0/4204679635 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381921648 0 0) 0x7fd6fc06d310 con 0x7fd6fc158a20 2023-07-14T11:53:49.806 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd6fb7fe700 1 -- 172.21.15.104:0/4204679635 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6ec0057e0 con 0x7fd6fc158a20 2023-07-14T11:53:49.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd70113d700 1 -- 172.21.15.104:0/4204679635 >> v2:172.21.15.121:3300/0 conn(0x7fd6fc06c220 msgr2=0x7fd6fc0cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:49.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd70113d700 1 --2- 172.21.15.104:0/4204679635 >> v2:172.21.15.121:3300/0 conn(0x7fd6fc06c220 0x7fd6fc0cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd70113d700 1 -- 172.21.15.104:0/4204679635 >> v1:172.21.15.104:6789/0 conn(0x7fd6fc158a20 legacy=0x7fd6fc0cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd70113d700 1 -- 172.21.15.104:0/4204679635 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc06ccc0 con 0x7fd6fc0cc260 2023-07-14T11:53:49.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.820+0000 7fd70113d700 1 --2- 172.21.15.104:0/4204679635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6fc0cc260 0x7fd6fc06d7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc06a630 tx=0x7fd6f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b208afffe898b7a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:49.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.824+0000 7fd6fb7fe700 1 -- 172.21.15.104:0/4204679635 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400e070 con 0x7fd6fc0cc260 2023-07-14T11:53:49.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.824+0000 7fd6fb7fe700 1 -- 172.21.15.104:0/4204679635 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f400bb20 con 0x7fd6fc0cc260 2023-07-14T11:53:49.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.824+0000 7fd6fb7fe700 1 -- 172.21.15.104:0/4204679635 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4012430 con 0x7fd6fc0cc260 2023-07-14T11:53:49.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.824+0000 7fd70193e700 1 -- 172.21.15.104:0/4204679635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6fc0cc260 msgr2=0x7fd6fc06d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.824+0000 7fd70193e700 1 --2- 172.21.15.104:0/4204679635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6fc0cc260 0x7fd6fc06d7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc06a630 tx=0x7fd6f400b330 comp rx=0 tx=0).stop 2023-07-14T11:53:49.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.824+0000 7fd70193e700 1 -- 172.21.15.104:0/4204679635 shutdown_connections 2023-07-14T11:53:49.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.824+0000 7fd70193e700 1 --2- 172.21.15.104:0/4204679635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6fc0cc260 0x7fd6fc06d7a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.824+0000 7fd70193e700 1 --2- 172.21.15.104:0/4204679635 >> v2:172.21.15.121:3300/0 conn(0x7fd6fc06c220 0x7fd6fc0cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.824+0000 7fd70193e700 1 -- 172.21.15.104:0/4204679635 >> 172.21.15.104:0/4204679635 conn(0x7fd6fc0afc60 msgr2=0x7fd6fc0b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:49.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.824+0000 7fd70193e700 1 -- 172.21.15.104:0/4204679635 shutdown_connections 2023-07-14T11:53:49.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.824+0000 7fd70193e700 1 -- 172.21.15.104:0/4204679635 wait complete. 2023-07-14T11:53:49.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.828+0000 7fd70193e700 1 Processor -- start 2023-07-14T11:53:49.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.828+0000 7fd70193e700 1 -- start start 2023-07-14T11:53:49.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.828+0000 7fd70193e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6fc06c220 0x7fd6fc0c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.828+0000 7fd70193e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6fc0cc260 0x7fd6fc0c1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70193e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc158a20 0x7fd6fc0b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70193e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc154ec0 con 0x7fd6fc06c220 2023-07-14T11:53:49.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70193e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc155480 con 0x7fd6fc158a20 2023-07-14T11:53:49.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70193e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0b3a20 con 0x7fd6fc0cc260 2023-07-14T11:53:49.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70093c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6fc06c220 0x7fd6fc0c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70093c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6fc06c220 0x7fd6fc0c3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38644/0 (socket says 172.21.15.104:38644) 2023-07-14T11:53:49.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70093c700 1 -- 172.21.15.104:0/518369391 learned_addr learned my addr 172.21.15.104:0/518369391 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:49.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70093c700 1 -- 172.21.15.104:0/518369391 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6fc0cc260 msgr2=0x7fd6fc0c1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70093c700 1 --2- 172.21.15.104:0/518369391 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6fc0cc260 0x7fd6fc0c1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70113d700 1 --2- 172.21.15.104:0/518369391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc158a20 0x7fd6fc0b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70093c700 1 -- 172.21.15.104:0/518369391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc158a20 msgr2=0x7fd6fc0b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:49.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70093c700 1 --2- 172.21.15.104:0/518369391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc158a20 0x7fd6fc0b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:49.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70093c700 1 -- 172.21.15.104:0/518369391 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f4009a70 con 0x7fd6fc06c220 2023-07-14T11:53:49.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70093c700 1 --2- 172.21.15.104:0/518369391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6fc06c220 0x7fd6fc0c3e30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c500 tx=0x7fd6f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:49.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd6f97fa700 1 -- 172.21.15.104:0/518369391 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0005d90 con 0x7fd6fc06c220 2023-07-14T11:53:49.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70193e700 1 -- 172.21.15.104:0/518369391 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6fc0b3d00 con 0x7fd6fc06c220 2023-07-14T11:53:49.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd6f97fa700 1 -- 172.21.15.104:0/518369391 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f00070d0 con 0x7fd6fc06c220 2023-07-14T11:53:49.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd6f97fa700 1 -- 172.21.15.104:0/518369391 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0007480 con 0x7fd6fc06c220 2023-07-14T11:53:49.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70193e700 1 -- 172.21.15.104:0/518369391 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6fc0b36b0 con 0x7fd6fc06c220 2023-07-14T11:53:49.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.832+0000 7fd70193e700 1 -- 172.21.15.104:0/518369391 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6fc04e500 con 0x7fd6fc06c220 2023-07-14T11:53:49.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.836+0000 7fd6f97fa700 1 -- 172.21.15.104:0/518369391 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd6f00075e0 con 0x7fd6fc06c220 2023-07-14T11:53:49.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.836+0000 7fd6f97fa700 1 --2- 172.21.15.104:0/518369391 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd6e00662f0 0x7fd6e00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:49.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.836+0000 7fd6f97fa700 1 -- 172.21.15.104:0/518369391 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd6f0083700 con 0x7fd6fc06c220 2023-07-14T11:53:49.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.836+0000 7fd6fbfff700 1 --2- 172.21.15.104:0/518369391 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd6e00662f0 0x7fd6e00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:49.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.836+0000 7fd6fbfff700 1 --2- 172.21.15.104:0/518369391 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd6e00662f0 0x7fd6e00687a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8006b40 tx=0x7fd6e8006ad0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:49.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:49.844+0000 7fd6f97fa700 1 -- 172.21.15.104:0/518369391 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fd6f00506e0 con 0x7fd6fc06c220 2023-07-14T11:53:50.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.016+0000 7fd70193e700 1 -- 172.21.15.104:0/518369391 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd6fc0b40d0 con 0x7fd6fc06c220 2023-07-14T11:53:50.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.016+0000 7fd6f97fa700 1 -- 172.21.15.104:0/518369391 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd6f0053d00 con 0x7fd6fc06c220 2023-07-14T11:53:50.001 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:53:50.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.020+0000 7fd70193e700 1 -- 172.21.15.104:0/518369391 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd6e00662f0 msgr2=0x7fd6e00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.020+0000 7fd70193e700 1 --2- 172.21.15.104:0/518369391 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd6e00662f0 0x7fd6e00687a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8006b40 tx=0x7fd6e8006ad0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.020+0000 7fd70193e700 1 -- 172.21.15.104:0/518369391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6fc06c220 msgr2=0x7fd6fc0c3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.020+0000 7fd70193e700 1 --2- 172.21.15.104:0/518369391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6fc06c220 0x7fd6fc0c3e30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c500 tx=0x7fd6f000c8c0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.020+0000 7fd70193e700 1 -- 172.21.15.104:0/518369391 shutdown_connections 2023-07-14T11:53:50.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.020+0000 7fd70193e700 1 --2- 172.21.15.104:0/518369391 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd6e00662f0 0x7fd6e00687a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.020+0000 7fd70193e700 1 --2- 172.21.15.104:0/518369391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6fc158a20 0x7fd6fc0b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.020+0000 7fd70193e700 1 --2- 172.21.15.104:0/518369391 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6fc0cc260 0x7fd6fc0c1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.020+0000 7fd70193e700 1 --2- 172.21.15.104:0/518369391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6fc06c220 0x7fd6fc0c3e30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.020+0000 7fd70193e700 1 -- 172.21.15.104:0/518369391 >> 172.21.15.104:0/518369391 conn(0x7fd6fc0afc60 msgr2=0x7fd6fc0c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:50.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.020+0000 7fd70193e700 1 -- 172.21.15.104:0/518369391 shutdown_connections 2023-07-14T11:53:50.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.020+0000 7fd70193e700 1 -- 172.21.15.104:0/518369391 wait complete. 2023-07-14T11:53:50.023 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:53:50.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.208+0000 7fb8e4ecb700 1 Processor -- start 2023-07-14T11:53:50.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.212+0000 7fb8e4ecb700 1 -- start start 2023-07-14T11:53:50.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.212+0000 7fb8e4ecb700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb8e015fa10 0x7fb8e015fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.212+0000 7fb8e4ecb700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e00d0930 con 0x7fb8e0160320 2023-07-14T11:53:50.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.212+0000 7fb8e4ecb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e00b4ae0 con 0x7fb8e00b01c0 2023-07-14T11:53:50.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.212+0000 7fb8e4ecb700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e00ce560 con 0x7fb8e015fa10 2023-07-14T11:53:50.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.212+0000 7fb8df7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb8e015fa10 0x7fb8e015fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.212+0000 7fb8df7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb8e015fa10 0x7fb8e015fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38660/0 (socket says 172.21.15.104:38660) 2023-07-14T11:53:50.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.212+0000 7fb8deffd700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb8e0160320 0x7fb8e00ccb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:42620/0 (socket says 172.21.15.104:42620) 2023-07-14T11:53:50.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.212+0000 7fb8df7fe700 1 -- 172.21.15.104:0/284982022 learned_addr learned my addr 172.21.15.104:0/284982022 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:50.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.212+0000 7fb8de7fc700 1 -- 172.21.15.104:0/284982022 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269299252 0 0) 0x7fb8e00d0930 con 0x7fb8e0160320 2023-07-14T11:53:50.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.212+0000 7fb8de7fc700 1 -- 172.21.15.104:0/284982022 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8c80035d0 con 0x7fb8e0160320 2023-07-14T11:53:50.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.212+0000 7fb8df7fe700 1 -- 172.21.15.104:0/284982022 >> v1:172.21.15.121:6789/0 conn(0x7fb8e00b01c0 legacy=0x7fb8e00cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.212+0000 7fb8df7fe700 1 -- 172.21.15.104:0/284982022 >> v1:172.21.15.104:6789/0 conn(0x7fb8e0160320 legacy=0x7fb8e00ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.212+0000 7fb8df7fe700 1 -- 172.21.15.104:0/284982022 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e00cd990 con 0x7fb8e015fa10 2023-07-14T11:53:50.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.216+0000 7fb8df7fe700 1 --2- 172.21.15.104:0/284982022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8e015fa10 0x7fb8e015fde0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009900 tx=0x7fb8d0009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341585f211c446ad server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:50.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.216+0000 7fb8de7fc700 1 -- 172.21.15.104:0/284982022 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0011830 con 0x7fb8e015fa10 2023-07-14T11:53:50.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.216+0000 7fb8de7fc700 1 -- 172.21.15.104:0/284982022 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d0011990 con 0x7fb8e015fa10 2023-07-14T11:53:50.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.216+0000 7fb8de7fc700 1 -- 172.21.15.104:0/284982022 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0011d40 con 0x7fb8e015fa10 2023-07-14T11:53:50.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.216+0000 7fb8e4ecb700 1 -- 172.21.15.104:0/284982022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8e015fa10 msgr2=0x7fb8e015fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.216+0000 7fb8e4ecb700 1 --2- 172.21.15.104:0/284982022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8e015fa10 0x7fb8e015fde0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009900 tx=0x7fb8d0009c10 comp rx=0 tx=0).stop 2023-07-14T11:53:50.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.216+0000 7fb8e4ecb700 1 -- 172.21.15.104:0/284982022 shutdown_connections 2023-07-14T11:53:50.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.216+0000 7fb8e4ecb700 1 --2- 172.21.15.104:0/284982022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8e015fa10 0x7fb8e015fde0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.216+0000 7fb8e4ecb700 1 -- 172.21.15.104:0/284982022 >> 172.21.15.104:0/284982022 conn(0x7fb8e009b850 msgr2=0x7fb8e00d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:50.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.216+0000 7fb8e4ecb700 1 -- 172.21.15.104:0/284982022 shutdown_connections 2023-07-14T11:53:50.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.216+0000 7fb8e4ecb700 1 -- 172.21.15.104:0/284982022 wait complete. 2023-07-14T11:53:50.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8e4ecb700 1 Processor -- start 2023-07-14T11:53:50.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8e4ecb700 1 -- start start 2023-07-14T11:53:50.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8e4ecb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e00b01c0 0x7fb8e01599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8e4ecb700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8e015fa10 0x7fb8e00b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8e4ecb700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8e0160320 0x7fb8e00b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8e4ecb700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e00ca550 con 0x7fb8e0160320 2023-07-14T11:53:50.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8e4ecb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e0071050 con 0x7fb8e00b01c0 2023-07-14T11:53:50.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8e4ecb700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e00c95f0 con 0x7fb8e015fa10 2023-07-14T11:53:50.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8dffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8e0160320 0x7fb8e00b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8dffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8e0160320 0x7fb8e00b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38672/0 (socket says 172.21.15.104:38672) 2023-07-14T11:53:50.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8dffff700 1 -- 172.21.15.104:0/1568163125 learned_addr learned my addr 172.21.15.104:0/1568163125 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:50.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8df7fe700 1 --2- 172.21.15.104:0/1568163125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e00b01c0 0x7fb8e01599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8deffd700 1 --2- 172.21.15.104:0/1568163125 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8e015fa10 0x7fb8e00b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8dffff700 1 -- 172.21.15.104:0/1568163125 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8e015fa10 msgr2=0x7fb8e00b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8dffff700 1 --2- 172.21.15.104:0/1568163125 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8e015fa10 0x7fb8e00b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8dffff700 1 -- 172.21.15.104:0/1568163125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e00b01c0 msgr2=0x7fb8e01599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8dffff700 1 --2- 172.21.15.104:0/1568163125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e00b01c0 0x7fb8e01599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8dffff700 1 -- 172.21.15.104:0/1568163125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d0009510 con 0x7fb8e0160320 2023-07-14T11:53:50.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8df7fe700 1 --2- 172.21.15.104:0/1568163125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e00b01c0 0x7fb8e01599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:50.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.220+0000 7fb8dffff700 1 --2- 172.21.15.104:0/1568163125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8e0160320 0x7fb8e00b15d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800cb50 tx=0x7fb8d800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:50.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.224+0000 7fb8c7fff700 1 -- 172.21.15.104:0/1568163125 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8003c20 con 0x7fb8e0160320 2023-07-14T11:53:50.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.224+0000 7fb8c7fff700 1 -- 172.21.15.104:0/1568163125 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d8003d80 con 0x7fb8e0160320 2023-07-14T11:53:50.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.224+0000 7fb8c7fff700 1 -- 172.21.15.104:0/1568163125 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d800aac0 con 0x7fb8e0160320 2023-07-14T11:53:50.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.224+0000 7fb8e4ecb700 1 -- 172.21.15.104:0/1568163125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e00c98d0 con 0x7fb8e0160320 2023-07-14T11:53:50.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.224+0000 7fb8e4ecb700 1 -- 172.21.15.104:0/1568163125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e00c9e90 con 0x7fb8e0160320 2023-07-14T11:53:50.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.224+0000 7fb8e4ecb700 1 -- 172.21.15.104:0/1568163125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e00d3430 con 0x7fb8e0160320 2023-07-14T11:53:50.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.224+0000 7fb8c7fff700 1 -- 172.21.15.104:0/1568163125 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb8d800ac20 con 0x7fb8e0160320 2023-07-14T11:53:50.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.228+0000 7fb8c7fff700 1 --2- 172.21.15.104:0/1568163125 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb8cc0662f0 0x7fb8cc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.228+0000 7fb8c7fff700 1 -- 172.21.15.104:0/1568163125 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb8d8083d70 con 0x7fb8e0160320 2023-07-14T11:53:50.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.228+0000 7fb8df7fe700 1 --2- 172.21.15.104:0/1568163125 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb8cc0662f0 0x7fb8cc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.228+0000 7fb8df7fe700 1 --2- 172.21.15.104:0/1568163125 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb8cc0662f0 0x7fb8cc0687a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0006f80 tx=0x7fb8d0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:50.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.236+0000 7fb8c7fff700 1 -- 172.21.15.104:0/1568163125 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb8d8051040 con 0x7fb8e0160320 2023-07-14T11:53:50.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.408+0000 7fb8e4ecb700 1 -- 172.21.15.104:0/1568163125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8e00b4ae0 con 0x7fb8e0160320 2023-07-14T11:53:50.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.408+0000 7fb8c7fff700 1 -- 172.21.15.104:0/1568163125 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb8e00b4ae0 con 0x7fb8e0160320 2023-07-14T11:53:50.393 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:53:50.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.412+0000 7fb8e4ecb700 1 -- 172.21.15.104:0/1568163125 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb8cc0662f0 msgr2=0x7fb8cc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.412+0000 7fb8e4ecb700 1 --2- 172.21.15.104:0/1568163125 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb8cc0662f0 0x7fb8cc0687a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0006f80 tx=0x7fb8d0008030 comp rx=0 tx=0).stop 2023-07-14T11:53:50.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.412+0000 7fb8e4ecb700 1 -- 172.21.15.104:0/1568163125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8e0160320 msgr2=0x7fb8e00b15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.412+0000 7fb8e4ecb700 1 --2- 172.21.15.104:0/1568163125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8e0160320 0x7fb8e00b15d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800cb50 tx=0x7fb8d800ce60 comp rx=0 tx=0).stop 2023-07-14T11:53:50.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.412+0000 7fb8e4ecb700 1 -- 172.21.15.104:0/1568163125 shutdown_connections 2023-07-14T11:53:50.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.412+0000 7fb8e4ecb700 1 --2- 172.21.15.104:0/1568163125 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb8cc0662f0 0x7fb8cc0687a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.412+0000 7fb8e4ecb700 1 --2- 172.21.15.104:0/1568163125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8e0160320 0x7fb8e00b15d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.412+0000 7fb8e4ecb700 1 --2- 172.21.15.104:0/1568163125 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8e015fa10 0x7fb8e00b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.412+0000 7fb8e4ecb700 1 --2- 172.21.15.104:0/1568163125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e00b01c0 0x7fb8e01599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.412+0000 7fb8e4ecb700 1 -- 172.21.15.104:0/1568163125 >> 172.21.15.104:0/1568163125 conn(0x7fb8e009b850 msgr2=0x7fb8e0071520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:50.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.412+0000 7fb8e4ecb700 1 -- 172.21.15.104:0/1568163125 shutdown_connections 2023-07-14T11:53:50.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.412+0000 7fb8e4ecb700 1 -- 172.21.15.104:0/1568163125 wait complete. 2023-07-14T11:53:50.418 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:53:50.594 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.608+0000 7f757e781700 1 Processor -- start 2023-07-14T11:53:50.594 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.608+0000 7f757e781700 1 -- start start 2023-07-14T11:53:50.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f757e781700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75780cf710 0x7f75780d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f757e781700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7578071d10 con 0x7f75780b1c50 2023-07-14T11:53:50.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f757e781700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757815c420 con 0x7f75780d0210 2023-07-14T11:53:50.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f757e781700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7578070840 con 0x7f75780cf710 2023-07-14T11:53:50.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f7577fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f75780b1c50 0x7f7578071600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:48384/0 (socket says 172.21.15.104:48384) 2023-07-14T11:53:50.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f7577fff700 1 -- 172.21.15.104:0/4087404740 learned_addr learned my addr 172.21.15.104:0/4087404740 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:50.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f757cd1e700 1 --2- 172.21.15.104:0/4087404740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75780cf710 0x7f75780d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f7576ffd700 1 -- 172.21.15.104:0/4087404740 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988820926 0 0) 0x7f7568001110 con 0x7f75780b1c50 2023-07-14T11:53:50.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f7576ffd700 1 -- 172.21.15.104:0/4087404740 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75600035d0 con 0x7f75780b1c50 2023-07-14T11:53:50.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f7576ffd700 1 -- 172.21.15.104:0/4087404740 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951508489 0 0) 0x7f757815c420 con 0x7f75780d0210 2023-07-14T11:53:50.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f7576ffd700 1 -- 172.21.15.104:0/4087404740 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7568001110 con 0x7f75780d0210 2023-07-14T11:53:50.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f7576ffd700 1 -- 172.21.15.104:0/4087404740 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 382674992 0 0) 0x7f75600035d0 con 0x7f75780b1c50 2023-07-14T11:53:50.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f757cd1e700 1 -- 172.21.15.104:0/4087404740 >> v1:172.21.15.121:6789/0 conn(0x7f75780d0210 legacy=0x7f75780cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f757cd1e700 1 -- 172.21.15.104:0/4087404740 >> v1:172.21.15.104:6789/0 conn(0x7f75780b1c50 legacy=0x7f7578071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f757cd1e700 1 -- 172.21.15.104:0/4087404740 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7578070b20 con 0x7f75780cf710 2023-07-14T11:53:50.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f757cd1e700 1 --2- 172.21.15.104:0/4087404740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75780cf710 0x7f75780d0640 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7570009dc0 tx=0x7f757000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40fecfa72debc48a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:50.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.612+0000 7f7576ffd700 1 -- 172.21.15.104:0/4087404740 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757000e070 con 0x7f75780cf710 2023-07-14T11:53:50.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.616+0000 7f7576ffd700 1 -- 172.21.15.104:0/4087404740 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7570003a40 con 0x7f75780cf710 2023-07-14T11:53:50.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.616+0000 7f7576ffd700 1 -- 172.21.15.104:0/4087404740 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568001820 con 0x7f75780cf710 2023-07-14T11:53:50.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.616+0000 7f757e781700 1 -- 172.21.15.104:0/4087404740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75780cf710 msgr2=0x7f75780d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.616+0000 7f757e781700 1 --2- 172.21.15.104:0/4087404740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75780cf710 0x7f75780d0640 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7570009dc0 tx=0x7f757000b330 comp rx=0 tx=0).stop 2023-07-14T11:53:50.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.616+0000 7f757e781700 1 -- 172.21.15.104:0/4087404740 shutdown_connections 2023-07-14T11:53:50.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.616+0000 7f757e781700 1 --2- 172.21.15.104:0/4087404740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75780cf710 0x7f75780d0640 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.616+0000 7f757e781700 1 -- 172.21.15.104:0/4087404740 >> 172.21.15.104:0/4087404740 conn(0x7f757809b9c0 msgr2=0x7f75780d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:50.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.616+0000 7f757e781700 1 -- 172.21.15.104:0/4087404740 shutdown_connections 2023-07-14T11:53:50.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.616+0000 7f757e781700 1 -- 172.21.15.104:0/4087404740 wait complete. 2023-07-14T11:53:50.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.616+0000 7f757e781700 1 Processor -- start 2023-07-14T11:53:50.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.616+0000 7f757e781700 1 -- start start 2023-07-14T11:53:50.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f757e781700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75780b1c50 0x7f757815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f757e781700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75780cf710 0x7f757809c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f757e781700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75780d0210 0x7f75780b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f757e781700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757809d270 con 0x7f75780cf710 2023-07-14T11:53:50.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f757e781700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757809d550 con 0x7f75780b1c50 2023-07-14T11:53:50.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f757e781700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757809d830 con 0x7f75780d0210 2023-07-14T11:53:50.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f7577fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75780b1c50 0x7f757815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f75777fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75780cf710 0x7f757809c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f757cd1e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75780d0210 0x7f75780b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f7577fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75780b1c50 0x7f757815a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:36830/0 (socket says 172.21.15.104:36830) 2023-07-14T11:53:50.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f7577fff700 1 -- 172.21.15.104:0/3096547196 learned_addr learned my addr 172.21.15.104:0/3096547196 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:50.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f757cd1e700 1 -- 172.21.15.104:0/3096547196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75780b1c50 msgr2=0x7f757815a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f757cd1e700 1 --2- 172.21.15.104:0/3096547196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75780b1c50 0x7f757815a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f757cd1e700 1 -- 172.21.15.104:0/3096547196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75780cf710 msgr2=0x7f757809c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f757cd1e700 1 --2- 172.21.15.104:0/3096547196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75780cf710 0x7f757809c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f757cd1e700 1 -- 172.21.15.104:0/3096547196 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7570009a70 con 0x7f75780d0210 2023-07-14T11:53:50.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f75777fe700 1 --2- 172.21.15.104:0/3096547196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75780cf710 0x7f757809c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:50.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f757cd1e700 1 --2- 172.21.15.104:0/3096547196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75780d0210 0x7f75780b3650 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7570009fd0 tx=0x7f7570004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:50.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f7574ff9700 1 -- 172.21.15.104:0/3096547196 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75700044c0 con 0x7f75780d0210 2023-07-14T11:53:50.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f7574ff9700 1 -- 172.21.15.104:0/3096547196 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7570004660 con 0x7f75780d0210 2023-07-14T11:53:50.608 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f757e781700 1 -- 172.21.15.104:0/3096547196 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757809db10 con 0x7f75780d0210 2023-07-14T11:53:50.608 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f7574ff9700 1 -- 172.21.15.104:0/3096547196 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757001d470 con 0x7f75780d0210 2023-07-14T11:53:50.608 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.620+0000 7f757e781700 1 -- 172.21.15.104:0/3096547196 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757809ddf0 con 0x7f75780d0210 2023-07-14T11:53:50.608 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.624+0000 7f757e781700 1 -- 172.21.15.104:0/3096547196 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757815dcf0 con 0x7f75780d0210 2023-07-14T11:53:50.609 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.624+0000 7f7574ff9700 1 -- 172.21.15.104:0/3096547196 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f757001d5d0 con 0x7f75780d0210 2023-07-14T11:53:50.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.624+0000 7f7574ff9700 1 --2- 172.21.15.104:0/3096547196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f75640662f0 0x7f75640687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.628+0000 7f7574ff9700 1 -- 172.21.15.104:0/3096547196 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7570085be0 con 0x7f75780d0210 2023-07-14T11:53:50.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.628+0000 7f7577fff700 1 --2- 172.21.15.104:0/3096547196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f75640662f0 0x7f75640687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.628+0000 7f7577fff700 1 --2- 172.21.15.104:0/3096547196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f75640662f0 0x7f75640687a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7568001390 tx=0x7f7568007c60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:50.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.636+0000 7f7574ff9700 1 -- 172.21.15.104:0/3096547196 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f757815dcf0 con 0x7f75780d0210 2023-07-14T11:53:50.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.808+0000 7f757e781700 1 -- 172.21.15.104:0/3096547196 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f757815dcf0 con 0x7f75780d0210 2023-07-14T11:53:50.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.808+0000 7f7574ff9700 1 -- 172.21.15.104:0/3096547196 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7570016020 con 0x7f75780d0210 2023-07-14T11:53:50.793 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:53:50.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.812+0000 7f757e781700 1 -- 172.21.15.104:0/3096547196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f75640662f0 msgr2=0x7f75640687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.812+0000 7f757e781700 1 --2- 172.21.15.104:0/3096547196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f75640662f0 0x7f75640687a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7568001390 tx=0x7f7568007c60 comp rx=0 tx=0).stop 2023-07-14T11:53:50.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.812+0000 7f757e781700 1 -- 172.21.15.104:0/3096547196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75780d0210 msgr2=0x7f75780b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.812+0000 7f757e781700 1 --2- 172.21.15.104:0/3096547196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75780d0210 0x7f75780b3650 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7570009fd0 tx=0x7f7570004af0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.812+0000 7f757e781700 1 -- 172.21.15.104:0/3096547196 shutdown_connections 2023-07-14T11:53:50.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.812+0000 7f757e781700 1 --2- 172.21.15.104:0/3096547196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f75640662f0 0x7f75640687a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.812+0000 7f757e781700 1 --2- 172.21.15.104:0/3096547196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75780cf710 0x7f757809c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.812+0000 7f757e781700 1 --2- 172.21.15.104:0/3096547196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75780d0210 0x7f75780b3650 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.812+0000 7f757e781700 1 --2- 172.21.15.104:0/3096547196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75780b1c50 0x7f757815a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.812+0000 7f757e781700 1 -- 172.21.15.104:0/3096547196 >> 172.21.15.104:0/3096547196 conn(0x7f757809b9c0 msgr2=0x7f75780b6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:50.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.812+0000 7f757e781700 1 -- 172.21.15.104:0/3096547196 shutdown_connections 2023-07-14T11:53:50.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:50.812+0000 7f757e781700 1 -- 172.21.15.104:0/3096547196 wait complete. 2023-07-14T11:53:50.819 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:53:50.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.004+0000 7f0fc80f5700 1 Processor -- start 2023-07-14T11:53:50.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.004+0000 7f0fc80f5700 1 -- start start 2023-07-14T11:53:50.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc80f5700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fc0160240 0x7f0fc00cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc80f5700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc0163fe0 con 0x7f0fc00b0490 2023-07-14T11:53:50.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc80f5700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc00c8c30 con 0x7f0fc0160610 2023-07-14T11:53:50.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc80f5700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00ca9e0 con 0x7f0fc0160240 2023-07-14T11:53:50.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc6692700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f0fc00b0490 0x7f0fc006cf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:48396/0 (socket says 172.21.15.104:48396) 2023-07-14T11:53:50.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc6692700 1 -- 172.21.15.104:0/1880389769 learned_addr learned my addr 172.21.15.104:0/1880389769 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:50.993 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc5e91700 1 --2- 172.21.15.104:0/1880389769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fc0160240 0x7f0fc00cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.993 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977407474 0 0) 0x7f0fc0163fe0 con 0x7f0fc00b0490 2023-07-14T11:53:50.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fac0035d0 con 0x7f0fc00b0490 2023-07-14T11:53:50.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172543622 0 0) 0x7f0fc00c8c30 con 0x7f0fc0160610 2023-07-14T11:53:50.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fc0163fe0 con 0x7f0fc0160610 2023-07-14T11:53:50.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2328653633 0 0) 0x7f0fac0035d0 con 0x7f0fc00b0490 2023-07-14T11:53:50.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fc0160240 msgr2=0x7f0fc00cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:50.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 --2- 172.21.15.104:0/1880389769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fc0160240 0x7f0fc00cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 >> v1:172.21.15.121:6789/0 conn(0x7f0fc0160610 legacy=0x7f0fc00afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc0164230 con 0x7f0fc00b0490 2023-07-14T11:53:50.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f0fb80019f0 con 0x7f0fc00b0490 2023-07-14T11:53:50.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 >> v1:172.21.15.104:6789/0 conn(0x7f0fc00b0490 legacy=0x7f0fc006cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 --2- 172.21.15.104:0/1880389769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fac003be0 0x7f0fac003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 --2- 172.21.15.104:0/1880389769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fac004870 0x7f0fac004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 --2- 172.21.15.104:0/1880389769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fc0160240 0x7f0fac005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:50.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00ca9e0 con 0x7f0fac004870 2023-07-14T11:53:50.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac00bb10 con 0x7f0fac003be0 2023-07-14T11:53:50.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac00bcd0 con 0x7f0fc0160240 2023-07-14T11:53:50.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc5690700 1 --2- 172.21.15.104:0/1880389769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fac004870 0x7f0fac004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc5e91700 1 --2- 172.21.15.104:0/1880389769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fac003be0 0x7f0fac003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.008+0000 7f0fc6692700 1 --2- 172.21.15.104:0/1880389769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fc0160240 0x7f0fac005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:50.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc5690700 1 -- 172.21.15.104:0/1880389769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fc0160240 msgr2=0x7f0fac005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:50.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc5690700 1 --2- 172.21.15.104:0/1880389769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fc0160240 0x7f0fac005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc5690700 1 -- 172.21.15.104:0/1880389769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fac003be0 msgr2=0x7f0fac003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:50.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc5690700 1 --2- 172.21.15.104:0/1880389769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fac003be0 0x7f0fac003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc5690700 1 -- 172.21.15.104:0/1880389769 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac0035d0 con 0x7f0fac004870 2023-07-14T11:53:50.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc5690700 1 --2- 172.21.15.104:0/1880389769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fac004870 0x7f0fac004ce0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0001960 tx=0x7f0fac0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f1557c1e81d82a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:50.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0001450 con 0x7f0fac004870 2023-07-14T11:53:50.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb00052f0 con 0x7f0fac004870 2023-07-14T11:53:50.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc4e8f700 1 -- 172.21.15.104:0/1880389769 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00036a0 con 0x7f0fac004870 2023-07-14T11:53:50.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc80f5700 1 -- 172.21.15.104:0/1880389769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fac004870 msgr2=0x7f0fac004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:50.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc80f5700 1 --2- 172.21.15.104:0/1880389769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fac004870 0x7f0fac004ce0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0001960 tx=0x7f0fac0058a0 comp rx=0 tx=0).stop 2023-07-14T11:53:50.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc6692700 1 -- 172.21.15.104:0/1880389769 reap_dead start 2023-07-14T11:53:50.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc80f5700 1 -- 172.21.15.104:0/1880389769 shutdown_connections 2023-07-14T11:53:50.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc80f5700 1 -- 172.21.15.104:0/1880389769 >> 172.21.15.104:0/1880389769 conn(0x7f0fc00cad80 msgr2=0x7f0fc00cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:50.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc80f5700 1 -- 172.21.15.104:0/1880389769 shutdown_connections 2023-07-14T11:53:50.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.012+0000 7f0fc80f5700 1 -- 172.21.15.104:0/1880389769 wait complete. 2023-07-14T11:53:51.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc80f5700 1 Processor -- start 2023-07-14T11:53:51.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc80f5700 1 -- start start 2023-07-14T11:53:51.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc80f5700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fac004870 0x7f0fc00c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:51.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc80f5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fc00b37c0 0x7f0fc00aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:51.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc80f5700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc00af360 0x7f0fc00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:51.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc80f5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00af7e0 con 0x7f0fc00b37c0 2023-07-14T11:53:51.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc80f5700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00c7ef0 con 0x7f0fc00af360 2023-07-14T11:53:51.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc80f5700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00b2720 con 0x7f0fac004870 2023-07-14T11:53:51.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc5690700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fc00b37c0 0x7f0fc00aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:51.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc5690700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fc00b37c0 0x7f0fc00aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38444/0 (socket says 172.21.15.104:38444) 2023-07-14T11:53:51.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc5690700 1 -- 172.21.15.104:0/2031155664 learned_addr learned my addr 172.21.15.104:0/2031155664 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:51.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc6692700 1 --2- 172.21.15.104:0/2031155664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc00af360 0x7f0fc00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:51.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc5e91700 1 --2- 172.21.15.104:0/2031155664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fac004870 0x7f0fc00c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:51.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc5690700 1 -- 172.21.15.104:0/2031155664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fac004870 msgr2=0x7f0fc00c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:51.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc5690700 1 --2- 172.21.15.104:0/2031155664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fac004870 0x7f0fc00c47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc5690700 1 -- 172.21.15.104:0/2031155664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc00af360 msgr2=0x7f0fc00b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:51.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc5690700 1 --2- 172.21.15.104:0/2031155664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc00af360 0x7f0fc00b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc5690700 1 -- 172.21.15.104:0/2031155664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc00b2a00 con 0x7f0fc00b37c0 2023-07-14T11:53:51.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc5690700 1 --2- 172.21.15.104:0/2031155664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fc00b37c0 0x7f0fc00aee20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c3c0 tx=0x7f0fb00012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:51.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fb67fc700 1 -- 172.21.15.104:0/2031155664 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0001450 con 0x7f0fc00b37c0 2023-07-14T11:53:51.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fb67fc700 1 -- 172.21.15.104:0/2031155664 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb0006c90 con 0x7f0fc00b37c0 2023-07-14T11:53:51.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc80f5700 1 -- 172.21.15.104:0/2031155664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc00b2ce0 con 0x7f0fc00b37c0 2023-07-14T11:53:51.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fb67fc700 1 -- 172.21.15.104:0/2031155664 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0007040 con 0x7f0fc00b37c0 2023-07-14T11:53:51.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.016+0000 7f0fc80f5700 1 -- 172.21.15.104:0/2031155664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc00b32a0 con 0x7f0fc00b37c0 2023-07-14T11:53:51.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.020+0000 7f0fc80f5700 1 -- 172.21.15.104:0/2031155664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc00af760 con 0x7f0fc00b37c0 2023-07-14T11:53:51.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.020+0000 7f0fb67fc700 1 -- 172.21.15.104:0/2031155664 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f0fb00071a0 con 0x7f0fc00b37c0 2023-07-14T11:53:51.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.020+0000 7f0fb67fc700 1 --2- 172.21.15.104:0/2031155664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0fa4066320 0x7f0fa40687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:51.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.024+0000 7f0fb67fc700 1 -- 172.21.15.104:0/2031155664 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0fb0084f20 con 0x7f0fc00b37c0 2023-07-14T11:53:51.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.024+0000 7f0fc5e91700 1 --2- 172.21.15.104:0/2031155664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0fa4066320 0x7f0fa40687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:51.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.024+0000 7f0fc5e91700 1 --2- 172.21.15.104:0/2031155664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0fa4066320 0x7f0fa40687d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009b90 tx=0x7f0fbc009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:51.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.028+0000 7f0fb67fc700 1 -- 172.21.15.104:0/2031155664 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f0fb00520a0 con 0x7f0fc00b37c0 2023-07-14T11:53:51.180 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.196+0000 7f0fc80f5700 1 -- 172.21.15.104:0/2031155664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0fc004dcb0 con 0x7f0fc00b37c0 2023-07-14T11:53:51.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.196+0000 7f0fb67fc700 1 -- 172.21.15.104:0/2031155664 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0fb00170a0 con 0x7f0fc00b37c0 2023-07-14T11:53:51.181 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:53:51.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.200+0000 7f0fc80f5700 1 -- 172.21.15.104:0/2031155664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0fa4066320 msgr2=0x7f0fa40687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:51.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.200+0000 7f0fc80f5700 1 --2- 172.21.15.104:0/2031155664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0fa4066320 0x7f0fa40687d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009b90 tx=0x7f0fbc009240 comp rx=0 tx=0).stop 2023-07-14T11:53:51.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.200+0000 7f0fc80f5700 1 -- 172.21.15.104:0/2031155664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fc00b37c0 msgr2=0x7f0fc00aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:51.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.200+0000 7f0fc80f5700 1 --2- 172.21.15.104:0/2031155664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fc00b37c0 0x7f0fc00aee20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c3c0 tx=0x7f0fb00012d0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.200+0000 7f0fc80f5700 1 -- 172.21.15.104:0/2031155664 shutdown_connections 2023-07-14T11:53:51.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.200+0000 7f0fc80f5700 1 --2- 172.21.15.104:0/2031155664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0fa4066320 0x7f0fa40687d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.200+0000 7f0fc80f5700 1 --2- 172.21.15.104:0/2031155664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fc00b37c0 0x7f0fc00aee20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.200+0000 7f0fc80f5700 1 --2- 172.21.15.104:0/2031155664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc00af360 0x7f0fc00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.200+0000 7f0fc80f5700 1 --2- 172.21.15.104:0/2031155664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fac004870 0x7f0fc00c47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.200+0000 7f0fc80f5700 1 -- 172.21.15.104:0/2031155664 >> 172.21.15.104:0/2031155664 conn(0x7f0fc00cad80 msgr2=0x7f0fc00cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:51.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.200+0000 7f0fc80f5700 1 -- 172.21.15.104:0/2031155664 shutdown_connections 2023-07-14T11:53:51.186 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.200+0000 7f0fc80f5700 1 -- 172.21.15.104:0/2031155664 wait complete. 2023-07-14T11:53:51.207 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:53:51.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.392+0000 7fb50305a700 1 Processor -- start 2023-07-14T11:53:51.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.392+0000 7fb50305a700 1 -- start start 2023-07-14T11:53:51.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.392+0000 7fb50305a700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb4fc0c9d90 0x7fb4fc0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:51.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.392+0000 7fb50305a700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb4fc159f20 0x7fb4fc0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:51.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.392+0000 7fb50305a700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4fc0b0ef0 con 0x7fb4fc0caf70 2023-07-14T11:53:51.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.392+0000 7fb50305a700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4fc0ce9c0 con 0x7fb4fc159f20 2023-07-14T11:53:51.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.392+0000 7fb50305a700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4fc0cee50 con 0x7fb4fc0c9d90 2023-07-14T11:53:51.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb502859700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb4fc0caf70 0x7fb4fc0c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:48412/0 (socket says 172.21.15.104:48412) 2023-07-14T11:53:51.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb501857700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb4fc159f20 0x7fb4fc0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:51.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb502859700 1 -- 172.21.15.104:0/805862292 learned_addr learned my addr 172.21.15.104:0/805862292 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:51.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb502058700 1 --2- 172.21.15.104:0/805862292 >> v2:172.21.15.121:3300/0 conn(0x7fb4fc0c9d90 0x7fb4fc0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:51.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb501857700 1 -- 172.21.15.104:0/805862292 >> v2:172.21.15.121:3300/0 conn(0x7fb4fc0c9d90 msgr2=0x7fb4fc0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:51.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb501857700 1 --2- 172.21.15.104:0/805862292 >> v2:172.21.15.121:3300/0 conn(0x7fb4fc0c9d90 0x7fb4fc0ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb501857700 1 -- 172.21.15.104:0/805862292 >> v1:172.21.15.104:6789/0 conn(0x7fb4fc0caf70 legacy=0x7fb4fc0c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:51.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb501857700 1 -- 172.21.15.104:0/805862292 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc0ca6a0 con 0x7fb4fc159f20 2023-07-14T11:53:51.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb501056700 1 -- 172.21.15.104:0/805862292 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794137990 0 0) 0x7fb4fc0b0ef0 con 0x7fb4fc0caf70 2023-07-14T11:53:51.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb501857700 1 --2- 172.21.15.104:0/805862292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4fc159f20 0x7fb4fc0caa30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0cdf30 tx=0x7fb4ec009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f0bd79668e5a0b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:51.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb501056700 1 -- 172.21.15.104:0/805862292 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec011830 con 0x7fb4fc159f20 2023-07-14T11:53:51.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb501056700 1 -- 172.21.15.104:0/805862292 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec011990 con 0x7fb4fc159f20 2023-07-14T11:53:51.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb501056700 1 -- 172.21.15.104:0/805862292 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec011d40 con 0x7fb4fc159f20 2023-07-14T11:53:51.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb50305a700 1 -- 172.21.15.104:0/805862292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4fc159f20 msgr2=0x7fb4fc0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:51.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb50305a700 1 --2- 172.21.15.104:0/805862292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4fc159f20 0x7fb4fc0caa30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0cdf30 tx=0x7fb4ec009b30 comp rx=0 tx=0).stop 2023-07-14T11:53:51.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb50305a700 1 -- 172.21.15.104:0/805862292 shutdown_connections 2023-07-14T11:53:51.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb50305a700 1 --2- 172.21.15.104:0/805862292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4fc159f20 0x7fb4fc0caa30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb50305a700 1 --2- 172.21.15.104:0/805862292 >> v2:172.21.15.121:3300/0 conn(0x7fb4fc0c9d90 0x7fb4fc0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb50305a700 1 -- 172.21.15.104:0/805862292 >> 172.21.15.104:0/805862292 conn(0x7fb4fc06e960 msgr2=0x7fb4fc06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:51.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb50305a700 1 -- 172.21.15.104:0/805862292 shutdown_connections 2023-07-14T11:53:51.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.396+0000 7fb50305a700 1 -- 172.21.15.104:0/805862292 wait complete. 2023-07-14T11:53:51.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.400+0000 7fb50305a700 1 Processor -- start 2023-07-14T11:53:51.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.400+0000 7fb50305a700 1 -- start start 2023-07-14T11:53:51.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.400+0000 7fb50305a700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4fc0c9d90 0x7fb4fc14e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:51.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.400+0000 7fb50305a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4fc0caf70 0x7fb4fc15b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:51.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.400+0000 7fb50305a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4fc159f20 0x7fb4fc15be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:51.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.400+0000 7fb50305a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0c3e00 con 0x7fb4fc159f20 2023-07-14T11:53:51.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.400+0000 7fb50305a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0c40e0 con 0x7fb4fc0caf70 2023-07-14T11:53:51.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.400+0000 7fb50305a700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0c43c0 con 0x7fb4fc0c9d90 2023-07-14T11:53:51.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb502859700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4fc159f20 0x7fb4fc15be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:51.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb502859700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4fc159f20 0x7fb4fc15be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38448/0 (socket says 172.21.15.104:38448) 2023-07-14T11:53:51.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb502859700 1 -- 172.21.15.104:0/28908013 learned_addr learned my addr 172.21.15.104:0/28908013 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:51.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb501857700 1 --2- 172.21.15.104:0/28908013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4fc0caf70 0x7fb4fc15b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:51.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb502058700 1 --2- 172.21.15.104:0/28908013 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4fc0c9d90 0x7fb4fc14e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:51.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb502859700 1 -- 172.21.15.104:0/28908013 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4fc0c9d90 msgr2=0x7fb4fc14e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:51.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb502859700 1 --2- 172.21.15.104:0/28908013 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4fc0c9d90 0x7fb4fc14e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb502859700 1 -- 172.21.15.104:0/28908013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4fc0caf70 msgr2=0x7fb4fc15b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:51.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb502859700 1 --2- 172.21.15.104:0/28908013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4fc0caf70 0x7fb4fc15b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb502859700 1 -- 172.21.15.104:0/28908013 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ec0094b0 con 0x7fb4fc159f20 2023-07-14T11:53:51.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb502859700 1 --2- 172.21.15.104:0/28908013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4fc159f20 0x7fb4fc15be20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4002230 tx=0x7fb4f400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:51.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb4f2ffd700 1 -- 172.21.15.104:0/28908013 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f400aac0 con 0x7fb4fc159f20 2023-07-14T11:53:51.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb4f2ffd700 1 -- 172.21.15.104:0/28908013 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f400ac40 con 0x7fb4fc159f20 2023-07-14T11:53:51.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb4f2ffd700 1 -- 172.21.15.104:0/28908013 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40070d0 con 0x7fb4fc159f20 2023-07-14T11:53:51.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb50305a700 1 -- 172.21.15.104:0/28908013 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4fc0c46a0 con 0x7fb4fc159f20 2023-07-14T11:53:51.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb50305a700 1 -- 172.21.15.104:0/28908013 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc14ecd0 con 0x7fb4fc159f20 2023-07-14T11:53:51.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.404+0000 7fb50305a700 1 -- 172.21.15.104:0/28908013 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4fc04e500 con 0x7fb4fc159f20 2023-07-14T11:53:51.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.408+0000 7fb4f2ffd700 1 -- 172.21.15.104:0/28908013 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb4f4007250 con 0x7fb4fc159f20 2023-07-14T11:53:51.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.408+0000 7fb4f2ffd700 1 --2- 172.21.15.104:0/28908013 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb4e00662f0 0x7fb4e00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:51.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.408+0000 7fb4f2ffd700 1 -- 172.21.15.104:0/28908013 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb4f4083a00 con 0x7fb4fc159f20 2023-07-14T11:53:51.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.408+0000 7fb502058700 1 --2- 172.21.15.104:0/28908013 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb4e00662f0 0x7fb4e00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:51.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.412+0000 7fb502058700 1 --2- 172.21.15.104:0/28908013 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb4e00662f0 0x7fb4e00687a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8007a60 tx=0x7fb4f8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:51.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.412+0000 7fb4f2ffd700 1 -- 172.21.15.104:0/28908013 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb4f4050b80 con 0x7fb4fc159f20 2023-07-14T11:53:51.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.624+0000 7fb50305a700 1 -- 172.21.15.104:0/28908013 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4fc070c90 con 0x7fb4fc159f20 2023-07-14T11:53:51.608 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.624+0000 7fb4f2ffd700 1 -- 172.21.15.104:0/28908013 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4717 (secure 0 0 0) 0x7fb4f4081e60 con 0x7fb4fc159f20 2023-07-14T11:53:51.608 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:51.608 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2103358277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2103358277},{"type":"v1","addr":"172.21.15.121:6833","nonce":2103358277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1841065287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1841065287},{"type":"v1","addr":"172.21.15.121:6835","nonce":1841065287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3342053518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3342053518},{"type":"v1","addr":"172.21.15.104:6835","nonce":3342053518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:53:15.029045+0000","modified":"2023-07-14T11:53:41.785898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/2666329866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2666329866},{"type":"v1","addr":"172.21.15.104:6837","nonce":2666329866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:53:51.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.624+0000 7fb50305a700 1 -- 172.21.15.104:0/28908013 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb4e00662f0 msgr2=0x7fb4e00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:51.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.628+0000 7fb50305a700 1 --2- 172.21.15.104:0/28908013 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb4e00662f0 0x7fb4e00687a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8007a60 tx=0x7fb4f8008040 comp rx=0 tx=0).stop 2023-07-14T11:53:51.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.628+0000 7fb50305a700 1 -- 172.21.15.104:0/28908013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4fc159f20 msgr2=0x7fb4fc15be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:51.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.628+0000 7fb50305a700 1 --2- 172.21.15.104:0/28908013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4fc159f20 0x7fb4fc15be20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4002230 tx=0x7fb4f400cf50 comp rx=0 tx=0).stop 2023-07-14T11:53:51.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.628+0000 7fb50305a700 1 -- 172.21.15.104:0/28908013 shutdown_connections 2023-07-14T11:53:51.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.628+0000 7fb50305a700 1 --2- 172.21.15.104:0/28908013 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb4e00662f0 0x7fb4e00687a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.628+0000 7fb50305a700 1 --2- 172.21.15.104:0/28908013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4fc159f20 0x7fb4fc15be20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.628+0000 7fb50305a700 1 --2- 172.21.15.104:0/28908013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4fc0caf70 0x7fb4fc15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.628+0000 7fb50305a700 1 --2- 172.21.15.104:0/28908013 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4fc0c9d90 0x7fb4fc14e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.628+0000 7fb50305a700 1 -- 172.21.15.104:0/28908013 >> 172.21.15.104:0/28908013 conn(0x7fb4fc06e960 msgr2=0x7fb4fc06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:51.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.628+0000 7fb50305a700 1 -- 172.21.15.104:0/28908013 shutdown_connections 2023-07-14T11:53:51.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.628+0000 7fb50305a700 1 -- 172.21.15.104:0/28908013 wait complete. 2023-07-14T11:53:51.613 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 9 2023-07-14T11:53:51.631 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:51.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.820+0000 7f68bdd9e700 1 Processor -- start 2023-07-14T11:53:51.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.820+0000 7f68bdd9e700 1 -- start start 2023-07-14T11:53:51.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68bdd9e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f68b80b0de0 0x7f68b81669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:51.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68bdd9e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b80af0a0 con 0x7f68b815bff0 2023-07-14T11:53:51.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68bdd9e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b80cf720 con 0x7f68b80cd240 2023-07-14T11:53:51.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68bdd9e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b80af2f0 con 0x7f68b80b0de0 2023-07-14T11:53:51.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68bd59d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f68b80b0de0 0x7f68b81669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:51.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68bd59d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f68b80b0de0 0x7f68b81669d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38464/0 (socket says 172.21.15.104:38464) 2023-07-14T11:53:51.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68bd59d700 1 -- 172.21.15.104:0/472678597 learned_addr learned my addr 172.21.15.104:0/472678597 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:51.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68bd59d700 1 -- 172.21.15.104:0/472678597 >> v1:172.21.15.121:6789/0 conn(0x7f68b80cd240 legacy=0x7f68b80ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:51.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68bd59d700 1 -- 172.21.15.104:0/472678597 >> v1:172.21.15.104:6789/0 conn(0x7f68b815bff0 legacy=0x7f68b80ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:51.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68bd59d700 1 -- 172.21.15.104:0/472678597 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b80af5d0 con 0x7f68b80b0de0 2023-07-14T11:53:51.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68bd59d700 1 --2- 172.21.15.104:0/472678597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68b80b0de0 0x7f68b81669d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009e50 tx=0x7f68b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56636015d2e667a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:51.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68b77fe700 1 -- 172.21.15.104:0/472678597 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b000e070 con 0x7f68b80b0de0 2023-07-14T11:53:51.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68b77fe700 1 -- 172.21.15.104:0/472678597 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b000b9c0 con 0x7f68b80b0de0 2023-07-14T11:53:51.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68b77fe700 1 -- 172.21.15.104:0/472678597 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b000bd70 con 0x7f68b80b0de0 2023-07-14T11:53:51.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68bdd9e700 1 -- 172.21.15.104:0/472678597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68b80b0de0 msgr2=0x7f68b81669d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:51.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.824+0000 7f68bdd9e700 1 --2- 172.21.15.104:0/472678597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68b80b0de0 0x7f68b81669d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009e50 tx=0x7f68b000b330 comp rx=0 tx=0).stop 2023-07-14T11:53:51.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.828+0000 7f68bdd9e700 1 -- 172.21.15.104:0/472678597 shutdown_connections 2023-07-14T11:53:51.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.828+0000 7f68bdd9e700 1 --2- 172.21.15.104:0/472678597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68b80b0de0 0x7f68b81669d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.828+0000 7f68bdd9e700 1 -- 172.21.15.104:0/472678597 >> 172.21.15.104:0/472678597 conn(0x7f68b8060c40 msgr2=0x7f68b809b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:51.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.828+0000 7f68bdd9e700 1 -- 172.21.15.104:0/472678597 shutdown_connections 2023-07-14T11:53:51.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.828+0000 7f68bdd9e700 1 -- 172.21.15.104:0/472678597 wait complete. 2023-07-14T11:53:51.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.828+0000 7f68bdd9e700 1 Processor -- start 2023-07-14T11:53:51.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bdd9e700 1 -- start start 2023-07-14T11:53:51.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bdd9e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68b80b0de0 0x7f68b80b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:51.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bdd9e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68b80cd240 0x7f68b80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:51.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bdd9e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68b815bff0 0x7f68b81505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:51.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bdd9e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b80b2910 con 0x7f68b815bff0 2023-07-14T11:53:51.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bdd9e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b80b2090 con 0x7f68b80cd240 2023-07-14T11:53:51.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bdd9e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b80b2bb0 con 0x7f68b80b0de0 2023-07-14T11:53:51.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bd59d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68b815bff0 0x7f68b81505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:51.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bd59d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68b815bff0 0x7f68b81505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38468/0 (socket says 172.21.15.104:38468) 2023-07-14T11:53:51.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bd59d700 1 -- 172.21.15.104:0/1724117248 learned_addr learned my addr 172.21.15.104:0/1724117248 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:51.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bcd9c700 1 --2- 172.21.15.104:0/1724117248 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68b80b0de0 0x7f68b80b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:51.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68b7fff700 1 --2- 172.21.15.104:0/1724117248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68b80cd240 0x7f68b80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:51.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bd59d700 1 -- 172.21.15.104:0/1724117248 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68b80b0de0 msgr2=0x7f68b80b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:51.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bd59d700 1 --2- 172.21.15.104:0/1724117248 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68b80b0de0 0x7f68b80b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bd59d700 1 -- 172.21.15.104:0/1724117248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68b80cd240 msgr2=0x7f68b80706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:51.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bd59d700 1 --2- 172.21.15.104:0/1724117248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68b80cd240 0x7f68b80706b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:51.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.832+0000 7f68bd59d700 1 -- 172.21.15.104:0/1724117248 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b0009b30 con 0x7f68b815bff0 2023-07-14T11:53:51.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.836+0000 7f68bd59d700 1 --2- 172.21.15.104:0/1724117248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68b815bff0 0x7f68b81505c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f68b80b50d0 tx=0x7f68b0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:51.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.836+0000 7f68b57fa700 1 -- 172.21.15.104:0/1724117248 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b0004800 con 0x7f68b815bff0 2023-07-14T11:53:51.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.836+0000 7f68b57fa700 1 -- 172.21.15.104:0/1724117248 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b00099b0 con 0x7f68b815bff0 2023-07-14T11:53:51.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.836+0000 7f68b57fa700 1 -- 172.21.15.104:0/1724117248 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b001c680 con 0x7f68b815bff0 2023-07-14T11:53:51.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.836+0000 7f68bdd9e700 1 -- 172.21.15.104:0/1724117248 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b80b2e90 con 0x7f68b815bff0 2023-07-14T11:53:51.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.836+0000 7f68bdd9e700 1 -- 172.21.15.104:0/1724117248 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b8070bf0 con 0x7f68b815bff0 2023-07-14T11:53:51.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.836+0000 7f68bdd9e700 1 -- 172.21.15.104:0/1724117248 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b80cf180 con 0x7f68b815bff0 2023-07-14T11:53:51.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.840+0000 7f68b57fa700 1 -- 172.21.15.104:0/1724117248 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f68b001c7e0 con 0x7f68b815bff0 2023-07-14T11:53:51.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.840+0000 7f68b57fa700 1 --2- 172.21.15.104:0/1724117248 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f689c0662f0 0x7f689c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:51.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.840+0000 7f68b57fa700 1 -- 172.21.15.104:0/1724117248 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f68b0085cb0 con 0x7f68b815bff0 2023-07-14T11:53:51.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.840+0000 7f68bcd9c700 1 --2- 172.21.15.104:0/1724117248 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f689c0662f0 0x7f689c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:51.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.840+0000 7f68bcd9c700 1 --2- 172.21.15.104:0/1724117248 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f689c0662f0 0x7f689c0687a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f68b80cca70 tx=0x7f68ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:51.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:51.848+0000 7f68b57fa700 1 -- 172.21.15.104:0/1724117248 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f68b0052ed0 con 0x7f68b815bff0 2023-07-14T11:53:52.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.020+0000 7f68bdd9e700 1 -- 172.21.15.104:0/1724117248 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68b804e500 con 0x7f68b815bff0 2023-07-14T11:53:52.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.024+0000 7f68b57fa700 1 -- 172.21.15.104:0/1724117248 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20156 (secure 0 0 0) 0x7f68b00564f0 con 0x7f68b815bff0 2023-07-14T11:53:52.008 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:52.009 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":28,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:20.091434+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:52.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.024+0000 7f68bdd9e700 1 -- 172.21.15.104:0/1724117248 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f689c0662f0 msgr2=0x7f689c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:52.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.024+0000 7f68bdd9e700 1 --2- 172.21.15.104:0/1724117248 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f689c0662f0 0x7f689c0687a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f68b80cca70 tx=0x7f68ac009040 comp rx=0 tx=0).stop 2023-07-14T11:53:52.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.028+0000 7f68bdd9e700 1 -- 172.21.15.104:0/1724117248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68b815bff0 msgr2=0x7f68b81505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:52.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.028+0000 7f68bdd9e700 1 --2- 172.21.15.104:0/1724117248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68b815bff0 0x7f68b81505c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f68b80b50d0 tx=0x7f68b0004af0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.028+0000 7f68bdd9e700 1 -- 172.21.15.104:0/1724117248 shutdown_connections 2023-07-14T11:53:52.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.028+0000 7f68bdd9e700 1 --2- 172.21.15.104:0/1724117248 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f689c0662f0 0x7f689c0687a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.028+0000 7f68bdd9e700 1 --2- 172.21.15.104:0/1724117248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68b815bff0 0x7f68b81505c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.028+0000 7f68bdd9e700 1 --2- 172.21.15.104:0/1724117248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68b80cd240 0x7f68b80706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.028+0000 7f68bdd9e700 1 --2- 172.21.15.104:0/1724117248 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68b80b0de0 0x7f68b80b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.028+0000 7f68bdd9e700 1 -- 172.21.15.104:0/1724117248 >> 172.21.15.104:0/1724117248 conn(0x7f68b8060c40 msgr2=0x7f68b80d2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:52.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.028+0000 7f68bdd9e700 1 -- 172.21.15.104:0/1724117248 shutdown_connections 2023-07-14T11:53:52.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.028+0000 7f68bdd9e700 1 -- 172.21.15.104:0/1724117248 wait complete. 2023-07-14T11:53:52.032 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:53:52.033 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:53:52.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.232+0000 7fdb8df9e700 1 Processor -- start 2023-07-14T11:53:52.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.232+0000 7fdb8df9e700 1 -- start start 2023-07-14T11:53:52.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb8df9e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdb880cd260 0x7fdb880d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:52.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb8df9e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb880d0610 0x7fdb880af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:52.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb8df9e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb880d1a60 con 0x7fdb880b0ee0 2023-07-14T11:53:52.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb8df9e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb880d1ef0 con 0x7fdb880cd260 2023-07-14T11:53:52.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb8df9e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb880d0e50 con 0x7fdb880d0610 2023-07-14T11:53:52.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb87fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdb880cd260 0x7fdb880d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:52.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb87fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdb880cd260 0x7fdb880d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38470/0 (socket says 172.21.15.104:38470) 2023-07-14T11:53:52.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb87fff700 1 -- 172.21.15.104:0/3865556660 learned_addr learned my addr 172.21.15.104:0/3865556660 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:52.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb8d79d700 1 --2- 172.21.15.104:0/3865556660 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb880d0610 0x7fdb880af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:52.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb87fff700 1 -- 172.21.15.104:0/3865556660 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb880d0610 msgr2=0x7fdb880af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:52.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb87fff700 1 --2- 172.21.15.104:0/3865556660 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb880d0610 0x7fdb880af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb87fff700 1 -- 172.21.15.104:0/3865556660 >> v1:172.21.15.121:6789/0 conn(0x7fdb880b0ee0 legacy=0x7fdb880ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:52.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb87fff700 1 -- 172.21.15.104:0/3865556660 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb880d1130 con 0x7fdb880cd260 2023-07-14T11:53:52.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb87fff700 1 --2- 172.21.15.104:0/3865556660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb880cd260 0x7fdb880d00d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009910 tx=0x7fdb74009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158024e849edfc6a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:52.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb877fe700 1 -- 172.21.15.104:0/3865556660 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7400ba60 con 0x7fdb880cd260 2023-07-14T11:53:52.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb877fe700 1 -- 172.21.15.104:0/3865556660 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7400bbc0 con 0x7fdb880cd260 2023-07-14T11:53:52.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb877fe700 1 -- 172.21.15.104:0/3865556660 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb74009e60 con 0x7fdb880cd260 2023-07-14T11:53:52.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb8df9e700 1 -- 172.21.15.104:0/3865556660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb880cd260 msgr2=0x7fdb880d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:52.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb8df9e700 1 --2- 172.21.15.104:0/3865556660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb880cd260 0x7fdb880d00d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009910 tx=0x7fdb74009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:52.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb8df9e700 1 -- 172.21.15.104:0/3865556660 shutdown_connections 2023-07-14T11:53:52.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb8df9e700 1 --2- 172.21.15.104:0/3865556660 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb880d0610 0x7fdb880af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb8df9e700 1 --2- 172.21.15.104:0/3865556660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb880cd260 0x7fdb880d00d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb8df9e700 1 -- 172.21.15.104:0/3865556660 >> 172.21.15.104:0/3865556660 conn(0x7fdb88060c40 msgr2=0x7fdb8809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:52.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb8df9e700 1 -- 172.21.15.104:0/3865556660 shutdown_connections 2023-07-14T11:53:52.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.236+0000 7fdb8df9e700 1 -- 172.21.15.104:0/3865556660 wait complete. 2023-07-14T11:53:52.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.240+0000 7fdb8df9e700 1 Processor -- start 2023-07-14T11:53:52.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.240+0000 7fdb8df9e700 1 -- start start 2023-07-14T11:53:52.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.240+0000 7fdb8df9e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb880b0ee0 0x7fdb880c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:52.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.240+0000 7fdb8df9e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb880cd260 0x7fdb8815b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:52.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.240+0000 7fdb8df9e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb880d0610 0x7fdb881f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:52.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.240+0000 7fdb8df9e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb880b1af0 con 0x7fdb880b0ee0 2023-07-14T11:53:52.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.240+0000 7fdb8df9e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb880b2070 con 0x7fdb880d0610 2023-07-14T11:53:52.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.240+0000 7fdb8df9e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb880c6550 con 0x7fdb880cd260 2023-07-14T11:53:52.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.240+0000 7fdb8cf9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb880b0ee0 0x7fdb880c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:52.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.240+0000 7fdb8d79d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb880d0610 0x7fdb881f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:52.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.240+0000 7fdb8d79d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb880d0610 0x7fdb881f14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:36884/0 (socket says 172.21.15.104:36884) 2023-07-14T11:53:52.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.240+0000 7fdb8cf9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb880b0ee0 0x7fdb880c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38478/0 (socket says 172.21.15.104:38478) 2023-07-14T11:53:52.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.240+0000 7fdb8d79d700 1 -- 172.21.15.104:0/495634873 learned_addr learned my addr 172.21.15.104:0/495634873 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:52.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.244+0000 7fdb87fff700 1 --2- 172.21.15.104:0/495634873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb880cd260 0x7fdb8815b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:52.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.244+0000 7fdb8d79d700 1 -- 172.21.15.104:0/495634873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb880cd260 msgr2=0x7fdb8815b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:52.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.244+0000 7fdb8d79d700 1 --2- 172.21.15.104:0/495634873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb880cd260 0x7fdb8815b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.244+0000 7fdb8d79d700 1 -- 172.21.15.104:0/495634873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb880b0ee0 msgr2=0x7fdb880c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:52.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.244+0000 7fdb8d79d700 1 --2- 172.21.15.104:0/495634873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb880b0ee0 0x7fdb880c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.244+0000 7fdb8d79d700 1 -- 172.21.15.104:0/495634873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb74009570 con 0x7fdb880d0610 2023-07-14T11:53:52.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.244+0000 7fdb87fff700 1 --2- 172.21.15.104:0/495634873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb880cd260 0x7fdb8815b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:52.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.244+0000 7fdb8cf9c700 1 --2- 172.21.15.104:0/495634873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb880b0ee0 0x7fdb880c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:52.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.244+0000 7fdb8d79d700 1 --2- 172.21.15.104:0/495634873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb880d0610 0x7fdb881f14a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdb80007e90 tx=0x7fdb8000e3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:52.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.244+0000 7fdb857fa700 1 -- 172.21.15.104:0/495634873 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8000ede0 con 0x7fdb880d0610 2023-07-14T11:53:52.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.244+0000 7fdb857fa700 1 -- 172.21.15.104:0/495634873 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb80014420 con 0x7fdb880d0610 2023-07-14T11:53:52.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.244+0000 7fdb8df9e700 1 -- 172.21.15.104:0/495634873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb881f1fe0 con 0x7fdb880d0610 2023-07-14T11:53:52.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.244+0000 7fdb857fa700 1 -- 172.21.15.104:0/495634873 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb800147d0 con 0x7fdb880d0610 2023-07-14T11:53:52.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.244+0000 7fdb8df9e700 1 -- 172.21.15.104:0/495634873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb880b1d00 con 0x7fdb880d0610 2023-07-14T11:53:52.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.244+0000 7fdb8df9e700 1 -- 172.21.15.104:0/495634873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8804e500 con 0x7fdb880d0610 2023-07-14T11:53:52.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.248+0000 7fdb857fa700 1 -- 172.21.15.104:0/495634873 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fdb80014930 con 0x7fdb880d0610 2023-07-14T11:53:52.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.248+0000 7fdb857fa700 1 --2- 172.21.15.104:0/495634873 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdb6c0662f0 0x7fdb6c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:52.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.248+0000 7fdb857fa700 1 -- 172.21.15.104:0/495634873 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdb80084730 con 0x7fdb880d0610 2023-07-14T11:53:52.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.248+0000 7fdb8cf9c700 1 --2- 172.21.15.104:0/495634873 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdb6c0662f0 0x7fdb6c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:52.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.252+0000 7fdb8cf9c700 1 --2- 172.21.15.104:0/495634873 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdb6c0662f0 0x7fdb6c0687a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c000f60 tx=0x7fdb7c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:52.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.260+0000 7fdb857fa700 1 -- 172.21.15.104:0/495634873 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fdb80051820 con 0x7fdb880d0610 2023-07-14T11:53:52.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.436+0000 7fdb8df9e700 1 -- 172.21.15.104:0/495634873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb880b21b0 con 0x7fdb880d0610 2023-07-14T11:53:52.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.436+0000 7fdb857fa700 1 -- 172.21.15.104:0/495634873 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb80054e40 con 0x7fdb880d0610 2023-07-14T11:53:52.421 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:53:52.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.440+0000 7fdb8df9e700 1 -- 172.21.15.104:0/495634873 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdb6c0662f0 msgr2=0x7fdb6c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:52.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.440+0000 7fdb8df9e700 1 --2- 172.21.15.104:0/495634873 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdb6c0662f0 0x7fdb6c0687a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c000f60 tx=0x7fdb7c009040 comp rx=0 tx=0).stop 2023-07-14T11:53:52.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.440+0000 7fdb8df9e700 1 -- 172.21.15.104:0/495634873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb880d0610 msgr2=0x7fdb881f14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:52.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.440+0000 7fdb8df9e700 1 --2- 172.21.15.104:0/495634873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb880d0610 0x7fdb881f14a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdb80007e90 tx=0x7fdb8000e3c0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.440+0000 7fdb8df9e700 1 -- 172.21.15.104:0/495634873 shutdown_connections 2023-07-14T11:53:52.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.440+0000 7fdb8df9e700 1 --2- 172.21.15.104:0/495634873 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdb6c0662f0 0x7fdb6c0687a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.440+0000 7fdb8df9e700 1 --2- 172.21.15.104:0/495634873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb880d0610 0x7fdb881f14a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.440+0000 7fdb8df9e700 1 --2- 172.21.15.104:0/495634873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb880cd260 0x7fdb8815b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.440+0000 7fdb8df9e700 1 --2- 172.21.15.104:0/495634873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb880b0ee0 0x7fdb880c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.440+0000 7fdb8df9e700 1 -- 172.21.15.104:0/495634873 >> 172.21.15.104:0/495634873 conn(0x7fdb88060c40 msgr2=0x7fdb880b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:52.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.440+0000 7fdb8df9e700 1 -- 172.21.15.104:0/495634873 shutdown_connections 2023-07-14T11:53:52.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.440+0000 7fdb8df9e700 1 -- 172.21.15.104:0/495634873 wait complete. 2023-07-14T11:53:52.447 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:53:52.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.628+0000 7f6840f15700 1 Processor -- start 2023-07-14T11:53:52.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.628+0000 7f6840f15700 1 -- start start 2023-07-14T11:53:52.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f6840f15700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f683c160000 0x7f683c06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:52.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f6840f15700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683c0cdfb0 con 0x7f683c0cd7f0 2023-07-14T11:53:52.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f6840f15700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683c0cdcd0 con 0x7f683c1603d0 2023-07-14T11:53:52.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f6840f15700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683c070e40 con 0x7f683c160000 2023-07-14T11:53:52.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f683bfff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f683c0cd7f0 0x7f683c0c0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:48438/0 (socket says 172.21.15.104:48438) 2023-07-14T11:53:52.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f683bfff700 1 -- 172.21.15.104:0/2053370466 learned_addr learned my addr 172.21.15.104:0/2053370466 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:52.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f683b7fe700 1 --2- 172.21.15.104:0/2053370466 >> v2:172.21.15.121:3300/0 conn(0x7f683c160000 0x7f683c06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:52.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f683affd700 1 -- 172.21.15.104:0/2053370466 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4060312012 0 0) 0x7f6834001110 con 0x7f683c0cd7f0 2023-07-14T11:53:52.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f683affd700 1 -- 172.21.15.104:0/2053370466 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68280035d0 con 0x7f683c0cd7f0 2023-07-14T11:53:52.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f683b7fe700 1 -- 172.21.15.104:0/2053370466 >> v1:172.21.15.121:6789/0 conn(0x7f683c1603d0 legacy=0x7f683c0c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:52.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f683affd700 1 -- 172.21.15.104:0/2053370466 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1618464380 0 0) 0x7f68280035d0 con 0x7f683c0cd7f0 2023-07-14T11:53:52.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f683b7fe700 1 -- 172.21.15.104:0/2053370466 >> v1:172.21.15.104:6789/0 conn(0x7f683c0cd7f0 legacy=0x7f683c0c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:52.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f683b7fe700 1 -- 172.21.15.104:0/2053370466 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c070780 con 0x7f683c160000 2023-07-14T11:53:52.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f683b7fe700 1 --2- 172.21.15.104:0/2053370466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f683c160000 0x7f683c06dd10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f682c009850 tx=0x7f682c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70f24c7ca33010f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:52.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f683affd700 1 -- 172.21.15.104:0/2053370466 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834001800 con 0x7f683c160000 2023-07-14T11:53:52.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f683affd700 1 -- 172.21.15.104:0/2053370466 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c004030 con 0x7f683c160000 2023-07-14T11:53:52.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f683affd700 1 -- 172.21.15.104:0/2053370466 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c0043c0 con 0x7f683c160000 2023-07-14T11:53:52.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f6840f15700 1 -- 172.21.15.104:0/2053370466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f683c160000 msgr2=0x7f683c06dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:52.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.632+0000 7f6840f15700 1 --2- 172.21.15.104:0/2053370466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f683c160000 0x7f683c06dd10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f682c009850 tx=0x7f682c009b60 comp rx=0 tx=0).stop 2023-07-14T11:53:52.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.636+0000 7f6840f15700 1 -- 172.21.15.104:0/2053370466 shutdown_connections 2023-07-14T11:53:52.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.636+0000 7f6840f15700 1 --2- 172.21.15.104:0/2053370466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f683c160000 0x7f683c06dd10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.636+0000 7f6840f15700 1 -- 172.21.15.104:0/2053370466 >> 172.21.15.104:0/2053370466 conn(0x7f683c0afc60 msgr2=0x7f683c0b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:52.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.636+0000 7f6840f15700 1 -- 172.21.15.104:0/2053370466 shutdown_connections 2023-07-14T11:53:52.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.636+0000 7f6840f15700 1 -- 172.21.15.104:0/2053370466 wait complete. 2023-07-14T11:53:52.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.636+0000 7f6840f15700 1 Processor -- start 2023-07-14T11:53:52.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.636+0000 7f6840f15700 1 -- start start 2023-07-14T11:53:52.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6840f15700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f683c0cd7f0 0x7f683c154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:52.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6840f15700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c160000 0x7f683c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:52.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6840f15700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f683c1603d0 0x7f683c0b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:52.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6840f15700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c0c63b0 con 0x7f683c160000 2023-07-14T11:53:52.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6840f15700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c0c5f20 con 0x7f683c0cd7f0 2023-07-14T11:53:52.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6840f15700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c0c5350 con 0x7f683c1603d0 2023-07-14T11:53:52.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6832ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c160000 0x7f683c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:52.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6832ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c160000 0x7f683c0b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38492/0 (socket says 172.21.15.104:38492) 2023-07-14T11:53:52.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6832ffd700 1 -- 172.21.15.104:0/30201333 learned_addr learned my addr 172.21.15.104:0/30201333 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:52.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f683bfff700 1 --2- 172.21.15.104:0/30201333 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f683c1603d0 0x7f683c0b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:52.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f683b7fe700 1 --2- 172.21.15.104:0/30201333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f683c0cd7f0 0x7f683c154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:52.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6832ffd700 1 -- 172.21.15.104:0/30201333 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f683c1603d0 msgr2=0x7f683c0b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:52.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6832ffd700 1 --2- 172.21.15.104:0/30201333 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f683c1603d0 0x7f683c0b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6832ffd700 1 -- 172.21.15.104:0/30201333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f683c0cd7f0 msgr2=0x7f683c154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:52.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6832ffd700 1 --2- 172.21.15.104:0/30201333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f683c0cd7f0 0x7f683c154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6832ffd700 1 -- 172.21.15.104:0/30201333 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c0094b0 con 0x7f683c160000 2023-07-14T11:53:52.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f683b7fe700 1 --2- 172.21.15.104:0/30201333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f683c0cd7f0 0x7f683c154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:52.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6832ffd700 1 --2- 172.21.15.104:0/30201333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c160000 0x7f683c0b3520 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f682400cbc0 tx=0x7f682400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:52.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6838ff9700 1 -- 172.21.15.104:0/30201333 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6824003c20 con 0x7f683c160000 2023-07-14T11:53:52.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6838ff9700 1 -- 172.21.15.104:0/30201333 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6824003d80 con 0x7f683c160000 2023-07-14T11:53:52.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6838ff9700 1 -- 172.21.15.104:0/30201333 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682400ab00 con 0x7f683c160000 2023-07-14T11:53:52.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6840f15700 1 -- 172.21.15.104:0/30201333 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683c154bc0 con 0x7f683c160000 2023-07-14T11:53:52.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.640+0000 7f6840f15700 1 -- 172.21.15.104:0/30201333 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683c0c58d0 con 0x7f683c160000 2023-07-14T11:53:52.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.644+0000 7f6840f15700 1 -- 172.21.15.104:0/30201333 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683c071240 con 0x7f683c160000 2023-07-14T11:53:52.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.644+0000 7f6838ff9700 1 -- 172.21.15.104:0/30201333 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f682400ac60 con 0x7f683c160000 2023-07-14T11:53:52.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.648+0000 7f6838ff9700 1 --2- 172.21.15.104:0/30201333 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f681c0662f0 0x7f681c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:52.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.648+0000 7f6838ff9700 1 -- 172.21.15.104:0/30201333 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6824083fc0 con 0x7f683c160000 2023-07-14T11:53:52.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.648+0000 7f683b7fe700 1 --2- 172.21.15.104:0/30201333 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f681c0662f0 0x7f681c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:52.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.648+0000 7f683b7fe700 1 --2- 172.21.15.104:0/30201333 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f681c0662f0 0x7f681c0687a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f682c0090b0 tx=0x7f682c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:52.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.652+0000 7f6838ff9700 1 -- 172.21.15.104:0/30201333 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f683c071240 con 0x7f683c160000 2023-07-14T11:53:52.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.824+0000 7f6840f15700 1 -- 172.21.15.104:0/30201333 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f683c071240 con 0x7f683c160000 2023-07-14T11:53:52.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.824+0000 7f6838ff9700 1 -- 172.21.15.104:0/30201333 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7f683c071240 con 0x7f683c160000 2023-07-14T11:53:52.808 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:53:52.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.828+0000 7f6840f15700 1 -- 172.21.15.104:0/30201333 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f681c0662f0 msgr2=0x7f681c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:52.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.828+0000 7f6840f15700 1 --2- 172.21.15.104:0/30201333 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f681c0662f0 0x7f681c0687a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f682c0090b0 tx=0x7f682c008030 comp rx=0 tx=0).stop 2023-07-14T11:53:52.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.828+0000 7f6840f15700 1 -- 172.21.15.104:0/30201333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c160000 msgr2=0x7f683c0b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:52.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.828+0000 7f6840f15700 1 --2- 172.21.15.104:0/30201333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c160000 0x7f683c0b3520 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f682400cbc0 tx=0x7f682400ced0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.828+0000 7f6840f15700 1 -- 172.21.15.104:0/30201333 shutdown_connections 2023-07-14T11:53:52.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.828+0000 7f6840f15700 1 --2- 172.21.15.104:0/30201333 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f681c0662f0 0x7f681c0687a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.828+0000 7f6840f15700 1 --2- 172.21.15.104:0/30201333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c160000 0x7f683c0b3520 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.828+0000 7f6840f15700 1 --2- 172.21.15.104:0/30201333 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f683c1603d0 0x7f683c0b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.828+0000 7f6840f15700 1 --2- 172.21.15.104:0/30201333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f683c0cd7f0 0x7f683c154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:52.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.828+0000 7f6840f15700 1 -- 172.21.15.104:0/30201333 >> 172.21.15.104:0/30201333 conn(0x7f683c0afc60 msgr2=0x7f683c0aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:52.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.828+0000 7f6840f15700 1 -- 172.21.15.104:0/30201333 shutdown_connections 2023-07-14T11:53:52.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:52.828+0000 7f6840f15700 1 -- 172.21.15.104:0/30201333 wait complete. 2023-07-14T11:53:52.833 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:53:53.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.020+0000 7f4bfd319700 1 Processor -- start 2023-07-14T11:53:53.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.020+0000 7f4bfd319700 1 -- start start 2023-07-14T11:53:53.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.024+0000 7f4bfd319700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4bf80cd7f0 0x7f4bf806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.024+0000 7f4bfd319700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf80cdfb0 con 0x7f4bf81603d0 2023-07-14T11:53:53.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.024+0000 7f4bfd319700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf80cdcd0 con 0x7f4bf8160000 2023-07-14T11:53:53.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.024+0000 7f4bfd319700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf8070e40 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.024+0000 7f4bf67fc700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f4bf81603d0 0x7f4bf80c0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:48444/0 (socket says 172.21.15.104:48444) 2023-07-14T11:53:53.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.024+0000 7f4bf67fc700 1 -- 172.21.15.104:0/286681260 learned_addr learned my addr 172.21.15.104:0/286681260 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:53.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.024+0000 7f4bf77fe700 1 --2- 172.21.15.104:0/286681260 >> v2:172.21.15.121:3300/0 conn(0x7f4bf80cd7f0 0x7f4bf806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.024+0000 7f4bf5ffb700 1 -- 172.21.15.104:0/286681260 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181826694 0 0) 0x7f4bec001110 con 0x7f4bf81603d0 2023-07-14T11:53:53.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.024+0000 7f4bf5ffb700 1 -- 172.21.15.104:0/286681260 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4be00035d0 con 0x7f4bf81603d0 2023-07-14T11:53:53.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.024+0000 7f4bf77fe700 1 -- 172.21.15.104:0/286681260 >> v1:172.21.15.121:6789/0 conn(0x7f4bf8160000 legacy=0x7f4bf80c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.024+0000 7f4bf77fe700 1 -- 172.21.15.104:0/286681260 >> v1:172.21.15.104:6789/0 conn(0x7f4bf81603d0 legacy=0x7f4bf80c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.024+0000 7f4bf77fe700 1 -- 172.21.15.104:0/286681260 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf8070780 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.024+0000 7f4bf77fe700 1 --2- 172.21.15.104:0/286681260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf80cd7f0 0x7f4bf806dd10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009d90 tx=0x7f4bf000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8af794b3dcab721f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:53.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.028+0000 7f4bf5ffb700 1 -- 172.21.15.104:0/286681260 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf000e070 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.028+0000 7f4bf5ffb700 1 -- 172.21.15.104:0/286681260 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf000bb20 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.028+0000 7f4bf5ffb700 1 -- 172.21.15.104:0/286681260 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0012430 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.028+0000 7f4bfd319700 1 -- 172.21.15.104:0/286681260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf80cd7f0 msgr2=0x7f4bf806dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.028+0000 7f4bfd319700 1 --2- 172.21.15.104:0/286681260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf80cd7f0 0x7f4bf806dd10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009d90 tx=0x7f4bf000b330 comp rx=0 tx=0).stop 2023-07-14T11:53:53.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.028+0000 7f4bfd319700 1 -- 172.21.15.104:0/286681260 shutdown_connections 2023-07-14T11:53:53.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.028+0000 7f4bfd319700 1 --2- 172.21.15.104:0/286681260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf80cd7f0 0x7f4bf806dd10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.028+0000 7f4bfd319700 1 -- 172.21.15.104:0/286681260 >> 172.21.15.104:0/286681260 conn(0x7f4bf80afc60 msgr2=0x7f4bf80b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:53.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.028+0000 7f4bfd319700 1 -- 172.21.15.104:0/286681260 shutdown_connections 2023-07-14T11:53:53.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.028+0000 7f4bfd319700 1 -- 172.21.15.104:0/286681260 wait complete. 2023-07-14T11:53:53.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.028+0000 7f4bfd319700 1 Processor -- start 2023-07-14T11:53:53.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.028+0000 7f4bfd319700 1 -- start start 2023-07-14T11:53:53.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bfd319700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf80cd7f0 0x7f4bf81f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bfd319700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bf8160000 0x7f4bf81f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bfd319700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf81603d0 0x7f4bf81f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bfd319700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf81f6500 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bfd319700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf80b3490 con 0x7f4bf81603d0 2023-07-14T11:53:53.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bfd319700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf81f6bc0 con 0x7f4bf8160000 2023-07-14T11:53:53.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bf6ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf80cd7f0 0x7f4bf81f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bf6ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf80cd7f0 0x7f4bf81f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38502/0 (socket says 172.21.15.104:38502) 2023-07-14T11:53:53.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bf6ffd700 1 -- 172.21.15.104:0/2831172544 learned_addr learned my addr 172.21.15.104:0/2831172544 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:53.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bf77fe700 1 --2- 172.21.15.104:0/2831172544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf81603d0 0x7f4bf81f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bf67fc700 1 --2- 172.21.15.104:0/2831172544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bf8160000 0x7f4bf81f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bf6ffd700 1 -- 172.21.15.104:0/2831172544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bf8160000 msgr2=0x7f4bf81f3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bf6ffd700 1 --2- 172.21.15.104:0/2831172544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bf8160000 0x7f4bf81f3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bf6ffd700 1 -- 172.21.15.104:0/2831172544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf81603d0 msgr2=0x7f4bf81f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bf6ffd700 1 --2- 172.21.15.104:0/2831172544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf81603d0 0x7f4bf81f5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bf6ffd700 1 -- 172.21.15.104:0/2831172544 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf0009a70 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bf6ffd700 1 --2- 172.21.15.104:0/2831172544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf80cd7f0 0x7f4bf81f5070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4be800c500 tx=0x7f4be800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:53.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bdf7fe700 1 -- 172.21.15.104:0/2831172544 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be80052d0 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bdf7fe700 1 -- 172.21.15.104:0/2831172544 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be8005430 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bfd319700 1 -- 172.21.15.104:0/2831172544 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf81f6ea0 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bdf7fe700 1 -- 172.21.15.104:0/2831172544 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be80055a0 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.032+0000 7f4bfd319700 1 -- 172.21.15.104:0/2831172544 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf81f7460 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.020 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.036+0000 7f4bfd319700 1 -- 172.21.15.104:0/2831172544 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf804e500 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.020 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.036+0000 7f4bdf7fe700 1 -- 172.21.15.104:0/2831172544 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f4be80038e0 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.036+0000 7f4bdf7fe700 1 --2- 172.21.15.104:0/2831172544 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4be40662f0 0x7f4be40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.036+0000 7f4bdf7fe700 1 -- 172.21.15.104:0/2831172544 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4be80833f0 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.036+0000 7f4bf67fc700 1 --2- 172.21.15.104:0/2831172544 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4be40662f0 0x7f4be40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.040+0000 7f4bf67fc700 1 --2- 172.21.15.104:0/2831172544 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4be40662f0 0x7f4be40687a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4bec007fd0 tx=0x7f4bec007c60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:53.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.044+0000 7f4bdf7fe700 1 -- 172.21.15.104:0/2831172544 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f4be8050500 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.212+0000 7f4bfd319700 1 -- 172.21.15.104:0/2831172544 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4bf80b40d0 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.216+0000 7f4bdf7fe700 1 -- 172.21.15.104:0/2831172544 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4be8053b20 con 0x7f4bf80cd7f0 2023-07-14T11:53:53.199 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:53:53.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.216+0000 7f4bfd319700 1 -- 172.21.15.104:0/2831172544 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4be40662f0 msgr2=0x7f4be40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.216+0000 7f4bfd319700 1 --2- 172.21.15.104:0/2831172544 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4be40662f0 0x7f4be40687a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4bec007fd0 tx=0x7f4bec007c60 comp rx=0 tx=0).stop 2023-07-14T11:53:53.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.216+0000 7f4bfd319700 1 -- 172.21.15.104:0/2831172544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf80cd7f0 msgr2=0x7f4bf81f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.216+0000 7f4bfd319700 1 --2- 172.21.15.104:0/2831172544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf80cd7f0 0x7f4bf81f5070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4be800c500 tx=0x7f4be800c810 comp rx=0 tx=0).stop 2023-07-14T11:53:53.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.216+0000 7f4bfd319700 1 -- 172.21.15.104:0/2831172544 shutdown_connections 2023-07-14T11:53:53.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.216+0000 7f4bfd319700 1 --2- 172.21.15.104:0/2831172544 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4be40662f0 0x7f4be40687a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.216+0000 7f4bfd319700 1 --2- 172.21.15.104:0/2831172544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf81603d0 0x7f4bf81f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.216+0000 7f4bfd319700 1 --2- 172.21.15.104:0/2831172544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bf8160000 0x7f4bf81f3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.216+0000 7f4bfd319700 1 --2- 172.21.15.104:0/2831172544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bf80cd7f0 0x7f4bf81f5070 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.216+0000 7f4bfd319700 1 -- 172.21.15.104:0/2831172544 >> 172.21.15.104:0/2831172544 conn(0x7f4bf80afc60 msgr2=0x7f4bf80ca1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:53.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.216+0000 7f4bfd319700 1 -- 172.21.15.104:0/2831172544 shutdown_connections 2023-07-14T11:53:53.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.216+0000 7f4bfd319700 1 -- 172.21.15.104:0/2831172544 wait complete. 2023-07-14T11:53:53.226 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:53:53.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.416+0000 7f46b5cf2700 1 Processor -- start 2023-07-14T11:53:53.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.416+0000 7f46b5cf2700 1 -- start start 2023-07-14T11:53:53.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.416+0000 7f46b5cf2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f46b0158a20 0x7f46b00cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.416+0000 7f46b5cf2700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f46b006c220 0x7f46b00cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46b5cf2700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b006d310 con 0x7f46b00cb820 2023-07-14T11:53:53.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46b5cf2700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b006c700 con 0x7f46b006c220 2023-07-14T11:53:53.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46b5cf2700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b006c9e0 con 0x7f46b0158a20 2023-07-14T11:53:53.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46affff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f46b006c220 0x7f46b00cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46affff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f46b006c220 0x7f46b00cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38518/0 (socket says 172.21.15.104:38518) 2023-07-14T11:53:53.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46b4cf0700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f46b0158a20 0x7f46b00cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46affff700 1 -- 172.21.15.104:0/877848462 learned_addr learned my addr 172.21.15.104:0/877848462 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:53.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46affff700 1 -- 172.21.15.104:0/877848462 >> v2:172.21.15.121:3300/0 conn(0x7f46b0158a20 msgr2=0x7f46b00cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:53.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46affff700 1 --2- 172.21.15.104:0/877848462 >> v2:172.21.15.121:3300/0 conn(0x7f46b0158a20 0x7f46b00cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46affff700 1 -- 172.21.15.104:0/877848462 >> v1:172.21.15.121:6789/0 conn(0x7f46b00cb820 legacy=0x7f46b006d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46affff700 1 -- 172.21.15.104:0/877848462 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b006ccc0 con 0x7f46b006c220 2023-07-14T11:53:53.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46b4cf0700 1 --2- 172.21.15.104:0/877848462 >> v2:172.21.15.121:3300/0 conn(0x7f46b0158a20 0x7f46b00cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:53.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46affff700 1 --2- 172.21.15.104:0/877848462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b006c220 0x7f46b00cbd20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f46b006b0f0 tx=0x7f469c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e7d6d6abf489bc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:53.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46af7fe700 1 -- 172.21.15.104:0/877848462 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c00dd30 con 0x7f46b006c220 2023-07-14T11:53:53.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46af7fe700 1 -- 172.21.15.104:0/877848462 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469c00de90 con 0x7f46b006c220 2023-07-14T11:53:53.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46af7fe700 1 -- 172.21.15.104:0/877848462 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c010690 con 0x7f46b006c220 2023-07-14T11:53:53.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46b5cf2700 1 -- 172.21.15.104:0/877848462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b006c220 msgr2=0x7f46b00cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.420+0000 7f46b5cf2700 1 --2- 172.21.15.104:0/877848462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b006c220 0x7f46b00cbd20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f46b006b0f0 tx=0x7f469c00be90 comp rx=0 tx=0).stop 2023-07-14T11:53:53.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.424+0000 7f46b5cf2700 1 -- 172.21.15.104:0/877848462 shutdown_connections 2023-07-14T11:53:53.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.424+0000 7f46b5cf2700 1 --2- 172.21.15.104:0/877848462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b006c220 0x7f46b00cbd20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.424+0000 7f46b5cf2700 1 --2- 172.21.15.104:0/877848462 >> v2:172.21.15.121:3300/0 conn(0x7f46b0158a20 0x7f46b00cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.424+0000 7f46b5cf2700 1 -- 172.21.15.104:0/877848462 >> 172.21.15.104:0/877848462 conn(0x7f46b00afc60 msgr2=0x7f46b00b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:53.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.424+0000 7f46b5cf2700 1 -- 172.21.15.104:0/877848462 shutdown_connections 2023-07-14T11:53:53.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.424+0000 7f46b5cf2700 1 -- 172.21.15.104:0/877848462 wait complete. 2023-07-14T11:53:53.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.424+0000 7f46b5cf2700 1 Processor -- start 2023-07-14T11:53:53.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.424+0000 7f46b5cf2700 1 -- start start 2023-07-14T11:53:53.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46b5cf2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b006c220 0x7f46b014d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46b5cf2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b00cb820 0x7f46b00c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46b5cf2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46b0158a20 0x7f46b0152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46b5cf2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b00c6c00 con 0x7f46b00cb820 2023-07-14T11:53:53.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46b5cf2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b00c6ee0 con 0x7f46b0158a20 2023-07-14T11:53:53.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46b5cf2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b01f4dd0 con 0x7f46b006c220 2023-07-14T11:53:53.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46affff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b00cb820 0x7f46b00c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46affff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b00cb820 0x7f46b00c63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38520/0 (socket says 172.21.15.104:38520) 2023-07-14T11:53:53.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46affff700 1 -- 172.21.15.104:0/3128268935 learned_addr learned my addr 172.21.15.104:0/3128268935 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:53.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46b4cf0700 1 --2- 172.21.15.104:0/3128268935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b006c220 0x7f46b014d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46b54f1700 1 --2- 172.21.15.104:0/3128268935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46b0158a20 0x7f46b0152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46affff700 1 -- 172.21.15.104:0/3128268935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b006c220 msgr2=0x7f46b014d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46affff700 1 --2- 172.21.15.104:0/3128268935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b006c220 0x7f46b014d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46affff700 1 -- 172.21.15.104:0/3128268935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46b0158a20 msgr2=0x7f46b0152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46affff700 1 --2- 172.21.15.104:0/3128268935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46b0158a20 0x7f46b0152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46affff700 1 -- 172.21.15.104:0/3128268935 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a40094b0 con 0x7f46b00cb820 2023-07-14T11:53:53.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46b54f1700 1 --2- 172.21.15.104:0/3128268935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46b0158a20 0x7f46b0152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:53.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46affff700 1 --2- 172.21.15.104:0/3128268935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b00cb820 0x7f46b00c63b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f469c015000 tx=0x7f469c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:53.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46ad7fa700 1 -- 172.21.15.104:0/3128268935 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c003d10 con 0x7f46b00cb820 2023-07-14T11:53:53.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46ad7fa700 1 -- 172.21.15.104:0/3128268935 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469c003e90 con 0x7f46b00cb820 2023-07-14T11:53:53.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46b5cf2700 1 -- 172.21.15.104:0/3128268935 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469c00b810 con 0x7f46b00cb820 2023-07-14T11:53:53.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46ad7fa700 1 -- 172.21.15.104:0/3128268935 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c0106b0 con 0x7f46b00cb820 2023-07-14T11:53:53.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.428+0000 7f46b5cf2700 1 -- 172.21.15.104:0/3128268935 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b01f5390 con 0x7f46b00cb820 2023-07-14T11:53:53.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.432+0000 7f46b5cf2700 1 -- 172.21.15.104:0/3128268935 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b01f4c30 con 0x7f46b00cb820 2023-07-14T11:53:53.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.432+0000 7f46ad7fa700 1 -- 172.21.15.104:0/3128268935 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f469c02c440 con 0x7f46b00cb820 2023-07-14T11:53:53.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.432+0000 7f46ad7fa700 1 --2- 172.21.15.104:0/3128268935 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4694066320 0x7f46940687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.432+0000 7f46ad7fa700 1 -- 172.21.15.104:0/3128268935 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f469c085960 con 0x7f46b00cb820 2023-07-14T11:53:53.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.432+0000 7f46b4cf0700 1 --2- 172.21.15.104:0/3128268935 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4694066320 0x7f46940687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.436+0000 7f46b4cf0700 1 --2- 172.21.15.104:0/3128268935 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4694066320 0x7f46940687d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009c70 tx=0x7f46a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:53.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.440+0000 7f46ad7fa700 1 -- 172.21.15.104:0/3128268935 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f46b01f4c30 con 0x7f46b00cb820 2023-07-14T11:53:53.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.612+0000 7f46b5cf2700 1 -- 172.21.15.104:0/3128268935 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f46b01f4c30 con 0x7f46b00cb820 2023-07-14T11:53:53.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.612+0000 7f46ad7fa700 1 -- 172.21.15.104:0/3128268935 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f46b01f4c30 con 0x7f46b00cb820 2023-07-14T11:53:53.598 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:53:53.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.616+0000 7f46b5cf2700 1 -- 172.21.15.104:0/3128268935 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4694066320 msgr2=0x7f46940687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.616+0000 7f46b5cf2700 1 --2- 172.21.15.104:0/3128268935 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4694066320 0x7f46940687d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009c70 tx=0x7f46a4008040 comp rx=0 tx=0).stop 2023-07-14T11:53:53.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.616+0000 7f46b5cf2700 1 -- 172.21.15.104:0/3128268935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b00cb820 msgr2=0x7f46b00c63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.616+0000 7f46b5cf2700 1 --2- 172.21.15.104:0/3128268935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b00cb820 0x7f46b00c63b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f469c015000 tx=0x7f469c004770 comp rx=0 tx=0).stop 2023-07-14T11:53:53.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.616+0000 7f46b5cf2700 1 -- 172.21.15.104:0/3128268935 shutdown_connections 2023-07-14T11:53:53.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.616+0000 7f46b5cf2700 1 --2- 172.21.15.104:0/3128268935 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4694066320 0x7f46940687d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.616+0000 7f46b5cf2700 1 --2- 172.21.15.104:0/3128268935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b00cb820 0x7f46b00c63b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.616+0000 7f46b5cf2700 1 --2- 172.21.15.104:0/3128268935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46b0158a20 0x7f46b0152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.616+0000 7f46b5cf2700 1 --2- 172.21.15.104:0/3128268935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46b006c220 0x7f46b014d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.616+0000 7f46b5cf2700 1 -- 172.21.15.104:0/3128268935 >> 172.21.15.104:0/3128268935 conn(0x7f46b00afc60 msgr2=0x7f46b00c8930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:53.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.616+0000 7f46b5cf2700 1 -- 172.21.15.104:0/3128268935 shutdown_connections 2023-07-14T11:53:53.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.616+0000 7f46b5cf2700 1 -- 172.21.15.104:0/3128268935 wait complete. 2023-07-14T11:53:53.621 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:53:53.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.820+0000 7f93cbe1d700 1 Processor -- start 2023-07-14T11:53:53.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.820+0000 7f93cbe1d700 1 -- start start 2023-07-14T11:53:53.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93cbe1d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93c4160610 0x7f93c40cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93cbe1d700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c4163fe0 con 0x7f93c40b0490 2023-07-14T11:53:53.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93cbe1d700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c40c8c30 con 0x7f93c4160240 2023-07-14T11:53:53.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93cbe1d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40ca9e0 con 0x7f93c4160610 2023-07-14T11:53:53.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93c93b8700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f93c40b0490 0x7f93c406cf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:48454/0 (socket says 172.21.15.104:48454) 2023-07-14T11:53:53.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93c9bb9700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f93c4160240 0x7f93c40afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:46994/0 (socket says 172.21.15.104:46994) 2023-07-14T11:53:53.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93c93b8700 1 -- 172.21.15.104:0/2749650664 learned_addr learned my addr 172.21.15.104:0/2749650664 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:53.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93ca3ba700 1 --2- 172.21.15.104:0/2749650664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93c4160610 0x7f93c40cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93c8bb7700 1 -- 172.21.15.104:0/2749650664 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145227735 0 0) 0x7f93c4163fe0 con 0x7f93c40b0490 2023-07-14T11:53:53.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93c8bb7700 1 -- 172.21.15.104:0/2749650664 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93b00035d0 con 0x7f93c40b0490 2023-07-14T11:53:53.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93c8bb7700 1 -- 172.21.15.104:0/2749650664 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537810017 0 0) 0x7f93c40c8c30 con 0x7f93c4160240 2023-07-14T11:53:53.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93c8bb7700 1 -- 172.21.15.104:0/2749650664 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93c4163fe0 con 0x7f93c4160240 2023-07-14T11:53:53.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93ca3ba700 1 -- 172.21.15.104:0/2749650664 >> v1:172.21.15.121:6789/0 conn(0x7f93c4160240 legacy=0x7f93c40afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93c8bb7700 1 -- 172.21.15.104:0/2749650664 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 246355906 0 0) 0x7f93b00035d0 con 0x7f93c40b0490 2023-07-14T11:53:53.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93ca3ba700 1 -- 172.21.15.104:0/2749650664 >> v1:172.21.15.104:6789/0 conn(0x7f93c40b0490 legacy=0x7f93c406cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.824+0000 7f93ca3ba700 1 -- 172.21.15.104:0/2749650664 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c4164230 con 0x7f93c4160610 2023-07-14T11:53:53.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.828+0000 7f93ca3ba700 1 --2- 172.21.15.104:0/2749650664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93c4160610 0x7f93c40cdc20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93bc009dc0 tx=0x7f93bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2ff72c6f598998 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:53.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.828+0000 7f93c8bb7700 1 -- 172.21.15.104:0/2749650664 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc00e070 con 0x7f93c4160610 2023-07-14T11:53:53.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.828+0000 7f93c8bb7700 1 -- 172.21.15.104:0/2749650664 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93bc003a40 con 0x7f93c4160610 2023-07-14T11:53:53.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.828+0000 7f93c8bb7700 1 -- 172.21.15.104:0/2749650664 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b40015a0 con 0x7f93c4160610 2023-07-14T11:53:53.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.828+0000 7f93cbe1d700 1 -- 172.21.15.104:0/2749650664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93c4160610 msgr2=0x7f93c40cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.828+0000 7f93cbe1d700 1 --2- 172.21.15.104:0/2749650664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93c4160610 0x7f93c40cdc20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93bc009dc0 tx=0x7f93bc00b330 comp rx=0 tx=0).stop 2023-07-14T11:53:53.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.828+0000 7f93cbe1d700 1 -- 172.21.15.104:0/2749650664 shutdown_connections 2023-07-14T11:53:53.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.828+0000 7f93cbe1d700 1 --2- 172.21.15.104:0/2749650664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93c4160610 0x7f93c40cdc20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.828+0000 7f93cbe1d700 1 -- 172.21.15.104:0/2749650664 >> 172.21.15.104:0/2749650664 conn(0x7f93c40cad80 msgr2=0x7f93c40cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:53.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.828+0000 7f93cbe1d700 1 -- 172.21.15.104:0/2749650664 shutdown_connections 2023-07-14T11:53:53.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.828+0000 7f93cbe1d700 1 -- 172.21.15.104:0/2749650664 wait complete. 2023-07-14T11:53:53.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93cbe1d700 1 Processor -- start 2023-07-14T11:53:53.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93cbe1d700 1 -- start start 2023-07-14T11:53:53.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93cbe1d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93c40b0490 0x7f93c41633d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93cbe1d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93c4160240 0x7f93c40af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93cbe1d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93c4160610 0x7f93c40afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93cbe1d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c4163910 con 0x7f93c40b0490 2023-07-14T11:53:53.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93cbe1d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40c5f10 con 0x7f93c4160610 2023-07-14T11:53:53.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93cbe1d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40c61f0 con 0x7f93c4160240 2023-07-14T11:53:53.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93c9bb9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93c40b0490 0x7f93c41633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93c9bb9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93c40b0490 0x7f93c41633d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38530/0 (socket says 172.21.15.104:38530) 2023-07-14T11:53:53.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93c9bb9700 1 -- 172.21.15.104:0/2397253564 learned_addr learned my addr 172.21.15.104:0/2397253564 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:53.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93ca3ba700 1 --2- 172.21.15.104:0/2397253564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93c4160610 0x7f93c40afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93c93b8700 1 --2- 172.21.15.104:0/2397253564 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93c4160240 0x7f93c40af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93c9bb9700 1 -- 172.21.15.104:0/2397253564 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93c4160240 msgr2=0x7f93c40af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93c9bb9700 1 --2- 172.21.15.104:0/2397253564 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93c4160240 0x7f93c40af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93c9bb9700 1 -- 172.21.15.104:0/2397253564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93c4160610 msgr2=0x7f93c40afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:53.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93c9bb9700 1 --2- 172.21.15.104:0/2397253564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93c4160610 0x7f93c40afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:53.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93c9bb9700 1 -- 172.21.15.104:0/2397253564 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93bc009a70 con 0x7f93c40b0490 2023-07-14T11:53:53.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93c93b8700 1 --2- 172.21.15.104:0/2397253564 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93c4160240 0x7f93c40af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:53:53.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.832+0000 7f93c9bb9700 1 --2- 172.21.15.104:0/2397253564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93c40b0490 0x7f93c41633d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f93c000c700 tx=0x7f93c000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:53.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.836+0000 7f93ba7fc700 1 -- 172.21.15.104:0/2397253564 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c0005cd0 con 0x7f93c40b0490 2023-07-14T11:53:53.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.836+0000 7f93ba7fc700 1 -- 172.21.15.104:0/2397253564 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c0005e30 con 0x7f93c40b0490 2023-07-14T11:53:53.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.836+0000 7f93ba7fc700 1 -- 172.21.15.104:0/2397253564 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c00072f0 con 0x7f93c40b0490 2023-07-14T11:53:53.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.836+0000 7f93cbe1d700 1 -- 172.21.15.104:0/2397253564 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c40c64d0 con 0x7f93c40b0490 2023-07-14T11:53:53.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.836+0000 7f93cbe1d700 1 -- 172.21.15.104:0/2397253564 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c40c6a90 con 0x7f93c40b0490 2023-07-14T11:53:53.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.836+0000 7f93cbe1d700 1 -- 172.21.15.104:0/2397253564 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c404dcb0 con 0x7f93c40b0490 2023-07-14T11:53:53.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.836+0000 7f93ba7fc700 1 -- 172.21.15.104:0/2397253564 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f93c0007450 con 0x7f93c40b0490 2023-07-14T11:53:53.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.840+0000 7f93ba7fc700 1 --2- 172.21.15.104:0/2397253564 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f93a80662f0 0x7f93a80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:53.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.840+0000 7f93ba7fc700 1 -- 172.21.15.104:0/2397253564 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f93c00834e0 con 0x7f93c40b0490 2023-07-14T11:53:53.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.840+0000 7f93c93b8700 1 --2- 172.21.15.104:0/2397253564 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f93a80662f0 0x7f93a80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:53.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.840+0000 7f93c93b8700 1 --2- 172.21.15.104:0/2397253564 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f93a80662f0 0x7f93a80687a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f93b40016d0 tx=0x7f93b4001540 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:53.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:53.844+0000 7f93ba7fc700 1 -- 172.21.15.104:0/2397253564 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f93c00506e0 con 0x7f93c40b0490 2023-07-14T11:53:54.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.016+0000 7f93cbe1d700 1 -- 172.21.15.104:0/2397253564 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f93c40b0bc0 con 0x7f93c40b0490 2023-07-14T11:53:54.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.016+0000 7f93ba7fc700 1 -- 172.21.15.104:0/2397253564 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f93c0053d00 con 0x7f93c40b0490 2023-07-14T11:53:54.002 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:53:54.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.024+0000 7f93cbe1d700 1 -- 172.21.15.104:0/2397253564 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f93a80662f0 msgr2=0x7f93a80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.024+0000 7f93cbe1d700 1 --2- 172.21.15.104:0/2397253564 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f93a80662f0 0x7f93a80687a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f93b40016d0 tx=0x7f93b4001540 comp rx=0 tx=0).stop 2023-07-14T11:53:54.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.024+0000 7f93cbe1d700 1 -- 172.21.15.104:0/2397253564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93c40b0490 msgr2=0x7f93c41633d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.024+0000 7f93cbe1d700 1 --2- 172.21.15.104:0/2397253564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93c40b0490 0x7f93c41633d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f93c000c700 tx=0x7f93c000ca10 comp rx=0 tx=0).stop 2023-07-14T11:53:54.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.024+0000 7f93cbe1d700 1 -- 172.21.15.104:0/2397253564 shutdown_connections 2023-07-14T11:53:54.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.024+0000 7f93cbe1d700 1 --2- 172.21.15.104:0/2397253564 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f93a80662f0 0x7f93a80687a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.024+0000 7f93cbe1d700 1 --2- 172.21.15.104:0/2397253564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93c4160610 0x7f93c40afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.024+0000 7f93cbe1d700 1 --2- 172.21.15.104:0/2397253564 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93c4160240 0x7f93c40af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.024+0000 7f93cbe1d700 1 --2- 172.21.15.104:0/2397253564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93c40b0490 0x7f93c41633d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.024+0000 7f93cbe1d700 1 -- 172.21.15.104:0/2397253564 >> 172.21.15.104:0/2397253564 conn(0x7f93c40cad80 msgr2=0x7f93c415e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:54.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.024+0000 7f93cbe1d700 1 -- 172.21.15.104:0/2397253564 shutdown_connections 2023-07-14T11:53:54.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.024+0000 7f93cbe1d700 1 -- 172.21.15.104:0/2397253564 wait complete. 2023-07-14T11:53:54.031 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:53:54.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.220+0000 7fa42fd81700 1 Processor -- start 2023-07-14T11:53:54.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42fd81700 1 -- start start 2023-07-14T11:53:54.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42fd81700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815c070 0x7fa4280ad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:54.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42fd81700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa4280b0440 0x7fa4280703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:54.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42fd81700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4280cd530 con 0x7fa4280ad8b0 2023-07-14T11:53:54.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42fd81700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa42815c440 con 0x7fa4280b0440 2023-07-14T11:53:54.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42fd81700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4280cc1b0 con 0x7fa42815c070 2023-07-14T11:53:54.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42e31e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa4280b0440 0x7fa4280703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:54.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42e31e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa4280b0440 0x7fa4280703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38542/0 (socket says 172.21.15.104:38542) 2023-07-14T11:53:54.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42e31e700 1 -- 172.21.15.104:0/110527650 learned_addr learned my addr 172.21.15.104:0/110527650 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:54.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42db1d700 1 --2- 172.21.15.104:0/110527650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815c070 0x7fa4280ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:54.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42e31e700 1 -- 172.21.15.104:0/110527650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815c070 msgr2=0x7fa4280ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:54.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42e31e700 1 --2- 172.21.15.104:0/110527650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815c070 0x7fa4280ad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42e31e700 1 -- 172.21.15.104:0/110527650 >> v1:172.21.15.104:6789/0 conn(0x7fa4280ad8b0 legacy=0x7fa4280afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42e31e700 1 -- 172.21.15.104:0/110527650 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4280cc490 con 0x7fa4280b0440 2023-07-14T11:53:54.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42db1d700 1 --2- 172.21.15.104:0/110527650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815c070 0x7fa4280ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:54.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.224+0000 7fa42e31e700 1 --2- 172.21.15.104:0/110527650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4280b0440 0x7fa4280703a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa420009910 tx=0x7fa420009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ede4d7eadeac507 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:54.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.228+0000 7fa42cb1b700 1 -- 172.21.15.104:0/110527650 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42000ba60 con 0x7fa4280b0440 2023-07-14T11:53:54.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.228+0000 7fa42cb1b700 1 -- 172.21.15.104:0/110527650 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42000bbc0 con 0x7fa4280b0440 2023-07-14T11:53:54.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.228+0000 7fa42cb1b700 1 -- 172.21.15.104:0/110527650 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420009e60 con 0x7fa4280b0440 2023-07-14T11:53:54.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.228+0000 7fa42fd81700 1 -- 172.21.15.104:0/110527650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4280b0440 msgr2=0x7fa4280703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.228+0000 7fa42fd81700 1 --2- 172.21.15.104:0/110527650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4280b0440 0x7fa4280703a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa420009910 tx=0x7fa420009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:54.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.228+0000 7fa42fd81700 1 -- 172.21.15.104:0/110527650 shutdown_connections 2023-07-14T11:53:54.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.228+0000 7fa42fd81700 1 --2- 172.21.15.104:0/110527650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4280b0440 0x7fa4280703a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.228+0000 7fa42fd81700 1 --2- 172.21.15.104:0/110527650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa42815c070 0x7fa4280ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.228+0000 7fa42fd81700 1 -- 172.21.15.104:0/110527650 >> 172.21.15.104:0/110527650 conn(0x7fa4280cad30 msgr2=0x7fa4280cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:54.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.228+0000 7fa42fd81700 1 -- 172.21.15.104:0/110527650 shutdown_connections 2023-07-14T11:53:54.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.228+0000 7fa42fd81700 1 -- 172.21.15.104:0/110527650 wait complete. 2023-07-14T11:53:54.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.228+0000 7fa42fd81700 1 Processor -- start 2023-07-14T11:53:54.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.228+0000 7fa42fd81700 1 -- start start 2023-07-14T11:53:54.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42fd81700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4280ad8b0 0x7fa4280c1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:54.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42fd81700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4280b0440 0x7fa4280b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:54.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42fd81700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa42815c070 0x7fa4281f9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:54.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42fd81700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa428154f60 con 0x7fa4280b0440 2023-07-14T11:53:54.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42fd81700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4280c1890 con 0x7fa42815c070 2023-07-14T11:53:54.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42fd81700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42815df80 con 0x7fa4280ad8b0 2023-07-14T11:53:54.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42d31c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4280b0440 0x7fa4280b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:54.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42d31c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4280b0440 0x7fa4280b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38554/0 (socket says 172.21.15.104:38554) 2023-07-14T11:53:54.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42d31c700 1 -- 172.21.15.104:0/3794261460 learned_addr learned my addr 172.21.15.104:0/3794261460 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:54.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42e31e700 1 --2- 172.21.15.104:0/3794261460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa42815c070 0x7fa4281f9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:54.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42db1d700 1 --2- 172.21.15.104:0/3794261460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4280ad8b0 0x7fa4280c1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:54.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42d31c700 1 -- 172.21.15.104:0/3794261460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4280ad8b0 msgr2=0x7fa4280c1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42d31c700 1 --2- 172.21.15.104:0/3794261460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4280ad8b0 0x7fa4280c1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42d31c700 1 -- 172.21.15.104:0/3794261460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa42815c070 msgr2=0x7fa4281f9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42d31c700 1 --2- 172.21.15.104:0/3794261460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa42815c070 0x7fa4281f9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42d31c700 1 -- 172.21.15.104:0/3794261460 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa420009570 con 0x7fa4280b0440 2023-07-14T11:53:54.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42d31c700 1 --2- 172.21.15.104:0/3794261460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4280b0440 0x7fa4280b2c90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa41800c530 tx=0x7fa41800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:54.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa41e7fc700 1 -- 172.21.15.104:0/3794261460 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4180052d0 con 0x7fa4280b0440 2023-07-14T11:53:54.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42fd81700 1 -- 172.21.15.104:0/3794261460 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42815d370 con 0x7fa4280b0440 2023-07-14T11:53:54.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa41e7fc700 1 -- 172.21.15.104:0/3794261460 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa418005430 con 0x7fa4280b0440 2023-07-14T11:53:54.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa41e7fc700 1 -- 172.21.15.104:0/3794261460 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4180055a0 con 0x7fa4280b0440 2023-07-14T11:53:54.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.232+0000 7fa42fd81700 1 -- 172.21.15.104:0/3794261460 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42815d700 con 0x7fa4280b0440 2023-07-14T11:53:54.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.236+0000 7fa42fd81700 1 -- 172.21.15.104:0/3794261460 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4280b10e0 con 0x7fa4280b0440 2023-07-14T11:53:54.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.236+0000 7fa41e7fc700 1 -- 172.21.15.104:0/3794261460 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa4180038e0 con 0x7fa4280b0440 2023-07-14T11:53:54.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.236+0000 7fa41e7fc700 1 --2- 172.21.15.104:0/3794261460 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa40c066320 0x7fa40c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:54.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.236+0000 7fa41e7fc700 1 -- 172.21.15.104:0/3794261460 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa418083520 con 0x7fa4280b0440 2023-07-14T11:53:54.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.236+0000 7fa42db1d700 1 --2- 172.21.15.104:0/3794261460 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa40c066320 0x7fa40c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:54.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.240+0000 7fa42db1d700 1 --2- 172.21.15.104:0/3794261460 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa40c066320 0x7fa40c0687d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa424007510 tx=0x7fa4240074a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:54.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.244+0000 7fa41e7fc700 1 -- 172.21.15.104:0/3794261460 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fa418050590 con 0x7fa4280b0440 2023-07-14T11:53:54.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.448+0000 7fa42fd81700 1 -- 172.21.15.104:0/3794261460 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa42804dcb0 con 0x7fa4280b0440 2023-07-14T11:53:54.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.448+0000 7fa41e7fc700 1 -- 172.21.15.104:0/3794261460 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4717 (secure 0 0 0) 0x7fa418053bb0 con 0x7fa4280b0440 2023-07-14T11:53:54.434 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:54.434 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2103358277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2103358277},{"type":"v1","addr":"172.21.15.121:6833","nonce":2103358277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1841065287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1841065287},{"type":"v1","addr":"172.21.15.121:6835","nonce":1841065287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3342053518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3342053518},{"type":"v1","addr":"172.21.15.104:6835","nonce":3342053518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:53:15.029045+0000","modified":"2023-07-14T11:53:41.785898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/2666329866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2666329866},{"type":"v1","addr":"172.21.15.104:6837","nonce":2666329866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:53:54.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.452+0000 7fa42fd81700 1 -- 172.21.15.104:0/3794261460 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa40c066320 msgr2=0x7fa40c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.452+0000 7fa42fd81700 1 --2- 172.21.15.104:0/3794261460 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa40c066320 0x7fa40c0687d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa424007510 tx=0x7fa4240074a0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.452+0000 7fa42fd81700 1 -- 172.21.15.104:0/3794261460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4280b0440 msgr2=0x7fa4280b2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.452+0000 7fa42fd81700 1 --2- 172.21.15.104:0/3794261460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4280b0440 0x7fa4280b2c90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa41800c530 tx=0x7fa41800c840 comp rx=0 tx=0).stop 2023-07-14T11:53:54.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.452+0000 7fa42fd81700 1 -- 172.21.15.104:0/3794261460 shutdown_connections 2023-07-14T11:53:54.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.452+0000 7fa42fd81700 1 --2- 172.21.15.104:0/3794261460 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa40c066320 0x7fa40c0687d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.452+0000 7fa42fd81700 1 --2- 172.21.15.104:0/3794261460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4280b0440 0x7fa4280b2c90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.452+0000 7fa42fd81700 1 --2- 172.21.15.104:0/3794261460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa42815c070 0x7fa4281f9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.452+0000 7fa42fd81700 1 --2- 172.21.15.104:0/3794261460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa4280ad8b0 0x7fa4280c1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.452+0000 7fa42fd81700 1 -- 172.21.15.104:0/3794261460 >> 172.21.15.104:0/3794261460 conn(0x7fa4280cad30 msgr2=0x7fa4280c9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:54.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.456+0000 7fa42fd81700 1 -- 172.21.15.104:0/3794261460 shutdown_connections 2023-07-14T11:53:54.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.456+0000 7fa42fd81700 1 -- 172.21.15.104:0/3794261460 wait complete. 2023-07-14T11:53:54.441 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 9 2023-07-14T11:53:54.461 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:53:54.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.648+0000 7ff7f4907700 1 Processor -- start 2023-07-14T11:53:54.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.648+0000 7ff7f4907700 1 -- start start 2023-07-14T11:53:54.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.652+0000 7ff7f4907700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f00cb130 0x7ff7f015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:54.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.652+0000 7ff7f4907700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f015d230 con 0x7ff7f0071000 2023-07-14T11:53:54.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.652+0000 7ff7f4907700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f00cde10 con 0x7ff7f01599d0 2023-07-14T11:53:54.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.652+0000 7ff7f4907700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00cd520 con 0x7ff7f00cb130 2023-07-14T11:53:54.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.652+0000 7ff7e659c700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ff7f0071000 0x7ff7f00caa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:48476/0 (socket says 172.21.15.104:48476) 2023-07-14T11:53:54.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.652+0000 7ff7e659c700 1 -- 172.21.15.104:0/4262451007 learned_addr learned my addr 172.21.15.104:0/4262451007 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:54.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.652+0000 7ff7ef59e700 1 --2- 172.21.15.104:0/4262451007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f00cb130 0x7ff7f015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:54.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.652+0000 7ff7ee59c700 1 -- 172.21.15.104:0/4262451007 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699251267 0 0) 0x7ff7f015d230 con 0x7ff7f0071000 2023-07-14T11:53:54.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.652+0000 7ff7ee59c700 1 -- 172.21.15.104:0/4262451007 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7dc0035d0 con 0x7ff7f0071000 2023-07-14T11:53:54.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.652+0000 7ff7ee59c700 1 -- 172.21.15.104:0/4262451007 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176581880 0 0) 0x7ff7f00cde10 con 0x7ff7f01599d0 2023-07-14T11:53:54.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.652+0000 7ff7ef59e700 1 -- 172.21.15.104:0/4262451007 >> v1:172.21.15.121:6789/0 conn(0x7ff7f01599d0 legacy=0x7ff7f00c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.652+0000 7ff7ef59e700 1 -- 172.21.15.104:0/4262451007 >> v1:172.21.15.104:6789/0 conn(0x7ff7f0071000 legacy=0x7ff7f00caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.652+0000 7ff7ef59e700 1 -- 172.21.15.104:0/4262451007 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f00715e0 con 0x7ff7f00cb130 2023-07-14T11:53:54.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.652+0000 7ff7ef59e700 1 --2- 172.21.15.104:0/4262451007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f00cb130 0x7ff7f015ccf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009dc0 tx=0x7ff7e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40b2428f6b97b34b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:54.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.656+0000 7ff7ee59c700 1 -- 172.21.15.104:0/4262451007 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800e070 con 0x7ff7f00cb130 2023-07-14T11:53:54.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.656+0000 7ff7ee59c700 1 -- 172.21.15.104:0/4262451007 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e800bb20 con 0x7ff7f00cb130 2023-07-14T11:53:54.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.656+0000 7ff7ee59c700 1 -- 172.21.15.104:0/4262451007 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800bed0 con 0x7ff7f00cb130 2023-07-14T11:53:54.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.656+0000 7ff7f4907700 1 -- 172.21.15.104:0/4262451007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f00cb130 msgr2=0x7ff7f015ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.656+0000 7ff7f4907700 1 --2- 172.21.15.104:0/4262451007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f00cb130 0x7ff7f015ccf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009dc0 tx=0x7ff7e800b330 comp rx=0 tx=0).stop 2023-07-14T11:53:54.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.656+0000 7ff7f4907700 1 -- 172.21.15.104:0/4262451007 shutdown_connections 2023-07-14T11:53:54.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.656+0000 7ff7f4907700 1 --2- 172.21.15.104:0/4262451007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f00cb130 0x7ff7f015ccf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.656+0000 7ff7f4907700 1 -- 172.21.15.104:0/4262451007 >> 172.21.15.104:0/4262451007 conn(0x7ff7f006e960 msgr2=0x7ff7f006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:54.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.656+0000 7ff7f4907700 1 -- 172.21.15.104:0/4262451007 shutdown_connections 2023-07-14T11:53:54.641 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.656+0000 7ff7f4907700 1 -- 172.21.15.104:0/4262451007 wait complete. 2023-07-14T11:53:54.642 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7f4907700 1 Processor -- start 2023-07-14T11:53:54.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7f4907700 1 -- start start 2023-07-14T11:53:54.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7f4907700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f0071000 0x7ff7f0153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:54.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7f4907700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f00cb130 0x7ff7f00b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:54.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7f4907700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f01599d0 0x7ff7f00ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:54.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7f4907700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00c3300 con 0x7ff7f0071000 2023-07-14T11:53:54.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7f4907700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00c3550 con 0x7ff7f01599d0 2023-07-14T11:53:54.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7f4907700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00c3830 con 0x7ff7f00cb130 2023-07-14T11:53:54.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7eed9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f0071000 0x7ff7f0153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:54.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7eed9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f0071000 0x7ff7f0153100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38568/0 (socket says 172.21.15.104:38568) 2023-07-14T11:53:54.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7eed9d700 1 -- 172.21.15.104:0/2471107284 learned_addr learned my addr 172.21.15.104:0/2471107284 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:54.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7ef59e700 1 --2- 172.21.15.104:0/2471107284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f01599d0 0x7ff7f00ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:54.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7e659c700 1 --2- 172.21.15.104:0/2471107284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f00cb130 0x7ff7f00b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:54.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7eed9d700 1 -- 172.21.15.104:0/2471107284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f00cb130 msgr2=0x7ff7f00b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7eed9d700 1 --2- 172.21.15.104:0/2471107284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f00cb130 0x7ff7f00b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7eed9d700 1 -- 172.21.15.104:0/2471107284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f01599d0 msgr2=0x7ff7f00ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7eed9d700 1 --2- 172.21.15.104:0/2471107284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f01599d0 0x7ff7f00ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7eed9d700 1 -- 172.21.15.104:0/2471107284 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e8009a70 con 0x7ff7f0071000 2023-07-14T11:53:54.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.660+0000 7ff7ef59e700 1 --2- 172.21.15.104:0/2471107284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f01599d0 0x7ff7f00ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:54.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.664+0000 7ff7eed9d700 1 --2- 172.21.15.104:0/2471107284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f0071000 0x7ff7f0153100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000c520 tx=0x7ff7e000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:54.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.664+0000 7ff7e7fff700 1 -- 172.21.15.104:0/2471107284 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00054e0 con 0x7ff7f0071000 2023-07-14T11:53:54.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.664+0000 7ff7e7fff700 1 -- 172.21.15.104:0/2471107284 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e00036a0 con 0x7ff7f0071000 2023-07-14T11:53:54.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.664+0000 7ff7e7fff700 1 -- 172.21.15.104:0/2471107284 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0003a50 con 0x7ff7f0071000 2023-07-14T11:53:54.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.664+0000 7ff7f4907700 1 -- 172.21.15.104:0/2471107284 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f00c3b10 con 0x7ff7f0071000 2023-07-14T11:53:54.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.664+0000 7ff7f4907700 1 -- 172.21.15.104:0/2471107284 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f00c40d0 con 0x7ff7f0071000 2023-07-14T11:53:54.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.664+0000 7ff7f4907700 1 -- 172.21.15.104:0/2471107284 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f00c42f0 con 0x7ff7f0071000 2023-07-14T11:53:54.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.668+0000 7ff7e7fff700 1 -- 172.21.15.104:0/2471107284 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff7e0003bb0 con 0x7ff7f0071000 2023-07-14T11:53:54.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.668+0000 7ff7e7fff700 1 --2- 172.21.15.104:0/2471107284 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff7d0066320 0x7ff7d00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:54.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.668+0000 7ff7e7fff700 1 -- 172.21.15.104:0/2471107284 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff7e0083430 con 0x7ff7f0071000 2023-07-14T11:53:54.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.668+0000 7ff7e659c700 1 --2- 172.21.15.104:0/2471107284 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff7d0066320 0x7ff7d00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:54.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.668+0000 7ff7e659c700 1 --2- 172.21.15.104:0/2471107284 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff7d0066320 0x7ff7d00687d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0154e00 tx=0x7ff7d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:54.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.672+0000 7ff7e7fff700 1 -- 172.21.15.104:0/2471107284 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff7e00504e0 con 0x7ff7f0071000 2023-07-14T11:53:54.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.844+0000 7ff7f4907700 1 -- 172.21.15.104:0/2471107284 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7f004e500 con 0x7ff7f0071000 2023-07-14T11:53:54.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.848+0000 7ff7e7fff700 1 -- 172.21.15.104:0/2471107284 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20251 (secure 0 0 0) 0x7ff7e0003e60 con 0x7ff7f0071000 2023-07-14T11:53:54.832 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:54.832 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":29,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:54.591870+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:53:54.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.852+0000 7ff7f4907700 1 -- 172.21.15.104:0/2471107284 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff7d0066320 msgr2=0x7ff7d00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.852+0000 7ff7f4907700 1 --2- 172.21.15.104:0/2471107284 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff7d0066320 0x7ff7d00687d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0154e00 tx=0x7ff7d8009040 comp rx=0 tx=0).stop 2023-07-14T11:53:54.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.852+0000 7ff7f4907700 1 -- 172.21.15.104:0/2471107284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f0071000 msgr2=0x7ff7f0153100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:54.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.852+0000 7ff7f4907700 1 --2- 172.21.15.104:0/2471107284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f0071000 0x7ff7f0153100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000c520 tx=0x7ff7e000c830 comp rx=0 tx=0).stop 2023-07-14T11:53:54.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.852+0000 7ff7f4907700 1 -- 172.21.15.104:0/2471107284 shutdown_connections 2023-07-14T11:53:54.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.852+0000 7ff7f4907700 1 --2- 172.21.15.104:0/2471107284 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff7d0066320 0x7ff7d00687d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.852+0000 7ff7f4907700 1 --2- 172.21.15.104:0/2471107284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f01599d0 0x7ff7f00ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.852+0000 7ff7f4907700 1 --2- 172.21.15.104:0/2471107284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f00cb130 0x7ff7f00b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.852+0000 7ff7f4907700 1 --2- 172.21.15.104:0/2471107284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f0071000 0x7ff7f0153100 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:54.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.852+0000 7ff7f4907700 1 -- 172.21.15.104:0/2471107284 >> 172.21.15.104:0/2471107284 conn(0x7ff7f006e960 msgr2=0x7ff7f006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:54.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.852+0000 7ff7f4907700 1 -- 172.21.15.104:0/2471107284 shutdown_connections 2023-07-14T11:53:54.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:54.852+0000 7ff7f4907700 1 -- 172.21.15.104:0/2471107284 wait complete. 2023-07-14T11:53:54.860 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:53:55.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.048+0000 7f3910cbc700 1 Processor -- start 2023-07-14T11:53:55.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.048+0000 7f3910cbc700 1 -- start start 2023-07-14T11:53:55.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f3910cbc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f390c06d860 0x7f390c0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f3910cbc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c06be70 0x7f390c06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f3910cbc700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390c0cc1b0 con 0x7f390c0c1e00 2023-07-14T11:53:55.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f3910cbc700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390c06e170 con 0x7f390c06d860 2023-07-14T11:53:55.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f3910cbc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c0cd690 con 0x7f390c06be70 2023-07-14T11:53:55.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f390ad9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f390c0c1e00 0x7f390c0cc640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:48482/0 (socket says 172.21.15.104:48482) 2023-07-14T11:53:55.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f390ad9d700 1 -- 172.21.15.104:0/1707409555 learned_addr learned my addr 172.21.15.104:0/1707409555 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:53:55.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f390a59c700 1 --2- 172.21.15.104:0/1707409555 >> v2:172.21.15.121:3300/0 conn(0x7f390c06d860 0x7f390c0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f3903fff700 1 --2- 172.21.15.104:0/1707409555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c06be70 0x7f390c06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f3909d9b700 1 -- 172.21.15.104:0/1707409555 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686027242 0 0) 0x7f390c0cc1b0 con 0x7f390c0c1e00 2023-07-14T11:53:55.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f3909d9b700 1 -- 172.21.15.104:0/1707409555 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38f80035d0 con 0x7f390c0c1e00 2023-07-14T11:53:55.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f390a59c700 1 -- 172.21.15.104:0/1707409555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c06be70 msgr2=0x7f390c06dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:55.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f390a59c700 1 --2- 172.21.15.104:0/1707409555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c06be70 0x7f390c06dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f390a59c700 1 -- 172.21.15.104:0/1707409555 >> v1:172.21.15.104:6789/0 conn(0x7f390c0c1e00 legacy=0x7f390c0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f390a59c700 1 -- 172.21.15.104:0/1707409555 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c0cd970 con 0x7f390c06d860 2023-07-14T11:53:55.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.052+0000 7f390a59c700 1 --2- 172.21.15.104:0/1707409555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f390c06d860 0x7f390c0cb840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009880 tx=0x7f38fc009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=677680150bcd7771 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:55.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.056+0000 7f3909d9b700 1 -- 172.21.15.104:0/1707409555 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc011830 con 0x7f390c06d860 2023-07-14T11:53:55.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.056+0000 7f3909d9b700 1 -- 172.21.15.104:0/1707409555 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc011990 con 0x7f390c06d860 2023-07-14T11:53:55.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.056+0000 7f3909d9b700 1 -- 172.21.15.104:0/1707409555 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc011d20 con 0x7f390c06d860 2023-07-14T11:53:55.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.056+0000 7f3910cbc700 1 -- 172.21.15.104:0/1707409555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f390c06d860 msgr2=0x7f390c0cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.056+0000 7f3910cbc700 1 --2- 172.21.15.104:0/1707409555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f390c06d860 0x7f390c0cb840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009880 tx=0x7f38fc009b90 comp rx=0 tx=0).stop 2023-07-14T11:53:55.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.056+0000 7f3910cbc700 1 -- 172.21.15.104:0/1707409555 shutdown_connections 2023-07-14T11:53:55.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.056+0000 7f3910cbc700 1 --2- 172.21.15.104:0/1707409555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c06be70 0x7f390c06dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.056+0000 7f3910cbc700 1 --2- 172.21.15.104:0/1707409555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f390c06d860 0x7f390c0cb840 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.056+0000 7f3910cbc700 1 -- 172.21.15.104:0/1707409555 >> 172.21.15.104:0/1707409555 conn(0x7f390c0cad30 msgr2=0x7f390c0cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:55.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.056+0000 7f3910cbc700 1 -- 172.21.15.104:0/1707409555 shutdown_connections 2023-07-14T11:53:55.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.056+0000 7f3910cbc700 1 -- 172.21.15.104:0/1707409555 wait complete. 2023-07-14T11:53:55.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f3910cbc700 1 Processor -- start 2023-07-14T11:53:55.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f3910cbc700 1 -- start start 2023-07-14T11:53:55.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f3910cbc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f390c06be70 0x7f390c1594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f3910cbc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c06d860 0x7f390c1f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f3910cbc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f390c0c1e00 0x7f390c1f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f3910cbc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c15b620 con 0x7f390c06be70 2023-07-14T11:53:55.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f3910cbc700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c1f32f0 con 0x7f390c0c1e00 2023-07-14T11:53:55.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f3910cbc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c15b310 con 0x7f390c06d860 2023-07-14T11:53:55.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f390a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f390c06be70 0x7f390c1594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f390a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f390c06be70 0x7f390c1594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38572/0 (socket says 172.21.15.104:38572) 2023-07-14T11:53:55.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f390a59c700 1 -- 172.21.15.104:0/3512029227 learned_addr learned my addr 172.21.15.104:0/3512029227 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:55.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f3903fff700 1 --2- 172.21.15.104:0/3512029227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c06d860 0x7f390c1f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f390ad9d700 1 --2- 172.21.15.104:0/3512029227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f390c0c1e00 0x7f390c1f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f390a59c700 1 -- 172.21.15.104:0/3512029227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c06d860 msgr2=0x7f390c1f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f390a59c700 1 --2- 172.21.15.104:0/3512029227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c06d860 0x7f390c1f2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f390a59c700 1 -- 172.21.15.104:0/3512029227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f390c0c1e00 msgr2=0x7f390c1f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f390a59c700 1 --2- 172.21.15.104:0/3512029227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f390c0c1e00 0x7f390c1f5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f390a59c700 1 -- 172.21.15.104:0/3512029227 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0094b0 con 0x7f390c06be70 2023-07-14T11:53:55.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f390ad9d700 1 --2- 172.21.15.104:0/3512029227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f390c0c1e00 0x7f390c1f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:53:55.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f390a59c700 1 --2- 172.21.15.104:0/3512029227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f390c06be70 0x7f390c1594f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f390c0700e0 tx=0x7f38fc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:55.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.060+0000 7f3902ffd700 1 -- 172.21.15.104:0/3512029227 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0044d0 con 0x7f390c06be70 2023-07-14T11:53:55.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.064+0000 7f3902ffd700 1 -- 172.21.15.104:0/3512029227 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc0036c0 con 0x7f390c06be70 2023-07-14T11:53:55.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.064+0000 7f3910cbc700 1 -- 172.21.15.104:0/3512029227 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c1f6060 con 0x7f390c06be70 2023-07-14T11:53:55.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.064+0000 7f3902ffd700 1 -- 172.21.15.104:0/3512029227 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc003a70 con 0x7f390c06be70 2023-07-14T11:53:55.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.064+0000 7f3910cbc700 1 -- 172.21.15.104:0/3512029227 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390c1f6620 con 0x7f390c06be70 2023-07-14T11:53:55.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.064+0000 7f3910cbc700 1 -- 172.21.15.104:0/3512029227 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390c04dcb0 con 0x7f390c06be70 2023-07-14T11:53:55.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.064+0000 7f3902ffd700 1 -- 172.21.15.104:0/3512029227 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f38fc003bd0 con 0x7f390c06be70 2023-07-14T11:53:55.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.064+0000 7f3902ffd700 1 --2- 172.21.15.104:0/3512029227 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f38ec0662f0 0x7f38ec0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.068+0000 7f3902ffd700 1 -- 172.21.15.104:0/3512029227 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f38fc021030 con 0x7f390c06be70 2023-07-14T11:53:55.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.068+0000 7f3903fff700 1 --2- 172.21.15.104:0/3512029227 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f38ec0662f0 0x7f38ec0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.068+0000 7f3903fff700 1 --2- 172.21.15.104:0/3512029227 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f38ec0662f0 0x7f38ec0687a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f390c0ab8c0 tx=0x7f38f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:55.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.072+0000 7f3902ffd700 1 -- 172.21.15.104:0/3512029227 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f38fc052650 con 0x7f390c06be70 2023-07-14T11:53:55.259 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.276+0000 7f3910cbc700 1 -- 172.21.15.104:0/3512029227 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f390c071440 con 0x7f390c06be70 2023-07-14T11:53:55.260 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.276+0000 7f3902ffd700 1 -- 172.21.15.104:0/3512029227 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4717 (secure 0 0 0) 0x7f38fc055c70 con 0x7f390c06be70 2023-07-14T11:53:55.260 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:53:55.260 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2103358277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2103358277},{"type":"v1","addr":"172.21.15.121:6833","nonce":2103358277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1841065287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1841065287},{"type":"v1","addr":"172.21.15.121:6835","nonce":1841065287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3342053518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3342053518},{"type":"v1","addr":"172.21.15.104:6835","nonce":3342053518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:53:15.029045+0000","modified":"2023-07-14T11:53:41.785898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/2666329866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2666329866},{"type":"v1","addr":"172.21.15.104:6837","nonce":2666329866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:53:55.264 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.280+0000 7f3910cbc700 1 -- 172.21.15.104:0/3512029227 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f38ec0662f0 msgr2=0x7f38ec0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.280+0000 7f3910cbc700 1 --2- 172.21.15.104:0/3512029227 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f38ec0662f0 0x7f38ec0687a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f390c0ab8c0 tx=0x7f38f4008040 comp rx=0 tx=0).stop 2023-07-14T11:53:55.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.280+0000 7f3910cbc700 1 -- 172.21.15.104:0/3512029227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f390c06be70 msgr2=0x7f390c1594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.280+0000 7f3910cbc700 1 --2- 172.21.15.104:0/3512029227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f390c06be70 0x7f390c1594f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f390c0700e0 tx=0x7f38fc004770 comp rx=0 tx=0).stop 2023-07-14T11:53:55.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.280+0000 7f3910cbc700 1 -- 172.21.15.104:0/3512029227 shutdown_connections 2023-07-14T11:53:55.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.280+0000 7f3910cbc700 1 --2- 172.21.15.104:0/3512029227 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f38ec0662f0 0x7f38ec0687a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.280+0000 7f3910cbc700 1 --2- 172.21.15.104:0/3512029227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f390c0c1e00 0x7f390c1f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.280+0000 7f3910cbc700 1 --2- 172.21.15.104:0/3512029227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f390c06d860 0x7f390c1f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.280+0000 7f3910cbc700 1 --2- 172.21.15.104:0/3512029227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f390c06be70 0x7f390c1594f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.280+0000 7f3910cbc700 1 -- 172.21.15.104:0/3512029227 >> 172.21.15.104:0/3512029227 conn(0x7f390c0cad30 msgr2=0x7f390c0c9480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:55.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.280+0000 7f3910cbc700 1 -- 172.21.15.104:0/3512029227 shutdown_connections 2023-07-14T11:53:55.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.280+0000 7f3910cbc700 1 -- 172.21.15.104:0/3512029227 wait complete. 2023-07-14T11:53:55.267 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 9 2023-07-14T11:53:55.687 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-14T11:53:15.029045+0000', 'modified': '2023-07-14T11:53:41.785898+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4275}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4275': {'gid': 4275, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6837/2666329866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 2666329866}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 2666329866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-14T11:53:55.688 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-14T11:53:55.688 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-14T11:53:55.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.884+0000 7fb756074700 1 Processor -- start 2023-07-14T11:53:55.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.884+0000 7fb756074700 1 -- start start 2023-07-14T11:53:55.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.884+0000 7fb756074700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb750160370 0x7fb7500caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.884+0000 7fb756074700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7500ca8a0 con 0x7fb75015ffa0 2023-07-14T11:53:55.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb756074700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7500cb3b0 con 0x7fb7500cafb0 2023-07-14T11:53:55.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb756074700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7500cbcd0 con 0x7fb750160370 2023-07-14T11:53:55.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb754871700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb750160370 0x7fb7500caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb754871700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb750160370 0x7fb7500caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:36994/0 (socket says 172.21.15.104:36994) 2023-07-14T11:53:55.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb754871700 1 -- 172.21.15.104:0/2417581376 learned_addr learned my addr 172.21.15.104:0/2417581376 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:55.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb755873700 1 --1- 172.21.15.104:0/2417581376 >> v1:172.21.15.121:6789/0 conn(0x7fb7500cafb0 0x7fb7500ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:47006/0 (socket says 172.21.15.104:47006) 2023-07-14T11:53:55.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb755072700 1 --1- 172.21.15.104:0/2417581376 >> v1:172.21.15.104:6789/0 conn(0x7fb75015ffa0 0x7fb75006d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:48486/0 (socket says 172.21.15.104:48486) 2023-07-14T11:53:55.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb747fff700 1 -- 172.21.15.104:0/2417581376 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334043671 0 0) 0x7fb74c001110 con 0x7fb75015ffa0 2023-07-14T11:53:55.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb747fff700 1 -- 172.21.15.104:0/2417581376 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb73c0035d0 con 0x7fb75015ffa0 2023-07-14T11:53:55.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb754871700 1 -- 172.21.15.104:0/2417581376 >> v1:172.21.15.121:6789/0 conn(0x7fb7500cafb0 legacy=0x7fb7500ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb754871700 1 -- 172.21.15.104:0/2417581376 >> v1:172.21.15.104:6789/0 conn(0x7fb75015ffa0 legacy=0x7fb75006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb754871700 1 -- 172.21.15.104:0/2417581376 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7500cbf70 con 0x7fb750160370 2023-07-14T11:53:55.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb754871700 1 --2- 172.21.15.104:0/2417581376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb750160370 0x7fb7500caa70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb740009910 tx=0x7fb740009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43312e831705c663 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:55.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb747fff700 1 -- 172.21.15.104:0/2417581376 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740011830 con 0x7fb750160370 2023-07-14T11:53:55.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb747fff700 1 -- 172.21.15.104:0/2417581376 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb740011990 con 0x7fb750160370 2023-07-14T11:53:55.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.888+0000 7fb747fff700 1 -- 172.21.15.104:0/2417581376 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740011d40 con 0x7fb750160370 2023-07-14T11:53:55.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.892+0000 7fb756074700 1 -- 172.21.15.104:0/2417581376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb750160370 msgr2=0x7fb7500caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.892+0000 7fb756074700 1 --2- 172.21.15.104:0/2417581376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb750160370 0x7fb7500caa70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb740009910 tx=0x7fb740009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:55.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.892+0000 7fb756074700 1 -- 172.21.15.104:0/2417581376 shutdown_connections 2023-07-14T11:53:55.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.892+0000 7fb756074700 1 --2- 172.21.15.104:0/2417581376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb750160370 0x7fb7500caa70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.892+0000 7fb756074700 1 -- 172.21.15.104:0/2417581376 >> 172.21.15.104:0/2417581376 conn(0x7fb75006e960 msgr2=0x7fb75006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:55.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.892+0000 7fb756074700 1 -- 172.21.15.104:0/2417581376 shutdown_connections 2023-07-14T11:53:55.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.892+0000 7fb756074700 1 -- 172.21.15.104:0/2417581376 wait complete. 2023-07-14T11:53:55.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.892+0000 7fb756074700 1 Processor -- start 2023-07-14T11:53:55.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb756074700 1 -- start start 2023-07-14T11:53:55.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb756074700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7500cafb0 0x7fb7501f9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb756074700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb75015ffa0 0x7fb7500ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb756074700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb750160370 0x7fb7501f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb756074700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7501fa230 con 0x7fb7500cafb0 2023-07-14T11:53:55.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb756074700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7501f6c50 con 0x7fb75015ffa0 2023-07-14T11:53:55.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb756074700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7501f6040 con 0x7fb750160370 2023-07-14T11:53:55.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb755072700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7500cafb0 0x7fb7501f9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb755072700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7500cafb0 0x7fb7501f9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38582/0 (socket says 172.21.15.104:38582) 2023-07-14T11:53:55.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb755072700 1 -- 172.21.15.104:0/4223493159 learned_addr learned my addr 172.21.15.104:0/4223493159 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:55.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb754871700 1 --2- 172.21.15.104:0/4223493159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb75015ffa0 0x7fb7500ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb755873700 1 --2- 172.21.15.104:0/4223493159 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb750160370 0x7fb7501f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb755072700 1 -- 172.21.15.104:0/4223493159 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb750160370 msgr2=0x7fb7501f2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb755072700 1 --2- 172.21.15.104:0/4223493159 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb750160370 0x7fb7501f2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb755072700 1 -- 172.21.15.104:0/4223493159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb75015ffa0 msgr2=0x7fb7500ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb755072700 1 --2- 172.21.15.104:0/4223493159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb75015ffa0 0x7fb7500ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb755072700 1 -- 172.21.15.104:0/4223493159 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb740009570 con 0x7fb7500cafb0 2023-07-14T11:53:55.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.896+0000 7fb755072700 1 --2- 172.21.15.104:0/4223493159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7500cafb0 0x7fb7501f9cf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00c4f0 tx=0x7fb74c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:55.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.900+0000 7fb745ffb700 1 -- 172.21.15.104:0/4223493159 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c005cd0 con 0x7fb7500cafb0 2023-07-14T11:53:55.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.900+0000 7fb745ffb700 1 -- 172.21.15.104:0/4223493159 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74c005e30 con 0x7fb7500cafb0 2023-07-14T11:53:55.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.900+0000 7fb745ffb700 1 -- 172.21.15.104:0/4223493159 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c0072f0 con 0x7fb7500cafb0 2023-07-14T11:53:55.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.900+0000 7fb756074700 1 -- 172.21.15.104:0/4223493159 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7501f6320 con 0x7fb7500cafb0 2023-07-14T11:53:55.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.900+0000 7fb756074700 1 -- 172.21.15.104:0/4223493159 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7501f68e0 con 0x7fb7500cafb0 2023-07-14T11:53:55.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.904+0000 7fb745ffb700 1 -- 172.21.15.104:0/4223493159 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb74c007450 con 0x7fb7500cafb0 2023-07-14T11:53:55.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.904+0000 7fb745ffb700 1 --2- 172.21.15.104:0/4223493159 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb7340662f0 0x7fb7340687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.904+0000 7fb745ffb700 1 -- 172.21.15.104:0/4223493159 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb74c083480 con 0x7fb7500cafb0 2023-07-14T11:53:55.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.904+0000 7fb75a55e740 1 Processor -- start 2023-07-14T11:53:55.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.904+0000 7fb754871700 1 --2- 172.21.15.104:0/4223493159 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb7340662f0 0x7fb7340687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.904+0000 7fb75a55e740 1 -- start start 2023-07-14T11:53:55.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.904+0000 7fb754871700 1 --2- 172.21.15.104:0/4223493159 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb7340662f0 0x7fb7340687a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb740009150 tx=0x7fb740008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:55.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.904+0000 7fb75a55e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3eb0 0x109cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.904+0000 7fb75a55e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3ae0 0x109d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.904+0000 7fb75a55e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3710 0x109da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.904+0000 7fb75a55e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0xf70270 con 0xde3ae0 2023-07-14T11:53:55.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.904+0000 7fb75a55e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xfc33a0 con 0xde3eb0 2023-07-14T11:53:55.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.904+0000 7fb75a55e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0xfbc600 con 0xde3710 2023-07-14T11:53:55.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb754871700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3ae0 0x109d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb754871700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3ae0 0x109d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38598/0 (socket says 172.21.15.104:38598) 2023-07-14T11:53:55.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb754871700 1 -- 172.21.15.104:0/330820759 learned_addr learned my addr 172.21.15.104:0/330820759 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:55.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb755072700 1 --2- 172.21.15.104:0/330820759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3eb0 0x109cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb755873700 1 --2- 172.21.15.104:0/330820759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3710 0x109da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb754871700 1 -- 172.21.15.104:0/330820759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3710 msgr2=0x109da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:55.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb754871700 1 --2- 172.21.15.104:0/330820759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3710 0x109da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb754871700 1 -- 172.21.15.104:0/330820759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3eb0 msgr2=0x109cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:55.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb754871700 1 --2- 172.21.15.104:0/330820759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3eb0 0x109cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb754871700 1 -- 172.21.15.104:0/330820759 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xe2f9e0 con 0xde3ae0 2023-07-14T11:53:55.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb754871700 1 --2- 172.21.15.104:0/330820759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3ae0 0x109d500 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb75006a810 tx=0x7fb740010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14d4b392dd89c1f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:55.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb732ffd700 1 -- 172.21.15.104:0/330820759 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74002c070 con 0xde3ae0 2023-07-14T11:53:55.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb732ffd700 1 -- 172.21.15.104:0/330820759 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb740011db0 con 0xde3ae0 2023-07-14T11:53:55.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb732ffd700 1 -- 172.21.15.104:0/330820759 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740020070 con 0xde3ae0 2023-07-14T11:53:55.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb75a55e740 1 -- 172.21.15.104:0/330820759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3ae0 msgr2=0x109d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb75a55e740 1 --2- 172.21.15.104:0/330820759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3ae0 0x109d500 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb75006a810 tx=0x7fb740010000 comp rx=0 tx=0).stop 2023-07-14T11:53:55.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb75a55e740 1 -- 172.21.15.104:0/330820759 shutdown_connections 2023-07-14T11:53:55.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb75a55e740 1 --2- 172.21.15.104:0/330820759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3ae0 0x109d500 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb75a55e740 1 --2- 172.21.15.104:0/330820759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3710 0x109da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb75a55e740 1 --2- 172.21.15.104:0/330820759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3eb0 0x109cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb75a55e740 1 -- 172.21.15.104:0/330820759 >> 172.21.15.104:0/330820759 conn(0x104db60 msgr2=0x1057e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:55.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb75a55e740 1 -- 172.21.15.104:0/330820759 shutdown_connections 2023-07-14T11:53:55.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb75a55e740 1 -- 172.21.15.104:0/330820759 wait complete. 2023-07-14T11:53:55.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb75a55e740 1 -- start start 2023-07-14T11:53:55.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.908+0000 7fb75a55e740 1 Processor -- start 2023-07-14T11:53:55.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb75a55e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3710 0x106e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb75a55e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3ae0 0x1082ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb75a55e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3eb0 0x1083530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb75a55e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0xf7eb00 con 0xde3710 2023-07-14T11:53:55.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb75a55e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xf7d470 con 0xde3eb0 2023-07-14T11:53:55.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb75a55e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0xe68b20 con 0xde3ae0 2023-07-14T11:53:55.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb75a55e740 10 client.0 _ll_register_callbacks cb 0xfe2530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:53:55.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb755072700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3710 0x106e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb755072700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3710 0x106e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38600/0 (socket says 172.21.15.104:38600) 2023-07-14T11:53:55.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb755072700 1 -- 172.21.15.104:0/1940816658 learned_addr learned my addr 172.21.15.104:0/1940816658 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:55.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb754871700 1 --2- 172.21.15.104:0/1940816658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3ae0 0x1082ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb755873700 1 --2- 172.21.15.104:0/1940816658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3eb0 0x1083530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb755072700 1 -- 172.21.15.104:0/1940816658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3ae0 msgr2=0x1082ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:55.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb755072700 1 --2- 172.21.15.104:0/1940816658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3ae0 0x1082ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb755072700 1 -- 172.21.15.104:0/1940816658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3eb0 msgr2=0x1083530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:55.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb755072700 1 --2- 172.21.15.104:0/1940816658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3eb0 0x1083530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb755072700 1 -- 172.21.15.104:0/1940816658 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7500cbf70 con 0xde3710 2023-07-14T11:53:55.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb755072700 1 --2- 172.21.15.104:0/1940816658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3710 0x106e8e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb74c050670 tx=0x7fb74c050560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e7d6394fa4b3bc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:55.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb72bfff700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-14T11:53:55.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb72bfff700 1 -- 172.21.15.104:0/1940816658 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c083070 con 0xde3710 2023-07-14T11:53:55.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb75a55e740 1 -- 172.21.15.104:0/1940816658 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xfb55e0 con 0xde3710 2023-07-14T11:53:55.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb72bfff700 1 -- 172.21.15.104:0/1940816658 <== mon.0 v2:172.21.15.104:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb74c0831d0 con 0xde3710 2023-07-14T11:53:55.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb72bfff700 1 -- 172.21.15.104:0/1940816658 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb74c04ff60 con 0xde3710 2023-07-14T11:53:55.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb75a55e740 10 client.4914 fetch_fsmap learned FSMap version 9 2023-07-14T11:53:55.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb75a55e740 1 -- 172.21.15.104:0/1940816658 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0xf82250 con 0xde3710 2023-07-14T11:53:55.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.912+0000 7fb72bfff700 1 -- 172.21.15.104:0/1940816658 <== mon.0 v2:172.21.15.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xfb55e0 con 0xde3710 2023-07-14T11:53:55.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb72bfff700 1 -- 172.21.15.104:0/1940816658 <== mon.0 v2:172.21.15.104:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fb74c0500c0 con 0xde3710 2023-07-14T11:53:55.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 10 client.4914 fetch_fsmap finished waiting for FSMap version 9 2023-07-14T11:53:55.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 10 client.4914 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-14T11:53:55.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 20 client.4914 populate_metadata read hostname 'smithi104' 2023-07-14T11:53:55.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 1 --2- 172.21.15.104:0/1940816658 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0xe2e790 0x10af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 4 client.4914 mds_command: new command op to 4275 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:53:55.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 1 -- 172.21.15.104:0/1940816658 --> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xfb5300 con 0xe2e790 2023-07-14T11:53:55.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb754871700 1 --2- 172.21.15.104:0/1940816658 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0xe2e790 0x10af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb72bfff700 10 client.4914 ms_handle_connect on v2:172.21.15.104:6836/2666329866 2023-07-14T11:53:55.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb754871700 1 --2- 172.21.15.104:0/1940816658 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0xe2e790 0x10af200 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a934456ba8a4ce53 server_cookie=1d1f573674b078c7 in_seq=0 out_seq=0 2023-07-14T11:53:55.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb72bfff700 1 -- 172.21.15.104:0/1940816658 <== mds.0 v2:172.21.15.104:6836/2666329866 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0xfb5300 con 0xe2e790 2023-07-14T11:53:55.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb72bfff700 10 client.4914 handle_command_reply: tid=0 2023-07-14T11:53:55.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 1 client.4914 shutdown 2023-07-14T11:53:55.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 1 -- 172.21.15.104:0/1940816658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3710 msgr2=0x106e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 1 --2- 172.21.15.104:0/1940816658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3710 0x106e8e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb74c050670 tx=0x7fb74c050560 comp rx=0 tx=0).stop 2023-07-14T11:53:55.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 1 -- 172.21.15.104:0/1940816658 shutdown_connections 2023-07-14T11:53:55.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 1 --2- 172.21.15.104:0/1940816658 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0xe2e790 0x10af200 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 1 --2- 172.21.15.104:0/1940816658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3eb0 0x1083530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb72bfff700 0 client.4914 ms_handle_reset on v2:172.21.15.104:6836/2666329866 2023-07-14T11:53:55.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 1 --2- 172.21.15.104:0/1940816658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3ae0 0x1082ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 1 --2- 172.21.15.104:0/1940816658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3710 0x106e8e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 1 -- 172.21.15.104:0/1940816658 >> 172.21.15.104:0/1940816658 conn(0x104db60 msgr2=0x1051490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:55.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 1 -- 172.21.15.104:0/1940816658 shutdown_connections 2023-07-14T11:53:55.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 1 -- 172.21.15.104:0/1940816658 wait complete. 2023-07-14T11:53:55.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.916+0000 7fb75a55e740 20 client.4914 trim_cache size 0 max 16384 2023-07-14T11:53:55.921 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.936+0000 7fb75a55e740 1 Processor -- start 2023-07-14T11:53:55.921 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.936+0000 7fb75a55e740 1 -- start start 2023-07-14T11:53:55.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.936+0000 7fb75a55e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3710 0x10a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.936+0000 7fb75a55e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3ae0 0x107f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.936+0000 7fb75a55e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3eb0 0x1081ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.936+0000 7fb75a55e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0xfb9e10 con 0xde3710 2023-07-14T11:53:55.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.936+0000 7fb75a55e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xfc47a0 con 0xde3ae0 2023-07-14T11:53:55.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.936+0000 7fb75a55e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0xfbbe10 con 0xde3eb0 2023-07-14T11:53:55.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb755072700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3710 0x10a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb754871700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3ae0 0x107f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb755072700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3710 0x10a4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38612/0 (socket says 172.21.15.104:38612) 2023-07-14T11:53:55.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb755072700 1 -- 172.21.15.104:0/3604265628 learned_addr learned my addr 172.21.15.104:0/3604265628 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:55.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb755873700 1 --2- 172.21.15.104:0/3604265628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3eb0 0x1081ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb754871700 1 -- 172.21.15.104:0/3604265628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3eb0 msgr2=0x1081ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:55.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb754871700 1 --2- 172.21.15.104:0/3604265628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3eb0 0x1081ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb754871700 1 -- 172.21.15.104:0/3604265628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3710 msgr2=0x10a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:55.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb754871700 1 --2- 172.21.15.104:0/3604265628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3710 0x10a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb754871700 1 -- 172.21.15.104:0/3604265628 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7501f3550 con 0xde3ae0 2023-07-14T11:53:55.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb755072700 1 --2- 172.21.15.104:0/3604265628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3710 0x10a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:55.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb754871700 1 --2- 172.21.15.104:0/3604265628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3ae0 0x107f780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb740031000 tx=0x7fb7400092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ce93380f0c4a811 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:55.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb7337fe700 1 -- 172.21.15.104:0/3604265628 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74002c040 con 0xde3ae0 2023-07-14T11:53:55.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb7337fe700 1 -- 172.21.15.104:0/3604265628 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb740004240 con 0xde3ae0 2023-07-14T11:53:55.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb7337fe700 1 -- 172.21.15.104:0/3604265628 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7400045b0 con 0xde3ae0 2023-07-14T11:53:55.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb75a55e740 1 -- 172.21.15.104:0/3604265628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3ae0 msgr2=0x107f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb75a55e740 1 --2- 172.21.15.104:0/3604265628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3ae0 0x107f780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb740031000 tx=0x7fb7400092e0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb75a55e740 1 -- 172.21.15.104:0/3604265628 shutdown_connections 2023-07-14T11:53:55.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb75a55e740 1 --2- 172.21.15.104:0/3604265628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3eb0 0x1081ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb75a55e740 1 --2- 172.21.15.104:0/3604265628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3ae0 0x107f780 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb75a55e740 1 --2- 172.21.15.104:0/3604265628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3710 0x10a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb75a55e740 1 -- 172.21.15.104:0/3604265628 >> 172.21.15.104:0/3604265628 conn(0x104db60 msgr2=0x10813c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:55.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb75a55e740 1 -- 172.21.15.104:0/3604265628 shutdown_connections 2023-07-14T11:53:55.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb75a55e740 1 -- 172.21.15.104:0/3604265628 wait complete. 2023-07-14T11:53:55.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb75a55e740 1 -- start start 2023-07-14T11:53:55.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.940+0000 7fb75a55e740 1 Processor -- start 2023-07-14T11:53:55.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb75a55e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3710 0x107f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb75a55e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3ae0 0x10fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb75a55e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3eb0 0x107e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb75a55e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x1052880 con 0xde3eb0 2023-07-14T11:53:55.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb75a55e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1080030 con 0xde3ae0 2023-07-14T11:53:55.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb75a55e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x107f340 con 0xde3710 2023-07-14T11:53:55.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb75a55e740 10 client.0 _ll_register_callbacks cb 0xf6de50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:53:55.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb755873700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3eb0 0x107e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb755873700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3eb0 0x107e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38616/0 (socket says 172.21.15.104:38616) 2023-07-14T11:53:55.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb755873700 1 -- 172.21.15.104:0/3583629619 learned_addr learned my addr 172.21.15.104:0/3583629619 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:55.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb754871700 1 --2- 172.21.15.104:0/3583629619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3ae0 0x10fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb755072700 1 --2- 172.21.15.104:0/3583629619 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3710 0x107f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb755873700 1 -- 172.21.15.104:0/3583629619 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3710 msgr2=0x107f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:55.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb755873700 1 --2- 172.21.15.104:0/3583629619 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3710 0x107f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb755873700 1 -- 172.21.15.104:0/3583629619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3ae0 msgr2=0x10fcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:55.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb755873700 1 --2- 172.21.15.104:0/3583629619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3ae0 0x10fcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb755873700 1 -- 172.21.15.104:0/3583629619 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xf82250 con 0xde3eb0 2023-07-14T11:53:55.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb754871700 1 --2- 172.21.15.104:0/3583629619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3ae0 0x10fcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:55.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb755873700 1 --2- 172.21.15.104:0/3583629619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3eb0 0x107e770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb748017ec0 tx=0x7fb748007330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4113048b8afc3f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:55.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb72bfff700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-14T11:53:55.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb72bfff700 1 -- 172.21.15.104:0/3583629619 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb748007a20 con 0xde3eb0 2023-07-14T11:53:55.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb75a55e740 1 -- 172.21.15.104:0/3583629619 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb740011db0 con 0xde3eb0 2023-07-14T11:53:55.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb72bfff700 1 -- 172.21.15.104:0/3583629619 <== mon.0 v2:172.21.15.104:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb748020440 con 0xde3eb0 2023-07-14T11:53:55.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb72bfff700 1 -- 172.21.15.104:0/3583629619 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb748020600 con 0xde3eb0 2023-07-14T11:53:55.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb75a55e740 10 client.4920 fetch_fsmap learned FSMap version 9 2023-07-14T11:53:55.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.944+0000 7fb75a55e740 1 -- 172.21.15.104:0/3583629619 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0xe2f9e0 con 0xde3eb0 2023-07-14T11:53:55.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb72bfff700 1 -- 172.21.15.104:0/3583629619 <== mon.0 v2:172.21.15.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb740011db0 con 0xde3eb0 2023-07-14T11:53:55.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb72bfff700 1 -- 172.21.15.104:0/3583629619 <== mon.0 v2:172.21.15.104:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fb748020760 con 0xde3eb0 2023-07-14T11:53:55.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 10 client.4920 fetch_fsmap finished waiting for FSMap version 9 2023-07-14T11:53:55.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 10 client.4920 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-14T11:53:55.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 20 client.4920 populate_metadata read hostname 'smithi104' 2023-07-14T11:53:55.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 1 --2- 172.21.15.104:0/3583629619 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0x10a8f30 0x1056830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 4 client.4920 mds_command: new command op to 4275 tid=0[{"prefix": "status"}] 2023-07-14T11:53:55.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 1 -- 172.21.15.104:0/3583629619 --> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] -- command(tid 0: {"prefix": "status"}) v1 -- 0xe3b4e0 con 0x10a8f30 2023-07-14T11:53:55.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb755072700 1 --2- 172.21.15.104:0/3583629619 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0x10a8f30 0x1056830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb72bfff700 10 client.4920 ms_handle_connect on v2:172.21.15.104:6836/2666329866 2023-07-14T11:53:55.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb755072700 1 --2- 172.21.15.104:0/3583629619 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0x10a8f30 0x1056830 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=528663411fa44f09 server_cookie=8c7c2d262701daa1 in_seq=0 out_seq=0 2023-07-14T11:53:55.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb72bfff700 1 -- 172.21.15.104:0/3583629619 <== mds.0 v2:172.21.15.104:6836/2666329866 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0xe3b4e0 con 0x10a8f30 2023-07-14T11:53:55.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb72bfff700 10 client.4920 handle_command_reply: tid=0 2023-07-14T11:53:55.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 1 client.4920 shutdown 2023-07-14T11:53:55.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 1 -- 172.21.15.104:0/3583629619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3eb0 msgr2=0x107e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 1 --2- 172.21.15.104:0/3583629619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3eb0 0x107e770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb748017ec0 tx=0x7fb748007330 comp rx=0 tx=0).stop 2023-07-14T11:53:55.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 1 -- 172.21.15.104:0/3583629619 shutdown_connections 2023-07-14T11:53:55.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 1 --2- 172.21.15.104:0/3583629619 >> [v2:172.21.15.104:6836/2666329866,v1:172.21.15.104:6837/2666329866] conn(0x10a8f30 0x1056830 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 1 --2- 172.21.15.104:0/3583629619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xde3eb0 0x107e770 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 1 --2- 172.21.15.104:0/3583629619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde3ae0 0x10fcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb72bfff700 0 client.4920 ms_handle_reset on v2:172.21.15.104:6836/2666329866 2023-07-14T11:53:55.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 1 --2- 172.21.15.104:0/3583629619 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0xde3710 0x107f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 1 -- 172.21.15.104:0/3583629619 >> 172.21.15.104:0/3583629619 conn(0x104db60 msgr2=0x10530d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:55.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 1 -- 172.21.15.104:0/3583629619 shutdown_connections 2023-07-14T11:53:55.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 1 -- 172.21.15.104:0/3583629619 wait complete. 2023-07-14T11:53:55.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.948+0000 7fb75a55e740 20 client.4920 trim_cache size 0 max 16384 2023-07-14T11:53:55.938 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:53:55.938 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_fsid": "c5576993-99e1-43b1-a253-659a3cee5212", 2023-07-14T11:53:55.938 INFO:teuthology.orchestra.run.smithi104.stdout: "whoami": 0, 2023-07-14T11:53:55.938 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 4275, 2023-07-14T11:53:55.938 INFO:teuthology.orchestra.run.smithi104.stdout: "want_state": "up:active", 2023-07-14T11:53:55.939 INFO:teuthology.orchestra.run.smithi104.stdout: "state": "up:active", 2023-07-14T11:53:55.939 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs", 2023-07-14T11:53:55.939 INFO:teuthology.orchestra.run.smithi104.stdout: "rank_uptime": 40.386639688000002, 2023-07-14T11:53:55.939 INFO:teuthology.orchestra.run.smithi104.stdout: "mdsmap_epoch": 9, 2023-07-14T11:53:55.939 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_epoch": 30, 2023-07-14T11:53:55.939 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_epoch_barrier": 28, 2023-07-14T11:53:55.939 INFO:teuthology.orchestra.run.smithi104.stdout: "uptime": 46.657203002999999 2023-07-14T11:53:55.939 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:53:55.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.952+0000 7fb756074700 1 -- 172.21.15.104:0/4223493159 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb7340662f0 msgr2=0x7fb7340687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.952+0000 7fb756074700 1 --2- 172.21.15.104:0/4223493159 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb7340662f0 0x7fb7340687a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb740009150 tx=0x7fb740008030 comp rx=0 tx=0).stop 2023-07-14T11:53:55.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.952+0000 7fb756074700 1 -- 172.21.15.104:0/4223493159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7500cafb0 msgr2=0x7fb7501f9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.952+0000 7fb756074700 1 --2- 172.21.15.104:0/4223493159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7500cafb0 0x7fb7501f9cf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00c4f0 tx=0x7fb74c00c800 comp rx=0 tx=0).stop 2023-07-14T11:53:55.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.952+0000 7fb756074700 1 -- 172.21.15.104:0/4223493159 shutdown_connections 2023-07-14T11:53:55.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.952+0000 7fb756074700 1 --2- 172.21.15.104:0/4223493159 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb7340662f0 0x7fb7340687a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.952+0000 7fb756074700 1 --2- 172.21.15.104:0/4223493159 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb750160370 0x7fb7501f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.952+0000 7fb756074700 1 --2- 172.21.15.104:0/4223493159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb75015ffa0 0x7fb7500ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.952+0000 7fb756074700 1 --2- 172.21.15.104:0/4223493159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7500cafb0 0x7fb7501f9cf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.952+0000 7fb756074700 1 -- 172.21.15.104:0/4223493159 >> 172.21.15.104:0/4223493159 conn(0x7fb75006e960 msgr2=0x7fb75006bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:55.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.952+0000 7fb756074700 1 -- 172.21.15.104:0/4223493159 shutdown_connections 2023-07-14T11:53:55.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:55.952+0000 7fb756074700 1 -- 172.21.15.104:0/4223493159 wait complete. 2023-07-14T11:53:55.956 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-07-14T11:53:55.967 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-07-14T11:53:55.967 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-07-14T11:53:55.968 INFO:tasks.ceph_fuse:client.0 config is {} 2023-07-14T11:53:55.968 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-14T11:53:55.968 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-14T11:53:55.968 INFO:tasks.ceph_fuse:client.2 config is {} 2023-07-14T11:53:55.969 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-14T11:53:55.969 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-14T11:53:55.969 INFO:tasks.ceph_fuse:client.1 config is {} 2023-07-14T11:53:55.969 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-14T11:53:55.969 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-14T11:53:55.970 INFO:tasks.ceph_fuse:client.3 config is {} 2023-07-14T11:53:55.970 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-14T11:53:55.970 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-14T11:53:55.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:55.970 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-07-14T11:53:55.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:55.991 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2023-07-14T11:53:56.079 INFO:teuthology.orchestra.run.smithi121.stderr:Cannot find device "ceph-brx" 2023-07-14T11:53:56.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:53:56.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:56.084 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2023-07-14T11:53:56.103 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-07-14T11:53:56.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:56.104 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-07-14T11:53:56.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:56.188 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link delete ceph-brx 2023-07-14T11:53:56.394 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-07-14T11:53:56.394 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-14T11:53:56.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.580+0000 7f67a96aa700 1 Processor -- start 2023-07-14T11:53:56.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.580+0000 7f67a96aa700 1 -- start start 2023-07-14T11:53:56.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.580+0000 7f67a96aa700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f67a4159f40 0x7f67a406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:56.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.580+0000 7f67a96aa700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67a406e230 0x7f67a406f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:56.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.580+0000 7f67a96aa700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a406f310 con 0x7f67a4159b70 2023-07-14T11:53:56.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.580+0000 7f67a96aa700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a40cba60 con 0x7f67a4159f40 2023-07-14T11:53:56.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.580+0000 7f67a96aa700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40cbd40 con 0x7f67a406e230 2023-07-14T11:53:56.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f679a7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f67a4159f40 0x7f67a406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:56.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f679a7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f67a4159f40 0x7f67a406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:37052/0 (socket says 172.21.15.104:37052) 2023-07-14T11:53:56.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f679a7fc700 1 -- 172.21.15.104:0/983277628 learned_addr learned my addr 172.21.15.104:0/983277628 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:56.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f67a3fff700 1 --2- 172.21.15.104:0/983277628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67a406e230 0x7f67a406f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:56.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f679a7fc700 1 -- 172.21.15.104:0/983277628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67a406e230 msgr2=0x7f67a406f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:56.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f679a7fc700 1 --2- 172.21.15.104:0/983277628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67a406e230 0x7f67a406f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:56.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f679a7fc700 1 -- 172.21.15.104:0/983277628 >> v1:172.21.15.121:6789/0 conn(0x7f67a4159b70 legacy=0x7f67a406ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:56.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f679a7fc700 1 -- 172.21.15.104:0/983277628 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a40cc020 con 0x7f67a4159f40 2023-07-14T11:53:56.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f679a7fc700 1 --2- 172.21.15.104:0/983277628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a4159f40 0x7f67a406dcf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f678c009ea0 tx=0x7f678c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90f9582d803525e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:56.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f67a27fc700 1 -- 172.21.15.104:0/983277628 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c00e070 con 0x7f67a4159f40 2023-07-14T11:53:56.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f67a27fc700 1 -- 172.21.15.104:0/983277628 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c0090d0 con 0x7f67a4159f40 2023-07-14T11:53:56.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f67a27fc700 1 -- 172.21.15.104:0/983277628 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c00bb50 con 0x7f67a4159f40 2023-07-14T11:53:56.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f67a96aa700 1 -- 172.21.15.104:0/983277628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a4159f40 msgr2=0x7f67a406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:56.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f67a96aa700 1 --2- 172.21.15.104:0/983277628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a4159f40 0x7f67a406dcf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f678c009ea0 tx=0x7f678c00b330 comp rx=0 tx=0).stop 2023-07-14T11:53:56.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f67a96aa700 1 -- 172.21.15.104:0/983277628 shutdown_connections 2023-07-14T11:53:56.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f67a96aa700 1 --2- 172.21.15.104:0/983277628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67a406e230 0x7f67a406f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:56.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f67a96aa700 1 --2- 172.21.15.104:0/983277628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a4159f40 0x7f67a406dcf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:56.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.584+0000 7f67a96aa700 1 -- 172.21.15.104:0/983277628 >> 172.21.15.104:0/983277628 conn(0x7f67a40cc500 msgr2=0x7f67a40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:56.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.588+0000 7f67a96aa700 1 -- 172.21.15.104:0/983277628 shutdown_connections 2023-07-14T11:53:56.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.588+0000 7f67a96aa700 1 -- 172.21.15.104:0/983277628 wait complete. 2023-07-14T11:53:56.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.588+0000 7f67a96aa700 1 Processor -- start 2023-07-14T11:53:56.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.588+0000 7f67a96aa700 1 -- start start 2023-07-14T11:53:56.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f67a96aa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a406e230 0x7f67a41f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:56.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f67a96aa700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a4159b70 0x7f67a406f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:56.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f67a96aa700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67a4159f40 0x7f67a415bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:56.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f67a96aa700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40c9710 con 0x7f67a4159b70 2023-07-14T11:53:56.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f67a96aa700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a415b440 con 0x7f67a406e230 2023-07-14T11:53:56.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f67a96aa700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a41f61b0 con 0x7f67a4159f40 2023-07-14T11:53:56.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f679a7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a4159b70 0x7f67a406f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:56.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f67a2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a406e230 0x7f67a41f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:56.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f679a7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a4159b70 0x7f67a406f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38618/0 (socket says 172.21.15.104:38618) 2023-07-14T11:53:56.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f679a7fc700 1 -- 172.21.15.104:0/2912105524 learned_addr learned my addr 172.21.15.104:0/2912105524 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:56.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f67a2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a406e230 0x7f67a41f5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:37064/0 (socket says 172.21.15.104:37064) 2023-07-14T11:53:56.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f67a3fff700 1 --2- 172.21.15.104:0/2912105524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67a4159f40 0x7f67a415bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:56.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f679a7fc700 1 -- 172.21.15.104:0/2912105524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67a4159f40 msgr2=0x7f67a415bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:56.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f679a7fc700 1 --2- 172.21.15.104:0/2912105524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67a4159f40 0x7f67a415bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:56.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f679a7fc700 1 -- 172.21.15.104:0/2912105524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a406e230 msgr2=0x7f67a41f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:56.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f679a7fc700 1 --2- 172.21.15.104:0/2912105524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a406e230 0x7f67a41f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:56.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f679a7fc700 1 -- 172.21.15.104:0/2912105524 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c009b30 con 0x7f67a4159b70 2023-07-14T11:53:56.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.592+0000 7f679a7fc700 1 --2- 172.21.15.104:0/2912105524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a4159b70 0x7f67a406f760 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f678c012000 tx=0x7f678c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:56.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.596+0000 7f679bfff700 1 -- 172.21.15.104:0/2912105524 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c004040 con 0x7f67a4159b70 2023-07-14T11:53:56.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.596+0000 7f679bfff700 1 -- 172.21.15.104:0/2912105524 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c0041e0 con 0x7f67a4159b70 2023-07-14T11:53:56.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.596+0000 7f67a96aa700 1 -- 172.21.15.104:0/2912105524 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a41f6490 con 0x7f67a4159b70 2023-07-14T11:53:56.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.596+0000 7f679bfff700 1 -- 172.21.15.104:0/2912105524 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c004590 con 0x7f67a4159b70 2023-07-14T11:53:56.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.596+0000 7f67a96aa700 1 -- 172.21.15.104:0/2912105524 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a415b650 con 0x7f67a4159b70 2023-07-14T11:53:56.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.596+0000 7f67a96aa700 1 -- 172.21.15.104:0/2912105524 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a415b840 con 0x7f67a4159b70 2023-07-14T11:53:56.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.596+0000 7f679bfff700 1 -- 172.21.15.104:0/2912105524 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f678c0046f0 con 0x7f67a4159b70 2023-07-14T11:53:56.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.600+0000 7f679bfff700 1 --2- 172.21.15.104:0/2912105524 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f67840662f0 0x7f67840687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:56.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.600+0000 7f679bfff700 1 -- 172.21.15.104:0/2912105524 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f678c085360 con 0x7f67a4159b70 2023-07-14T11:53:56.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.600+0000 7f67a2ffd700 1 --2- 172.21.15.104:0/2912105524 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f67840662f0 0x7f67840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:56.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.600+0000 7f67a2ffd700 1 --2- 172.21.15.104:0/2912105524 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f67840662f0 0x7f67840687a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f67940018a0 tx=0x7f6794009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:56.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.604+0000 7f679bfff700 1 -- 172.21.15.104:0/2912105524 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f678c0524c0 con 0x7f67a4159b70 2023-07-14T11:53:56.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.812+0000 7f67a96aa700 1 -- 172.21.15.104:0/2912105524 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f67a404dcb0 con 0x7f67a4159b70 2023-07-14T11:53:56.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.816+0000 7f679bfff700 1 -- 172.21.15.104:0/2912105524 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f678c020070 con 0x7f67a4159b70 2023-07-14T11:53:56.799 INFO:teuthology.orchestra.run.smithi104.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-14T11:53:56.803 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.820+0000 7f67a96aa700 1 -- 172.21.15.104:0/2912105524 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f67840662f0 msgr2=0x7f67840687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:56.803 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.820+0000 7f67a96aa700 1 --2- 172.21.15.104:0/2912105524 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f67840662f0 0x7f67840687a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f67940018a0 tx=0x7f6794009040 comp rx=0 tx=0).stop 2023-07-14T11:53:56.803 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.820+0000 7f67a96aa700 1 -- 172.21.15.104:0/2912105524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a4159b70 msgr2=0x7f67a406f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:56.803 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.820+0000 7f67a96aa700 1 --2- 172.21.15.104:0/2912105524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a4159b70 0x7f67a406f760 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f678c012000 tx=0x7f678c004af0 comp rx=0 tx=0).stop 2023-07-14T11:53:56.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.820+0000 7f67a96aa700 1 -- 172.21.15.104:0/2912105524 shutdown_connections 2023-07-14T11:53:56.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.820+0000 7f67a96aa700 1 --2- 172.21.15.104:0/2912105524 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f67840662f0 0x7f67840687a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:56.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.820+0000 7f67a96aa700 1 --2- 172.21.15.104:0/2912105524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a4159b70 0x7f67a406f760 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:56.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.820+0000 7f67a96aa700 1 --2- 172.21.15.104:0/2912105524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67a4159f40 0x7f67a415bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:56.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.820+0000 7f67a96aa700 1 --2- 172.21.15.104:0/2912105524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a406e230 0x7f67a41f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:56.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.820+0000 7f67a96aa700 1 -- 172.21.15.104:0/2912105524 >> 172.21.15.104:0/2912105524 conn(0x7f67a40cc500 msgr2=0x7f67a4159180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:56.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.820+0000 7f67a96aa700 1 -- 172.21.15.104:0/2912105524 shutdown_connections 2023-07-14T11:53:56.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:56.820+0000 7f67a96aa700 1 -- 172.21.15.104:0/2912105524 wait complete. 2023-07-14T11:53:56.824 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-14T11:53:56.825 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T11:53:56.825 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2023-07-14T11:53:56.825 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-14T11:53:56.825 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-14T11:53:56.825 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-14T11:53:56.826 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T11:53:56.826 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T11:53:56.826 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:53:56.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:56.827 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2023-07-14T11:53:56.842 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:53:56.842 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:53:56.842 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:53:56.842 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:56.842 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2023-07-14T11:53:56.842 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:56.843 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:53:56.843 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:56.843 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2023-07-14T11:53:56.843 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:53:56.843 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:56.843 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2023-07-14T11:53:56.843 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:53:56.844 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:56.844 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T11:53:56.844 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:56.844 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-07-14T11:53:56.844 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42359sec preferred_lft 42359sec 2023-07-14T11:53:56.844 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2023-07-14T11:53:56.844 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:56.844 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-07-14T11:53:56.845 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:a2:a1:53:16 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:56.845 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-07-14T11:53:56.845 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:56.846 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-14T11:53:56.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:56.846 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:53:56.847 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:53:56.847 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link add name ceph-brx type bridge 2023-07-14T11:53:56.847 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip addr flush dev ceph-brx 2023-07-14T11:53:56.847 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set ceph-brx up 2023-07-14T11:53:56.847 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-14T11:53:56.848 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:53:56.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:56.997 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T11:53:57.031 INFO:teuthology.orchestra.run.smithi104.stdout:1 2023-07-14T11:53:57.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:57.034 DEBUG:teuthology.orchestra.run.smithi104:> route 2023-07-14T11:53:57.094 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2023-07-14T11:53:57.094 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T11:53:57.095 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-07-14T11:53:57.095 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T11:53:57.095 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-07-14T11:53:57.095 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-07-14T11:53:57.095 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-07-14T11:53:57.095 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T11:53:57.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:57.096 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:53:57.097 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:53:57.097 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T11:53:57.097 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T11:53:57.097 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T11:53:57.098 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:53:57.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:57.229 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2023-07-14T11:53:57.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:57.249 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list-id 2023-07-14T11:53:57.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:57.309 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:53:57.309 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:53:57.309 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-14T11:53:57.310 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-14T11:53:57.310 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:53:57.434 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-14T11:53:57.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:57.435 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:53:57.435 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:53:57.436 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-14T11:53:57.436 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-14T11:53:57.436 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-14T11:53:57.436 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-14T11:53:57.436 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-14T11:53:57.437 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:53:57.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:57.590 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:53:57.590 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:53:57.591 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2023-07-14T11:53:57.591 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set dev brx.0 master ceph-brx 2023-07-14T11:53:57.591 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:53:57.656 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-14T11:53:57.656 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:53:57.656 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-14T11:53:57.669 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:53:57.669 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:53:57.670 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:53:57.725 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:53:57.803 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:53:57.803 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:53:57.879 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:53:57.882 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:53:57.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:57.883 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:53:57.946 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-14T11:53:57.947 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-14T11:53:57.996 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:53:58.030 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:53:58.031 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:53:58.112 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:53:58.115 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:53:58.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:58.116 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:53:58.132 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:2023-07-14T11:53:58.148+0000 7f9ea9cab640 -1 init, newargv = 0x55b19c413270 newargc=15 2023-07-14T11:53:58.132 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[43223]: starting ceph client 2023-07-14T11:53:58.143 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[43223]: starting fuse 2023-07-14T11:53:58.146 INFO:teuthology.orchestra.run.smithi104.stdout:49 2023-07-14T11:53:58.146 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-07-14T11:53:58.147 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T11:53:58.147 DEBUG:teuthology.orchestra.run.smithi104:> import glob 2023-07-14T11:53:58.147 DEBUG:teuthology.orchestra.run.smithi104:> import re 2023-07-14T11:53:58.147 DEBUG:teuthology.orchestra.run.smithi104:> import os 2023-07-14T11:53:58.148 DEBUG:teuthology.orchestra.run.smithi104:> import subprocess 2023-07-14T11:53:58.148 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:53:58.148 DEBUG:teuthology.orchestra.run.smithi104:> def _find_admin_socket(client_name): 2023-07-14T11:53:58.148 DEBUG:teuthology.orchestra.run.smithi104:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-14T11:53:58.149 DEBUG:teuthology.orchestra.run.smithi104:> files = glob.glob(asok_path) 2023-07-14T11:53:58.149 DEBUG:teuthology.orchestra.run.smithi104:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-14T11:53:58.149 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:53:58.149 DEBUG:teuthology.orchestra.run.smithi104:> # Given a non-glob path, it better be there 2023-07-14T11:53:58.150 DEBUG:teuthology.orchestra.run.smithi104:> if "*" not in asok_path: 2023-07-14T11:53:58.150 DEBUG:teuthology.orchestra.run.smithi104:> assert(len(files) == 1) 2023-07-14T11:53:58.150 DEBUG:teuthology.orchestra.run.smithi104:> return files[0] 2023-07-14T11:53:58.150 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:53:58.150 DEBUG:teuthology.orchestra.run.smithi104:> for f in files: 2023-07-14T11:53:58.151 DEBUG:teuthology.orchestra.run.smithi104:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-14T11:53:58.151 DEBUG:teuthology.orchestra.run.smithi104:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-14T11:53:58.151 DEBUG:teuthology.orchestra.run.smithi104:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-14T11:53:58.151 DEBUG:teuthology.orchestra.run.smithi104:> contents = proc_f.read() 2023-07-14T11:53:58.151 DEBUG:teuthology.orchestra.run.smithi104:> if mountpoint in contents: 2023-07-14T11:53:58.152 DEBUG:teuthology.orchestra.run.smithi104:> return f 2023-07-14T11:53:58.152 DEBUG:teuthology.orchestra.run.smithi104:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-14T11:53:58.152 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:53:58.152 DEBUG:teuthology.orchestra.run.smithi104:> print(_find_admin_socket("client.0")) 2023-07-14T11:53:58.153 DEBUG:teuthology.orchestra.run.smithi104:> ' 2023-07-14T11:53:58.310 INFO:teuthology.orchestra.run.smithi104.stdout:/var/run/ceph/ceph-client.0.43223.asok 2023-07-14T11:53:58.499 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43223.asok 2023-07-14T11:53:58.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:58.500 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43223.asok status 2023-07-14T11:53:58.697 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:53:58.697 INFO:teuthology.orchestra.run.smithi104.stdout: "metadata": { 2023-07-14T11:53:58.697 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-14T11:53:58.698 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:53:58.698 INFO:teuthology.orchestra.run.smithi104.stdout: "entity_id": "0", 2023-07-14T11:53:58.698 INFO:teuthology.orchestra.run.smithi104.stdout: "hostname": "smithi104", 2023-07-14T11:53:58.698 INFO:teuthology.orchestra.run.smithi104.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-14T11:53:58.698 INFO:teuthology.orchestra.run.smithi104.stdout: "pid": "43223", 2023-07-14T11:53:58.698 INFO:teuthology.orchestra.run.smithi104.stdout: "root": "/" 2023-07-14T11:53:58.699 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:53:58.699 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_count": 0, 2023-07-14T11:53:58.699 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_pinned_count": 0, 2023-07-14T11:53:58.699 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 4929, 2023-07-14T11:53:58.700 INFO:teuthology.orchestra.run.smithi104.stdout: "inst": { 2023-07-14T11:53:58.700 INFO:teuthology.orchestra.run.smithi104.stdout: "name": { 2023-07-14T11:53:58.700 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "client", 2023-07-14T11:53:58.700 INFO:teuthology.orchestra.run.smithi104.stdout: "num": 4929 2023-07-14T11:53:58.700 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:53:58.700 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:53:58.700 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:53:58.700 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:53:58.701 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1715520903 2023-07-14T11:53:58.701 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:53:58.701 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:53:58.701 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:53:58.701 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:53:58.701 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:53:58.701 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1715520903 2023-07-14T11:53:58.702 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:53:58.702 INFO:teuthology.orchestra.run.smithi104.stdout: "inst_str": "client.4929 192.168.0.1:0/1715520903", 2023-07-14T11:53:58.702 INFO:teuthology.orchestra.run.smithi104.stdout: "addr_str": "192.168.0.1:0/1715520903", 2023-07-14T11:53:58.702 INFO:teuthology.orchestra.run.smithi104.stdout: "inode_count": 1, 2023-07-14T11:53:58.702 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_epoch": 9, 2023-07-14T11:53:58.702 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch": 30, 2023-07-14T11:53:58.702 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch_barrier": 0, 2023-07-14T11:53:58.702 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklisted": false, 2023-07-14T11:53:58.703 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs" 2023-07-14T11:53:58.703 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:53:58.712 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-14T11:53:58.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.896+0000 7feb09834700 1 Processor -- start 2023-07-14T11:53:58.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.900+0000 7feb09834700 1 -- start start 2023-07-14T11:53:58.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.900+0000 7feb09834700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7feb040cd240 0x7feb040ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:58.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.900+0000 7feb09834700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feb040ceff0 0x7feb040afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:58.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.900+0000 7feb09834700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb040af0a0 con 0x7feb0415bff0 2023-07-14T11:53:58.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.900+0000 7feb09834700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb040cf720 con 0x7feb040cd240 2023-07-14T11:53:58.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.900+0000 7feb09834700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb040af2f0 con 0x7feb040ceff0 2023-07-14T11:53:58.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.900+0000 7feb027fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7feb040cd240 0x7feb040ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:58.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.900+0000 7feb027fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7feb040cd240 0x7feb040ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38630/0 (socket says 172.21.15.104:38630) 2023-07-14T11:53:58.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.900+0000 7feb027fc700 1 -- 172.21.15.104:0/271259818 learned_addr learned my addr 172.21.15.104:0/271259818 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:58.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.900+0000 7feb037fe700 1 --2- 172.21.15.104:0/271259818 >> v2:172.21.15.121:3300/0 conn(0x7feb040ceff0 0x7feb040afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:58.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.900+0000 7feb027fc700 1 -- 172.21.15.104:0/271259818 >> v2:172.21.15.121:3300/0 conn(0x7feb040ceff0 msgr2=0x7feb040afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:53:58.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.900+0000 7feb027fc700 1 --2- 172.21.15.104:0/271259818 >> v2:172.21.15.121:3300/0 conn(0x7feb040ceff0 0x7feb040afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:58.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.900+0000 7feb027fc700 1 -- 172.21.15.104:0/271259818 >> v1:172.21.15.121:6789/0 conn(0x7feb0415bff0 legacy=0x7feb040ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:58.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.900+0000 7feb027fc700 1 -- 172.21.15.104:0/271259818 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb040af5d0 con 0x7feb040cd240 2023-07-14T11:53:58.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.900+0000 7feb027fc700 1 --2- 172.21.15.104:0/271259818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb040cd240 0x7feb040ceab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feb0408b700 tx=0x7feaf8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8843f24aa8a2805a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:58.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.904+0000 7feb01ffb700 1 -- 172.21.15.104:0/271259818 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf8011830 con 0x7feb040cd240 2023-07-14T11:53:58.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.904+0000 7feb01ffb700 1 -- 172.21.15.104:0/271259818 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf8011990 con 0x7feb040cd240 2023-07-14T11:53:58.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.904+0000 7feb01ffb700 1 -- 172.21.15.104:0/271259818 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf8011d20 con 0x7feb040cd240 2023-07-14T11:53:58.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.904+0000 7feb09834700 1 -- 172.21.15.104:0/271259818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb040cd240 msgr2=0x7feb040ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:58.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.904+0000 7feb09834700 1 --2- 172.21.15.104:0/271259818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb040cd240 0x7feb040ceab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feb0408b700 tx=0x7feaf8009bf0 comp rx=0 tx=0).stop 2023-07-14T11:53:58.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.904+0000 7feb09834700 1 -- 172.21.15.104:0/271259818 shutdown_connections 2023-07-14T11:53:58.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.904+0000 7feb09834700 1 --2- 172.21.15.104:0/271259818 >> v2:172.21.15.121:3300/0 conn(0x7feb040ceff0 0x7feb040afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:58.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.904+0000 7feb09834700 1 --2- 172.21.15.104:0/271259818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb040cd240 0x7feb040ceab0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:58.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.904+0000 7feb09834700 1 -- 172.21.15.104:0/271259818 >> 172.21.15.104:0/271259818 conn(0x7feb04060c40 msgr2=0x7feb0409b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:58.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.904+0000 7feb09834700 1 -- 172.21.15.104:0/271259818 shutdown_connections 2023-07-14T11:53:58.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.904+0000 7feb09834700 1 -- 172.21.15.104:0/271259818 wait complete. 2023-07-14T11:53:58.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb09834700 1 Processor -- start 2023-07-14T11:53:58.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb09834700 1 -- start start 2023-07-14T11:53:58.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb09834700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb040cd240 0x7feb041f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:58.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb09834700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb040ceff0 0x7feb041f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:58.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb09834700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb0415bff0 0x7feb04070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:58.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb09834700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb041f45c0 con 0x7feb0415bff0 2023-07-14T11:53:58.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb09834700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb041f1730 con 0x7feb040ceff0 2023-07-14T11:53:58.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb09834700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb040b7f80 con 0x7feb040cd240 2023-07-14T11:53:58.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb037fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb0415bff0 0x7feb04070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:58.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb037fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb0415bff0 0x7feb04070b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38638/0 (socket says 172.21.15.104:38638) 2023-07-14T11:53:58.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb02ffd700 1 --2- 172.21.15.104:0/1336599960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb040cd240 0x7feb041f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:58.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb037fe700 1 -- 172.21.15.104:0/1336599960 learned_addr learned my addr 172.21.15.104:0/1336599960 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:53:58.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb027fc700 1 --2- 172.21.15.104:0/1336599960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb040ceff0 0x7feb041f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:58.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb037fe700 1 -- 172.21.15.104:0/1336599960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb040cd240 msgr2=0x7feb041f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:58.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb037fe700 1 --2- 172.21.15.104:0/1336599960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb040cd240 0x7feb041f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:58.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb037fe700 1 -- 172.21.15.104:0/1336599960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb040ceff0 msgr2=0x7feb041f11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:58.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb037fe700 1 --2- 172.21.15.104:0/1336599960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb040ceff0 0x7feb041f11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:58.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb037fe700 1 -- 172.21.15.104:0/1336599960 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf8009570 con 0x7feb0415bff0 2023-07-14T11:53:58.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb027fc700 1 --2- 172.21.15.104:0/1336599960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb040ceff0 0x7feb041f11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:53:58.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feb037fe700 1 --2- 172.21.15.104:0/1336599960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb0415bff0 0x7feb04070b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feafc00efc0 tx=0x7feafc00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:58.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.908+0000 7feaebfff700 1 -- 172.21.15.104:0/1336599960 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00cad0 con 0x7feb0415bff0 2023-07-14T11:53:58.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.912+0000 7feaebfff700 1 -- 172.21.15.104:0/1336599960 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc00f040 con 0x7feb0415bff0 2023-07-14T11:53:58.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.912+0000 7feb09834700 1 -- 172.21.15.104:0/1336599960 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb040b8260 con 0x7feb0415bff0 2023-07-14T11:53:58.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.912+0000 7feaebfff700 1 -- 172.21.15.104:0/1336599960 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00ce70 con 0x7feb0415bff0 2023-07-14T11:53:58.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.912+0000 7feb09834700 1 -- 172.21.15.104:0/1336599960 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0415c960 con 0x7feb0415bff0 2023-07-14T11:53:58.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.912+0000 7feb09834700 1 -- 172.21.15.104:0/1336599960 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0404e500 con 0x7feb0415bff0 2023-07-14T11:53:58.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.912+0000 7feaebfff700 1 -- 172.21.15.104:0/1336599960 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7feafc014480 con 0x7feb0415bff0 2023-07-14T11:53:58.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.916+0000 7feaebfff700 1 --2- 172.21.15.104:0/1336599960 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feaf00662f0 0x7feaf00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:58.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.916+0000 7feaebfff700 1 -- 172.21.15.104:0/1336599960 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7feafc0847d0 con 0x7feb0415bff0 2023-07-14T11:53:58.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.916+0000 7feb02ffd700 1 --2- 172.21.15.104:0/1336599960 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feaf00662f0 0x7feaf00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:58.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.916+0000 7feb02ffd700 1 --2- 172.21.15.104:0/1336599960 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feaf00662f0 0x7feaf00687a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000f60 tx=0x7feaf4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:58.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:58.920+0000 7feaebfff700 1 -- 172.21.15.104:0/1336599960 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7feafc051880 con 0x7feb0415bff0 2023-07-14T11:53:59.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:59.128+0000 7feb09834700 1 -- 172.21.15.104:0/1336599960 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feb040b08f0 con 0x7feb0415bff0 2023-07-14T11:53:59.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:59.132+0000 7feaebfff700 1 -- 172.21.15.104:0/1336599960 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7feafc054ea0 con 0x7feb0415bff0 2023-07-14T11:53:59.114 INFO:teuthology.orchestra.run.smithi104.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-14T11:53:59.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:59.132+0000 7feb09834700 1 -- 172.21.15.104:0/1336599960 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feaf00662f0 msgr2=0x7feaf00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:59.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:59.132+0000 7feb09834700 1 --2- 172.21.15.104:0/1336599960 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feaf00662f0 0x7feaf00687a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000f60 tx=0x7feaf4009040 comp rx=0 tx=0).stop 2023-07-14T11:53:59.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:59.132+0000 7feb09834700 1 -- 172.21.15.104:0/1336599960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb0415bff0 msgr2=0x7feb04070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:59.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:59.132+0000 7feb09834700 1 --2- 172.21.15.104:0/1336599960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb0415bff0 0x7feb04070b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feafc00efc0 tx=0x7feafc00c0c0 comp rx=0 tx=0).stop 2023-07-14T11:53:59.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:59.136+0000 7feb09834700 1 -- 172.21.15.104:0/1336599960 shutdown_connections 2023-07-14T11:53:59.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:59.136+0000 7feb09834700 1 --2- 172.21.15.104:0/1336599960 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feaf00662f0 0x7feaf00687a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:59.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:59.136+0000 7feb09834700 1 --2- 172.21.15.104:0/1336599960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb0415bff0 0x7feb04070b90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:59.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:59.136+0000 7feb09834700 1 --2- 172.21.15.104:0/1336599960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb040ceff0 0x7feb041f11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:59.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:59.136+0000 7feb09834700 1 --2- 172.21.15.104:0/1336599960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb040cd240 0x7feb041f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:59.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:59.136+0000 7feb09834700 1 -- 172.21.15.104:0/1336599960 >> 172.21.15.104:0/1336599960 conn(0x7feb04060c40 msgr2=0x7feb040afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:59.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:59.136+0000 7feb09834700 1 -- 172.21.15.104:0/1336599960 shutdown_connections 2023-07-14T11:53:59.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:53:59.136+0000 7feb09834700 1 -- 172.21.15.104:0/1336599960 wait complete. 2023-07-14T11:53:59.140 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-14T11:53:59.140 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T11:53:59.140 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2023-07-14T11:53:59.141 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-07-14T11:53:59.141 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-14T11:53:59.141 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-14T11:53:59.141 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T11:53:59.141 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T11:53:59.142 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-07-14T11:53:59.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:59.142 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2023-07-14T11:53:59.159 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:53:59.160 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:53:59.160 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:53:59.160 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:59.160 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2023-07-14T11:53:59.160 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:59.161 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:53:59.161 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:59.161 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2023-07-14T11:53:59.161 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:53:59.161 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:59.161 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2023-07-14T11:53:59.161 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:53:59.161 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:59.162 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T11:53:59.162 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:59.162 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-07-14T11:53:59.162 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42357sec preferred_lft 42357sec 2023-07-14T11:53:59.162 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2023-07-14T11:53:59.162 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:59.162 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-07-14T11:53:59.162 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:a2:a1:53:16 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:59.163 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-07-14T11:53:59.163 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:59.163 INFO:teuthology.orchestra.run.smithi104.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-14T11:53:59.163 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:b8:27:80:0d:ab brd ff:ff:ff:ff:ff:ff 2023-07-14T11:53:59.163 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-14T11:53:59.163 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:59.163 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5020:39ff:fe23:761d/64 scope link 2023-07-14T11:53:59.163 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:59.164 INFO:teuthology.orchestra.run.smithi104.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-14T11:53:59.164 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:b8:27:80:0d:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-14T11:53:59.164 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::e8b8:27ff:fe80:dab/64 scope link 2023-07-14T11:53:59.164 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:59.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:59.165 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T11:53:59.245 INFO:teuthology.orchestra.run.smithi104.stdout:1 2023-07-14T11:53:59.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:59.248 DEBUG:teuthology.orchestra.run.smithi104:> route 2023-07-14T11:53:59.308 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2023-07-14T11:53:59.308 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T11:53:59.308 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-07-14T11:53:59.308 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T11:53:59.309 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-07-14T11:53:59.309 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-07-14T11:53:59.309 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-07-14T11:53:59.309 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T11:53:59.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:59.310 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:53:59.310 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:53:59.311 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T11:53:59.311 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T11:53:59.311 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T11:53:59.311 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:53:59.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:59.451 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2023-07-14T11:53:59.468 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-14T11:53:59.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:59.469 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list-id 2023-07-14T11:53:59.528 INFO:teuthology.orchestra.run.smithi104.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-07-14T11:53:59.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:59.530 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:53:59.530 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:53:59.530 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-14T11:53:59.530 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-07-14T11:53:59.531 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:53:59.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:59.645 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-14T11:53:59.678 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:53:59.678 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:53:59.678 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:53:59.678 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:59.679 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2023-07-14T11:53:59.679 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:59.679 INFO:teuthology.orchestra.run.smithi104.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-14T11:53:59.679 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 8a:0f:7f:d4:34:7e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-14T11:53:59.679 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-14T11:53:59.679 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:59.680 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::880f:7fff:fed4:347e/64 scope link 2023-07-14T11:53:59.680 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:59.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:59.682 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-14T11:53:59.764 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:53:59.764 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:53:59.764 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:53:59.765 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:59.765 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2023-07-14T11:53:59.765 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:59.765 INFO:teuthology.orchestra.run.smithi104.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-14T11:53:59.765 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 8a:0f:7f:d4:34:7e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-14T11:53:59.765 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-14T11:53:59.765 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:59.766 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::880f:7fff:fed4:347e/64 scope link 2023-07-14T11:53:59.766 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:53:59.768 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-07-14T11:53:59.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:59.768 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:53:59.769 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:53:59.769 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-07-14T11:53:59.769 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-14T11:53:59.769 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-07-14T11:53:59.770 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-07-14T11:53:59.770 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-07-14T11:53:59.770 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:53:59.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:53:59.962 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:53:59.962 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:53:59.963 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.1 up 2023-07-14T11:53:59.963 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set dev brx.1 master ceph-brx 2023-07-14T11:53:59.963 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:54:00.028 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-07-14T11:54:00.028 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:54:00.028 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-07-14T11:54:00.045 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-07-14T11:54:00.046 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:54:00.046 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-07-14T11:54:00.108 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:54:00.189 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:54:00.190 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:54:00.264 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:54:00.266 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:54:00.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:00.267 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:54:00.326 INFO:teuthology.orchestra.run.smithi104.stdout:49 2023-07-14T11:54:00.326 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [49] 2023-07-14T11:54:00.327 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-07-14T11:54:00.381 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:54:00.409 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:54:00.410 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:54:00.483 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:54:00.485 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:54:00.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:00.486 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:54:00.519 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi104.stderr:2023-07-14T11:54:00.536+0000 7f8506dfb640 -1 init, newargv = 0x5563f9458ed0 newargc=15 2023-07-14T11:54:00.519 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi104.stderr:ceph-fuse[43359]: starting ceph client 2023-07-14T11:54:00.530 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi104.stderr:ceph-fuse[43359]: starting fuse 2023-07-14T11:54:00.532 INFO:teuthology.orchestra.run.smithi104.stdout:49 2023-07-14T11:54:00.532 INFO:teuthology.orchestra.run.smithi104.stdout:50 2023-07-14T11:54:00.533 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49, 50] 2023-07-14T11:54:00.533 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T11:54:00.533 DEBUG:teuthology.orchestra.run.smithi104:> import glob 2023-07-14T11:54:00.534 DEBUG:teuthology.orchestra.run.smithi104:> import re 2023-07-14T11:54:00.534 DEBUG:teuthology.orchestra.run.smithi104:> import os 2023-07-14T11:54:00.534 DEBUG:teuthology.orchestra.run.smithi104:> import subprocess 2023-07-14T11:54:00.534 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:54:00.535 DEBUG:teuthology.orchestra.run.smithi104:> def _find_admin_socket(client_name): 2023-07-14T11:54:00.535 DEBUG:teuthology.orchestra.run.smithi104:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-07-14T11:54:00.535 DEBUG:teuthology.orchestra.run.smithi104:> files = glob.glob(asok_path) 2023-07-14T11:54:00.535 DEBUG:teuthology.orchestra.run.smithi104:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-07-14T11:54:00.535 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:54:00.536 DEBUG:teuthology.orchestra.run.smithi104:> # Given a non-glob path, it better be there 2023-07-14T11:54:00.536 DEBUG:teuthology.orchestra.run.smithi104:> if "*" not in asok_path: 2023-07-14T11:54:00.536 DEBUG:teuthology.orchestra.run.smithi104:> assert(len(files) == 1) 2023-07-14T11:54:00.536 DEBUG:teuthology.orchestra.run.smithi104:> return files[0] 2023-07-14T11:54:00.537 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:54:00.537 DEBUG:teuthology.orchestra.run.smithi104:> for f in files: 2023-07-14T11:54:00.537 DEBUG:teuthology.orchestra.run.smithi104:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-14T11:54:00.537 DEBUG:teuthology.orchestra.run.smithi104:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-14T11:54:00.537 DEBUG:teuthology.orchestra.run.smithi104:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-14T11:54:00.538 DEBUG:teuthology.orchestra.run.smithi104:> contents = proc_f.read() 2023-07-14T11:54:00.538 DEBUG:teuthology.orchestra.run.smithi104:> if mountpoint in contents: 2023-07-14T11:54:00.538 DEBUG:teuthology.orchestra.run.smithi104:> return f 2023-07-14T11:54:00.538 DEBUG:teuthology.orchestra.run.smithi104:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-14T11:54:00.539 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:54:00.539 DEBUG:teuthology.orchestra.run.smithi104:> print(_find_admin_socket("client.2")) 2023-07-14T11:54:00.539 DEBUG:teuthology.orchestra.run.smithi104:> ' 2023-07-14T11:54:00.697 INFO:teuthology.orchestra.run.smithi104.stdout:/var/run/ceph/ceph-client.2.43359.asok 2023-07-14T11:54:00.881 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.43359.asok 2023-07-14T11:54:00.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:00.882 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.43359.asok status 2023-07-14T11:54:01.071 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:54:01.072 INFO:teuthology.orchestra.run.smithi104.stdout: "metadata": { 2023-07-14T11:54:01.072 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-14T11:54:01.072 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:54:01.072 INFO:teuthology.orchestra.run.smithi104.stdout: "entity_id": "2", 2023-07-14T11:54:01.072 INFO:teuthology.orchestra.run.smithi104.stdout: "hostname": "smithi104", 2023-07-14T11:54:01.072 INFO:teuthology.orchestra.run.smithi104.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-07-14T11:54:01.073 INFO:teuthology.orchestra.run.smithi104.stdout: "pid": "43359", 2023-07-14T11:54:01.073 INFO:teuthology.orchestra.run.smithi104.stdout: "root": "/" 2023-07-14T11:54:01.073 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:01.073 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_count": 0, 2023-07-14T11:54:01.073 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_pinned_count": 0, 2023-07-14T11:54:01.073 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 4941, 2023-07-14T11:54:01.073 INFO:teuthology.orchestra.run.smithi104.stdout: "inst": { 2023-07-14T11:54:01.074 INFO:teuthology.orchestra.run.smithi104.stdout: "name": { 2023-07-14T11:54:01.074 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "client", 2023-07-14T11:54:01.074 INFO:teuthology.orchestra.run.smithi104.stdout: "num": 4941 2023-07-14T11:54:01.074 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:01.074 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:54:01.074 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:54:01.074 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.2:0", 2023-07-14T11:54:01.075 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 519441777 2023-07-14T11:54:01.075 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:01.075 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:01.075 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:54:01.075 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:54:01.075 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.2:0", 2023-07-14T11:54:01.076 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 519441777 2023-07-14T11:54:01.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:01.076 INFO:teuthology.orchestra.run.smithi104.stdout: "inst_str": "client.4941 192.168.0.2:0/519441777", 2023-07-14T11:54:01.076 INFO:teuthology.orchestra.run.smithi104.stdout: "addr_str": "192.168.0.2:0/519441777", 2023-07-14T11:54:01.076 INFO:teuthology.orchestra.run.smithi104.stdout: "inode_count": 1, 2023-07-14T11:54:01.076 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_epoch": 9, 2023-07-14T11:54:01.076 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch": 30, 2023-07-14T11:54:01.076 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch_barrier": 0, 2023-07-14T11:54:01.077 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklisted": false, 2023-07-14T11:54:01.077 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs" 2023-07-14T11:54:01.077 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:54:01.084 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-14T11:54:01.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.272+0000 7fb4adbbe700 1 Processor -- start 2023-07-14T11:54:01.259 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.276+0000 7fb4adbbe700 1 -- start start 2023-07-14T11:54:01.260 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.276+0000 7fb4adbbe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb4a8159f20 0x7fb4a80caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:01.261 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.276+0000 7fb4adbbe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb4a80caf70 0x7fb4a815d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:01.261 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.276+0000 7fb4adbbe700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a80b0ef0 con 0x7fb4a80c9d90 2023-07-14T11:54:01.261 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.276+0000 7fb4adbbe700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a80ce9c0 con 0x7fb4a80caf70 2023-07-14T11:54:01.261 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.276+0000 7fb4adbbe700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a80cee50 con 0x7fb4a8159f20 2023-07-14T11:54:01.262 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.276+0000 7fb4ad3bd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb4a80caf70 0x7fb4a815d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:01.262 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.276+0000 7fb4acbbc700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb4a80c9d90 0x7fb4a80ca160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:39134/0 (socket says 172.21.15.104:39134) 2023-07-14T11:54:01.262 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.276+0000 7fb4acbbc700 1 -- 172.21.15.104:0/4133632614 learned_addr learned my addr 172.21.15.104:0/4133632614 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:01.262 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.276+0000 7fb49ffff700 1 --2- 172.21.15.104:0/4133632614 >> v2:172.21.15.121:3300/0 conn(0x7fb4a8159f20 0x7fb4a80caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:01.263 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.276+0000 7fb4ad3bd700 1 -- 172.21.15.104:0/4133632614 >> v2:172.21.15.121:3300/0 conn(0x7fb4a8159f20 msgr2=0x7fb4a80caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:01.263 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.276+0000 7fb4ad3bd700 1 --2- 172.21.15.104:0/4133632614 >> v2:172.21.15.121:3300/0 conn(0x7fb4a8159f20 0x7fb4a80caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:01.263 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.276+0000 7fb49f7fe700 1 -- 172.21.15.104:0/4133632614 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537586450 0 0) 0x7fb4a80b0ef0 con 0x7fb4a80c9d90 2023-07-14T11:54:01.263 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.276+0000 7fb4ad3bd700 1 -- 172.21.15.104:0/4133632614 >> v1:172.21.15.121:6789/0 conn(0x7fb4a80c9d90 legacy=0x7fb4a80ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:01.263 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.276+0000 7fb4ad3bd700 1 -- 172.21.15.104:0/4133632614 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a80cb4f0 con 0x7fb4a80caf70 2023-07-14T11:54:01.264 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.280+0000 7fb4ad3bd700 1 --2- 172.21.15.104:0/4133632614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4a80caf70 0x7fb4a815d7f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb4a806aa00 tx=0x7fb4a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d478ab952accedf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:01.264 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.280+0000 7fb49f7fe700 1 -- 172.21.15.104:0/4133632614 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a000e070 con 0x7fb4a80caf70 2023-07-14T11:54:01.264 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.280+0000 7fb49f7fe700 1 -- 172.21.15.104:0/4133632614 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a000bb20 con 0x7fb4a80caf70 2023-07-14T11:54:01.264 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.280+0000 7fb49f7fe700 1 -- 172.21.15.104:0/4133632614 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0012430 con 0x7fb4a80caf70 2023-07-14T11:54:01.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.280+0000 7fb4adbbe700 1 -- 172.21.15.104:0/4133632614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4a80caf70 msgr2=0x7fb4a815d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:01.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.280+0000 7fb4adbbe700 1 --2- 172.21.15.104:0/4133632614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4a80caf70 0x7fb4a815d7f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb4a806aa00 tx=0x7fb4a000b330 comp rx=0 tx=0).stop 2023-07-14T11:54:01.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.280+0000 7fb4adbbe700 1 -- 172.21.15.104:0/4133632614 shutdown_connections 2023-07-14T11:54:01.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.280+0000 7fb4adbbe700 1 --2- 172.21.15.104:0/4133632614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4a80caf70 0x7fb4a815d7f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:01.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.280+0000 7fb4adbbe700 1 --2- 172.21.15.104:0/4133632614 >> v2:172.21.15.121:3300/0 conn(0x7fb4a8159f20 0x7fb4a80caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:01.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.280+0000 7fb4adbbe700 1 -- 172.21.15.104:0/4133632614 >> 172.21.15.104:0/4133632614 conn(0x7fb4a806e960 msgr2=0x7fb4a806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:01.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.280+0000 7fb4adbbe700 1 -- 172.21.15.104:0/4133632614 shutdown_connections 2023-07-14T11:54:01.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.280+0000 7fb4adbbe700 1 -- 172.21.15.104:0/4133632614 wait complete. 2023-07-14T11:54:01.267 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4adbbe700 1 Processor -- start 2023-07-14T11:54:01.267 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4adbbe700 1 -- start start 2023-07-14T11:54:01.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4adbbe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4a80c9d90 0x7fb4a80c3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:01.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4adbbe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4a80caf70 0x7fb4a80b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:01.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4adbbe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4a8159f20 0x7fb4a8157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:01.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4adbbe700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a8152df0 con 0x7fb4a8159f20 2023-07-14T11:54:01.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4adbbe700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a81f2770 con 0x7fb4a80c9d90 2023-07-14T11:54:01.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4adbbe700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a81f2a50 con 0x7fb4a80caf70 2023-07-14T11:54:01.273 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4ad3bd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4a8159f20 0x7fb4a8157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:01.273 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4ad3bd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4a8159f20 0x7fb4a8157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55184/0 (socket says 172.21.15.104:55184) 2023-07-14T11:54:01.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4ad3bd700 1 -- 172.21.15.104:0/4016978662 learned_addr learned my addr 172.21.15.104:0/4016978662 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:01.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4acbbc700 1 --2- 172.21.15.104:0/4016978662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4a80c9d90 0x7fb4a80c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:01.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb49ffff700 1 --2- 172.21.15.104:0/4016978662 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4a80caf70 0x7fb4a80b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:01.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4ad3bd700 1 -- 172.21.15.104:0/4016978662 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4a80caf70 msgr2=0x7fb4a80b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:01.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4ad3bd700 1 --2- 172.21.15.104:0/4016978662 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4a80caf70 0x7fb4a80b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:01.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4ad3bd700 1 -- 172.21.15.104:0/4016978662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4a80c9d90 msgr2=0x7fb4a80c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:01.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4ad3bd700 1 --2- 172.21.15.104:0/4016978662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4a80c9d90 0x7fb4a80c3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:01.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb4ad3bd700 1 -- 172.21.15.104:0/4016978662 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a0009a70 con 0x7fb4a8159f20 2023-07-14T11:54:01.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.284+0000 7fb49ffff700 1 --2- 172.21.15.104:0/4016978662 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4a80caf70 0x7fb4a80b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:01.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.288+0000 7fb4ad3bd700 1 --2- 172.21.15.104:0/4016978662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4a8159f20 0x7fb4a8157e40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb4a806a6f0 tx=0x7fb4a0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:01.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.288+0000 7fb49d7fa700 1 -- 172.21.15.104:0/4016978662 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0012430 con 0x7fb4a8159f20 2023-07-14T11:54:01.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.288+0000 7fb49d7fa700 1 -- 172.21.15.104:0/4016978662 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a0020070 con 0x7fb4a8159f20 2023-07-14T11:54:01.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.288+0000 7fb4adbbe700 1 -- 172.21.15.104:0/4016978662 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a81f2d30 con 0x7fb4a8159f20 2023-07-14T11:54:01.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.288+0000 7fb49d7fa700 1 -- 172.21.15.104:0/4016978662 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0004280 con 0x7fb4a8159f20 2023-07-14T11:54:01.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.288+0000 7fb4adbbe700 1 -- 172.21.15.104:0/4016978662 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a81f32f0 con 0x7fb4a8159f20 2023-07-14T11:54:01.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.288+0000 7fb4adbbe700 1 -- 172.21.15.104:0/4016978662 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a804e500 con 0x7fb4a8159f20 2023-07-14T11:54:01.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.292+0000 7fb49d7fa700 1 -- 172.21.15.104:0/4016978662 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb4a00043e0 con 0x7fb4a8159f20 2023-07-14T11:54:01.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.292+0000 7fb49d7fa700 1 --2- 172.21.15.104:0/4016978662 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb494066320 0x7fb4940687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:01.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.292+0000 7fb49d7fa700 1 -- 172.21.15.104:0/4016978662 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb4a0022070 con 0x7fb4a8159f20 2023-07-14T11:54:01.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.292+0000 7fb4acbbc700 1 --2- 172.21.15.104:0/4016978662 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb494066320 0x7fb4940687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:01.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.292+0000 7fb4acbbc700 1 --2- 172.21.15.104:0/4016978662 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb494066320 0x7fb4940687d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4000f60 tx=0x7fb4a4007e90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:01.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.300+0000 7fb49d7fa700 1 -- 172.21.15.104:0/4016978662 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb4a0052dd0 con 0x7fb4a8159f20 2023-07-14T11:54:01.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.504+0000 7fb4adbbe700 1 -- 172.21.15.104:0/4016978662 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb4a80714b0 con 0x7fb4a8159f20 2023-07-14T11:54:01.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.504+0000 7fb49d7fa700 1 -- 172.21.15.104:0/4016978662 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb4a002c070 con 0x7fb4a8159f20 2023-07-14T11:54:01.490 INFO:teuthology.orchestra.run.smithi104.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-14T11:54:01.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.508+0000 7fb4adbbe700 1 -- 172.21.15.104:0/4016978662 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb494066320 msgr2=0x7fb4940687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:01.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.508+0000 7fb4adbbe700 1 --2- 172.21.15.104:0/4016978662 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb494066320 0x7fb4940687d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4000f60 tx=0x7fb4a4007e90 comp rx=0 tx=0).stop 2023-07-14T11:54:01.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.508+0000 7fb4adbbe700 1 -- 172.21.15.104:0/4016978662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4a8159f20 msgr2=0x7fb4a8157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:01.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.508+0000 7fb4adbbe700 1 --2- 172.21.15.104:0/4016978662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4a8159f20 0x7fb4a8157e40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb4a806a6f0 tx=0x7fb4a0004af0 comp rx=0 tx=0).stop 2023-07-14T11:54:01.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.512+0000 7fb4adbbe700 1 -- 172.21.15.104:0/4016978662 shutdown_connections 2023-07-14T11:54:01.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.512+0000 7fb4adbbe700 1 --2- 172.21.15.104:0/4016978662 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb494066320 0x7fb4940687d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:01.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.512+0000 7fb4adbbe700 1 --2- 172.21.15.104:0/4016978662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4a8159f20 0x7fb4a8157e40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:01.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.512+0000 7fb4adbbe700 1 --2- 172.21.15.104:0/4016978662 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4a80caf70 0x7fb4a80b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:01.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.512+0000 7fb4adbbe700 1 --2- 172.21.15.104:0/4016978662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4a80c9d90 0x7fb4a80c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:01.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.512+0000 7fb4adbbe700 1 -- 172.21.15.104:0/4016978662 >> 172.21.15.104:0/4016978662 conn(0x7fb4a806e960 msgr2=0x7fb4a806d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:01.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.512+0000 7fb4adbbe700 1 -- 172.21.15.104:0/4016978662 shutdown_connections 2023-07-14T11:54:01.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:01.512+0000 7fb4adbbe700 1 -- 172.21.15.104:0/4016978662 wait complete. 2023-07-14T11:54:01.515 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-14T11:54:01.515 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T11:54:01.515 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-07-14T11:54:01.516 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-14T11:54:01.516 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-14T11:54:01.516 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-14T11:54:01.516 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T11:54:01.516 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T11:54:01.517 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-14T11:54:01.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:01.517 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-07-14T11:54:01.537 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:54:01.538 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:54:01.538 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:54:01.538 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:01.538 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-07-14T11:54:01.538 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:01.539 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:54:01.539 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:54:01.539 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-07-14T11:54:01.539 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:54:01.539 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:54:01.539 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-07-14T11:54:01.539 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:54:01.540 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:54:01.540 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T11:54:01.540 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:54:01.540 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-07-14T11:54:01.540 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42344sec preferred_lft 42344sec 2023-07-14T11:54:01.540 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-07-14T11:54:01.540 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:01.541 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-07-14T11:54:01.541 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d2:f6:ad:5d brd ff:ff:ff:ff:ff:ff 2023-07-14T11:54:01.541 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-07-14T11:54:01.541 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:01.542 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-14T11:54:01.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:01.542 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-07-14T11:54:01.543 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-07-14T11:54:01.543 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add name ceph-brx type bridge 2023-07-14T11:54:01.543 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr flush dev ceph-brx 2023-07-14T11:54:01.543 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set ceph-brx up 2023-07-14T11:54:01.544 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-14T11:54:01.544 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-07-14T11:54:01.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:01.697 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T11:54:01.728 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-07-14T11:54:01.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:01.732 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-07-14T11:54:01.815 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-07-14T11:54:01.815 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T11:54:01.815 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-07-14T11:54:01.815 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T11:54:01.815 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-07-14T11:54:01.816 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-07-14T11:54:01.816 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-07-14T11:54:01.816 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T11:54:01.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:01.817 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-07-14T11:54:01.817 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-07-14T11:54:01.817 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T11:54:01.817 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T11:54:01.817 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T11:54:01.818 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-07-14T11:54:01.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:01.950 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-07-14T11:54:01.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:01.965 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list-id 2023-07-14T11:54:02.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:02.027 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-07-14T11:54:02.027 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-07-14T11:54:02.027 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-14T11:54:02.028 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-07-14T11:54:02.028 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-07-14T11:54:02.146 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-07-14T11:54:02.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:02.147 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-07-14T11:54:02.147 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-07-14T11:54:02.147 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-07-14T11:54:02.147 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-14T11:54:02.148 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-07-14T11:54:02.148 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-07-14T11:54:02.148 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-07-14T11:54:02.148 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-07-14T11:54:02.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:02.317 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-07-14T11:54:02.317 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-07-14T11:54:02.318 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-07-14T11:54:02.318 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set dev brx.0 master ceph-brx 2023-07-14T11:54:02.318 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-07-14T11:54:02.391 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-14T11:54:02.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:54:02.391 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-14T11:54:02.407 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-14T11:54:02.408 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:54:02.409 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-14T11:54:02.473 DEBUG:teuthology.orchestra.run.smithi121:> sudo modprobe fuse 2023-07-14T11:54:02.555 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:54:02.555 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:54:02.630 INFO:teuthology.orchestra.run.smithi121.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:54:02.634 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:54:02.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:02.635 DEBUG:teuthology.orchestra.run.smithi121:> ls /sys/fs/fuse/connections 2023-07-14T11:54:02.698 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-14T11:54:02.699 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-14T11:54:02.745 DEBUG:teuthology.orchestra.run.smithi121:> sudo modprobe fuse 2023-07-14T11:54:02.779 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:54:02.779 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:54:02.856 INFO:teuthology.orchestra.run.smithi121.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:54:02.858 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:54:02.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:02.859 DEBUG:teuthology.orchestra.run.smithi121:> ls /sys/fs/fuse/connections 2023-07-14T11:54:02.889 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi121.stderr:2023-07-14T11:54:02.903+0000 7fccae1de640 -1 init, newargv = 0x55cc897d11e0 newargc=15 2023-07-14T11:54:02.889 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi121.stderr:ceph-fuse[39418]: starting ceph client 2023-07-14T11:54:02.901 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi121.stderr:ceph-fuse[39418]: starting fuse 2023-07-14T11:54:02.901 INFO:teuthology.orchestra.run.smithi121.stdout:49 2023-07-14T11:54:02.902 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-07-14T11:54:02.902 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T11:54:02.902 DEBUG:teuthology.orchestra.run.smithi121:> import glob 2023-07-14T11:54:02.903 DEBUG:teuthology.orchestra.run.smithi121:> import re 2023-07-14T11:54:02.903 DEBUG:teuthology.orchestra.run.smithi121:> import os 2023-07-14T11:54:02.903 DEBUG:teuthology.orchestra.run.smithi121:> import subprocess 2023-07-14T11:54:02.903 DEBUG:teuthology.orchestra.run.smithi121:> 2023-07-14T11:54:02.904 DEBUG:teuthology.orchestra.run.smithi121:> def _find_admin_socket(client_name): 2023-07-14T11:54:02.904 DEBUG:teuthology.orchestra.run.smithi121:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-14T11:54:02.904 DEBUG:teuthology.orchestra.run.smithi121:> files = glob.glob(asok_path) 2023-07-14T11:54:02.904 DEBUG:teuthology.orchestra.run.smithi121:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-14T11:54:02.904 DEBUG:teuthology.orchestra.run.smithi121:> 2023-07-14T11:54:02.905 DEBUG:teuthology.orchestra.run.smithi121:> # Given a non-glob path, it better be there 2023-07-14T11:54:02.905 DEBUG:teuthology.orchestra.run.smithi121:> if "*" not in asok_path: 2023-07-14T11:54:02.905 DEBUG:teuthology.orchestra.run.smithi121:> assert(len(files) == 1) 2023-07-14T11:54:02.905 DEBUG:teuthology.orchestra.run.smithi121:> return files[0] 2023-07-14T11:54:02.906 DEBUG:teuthology.orchestra.run.smithi121:> 2023-07-14T11:54:02.906 DEBUG:teuthology.orchestra.run.smithi121:> for f in files: 2023-07-14T11:54:02.906 DEBUG:teuthology.orchestra.run.smithi121:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-14T11:54:02.906 DEBUG:teuthology.orchestra.run.smithi121:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-14T11:54:02.906 DEBUG:teuthology.orchestra.run.smithi121:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-14T11:54:02.907 DEBUG:teuthology.orchestra.run.smithi121:> contents = proc_f.read() 2023-07-14T11:54:02.907 DEBUG:teuthology.orchestra.run.smithi121:> if mountpoint in contents: 2023-07-14T11:54:02.907 DEBUG:teuthology.orchestra.run.smithi121:> return f 2023-07-14T11:54:02.907 DEBUG:teuthology.orchestra.run.smithi121:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-14T11:54:02.908 DEBUG:teuthology.orchestra.run.smithi121:> 2023-07-14T11:54:02.908 DEBUG:teuthology.orchestra.run.smithi121:> print(_find_admin_socket("client.1")) 2023-07-14T11:54:02.908 DEBUG:teuthology.orchestra.run.smithi121:> ' 2023-07-14T11:54:03.062 INFO:teuthology.orchestra.run.smithi121.stdout:/var/run/ceph/ceph-client.1.39418.asok 2023-07-14T11:54:03.246 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39418.asok 2023-07-14T11:54:03.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:03.247 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39418.asok status 2023-07-14T11:54:03.443 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-07-14T11:54:03.444 INFO:teuthology.orchestra.run.smithi121.stdout: "metadata": { 2023-07-14T11:54:03.444 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-14T11:54:03.444 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:54:03.444 INFO:teuthology.orchestra.run.smithi121.stdout: "entity_id": "1", 2023-07-14T11:54:03.444 INFO:teuthology.orchestra.run.smithi121.stdout: "hostname": "smithi121", 2023-07-14T11:54:03.444 INFO:teuthology.orchestra.run.smithi121.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-14T11:54:03.444 INFO:teuthology.orchestra.run.smithi121.stdout: "pid": "39418", 2023-07-14T11:54:03.445 INFO:teuthology.orchestra.run.smithi121.stdout: "root": "/" 2023-07-14T11:54:03.445 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-14T11:54:03.445 INFO:teuthology.orchestra.run.smithi121.stdout: "dentry_count": 0, 2023-07-14T11:54:03.445 INFO:teuthology.orchestra.run.smithi121.stdout: "dentry_pinned_count": 0, 2023-07-14T11:54:03.445 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 4891, 2023-07-14T11:54:03.445 INFO:teuthology.orchestra.run.smithi121.stdout: "inst": { 2023-07-14T11:54:03.445 INFO:teuthology.orchestra.run.smithi121.stdout: "name": { 2023-07-14T11:54:03.445 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "client", 2023-07-14T11:54:03.446 INFO:teuthology.orchestra.run.smithi121.stdout: "num": 4891 2023-07-14T11:54:03.446 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-14T11:54:03.446 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": { 2023-07-14T11:54:03.446 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "v1", 2023-07-14T11:54:03.446 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:54:03.446 INFO:teuthology.orchestra.run.smithi121.stdout: "nonce": 1782400104 2023-07-14T11:54:03.446 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-14T11:54:03.447 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-14T11:54:03.447 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": { 2023-07-14T11:54:03.447 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "v1", 2023-07-14T11:54:03.447 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:54:03.447 INFO:teuthology.orchestra.run.smithi121.stdout: "nonce": 1782400104 2023-07-14T11:54:03.447 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-14T11:54:03.448 INFO:teuthology.orchestra.run.smithi121.stdout: "inst_str": "client.4891 192.168.0.1:0/1782400104", 2023-07-14T11:54:03.448 INFO:teuthology.orchestra.run.smithi121.stdout: "addr_str": "192.168.0.1:0/1782400104", 2023-07-14T11:54:03.448 INFO:teuthology.orchestra.run.smithi121.stdout: "inode_count": 1, 2023-07-14T11:54:03.448 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_epoch": 9, 2023-07-14T11:54:03.448 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_epoch": 30, 2023-07-14T11:54:03.448 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_epoch_barrier": 0, 2023-07-14T11:54:03.448 INFO:teuthology.orchestra.run.smithi121.stdout: "blocklisted": false, 2023-07-14T11:54:03.448 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs" 2023-07-14T11:54:03.449 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-07-14T11:54:03.458 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-14T11:54:03.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.660+0000 7f57bf55d700 1 Processor -- start 2023-07-14T11:54:03.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.660+0000 7f57bf55d700 1 -- start start 2023-07-14T11:54:03.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.660+0000 7f57bf55d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f57b80ce520 0x7f57b80addc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:03.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.660+0000 7f57bf55d700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b80ad930 con 0x7f57b8159cc0 2023-07-14T11:54:03.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.660+0000 7f57bf55d700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b8099b90 con 0x7f57b80cfe00 2023-07-14T11:54:03.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.660+0000 7f57bf55d700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b80cbc60 con 0x7f57b80ce520 2023-07-14T11:54:03.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57be55b700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f57b8159cc0 0x7f57b80cde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:50142/0 (socket says 172.21.15.104:50142) 2023-07-14T11:54:03.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57be55b700 1 -- 172.21.15.104:0/1141075990 learned_addr learned my addr 172.21.15.104:0/1141075990 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:03.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57bdd5a700 1 --2- 172.21.15.104:0/1141075990 >> v2:172.21.15.121:3300/0 conn(0x7f57b80ce520 0x7f57b80addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:03.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57bd559700 1 -- 172.21.15.104:0/1141075990 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978785840 0 0) 0x7f57b80ad930 con 0x7f57b8159cc0 2023-07-14T11:54:03.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57bd559700 1 -- 172.21.15.104:0/1141075990 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a40035d0 con 0x7f57b8159cc0 2023-07-14T11:54:03.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57bd559700 1 -- 172.21.15.104:0/1141075990 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384633492 0 0) 0x7f57b8099b90 con 0x7f57b80cfe00 2023-07-14T11:54:03.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57bd559700 1 -- 172.21.15.104:0/1141075990 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57b80ad930 con 0x7f57b80cfe00 2023-07-14T11:54:03.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57bdd5a700 1 -- 172.21.15.104:0/1141075990 >> v1:172.21.15.121:6789/0 conn(0x7f57b80cfe00 legacy=0x7f57b80ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:03.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57bdd5a700 1 -- 172.21.15.104:0/1141075990 >> v1:172.21.15.104:6789/0 conn(0x7f57b8159cc0 legacy=0x7f57b80cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:03.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57bdd5a700 1 -- 172.21.15.104:0/1141075990 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b815a090 con 0x7f57b80ce520 2023-07-14T11:54:03.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57bdd5a700 1 --2- 172.21.15.104:0/1141075990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57b80ce520 0x7f57b80addc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009850 tx=0x7f57a8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2be9693fa2549b32 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:03.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57bd559700 1 -- 172.21.15.104:0/1141075990 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8011830 con 0x7f57b80ce520 2023-07-14T11:54:03.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57bd559700 1 -- 172.21.15.104:0/1141075990 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a8011990 con 0x7f57b80ce520 2023-07-14T11:54:03.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57bd559700 1 -- 172.21.15.104:0/1141075990 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8011d40 con 0x7f57b80ce520 2023-07-14T11:54:03.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57bf55d700 1 -- 172.21.15.104:0/1141075990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57b80ce520 msgr2=0x7f57b80addc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:03.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.664+0000 7f57bf55d700 1 --2- 172.21.15.104:0/1141075990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57b80ce520 0x7f57b80addc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009850 tx=0x7f57a8009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:03.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57bf55d700 1 -- 172.21.15.104:0/1141075990 shutdown_connections 2023-07-14T11:54:03.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57bf55d700 1 --2- 172.21.15.104:0/1141075990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57b80ce520 0x7f57b80addc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57bf55d700 1 -- 172.21.15.104:0/1141075990 >> 172.21.15.104:0/1141075990 conn(0x7f57b8099600 msgr2=0x7f57b80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:03.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57bf55d700 1 -- 172.21.15.104:0/1141075990 shutdown_connections 2023-07-14T11:54:03.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57bf55d700 1 -- 172.21.15.104:0/1141075990 wait complete. 2023-07-14T11:54:03.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57bf55d700 1 Processor -- start 2023-07-14T11:54:03.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57bf55d700 1 -- start start 2023-07-14T11:54:03.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57bf55d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57b80ce520 0x7f57b8154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:03.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57bf55d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57b80cfe00 0x7f57b815ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:03.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57bf55d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57b8159cc0 0x7f57b80b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:03.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57bf55d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80afc30 con 0x7f57b80ce520 2023-07-14T11:54:03.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57bf55d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80b07b0 con 0x7f57b80cfe00 2023-07-14T11:54:03.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57bf55d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80afe80 con 0x7f57b8159cc0 2023-07-14T11:54:03.655 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57be55b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57b80ce520 0x7f57b8154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:03.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57be55b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57b80ce520 0x7f57b8154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55198/0 (socket says 172.21.15.104:55198) 2023-07-14T11:54:03.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.668+0000 7f57be55b700 1 -- 172.21.15.104:0/2684563571 learned_addr learned my addr 172.21.15.104:0/2684563571 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:03.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.672+0000 7f57bdd5a700 1 --2- 172.21.15.104:0/2684563571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57b80cfe00 0x7f57b815ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:03.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.672+0000 7f57bed5c700 1 --2- 172.21.15.104:0/2684563571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57b8159cc0 0x7f57b80b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:03.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.672+0000 7f57be55b700 1 -- 172.21.15.104:0/2684563571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57b8159cc0 msgr2=0x7f57b80b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:03.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.672+0000 7f57be55b700 1 --2- 172.21.15.104:0/2684563571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57b8159cc0 0x7f57b80b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.672+0000 7f57be55b700 1 -- 172.21.15.104:0/2684563571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57b80cfe00 msgr2=0x7f57b815ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:03.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.672+0000 7f57be55b700 1 --2- 172.21.15.104:0/2684563571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57b80cfe00 0x7f57b815ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.672+0000 7f57be55b700 1 -- 172.21.15.104:0/2684563571 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a80094b0 con 0x7f57b80ce520 2023-07-14T11:54:03.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.672+0000 7f57be55b700 1 --2- 172.21.15.104:0/2684563571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57b80ce520 0x7f57b8154360 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f57b4001eb0 tx=0x7f57b400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:03.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.672+0000 7f57aeffd700 1 -- 172.21.15.104:0/2684563571 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b40074e0 con 0x7f57b80ce520 2023-07-14T11:54:03.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.672+0000 7f57aeffd700 1 -- 172.21.15.104:0/2684563571 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b4007640 con 0x7f57b80ce520 2023-07-14T11:54:03.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.672+0000 7f57bf55d700 1 -- 172.21.15.104:0/2684563571 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b80b0160 con 0x7f57b80ce520 2023-07-14T11:54:03.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.672+0000 7f57aeffd700 1 -- 172.21.15.104:0/2684563571 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4005a50 con 0x7f57b80ce520 2023-07-14T11:54:03.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.672+0000 7f57bf55d700 1 -- 172.21.15.104:0/2684563571 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b80b52d0 con 0x7f57b80ce520 2023-07-14T11:54:03.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.672+0000 7f57bf55d700 1 -- 172.21.15.104:0/2684563571 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b80b5c30 con 0x7f57b80ce520 2023-07-14T11:54:03.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.672+0000 7f57aeffd700 1 -- 172.21.15.104:0/2684563571 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f57b4005bb0 con 0x7f57b80ce520 2023-07-14T11:54:03.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.676+0000 7f57aeffd700 1 --2- 172.21.15.104:0/2684563571 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f579c0662f0 0x7f579c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:03.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.676+0000 7f57aeffd700 1 -- 172.21.15.104:0/2684563571 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f57b4083340 con 0x7f57b80ce520 2023-07-14T11:54:03.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.676+0000 7f57bdd5a700 1 --2- 172.21.15.104:0/2684563571 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f579c0662f0 0x7f579c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:03.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.676+0000 7f57bdd5a700 1 --2- 172.21.15.104:0/2684563571 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f579c0662f0 0x7f579c0687a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f57a80090b0 tx=0x7f57a8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:03.664 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.680+0000 7f57aeffd700 1 -- 172.21.15.104:0/2684563571 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f57b4050540 con 0x7f57b80ce520 2023-07-14T11:54:03.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.880+0000 7f57bf55d700 1 -- 172.21.15.104:0/2684563571 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f57b804e500 con 0x7f57b80ce520 2023-07-14T11:54:03.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.884+0000 7f57aeffd700 1 -- 172.21.15.104:0/2684563571 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f57b4017070 con 0x7f57b80ce520 2023-07-14T11:54:03.867 INFO:teuthology.orchestra.run.smithi104.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-14T11:54:03.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.888+0000 7f57bf55d700 1 -- 172.21.15.104:0/2684563571 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f579c0662f0 msgr2=0x7f579c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:03.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.888+0000 7f57bf55d700 1 --2- 172.21.15.104:0/2684563571 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f579c0662f0 0x7f579c0687a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f57a80090b0 tx=0x7f57a8008030 comp rx=0 tx=0).stop 2023-07-14T11:54:03.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.888+0000 7f57bf55d700 1 -- 172.21.15.104:0/2684563571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57b80ce520 msgr2=0x7f57b8154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:03.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.888+0000 7f57bf55d700 1 --2- 172.21.15.104:0/2684563571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57b80ce520 0x7f57b8154360 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f57b4001eb0 tx=0x7f57b400c7b0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.888+0000 7f57bf55d700 1 -- 172.21.15.104:0/2684563571 shutdown_connections 2023-07-14T11:54:03.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.888+0000 7f57bf55d700 1 --2- 172.21.15.104:0/2684563571 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f579c0662f0 0x7f579c0687a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.888+0000 7f57bf55d700 1 --2- 172.21.15.104:0/2684563571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57b8159cc0 0x7f57b80b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.888+0000 7f57bf55d700 1 --2- 172.21.15.104:0/2684563571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57b80cfe00 0x7f57b815ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.888+0000 7f57bf55d700 1 --2- 172.21.15.104:0/2684563571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57b80ce520 0x7f57b8154360 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.888+0000 7f57bf55d700 1 -- 172.21.15.104:0/2684563571 >> 172.21.15.104:0/2684563571 conn(0x7f57b8099600 msgr2=0x7f57b806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:03.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.888+0000 7f57bf55d700 1 -- 172.21.15.104:0/2684563571 shutdown_connections 2023-07-14T11:54:03.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:03.888+0000 7f57bf55d700 1 -- 172.21.15.104:0/2684563571 wait complete. 2023-07-14T11:54:03.892 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-14T11:54:03.892 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T11:54:03.892 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-07-14T11:54:03.892 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-07-14T11:54:03.893 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-14T11:54:03.893 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-14T11:54:03.893 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T11:54:03.893 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T11:54:03.894 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-07-14T11:54:03.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:03.894 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-07-14T11:54:03.911 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:54:03.911 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:54:03.911 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:54:03.911 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:03.911 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-07-14T11:54:03.912 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:03.912 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:54:03.912 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:54:03.912 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-07-14T11:54:03.912 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:54:03.912 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:54:03.912 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-07-14T11:54:03.912 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:54:03.913 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:54:03.913 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T11:54:03.913 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:54:03.913 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-07-14T11:54:03.913 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42342sec preferred_lft 42342sec 2023-07-14T11:54:03.913 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-07-14T11:54:03.913 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:03.913 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-07-14T11:54:03.914 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d2:f6:ad:5d brd ff:ff:ff:ff:ff:ff 2023-07-14T11:54:03.914 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-07-14T11:54:03.914 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:03.914 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-14T11:54:03.914 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 9e:02:6a:77:df:fb brd ff:ff:ff:ff:ff:ff 2023-07-14T11:54:03.914 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-14T11:54:03.914 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:03.914 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::d07b:d7ff:fe77:a7e4/64 scope link 2023-07-14T11:54:03.915 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:03.915 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-14T11:54:03.915 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 9e:02:6a:77:df:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-14T11:54:03.915 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::9c02:6aff:fe77:dffb/64 scope link tentative 2023-07-14T11:54:03.915 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:03.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:03.916 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T11:54:03.996 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-07-14T11:54:03.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:04.000 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-07-14T11:54:04.062 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-07-14T11:54:04.062 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T11:54:04.062 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-07-14T11:54:04.062 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T11:54:04.062 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-07-14T11:54:04.063 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-07-14T11:54:04.063 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-07-14T11:54:04.063 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T11:54:04.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:04.064 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-07-14T11:54:04.064 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-07-14T11:54:04.065 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T11:54:04.065 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T11:54:04.065 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T11:54:04.065 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-07-14T11:54:04.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:04.200 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-07-14T11:54:04.213 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-14T11:54:04.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:04.215 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list-id 2023-07-14T11:54:04.273 INFO:teuthology.orchestra.run.smithi121.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-07-14T11:54:04.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:04.275 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-07-14T11:54:04.275 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-07-14T11:54:04.275 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-14T11:54:04.276 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-07-14T11:54:04.276 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-07-14T11:54:04.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:04.388 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-14T11:54:04.423 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:54:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:54:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:54:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-07-14T11:54:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:04.424 INFO:teuthology.orchestra.run.smithi121.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-14T11:54:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0a:ef:9c:5a:10:f5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-14T11:54:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-14T11:54:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::8ef:9cff:fe5a:10f5/64 scope link 2023-07-14T11:54:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:04.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:04.426 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-14T11:54:04.503 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:54:04.504 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:54:04.504 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:54:04.504 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:04.504 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-07-14T11:54:04.504 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:04.505 INFO:teuthology.orchestra.run.smithi121.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-14T11:54:04.505 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0a:ef:9c:5a:10:f5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-14T11:54:04.505 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-14T11:54:04.505 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:04.505 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::8ef:9cff:fe5a:10f5/64 scope link 2023-07-14T11:54:04.505 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:54:04.506 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-07-14T11:54:04.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:04.507 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-07-14T11:54:04.508 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-07-14T11:54:04.508 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-07-14T11:54:04.508 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-14T11:54:04.509 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-07-14T11:54:04.509 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-07-14T11:54:04.509 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-07-14T11:54:04.509 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-07-14T11:54:04.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:04.703 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-07-14T11:54:04.703 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-07-14T11:54:04.704 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.1 up 2023-07-14T11:54:04.704 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set dev brx.1 master ceph-brx 2023-07-14T11:54:04.704 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-07-14T11:54:04.774 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-07-14T11:54:04.775 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:54:04.775 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-07-14T11:54:04.793 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-07-14T11:54:04.793 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:54:04.794 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-07-14T11:54:04.851 DEBUG:teuthology.orchestra.run.smithi121:> sudo modprobe fuse 2023-07-14T11:54:04.928 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:54:04.928 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:54:05.006 INFO:teuthology.orchestra.run.smithi121.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:54:05.009 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:54:05.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:05.010 DEBUG:teuthology.orchestra.run.smithi121:> ls /sys/fs/fuse/connections 2023-07-14T11:54:05.070 INFO:teuthology.orchestra.run.smithi121.stdout:49 2023-07-14T11:54:05.070 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [49] 2023-07-14T11:54:05.071 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-07-14T11:54:05.122 DEBUG:teuthology.orchestra.run.smithi121:> sudo modprobe fuse 2023-07-14T11:54:05.154 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:54:05.155 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:54:05.233 INFO:teuthology.orchestra.run.smithi121.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:54:05.235 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:54:05.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:05.236 DEBUG:teuthology.orchestra.run.smithi121:> ls /sys/fs/fuse/connections 2023-07-14T11:54:05.262 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi121.stderr:2023-07-14T11:54:05.275+0000 7f65b65ae640 -1 init, newargv = 0x55cb35e4bcf0 newargc=15 2023-07-14T11:54:05.262 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi121.stderr:ceph-fuse[39527]: starting ceph client 2023-07-14T11:54:05.273 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi121.stderr:ceph-fuse[39527]: starting fuse 2023-07-14T11:54:05.277 INFO:teuthology.orchestra.run.smithi121.stdout:49 2023-07-14T11:54:05.277 INFO:teuthology.orchestra.run.smithi121.stdout:50 2023-07-14T11:54:05.278 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49, 50] 2023-07-14T11:54:05.278 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T11:54:05.278 DEBUG:teuthology.orchestra.run.smithi121:> import glob 2023-07-14T11:54:05.278 DEBUG:teuthology.orchestra.run.smithi121:> import re 2023-07-14T11:54:05.279 DEBUG:teuthology.orchestra.run.smithi121:> import os 2023-07-14T11:54:05.279 DEBUG:teuthology.orchestra.run.smithi121:> import subprocess 2023-07-14T11:54:05.279 DEBUG:teuthology.orchestra.run.smithi121:> 2023-07-14T11:54:05.279 DEBUG:teuthology.orchestra.run.smithi121:> def _find_admin_socket(client_name): 2023-07-14T11:54:05.280 DEBUG:teuthology.orchestra.run.smithi121:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-07-14T11:54:05.280 DEBUG:teuthology.orchestra.run.smithi121:> files = glob.glob(asok_path) 2023-07-14T11:54:05.280 DEBUG:teuthology.orchestra.run.smithi121:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-07-14T11:54:05.280 DEBUG:teuthology.orchestra.run.smithi121:> 2023-07-14T11:54:05.281 DEBUG:teuthology.orchestra.run.smithi121:> # Given a non-glob path, it better be there 2023-07-14T11:54:05.281 DEBUG:teuthology.orchestra.run.smithi121:> if "*" not in asok_path: 2023-07-14T11:54:05.281 DEBUG:teuthology.orchestra.run.smithi121:> assert(len(files) == 1) 2023-07-14T11:54:05.281 DEBUG:teuthology.orchestra.run.smithi121:> return files[0] 2023-07-14T11:54:05.282 DEBUG:teuthology.orchestra.run.smithi121:> 2023-07-14T11:54:05.282 DEBUG:teuthology.orchestra.run.smithi121:> for f in files: 2023-07-14T11:54:05.282 DEBUG:teuthology.orchestra.run.smithi121:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-14T11:54:05.282 DEBUG:teuthology.orchestra.run.smithi121:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-14T11:54:05.282 DEBUG:teuthology.orchestra.run.smithi121:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-14T11:54:05.283 DEBUG:teuthology.orchestra.run.smithi121:> contents = proc_f.read() 2023-07-14T11:54:05.283 DEBUG:teuthology.orchestra.run.smithi121:> if mountpoint in contents: 2023-07-14T11:54:05.283 DEBUG:teuthology.orchestra.run.smithi121:> return f 2023-07-14T11:54:05.283 DEBUG:teuthology.orchestra.run.smithi121:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-14T11:54:05.284 DEBUG:teuthology.orchestra.run.smithi121:> 2023-07-14T11:54:05.284 DEBUG:teuthology.orchestra.run.smithi121:> print(_find_admin_socket("client.3")) 2023-07-14T11:54:05.284 DEBUG:teuthology.orchestra.run.smithi121:> ' 2023-07-14T11:54:05.436 INFO:teuthology.orchestra.run.smithi121.stdout:/var/run/ceph/ceph-client.3.39527.asok 2023-07-14T11:54:05.620 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.39527.asok 2023-07-14T11:54:05.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:05.621 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.39527.asok status 2023-07-14T11:54:05.813 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-07-14T11:54:05.814 INFO:teuthology.orchestra.run.smithi121.stdout: "metadata": { 2023-07-14T11:54:05.814 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-14T11:54:05.814 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:54:05.814 INFO:teuthology.orchestra.run.smithi121.stdout: "entity_id": "3", 2023-07-14T11:54:05.814 INFO:teuthology.orchestra.run.smithi121.stdout: "hostname": "smithi121", 2023-07-14T11:54:05.815 INFO:teuthology.orchestra.run.smithi121.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-07-14T11:54:05.815 INFO:teuthology.orchestra.run.smithi121.stdout: "pid": "39527", 2023-07-14T11:54:05.815 INFO:teuthology.orchestra.run.smithi121.stdout: "root": "/" 2023-07-14T11:54:05.815 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-14T11:54:05.815 INFO:teuthology.orchestra.run.smithi121.stdout: "dentry_count": 0, 2023-07-14T11:54:05.815 INFO:teuthology.orchestra.run.smithi121.stdout: "dentry_pinned_count": 0, 2023-07-14T11:54:05.815 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 4691, 2023-07-14T11:54:05.816 INFO:teuthology.orchestra.run.smithi121.stdout: "inst": { 2023-07-14T11:54:05.816 INFO:teuthology.orchestra.run.smithi121.stdout: "name": { 2023-07-14T11:54:05.816 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "client", 2023-07-14T11:54:05.816 INFO:teuthology.orchestra.run.smithi121.stdout: "num": 4691 2023-07-14T11:54:05.816 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-14T11:54:05.816 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": { 2023-07-14T11:54:05.816 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "v1", 2023-07-14T11:54:05.817 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": "192.168.0.2:0", 2023-07-14T11:54:05.817 INFO:teuthology.orchestra.run.smithi121.stdout: "nonce": 2231923714 2023-07-14T11:54:05.817 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-14T11:54:05.817 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-14T11:54:05.817 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": { 2023-07-14T11:54:05.817 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "v1", 2023-07-14T11:54:05.817 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": "192.168.0.2:0", 2023-07-14T11:54:05.818 INFO:teuthology.orchestra.run.smithi121.stdout: "nonce": 2231923714 2023-07-14T11:54:05.818 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-14T11:54:05.818 INFO:teuthology.orchestra.run.smithi121.stdout: "inst_str": "client.4691 192.168.0.2:0/2231923714", 2023-07-14T11:54:05.818 INFO:teuthology.orchestra.run.smithi121.stdout: "addr_str": "192.168.0.2:0/2231923714", 2023-07-14T11:54:05.818 INFO:teuthology.orchestra.run.smithi121.stdout: "inode_count": 1, 2023-07-14T11:54:05.818 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_epoch": 9, 2023-07-14T11:54:05.818 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_epoch": 30, 2023-07-14T11:54:05.819 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_epoch_barrier": 0, 2023-07-14T11:54:05.819 INFO:teuthology.orchestra.run.smithi121.stdout: "blocklisted": false, 2023-07-14T11:54:05.819 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs" 2023-07-14T11:54:05.819 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-07-14T11:54:05.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:05.828 DEBUG:teuthology.orchestra.run.smithi104:> stat --file-system '--printf=%T 2023-07-14T11:54:05.829 DEBUG:teuthology.orchestra.run.smithi104:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-14T11:54:05.849 INFO:teuthology.orchestra.run.smithi104.stdout:fuseblk 2023-07-14T11:54:05.850 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-14T11:54:05.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:05.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:54:05.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:05.923 DEBUG:teuthology.orchestra.run.smithi104:> stat --file-system '--printf=%T 2023-07-14T11:54:05.924 DEBUG:teuthology.orchestra.run.smithi104:> ' -- /home/ubuntu/cephtest/mnt.2 2023-07-14T11:54:05.990 INFO:teuthology.orchestra.run.smithi104.stdout:fuseblk 2023-07-14T11:54:05.990 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-07-14T11:54:05.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:05.991 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-07-14T11:54:06.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:06.069 DEBUG:teuthology.orchestra.run.smithi121:> stat --file-system '--printf=%T 2023-07-14T11:54:06.069 DEBUG:teuthology.orchestra.run.smithi121:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-14T11:54:06.087 INFO:teuthology.orchestra.run.smithi121.stdout:fuseblk 2023-07-14T11:54:06.088 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-14T11:54:06.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:06.089 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-14T11:54:06.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:06.166 DEBUG:teuthology.orchestra.run.smithi121:> stat --file-system '--printf=%T 2023-07-14T11:54:06.166 DEBUG:teuthology.orchestra.run.smithi121:> ' -- /home/ubuntu/cephtest/mnt.3 2023-07-14T11:54:06.226 INFO:teuthology.orchestra.run.smithi121.stdout:fuseblk 2023-07-14T11:54:06.227 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-07-14T11:54:06.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:06.227 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-07-14T11:54:06.301 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-07-14T11:54:06.319 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:06.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.512+0000 7f9cde59e700 1 Processor -- start 2023-07-14T11:54:06.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.512+0000 7f9cde59e700 1 -- start start 2023-07-14T11:54:06.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.512+0000 7f9cde59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9cd815ffa0 0x7f9cd806d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.512+0000 7f9cde59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806dc50 0x7f9cd80caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.512+0000 7f9cde59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd80ca8a0 con 0x7f9cd8160370 2023-07-14T11:54:06.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.512+0000 7f9cde59e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd80cbcd0 con 0x7f9cd815ffa0 2023-07-14T11:54:06.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.512+0000 7f9cde59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd80cbf70 con 0x7f9cd806dc50 2023-07-14T11:54:06.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cdd59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9cd815ffa0 0x7f9cd806d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cdd59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9cd815ffa0 0x7f9cd806d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55214/0 (socket says 172.21.15.104:55214) 2023-07-14T11:54:06.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cdd59c700 1 -- 172.21.15.104:0/138448265 learned_addr learned my addr 172.21.15.104:0/138448265 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:06.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cdcd9b700 1 --2- 172.21.15.104:0/138448265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806dc50 0x7f9cd80caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cdd59c700 1 -- 172.21.15.104:0/138448265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806dc50 msgr2=0x7f9cd80caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:06.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cdd59c700 1 --2- 172.21.15.104:0/138448265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806dc50 0x7f9cd80caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cdd59c700 1 -- 172.21.15.104:0/138448265 >> v1:172.21.15.121:6789/0 conn(0x7f9cd8160370 legacy=0x7f9cd80ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:06.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cdd59c700 1 -- 172.21.15.104:0/138448265 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd80cc250 con 0x7f9cd815ffa0 2023-07-14T11:54:06.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cdd59c700 1 --2- 172.21.15.104:0/138448265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cd815ffa0 0x7f9cd806d710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009910 tx=0x7f9cd4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbbb3910526713f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:06.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9ccffff700 1 -- 172.21.15.104:0/138448265 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd400ba60 con 0x7f9cd815ffa0 2023-07-14T11:54:06.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9ccffff700 1 -- 172.21.15.104:0/138448265 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd400bbc0 con 0x7f9cd815ffa0 2023-07-14T11:54:06.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9ccffff700 1 -- 172.21.15.104:0/138448265 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4009e60 con 0x7f9cd815ffa0 2023-07-14T11:54:06.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cde59e700 1 -- 172.21.15.104:0/138448265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cd815ffa0 msgr2=0x7f9cd806d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:06.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cde59e700 1 --2- 172.21.15.104:0/138448265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cd815ffa0 0x7f9cd806d710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009910 tx=0x7f9cd4009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:06.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cde59e700 1 -- 172.21.15.104:0/138448265 shutdown_connections 2023-07-14T11:54:06.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cde59e700 1 --2- 172.21.15.104:0/138448265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806dc50 0x7f9cd80caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cde59e700 1 --2- 172.21.15.104:0/138448265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cd815ffa0 0x7f9cd806d710 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cde59e700 1 -- 172.21.15.104:0/138448265 >> 172.21.15.104:0/138448265 conn(0x7f9cd806e960 msgr2=0x7f9cd806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:06.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cde59e700 1 -- 172.21.15.104:0/138448265 shutdown_connections 2023-07-14T11:54:06.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.516+0000 7f9cde59e700 1 -- 172.21.15.104:0/138448265 wait complete. 2023-07-14T11:54:06.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.520+0000 7f9cde59e700 1 Processor -- start 2023-07-14T11:54:06.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.520+0000 7f9cde59e700 1 -- start start 2023-07-14T11:54:06.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.520+0000 7f9cde59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806dc50 0x7f9cd80c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.520+0000 7f9cde59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd815ffa0 0x7f9cd80c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.520+0000 7f9cde59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cd8160370 0x7f9cd81f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.520+0000 7f9cde59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd80b09d0 con 0x7f9cd8160370 2023-07-14T11:54:06.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.520+0000 7f9cde59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd8155ec0 con 0x7f9cd815ffa0 2023-07-14T11:54:06.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.520+0000 7f9cde59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd81f7040 con 0x7f9cd806dc50 2023-07-14T11:54:06.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9cddd9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cd8160370 0x7f9cd81f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9cdd59c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806dc50 0x7f9cd80c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9cddd9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cd8160370 0x7f9cd81f8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55226/0 (socket says 172.21.15.104:55226) 2023-07-14T11:54:06.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9cddd9d700 1 -- 172.21.15.104:0/2848884291 learned_addr learned my addr 172.21.15.104:0/2848884291 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:06.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9cdcd9b700 1 --2- 172.21.15.104:0/2848884291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd815ffa0 0x7f9cd80c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9cddd9d700 1 -- 172.21.15.104:0/2848884291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806dc50 msgr2=0x7f9cd80c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:06.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9cddd9d700 1 --2- 172.21.15.104:0/2848884291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806dc50 0x7f9cd80c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9cddd9d700 1 -- 172.21.15.104:0/2848884291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd815ffa0 msgr2=0x7f9cd80c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:06.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9cddd9d700 1 --2- 172.21.15.104:0/2848884291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd815ffa0 0x7f9cd80c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9cddd9d700 1 -- 172.21.15.104:0/2848884291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd4009570 con 0x7f9cd8160370 2023-07-14T11:54:06.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9cdd59c700 1 --2- 172.21.15.104:0/2848884291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806dc50 0x7f9cd80c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:06.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9cddd9d700 1 --2- 172.21.15.104:0/2848884291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cd8160370 0x7f9cd81f8f60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000cc50 tx=0x7f9cd000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:06.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9ccdffb700 1 -- 172.21.15.104:0/2848884291 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0003c20 con 0x7f9cd8160370 2023-07-14T11:54:06.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9ccdffb700 1 -- 172.21.15.104:0/2848884291 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd0003d80 con 0x7f9cd8160370 2023-07-14T11:54:06.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9ccdffb700 1 -- 172.21.15.104:0/2848884291 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd000aab0 con 0x7f9cd8160370 2023-07-14T11:54:06.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9cde59e700 1 -- 172.21.15.104:0/2848884291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd80c5c20 con 0x7f9cd8160370 2023-07-14T11:54:06.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.524+0000 7f9cde59e700 1 -- 172.21.15.104:0/2848884291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd80b06f0 con 0x7f9cd8160370 2023-07-14T11:54:06.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.528+0000 7f9cde59e700 1 -- 172.21.15.104:0/2848884291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd8071620 con 0x7f9cd8160370 2023-07-14T11:54:06.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.528+0000 7f9ccdffb700 1 -- 172.21.15.104:0/2848884291 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9cd000ac10 con 0x7f9cd8160370 2023-07-14T11:54:06.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.528+0000 7f9ccdffb700 1 --2- 172.21.15.104:0/2848884291 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cbc0662f0 0x7f9cbc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.528+0000 7f9ccdffb700 1 -- 172.21.15.104:0/2848884291 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9cd0083dd0 con 0x7f9cd8160370 2023-07-14T11:54:06.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.528+0000 7f9cdd59c700 1 --2- 172.21.15.104:0/2848884291 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cbc0662f0 0x7f9cbc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.532+0000 7f9cdd59c700 1 --2- 172.21.15.104:0/2848884291 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cbc0662f0 0x7f9cbc0687a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4000c00 tx=0x7f9cd4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:06.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.536+0000 7f9ccdffb700 1 -- 172.21.15.104:0/2848884291 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f9cd0051060 con 0x7f9cd8160370 2023-07-14T11:54:06.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.712+0000 7f9cde59e700 1 -- 172.21.15.104:0/2848884291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cd80ca8a0 con 0x7f9cd8160370 2023-07-14T11:54:06.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.712+0000 7f9ccdffb700 1 -- 172.21.15.104:0/2848884291 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20251 (secure 0 0 0) 0x7f9cd80ca8a0 con 0x7f9cd8160370 2023-07-14T11:54:06.698 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:06.699 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":30,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:55.601783+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:06.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.716+0000 7f9cde59e700 1 -- 172.21.15.104:0/2848884291 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cbc0662f0 msgr2=0x7f9cbc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:06.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.716+0000 7f9cde59e700 1 --2- 172.21.15.104:0/2848884291 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cbc0662f0 0x7f9cbc0687a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4000c00 tx=0x7f9cd4008030 comp rx=0 tx=0).stop 2023-07-14T11:54:06.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.716+0000 7f9cde59e700 1 -- 172.21.15.104:0/2848884291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cd8160370 msgr2=0x7f9cd81f8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:06.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.716+0000 7f9cde59e700 1 --2- 172.21.15.104:0/2848884291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cd8160370 0x7f9cd81f8f60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000cc50 tx=0x7f9cd000cf60 comp rx=0 tx=0).stop 2023-07-14T11:54:06.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.716+0000 7f9cde59e700 1 -- 172.21.15.104:0/2848884291 shutdown_connections 2023-07-14T11:54:06.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.716+0000 7f9cde59e700 1 --2- 172.21.15.104:0/2848884291 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cbc0662f0 0x7f9cbc0687a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.716+0000 7f9cde59e700 1 --2- 172.21.15.104:0/2848884291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cd8160370 0x7f9cd81f8f60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.716+0000 7f9cde59e700 1 --2- 172.21.15.104:0/2848884291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd815ffa0 0x7f9cd80c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.716+0000 7f9cde59e700 1 --2- 172.21.15.104:0/2848884291 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cd806dc50 0x7f9cd80c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.716+0000 7f9cde59e700 1 -- 172.21.15.104:0/2848884291 >> 172.21.15.104:0/2848884291 conn(0x7f9cd806e960 msgr2=0x7f9cd80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:06.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.716+0000 7f9cde59e700 1 -- 172.21.15.104:0/2848884291 shutdown_connections 2023-07-14T11:54:06.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.716+0000 7f9cde59e700 1 -- 172.21.15.104:0/2848884291 wait complete. 2023-07-14T11:54:06.727 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:54:06.728 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:54:06.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.916+0000 7ffbb971f700 1 Processor -- start 2023-07-14T11:54:06.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.916+0000 7ffbb971f700 1 -- start start 2023-07-14T11:54:06.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.920+0000 7ffbb971f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffbb40b0ee0 0x7ffbb40ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.920+0000 7ffbb971f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ffbb40d07e0 0x7ffbb40af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.920+0000 7ffbb971f700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb40cd4e0 con 0x7ffbb40cd090 2023-07-14T11:54:06.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.920+0000 7ffbb971f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb40d0c60 con 0x7ffbb40d07e0 2023-07-14T11:54:06.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.920+0000 7ffbb971f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb40d1a60 con 0x7ffbb40b0ee0 2023-07-14T11:54:06.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.920+0000 7ffbb8f1e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ffbb40d07e0 0x7ffbb40af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.920+0000 7ffbb8f1e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ffbb40d07e0 0x7ffbb40af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55236/0 (socket says 172.21.15.104:55236) 2023-07-14T11:54:06.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.920+0000 7ffbb8f1e700 1 -- 172.21.15.104:0/2547786363 learned_addr learned my addr 172.21.15.104:0/2547786363 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:06.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.920+0000 7ffbb3fff700 1 --2- 172.21.15.104:0/2547786363 >> v2:172.21.15.121:3300/0 conn(0x7ffbb40b0ee0 0x7ffbb40ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.920+0000 7ffbb8f1e700 1 -- 172.21.15.104:0/2547786363 >> v2:172.21.15.121:3300/0 conn(0x7ffbb40b0ee0 msgr2=0x7ffbb40ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:06.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.920+0000 7ffbb8f1e700 1 --2- 172.21.15.104:0/2547786363 >> v2:172.21.15.121:3300/0 conn(0x7ffbb40b0ee0 0x7ffbb40ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.920+0000 7ffbb8f1e700 1 -- 172.21.15.104:0/2547786363 >> v1:172.21.15.104:6789/0 conn(0x7ffbb40cd090 legacy=0x7ffbb40d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:06.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.920+0000 7ffbb8f1e700 1 -- 172.21.15.104:0/2547786363 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb40d1ef0 con 0x7ffbb40d07e0 2023-07-14T11:54:06.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.920+0000 7ffbb8f1e700 1 --2- 172.21.15.104:0/2547786363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb40d07e0 0x7ffbb40af0c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffbb408b380 tx=0x7ffbac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1390e418cc9e657 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:06.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.920+0000 7ffbb2ffd700 1 -- 172.21.15.104:0/2547786363 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac00e070 con 0x7ffbb40d07e0 2023-07-14T11:54:06.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.924+0000 7ffbb2ffd700 1 -- 172.21.15.104:0/2547786363 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac00b910 con 0x7ffbb40d07e0 2023-07-14T11:54:06.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.924+0000 7ffbb2ffd700 1 -- 172.21.15.104:0/2547786363 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac00bcc0 con 0x7ffbb40d07e0 2023-07-14T11:54:06.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.924+0000 7ffbb971f700 1 -- 172.21.15.104:0/2547786363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb40d07e0 msgr2=0x7ffbb40af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:06.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.924+0000 7ffbb971f700 1 --2- 172.21.15.104:0/2547786363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb40d07e0 0x7ffbb40af0c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffbb408b380 tx=0x7ffbac00b330 comp rx=0 tx=0).stop 2023-07-14T11:54:06.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.924+0000 7ffbb971f700 1 -- 172.21.15.104:0/2547786363 shutdown_connections 2023-07-14T11:54:06.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.924+0000 7ffbb971f700 1 --2- 172.21.15.104:0/2547786363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb40d07e0 0x7ffbb40af0c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.924+0000 7ffbb971f700 1 --2- 172.21.15.104:0/2547786363 >> v2:172.21.15.121:3300/0 conn(0x7ffbb40b0ee0 0x7ffbb40ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.924+0000 7ffbb971f700 1 -- 172.21.15.104:0/2547786363 >> 172.21.15.104:0/2547786363 conn(0x7ffbb4060c40 msgr2=0x7ffbb409b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:06.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.924+0000 7ffbb971f700 1 -- 172.21.15.104:0/2547786363 shutdown_connections 2023-07-14T11:54:06.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.924+0000 7ffbb971f700 1 -- 172.21.15.104:0/2547786363 wait complete. 2023-07-14T11:54:06.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.924+0000 7ffbb971f700 1 Processor -- start 2023-07-14T11:54:06.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb971f700 1 -- start start 2023-07-14T11:54:06.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb971f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40b0ee0 0x7ffbb40c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb971f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb40cd090 0x7ffbb40b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb971f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb40d07e0 0x7ffbb40c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb971f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb41595c0 con 0x7ffbb40cd090 2023-07-14T11:54:06.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb971f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40c95d0 con 0x7ffbb40d07e0 2023-07-14T11:54:06.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb971f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40c98b0 con 0x7ffbb40b0ee0 2023-07-14T11:54:06.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb40cd090 0x7ffbb40b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb40cd090 0x7ffbb40b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55248/0 (socket says 172.21.15.104:55248) 2023-07-14T11:54:06.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb37fe700 1 -- 172.21.15.104:0/2448989480 learned_addr learned my addr 172.21.15.104:0/2448989480 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:06.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb8f1e700 1 --2- 172.21.15.104:0/2448989480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb40d07e0 0x7ffbb40c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb3fff700 1 --2- 172.21.15.104:0/2448989480 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40b0ee0 0x7ffbb40c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb37fe700 1 -- 172.21.15.104:0/2448989480 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40b0ee0 msgr2=0x7ffbb40c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:06.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb37fe700 1 --2- 172.21.15.104:0/2448989480 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40b0ee0 0x7ffbb40c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb37fe700 1 -- 172.21.15.104:0/2448989480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb40d07e0 msgr2=0x7ffbb40c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:06.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb37fe700 1 --2- 172.21.15.104:0/2448989480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb40d07e0 0x7ffbb40c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb37fe700 1 -- 172.21.15.104:0/2448989480 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac009b30 con 0x7ffbb40cd090 2023-07-14T11:54:06.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb3fff700 1 --2- 172.21.15.104:0/2448989480 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40b0ee0 0x7ffbb40c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:06.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb37fe700 1 --2- 172.21.15.104:0/2448989480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb40cd090 0x7ffbb40b2500 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffba800c530 tx=0x7ffba800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:06.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb0ff9700 1 -- 172.21.15.104:0/2448989480 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba80052d0 con 0x7ffbb40cd090 2023-07-14T11:54:06.916 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb0ff9700 1 -- 172.21.15.104:0/2448989480 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba8005430 con 0x7ffbb40cd090 2023-07-14T11:54:06.916 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb0ff9700 1 -- 172.21.15.104:0/2448989480 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba80055a0 con 0x7ffbb40cd090 2023-07-14T11:54:06.916 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb971f700 1 -- 172.21.15.104:0/2448989480 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb40b2a40 con 0x7ffbb40cd090 2023-07-14T11:54:06.916 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.928+0000 7ffbb971f700 1 -- 172.21.15.104:0/2448989480 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb40b2f80 con 0x7ffbb40cd090 2023-07-14T11:54:06.916 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.932+0000 7ffbb971f700 1 -- 172.21.15.104:0/2448989480 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb404e500 con 0x7ffbb40cd090 2023-07-14T11:54:06.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.932+0000 7ffbb0ff9700 1 -- 172.21.15.104:0/2448989480 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ffba80038e0 con 0x7ffbb40cd090 2023-07-14T11:54:06.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.932+0000 7ffbb0ff9700 1 --2- 172.21.15.104:0/2448989480 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffba00662f0 0x7ffba00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.936+0000 7ffbb0ff9700 1 -- 172.21.15.104:0/2448989480 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ffba80833b0 con 0x7ffbb40cd090 2023-07-14T11:54:06.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.936+0000 7ffbb3fff700 1 --2- 172.21.15.104:0/2448989480 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffba00662f0 0x7ffba00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.920 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.936+0000 7ffbb3fff700 1 --2- 172.21.15.104:0/2448989480 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffba00662f0 0x7ffba00687a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009ba0 tx=0x7ffba4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:06.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:06.940+0000 7ffbb0ff9700 1 -- 172.21.15.104:0/2448989480 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ffba8050460 con 0x7ffbb40cd090 2023-07-14T11:54:07.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.112+0000 7ffbb971f700 1 -- 172.21.15.104:0/2448989480 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffbb40cd4e0 con 0x7ffbb40cd090 2023-07-14T11:54:07.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.112+0000 7ffbb0ff9700 1 -- 172.21.15.104:0/2448989480 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffbb40cd4e0 con 0x7ffbb40cd090 2023-07-14T11:54:07.098 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:54:07.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.116+0000 7ffbb971f700 1 -- 172.21.15.104:0/2448989480 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffba00662f0 msgr2=0x7ffba00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:07.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.116+0000 7ffbb971f700 1 --2- 172.21.15.104:0/2448989480 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffba00662f0 0x7ffba00687a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009ba0 tx=0x7ffba4008040 comp rx=0 tx=0).stop 2023-07-14T11:54:07.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.116+0000 7ffbb971f700 1 -- 172.21.15.104:0/2448989480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb40cd090 msgr2=0x7ffbb40b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:07.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.116+0000 7ffbb971f700 1 --2- 172.21.15.104:0/2448989480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb40cd090 0x7ffbb40b2500 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffba800c530 tx=0x7ffba800c840 comp rx=0 tx=0).stop 2023-07-14T11:54:07.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.116+0000 7ffbb971f700 1 -- 172.21.15.104:0/2448989480 shutdown_connections 2023-07-14T11:54:07.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.116+0000 7ffbb971f700 1 --2- 172.21.15.104:0/2448989480 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffba00662f0 0x7ffba00687a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.116+0000 7ffbb971f700 1 --2- 172.21.15.104:0/2448989480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb40cd090 0x7ffbb40b2500 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.116+0000 7ffbb971f700 1 --2- 172.21.15.104:0/2448989480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb40d07e0 0x7ffbb40c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.116+0000 7ffbb971f700 1 --2- 172.21.15.104:0/2448989480 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40b0ee0 0x7ffbb40c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.116+0000 7ffbb971f700 1 -- 172.21.15.104:0/2448989480 >> 172.21.15.104:0/2448989480 conn(0x7ffbb4060c40 msgr2=0x7ffbb40b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:07.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.116+0000 7ffbb971f700 1 -- 172.21.15.104:0/2448989480 shutdown_connections 2023-07-14T11:54:07.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.120+0000 7ffbb971f700 1 -- 172.21.15.104:0/2448989480 wait complete. 2023-07-14T11:54:07.124 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:54:07.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.320+0000 7fe71359e700 1 Processor -- start 2023-07-14T11:54:07.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.320+0000 7fe71359e700 1 -- start start 2023-07-14T11:54:07.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe71359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe714159c60 0x7fe71406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:07.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe71359e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe7140cab00 0x7fe7140caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe71359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7140cb4c0 con 0x7fe7140b0770 2023-07-14T11:54:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe71359e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7140714f0 con 0x7fe7140cab00 2023-07-14T11:54:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe71359e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71406da00 con 0x7fe714159c60 2023-07-14T11:54:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe712d9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe7140cab00 0x7fe7140caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe712d9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe7140cab00 0x7fe7140caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55264/0 (socket says 172.21.15.104:55264) 2023-07-14T11:54:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe712d9d700 1 -- 172.21.15.104:0/375757283 learned_addr learned my addr 172.21.15.104:0/375757283 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe71259c700 1 --2- 172.21.15.104:0/375757283 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe714159c60 0x7fe71406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe712d9d700 1 -- 172.21.15.104:0/375757283 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe714159c60 msgr2=0x7fe71406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe712d9d700 1 --2- 172.21.15.104:0/375757283 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe714159c60 0x7fe71406d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe712d9d700 1 -- 172.21.15.104:0/375757283 >> v1:172.21.15.121:6789/0 conn(0x7fe7140b0770 legacy=0x7fe714070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe712d9d700 1 -- 172.21.15.104:0/375757283 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71406dce0 con 0x7fe7140cab00 2023-07-14T11:54:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe71259c700 1 --2- 172.21.15.104:0/375757283 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe714159c60 0x7fe71406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe712d9d700 1 --2- 172.21.15.104:0/375757283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7140cab00 0x7fe7140caf80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009e50 tx=0x7fe70c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3e2910acfc0300 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe71159a700 1 -- 172.21.15.104:0/375757283 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00e070 con 0x7fe7140cab00 2023-07-14T11:54:07.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe71159a700 1 -- 172.21.15.104:0/375757283 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c003a40 con 0x7fe7140cab00 2023-07-14T11:54:07.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.324+0000 7fe71159a700 1 -- 172.21.15.104:0/375757283 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c003df0 con 0x7fe7140cab00 2023-07-14T11:54:07.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.328+0000 7fe71359e700 1 -- 172.21.15.104:0/375757283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7140cab00 msgr2=0x7fe7140caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:07.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.328+0000 7fe71359e700 1 --2- 172.21.15.104:0/375757283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7140cab00 0x7fe7140caf80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009e50 tx=0x7fe70c00b330 comp rx=0 tx=0).stop 2023-07-14T11:54:07.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.328+0000 7fe71359e700 1 -- 172.21.15.104:0/375757283 shutdown_connections 2023-07-14T11:54:07.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.328+0000 7fe71359e700 1 --2- 172.21.15.104:0/375757283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7140cab00 0x7fe7140caf80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.328+0000 7fe71359e700 1 --2- 172.21.15.104:0/375757283 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe714159c60 0x7fe71406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.328+0000 7fe71359e700 1 -- 172.21.15.104:0/375757283 >> 172.21.15.104:0/375757283 conn(0x7fe7140afc60 msgr2=0x7fe7140b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:07.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.328+0000 7fe71359e700 1 -- 172.21.15.104:0/375757283 shutdown_connections 2023-07-14T11:54:07.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.328+0000 7fe71359e700 1 -- 172.21.15.104:0/375757283 wait complete. 2023-07-14T11:54:07.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.328+0000 7fe71359e700 1 Processor -- start 2023-07-14T11:54:07.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.328+0000 7fe71359e700 1 -- start start 2023-07-14T11:54:07.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71359e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7140b0770 0x7fe7140c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:07.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7140cab00 0x7fe7141e7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:07.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe714159c60 0x7fe71414d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:07.316 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71359e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7141f5920 con 0x7fe7140b0770 2023-07-14T11:54:07.316 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7140c6410 con 0x7fe7140cab00 2023-07-14T11:54:07.316 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71359e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe714156780 con 0x7fe714159c60 2023-07-14T11:54:07.317 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7140b0770 0x7fe7140c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:07.317 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7140b0770 0x7fe7140c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55270/0 (socket says 172.21.15.104:55270) 2023-07-14T11:54:07.317 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71259c700 1 -- 172.21.15.104:0/587852787 learned_addr learned my addr 172.21.15.104:0/587852787 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:07.317 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe712d9d700 1 --2- 172.21.15.104:0/587852787 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe714159c60 0x7fe71414d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:07.317 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe711d9b700 1 --2- 172.21.15.104:0/587852787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7140cab00 0x7fe7141e7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:07.317 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71259c700 1 -- 172.21.15.104:0/587852787 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe714159c60 msgr2=0x7fe71414d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:07.318 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71259c700 1 --2- 172.21.15.104:0/587852787 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe714159c60 0x7fe71414d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.318 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71259c700 1 -- 172.21.15.104:0/587852787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7140cab00 msgr2=0x7fe7141e7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:07.318 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71259c700 1 --2- 172.21.15.104:0/587852787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7140cab00 0x7fe7141e7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.318 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71259c700 1 -- 172.21.15.104:0/587852787 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c009b30 con 0x7fe7140b0770 2023-07-14T11:54:07.318 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71259c700 1 --2- 172.21.15.104:0/587852787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7140b0770 0x7fe7140c5220 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe70400c770 tx=0x7fe70400ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:07.320 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe702ffd700 1 -- 172.21.15.104:0/587852787 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704011800 con 0x7fe7140b0770 2023-07-14T11:54:07.320 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe702ffd700 1 -- 172.21.15.104:0/587852787 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe704011960 con 0x7fe7140b0770 2023-07-14T11:54:07.320 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71359e700 1 -- 172.21.15.104:0/587852787 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71414d9f0 con 0x7fe7140b0770 2023-07-14T11:54:07.320 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe702ffd700 1 -- 172.21.15.104:0/587852787 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704011d10 con 0x7fe7140b0770 2023-07-14T11:54:07.320 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.332+0000 7fe71359e700 1 -- 172.21.15.104:0/587852787 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7140c5ae0 con 0x7fe7140b0770 2023-07-14T11:54:07.321 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.336+0000 7fe71359e700 1 -- 172.21.15.104:0/587852787 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71404e500 con 0x7fe7140b0770 2023-07-14T11:54:07.321 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.336+0000 7fe702ffd700 1 -- 172.21.15.104:0/587852787 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe704018360 con 0x7fe7140b0770 2023-07-14T11:54:07.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.336+0000 7fe702ffd700 1 --2- 172.21.15.104:0/587852787 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6f40662f0 0x7fe6f40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:07.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.340+0000 7fe702ffd700 1 -- 172.21.15.104:0/587852787 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe704083f90 con 0x7fe7140b0770 2023-07-14T11:54:07.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.340+0000 7fe711d9b700 1 --2- 172.21.15.104:0/587852787 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6f40662f0 0x7fe6f40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:07.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.340+0000 7fe711d9b700 1 --2- 172.21.15.104:0/587852787 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6f40662f0 0x7fe6f40687a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe7140ca8b0 tx=0x7fe7080090e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:07.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.344+0000 7fe702ffd700 1 -- 172.21.15.104:0/587852787 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe7040510d0 con 0x7fe7140b0770 2023-07-14T11:54:07.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.516+0000 7fe71359e700 1 -- 172.21.15.104:0/587852787 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe71406cf90 con 0x7fe7140b0770 2023-07-14T11:54:07.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.516+0000 7fe702ffd700 1 -- 172.21.15.104:0/587852787 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe7040546f0 con 0x7fe7140b0770 2023-07-14T11:54:07.500 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:54:07.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.516+0000 7fe71359e700 1 -- 172.21.15.104:0/587852787 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6f40662f0 msgr2=0x7fe6f40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:07.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.516+0000 7fe71359e700 1 --2- 172.21.15.104:0/587852787 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6f40662f0 0x7fe6f40687a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe7140ca8b0 tx=0x7fe7080090e0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.520+0000 7fe71359e700 1 -- 172.21.15.104:0/587852787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7140b0770 msgr2=0x7fe7140c5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:07.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.520+0000 7fe71359e700 1 --2- 172.21.15.104:0/587852787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7140b0770 0x7fe7140c5220 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe70400c770 tx=0x7fe70400ca80 comp rx=0 tx=0).stop 2023-07-14T11:54:07.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.520+0000 7fe71359e700 1 -- 172.21.15.104:0/587852787 shutdown_connections 2023-07-14T11:54:07.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.520+0000 7fe71359e700 1 --2- 172.21.15.104:0/587852787 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6f40662f0 0x7fe6f40687a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.520+0000 7fe71359e700 1 --2- 172.21.15.104:0/587852787 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe714159c60 0x7fe71414d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.520+0000 7fe71359e700 1 --2- 172.21.15.104:0/587852787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7140cab00 0x7fe7141e7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.520+0000 7fe71359e700 1 --2- 172.21.15.104:0/587852787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7140b0770 0x7fe7140c5220 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.520+0000 7fe71359e700 1 -- 172.21.15.104:0/587852787 >> 172.21.15.104:0/587852787 conn(0x7fe7140afc60 msgr2=0x7fe714070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:07.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.520+0000 7fe71359e700 1 -- 172.21.15.104:0/587852787 shutdown_connections 2023-07-14T11:54:07.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.520+0000 7fe71359e700 1 -- 172.21.15.104:0/587852787 wait complete. 2023-07-14T11:54:07.521 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:54:07.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.712+0000 7f184b240700 1 Processor -- start 2023-07-14T11:54:07.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.712+0000 7f184b240700 1 -- start start 2023-07-14T11:54:07.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184b240700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1844159c60 0x7f184406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:07.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184b240700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18440b0770 0x7f1844070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:07.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184b240700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f184406da00 con 0x7f18440cab00 2023-07-14T11:54:07.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184b240700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184406dce0 con 0x7f1844159c60 2023-07-14T11:54:07.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184b240700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184406dfc0 con 0x7f18440b0770 2023-07-14T11:54:07.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184a23e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1844159c60 0x7f184406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:07.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184a23e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1844159c60 0x7f184406d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:46446/0 (socket says 172.21.15.104:46446) 2023-07-14T11:54:07.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184a23e700 1 -- 172.21.15.104:0/3686687010 learned_addr learned my addr 172.21.15.104:0/3686687010 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:07.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f1849a3d700 1 --2- 172.21.15.104:0/3686687010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18440b0770 0x7f1844070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:07.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184a23e700 1 -- 172.21.15.104:0/3686687010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18440b0770 msgr2=0x7f1844070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:07.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184a23e700 1 --2- 172.21.15.104:0/3686687010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18440b0770 0x7f1844070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184a23e700 1 -- 172.21.15.104:0/3686687010 >> v1:172.21.15.121:6789/0 conn(0x7f18440cab00 legacy=0x7f18440c1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:07.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184a23e700 1 -- 172.21.15.104:0/3686687010 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18440705e0 con 0x7f1844159c60 2023-07-14T11:54:07.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184a23e700 1 --2- 172.21.15.104:0/3686687010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844159c60 0x7f184406d3b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1840009910 tx=0x7f1840009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ee3ccce1fa85789 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:07.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184923c700 1 -- 172.21.15.104:0/3686687010 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840011830 con 0x7f1844159c60 2023-07-14T11:54:07.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184923c700 1 -- 172.21.15.104:0/3686687010 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1840011990 con 0x7f1844159c60 2023-07-14T11:54:07.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.716+0000 7f184923c700 1 -- 172.21.15.104:0/3686687010 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840011d40 con 0x7f1844159c60 2023-07-14T11:54:07.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.720+0000 7f184b240700 1 -- 172.21.15.104:0/3686687010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844159c60 msgr2=0x7f184406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:07.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.720+0000 7f184b240700 1 --2- 172.21.15.104:0/3686687010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844159c60 0x7f184406d3b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1840009910 tx=0x7f1840009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:07.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.720+0000 7f184b240700 1 -- 172.21.15.104:0/3686687010 shutdown_connections 2023-07-14T11:54:07.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.720+0000 7f184b240700 1 --2- 172.21.15.104:0/3686687010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18440b0770 0x7f1844070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.706 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.720+0000 7f184b240700 1 --2- 172.21.15.104:0/3686687010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844159c60 0x7f184406d3b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.706 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.720+0000 7f184b240700 1 -- 172.21.15.104:0/3686687010 >> 172.21.15.104:0/3686687010 conn(0x7f18440afc60 msgr2=0x7f18440b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:07.706 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.720+0000 7f184b240700 1 -- 172.21.15.104:0/3686687010 shutdown_connections 2023-07-14T11:54:07.706 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.720+0000 7f184b240700 1 -- 172.21.15.104:0/3686687010 wait complete. 2023-07-14T11:54:07.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.720+0000 7f184b240700 1 Processor -- start 2023-07-14T11:54:07.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f184b240700 1 -- start start 2023-07-14T11:54:07.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f184b240700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18440b0770 0x7f184414d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:07.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f184b240700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18440cab00 0x7f18440c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:07.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f184b240700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844159c60 0x7f18440c26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:07.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f184b240700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184414db90 con 0x7f18440cab00 2023-07-14T11:54:07.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f184b240700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1844155250 con 0x7f1844159c60 2023-07-14T11:54:07.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f184b240700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18441f14b0 con 0x7f18440b0770 2023-07-14T11:54:07.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f1849a3d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18440cab00 0x7f18440c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:07.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f1849a3d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18440cab00 0x7f18440c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55272/0 (socket says 172.21.15.104:55272) 2023-07-14T11:54:07.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f1849a3d700 1 -- 172.21.15.104:0/1576374180 learned_addr learned my addr 172.21.15.104:0/1576374180 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:07.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f184a23e700 1 --2- 172.21.15.104:0/1576374180 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18440b0770 0x7f184414d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:07.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f184aa3f700 1 --2- 172.21.15.104:0/1576374180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844159c60 0x7f18440c26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:07.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f1849a3d700 1 -- 172.21.15.104:0/1576374180 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18440b0770 msgr2=0x7f184414d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:07.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f1849a3d700 1 --2- 172.21.15.104:0/1576374180 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18440b0770 0x7f184414d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f1849a3d700 1 -- 172.21.15.104:0/1576374180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844159c60 msgr2=0x7f18440c26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:07.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f1849a3d700 1 --2- 172.21.15.104:0/1576374180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844159c60 0x7f18440c26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f1849a3d700 1 -- 172.21.15.104:0/1576374180 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1840009570 con 0x7f18440cab00 2023-07-14T11:54:07.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f1849a3d700 1 --2- 172.21.15.104:0/1576374180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18440cab00 0x7f18440c7a30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f183400e9d0 tx=0x7f183400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:07.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f183affd700 1 -- 172.21.15.104:0/1576374180 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834009720 con 0x7f18440cab00 2023-07-14T11:54:07.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f183affd700 1 -- 172.21.15.104:0/1576374180 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183400c850 con 0x7f18440cab00 2023-07-14T11:54:07.712 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f184b240700 1 -- 172.21.15.104:0/1576374180 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18441f4f60 con 0x7f18440cab00 2023-07-14T11:54:07.712 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.724+0000 7f183affd700 1 -- 172.21.15.104:0/1576374180 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183400cbc0 con 0x7f18440cab00 2023-07-14T11:54:07.712 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.728+0000 7f184b240700 1 -- 172.21.15.104:0/1576374180 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18441f5520 con 0x7f18440cab00 2023-07-14T11:54:07.713 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.728+0000 7f184b240700 1 -- 172.21.15.104:0/1576374180 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184404e500 con 0x7f18440cab00 2023-07-14T11:54:07.713 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.728+0000 7f183affd700 1 -- 172.21.15.104:0/1576374180 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f183400cd20 con 0x7f18440cab00 2023-07-14T11:54:07.714 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.728+0000 7f183affd700 1 --2- 172.21.15.104:0/1576374180 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f18280662f0 0x7f18280687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:07.715 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.732+0000 7f183affd700 1 -- 172.21.15.104:0/1576374180 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f18340840e0 con 0x7f18440cab00 2023-07-14T11:54:07.715 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.732+0000 7f184a23e700 1 --2- 172.21.15.104:0/1576374180 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f18280662f0 0x7f18280687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:07.716 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.732+0000 7f184a23e700 1 --2- 172.21.15.104:0/1576374180 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f18280662f0 0x7f18280687a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1840014040 tx=0x7f1840008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:07.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.736+0000 7f183affd700 1 -- 172.21.15.104:0/1576374180 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f1834051220 con 0x7f18440cab00 2023-07-14T11:54:07.724 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:54:07.729 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:54:07.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.912+0000 7f184b240700 1 -- 172.21.15.104:0/1576374180 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f184406c410 con 0x7f18440cab00 2023-07-14T11:54:07.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.912+0000 7f183affd700 1 -- 172.21.15.104:0/1576374180 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1834054840 con 0x7f18440cab00 2023-07-14T11:54:07.895 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:07.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.912+0000 7f184b240700 1 -- 172.21.15.104:0/1576374180 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f18280662f0 msgr2=0x7f18280687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:07.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.912+0000 7f184b240700 1 --2- 172.21.15.104:0/1576374180 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f18280662f0 0x7f18280687a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1840014040 tx=0x7f1840008030 comp rx=0 tx=0).stop 2023-07-14T11:54:07.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.912+0000 7f184b240700 1 -- 172.21.15.104:0/1576374180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18440cab00 msgr2=0x7f18440c7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:07.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.912+0000 7f184b240700 1 --2- 172.21.15.104:0/1576374180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18440cab00 0x7f18440c7a30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f183400e9d0 tx=0x7f183400ece0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.912+0000 7f184b240700 1 -- 172.21.15.104:0/1576374180 shutdown_connections 2023-07-14T11:54:07.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.912+0000 7f184b240700 1 --2- 172.21.15.104:0/1576374180 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f18280662f0 0x7f18280687a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.912+0000 7f184b240700 1 --2- 172.21.15.104:0/1576374180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18440cab00 0x7f18440c7a30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.912+0000 7f184b240700 1 --2- 172.21.15.104:0/1576374180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844159c60 0x7f18440c26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.912+0000 7f184b240700 1 --2- 172.21.15.104:0/1576374180 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18440b0770 0x7f184414d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:07.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.912+0000 7f184b240700 1 -- 172.21.15.104:0/1576374180 >> 172.21.15.104:0/1576374180 conn(0x7f18440afc60 msgr2=0x7f18440ae6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:07.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.916+0000 7f184b240700 1 -- 172.21.15.104:0/1576374180 shutdown_connections 2023-07-14T11:54:07.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:07.916+0000 7f184b240700 1 -- 172.21.15.104:0/1576374180 wait complete. 2023-07-14T11:54:07.920 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:54:08.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.116+0000 7fcd0ee5a700 1 Processor -- start 2023-07-14T11:54:08.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.116+0000 7fcd0ee5a700 1 -- start start 2023-07-14T11:54:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0ee5a700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd080cee50 0x7fcd080cf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0ee5a700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fcd0815d260 0x7fcd080cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0ee5a700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd080d2df0 con 0x7fcd080b14d0 2023-07-14T11:54:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0ee5a700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd080b3890 con 0x7fcd0815d260 2023-07-14T11:54:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0ee5a700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080d0470 con 0x7fcd080cee50 2023-07-14T11:54:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0e659700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fcd0815d260 0x7fcd080cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0e659700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fcd0815d260 0x7fcd080cf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55288/0 (socket says 172.21.15.104:55288) 2023-07-14T11:54:08.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0e659700 1 -- 172.21.15.104:0/872529217 learned_addr learned my addr 172.21.15.104:0/872529217 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:08.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0de58700 1 --2- 172.21.15.104:0/872529217 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd080cee50 0x7fcd080cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0e659700 1 -- 172.21.15.104:0/872529217 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd080cee50 msgr2=0x7fcd080cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:08.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0e659700 1 --2- 172.21.15.104:0/872529217 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd080cee50 0x7fcd080cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0e659700 1 -- 172.21.15.104:0/872529217 >> v1:172.21.15.121:6789/0 conn(0x7fcd080b14d0 legacy=0x7fcd080b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0e659700 1 -- 172.21.15.104:0/872529217 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd080cfca0 con 0x7fcd0815d260 2023-07-14T11:54:08.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0de58700 1 --2- 172.21.15.104:0/872529217 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd080cee50 0x7fcd080cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:08.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0e659700 1 --2- 172.21.15.104:0/872529217 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd0815d260 0x7fcd080cf760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009e50 tx=0x7fcd0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11a7ac2be7c0839 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:08.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0ce56700 1 -- 172.21.15.104:0/872529217 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0000e070 con 0x7fcd0815d260 2023-07-14T11:54:08.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0ce56700 1 -- 172.21.15.104:0/872529217 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0000b9c0 con 0x7fcd0815d260 2023-07-14T11:54:08.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.120+0000 7fcd0ce56700 1 -- 172.21.15.104:0/872529217 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0000bd70 con 0x7fcd0815d260 2023-07-14T11:54:08.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.124+0000 7fcd0ee5a700 1 -- 172.21.15.104:0/872529217 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd0815d260 msgr2=0x7fcd080cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.124+0000 7fcd0ee5a700 1 --2- 172.21.15.104:0/872529217 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd0815d260 0x7fcd080cf760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009e50 tx=0x7fcd0000b330 comp rx=0 tx=0).stop 2023-07-14T11:54:08.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.124+0000 7fcd0ee5a700 1 -- 172.21.15.104:0/872529217 shutdown_connections 2023-07-14T11:54:08.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.124+0000 7fcd0ee5a700 1 --2- 172.21.15.104:0/872529217 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd0815d260 0x7fcd080cf760 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.124+0000 7fcd0ee5a700 1 --2- 172.21.15.104:0/872529217 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd080cee50 0x7fcd080cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.124+0000 7fcd0ee5a700 1 -- 172.21.15.104:0/872529217 >> 172.21.15.104:0/872529217 conn(0x7fcd0809b770 msgr2=0x7fcd080d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:08.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.124+0000 7fcd0ee5a700 1 -- 172.21.15.104:0/872529217 shutdown_connections 2023-07-14T11:54:08.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.124+0000 7fcd0ee5a700 1 -- 172.21.15.104:0/872529217 wait complete. 2023-07-14T11:54:08.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.124+0000 7fcd0ee5a700 1 Processor -- start 2023-07-14T11:54:08.110 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.124+0000 7fcd0ee5a700 1 -- start start 2023-07-14T11:54:08.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0ee5a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd080b14d0 0x7fcd08157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0ee5a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd080cee50 0x7fcd080b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0ee5a700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd0815d260 0x7fcd080afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0ee5a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080b60d0 con 0x7fcd080cee50 2023-07-14T11:54:08.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0ee5a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080b63b0 con 0x7fcd080b14d0 2023-07-14T11:54:08.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0ee5a700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080b6690 con 0x7fcd0815d260 2023-07-14T11:54:08.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0d657700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd080cee50 0x7fcd080b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0d657700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd080cee50 0x7fcd080b7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55296/0 (socket says 172.21.15.104:55296) 2023-07-14T11:54:08.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0d657700 1 -- 172.21.15.104:0/1196987111 learned_addr learned my addr 172.21.15.104:0/1196987111 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:08.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0de58700 1 --2- 172.21.15.104:0/1196987111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd080b14d0 0x7fcd08157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0e659700 1 --2- 172.21.15.104:0/1196987111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd0815d260 0x7fcd080afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0d657700 1 -- 172.21.15.104:0/1196987111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd0815d260 msgr2=0x7fcd080afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0d657700 1 --2- 172.21.15.104:0/1196987111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd0815d260 0x7fcd080afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0d657700 1 -- 172.21.15.104:0/1196987111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd080b14d0 msgr2=0x7fcd08157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0d657700 1 --2- 172.21.15.104:0/1196987111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd080b14d0 0x7fcd08157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0d657700 1 -- 172.21.15.104:0/1196987111 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd00009b30 con 0x7fcd080cee50 2023-07-14T11:54:08.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0de58700 1 --2- 172.21.15.104:0/1196987111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd080b14d0 0x7fcd08157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:08.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0d657700 1 --2- 172.21.15.104:0/1196987111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd080cee50 0x7fcd080b7ad0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fccf800c500 tx=0x7fccf800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:08.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fccfe7fc700 1 -- 172.21.15.104:0/1196987111 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf80052d0 con 0x7fcd080cee50 2023-07-14T11:54:08.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fccfe7fc700 1 -- 172.21.15.104:0/1196987111 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf8005430 con 0x7fcd080cee50 2023-07-14T11:54:08.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fccfe7fc700 1 -- 172.21.15.104:0/1196987111 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf80055a0 con 0x7fcd080cee50 2023-07-14T11:54:08.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0ee5a700 1 -- 172.21.15.104:0/1196987111 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd080b6970 con 0x7fcd080cee50 2023-07-14T11:54:08.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.128+0000 7fcd0ee5a700 1 -- 172.21.15.104:0/1196987111 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0809d6c0 con 0x7fcd080cee50 2023-07-14T11:54:08.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.132+0000 7fcd0ee5a700 1 -- 172.21.15.104:0/1196987111 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd080cfe60 con 0x7fcd080cee50 2023-07-14T11:54:08.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.132+0000 7fccfe7fc700 1 -- 172.21.15.104:0/1196987111 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fccf80038e0 con 0x7fcd080cee50 2023-07-14T11:54:08.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.132+0000 7fccfe7fc700 1 --2- 172.21.15.104:0/1196987111 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fccec066320 0x7fccec0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.132+0000 7fccfe7fc700 1 -- 172.21.15.104:0/1196987111 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fccf80834c0 con 0x7fcd080cee50 2023-07-14T11:54:08.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.132+0000 7fcd0de58700 1 --2- 172.21.15.104:0/1196987111 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fccec066320 0x7fccec0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.136+0000 7fcd0de58700 1 --2- 172.21.15.104:0/1196987111 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fccec066320 0x7fccec0687d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009a10 tx=0x7fcd04008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:08.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.140+0000 7fccfe7fc700 1 -- 172.21.15.104:0/1196987111 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fccf8050600 con 0x7fcd080cee50 2023-07-14T11:54:08.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.312+0000 7fcd0ee5a700 1 -- 172.21.15.104:0/1196987111 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd0806ee20 con 0x7fcd080cee50 2023-07-14T11:54:08.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.312+0000 7fccfe7fc700 1 -- 172.21.15.104:0/1196987111 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7fccf8053c20 con 0x7fcd080cee50 2023-07-14T11:54:08.296 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:54:08.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.312+0000 7fcd0ee5a700 1 -- 172.21.15.104:0/1196987111 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fccec066320 msgr2=0x7fccec0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.312+0000 7fcd0ee5a700 1 --2- 172.21.15.104:0/1196987111 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fccec066320 0x7fccec0687d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009a10 tx=0x7fcd04008040 comp rx=0 tx=0).stop 2023-07-14T11:54:08.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.312+0000 7fcd0ee5a700 1 -- 172.21.15.104:0/1196987111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd080cee50 msgr2=0x7fcd080b7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.312+0000 7fcd0ee5a700 1 --2- 172.21.15.104:0/1196987111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd080cee50 0x7fcd080b7ad0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fccf800c500 tx=0x7fccf800c8c0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.316+0000 7fcd0ee5a700 1 -- 172.21.15.104:0/1196987111 shutdown_connections 2023-07-14T11:54:08.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.316+0000 7fcd0ee5a700 1 --2- 172.21.15.104:0/1196987111 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fccec066320 0x7fccec0687d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.316+0000 7fcd0ee5a700 1 --2- 172.21.15.104:0/1196987111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd080cee50 0x7fcd080b7ad0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.316+0000 7fcd0ee5a700 1 --2- 172.21.15.104:0/1196987111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd0815d260 0x7fcd080afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.316+0000 7fcd0ee5a700 1 --2- 172.21.15.104:0/1196987111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd080b14d0 0x7fcd08157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.316+0000 7fcd0ee5a700 1 -- 172.21.15.104:0/1196987111 >> 172.21.15.104:0/1196987111 conn(0x7fcd0809b770 msgr2=0x7fcd080b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:08.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.316+0000 7fcd0ee5a700 1 -- 172.21.15.104:0/1196987111 shutdown_connections 2023-07-14T11:54:08.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.316+0000 7fcd0ee5a700 1 -- 172.21.15.104:0/1196987111 wait complete. 2023-07-14T11:54:08.319 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:54:08.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.512+0000 7f057f59e700 1 Processor -- start 2023-07-14T11:54:08.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.512+0000 7f057f59e700 1 -- start start 2023-07-14T11:54:08.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057f59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f05801603d0 0x7f058006dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057f59e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05800cdfb0 con 0x7f05800cd7f0 2023-07-14T11:54:08.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057f59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05800cdcd0 con 0x7f0580160000 2023-07-14T11:54:08.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057f59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0580070e40 con 0x7f05801603d0 2023-07-14T11:54:08.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057ed9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f05800cd7f0 0x7f05800c0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:50160/0 (socket says 172.21.15.104:50160) 2023-07-14T11:54:08.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057ed9d700 1 -- 172.21.15.104:0/814484447 learned_addr learned my addr 172.21.15.104:0/814484447 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:08.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057dd9b700 1 --2- 172.21.15.104:0/814484447 >> v2:172.21.15.121:3300/0 conn(0x7f05801603d0 0x7f058006dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057d59a700 1 -- 172.21.15.104:0/814484447 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 990213862 0 0) 0x7f0578001490 con 0x7f05800cd7f0 2023-07-14T11:54:08.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057d59a700 1 -- 172.21.15.104:0/814484447 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05680035d0 con 0x7f05800cd7f0 2023-07-14T11:54:08.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057dd9b700 1 -- 172.21.15.104:0/814484447 >> v1:172.21.15.121:6789/0 conn(0x7f0580160000 legacy=0x7f05800c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057dd9b700 1 -- 172.21.15.104:0/814484447 >> v1:172.21.15.104:6789/0 conn(0x7f05800cd7f0 legacy=0x7f05800c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057dd9b700 1 -- 172.21.15.104:0/814484447 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0580070780 con 0x7f05801603d0 2023-07-14T11:54:08.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057dd9b700 1 --2- 172.21.15.104:0/814484447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05801603d0 0x7f058006dd10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0574009850 tx=0x7f0574009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50b3eabb65226e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:08.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057d59a700 1 -- 172.21.15.104:0/814484447 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574011830 con 0x7f05801603d0 2023-07-14T11:54:08.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057d59a700 1 -- 172.21.15.104:0/814484447 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0574011990 con 0x7f05801603d0 2023-07-14T11:54:08.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.516+0000 7f057d59a700 1 -- 172.21.15.104:0/814484447 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574011d40 con 0x7f05801603d0 2023-07-14T11:54:08.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.520+0000 7f057f59e700 1 -- 172.21.15.104:0/814484447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05801603d0 msgr2=0x7f058006dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.520+0000 7f057f59e700 1 --2- 172.21.15.104:0/814484447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05801603d0 0x7f058006dd10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0574009850 tx=0x7f0574009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:08.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.520+0000 7f057f59e700 1 -- 172.21.15.104:0/814484447 shutdown_connections 2023-07-14T11:54:08.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.520+0000 7f057f59e700 1 --2- 172.21.15.104:0/814484447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05801603d0 0x7f058006dd10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.520+0000 7f057f59e700 1 -- 172.21.15.104:0/814484447 >> 172.21.15.104:0/814484447 conn(0x7f05800afc60 msgr2=0x7f05800b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:08.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.520+0000 7f057f59e700 1 -- 172.21.15.104:0/814484447 shutdown_connections 2023-07-14T11:54:08.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.520+0000 7f057f59e700 1 -- 172.21.15.104:0/814484447 wait complete. 2023-07-14T11:54:08.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.520+0000 7f057f59e700 1 Processor -- start 2023-07-14T11:54:08.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.520+0000 7f057f59e700 1 -- start start 2023-07-14T11:54:08.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057f59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05800cd7f0 0x7f0580162e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0580160000 0x7f05800b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057f59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05801603d0 0x7f05801505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057f59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800b38e0 con 0x7f05800cd7f0 2023-07-14T11:54:08.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057f59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800b3040 con 0x7f0580160000 2023-07-14T11:54:08.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057f59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05800b3320 con 0x7f05801603d0 2023-07-14T11:54:08.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05800cd7f0 0x7f0580162e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05800cd7f0 0x7f0580162e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55298/0 (socket says 172.21.15.104:55298) 2023-07-14T11:54:08.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057e59c700 1 -- 172.21.15.104:0/2524172209 learned_addr learned my addr 172.21.15.104:0/2524172209 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:08.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057dd9b700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0580160000 0x7f05800b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057e59c700 1 -- 172.21.15.104:0/2524172209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05801603d0 msgr2=0x7f05801505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057ed9d700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05801603d0 0x7f05801505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057e59c700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05801603d0 0x7f05801505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057e59c700 1 -- 172.21.15.104:0/2524172209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0580160000 msgr2=0x7f05800b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057e59c700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0580160000 0x7f05800b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057e59c700 1 -- 172.21.15.104:0/2524172209 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05740094b0 con 0x7f05800cd7f0 2023-07-14T11:54:08.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057dd9b700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0580160000 0x7f05800b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:08.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057ed9d700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05801603d0 0x7f05801505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:08.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057e59c700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05800cd7f0 0x7f0580162e70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f057000c500 tx=0x7f057000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:08.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f056effd700 1 -- 172.21.15.104:0/2524172209 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05700052d0 con 0x7f05800cd7f0 2023-07-14T11:54:08.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f056effd700 1 -- 172.21.15.104:0/2524172209 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0570005430 con 0x7f05800cd7f0 2023-07-14T11:54:08.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f056effd700 1 -- 172.21.15.104:0/2524172209 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05700055a0 con 0x7f05800cd7f0 2023-07-14T11:54:08.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057f59e700 1 -- 172.21.15.104:0/2524172209 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05800b3600 con 0x7f05800cd7f0 2023-07-14T11:54:08.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.524+0000 7f057f59e700 1 -- 172.21.15.104:0/2524172209 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05801633b0 con 0x7f05800cd7f0 2023-07-14T11:54:08.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.528+0000 7f057f59e700 1 -- 172.21.15.104:0/2524172209 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05801607a0 con 0x7f05800cd7f0 2023-07-14T11:54:08.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.528+0000 7f056effd700 1 -- 172.21.15.104:0/2524172209 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f05700036a0 con 0x7f05800cd7f0 2023-07-14T11:54:08.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.528+0000 7f056effd700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f05600662f0 0x7f05600687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.528+0000 7f056effd700 1 -- 172.21.15.104:0/2524172209 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f05700832c0 con 0x7f05800cd7f0 2023-07-14T11:54:08.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.528+0000 7f057dd9b700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f05600662f0 0x7f05600687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.532+0000 7f057dd9b700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f05600662f0 0x7f05600687a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f05800b2250 tx=0x7f0574008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:08.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.536+0000 7f056effd700 1 -- 172.21.15.104:0/2524172209 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f05700504a0 con 0x7f05800cd7f0 2023-07-14T11:54:08.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.708+0000 7f057f59e700 1 -- 172.21.15.104:0/2524172209 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f058004e500 con 0x7f05800cd7f0 2023-07-14T11:54:08.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.708+0000 7f056effd700 1 -- 172.21.15.104:0/2524172209 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0570053ac0 con 0x7f05800cd7f0 2023-07-14T11:54:08.693 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:08.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.712+0000 7f057f59e700 1 -- 172.21.15.104:0/2524172209 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f05600662f0 msgr2=0x7f05600687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.712+0000 7f057f59e700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f05600662f0 0x7f05600687a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f05800b2250 tx=0x7f0574008030 comp rx=0 tx=0).stop 2023-07-14T11:54:08.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.712+0000 7f057f59e700 1 -- 172.21.15.104:0/2524172209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05800cd7f0 msgr2=0x7f0580162e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.712+0000 7f057f59e700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05800cd7f0 0x7f0580162e70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f057000c500 tx=0x7f057000c8c0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.712+0000 7f057f59e700 1 -- 172.21.15.104:0/2524172209 shutdown_connections 2023-07-14T11:54:08.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.712+0000 7f057f59e700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f05600662f0 0x7f05600687a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.712+0000 7f057f59e700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05801603d0 0x7f05801505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.712+0000 7f057f59e700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0580160000 0x7f05800b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.712+0000 7f057f59e700 1 --2- 172.21.15.104:0/2524172209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05800cd7f0 0x7f0580162e70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.712+0000 7f057f59e700 1 -- 172.21.15.104:0/2524172209 >> 172.21.15.104:0/2524172209 conn(0x7f05800afc60 msgr2=0x7f058006d110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:08.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.712+0000 7f057f59e700 1 -- 172.21.15.104:0/2524172209 shutdown_connections 2023-07-14T11:54:08.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.712+0000 7f057f59e700 1 -- 172.21.15.104:0/2524172209 wait complete. 2023-07-14T11:54:08.720 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:08.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.904+0000 7f46327d9700 1 Processor -- start 2023-07-14T11:54:08.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.904+0000 7f46327d9700 1 -- start start 2023-07-14T11:54:08.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.904+0000 7f46327d9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f462c15ffa0 0x7f462c06d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.904+0000 7f46327d9700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462c0cb180 0x7f462c0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.904+0000 7f46327d9700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f462c0ca8a0 con 0x7f462c06dc50 2023-07-14T11:54:08.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.904+0000 7f46327d9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f462c0cbcd0 con 0x7f462c15ffa0 2023-07-14T11:54:08.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.904+0000 7f46327d9700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462c0cbf70 con 0x7f462c0cb180 2023-07-14T11:54:08.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.904+0000 7f4630fd6700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f462c06dc50 0x7f462c0caa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:50166/0 (socket says 172.21.15.104:50166) 2023-07-14T11:54:08.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.904+0000 7f4630fd6700 1 -- 172.21.15.104:0/1036756151 learned_addr learned my addr 172.21.15.104:0/1036756151 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:08.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.904+0000 7f46317d7700 1 --2- 172.21.15.104:0/1036756151 >> v2:172.21.15.121:3300/0 conn(0x7f462c15ffa0 0x7f462c06d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.904+0000 7f4631fd8700 1 --2- 172.21.15.104:0/1036756151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462c0cb180 0x7f462c0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f4623fff700 1 -- 172.21.15.104:0/1036756151 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514299109 0 0) 0x7f461c001110 con 0x7f462c06dc50 2023-07-14T11:54:08.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f4623fff700 1 -- 172.21.15.104:0/1036756151 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46180035d0 con 0x7f462c06dc50 2023-07-14T11:54:08.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f4631fd8700 1 -- 172.21.15.104:0/1036756151 >> v2:172.21.15.121:3300/0 conn(0x7f462c15ffa0 msgr2=0x7f462c06d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:08.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f4631fd8700 1 --2- 172.21.15.104:0/1036756151 >> v2:172.21.15.121:3300/0 conn(0x7f462c15ffa0 0x7f462c06d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f4631fd8700 1 -- 172.21.15.104:0/1036756151 >> v1:172.21.15.104:6789/0 conn(0x7f462c06dc50 legacy=0x7f462c0caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f4631fd8700 1 -- 172.21.15.104:0/1036756151 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c0cc250 con 0x7f462c0cb180 2023-07-14T11:54:08.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f4631fd8700 1 --2- 172.21.15.104:0/1036756151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462c0cb180 0x7f462c0ce340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4624009d90 tx=0x7f462400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a3216b254214dc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:08.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f4623fff700 1 -- 172.21.15.104:0/1036756151 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462400e070 con 0x7f462c0cb180 2023-07-14T11:54:08.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f4623fff700 1 -- 172.21.15.104:0/1036756151 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462400bb20 con 0x7f462c0cb180 2023-07-14T11:54:08.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f4623fff700 1 -- 172.21.15.104:0/1036756151 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624012430 con 0x7f462c0cb180 2023-07-14T11:54:08.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f46327d9700 1 -- 172.21.15.104:0/1036756151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462c0cb180 msgr2=0x7f462c0ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f46327d9700 1 --2- 172.21.15.104:0/1036756151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462c0cb180 0x7f462c0ce340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4624009d90 tx=0x7f462400b330 comp rx=0 tx=0).stop 2023-07-14T11:54:08.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f46327d9700 1 -- 172.21.15.104:0/1036756151 shutdown_connections 2023-07-14T11:54:08.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f46327d9700 1 --2- 172.21.15.104:0/1036756151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462c0cb180 0x7f462c0ce340 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f46327d9700 1 --2- 172.21.15.104:0/1036756151 >> v2:172.21.15.121:3300/0 conn(0x7f462c15ffa0 0x7f462c06d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f46327d9700 1 -- 172.21.15.104:0/1036756151 >> 172.21.15.104:0/1036756151 conn(0x7f462c06e960 msgr2=0x7f462c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:08.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f46327d9700 1 -- 172.21.15.104:0/1036756151 shutdown_connections 2023-07-14T11:54:08.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.908+0000 7f46327d9700 1 -- 172.21.15.104:0/1036756151 wait complete. 2023-07-14T11:54:08.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.912+0000 7f46327d9700 1 Processor -- start 2023-07-14T11:54:08.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.912+0000 7f46327d9700 1 -- start start 2023-07-14T11:54:08.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f46327d9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462c06dc50 0x7f462c0bcc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f46327d9700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462c0cb180 0x7f462c0c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f46327d9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f462c15ffa0 0x7f462c0c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f46327d9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c14b250 con 0x7f462c15ffa0 2023-07-14T11:54:08.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f46327d9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c15ebe0 con 0x7f462c06dc50 2023-07-14T11:54:08.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f46327d9700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462c0c4da0 con 0x7f462c0cb180 2023-07-14T11:54:08.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f4631fd8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f462c15ffa0 0x7f462c0c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f4631fd8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f462c15ffa0 0x7f462c0c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55306/0 (socket says 172.21.15.104:55306) 2023-07-14T11:54:08.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f4631fd8700 1 -- 172.21.15.104:0/2057255625 learned_addr learned my addr 172.21.15.104:0/2057255625 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:08.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f4630fd6700 1 --2- 172.21.15.104:0/2057255625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462c0cb180 0x7f462c0c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f46317d7700 1 --2- 172.21.15.104:0/2057255625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462c06dc50 0x7f462c0bcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f4631fd8700 1 -- 172.21.15.104:0/2057255625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462c0cb180 msgr2=0x7f462c0c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f4631fd8700 1 --2- 172.21.15.104:0/2057255625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462c0cb180 0x7f462c0c4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f4631fd8700 1 -- 172.21.15.104:0/2057255625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462c06dc50 msgr2=0x7f462c0bcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:08.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f4631fd8700 1 --2- 172.21.15.104:0/2057255625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462c06dc50 0x7f462c0bcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:08.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f4631fd8700 1 -- 172.21.15.104:0/2057255625 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4624009a70 con 0x7f462c15ffa0 2023-07-14T11:54:08.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f4631fd8700 1 --2- 172.21.15.104:0/2057255625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f462c15ffa0 0x7f462c0c6180 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4624013000 tx=0x7f4624004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:08.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f4621ffb700 1 -- 172.21.15.104:0/2057255625 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624012430 con 0x7f462c15ffa0 2023-07-14T11:54:08.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f4621ffb700 1 -- 172.21.15.104:0/2057255625 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4624020070 con 0x7f462c15ffa0 2023-07-14T11:54:08.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f4621ffb700 1 -- 172.21.15.104:0/2057255625 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624004280 con 0x7f462c15ffa0 2023-07-14T11:54:08.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f46327d9700 1 -- 172.21.15.104:0/2057255625 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462c0bd200 con 0x7f462c15ffa0 2023-07-14T11:54:08.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.916+0000 7f46327d9700 1 -- 172.21.15.104:0/2057255625 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462c14b780 con 0x7f462c15ffa0 2023-07-14T11:54:08.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.920+0000 7f46327d9700 1 -- 172.21.15.104:0/2057255625 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462c071620 con 0x7f462c15ffa0 2023-07-14T11:54:08.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.920+0000 7f4621ffb700 1 -- 172.21.15.104:0/2057255625 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f46240043e0 con 0x7f462c15ffa0 2023-07-14T11:54:08.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.920+0000 7f4621ffb700 1 --2- 172.21.15.104:0/2057255625 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46100662f0 0x7f46100687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:08.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.920+0000 7f4621ffb700 1 -- 172.21.15.104:0/2057255625 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4624022070 con 0x7f462c15ffa0 2023-07-14T11:54:08.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.920+0000 7f46317d7700 1 --2- 172.21.15.104:0/2057255625 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46100662f0 0x7f46100687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:08.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.920+0000 7f46317d7700 1 --2- 172.21.15.104:0/2057255625 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46100662f0 0x7f46100687a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4628009b80 tx=0x7f4628008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:08.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:08.924+0000 7f4621ffb700 1 -- 172.21.15.104:0/2057255625 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f4624052de0 con 0x7f462c15ffa0 2023-07-14T11:54:09.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.092+0000 7f46327d9700 1 -- 172.21.15.104:0/2057255625 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f462c04e500 con 0x7f462c15ffa0 2023-07-14T11:54:09.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.096+0000 7f4621ffb700 1 -- 172.21.15.104:0/2057255625 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20251 (secure 0 0 0) 0x7f462402c090 con 0x7f462c15ffa0 2023-07-14T11:54:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:09.082 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":30,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:55.601783+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:09.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.100+0000 7f46327d9700 1 -- 172.21.15.104:0/2057255625 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46100662f0 msgr2=0x7f46100687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.100+0000 7f46327d9700 1 --2- 172.21.15.104:0/2057255625 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46100662f0 0x7f46100687a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4628009b80 tx=0x7f4628008040 comp rx=0 tx=0).stop 2023-07-14T11:54:09.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.100+0000 7f46327d9700 1 -- 172.21.15.104:0/2057255625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f462c15ffa0 msgr2=0x7f462c0c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.100+0000 7f46327d9700 1 --2- 172.21.15.104:0/2057255625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f462c15ffa0 0x7f462c0c6180 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4624013000 tx=0x7f4624004af0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.100+0000 7f46327d9700 1 -- 172.21.15.104:0/2057255625 shutdown_connections 2023-07-14T11:54:09.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.100+0000 7f46327d9700 1 --2- 172.21.15.104:0/2057255625 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46100662f0 0x7f46100687a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.100+0000 7f46327d9700 1 --2- 172.21.15.104:0/2057255625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f462c15ffa0 0x7f462c0c6180 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.100+0000 7f46327d9700 1 --2- 172.21.15.104:0/2057255625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462c0cb180 0x7f462c0c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.100+0000 7f46327d9700 1 --2- 172.21.15.104:0/2057255625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462c06dc50 0x7f462c0bcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.100+0000 7f46327d9700 1 -- 172.21.15.104:0/2057255625 >> 172.21.15.104:0/2057255625 conn(0x7f462c06e960 msgr2=0x7f462c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:09.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.100+0000 7f46327d9700 1 -- 172.21.15.104:0/2057255625 shutdown_connections 2023-07-14T11:54:09.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.100+0000 7f46327d9700 1 -- 172.21.15.104:0/2057255625 wait complete. 2023-07-14T11:54:09.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:54:09.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:54:09.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.304+0000 7f841759e700 1 Processor -- start 2023-07-14T11:54:09.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.304+0000 7f841759e700 1 -- start start 2023-07-14T11:54:09.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f841759e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f84180cd260 0x7f84180d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f841759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84180d0610 0x7f84180af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f841759e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84180d1a60 con 0x7f84180b0ee0 2023-07-14T11:54:09.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f841759e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84180d1ef0 con 0x7f84180cd260 2023-07-14T11:54:09.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f841759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84180d0e50 con 0x7f84180d0610 2023-07-14T11:54:09.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f8415d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f84180cd260 0x7f84180d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f8415d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f84180cd260 0x7f84180d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55314/0 (socket says 172.21.15.104:55314) 2023-07-14T11:54:09.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f8415d9b700 1 -- 172.21.15.104:0/3916192473 learned_addr learned my addr 172.21.15.104:0/3916192473 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:09.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f841659c700 1 --1- 172.21.15.104:0/3916192473 >> v1:172.21.15.104:6789/0 conn(0x7f84180b0ee0 0x7f84180ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:50178/0 (socket says 172.21.15.104:50178) 2023-07-14T11:54:09.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f8416d9d700 1 --2- 172.21.15.104:0/3916192473 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84180d0610 0x7f84180af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f8415d9b700 1 -- 172.21.15.104:0/3916192473 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84180d0610 msgr2=0x7f84180af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:09.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f8415d9b700 1 --2- 172.21.15.104:0/3916192473 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84180d0610 0x7f84180af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f8415d9b700 1 -- 172.21.15.104:0/3916192473 >> v1:172.21.15.104:6789/0 conn(0x7f84180b0ee0 legacy=0x7f84180ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f8415d9b700 1 -- 172.21.15.104:0/3916192473 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84180d1130 con 0x7f84180cd260 2023-07-14T11:54:09.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f8415d9b700 1 --2- 172.21.15.104:0/3916192473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84180cd260 0x7f84180d00d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f840c009910 tx=0x7f840c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344cc335e615c88e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:09.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f841559a700 1 -- 172.21.15.104:0/3916192473 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c011830 con 0x7f84180cd260 2023-07-14T11:54:09.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f841559a700 1 -- 172.21.15.104:0/3916192473 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c011990 con 0x7f84180cd260 2023-07-14T11:54:09.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.308+0000 7f841559a700 1 -- 172.21.15.104:0/3916192473 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c011d40 con 0x7f84180cd260 2023-07-14T11:54:09.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.312+0000 7f841759e700 1 -- 172.21.15.104:0/3916192473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84180cd260 msgr2=0x7f84180d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.312+0000 7f841759e700 1 --2- 172.21.15.104:0/3916192473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84180cd260 0x7f84180d00d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f840c009910 tx=0x7f840c009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:09.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.312+0000 7f841759e700 1 -- 172.21.15.104:0/3916192473 shutdown_connections 2023-07-14T11:54:09.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.312+0000 7f841759e700 1 --2- 172.21.15.104:0/3916192473 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84180d0610 0x7f84180af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.312+0000 7f841759e700 1 --2- 172.21.15.104:0/3916192473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84180cd260 0x7f84180d00d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.312+0000 7f841759e700 1 -- 172.21.15.104:0/3916192473 >> 172.21.15.104:0/3916192473 conn(0x7f8418060c40 msgr2=0x7f841809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:09.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.312+0000 7f841759e700 1 -- 172.21.15.104:0/3916192473 shutdown_connections 2023-07-14T11:54:09.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.312+0000 7f841759e700 1 -- 172.21.15.104:0/3916192473 wait complete. 2023-07-14T11:54:09.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.312+0000 7f841759e700 1 Processor -- start 2023-07-14T11:54:09.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841759e700 1 -- start start 2023-07-14T11:54:09.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841759e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84180b0ee0 0x7f84180c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84180cd260 0x7f841815b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84180d0610 0x7f84181f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841759e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84180b1af0 con 0x7f84180b0ee0 2023-07-14T11:54:09.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84180b2070 con 0x7f84180d0610 2023-07-14T11:54:09.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84180c6550 con 0x7f84180cd260 2023-07-14T11:54:09.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84180b0ee0 0x7f84180c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84180b0ee0 0x7f84180c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55326/0 (socket says 172.21.15.104:55326) 2023-07-14T11:54:09.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841659c700 1 -- 172.21.15.104:0/10413439 learned_addr learned my addr 172.21.15.104:0/10413439 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:09.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f8416d9d700 1 --2- 172.21.15.104:0/10413439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84180d0610 0x7f84181f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f8415d9b700 1 --2- 172.21.15.104:0/10413439 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84180cd260 0x7f841815b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841659c700 1 -- 172.21.15.104:0/10413439 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84180cd260 msgr2=0x7f841815b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841659c700 1 --2- 172.21.15.104:0/10413439 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84180cd260 0x7f841815b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841659c700 1 -- 172.21.15.104:0/10413439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84180d0610 msgr2=0x7f84181f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841659c700 1 --2- 172.21.15.104:0/10413439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84180d0610 0x7f84181f14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841659c700 1 -- 172.21.15.104:0/10413439 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c009570 con 0x7f84180b0ee0 2023-07-14T11:54:09.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841659c700 1 --2- 172.21.15.104:0/10413439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84180b0ee0 0x7f84180c6010 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f840800c730 tx=0x7f840800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:09.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f8406ffd700 1 -- 172.21.15.104:0/10413439 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408005cd0 con 0x7f84180b0ee0 2023-07-14T11:54:09.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f8406ffd700 1 -- 172.21.15.104:0/10413439 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8408005e30 con 0x7f84180b0ee0 2023-07-14T11:54:09.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841759e700 1 -- 172.21.15.104:0/10413439 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84181f1fe0 con 0x7f84180b0ee0 2023-07-14T11:54:09.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f8406ffd700 1 -- 172.21.15.104:0/10413439 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84080072d0 con 0x7f84180b0ee0 2023-07-14T11:54:09.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.316+0000 7f841759e700 1 -- 172.21.15.104:0/10413439 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84180b1d00 con 0x7f84180b0ee0 2023-07-14T11:54:09.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.320+0000 7f841759e700 1 -- 172.21.15.104:0/10413439 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841804e500 con 0x7f84180b0ee0 2023-07-14T11:54:09.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.320+0000 7f8406ffd700 1 -- 172.21.15.104:0/10413439 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8408007430 con 0x7f84180b0ee0 2023-07-14T11:54:09.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.324+0000 7f8406ffd700 1 --2- 172.21.15.104:0/10413439 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83f80662f0 0x7f83f80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.324+0000 7f8406ffd700 1 -- 172.21.15.104:0/10413439 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8408083330 con 0x7f84180b0ee0 2023-07-14T11:54:09.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.324+0000 7f8415d9b700 1 --2- 172.21.15.104:0/10413439 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83f80662f0 0x7f83f80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.324+0000 7f8415d9b700 1 --2- 172.21.15.104:0/10413439 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83f80662f0 0x7f83f80687a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f84180cd650 tx=0x7f840c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:09.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.328+0000 7f8406ffd700 1 -- 172.21.15.104:0/10413439 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f8408050490 con 0x7f84180b0ee0 2023-07-14T11:54:09.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.496+0000 7f841759e700 1 -- 172.21.15.104:0/10413439 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84180b18d0 con 0x7f84180b0ee0 2023-07-14T11:54:09.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.500+0000 7f8406ffd700 1 -- 172.21.15.104:0/10413439 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8408017020 con 0x7f84180b0ee0 2023-07-14T11:54:09.483 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:54:09.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.500+0000 7f841759e700 1 -- 172.21.15.104:0/10413439 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83f80662f0 msgr2=0x7f83f80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.500+0000 7f841759e700 1 --2- 172.21.15.104:0/10413439 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83f80662f0 0x7f83f80687a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f84180cd650 tx=0x7f840c008030 comp rx=0 tx=0).stop 2023-07-14T11:54:09.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.500+0000 7f841759e700 1 -- 172.21.15.104:0/10413439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84180b0ee0 msgr2=0x7f84180c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.500+0000 7f841759e700 1 --2- 172.21.15.104:0/10413439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84180b0ee0 0x7f84180c6010 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f840800c730 tx=0x7f840800ca40 comp rx=0 tx=0).stop 2023-07-14T11:54:09.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.500+0000 7f841759e700 1 -- 172.21.15.104:0/10413439 shutdown_connections 2023-07-14T11:54:09.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.500+0000 7f841759e700 1 --2- 172.21.15.104:0/10413439 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83f80662f0 0x7f83f80687a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.500+0000 7f841759e700 1 --2- 172.21.15.104:0/10413439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84180d0610 0x7f84181f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.500+0000 7f841759e700 1 --2- 172.21.15.104:0/10413439 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84180cd260 0x7f841815b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.500+0000 7f841759e700 1 --2- 172.21.15.104:0/10413439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84180b0ee0 0x7f84180c6010 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.500+0000 7f841759e700 1 -- 172.21.15.104:0/10413439 >> 172.21.15.104:0/10413439 conn(0x7f8418060c40 msgr2=0x7f84180b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:09.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.500+0000 7f841759e700 1 -- 172.21.15.104:0/10413439 shutdown_connections 2023-07-14T11:54:09.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.504+0000 7f841759e700 1 -- 172.21.15.104:0/10413439 wait complete. 2023-07-14T11:54:09.507 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:54:09.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.700+0000 7f1d41584700 1 Processor -- start 2023-07-14T11:54:09.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.700+0000 7f1d41584700 1 -- start start 2023-07-14T11:54:09.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.700+0000 7f1d41584700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d3c15fa10 0x7f1d3c15fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.700+0000 7f1d41584700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d3c0b01c0 0x7f1d3c0cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.700+0000 7f1d41584700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3c0d0930 con 0x7f1d3c160320 2023-07-14T11:54:09.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.700+0000 7f1d41584700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d3c0b4ae0 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.700+0000 7f1d41584700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c0ce560 con 0x7f1d3c15fa10 2023-07-14T11:54:09.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d40d83700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d3c0b01c0 0x7f1d3c0cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d3b7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f1d3c160320 0x7f1d3c0ccb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:39220/0 (socket says 172.21.15.104:39220) 2023-07-14T11:54:09.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d3b7fe700 1 -- 172.21.15.104:0/3000127503 learned_addr learned my addr 172.21.15.104:0/3000127503 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:09.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d3bfff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d3c15fa10 0x7f1d3c15fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d3affd700 1 -- 172.21.15.104:0/3000127503 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252916327 0 0) 0x7f1d3c0d0930 con 0x7f1d3c160320 2023-07-14T11:54:09.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d3affd700 1 -- 172.21.15.104:0/3000127503 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d240035d0 con 0x7f1d3c160320 2023-07-14T11:54:09.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d40d83700 1 -- 172.21.15.104:0/3000127503 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d3c15fa10 msgr2=0x7f1d3c15fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:09.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d40d83700 1 --2- 172.21.15.104:0/3000127503 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d3c15fa10 0x7f1d3c15fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d40d83700 1 -- 172.21.15.104:0/3000127503 >> v1:172.21.15.121:6789/0 conn(0x7f1d3c160320 legacy=0x7f1d3c0ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d40d83700 1 -- 172.21.15.104:0/3000127503 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c15c300 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d3bfff700 1 --2- 172.21.15.104:0/3000127503 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d3c15fa10 0x7f1d3c15fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:09.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d40d83700 1 --2- 172.21.15.104:0/3000127503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d3c0b01c0 0x7f1d3c0cd280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009d90 tx=0x7f1d3400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5317aa512129955 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:09.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d3affd700 1 -- 172.21.15.104:0/3000127503 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3400e070 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d3affd700 1 -- 172.21.15.104:0/3000127503 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3400bb20 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d3affd700 1 -- 172.21.15.104:0/3000127503 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34012430 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d41584700 1 -- 172.21.15.104:0/3000127503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d3c0b01c0 msgr2=0x7f1d3c0cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.704+0000 7f1d41584700 1 --2- 172.21.15.104:0/3000127503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d3c0b01c0 0x7f1d3c0cd280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009d90 tx=0x7f1d3400b330 comp rx=0 tx=0).stop 2023-07-14T11:54:09.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.708+0000 7f1d41584700 1 -- 172.21.15.104:0/3000127503 shutdown_connections 2023-07-14T11:54:09.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.708+0000 7f1d41584700 1 --2- 172.21.15.104:0/3000127503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d3c0b01c0 0x7f1d3c0cd280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.708+0000 7f1d41584700 1 --2- 172.21.15.104:0/3000127503 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d3c15fa10 0x7f1d3c15fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.708+0000 7f1d41584700 1 -- 172.21.15.104:0/3000127503 >> 172.21.15.104:0/3000127503 conn(0x7f1d3c09b850 msgr2=0x7f1d3c0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:09.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.708+0000 7f1d41584700 1 -- 172.21.15.104:0/3000127503 shutdown_connections 2023-07-14T11:54:09.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.708+0000 7f1d41584700 1 -- 172.21.15.104:0/3000127503 wait complete. 2023-07-14T11:54:09.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d41584700 1 Processor -- start 2023-07-14T11:54:09.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d41584700 1 -- start start 2023-07-14T11:54:09.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d41584700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3c0b01c0 0x7f1d3c0cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d41584700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d3c15fa10 0x7f1d3c1f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d41584700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d3c160320 0x7f1d3c0cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d41584700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c15d8b0 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d41584700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c1f6320 con 0x7f1d3c160320 2023-07-14T11:54:09.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d41584700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c1f6600 con 0x7f1d3c15fa10 2023-07-14T11:54:09.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d3bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3c0b01c0 0x7f1d3c0cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d3bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3c0b01c0 0x7f1d3c0cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55336/0 (socket says 172.21.15.104:55336) 2023-07-14T11:54:09.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d3bfff700 1 -- 172.21.15.104:0/2713731171 learned_addr learned my addr 172.21.15.104:0/2713731171 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:09.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d40d83700 1 --2- 172.21.15.104:0/2713731171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d3c160320 0x7f1d3c0cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d3b7fe700 1 --2- 172.21.15.104:0/2713731171 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d3c15fa10 0x7f1d3c1f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d3bfff700 1 -- 172.21.15.104:0/2713731171 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d3c15fa10 msgr2=0x7f1d3c1f59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d3bfff700 1 --2- 172.21.15.104:0/2713731171 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d3c15fa10 0x7f1d3c1f59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d3bfff700 1 -- 172.21.15.104:0/2713731171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d3c160320 msgr2=0x7f1d3c0cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d3bfff700 1 --2- 172.21.15.104:0/2713731171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d3c160320 0x7f1d3c0cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d3bfff700 1 -- 172.21.15.104:0/2713731171 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c009450 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d3bfff700 1 --2- 172.21.15.104:0/2713731171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3c0b01c0 0x7f1d3c0cbfa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00ebe0 tx=0x7f1d2c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:09.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d38ff9700 1 -- 172.21.15.104:0/2713731171 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c005a40 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d38ff9700 1 -- 172.21.15.104:0/2713731171 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2c005ba0 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d38ff9700 1 -- 172.21.15.104:0/2713731171 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c00c850 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.712+0000 7f1d41584700 1 -- 172.21.15.104:0/2713731171 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d34009a70 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.716+0000 7f1d41584700 1 -- 172.21.15.104:0/2713731171 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3c15df20 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.716+0000 7f1d41584700 1 -- 172.21.15.104:0/2713731171 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3c0d3430 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.716+0000 7f1d38ff9700 1 -- 172.21.15.104:0/2713731171 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1d2c00c9b0 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.720+0000 7f1d38ff9700 1 --2- 172.21.15.104:0/2713731171 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1d280662f0 0x7f1d280687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.720+0000 7f1d38ff9700 1 -- 172.21.15.104:0/2713731171 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1d2c0841d0 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.720+0000 7f1d3b7fe700 1 --2- 172.21.15.104:0/2713731171 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1d280662f0 0x7f1d280687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.720+0000 7f1d3b7fe700 1 --2- 172.21.15.104:0/2713731171 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1d280662f0 0x7f1d280687a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1d30007fd0 tx=0x7f1d30007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:09.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.724+0000 7f1d38ff9700 1 -- 172.21.15.104:0/2713731171 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f1d2c0513b0 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.900+0000 7f1d41584700 1 -- 172.21.15.104:0/2713731171 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1d3c04e500 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.900+0000 7f1d38ff9700 1 -- 172.21.15.104:0/2713731171 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1d2c0549d0 con 0x7f1d3c0b01c0 2023-07-14T11:54:09.884 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:54:09.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.900+0000 7f1d41584700 1 -- 172.21.15.104:0/2713731171 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1d280662f0 msgr2=0x7f1d280687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.900+0000 7f1d41584700 1 --2- 172.21.15.104:0/2713731171 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1d280662f0 0x7f1d280687a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1d30007fd0 tx=0x7f1d30007de0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.900+0000 7f1d41584700 1 -- 172.21.15.104:0/2713731171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3c0b01c0 msgr2=0x7f1d3c0cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.900+0000 7f1d41584700 1 --2- 172.21.15.104:0/2713731171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3c0b01c0 0x7f1d3c0cbfa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00ebe0 tx=0x7f1d2c00eef0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.904+0000 7f1d41584700 1 -- 172.21.15.104:0/2713731171 shutdown_connections 2023-07-14T11:54:09.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.904+0000 7f1d41584700 1 --2- 172.21.15.104:0/2713731171 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1d280662f0 0x7f1d280687a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.904+0000 7f1d41584700 1 --2- 172.21.15.104:0/2713731171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d3c160320 0x7f1d3c0cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.904+0000 7f1d41584700 1 --2- 172.21.15.104:0/2713731171 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d3c15fa10 0x7f1d3c1f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.904+0000 7f1d41584700 1 --2- 172.21.15.104:0/2713731171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3c0b01c0 0x7f1d3c0cbfa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.904+0000 7f1d41584700 1 -- 172.21.15.104:0/2713731171 >> 172.21.15.104:0/2713731171 conn(0x7f1d3c09b850 msgr2=0x7f1d3c0d2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:09.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.904+0000 7f1d41584700 1 -- 172.21.15.104:0/2713731171 shutdown_connections 2023-07-14T11:54:09.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:09.904+0000 7f1d41584700 1 -- 172.21.15.104:0/2713731171 wait complete. 2023-07-14T11:54:09.907 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:54:10.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.104+0000 7f8a1ca2f700 1 Processor -- start 2023-07-14T11:54:10.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.104+0000 7f8a1ca2f700 1 -- start start 2023-07-14T11:54:10.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a1ca2f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8a180b0ee0 0x7f8a180ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a1ca2f700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a180cd4e0 con 0x7f8a180d07e0 2023-07-14T11:54:10.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a1ca2f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a180d1a60 con 0x7f8a180cd090 2023-07-14T11:54:10.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a1ca2f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a180d0c60 con 0x7f8a180b0ee0 2023-07-14T11:54:10.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a177fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8a180b0ee0 0x7f8a180ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a17fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f8a180d07e0 0x7f8a180af0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:50180/0 (socket says 172.21.15.104:50180) 2023-07-14T11:54:10.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a177fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8a180b0ee0 0x7f8a180ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55350/0 (socket says 172.21.15.104:55350) 2023-07-14T11:54:10.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a17fff700 1 -- 172.21.15.104:0/435553431 learned_addr learned my addr 172.21.15.104:0/435553431 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:10.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a16ffd700 1 -- 172.21.15.104:0/435553431 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2125363389 0 0) 0x7f8a10001490 con 0x7f8a180d07e0 2023-07-14T11:54:10.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a16ffd700 1 -- 172.21.15.104:0/435553431 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a040035d0 con 0x7f8a180d07e0 2023-07-14T11:54:10.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a177fe700 1 -- 172.21.15.104:0/435553431 >> v1:172.21.15.121:6789/0 conn(0x7f8a180cd090 legacy=0x7f8a180d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a177fe700 1 -- 172.21.15.104:0/435553431 >> v1:172.21.15.104:6789/0 conn(0x7f8a180d07e0 legacy=0x7f8a180af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a177fe700 1 -- 172.21.15.104:0/435553431 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a180d1ef0 con 0x7f8a180b0ee0 2023-07-14T11:54:10.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a177fe700 1 --2- 172.21.15.104:0/435553431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a180b0ee0 0x7f8a180ccb50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009850 tx=0x7f8a08009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836a0f94c2a3d0c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:10.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a16ffd700 1 -- 172.21.15.104:0/435553431 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08011830 con 0x7f8a180b0ee0 2023-07-14T11:54:10.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a16ffd700 1 -- 172.21.15.104:0/435553431 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a08011990 con 0x7f8a180b0ee0 2023-07-14T11:54:10.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.108+0000 7f8a16ffd700 1 -- 172.21.15.104:0/435553431 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08011d40 con 0x7f8a180b0ee0 2023-07-14T11:54:10.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.112+0000 7f8a1ca2f700 1 -- 172.21.15.104:0/435553431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a180b0ee0 msgr2=0x7f8a180ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.112+0000 7f8a1ca2f700 1 --2- 172.21.15.104:0/435553431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a180b0ee0 0x7f8a180ccb50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009850 tx=0x7f8a08009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:10.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.112+0000 7f8a1ca2f700 1 -- 172.21.15.104:0/435553431 shutdown_connections 2023-07-14T11:54:10.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.112+0000 7f8a1ca2f700 1 --2- 172.21.15.104:0/435553431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a180b0ee0 0x7f8a180ccb50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.112+0000 7f8a1ca2f700 1 -- 172.21.15.104:0/435553431 >> 172.21.15.104:0/435553431 conn(0x7f8a18060c40 msgr2=0x7f8a1809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:10.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.112+0000 7f8a1ca2f700 1 -- 172.21.15.104:0/435553431 shutdown_connections 2023-07-14T11:54:10.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.112+0000 7f8a1ca2f700 1 -- 172.21.15.104:0/435553431 wait complete. 2023-07-14T11:54:10.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.112+0000 7f8a1ca2f700 1 Processor -- start 2023-07-14T11:54:10.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a1ca2f700 1 -- start start 2023-07-14T11:54:10.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a1ca2f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a180b0ee0 0x7f8a18158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a1ca2f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a180cd090 0x7f8a180b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a1ca2f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a180d07e0 0x7f8a180b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a1ca2f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181590b0 con 0x7f8a180b0ee0 2023-07-14T11:54:10.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a1ca2f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a180b2060 con 0x7f8a180d07e0 2023-07-14T11:54:10.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a1ca2f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a180b2340 con 0x7f8a180cd090 2023-07-14T11:54:10.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a177fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a180b0ee0 0x7f8a18158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a177fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a180b0ee0 0x7f8a18158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55362/0 (socket says 172.21.15.104:55362) 2023-07-14T11:54:10.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a177fe700 1 -- 172.21.15.104:0/1639516315 learned_addr learned my addr 172.21.15.104:0/1639516315 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:10.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a17fff700 1 --2- 172.21.15.104:0/1639516315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a180d07e0 0x7f8a180b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a0ffff700 1 --2- 172.21.15.104:0/1639516315 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a180cd090 0x7f8a180b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a177fe700 1 -- 172.21.15.104:0/1639516315 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a180cd090 msgr2=0x7f8a180b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a177fe700 1 --2- 172.21.15.104:0/1639516315 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a180cd090 0x7f8a180b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a177fe700 1 -- 172.21.15.104:0/1639516315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a180d07e0 msgr2=0x7f8a180b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a177fe700 1 --2- 172.21.15.104:0/1639516315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a180d07e0 0x7f8a180b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a177fe700 1 -- 172.21.15.104:0/1639516315 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a080094b0 con 0x7f8a180b0ee0 2023-07-14T11:54:10.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.116+0000 7f8a17fff700 1 --2- 172.21.15.104:0/1639516315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a180d07e0 0x7f8a180b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:10.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.120+0000 7f8a177fe700 1 --2- 172.21.15.104:0/1639516315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a180b0ee0 0x7f8a18158b70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8a08014040 tx=0x7f8a08004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:10.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.120+0000 7f8a14ff9700 1 -- 172.21.15.104:0/1639516315 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08019980 con 0x7f8a180b0ee0 2023-07-14T11:54:10.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.120+0000 7f8a14ff9700 1 -- 172.21.15.104:0/1639516315 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0801e070 con 0x7f8a180b0ee0 2023-07-14T11:54:10.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.120+0000 7f8a14ff9700 1 -- 172.21.15.104:0/1639516315 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08019d20 con 0x7f8a180b0ee0 2023-07-14T11:54:10.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.120+0000 7f8a1ca2f700 1 -- 172.21.15.104:0/1639516315 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a180b2620 con 0x7f8a180b0ee0 2023-07-14T11:54:10.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.120+0000 7f8a1ca2f700 1 -- 172.21.15.104:0/1639516315 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a180b2be0 con 0x7f8a180b0ee0 2023-07-14T11:54:10.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.120+0000 7f8a1ca2f700 1 -- 172.21.15.104:0/1639516315 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1804e500 con 0x7f8a180b0ee0 2023-07-14T11:54:10.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.120+0000 7f8a14ff9700 1 -- 172.21.15.104:0/1639516315 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8a0802d450 con 0x7f8a180b0ee0 2023-07-14T11:54:10.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.124+0000 7f8a14ff9700 1 --2- 172.21.15.104:0/1639516315 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f89f80662f0 0x7f89f80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.124+0000 7f8a14ff9700 1 -- 172.21.15.104:0/1639516315 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8a08084f30 con 0x7f8a180b0ee0 2023-07-14T11:54:10.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.124+0000 7f8a0ffff700 1 --2- 172.21.15.104:0/1639516315 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f89f80662f0 0x7f89f80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.124+0000 7f8a0ffff700 1 --2- 172.21.15.104:0/1639516315 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f89f80662f0 0x7f89f80687a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8a180b3aa0 tx=0x7f8a00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:10.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.128+0000 7f8a14ff9700 1 -- 172.21.15.104:0/1639516315 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f8a08052110 con 0x7f8a180b0ee0 2023-07-14T11:54:10.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.296+0000 7f8a1ca2f700 1 -- 172.21.15.104:0/1639516315 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a180b7f10 con 0x7f8a180b0ee0 2023-07-14T11:54:10.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.300+0000 7f8a14ff9700 1 -- 172.21.15.104:0/1639516315 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a080150a0 con 0x7f8a180b0ee0 2023-07-14T11:54:10.283 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:10.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.300+0000 7f8a1ca2f700 1 -- 172.21.15.104:0/1639516315 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f89f80662f0 msgr2=0x7f89f80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.300+0000 7f8a1ca2f700 1 --2- 172.21.15.104:0/1639516315 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f89f80662f0 0x7f89f80687a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8a180b3aa0 tx=0x7f8a00009040 comp rx=0 tx=0).stop 2023-07-14T11:54:10.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.300+0000 7f8a1ca2f700 1 -- 172.21.15.104:0/1639516315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a180b0ee0 msgr2=0x7f8a18158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.300+0000 7f8a1ca2f700 1 --2- 172.21.15.104:0/1639516315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a180b0ee0 0x7f8a18158b70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8a08014040 tx=0x7f8a08004770 comp rx=0 tx=0).stop 2023-07-14T11:54:10.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.300+0000 7f8a1ca2f700 1 -- 172.21.15.104:0/1639516315 shutdown_connections 2023-07-14T11:54:10.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.300+0000 7f8a1ca2f700 1 --2- 172.21.15.104:0/1639516315 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f89f80662f0 0x7f89f80687a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.300+0000 7f8a1ca2f700 1 --2- 172.21.15.104:0/1639516315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a180d07e0 0x7f8a180b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.300+0000 7f8a1ca2f700 1 --2- 172.21.15.104:0/1639516315 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a180cd090 0x7f8a180b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.300+0000 7f8a1ca2f700 1 --2- 172.21.15.104:0/1639516315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a180b0ee0 0x7f8a18158b70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.300+0000 7f8a1ca2f700 1 -- 172.21.15.104:0/1639516315 >> 172.21.15.104:0/1639516315 conn(0x7f8a18060c40 msgr2=0x7f8a1809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:10.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.300+0000 7f8a1ca2f700 1 -- 172.21.15.104:0/1639516315 shutdown_connections 2023-07-14T11:54:10.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.300+0000 7f8a1ca2f700 1 -- 172.21.15.104:0/1639516315 wait complete. 2023-07-14T11:54:10.306 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:54:10.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.508+0000 7f79050ee700 1 Processor -- start 2023-07-14T11:54:10.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.508+0000 7f79050ee700 1 -- start start 2023-07-14T11:54:10.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.508+0000 7f79050ee700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7900158e90 0x7f790006deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.508+0000 7f79050ee700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f79000afd80 0x7f79000b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.508+0000 7f79050ee700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79000cbc20 con 0x7f79000c9360 2023-07-14T11:54:10.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.508+0000 7f79050ee700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79000c9790 con 0x7f7900158e90 2023-07-14T11:54:10.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.508+0000 7f79050ee700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790006ce80 con 0x7f79000afd80 2023-07-14T11:54:10.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f78f7fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f79000c9360 0x7f790006d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:44750/0 (socket says 172.21.15.104:44750) 2023-07-14T11:54:10.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f78f7fff700 1 -- 172.21.15.104:0/1732539460 learned_addr learned my addr 172.21.15.104:0/1732539460 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:10.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f78fed9d700 1 --2- 172.21.15.104:0/1732539460 >> v2:172.21.15.104:3300/0 conn(0x7f7900158e90 0x7f790006deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f78ff59e700 1 --2- 172.21.15.104:0/1732539460 >> v2:172.21.15.121:3300/0 conn(0x7f79000afd80 0x7f79000b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f78fe59c700 1 -- 172.21.15.104:0/1732539460 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117922088 0 0) 0x7f79000cbc20 con 0x7f79000c9360 2023-07-14T11:54:10.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f78fe59c700 1 -- 172.21.15.104:0/1732539460 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78ec0035d0 con 0x7f79000c9360 2023-07-14T11:54:10.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f78fed9d700 1 -- 172.21.15.104:0/1732539460 >> v2:172.21.15.121:3300/0 conn(0x7f79000afd80 msgr2=0x7f79000b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:10.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f78fed9d700 1 --2- 172.21.15.104:0/1732539460 >> v2:172.21.15.121:3300/0 conn(0x7f79000afd80 0x7f79000b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f78fed9d700 1 -- 172.21.15.104:0/1732539460 >> v1:172.21.15.104:6789/0 conn(0x7f79000c9360 legacy=0x7f790006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f78fed9d700 1 -- 172.21.15.104:0/1732539460 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79000b0700 con 0x7f7900158e90 2023-07-14T11:54:10.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f78fed9d700 1 --2- 172.21.15.104:0/1732539460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7900158e90 0x7f790006deb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f79000ad6d0 tx=0x7f78f0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c53b2ccc4b15770 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:10.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f78fe59c700 1 -- 172.21.15.104:0/1732539460 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0011830 con 0x7f7900158e90 2023-07-14T11:54:10.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f78fe59c700 1 -- 172.21.15.104:0/1732539460 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f0011990 con 0x7f7900158e90 2023-07-14T11:54:10.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f78fe59c700 1 -- 172.21.15.104:0/1732539460 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0011d20 con 0x7f7900158e90 2023-07-14T11:54:10.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f79050ee700 1 -- 172.21.15.104:0/1732539460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7900158e90 msgr2=0x7f790006deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f79050ee700 1 --2- 172.21.15.104:0/1732539460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7900158e90 0x7f790006deb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f79000ad6d0 tx=0x7f78f0009b30 comp rx=0 tx=0).stop 2023-07-14T11:54:10.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f79050ee700 1 -- 172.21.15.104:0/1732539460 shutdown_connections 2023-07-14T11:54:10.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f79050ee700 1 --2- 172.21.15.104:0/1732539460 >> v2:172.21.15.121:3300/0 conn(0x7f79000afd80 0x7f79000b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f79050ee700 1 --2- 172.21.15.104:0/1732539460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7900158e90 0x7f790006deb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f79050ee700 1 -- 172.21.15.104:0/1732539460 >> 172.21.15.104:0/1732539460 conn(0x7f79000cad80 msgr2=0x7f79000cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:10.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f79050ee700 1 -- 172.21.15.104:0/1732539460 shutdown_connections 2023-07-14T11:54:10.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.512+0000 7f79050ee700 1 -- 172.21.15.104:0/1732539460 wait complete. 2023-07-14T11:54:10.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.516+0000 7f79050ee700 1 Processor -- start 2023-07-14T11:54:10.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.516+0000 7f79050ee700 1 -- start start 2023-07-14T11:54:10.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f79050ee700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79000afd80 0x7f79001f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f79050ee700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79000c9360 0x7f790015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f79050ee700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900158e90 0x7f79000b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f79050ee700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79000b3020 con 0x7f79000afd80 2023-07-14T11:54:10.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f79050ee700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79000cde30 con 0x7f7900158e90 2023-07-14T11:54:10.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f79050ee700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79000b26f0 con 0x7f79000c9360 2023-07-14T11:54:10.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f78fed9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79000afd80 0x7f79001f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f78fed9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79000afd80 0x7f79001f2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35000/0 (socket says 172.21.15.104:35000) 2023-07-14T11:54:10.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f78fed9d700 1 -- 172.21.15.104:0/2433750865 learned_addr learned my addr 172.21.15.104:0/2433750865 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:10.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f78ff59e700 1 --2- 172.21.15.104:0/2433750865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900158e90 0x7f79000b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f78f7fff700 1 --2- 172.21.15.104:0/2433750865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79000c9360 0x7f790015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f78fed9d700 1 -- 172.21.15.104:0/2433750865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79000c9360 msgr2=0x7f790015cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f78fed9d700 1 --2- 172.21.15.104:0/2433750865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79000c9360 0x7f790015cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f78fed9d700 1 -- 172.21.15.104:0/2433750865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900158e90 msgr2=0x7f79000b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f78fed9d700 1 --2- 172.21.15.104:0/2433750865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900158e90 0x7f79000b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f78fed9d700 1 -- 172.21.15.104:0/2433750865 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f00094b0 con 0x7f79000afd80 2023-07-14T11:54:10.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f78fed9d700 1 --2- 172.21.15.104:0/2433750865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79000afd80 0x7f79001f2450 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f78f0014040 tx=0x7f78f0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:10.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f78f77fe700 1 -- 172.21.15.104:0/2433750865 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f00036c0 con 0x7f79000afd80 2023-07-14T11:54:10.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f78f77fe700 1 -- 172.21.15.104:0/2433750865 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f001e070 con 0x7f79000afd80 2023-07-14T11:54:10.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f78f77fe700 1 -- 172.21.15.104:0/2433750865 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0003a40 con 0x7f79000afd80 2023-07-14T11:54:10.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f79050ee700 1 -- 172.21.15.104:0/2433750865 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79000b29d0 con 0x7f79000afd80 2023-07-14T11:54:10.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.520+0000 7f79050ee700 1 -- 172.21.15.104:0/2433750865 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79001f2990 con 0x7f79000afd80 2023-07-14T11:54:10.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.524+0000 7f79050ee700 1 -- 172.21.15.104:0/2433750865 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f790004dcb0 con 0x7f79000afd80 2023-07-14T11:54:10.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.524+0000 7f78f77fe700 1 -- 172.21.15.104:0/2433750865 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f78f0003ba0 con 0x7f79000afd80 2023-07-14T11:54:10.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.524+0000 7f78f77fe700 1 --2- 172.21.15.104:0/2433750865 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f78e00662f0 0x7f78e00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.524+0000 7f78f77fe700 1 -- 172.21.15.104:0/2433750865 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f78f0085180 con 0x7f79000afd80 2023-07-14T11:54:10.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.524+0000 7f78f7fff700 1 --2- 172.21.15.104:0/2433750865 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f78e00662f0 0x7f78e00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.528+0000 7f78f7fff700 1 --2- 172.21.15.104:0/2433750865 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f78e00662f0 0x7f78e00687a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f790015c370 tx=0x7f78e8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:10.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.528+0000 7f78f77fe700 1 -- 172.21.15.104:0/2433750865 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f78f00522c0 con 0x7f79000afd80 2023-07-14T11:54:10.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.704+0000 7f79050ee700 1 -- 172.21.15.104:0/2433750865 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f79000b4280 con 0x7f79000afd80 2023-07-14T11:54:10.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.704+0000 7f78f77fe700 1 -- 172.21.15.104:0/2433750865 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78f00558e0 con 0x7f79000afd80 2023-07-14T11:54:10.688 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:54:10.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.704+0000 7f79050ee700 1 -- 172.21.15.104:0/2433750865 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f78e00662f0 msgr2=0x7f78e00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.704+0000 7f79050ee700 1 --2- 172.21.15.104:0/2433750865 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f78e00662f0 0x7f78e00687a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f790015c370 tx=0x7f78e8009040 comp rx=0 tx=0).stop 2023-07-14T11:54:10.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.704+0000 7f79050ee700 1 -- 172.21.15.104:0/2433750865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79000afd80 msgr2=0x7f79001f2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.704+0000 7f79050ee700 1 --2- 172.21.15.104:0/2433750865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79000afd80 0x7f79001f2450 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f78f0014040 tx=0x7f78f0004770 comp rx=0 tx=0).stop 2023-07-14T11:54:10.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.704+0000 7f79050ee700 1 -- 172.21.15.104:0/2433750865 shutdown_connections 2023-07-14T11:54:10.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.704+0000 7f79050ee700 1 --2- 172.21.15.104:0/2433750865 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f78e00662f0 0x7f78e00687a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.704+0000 7f79050ee700 1 --2- 172.21.15.104:0/2433750865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900158e90 0x7f79000b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.704+0000 7f79050ee700 1 --2- 172.21.15.104:0/2433750865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79000c9360 0x7f790015cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.704+0000 7f79050ee700 1 --2- 172.21.15.104:0/2433750865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79000afd80 0x7f79001f2450 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.704+0000 7f79050ee700 1 -- 172.21.15.104:0/2433750865 >> 172.21.15.104:0/2433750865 conn(0x7f79000cad80 msgr2=0x7f79000c3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:10.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.708+0000 7f79050ee700 1 -- 172.21.15.104:0/2433750865 shutdown_connections 2023-07-14T11:54:10.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.708+0000 7f79050ee700 1 -- 172.21.15.104:0/2433750865 wait complete. 2023-07-14T11:54:10.710 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:54:10.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.900+0000 7f227cb3e700 1 Processor -- start 2023-07-14T11:54:10.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.900+0000 7f227cb3e700 1 -- start start 2023-07-14T11:54:10.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.900+0000 7f227cb3e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f22780b0770 0x7f227806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.900+0000 7f227cb3e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22780714f0 con 0x7f22780cab00 2023-07-14T11:54:10.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.900+0000 7f227cb3e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227806da00 con 0x7f2278159c60 2023-07-14T11:54:10.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.900+0000 7f227cb3e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f227806dce0 con 0x7f22780b0770 2023-07-14T11:54:10.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f2277fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f22780cab00 0x7f22780c1ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:44766/0 (socket says 172.21.15.104:44766) 2023-07-14T11:54:10.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f2277fff700 1 -- 172.21.15.104:0/2104709397 learned_addr learned my addr 172.21.15.104:0/2104709397 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:10.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f2276ffd700 1 --2- 172.21.15.104:0/2104709397 >> v2:172.21.15.121:3300/0 conn(0x7f22780b0770 0x7f227806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f22767fc700 1 -- 172.21.15.104:0/2104709397 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404841677 0 0) 0x7f22780714f0 con 0x7f22780cab00 2023-07-14T11:54:10.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f22767fc700 1 -- 172.21.15.104:0/2104709397 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22600035d0 con 0x7f22780cab00 2023-07-14T11:54:10.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f2276ffd700 1 -- 172.21.15.104:0/2104709397 >> v1:172.21.15.121:6789/0 conn(0x7f2278159c60 legacy=0x7f2278070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f2276ffd700 1 -- 172.21.15.104:0/2104709397 >> v1:172.21.15.104:6789/0 conn(0x7f22780cab00 legacy=0x7f22780c1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f2276ffd700 1 -- 172.21.15.104:0/2104709397 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227806dfc0 con 0x7f22780b0770 2023-07-14T11:54:10.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f2276ffd700 1 --2- 172.21.15.104:0/2104709397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22780b0770 0x7f227806d3b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f226c009850 tx=0x7f226c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45b13a8dd632a37c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:10.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f22767fc700 1 -- 172.21.15.104:0/2104709397 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c011830 con 0x7f22780b0770 2023-07-14T11:54:10.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f22767fc700 1 -- 172.21.15.104:0/2104709397 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226c011990 con 0x7f22780b0770 2023-07-14T11:54:10.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f22767fc700 1 -- 172.21.15.104:0/2104709397 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c011d40 con 0x7f22780b0770 2023-07-14T11:54:10.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f227cb3e700 1 -- 172.21.15.104:0/2104709397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22780b0770 msgr2=0x7f227806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f227cb3e700 1 --2- 172.21.15.104:0/2104709397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22780b0770 0x7f227806d3b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f226c009850 tx=0x7f226c009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:10.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f227cb3e700 1 -- 172.21.15.104:0/2104709397 shutdown_connections 2023-07-14T11:54:10.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f227cb3e700 1 --2- 172.21.15.104:0/2104709397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22780b0770 0x7f227806d3b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f227cb3e700 1 -- 172.21.15.104:0/2104709397 >> 172.21.15.104:0/2104709397 conn(0x7f22780afc60 msgr2=0x7f22780b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:10.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f227cb3e700 1 -- 172.21.15.104:0/2104709397 shutdown_connections 2023-07-14T11:54:10.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.904+0000 7f227cb3e700 1 -- 172.21.15.104:0/2104709397 wait complete. 2023-07-14T11:54:10.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.908+0000 7f227cb3e700 1 Processor -- start 2023-07-14T11:54:10.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.908+0000 7f227cb3e700 1 -- start start 2023-07-14T11:54:10.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.908+0000 7f227cb3e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22780b0770 0x7f2278152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.908+0000 7f227cb3e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22780cab00 0x7f22780c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.908+0000 7f227cb3e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2278159c60 0x7f22780b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.908+0000 7f227cb3e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2278152690 con 0x7f2278159c60 2023-07-14T11:54:10.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.908+0000 7f227cb3e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22780c38e0 con 0x7f22780b0770 2023-07-14T11:54:10.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.908+0000 7f227cb3e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22780c2d60 con 0x7f22780cab00 2023-07-14T11:54:10.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.908+0000 7f2277fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2278159c60 0x7f22780b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.908+0000 7f2277fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2278159c60 0x7f22780b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35004/0 (socket says 172.21.15.104:35004) 2023-07-14T11:54:10.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.908+0000 7f2277fff700 1 -- 172.21.15.104:0/1627495920 learned_addr learned my addr 172.21.15.104:0/1627495920 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:10.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.908+0000 7f2276ffd700 1 --2- 172.21.15.104:0/1627495920 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22780cab00 0x7f22780c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.908+0000 7f22777fe700 1 --2- 172.21.15.104:0/1627495920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22780b0770 0x7f2278152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.912+0000 7f2277fff700 1 -- 172.21.15.104:0/1627495920 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22780cab00 msgr2=0x7f22780c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.912+0000 7f2277fff700 1 --2- 172.21.15.104:0/1627495920 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22780cab00 0x7f22780c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.912+0000 7f2277fff700 1 -- 172.21.15.104:0/1627495920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22780b0770 msgr2=0x7f2278152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:10.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.912+0000 7f2277fff700 1 --2- 172.21.15.104:0/1627495920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22780b0770 0x7f2278152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:10.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.912+0000 7f2277fff700 1 -- 172.21.15.104:0/1627495920 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c0094b0 con 0x7f2278159c60 2023-07-14T11:54:10.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.912+0000 7f2277fff700 1 --2- 172.21.15.104:0/1627495920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2278159c60 0x7f22780b1460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f227000cb40 tx=0x7f227000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:10.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.912+0000 7f225ffff700 1 -- 172.21.15.104:0/1627495920 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227000a9e0 con 0x7f2278159c60 2023-07-14T11:54:10.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.912+0000 7f225ffff700 1 -- 172.21.15.104:0/1627495920 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227000ab40 con 0x7f2278159c60 2023-07-14T11:54:10.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.912+0000 7f227cb3e700 1 -- 172.21.15.104:0/1627495920 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22780c3040 con 0x7f2278159c60 2023-07-14T11:54:10.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.912+0000 7f225ffff700 1 -- 172.21.15.104:0/1627495920 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227000ce60 con 0x7f2278159c60 2023-07-14T11:54:10.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.912+0000 7f227cb3e700 1 -- 172.21.15.104:0/1627495920 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22780c3600 con 0x7f2278159c60 2023-07-14T11:54:10.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.912+0000 7f227cb3e700 1 -- 172.21.15.104:0/1627495920 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227804e500 con 0x7f2278159c60 2023-07-14T11:54:10.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.916+0000 7f225ffff700 1 -- 172.21.15.104:0/1627495920 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f227001b3b0 con 0x7f2278159c60 2023-07-14T11:54:10.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.916+0000 7f225ffff700 1 --2- 172.21.15.104:0/1627495920 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2264066320 0x7f22640687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:10.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.916+0000 7f225ffff700 1 -- 172.21.15.104:0/1627495920 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2270083b30 con 0x7f2278159c60 2023-07-14T11:54:10.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.916+0000 7f22777fe700 1 --2- 172.21.15.104:0/1627495920 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2264066320 0x7f22640687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:10.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.916+0000 7f22777fe700 1 --2- 172.21.15.104:0/1627495920 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2264066320 0x7f22640687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2268000f60 tx=0x7f2268009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:10.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:10.924+0000 7f225ffff700 1 -- 172.21.15.104:0/1627495920 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f2270050c70 con 0x7f2278159c60 2023-07-14T11:54:11.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.092+0000 7f227cb3e700 1 -- 172.21.15.104:0/1627495920 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f22780b11c0 con 0x7f2278159c60 2023-07-14T11:54:11.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.096+0000 7f225ffff700 1 -- 172.21.15.104:0/1627495920 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2270054290 con 0x7f2278159c60 2023-07-14T11:54:11.079 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:11.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.096+0000 7f227cb3e700 1 -- 172.21.15.104:0/1627495920 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2264066320 msgr2=0x7f22640687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:11.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.096+0000 7f227cb3e700 1 --2- 172.21.15.104:0/1627495920 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2264066320 0x7f22640687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2268000f60 tx=0x7f2268009040 comp rx=0 tx=0).stop 2023-07-14T11:54:11.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.096+0000 7f227cb3e700 1 -- 172.21.15.104:0/1627495920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2278159c60 msgr2=0x7f22780b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:11.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.096+0000 7f227cb3e700 1 --2- 172.21.15.104:0/1627495920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2278159c60 0x7f22780b1460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f227000cb40 tx=0x7f227000cc20 comp rx=0 tx=0).stop 2023-07-14T11:54:11.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.096+0000 7f227cb3e700 1 -- 172.21.15.104:0/1627495920 shutdown_connections 2023-07-14T11:54:11.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.096+0000 7f227cb3e700 1 --2- 172.21.15.104:0/1627495920 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2264066320 0x7f22640687d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.096+0000 7f227cb3e700 1 --2- 172.21.15.104:0/1627495920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2278159c60 0x7f22780b1460 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.096+0000 7f227cb3e700 1 --2- 172.21.15.104:0/1627495920 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22780cab00 0x7f22780c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.096+0000 7f227cb3e700 1 --2- 172.21.15.104:0/1627495920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22780b0770 0x7f2278152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.096+0000 7f227cb3e700 1 -- 172.21.15.104:0/1627495920 >> 172.21.15.104:0/1627495920 conn(0x7f22780afc60 msgr2=0x7f22780b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:11.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.096+0000 7f227cb3e700 1 -- 172.21.15.104:0/1627495920 shutdown_connections 2023-07-14T11:54:11.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.096+0000 7f227cb3e700 1 -- 172.21.15.104:0/1627495920 wait complete. 2023-07-14T11:54:11.103 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:11.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.288+0000 7f9e171e8700 1 Processor -- start 2023-07-14T11:54:11.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.288+0000 7f9e171e8700 1 -- start start 2023-07-14T11:54:11.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e171e8700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e10071000 0x7f9e1015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:11.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e171e8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9e100caa20 0x7f9e100cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:11.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e171e8700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1015d230 con 0x7f9e101599d0 2023-07-14T11:54:11.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e171e8700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e100cde10 con 0x7f9e100caa20 2023-07-14T11:54:11.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e171e8700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e100cb3c0 con 0x7f9e10071000 2023-07-14T11:54:11.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e161e6700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f9e101599d0 0x7f9e100c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:44768/0 (socket says 172.21.15.104:44768) 2023-07-14T11:54:11.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e161e6700 1 -- 172.21.15.104:0/3730614549 learned_addr learned my addr 172.21.15.104:0/3730614549 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:11.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e159e5700 1 --2- 172.21.15.104:0/3730614549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e10071000 0x7f9e1015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:11.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e169e7700 1 --2- 172.21.15.104:0/3730614549 >> v2:172.21.15.121:3300/0 conn(0x7f9e100caa20 0x7f9e100cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:11.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e151e4700 1 -- 172.21.15.104:0/3730614549 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102821341 0 0) 0x7f9e1015d230 con 0x7f9e101599d0 2023-07-14T11:54:11.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e151e4700 1 -- 172.21.15.104:0/3730614549 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dfc0035d0 con 0x7f9e101599d0 2023-07-14T11:54:11.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e159e5700 1 -- 172.21.15.104:0/3730614549 >> v2:172.21.15.121:3300/0 conn(0x7f9e100caa20 msgr2=0x7f9e100cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:11.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e159e5700 1 --2- 172.21.15.104:0/3730614549 >> v2:172.21.15.121:3300/0 conn(0x7f9e100caa20 0x7f9e100cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e159e5700 1 -- 172.21.15.104:0/3730614549 >> v1:172.21.15.104:6789/0 conn(0x7f9e101599d0 legacy=0x7f9e100c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:11.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e159e5700 1 -- 172.21.15.104:0/3730614549 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e100cd520 con 0x7f9e10071000 2023-07-14T11:54:11.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e159e5700 1 --2- 172.21.15.104:0/3730614549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e10071000 0x7f9e1015ccf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009850 tx=0x7f9e00009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=913e79f0f477f331 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:11.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e151e4700 1 -- 172.21.15.104:0/3730614549 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00011830 con 0x7f9e10071000 2023-07-14T11:54:11.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e151e4700 1 -- 172.21.15.104:0/3730614549 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e00011990 con 0x7f9e10071000 2023-07-14T11:54:11.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.292+0000 7f9e151e4700 1 -- 172.21.15.104:0/3730614549 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00011d40 con 0x7f9e10071000 2023-07-14T11:54:11.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.296+0000 7f9e171e8700 1 -- 172.21.15.104:0/3730614549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e10071000 msgr2=0x7f9e1015ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:11.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.296+0000 7f9e171e8700 1 --2- 172.21.15.104:0/3730614549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e10071000 0x7f9e1015ccf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009850 tx=0x7f9e00009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:11.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.296+0000 7f9e171e8700 1 -- 172.21.15.104:0/3730614549 shutdown_connections 2023-07-14T11:54:11.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.296+0000 7f9e171e8700 1 --2- 172.21.15.104:0/3730614549 >> v2:172.21.15.121:3300/0 conn(0x7f9e100caa20 0x7f9e100cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.296+0000 7f9e171e8700 1 --2- 172.21.15.104:0/3730614549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e10071000 0x7f9e1015ccf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.296+0000 7f9e171e8700 1 -- 172.21.15.104:0/3730614549 >> 172.21.15.104:0/3730614549 conn(0x7f9e1006e960 msgr2=0x7f9e1006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:11.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.296+0000 7f9e171e8700 1 -- 172.21.15.104:0/3730614549 shutdown_connections 2023-07-14T11:54:11.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.296+0000 7f9e171e8700 1 -- 172.21.15.104:0/3730614549 wait complete. 2023-07-14T11:54:11.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e171e8700 1 Processor -- start 2023-07-14T11:54:11.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e171e8700 1 -- start start 2023-07-14T11:54:11.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e171e8700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e10071000 0x7f9e10158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:11.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e171e8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e100caa20 0x7f9e100b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:11.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e171e8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e101599d0 0x7f9e1006cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:11.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e171e8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e100add30 con 0x7f9e100caa20 2023-07-14T11:54:11.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e171e8700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e100adfd0 con 0x7f9e101599d0 2023-07-14T11:54:11.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e171e8700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e100ae2b0 con 0x7f9e10071000 2023-07-14T11:54:11.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e159e5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e100caa20 0x7f9e100b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:11.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e159e5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e100caa20 0x7f9e100b3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35014/0 (socket says 172.21.15.104:35014) 2023-07-14T11:54:11.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e161e6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e10071000 0x7f9e10158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:11.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e159e5700 1 -- 172.21.15.104:0/2085131535 learned_addr learned my addr 172.21.15.104:0/2085131535 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:11.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e169e7700 1 --2- 172.21.15.104:0/2085131535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e101599d0 0x7f9e1006cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:11.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e161e6700 1 -- 172.21.15.104:0/2085131535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e101599d0 msgr2=0x7f9e1006cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:11.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e161e6700 1 --2- 172.21.15.104:0/2085131535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e101599d0 0x7f9e1006cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e161e6700 1 -- 172.21.15.104:0/2085131535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e100caa20 msgr2=0x7f9e100b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:11.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e161e6700 1 --2- 172.21.15.104:0/2085131535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e100caa20 0x7f9e100b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.300+0000 7f9e161e6700 1 -- 172.21.15.104:0/2085131535 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e000094b0 con 0x7f9e10071000 2023-07-14T11:54:11.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.304+0000 7f9e159e5700 1 --2- 172.21.15.104:0/2085131535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e100caa20 0x7f9e100b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:11.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.304+0000 7f9e161e6700 1 --2- 172.21.15.104:0/2085131535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e10071000 0x7f9e10158690 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00c500 tx=0x7f9e0c00c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:11.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.304+0000 7f9e06ffd700 1 -- 172.21.15.104:0/2085131535 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c005d90 con 0x7f9e10071000 2023-07-14T11:54:11.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.304+0000 7f9e06ffd700 1 -- 172.21.15.104:0/2085131535 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0c0070d0 con 0x7f9e10071000 2023-07-14T11:54:11.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.304+0000 7f9e171e8700 1 -- 172.21.15.104:0/2085131535 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e100ae590 con 0x7f9e10071000 2023-07-14T11:54:11.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.304+0000 7f9e06ffd700 1 -- 172.21.15.104:0/2085131535 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c007440 con 0x7f9e10071000 2023-07-14T11:54:11.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.304+0000 7f9e171e8700 1 -- 172.21.15.104:0/2085131535 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e10158bd0 con 0x7f9e10071000 2023-07-14T11:54:11.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.304+0000 7f9e171e8700 1 -- 172.21.15.104:0/2085131535 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e1004e500 con 0x7f9e10071000 2023-07-14T11:54:11.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.308+0000 7f9e06ffd700 1 -- 172.21.15.104:0/2085131535 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9e0c0075a0 con 0x7f9e10071000 2023-07-14T11:54:11.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.308+0000 7f9e06ffd700 1 --2- 172.21.15.104:0/2085131535 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9df40662f0 0x7f9df40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:11.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.308+0000 7f9e06ffd700 1 -- 172.21.15.104:0/2085131535 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9e0c082ff0 con 0x7f9e10071000 2023-07-14T11:54:11.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.308+0000 7f9e159e5700 1 --2- 172.21.15.104:0/2085131535 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9df40662f0 0x7f9df40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:11.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.308+0000 7f9e159e5700 1 --2- 172.21.15.104:0/2085131535 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9df40662f0 0x7f9df40687a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9e1015c1d0 tx=0x7f9e0001c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:11.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.316+0000 7f9e06ffd700 1 -- 172.21.15.104:0/2085131535 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f9e0c0b3720 con 0x7f9e10071000 2023-07-14T11:54:11.475 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.492+0000 7f9e171e8700 1 -- 172.21.15.104:0/2085131535 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e10070de0 con 0x7f9e10071000 2023-07-14T11:54:11.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.492+0000 7f9e06ffd700 1 -- 172.21.15.104:0/2085131535 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20251 (secure 0 0 0) 0x7f9e0c050190 con 0x7f9e10071000 2023-07-14T11:54:11.479 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:11.479 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":30,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:55.601783+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:11.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.496+0000 7f9e171e8700 1 -- 172.21.15.104:0/2085131535 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9df40662f0 msgr2=0x7f9df40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:11.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.496+0000 7f9e171e8700 1 --2- 172.21.15.104:0/2085131535 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9df40662f0 0x7f9df40687a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9e1015c1d0 tx=0x7f9e0001c040 comp rx=0 tx=0).stop 2023-07-14T11:54:11.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.496+0000 7f9e171e8700 1 -- 172.21.15.104:0/2085131535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e10071000 msgr2=0x7f9e10158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:11.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.496+0000 7f9e171e8700 1 --2- 172.21.15.104:0/2085131535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e10071000 0x7f9e10158690 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00c500 tx=0x7f9e0c00c810 comp rx=0 tx=0).stop 2023-07-14T11:54:11.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.500+0000 7f9e171e8700 1 -- 172.21.15.104:0/2085131535 shutdown_connections 2023-07-14T11:54:11.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.500+0000 7f9e171e8700 1 --2- 172.21.15.104:0/2085131535 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9df40662f0 0x7f9df40687a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.500+0000 7f9e171e8700 1 --2- 172.21.15.104:0/2085131535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e100caa20 0x7f9e100b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.500+0000 7f9e171e8700 1 --2- 172.21.15.104:0/2085131535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e101599d0 0x7f9e1006cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.500+0000 7f9e171e8700 1 --2- 172.21.15.104:0/2085131535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e10071000 0x7f9e10158690 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.500+0000 7f9e171e8700 1 -- 172.21.15.104:0/2085131535 >> 172.21.15.104:0/2085131535 conn(0x7f9e1006e960 msgr2=0x7f9e10153590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:11.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.500+0000 7f9e171e8700 1 -- 172.21.15.104:0/2085131535 shutdown_connections 2023-07-14T11:54:11.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.500+0000 7f9e171e8700 1 -- 172.21.15.104:0/2085131535 wait complete. 2023-07-14T11:54:11.505 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:54:11.506 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:54:11.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.700+0000 7faed8ee4700 1 Processor -- start 2023-07-14T11:54:11.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.700+0000 7faed8ee4700 1 -- start start 2023-07-14T11:54:11.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed8ee4700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7faed4069ca0 0x7faed40ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:11.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed8ee4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7faed406aaf0 0x7faed406e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:11.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed8ee4700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed415fd90 con 0x7faed406a680 2023-07-14T11:54:11.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed8ee4700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed4159eb0 con 0x7faed4069ca0 2023-07-14T11:54:11.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed8ee4700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed415fa90 con 0x7faed406aaf0 2023-07-14T11:54:11.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed37fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7faed4069ca0 0x7faed40ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:11.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed37fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7faed4069ca0 0x7faed40ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35020/0 (socket says 172.21.15.104:35020) 2023-07-14T11:54:11.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed37fe700 1 -- 172.21.15.104:0/2652085197 learned_addr learned my addr 172.21.15.104:0/2652085197 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:11.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed3fff700 1 --2- 172.21.15.104:0/2652085197 >> v2:172.21.15.121:3300/0 conn(0x7faed406aaf0 0x7faed406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:11.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed37fe700 1 -- 172.21.15.104:0/2652085197 >> v2:172.21.15.121:3300/0 conn(0x7faed406aaf0 msgr2=0x7faed406e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:11.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed37fe700 1 --2- 172.21.15.104:0/2652085197 >> v2:172.21.15.121:3300/0 conn(0x7faed406aaf0 0x7faed406e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed37fe700 1 -- 172.21.15.104:0/2652085197 >> v1:172.21.15.121:6789/0 conn(0x7faed406a680 legacy=0x7faed4160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:11.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed37fe700 1 -- 172.21.15.104:0/2652085197 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed4160930 con 0x7faed4069ca0 2023-07-14T11:54:11.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed37fe700 1 --2- 172.21.15.104:0/2652085197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faed4069ca0 0x7faed40ce2d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faed406e430 tx=0x7faec4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d4c6e1654aa19f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:11.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed27fc700 1 -- 172.21.15.104:0/2652085197 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec4004230 con 0x7faed4069ca0 2023-07-14T11:54:11.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed27fc700 1 -- 172.21.15.104:0/2652085197 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec4004390 con 0x7faed4069ca0 2023-07-14T11:54:11.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed27fc700 1 -- 172.21.15.104:0/2652085197 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec40036c0 con 0x7faed4069ca0 2023-07-14T11:54:11.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed8ee4700 1 -- 172.21.15.104:0/2652085197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faed4069ca0 msgr2=0x7faed40ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:11.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed8ee4700 1 --2- 172.21.15.104:0/2652085197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faed4069ca0 0x7faed40ce2d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faed406e430 tx=0x7faec4009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed8ee4700 1 -- 172.21.15.104:0/2652085197 shutdown_connections 2023-07-14T11:54:11.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed8ee4700 1 --2- 172.21.15.104:0/2652085197 >> v2:172.21.15.121:3300/0 conn(0x7faed406aaf0 0x7faed406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed8ee4700 1 --2- 172.21.15.104:0/2652085197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faed4069ca0 0x7faed40ce2d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.704+0000 7faed8ee4700 1 -- 172.21.15.104:0/2652085197 >> 172.21.15.104:0/2652085197 conn(0x7faed406fa70 msgr2=0x7faed406fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:11.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.708+0000 7faed8ee4700 1 -- 172.21.15.104:0/2652085197 shutdown_connections 2023-07-14T11:54:11.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.708+0000 7faed8ee4700 1 -- 172.21.15.104:0/2652085197 wait complete. 2023-07-14T11:54:11.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.708+0000 7faed8ee4700 1 Processor -- start 2023-07-14T11:54:11.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.708+0000 7faed8ee4700 1 -- start start 2023-07-14T11:54:11.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.708+0000 7faed8ee4700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faed4069ca0 0x7faed41f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:11.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.708+0000 7faed8ee4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faed406a680 0x7faed4156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:11.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.708+0000 7faed8ee4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faed406aaf0 0x7faed40c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:11.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.708+0000 7faed8ee4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed41f2b90 con 0x7faed406a680 2023-07-14T11:54:11.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.708+0000 7faed8ee4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed40c67d0 con 0x7faed406aaf0 2023-07-14T11:54:11.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.708+0000 7faed8ee4700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed40c6a30 con 0x7faed4069ca0 2023-07-14T11:54:11.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faed3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faed406aaf0 0x7faed40c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:11.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faed37fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faed4069ca0 0x7faed41f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:11.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faed3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faed406aaf0 0x7faed40c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:53112/0 (socket says 172.21.15.104:53112) 2023-07-14T11:54:11.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faed2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faed406a680 0x7faed4156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:11.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faed3fff700 1 -- 172.21.15.104:0/2698640752 learned_addr learned my addr 172.21.15.104:0/2698640752 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:11.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faed3fff700 1 -- 172.21.15.104:0/2698640752 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faed4069ca0 msgr2=0x7faed41f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:11.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faed3fff700 1 --2- 172.21.15.104:0/2698640752 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faed4069ca0 0x7faed41f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faed3fff700 1 -- 172.21.15.104:0/2698640752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faed406a680 msgr2=0x7faed4156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:11.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faed3fff700 1 --2- 172.21.15.104:0/2698640752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faed406a680 0x7faed4156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faed3fff700 1 -- 172.21.15.104:0/2698640752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec4009570 con 0x7faed406aaf0 2023-07-14T11:54:11.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faed3fff700 1 --2- 172.21.15.104:0/2698640752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faed406aaf0 0x7faed40c6290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faecc007d40 tx=0x7faecc00e3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:11.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faebbfff700 1 -- 172.21.15.104:0/2698640752 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc00ed30 con 0x7faed406aaf0 2023-07-14T11:54:11.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faebbfff700 1 -- 172.21.15.104:0/2698640752 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faecc007e90 con 0x7faed406aaf0 2023-07-14T11:54:11.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faebbfff700 1 -- 172.21.15.104:0/2698640752 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc014660 con 0x7faed406aaf0 2023-07-14T11:54:11.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faed8ee4700 1 -- 172.21.15.104:0/2698640752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed40c6d10 con 0x7faed406aaf0 2023-07-14T11:54:11.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.712+0000 7faed8ee4700 1 -- 172.21.15.104:0/2698640752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed41f2820 con 0x7faed406aaf0 2023-07-14T11:54:11.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.716+0000 7faed8ee4700 1 -- 172.21.15.104:0/2698640752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed40c9660 con 0x7faed406aaf0 2023-07-14T11:54:11.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.716+0000 7faebbfff700 1 -- 172.21.15.104:0/2698640752 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7faecc0147c0 con 0x7faed406aaf0 2023-07-14T11:54:11.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.716+0000 7faebbfff700 1 --2- 172.21.15.104:0/2698640752 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7faec00662f0 0x7faec00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:11.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.716+0000 7faebbfff700 1 -- 172.21.15.104:0/2698640752 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7faecc084620 con 0x7faed406aaf0 2023-07-14T11:54:11.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.716+0000 7faed37fe700 1 --2- 172.21.15.104:0/2698640752 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7faec00662f0 0x7faec00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:11.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.716+0000 7faed37fe700 1 --2- 172.21.15.104:0/2698640752 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7faec00662f0 0x7faec00687a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faed406e430 tx=0x7faec4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:11.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.724+0000 7faebbfff700 1 -- 172.21.15.104:0/2698640752 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7faecc0516d0 con 0x7faed406aaf0 2023-07-14T11:54:11.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.896+0000 7faed8ee4700 1 -- 172.21.15.104:0/2698640752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faed41f2680 con 0x7faed406aaf0 2023-07-14T11:54:11.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.896+0000 7faebbfff700 1 -- 172.21.15.104:0/2698640752 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7faed41f2680 con 0x7faed406aaf0 2023-07-14T11:54:11.883 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:54:11.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.900+0000 7faed8ee4700 1 -- 172.21.15.104:0/2698640752 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7faec00662f0 msgr2=0x7faec00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:11.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.900+0000 7faed8ee4700 1 --2- 172.21.15.104:0/2698640752 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7faec00662f0 0x7faec00687a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faed406e430 tx=0x7faec4008030 comp rx=0 tx=0).stop 2023-07-14T11:54:11.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.900+0000 7faed8ee4700 1 -- 172.21.15.104:0/2698640752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faed406aaf0 msgr2=0x7faed40c6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:11.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.900+0000 7faed8ee4700 1 --2- 172.21.15.104:0/2698640752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faed406aaf0 0x7faed40c6290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faecc007d40 tx=0x7faecc00e3c0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.900+0000 7faed8ee4700 1 -- 172.21.15.104:0/2698640752 shutdown_connections 2023-07-14T11:54:11.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.900+0000 7faed8ee4700 1 --2- 172.21.15.104:0/2698640752 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7faec00662f0 0x7faec00687a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.900+0000 7faed8ee4700 1 --2- 172.21.15.104:0/2698640752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faed406a680 0x7faed4156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.900+0000 7faed8ee4700 1 --2- 172.21.15.104:0/2698640752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faed406aaf0 0x7faed40c6290 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.900+0000 7faed8ee4700 1 --2- 172.21.15.104:0/2698640752 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faed4069ca0 0x7faed41f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:11.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.900+0000 7faed8ee4700 1 -- 172.21.15.104:0/2698640752 >> 172.21.15.104:0/2698640752 conn(0x7faed406fa70 msgr2=0x7faed4160220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:11.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.900+0000 7faed8ee4700 1 -- 172.21.15.104:0/2698640752 shutdown_connections 2023-07-14T11:54:11.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:11.900+0000 7faed8ee4700 1 -- 172.21.15.104:0/2698640752 wait complete. 2023-07-14T11:54:11.909 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:54:12.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.100+0000 7f73e7fff700 1 Processor -- start 2023-07-14T11:54:12.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.100+0000 7f73e7fff700 1 -- start start 2023-07-14T11:54:12.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e7fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73e8160000 0x7f73e806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e7fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f73e81603d0 0x7f73e80c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e7fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f73e80cd7f0 0x7f73e80ca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e7fff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e8070e40 con 0x7f73e80cd7f0 2023-07-14T11:54:12.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e7fff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e80cdfb0 con 0x7f73e81603d0 2023-07-14T11:54:12.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e7fff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80ca8c0 con 0x7f73e8160000 2023-07-14T11:54:12.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e77fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f73e80cd7f0 0x7f73e80ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e77fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f73e80cd7f0 0x7f73e80ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35028/0 (socket says 172.21.15.104:35028) 2023-07-14T11:54:12.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e77fe700 1 -- 172.21.15.104:0/1089342231 learned_addr learned my addr 172.21.15.104:0/1089342231 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:12.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e6ffd700 1 --2- 172.21.15.104:0/1089342231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73e8160000 0x7f73e806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e67fc700 1 --2- 172.21.15.104:0/1089342231 >> v2:172.21.15.121:3300/0 conn(0x7f73e81603d0 0x7f73e80c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e77fe700 1 -- 172.21.15.104:0/1089342231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73e8160000 msgr2=0x7f73e806dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:12.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e77fe700 1 --2- 172.21.15.104:0/1089342231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73e8160000 0x7f73e806dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e77fe700 1 -- 172.21.15.104:0/1089342231 >> v2:172.21.15.121:3300/0 conn(0x7f73e81603d0 msgr2=0x7f73e80c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:12.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e77fe700 1 --2- 172.21.15.104:0/1089342231 >> v2:172.21.15.121:3300/0 conn(0x7f73e81603d0 0x7f73e80c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e77fe700 1 -- 172.21.15.104:0/1089342231 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e80cdcd0 con 0x7f73e80cd7f0 2023-07-14T11:54:12.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e77fe700 1 --2- 172.21.15.104:0/1089342231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73e80cd7f0 0x7f73e80ca380 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f73e806b120 tx=0x7f73e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b426a325d8bee4e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:12.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e5ffb700 1 -- 172.21.15.104:0/1089342231 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e000e070 con 0x7f73e80cd7f0 2023-07-14T11:54:12.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e5ffb700 1 -- 172.21.15.104:0/1089342231 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e000b9c0 con 0x7f73e80cd7f0 2023-07-14T11:54:12.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.104+0000 7f73e5ffb700 1 -- 172.21.15.104:0/1089342231 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e000bd70 con 0x7f73e80cd7f0 2023-07-14T11:54:12.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.108+0000 7f73e7fff700 1 -- 172.21.15.104:0/1089342231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73e80cd7f0 msgr2=0x7f73e80ca380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:12.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.108+0000 7f73e7fff700 1 --2- 172.21.15.104:0/1089342231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73e80cd7f0 0x7f73e80ca380 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f73e806b120 tx=0x7f73e000b330 comp rx=0 tx=0).stop 2023-07-14T11:54:12.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.108+0000 7f73e7fff700 1 -- 172.21.15.104:0/1089342231 shutdown_connections 2023-07-14T11:54:12.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.108+0000 7f73e7fff700 1 --2- 172.21.15.104:0/1089342231 >> v2:172.21.15.121:3300/0 conn(0x7f73e81603d0 0x7f73e80c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.108+0000 7f73e7fff700 1 --2- 172.21.15.104:0/1089342231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73e80cd7f0 0x7f73e80ca380 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.108+0000 7f73e7fff700 1 --2- 172.21.15.104:0/1089342231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73e8160000 0x7f73e806dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.108+0000 7f73e7fff700 1 -- 172.21.15.104:0/1089342231 >> 172.21.15.104:0/1089342231 conn(0x7f73e80afc60 msgr2=0x7f73e80b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:12.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.108+0000 7f73e7fff700 1 -- 172.21.15.104:0/1089342231 shutdown_connections 2023-07-14T11:54:12.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.108+0000 7f73e7fff700 1 -- 172.21.15.104:0/1089342231 wait complete. 2023-07-14T11:54:12.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.108+0000 7f73e7fff700 1 Processor -- start 2023-07-14T11:54:12.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.108+0000 7f73e7fff700 1 -- start start 2023-07-14T11:54:12.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e7fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73e80cd7f0 0x7f73e81641d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73e8160000 0x7f73e80c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73e81603d0 0x7f73e80add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e7fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e8161440 con 0x7f73e8160000 2023-07-14T11:54:12.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e7fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80c72f0 con 0x7f73e81603d0 2023-07-14T11:54:12.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e7fff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e81607c0 con 0x7f73e80cd7f0 2023-07-14T11:54:12.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e67fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73e8160000 0x7f73e80c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e67fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73e8160000 0x7f73e80c6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35036/0 (socket says 172.21.15.104:35036) 2023-07-14T11:54:12.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e67fc700 1 -- 172.21.15.104:0/1936556474 learned_addr learned my addr 172.21.15.104:0/1936556474 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:12.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e77fe700 1 --2- 172.21.15.104:0/1936556474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73e81603d0 0x7f73e80add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e6ffd700 1 --2- 172.21.15.104:0/1936556474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73e80cd7f0 0x7f73e81641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e67fc700 1 -- 172.21.15.104:0/1936556474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73e80cd7f0 msgr2=0x7f73e81641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:12.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e67fc700 1 --2- 172.21.15.104:0/1936556474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73e80cd7f0 0x7f73e81641d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e67fc700 1 -- 172.21.15.104:0/1936556474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73e81603d0 msgr2=0x7f73e80add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:12.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e67fc700 1 --2- 172.21.15.104:0/1936556474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73e81603d0 0x7f73e80add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e67fc700 1 -- 172.21.15.104:0/1936556474 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e0009b30 con 0x7f73e8160000 2023-07-14T11:54:12.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e77fe700 1 --2- 172.21.15.104:0/1936556474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73e81603d0 0x7f73e80add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:12.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e67fc700 1 --2- 172.21.15.104:0/1936556474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73e8160000 0x7f73e80c6db0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00e760 tx=0x7f73dc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:12.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73cf7fe700 1 -- 172.21.15.104:0/1936556474 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc013610 con 0x7f73e8160000 2023-07-14T11:54:12.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e7fff700 1 -- 172.21.15.104:0/1936556474 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e8160aa0 con 0x7f73e8160000 2023-07-14T11:54:12.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73cf7fe700 1 -- 172.21.15.104:0/1936556474 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc013770 con 0x7f73e8160000 2023-07-14T11:54:12.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73cf7fe700 1 -- 172.21.15.104:0/1936556474 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc013ae0 con 0x7f73e8160000 2023-07-14T11:54:12.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.112+0000 7f73e7fff700 1 -- 172.21.15.104:0/1936556474 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e8161060 con 0x7f73e8160000 2023-07-14T11:54:12.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.116+0000 7f73e7fff700 1 -- 172.21.15.104:0/1936556474 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e80cf020 con 0x7f73e8160000 2023-07-14T11:54:12.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.116+0000 7f73cf7fe700 1 -- 172.21.15.104:0/1936556474 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f73dc013c40 con 0x7f73e8160000 2023-07-14T11:54:12.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.116+0000 7f73cf7fe700 1 --2- 172.21.15.104:0/1936556474 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f73d40662f0 0x7f73d40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.116+0000 7f73cf7fe700 1 -- 172.21.15.104:0/1936556474 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f73dc083e00 con 0x7f73e8160000 2023-07-14T11:54:12.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.116+0000 7f73e6ffd700 1 --2- 172.21.15.104:0/1936556474 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f73d40662f0 0x7f73d40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.120+0000 7f73e6ffd700 1 --2- 172.21.15.104:0/1936556474 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f73d40662f0 0x7f73d40687a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f73d800c920 tx=0x7f73d8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:12.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.124+0000 7f73cf7fe700 1 -- 172.21.15.104:0/1936556474 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f73dc050ed0 con 0x7f73e8160000 2023-07-14T11:54:12.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.292+0000 7f73e7fff700 1 -- 172.21.15.104:0/1936556474 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f73e804e500 con 0x7f73e8160000 2023-07-14T11:54:12.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.292+0000 7f73cf7fe700 1 -- 172.21.15.104:0/1936556474 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f73dc0544f0 con 0x7f73e8160000 2023-07-14T11:54:12.278 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:54:12.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.296+0000 7f73e7fff700 1 -- 172.21.15.104:0/1936556474 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f73d40662f0 msgr2=0x7f73d40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:12.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.296+0000 7f73e7fff700 1 --2- 172.21.15.104:0/1936556474 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f73d40662f0 0x7f73d40687a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f73d800c920 tx=0x7f73d8008040 comp rx=0 tx=0).stop 2023-07-14T11:54:12.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.296+0000 7f73e7fff700 1 -- 172.21.15.104:0/1936556474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73e8160000 msgr2=0x7f73e80c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:12.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.296+0000 7f73e7fff700 1 --2- 172.21.15.104:0/1936556474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73e8160000 0x7f73e80c6db0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00e760 tx=0x7f73dc00eb20 comp rx=0 tx=0).stop 2023-07-14T11:54:12.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.296+0000 7f73e7fff700 1 -- 172.21.15.104:0/1936556474 shutdown_connections 2023-07-14T11:54:12.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.296+0000 7f73e7fff700 1 --2- 172.21.15.104:0/1936556474 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f73d40662f0 0x7f73d40687a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.296+0000 7f73e7fff700 1 --2- 172.21.15.104:0/1936556474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73e8160000 0x7f73e80c6db0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.296+0000 7f73e7fff700 1 --2- 172.21.15.104:0/1936556474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73e81603d0 0x7f73e80add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.296+0000 7f73e7fff700 1 --2- 172.21.15.104:0/1936556474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73e80cd7f0 0x7f73e81641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.296+0000 7f73e7fff700 1 -- 172.21.15.104:0/1936556474 >> 172.21.15.104:0/1936556474 conn(0x7f73e80afc60 msgr2=0x7f73e80c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:12.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.296+0000 7f73e7fff700 1 -- 172.21.15.104:0/1936556474 shutdown_connections 2023-07-14T11:54:12.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.296+0000 7f73e7fff700 1 -- 172.21.15.104:0/1936556474 wait complete. 2023-07-14T11:54:12.303 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:54:12.475 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.492+0000 7fef9f59e700 1 Processor -- start 2023-07-14T11:54:12.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.492+0000 7fef9f59e700 1 -- start start 2023-07-14T11:54:12.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.492+0000 7fef9f59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefa0159c60 0x7fefa006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.492+0000 7fef9f59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fefa00b0770 0x7fefa0070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.492+0000 7fef9f59e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa006da00 con 0x7fefa00cab00 2023-07-14T11:54:12.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.492+0000 7fef9f59e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa006dce0 con 0x7fefa00b0770 2023-07-14T11:54:12.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.492+0000 7fef9f59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa006dfc0 con 0x7fefa0159c60 2023-07-14T11:54:12.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9ed9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fefa00cab00 0x7fefa00c1ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:44782/0 (socket says 172.21.15.104:44782) 2023-07-14T11:54:12.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9ed9d700 1 -- 172.21.15.104:0/1009796646 learned_addr learned my addr 172.21.15.104:0/1009796646 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:12.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9dd9b700 1 --2- 172.21.15.104:0/1009796646 >> v2:172.21.15.104:3300/0 conn(0x7fefa00b0770 0x7fefa0070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9e59c700 1 --2- 172.21.15.104:0/1009796646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefa0159c60 0x7fefa006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9d59a700 1 -- 172.21.15.104:0/1009796646 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251106169 0 0) 0x7fefa006da00 con 0x7fefa00cab00 2023-07-14T11:54:12.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9d59a700 1 -- 172.21.15.104:0/1009796646 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef880035d0 con 0x7fefa00cab00 2023-07-14T11:54:12.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9dd9b700 1 -- 172.21.15.104:0/1009796646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefa0159c60 msgr2=0x7fefa006d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:12.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9dd9b700 1 --2- 172.21.15.104:0/1009796646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefa0159c60 0x7fefa006d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9dd9b700 1 -- 172.21.15.104:0/1009796646 >> v1:172.21.15.104:6789/0 conn(0x7fefa00cab00 legacy=0x7fefa00c1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:12.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9dd9b700 1 -- 172.21.15.104:0/1009796646 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa00705e0 con 0x7fefa00b0770 2023-07-14T11:54:12.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9dd9b700 1 --2- 172.21.15.104:0/1009796646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefa00b0770 0x7fefa0070de0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fef94009850 tx=0x7fef94009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35886784c667b5a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:12.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9d59a700 1 -- 172.21.15.104:0/1009796646 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94011830 con 0x7fefa00b0770 2023-07-14T11:54:12.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9d59a700 1 -- 172.21.15.104:0/1009796646 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef94011990 con 0x7fefa00b0770 2023-07-14T11:54:12.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9d59a700 1 -- 172.21.15.104:0/1009796646 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94011d00 con 0x7fefa00b0770 2023-07-14T11:54:12.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9f59e700 1 -- 172.21.15.104:0/1009796646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefa00b0770 msgr2=0x7fefa0070de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:12.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.496+0000 7fef9f59e700 1 --2- 172.21.15.104:0/1009796646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefa00b0770 0x7fefa0070de0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fef94009850 tx=0x7fef94009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:12.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.500+0000 7fef9f59e700 1 -- 172.21.15.104:0/1009796646 shutdown_connections 2023-07-14T11:54:12.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.500+0000 7fef9f59e700 1 --2- 172.21.15.104:0/1009796646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefa00b0770 0x7fefa0070de0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.500+0000 7fef9f59e700 1 --2- 172.21.15.104:0/1009796646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefa0159c60 0x7fefa006d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.500+0000 7fef9f59e700 1 -- 172.21.15.104:0/1009796646 >> 172.21.15.104:0/1009796646 conn(0x7fefa00afc60 msgr2=0x7fefa00b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:12.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.500+0000 7fef9f59e700 1 -- 172.21.15.104:0/1009796646 shutdown_connections 2023-07-14T11:54:12.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.500+0000 7fef9f59e700 1 -- 172.21.15.104:0/1009796646 wait complete. 2023-07-14T11:54:12.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.500+0000 7fef9f59e700 1 Processor -- start 2023-07-14T11:54:12.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9f59e700 1 -- start start 2023-07-14T11:54:12.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9f59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefa00b0770 0x7fefa01e1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9f59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefa00cab00 0x7fefa00c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefa0159c60 0x7fefa00c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9f59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00c7210 con 0x7fefa00b0770 2023-07-14T11:54:12.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9f59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00c6c50 con 0x7fefa0159c60 2023-07-14T11:54:12.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9f59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00c6f30 con 0x7fefa00cab00 2023-07-14T11:54:12.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefa00b0770 0x7fefa01e1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefa00b0770 0x7fefa01e1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35062/0 (socket says 172.21.15.104:35062) 2023-07-14T11:54:12.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9e59c700 1 -- 172.21.15.104:0/3165003076 learned_addr learned my addr 172.21.15.104:0/3165003076 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:12.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9dd9b700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefa00cab00 0x7fefa00c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9ed9d700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefa0159c60 0x7fefa00c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9e59c700 1 -- 172.21.15.104:0/3165003076 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefa00cab00 msgr2=0x7fefa00c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:12.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9e59c700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefa00cab00 0x7fefa00c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9e59c700 1 -- 172.21.15.104:0/3165003076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefa0159c60 msgr2=0x7fefa00c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:12.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9e59c700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefa0159c60 0x7fefa00c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9e59c700 1 -- 172.21.15.104:0/3165003076 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef940094b0 con 0x7fefa00b0770 2023-07-14T11:54:12.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9dd9b700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefa00cab00 0x7fefa00c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:12.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9ed9d700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefa0159c60 0x7fefa00c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:12.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.504+0000 7fef9e59c700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefa00b0770 0x7fefa01e1be0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fef9000e840 tx=0x7fef9000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:12.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.508+0000 7fef8effd700 1 -- 172.21.15.104:0/3165003076 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef900095a0 con 0x7fefa00b0770 2023-07-14T11:54:12.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.508+0000 7fef8effd700 1 -- 172.21.15.104:0/3165003076 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9000c690 con 0x7fefa00b0770 2023-07-14T11:54:12.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.508+0000 7fef9f59e700 1 -- 172.21.15.104:0/3165003076 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa00cbe50 con 0x7fefa00b0770 2023-07-14T11:54:12.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.508+0000 7fef8effd700 1 -- 172.21.15.104:0/3165003076 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9000ca00 con 0x7fefa00b0770 2023-07-14T11:54:12.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.508+0000 7fef9f59e700 1 -- 172.21.15.104:0/3165003076 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa01e2120 con 0x7fefa00b0770 2023-07-14T11:54:12.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.508+0000 7fef9f59e700 1 -- 172.21.15.104:0/3165003076 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa004e500 con 0x7fefa00b0770 2023-07-14T11:54:12.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.508+0000 7fef8effd700 1 -- 172.21.15.104:0/3165003076 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fef9000cb60 con 0x7fefa00b0770 2023-07-14T11:54:12.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.512+0000 7fef8effd700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fef800662f0 0x7fef800687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.512+0000 7fef8effd700 1 -- 172.21.15.104:0/3165003076 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fef90083fe0 con 0x7fefa00b0770 2023-07-14T11:54:12.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.512+0000 7fef9dd9b700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fef800662f0 0x7fef800687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.512+0000 7fef9dd9b700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fef800662f0 0x7fef800687a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fef940090b0 tx=0x7fef94008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:12.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.516+0000 7fef8effd700 1 -- 172.21.15.104:0/3165003076 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fef90054bf0 con 0x7fefa00b0770 2023-07-14T11:54:12.679 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.696+0000 7fef9f59e700 1 -- 172.21.15.104:0/3165003076 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fefa0158110 con 0x7fefa00b0770 2023-07-14T11:54:12.680 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.696+0000 7fef8effd700 1 -- 172.21.15.104:0/3165003076 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef90017020 con 0x7fefa00b0770 2023-07-14T11:54:12.681 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:12.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.700+0000 7fef9f59e700 1 -- 172.21.15.104:0/3165003076 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fef800662f0 msgr2=0x7fef800687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:12.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.700+0000 7fef9f59e700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fef800662f0 0x7fef800687a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fef940090b0 tx=0x7fef94008030 comp rx=0 tx=0).stop 2023-07-14T11:54:12.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.700+0000 7fef9f59e700 1 -- 172.21.15.104:0/3165003076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefa00b0770 msgr2=0x7fefa01e1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:12.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.700+0000 7fef9f59e700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefa00b0770 0x7fefa01e1be0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fef9000e840 tx=0x7fef9000ec00 comp rx=0 tx=0).stop 2023-07-14T11:54:12.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.700+0000 7fef9f59e700 1 -- 172.21.15.104:0/3165003076 shutdown_connections 2023-07-14T11:54:12.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.700+0000 7fef9f59e700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fef800662f0 0x7fef800687a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.700+0000 7fef9f59e700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefa0159c60 0x7fefa00c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.700+0000 7fef9f59e700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fefa00cab00 0x7fefa00c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.700+0000 7fef9f59e700 1 --2- 172.21.15.104:0/3165003076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefa00b0770 0x7fefa01e1be0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.700+0000 7fef9f59e700 1 -- 172.21.15.104:0/3165003076 >> 172.21.15.104:0/3165003076 conn(0x7fefa00afc60 msgr2=0x7fefa0157020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:12.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.700+0000 7fef9f59e700 1 -- 172.21.15.104:0/3165003076 shutdown_connections 2023-07-14T11:54:12.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.700+0000 7fef9f59e700 1 -- 172.21.15.104:0/3165003076 wait complete. 2023-07-14T11:54:12.705 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:54:12.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.900+0000 7fcea8192700 1 Processor -- start 2023-07-14T11:54:12.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.900+0000 7fcea8192700 1 -- start start 2023-07-14T11:54:12.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea8192700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcea015df10 0x7fcea015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea8192700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcea00c8950 0x7fcea00c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea8192700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea0070a30 con 0x7fcea00afd80 2023-07-14T11:54:12.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea8192700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea00c92f0 con 0x7fcea00c8950 2023-07-14T11:54:12.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea8192700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea015e830 con 0x7fcea015df10 2023-07-14T11:54:12.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea572d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fcea00afd80 0x7fcea00711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:44798/0 (socket says 172.21.15.104:44798) 2023-07-14T11:54:12.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea572d700 1 -- 172.21.15.104:0/2398744455 learned_addr learned my addr 172.21.15.104:0/2398744455 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:12.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea672f700 1 --2- 172.21.15.104:0/2398744455 >> v2:172.21.15.121:3300/0 conn(0x7fcea00c8950 0x7fcea00c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea5f2e700 1 --2- 172.21.15.104:0/2398744455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcea015df10 0x7fcea015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea4f2c700 1 -- 172.21.15.104:0/2398744455 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974194003 0 0) 0x7fcea0070a30 con 0x7fcea00afd80 2023-07-14T11:54:12.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea4f2c700 1 -- 172.21.15.104:0/2398744455 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce8c0035d0 con 0x7fcea00afd80 2023-07-14T11:54:12.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea5f2e700 1 -- 172.21.15.104:0/2398744455 >> v2:172.21.15.121:3300/0 conn(0x7fcea00c8950 msgr2=0x7fcea00c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:12.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea5f2e700 1 --2- 172.21.15.104:0/2398744455 >> v2:172.21.15.121:3300/0 conn(0x7fcea00c8950 0x7fcea00c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea5f2e700 1 -- 172.21.15.104:0/2398744455 >> v1:172.21.15.104:6789/0 conn(0x7fcea00afd80 legacy=0x7fcea00711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:12.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea5f2e700 1 -- 172.21.15.104:0/2398744455 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea00b0210 con 0x7fcea015df10 2023-07-14T11:54:12.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea5f2e700 1 --2- 172.21.15.104:0/2398744455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcea015df10 0x7fcea015e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009850 tx=0x7fce9c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac320a14e1a266e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:12.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea4f2c700 1 -- 172.21.15.104:0/2398744455 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c011830 con 0x7fcea015df10 2023-07-14T11:54:12.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea4f2c700 1 -- 172.21.15.104:0/2398744455 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c011990 con 0x7fcea015df10 2023-07-14T11:54:12.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.904+0000 7fcea4f2c700 1 -- 172.21.15.104:0/2398744455 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c011d40 con 0x7fcea015df10 2023-07-14T11:54:12.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.908+0000 7fcea8192700 1 -- 172.21.15.104:0/2398744455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcea015df10 msgr2=0x7fcea015e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:12.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.908+0000 7fcea8192700 1 --2- 172.21.15.104:0/2398744455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcea015df10 0x7fcea015e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009850 tx=0x7fce9c009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:12.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.908+0000 7fcea8192700 1 -- 172.21.15.104:0/2398744455 shutdown_connections 2023-07-14T11:54:12.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.908+0000 7fcea8192700 1 --2- 172.21.15.104:0/2398744455 >> v2:172.21.15.121:3300/0 conn(0x7fcea00c8950 0x7fcea00c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.908+0000 7fcea8192700 1 --2- 172.21.15.104:0/2398744455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcea015df10 0x7fcea015e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.908+0000 7fcea8192700 1 -- 172.21.15.104:0/2398744455 >> 172.21.15.104:0/2398744455 conn(0x7fcea00cad80 msgr2=0x7fcea00cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:12.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.908+0000 7fcea8192700 1 -- 172.21.15.104:0/2398744455 shutdown_connections 2023-07-14T11:54:12.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.908+0000 7fcea8192700 1 -- 172.21.15.104:0/2398744455 wait complete. 2023-07-14T11:54:12.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.908+0000 7fcea8192700 1 Processor -- start 2023-07-14T11:54:12.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea8192700 1 -- start start 2023-07-14T11:54:12.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea8192700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcea00afd80 0x7fcea0155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea8192700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcea00c8950 0x7fcea00cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea8192700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcea00cd760 0x7fcea00c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea8192700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea00ad3c0 con 0x7fcea00c8950 2023-07-14T11:54:12.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea8192700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea00b28a0 con 0x7fcea00cd760 2023-07-14T11:54:12.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea8192700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea00ac560 con 0x7fcea00afd80 2023-07-14T11:54:12.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea572d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcea00c8950 0x7fcea00cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea572d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcea00c8950 0x7fcea00cbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35068/0 (socket says 172.21.15.104:35068) 2023-07-14T11:54:12.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea572d700 1 -- 172.21.15.104:0/2684767855 learned_addr learned my addr 172.21.15.104:0/2684767855 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:12.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea5f2e700 1 --2- 172.21.15.104:0/2684767855 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcea00afd80 0x7fcea0155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea672f700 1 --2- 172.21.15.104:0/2684767855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcea00cd760 0x7fcea00c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea572d700 1 -- 172.21.15.104:0/2684767855 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcea00afd80 msgr2=0x7fcea0155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:12.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea572d700 1 --2- 172.21.15.104:0/2684767855 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcea00afd80 0x7fcea0155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea572d700 1 -- 172.21.15.104:0/2684767855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcea00cd760 msgr2=0x7fcea00c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:12.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea572d700 1 --2- 172.21.15.104:0/2684767855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcea00cd760 0x7fcea00c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:12.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea572d700 1 -- 172.21.15.104:0/2684767855 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9c0094b0 con 0x7fcea00c8950 2023-07-14T11:54:12.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fcea572d700 1 --2- 172.21.15.104:0/2684767855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcea00c8950 0x7fcea00cbed0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fce9000c730 tx=0x7fce9000ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:12.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fce967fc700 1 -- 172.21.15.104:0/2684767855 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce90005d90 con 0x7fcea00c8950 2023-07-14T11:54:12.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.912+0000 7fce967fc700 1 -- 172.21.15.104:0/2684767855 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce900070d0 con 0x7fcea00c8950 2023-07-14T11:54:12.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.916+0000 7fcea8192700 1 -- 172.21.15.104:0/2684767855 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea00ac7b0 con 0x7fcea00c8950 2023-07-14T11:54:12.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.916+0000 7fce967fc700 1 -- 172.21.15.104:0/2684767855 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce90007460 con 0x7fcea00c8950 2023-07-14T11:54:12.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.916+0000 7fcea8192700 1 -- 172.21.15.104:0/2684767855 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea00acd70 con 0x7fcea00c8950 2023-07-14T11:54:12.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.916+0000 7fcea8192700 1 -- 172.21.15.104:0/2684767855 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea00b29e0 con 0x7fcea00c8950 2023-07-14T11:54:12.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.916+0000 7fce967fc700 1 -- 172.21.15.104:0/2684767855 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fce900075c0 con 0x7fcea00c8950 2023-07-14T11:54:12.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.920+0000 7fce967fc700 1 --2- 172.21.15.104:0/2684767855 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fce840662f0 0x7fce840687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:12.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.920+0000 7fce967fc700 1 -- 172.21.15.104:0/2684767855 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fce90083560 con 0x7fcea00c8950 2023-07-14T11:54:12.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.920+0000 7fcea5f2e700 1 --2- 172.21.15.104:0/2684767855 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fce840662f0 0x7fce840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:12.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.920+0000 7fcea5f2e700 1 --2- 172.21.15.104:0/2684767855 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fce840662f0 0x7fce840687a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0090b0 tx=0x7fce9c01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:12.911 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:12.928+0000 7fce967fc700 1 -- 172.21.15.104:0/2684767855 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fce90050610 con 0x7fcea00c8950 2023-07-14T11:54:13.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.096+0000 7fcea8192700 1 -- 172.21.15.104:0/2684767855 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcea004dcb0 con 0x7fcea00c8950 2023-07-14T11:54:13.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.100+0000 7fce967fc700 1 -- 172.21.15.104:0/2684767855 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7fce90053c30 con 0x7fcea00c8950 2023-07-14T11:54:13.083 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:54:13.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.104+0000 7fcea8192700 1 -- 172.21.15.104:0/2684767855 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fce840662f0 msgr2=0x7fce840687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.104+0000 7fcea8192700 1 --2- 172.21.15.104:0/2684767855 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fce840662f0 0x7fce840687a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0090b0 tx=0x7fce9c01c040 comp rx=0 tx=0).stop 2023-07-14T11:54:13.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.104+0000 7fcea8192700 1 -- 172.21.15.104:0/2684767855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcea00c8950 msgr2=0x7fcea00cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.104+0000 7fcea8192700 1 --2- 172.21.15.104:0/2684767855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcea00c8950 0x7fcea00cbed0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fce9000c730 tx=0x7fce9000ca40 comp rx=0 tx=0).stop 2023-07-14T11:54:13.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.104+0000 7fcea8192700 1 -- 172.21.15.104:0/2684767855 shutdown_connections 2023-07-14T11:54:13.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.104+0000 7fcea8192700 1 --2- 172.21.15.104:0/2684767855 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fce840662f0 0x7fce840687a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.104+0000 7fcea8192700 1 --2- 172.21.15.104:0/2684767855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcea00c8950 0x7fcea00cbed0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.104+0000 7fcea8192700 1 --2- 172.21.15.104:0/2684767855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcea00cd760 0x7fcea00c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.104+0000 7fcea8192700 1 --2- 172.21.15.104:0/2684767855 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcea00afd80 0x7fcea0155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.104+0000 7fcea8192700 1 -- 172.21.15.104:0/2684767855 >> 172.21.15.104:0/2684767855 conn(0x7fcea00cad80 msgr2=0x7fcea015b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:13.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.104+0000 7fcea8192700 1 -- 172.21.15.104:0/2684767855 shutdown_connections 2023-07-14T11:54:13.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.104+0000 7fcea8192700 1 -- 172.21.15.104:0/2684767855 wait complete. 2023-07-14T11:54:13.108 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:54:13.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.308+0000 7fe81cae4700 1 Processor -- start 2023-07-14T11:54:13.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.308+0000 7fe81cae4700 1 -- start start 2023-07-14T11:54:13.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.308+0000 7fe81cae4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe8180b0ee0 0x7fe8180ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:13.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe81cae4700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8180cd4e0 con 0x7fe8180d07e0 2023-07-14T11:54:13.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe81cae4700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8180d1a60 con 0x7fe8180cd090 2023-07-14T11:54:13.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe81cae4700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8180d0c60 con 0x7fe8180b0ee0 2023-07-14T11:54:13.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe8177fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe8180b0ee0 0x7fe8180ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:13.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe817fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fe8180d07e0 0x7fe8180af0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:44814/0 (socket says 172.21.15.104:44814) 2023-07-14T11:54:13.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe817fff700 1 -- 172.21.15.104:0/2705298719 learned_addr learned my addr 172.21.15.104:0/2705298719 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:13.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe816ffd700 1 -- 172.21.15.104:0/2705298719 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975962264 0 0) 0x7fe810001110 con 0x7fe8180d07e0 2023-07-14T11:54:13.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe816ffd700 1 -- 172.21.15.104:0/2705298719 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8040035d0 con 0x7fe8180d07e0 2023-07-14T11:54:13.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe8177fe700 1 -- 172.21.15.104:0/2705298719 >> v1:172.21.15.121:6789/0 conn(0x7fe8180cd090 legacy=0x7fe8180d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe8177fe700 1 -- 172.21.15.104:0/2705298719 >> v1:172.21.15.104:6789/0 conn(0x7fe8180d07e0 legacy=0x7fe8180af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe8177fe700 1 -- 172.21.15.104:0/2705298719 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8180d1ef0 con 0x7fe8180b0ee0 2023-07-14T11:54:13.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe8177fe700 1 --2- 172.21.15.104:0/2705298719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8180b0ee0 0x7fe8180ccb50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe808009d90 tx=0x7fe80800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5814255c44f2832f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:13.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe816ffd700 1 -- 172.21.15.104:0/2705298719 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80800e070 con 0x7fe8180b0ee0 2023-07-14T11:54:13.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe816ffd700 1 -- 172.21.15.104:0/2705298719 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80800bb20 con 0x7fe8180b0ee0 2023-07-14T11:54:13.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe816ffd700 1 -- 172.21.15.104:0/2705298719 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808012430 con 0x7fe8180b0ee0 2023-07-14T11:54:13.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe81cae4700 1 -- 172.21.15.104:0/2705298719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8180b0ee0 msgr2=0x7fe8180ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe81cae4700 1 --2- 172.21.15.104:0/2705298719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8180b0ee0 0x7fe8180ccb50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe808009d90 tx=0x7fe80800b330 comp rx=0 tx=0).stop 2023-07-14T11:54:13.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.312+0000 7fe81cae4700 1 -- 172.21.15.104:0/2705298719 shutdown_connections 2023-07-14T11:54:13.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.316+0000 7fe81cae4700 1 --2- 172.21.15.104:0/2705298719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8180b0ee0 0x7fe8180ccb50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.316+0000 7fe81cae4700 1 -- 172.21.15.104:0/2705298719 >> 172.21.15.104:0/2705298719 conn(0x7fe818060c40 msgr2=0x7fe81809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:13.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.316+0000 7fe81cae4700 1 -- 172.21.15.104:0/2705298719 shutdown_connections 2023-07-14T11:54:13.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.316+0000 7fe81cae4700 1 -- 172.21.15.104:0/2705298719 wait complete. 2023-07-14T11:54:13.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe81cae4700 1 Processor -- start 2023-07-14T11:54:13.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe81cae4700 1 -- start start 2023-07-14T11:54:13.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe81cae4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8180b0ee0 0x7fe8180cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:13.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe81cae4700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8180cd090 0x7fe8181f0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:13.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe81cae4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8180d07e0 0x7fe8180b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:13.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe81cae4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8180b37f0 con 0x7fe8180b0ee0 2023-07-14T11:54:13.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe81cae4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8180b3c80 con 0x7fe8180d07e0 2023-07-14T11:54:13.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe81cae4700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8180b2be0 con 0x7fe8180cd090 2023-07-14T11:54:13.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe8177fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8180b0ee0 0x7fe8180cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:13.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe8177fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8180b0ee0 0x7fe8180cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35084/0 (socket says 172.21.15.104:35084) 2023-07-14T11:54:13.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe8177fe700 1 -- 172.21.15.104:0/2787452160 learned_addr learned my addr 172.21.15.104:0/2787452160 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:13.309 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe817fff700 1 --2- 172.21.15.104:0/2787452160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8180d07e0 0x7fe8180b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:13.309 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe80effd700 1 --2- 172.21.15.104:0/2787452160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8180cd090 0x7fe8181f0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:13.309 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe8177fe700 1 -- 172.21.15.104:0/2787452160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8180cd090 msgr2=0x7fe8181f0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.309 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe8177fe700 1 --2- 172.21.15.104:0/2787452160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8180cd090 0x7fe8181f0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.309 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe8177fe700 1 -- 172.21.15.104:0/2787452160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8180d07e0 msgr2=0x7fe8180b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe8177fe700 1 --2- 172.21.15.104:0/2787452160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8180d07e0 0x7fe8180b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe8177fe700 1 -- 172.21.15.104:0/2787452160 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe808009a70 con 0x7fe8180b0ee0 2023-07-14T11:54:13.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe80effd700 1 --2- 172.21.15.104:0/2787452160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8180cd090 0x7fe8181f0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:13.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.320+0000 7fe8177fe700 1 --2- 172.21.15.104:0/2787452160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8180b0ee0 0x7fe8180cb5c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe808014040 tx=0x7fe808004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:13.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.324+0000 7fe814ff9700 1 -- 172.21.15.104:0/2787452160 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808012430 con 0x7fe8180b0ee0 2023-07-14T11:54:13.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.324+0000 7fe814ff9700 1 -- 172.21.15.104:0/2787452160 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe808020070 con 0x7fe8180b0ee0 2023-07-14T11:54:13.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.324+0000 7fe81cae4700 1 -- 172.21.15.104:0/2787452160 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8180b2ec0 con 0x7fe8180b0ee0 2023-07-14T11:54:13.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.324+0000 7fe814ff9700 1 -- 172.21.15.104:0/2787452160 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808004280 con 0x7fe8180b0ee0 2023-07-14T11:54:13.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.324+0000 7fe81cae4700 1 -- 172.21.15.104:0/2787452160 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8180b3480 con 0x7fe8180b0ee0 2023-07-14T11:54:13.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.324+0000 7fe81cae4700 1 -- 172.21.15.104:0/2787452160 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8180b3610 con 0x7fe8180b0ee0 2023-07-14T11:54:13.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.328+0000 7fe814ff9700 1 -- 172.21.15.104:0/2787452160 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe8080043e0 con 0x7fe8180b0ee0 2023-07-14T11:54:13.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.328+0000 7fe814ff9700 1 --2- 172.21.15.104:0/2787452160 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe7f80662f0 0x7fe7f80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:13.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.328+0000 7fe814ff9700 1 -- 172.21.15.104:0/2787452160 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe808022070 con 0x7fe8180b0ee0 2023-07-14T11:54:13.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.328+0000 7fe80effd700 1 --2- 172.21.15.104:0/2787452160 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe7f80662f0 0x7fe7f80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:13.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.328+0000 7fe80effd700 1 --2- 172.21.15.104:0/2787452160 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe7f80662f0 0x7fe7f80687a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe8180cef00 tx=0x7fe800009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:13.318 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.332+0000 7fe814ff9700 1 -- 172.21.15.104:0/2787452160 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe808053270 con 0x7fe8180b0ee0 2023-07-14T11:54:13.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.500+0000 7fe81cae4700 1 -- 172.21.15.104:0/2787452160 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe8180b4dd0 con 0x7fe8180b0ee0 2023-07-14T11:54:13.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.504+0000 7fe814ff9700 1 -- 172.21.15.104:0/2787452160 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe808056890 con 0x7fe8180b0ee0 2023-07-14T11:54:13.487 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:13.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.504+0000 7fe81cae4700 1 -- 172.21.15.104:0/2787452160 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe7f80662f0 msgr2=0x7fe7f80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.504+0000 7fe81cae4700 1 --2- 172.21.15.104:0/2787452160 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe7f80662f0 0x7fe7f80687a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe8180cef00 tx=0x7fe800009040 comp rx=0 tx=0).stop 2023-07-14T11:54:13.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.504+0000 7fe81cae4700 1 -- 172.21.15.104:0/2787452160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8180b0ee0 msgr2=0x7fe8180cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.504+0000 7fe81cae4700 1 --2- 172.21.15.104:0/2787452160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8180b0ee0 0x7fe8180cb5c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe808014040 tx=0x7fe808004af0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.504+0000 7fe81cae4700 1 -- 172.21.15.104:0/2787452160 shutdown_connections 2023-07-14T11:54:13.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.504+0000 7fe81cae4700 1 --2- 172.21.15.104:0/2787452160 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe7f80662f0 0x7fe7f80687a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.504+0000 7fe81cae4700 1 --2- 172.21.15.104:0/2787452160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8180d07e0 0x7fe8180b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.504+0000 7fe81cae4700 1 --2- 172.21.15.104:0/2787452160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8180cd090 0x7fe8181f0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.504+0000 7fe81cae4700 1 --2- 172.21.15.104:0/2787452160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8180b0ee0 0x7fe8180cb5c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.504+0000 7fe81cae4700 1 -- 172.21.15.104:0/2787452160 >> 172.21.15.104:0/2787452160 conn(0x7fe818060c40 msgr2=0x7fe81809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:13.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.504+0000 7fe81cae4700 1 -- 172.21.15.104:0/2787452160 shutdown_connections 2023-07-14T11:54:13.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.504+0000 7fe81cae4700 1 -- 172.21.15.104:0/2787452160 wait complete. 2023-07-14T11:54:13.511 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:13.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.704+0000 7f8680bae700 1 Processor -- start 2023-07-14T11:54:13.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.704+0000 7f8680bae700 1 -- start start 2023-07-14T11:54:13.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.704+0000 7f8680bae700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f867c160640 0x7f867c164000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:13.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.704+0000 7f8680bae700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867c0c8900 con 0x7f867c0cddb0 2023-07-14T11:54:13.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.704+0000 7f8680bae700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867c0caae0 con 0x7f867c164540 2023-07-14T11:54:13.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.704+0000 7f8680bae700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867c0c8fc0 con 0x7f867c160640 2023-07-14T11:54:13.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.704+0000 7f867ad9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f867c0cddb0 0x7f867c0afd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:44822/0 (socket says 172.21.15.104:44822) 2023-07-14T11:54:13.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.704+0000 7f867ad9d700 1 -- 172.21.15.104:0/2720962840 learned_addr learned my addr 172.21.15.104:0/2720962840 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:13.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.704+0000 7f867a59c700 1 --2- 172.21.15.104:0/2720962840 >> v2:172.21.15.121:3300/0 conn(0x7f867c160640 0x7f867c164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:13.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f8679d9b700 1 -- 172.21.15.104:0/2720962840 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652478247 0 0) 0x7f867c0c8900 con 0x7f867c0cddb0 2023-07-14T11:54:13.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f8679d9b700 1 -- 172.21.15.104:0/2720962840 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f865c0035d0 con 0x7f867c0cddb0 2023-07-14T11:54:13.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f867a59c700 1 -- 172.21.15.104:0/2720962840 >> v1:172.21.15.121:6789/0 conn(0x7f867c164540 legacy=0x7f867c070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f867a59c700 1 -- 172.21.15.104:0/2720962840 >> v1:172.21.15.104:6789/0 conn(0x7f867c0cddb0 legacy=0x7f867c0afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f867a59c700 1 -- 172.21.15.104:0/2720962840 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c0ca520 con 0x7f867c160640 2023-07-14T11:54:13.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f867a59c700 1 --2- 172.21.15.104:0/2720962840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c160640 0x7f867c164000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f866c009850 tx=0x7f866c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48b2f92d993fa0d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:13.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f8679d9b700 1 -- 172.21.15.104:0/2720962840 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c011830 con 0x7f867c160640 2023-07-14T11:54:13.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f8679d9b700 1 -- 172.21.15.104:0/2720962840 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c011990 con 0x7f867c160640 2023-07-14T11:54:13.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f8679d9b700 1 -- 172.21.15.104:0/2720962840 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c011d40 con 0x7f867c160640 2023-07-14T11:54:13.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f8680bae700 1 -- 172.21.15.104:0/2720962840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c160640 msgr2=0x7f867c164000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f8680bae700 1 --2- 172.21.15.104:0/2720962840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c160640 0x7f867c164000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f866c009850 tx=0x7f866c009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:13.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f8680bae700 1 -- 172.21.15.104:0/2720962840 shutdown_connections 2023-07-14T11:54:13.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f8680bae700 1 --2- 172.21.15.104:0/2720962840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c160640 0x7f867c164000 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f8680bae700 1 -- 172.21.15.104:0/2720962840 >> 172.21.15.104:0/2720962840 conn(0x7f867c0cad30 msgr2=0x7f867c0cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:13.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f8680bae700 1 -- 172.21.15.104:0/2720962840 shutdown_connections 2023-07-14T11:54:13.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.708+0000 7f8680bae700 1 -- 172.21.15.104:0/2720962840 wait complete. 2023-07-14T11:54:13.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.712+0000 7f8680bae700 1 Processor -- start 2023-07-14T11:54:13.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.712+0000 7f8680bae700 1 -- start start 2023-07-14T11:54:13.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.712+0000 7f8680bae700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f867c0cddb0 0x7f867c1f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:13.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.712+0000 7f8680bae700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f867c160640 0x7f867c0b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:13.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.712+0000 7f8680bae700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c164540 0x7f867c0c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:13.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.712+0000 7f8680bae700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c070540 con 0x7f867c160640 2023-07-14T11:54:13.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.712+0000 7f8680bae700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c0b2b90 con 0x7f867c164540 2023-07-14T11:54:13.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.712+0000 7f8680bae700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c0b1f80 con 0x7f867c0cddb0 2023-07-14T11:54:13.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.712+0000 7f8673fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f867c160640 0x7f867c0b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:13.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.716+0000 7f8673fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f867c160640 0x7f867c0b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35094/0 (socket says 172.21.15.104:35094) 2023-07-14T11:54:13.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.716+0000 7f8673fff700 1 -- 172.21.15.104:0/235526113 learned_addr learned my addr 172.21.15.104:0/235526113 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:13.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.716+0000 7f867a59c700 1 --2- 172.21.15.104:0/235526113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f867c0cddb0 0x7f867c1f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:13.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.716+0000 7f867ad9d700 1 --2- 172.21.15.104:0/235526113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c164540 0x7f867c0c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:13.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.716+0000 7f8673fff700 1 -- 172.21.15.104:0/235526113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f867c0cddb0 msgr2=0x7f867c1f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.716+0000 7f8673fff700 1 --2- 172.21.15.104:0/235526113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f867c0cddb0 0x7f867c1f55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.716+0000 7f8673fff700 1 -- 172.21.15.104:0/235526113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c164540 msgr2=0x7f867c0c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.716+0000 7f8673fff700 1 --2- 172.21.15.104:0/235526113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c164540 0x7f867c0c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.716+0000 7f8673fff700 1 -- 172.21.15.104:0/235526113 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c0094b0 con 0x7f867c160640 2023-07-14T11:54:13.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.716+0000 7f8673fff700 1 --2- 172.21.15.104:0/235526113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f867c160640 0x7f867c0b38b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f866400c500 tx=0x7f866400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:13.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.720+0000 7f8672ffd700 1 -- 172.21.15.104:0/235526113 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86640052d0 con 0x7f867c160640 2023-07-14T11:54:13.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.720+0000 7f8680bae700 1 -- 172.21.15.104:0/235526113 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867c0b2260 con 0x7f867c160640 2023-07-14T11:54:13.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.720+0000 7f8672ffd700 1 -- 172.21.15.104:0/235526113 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8664005430 con 0x7f867c160640 2023-07-14T11:54:13.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.720+0000 7f8672ffd700 1 -- 172.21.15.104:0/235526113 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86640055a0 con 0x7f867c160640 2023-07-14T11:54:13.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.720+0000 7f8680bae700 1 -- 172.21.15.104:0/235526113 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867c0b2820 con 0x7f867c160640 2023-07-14T11:54:13.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.720+0000 7f8680bae700 1 -- 172.21.15.104:0/235526113 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867c04dcb0 con 0x7f867c160640 2023-07-14T11:54:13.706 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.720+0000 7f8672ffd700 1 -- 172.21.15.104:0/235526113 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f86640038e0 con 0x7f867c160640 2023-07-14T11:54:13.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.724+0000 7f8672ffd700 1 --2- 172.21.15.104:0/235526113 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86600662f0 0x7f86600687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:13.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.724+0000 7f8672ffd700 1 -- 172.21.15.104:0/235526113 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8664083330 con 0x7f867c160640 2023-07-14T11:54:13.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.724+0000 7f867a59c700 1 --2- 172.21.15.104:0/235526113 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86600662f0 0x7f86600687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:13.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.724+0000 7f867a59c700 1 --2- 172.21.15.104:0/235526113 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86600662f0 0x7f86600687a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f866c0090b0 tx=0x7f866c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:13.714 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.728+0000 7f8672ffd700 1 -- 172.21.15.104:0/235526113 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f8664050490 con 0x7f867c160640 2023-07-14T11:54:13.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.900+0000 7f8680bae700 1 -- 172.21.15.104:0/235526113 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f867c0713a0 con 0x7f867c160640 2023-07-14T11:54:13.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.904+0000 7f8672ffd700 1 -- 172.21.15.104:0/235526113 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20251 (secure 0 0 0) 0x7f8664053ab0 con 0x7f867c160640 2023-07-14T11:54:13.888 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:13.888 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":30,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:55.601783+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:13.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.908+0000 7f8680bae700 1 -- 172.21.15.104:0/235526113 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86600662f0 msgr2=0x7f86600687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.908+0000 7f8680bae700 1 --2- 172.21.15.104:0/235526113 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86600662f0 0x7f86600687a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f866c0090b0 tx=0x7f866c008030 comp rx=0 tx=0).stop 2023-07-14T11:54:13.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.908+0000 7f8680bae700 1 -- 172.21.15.104:0/235526113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f867c160640 msgr2=0x7f867c0b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:13.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.908+0000 7f8680bae700 1 --2- 172.21.15.104:0/235526113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f867c160640 0x7f867c0b38b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f866400c500 tx=0x7f866400c810 comp rx=0 tx=0).stop 2023-07-14T11:54:13.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.908+0000 7f8680bae700 1 -- 172.21.15.104:0/235526113 shutdown_connections 2023-07-14T11:54:13.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.908+0000 7f8680bae700 1 --2- 172.21.15.104:0/235526113 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86600662f0 0x7f86600687a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.908+0000 7f8680bae700 1 --2- 172.21.15.104:0/235526113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f867c160640 0x7f867c0b38b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.908+0000 7f8680bae700 1 --2- 172.21.15.104:0/235526113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c164540 0x7f867c0c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.908+0000 7f8680bae700 1 --2- 172.21.15.104:0/235526113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f867c0cddb0 0x7f867c1f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:13.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.908+0000 7f8680bae700 1 -- 172.21.15.104:0/235526113 >> 172.21.15.104:0/235526113 conn(0x7f867c0cad30 msgr2=0x7f867c0addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:13.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.908+0000 7f8680bae700 1 -- 172.21.15.104:0/235526113 shutdown_connections 2023-07-14T11:54:13.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:13.908+0000 7f8680bae700 1 -- 172.21.15.104:0/235526113 wait complete. 2023-07-14T11:54:13.914 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:54:13.914 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:54:14.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.104+0000 7f6b52a20700 1 Processor -- start 2023-07-14T11:54:14.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.104+0000 7f6b52a20700 1 -- start start 2023-07-14T11:54:14.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b52a20700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b4c0b06c0 0x7f6b4c0b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:14.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b52a20700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6b4c0ce610 0x7f6b4c0cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:14.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b52a20700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4c0cbb60 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b52a20700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b4c0b0f60 con 0x7f6b4c0ce610 2023-07-14T11:54:14.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b52a20700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c0cb510 con 0x7f6b4c0b06c0 2023-07-14T11:54:14.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b5121d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f6b4c0b18d0 0x7f6b4c0cdf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:44830/0 (socket says 172.21.15.104:44830) 2023-07-14T11:54:14.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b5121d700 1 -- 172.21.15.104:0/2916657644 learned_addr learned my addr 172.21.15.104:0/2916657644 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:14.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b5221f700 1 --2- 172.21.15.104:0/2916657644 >> v2:172.21.15.121:3300/0 conn(0x7f6b4c0ce610 0x7f6b4c0cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:14.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b51a1e700 1 --2- 172.21.15.104:0/2916657644 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b4c0b06c0 0x7f6b4c0b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:14.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b50a1c700 1 -- 172.21.15.104:0/2916657644 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254788963 0 0) 0x7f6b4c0cbb60 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b50a1c700 1 -- 172.21.15.104:0/2916657644 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b380035d0 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b5221f700 1 -- 172.21.15.104:0/2916657644 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b4c0b06c0 msgr2=0x7f6b4c0b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:14.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b5221f700 1 --2- 172.21.15.104:0/2916657644 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b4c0b06c0 0x7f6b4c0b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b5221f700 1 -- 172.21.15.104:0/2916657644 >> v1:172.21.15.104:6789/0 conn(0x7f6b4c0b18d0 legacy=0x7f6b4c0cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:14.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b5221f700 1 -- 172.21.15.104:0/2916657644 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c0cb7f0 con 0x7f6b4c0ce610 2023-07-14T11:54:14.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b5221f700 1 --2- 172.21.15.104:0/2916657644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b4c0ce610 0x7f6b4c0cbff0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009850 tx=0x7f6b44009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c9b88c1bbf4308 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:14.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b50a1c700 1 -- 172.21.15.104:0/2916657644 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44011830 con 0x7f6b4c0ce610 2023-07-14T11:54:14.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b50a1c700 1 -- 172.21.15.104:0/2916657644 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b44011990 con 0x7f6b4c0ce610 2023-07-14T11:54:14.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.108+0000 7f6b50a1c700 1 -- 172.21.15.104:0/2916657644 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44011d00 con 0x7f6b4c0ce610 2023-07-14T11:54:14.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.112+0000 7f6b52a20700 1 -- 172.21.15.104:0/2916657644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b4c0ce610 msgr2=0x7f6b4c0cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:14.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.112+0000 7f6b52a20700 1 --2- 172.21.15.104:0/2916657644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b4c0ce610 0x7f6b4c0cbff0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009850 tx=0x7f6b44009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:14.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.112+0000 7f6b52a20700 1 -- 172.21.15.104:0/2916657644 shutdown_connections 2023-07-14T11:54:14.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.112+0000 7f6b52a20700 1 --2- 172.21.15.104:0/2916657644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b4c0ce610 0x7f6b4c0cbff0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.112+0000 7f6b52a20700 1 --2- 172.21.15.104:0/2916657644 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b4c0b06c0 0x7f6b4c0b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.112+0000 7f6b52a20700 1 -- 172.21.15.104:0/2916657644 >> 172.21.15.104:0/2916657644 conn(0x7f6b4c099600 msgr2=0x7f6b4c0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:14.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.112+0000 7f6b52a20700 1 -- 172.21.15.104:0/2916657644 shutdown_connections 2023-07-14T11:54:14.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.112+0000 7f6b52a20700 1 -- 172.21.15.104:0/2916657644 wait complete. 2023-07-14T11:54:14.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.112+0000 7f6b52a20700 1 Processor -- start 2023-07-14T11:54:14.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.112+0000 7f6b52a20700 1 -- start start 2023-07-14T11:54:14.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b52a20700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b4c0b06c0 0x7f6b4c0b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:14.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b52a20700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b4c0b18d0 0x7f6b4c151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:14.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b52a20700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b4c0ce610 0x7f6b4c1e0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:14.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b52a20700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c1515c0 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b52a20700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c14f0f0 con 0x7f6b4c0b06c0 2023-07-14T11:54:14.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b52a20700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c14f6c0 con 0x7f6b4c0ce610 2023-07-14T11:54:14.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b5121d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b4c0b18d0 0x7f6b4c151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:14.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b5121d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b4c0b18d0 0x7f6b4c151080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35098/0 (socket says 172.21.15.104:35098) 2023-07-14T11:54:14.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b5121d700 1 -- 172.21.15.104:0/3875789726 learned_addr learned my addr 172.21.15.104:0/3875789726 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:14.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b5221f700 1 --2- 172.21.15.104:0/3875789726 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b4c0ce610 0x7f6b4c1e0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:14.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b51a1e700 1 --2- 172.21.15.104:0/3875789726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b4c0b06c0 0x7f6b4c0b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:14.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b5121d700 1 -- 172.21.15.104:0/3875789726 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b4c0ce610 msgr2=0x7f6b4c1e0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:14.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b5121d700 1 --2- 172.21.15.104:0/3875789726 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b4c0ce610 0x7f6b4c1e0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b5121d700 1 -- 172.21.15.104:0/3875789726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b4c0b06c0 msgr2=0x7f6b4c0b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:14.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b5121d700 1 --2- 172.21.15.104:0/3875789726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b4c0b06c0 0x7f6b4c0b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b5121d700 1 -- 172.21.15.104:0/3875789726 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b440094b0 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b51a1e700 1 --2- 172.21.15.104:0/3875789726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b4c0b06c0 0x7f6b4c0b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:14.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b5121d700 1 --2- 172.21.15.104:0/3875789726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b4c0b18d0 0x7f6b4c151080 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c0010e0 tx=0x7f6b3c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:14.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b427fc700 1 -- 172.21.15.104:0/3875789726 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c005d90 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b427fc700 1 -- 172.21.15.104:0/3875789726 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3c0070d0 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b52a20700 1 -- 172.21.15.104:0/3875789726 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4c0710c0 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b427fc700 1 -- 172.21.15.104:0/3875789726 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c007460 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.116+0000 7f6b52a20700 1 -- 172.21.15.104:0/3875789726 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4c14f350 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.120+0000 7f6b52a20700 1 -- 172.21.15.104:0/3875789726 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4c04e500 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.120+0000 7f6b427fc700 1 -- 172.21.15.104:0/3875789726 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6b3c0075c0 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.120+0000 7f6b427fc700 1 --2- 172.21.15.104:0/3875789726 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6b300662f0 0x7f6b300687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:14.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.120+0000 7f6b427fc700 1 -- 172.21.15.104:0/3875789726 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6b3c083570 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.120+0000 7f6b51a1e700 1 --2- 172.21.15.104:0/3875789726 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6b300662f0 0x7f6b300687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:14.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.124+0000 7f6b51a1e700 1 --2- 172.21.15.104:0/3875789726 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6b300662f0 0x7f6b300687a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0c6b70 tx=0x7f6b4800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:14.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.128+0000 7f6b427fc700 1 -- 172.21.15.104:0/3875789726 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f6b3c050620 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.300+0000 7f6b52a20700 1 -- 172.21.15.104:0/3875789726 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b4c14f800 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.300+0000 7f6b427fc700 1 -- 172.21.15.104:0/3875789726 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b3c053c40 con 0x7f6b4c0b18d0 2023-07-14T11:54:14.286 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:54:14.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.304+0000 7f6b52a20700 1 -- 172.21.15.104:0/3875789726 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6b300662f0 msgr2=0x7f6b300687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:14.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.304+0000 7f6b52a20700 1 --2- 172.21.15.104:0/3875789726 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6b300662f0 0x7f6b300687a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0c6b70 tx=0x7f6b4800b040 comp rx=0 tx=0).stop 2023-07-14T11:54:14.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.304+0000 7f6b52a20700 1 -- 172.21.15.104:0/3875789726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b4c0b18d0 msgr2=0x7f6b4c151080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:14.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.304+0000 7f6b52a20700 1 --2- 172.21.15.104:0/3875789726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b4c0b18d0 0x7f6b4c151080 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c0010e0 tx=0x7f6b3c00c7d0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.304+0000 7f6b52a20700 1 -- 172.21.15.104:0/3875789726 shutdown_connections 2023-07-14T11:54:14.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.304+0000 7f6b52a20700 1 --2- 172.21.15.104:0/3875789726 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6b300662f0 0x7f6b300687a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.304+0000 7f6b52a20700 1 --2- 172.21.15.104:0/3875789726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b4c0b18d0 0x7f6b4c151080 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.304+0000 7f6b52a20700 1 --2- 172.21.15.104:0/3875789726 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b4c0ce610 0x7f6b4c1e0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.304+0000 7f6b52a20700 1 --2- 172.21.15.104:0/3875789726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b4c0b06c0 0x7f6b4c0b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.304+0000 7f6b52a20700 1 -- 172.21.15.104:0/3875789726 >> 172.21.15.104:0/3875789726 conn(0x7f6b4c099600 msgr2=0x7f6b4c1e12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:14.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.304+0000 7f6b52a20700 1 -- 172.21.15.104:0/3875789726 shutdown_connections 2023-07-14T11:54:14.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.304+0000 7f6b52a20700 1 -- 172.21.15.104:0/3875789726 wait complete. 2023-07-14T11:54:14.443 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:54:14.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.640+0000 7f6971b62700 1 Processor -- start 2023-07-14T11:54:14.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.640+0000 7f6971b62700 1 -- start start 2023-07-14T11:54:14.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f6971b62700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f696c0afd80 0x7f696c0cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:14.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f6971b62700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f696c15c200 0x7f696c0b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:14.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f6971b62700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f696c0cbed0 con 0x7f696c0ad0f0 2023-07-14T11:54:14.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f6971b62700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696c071820 con 0x7f696c0afd80 2023-07-14T11:54:14.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f6971b62700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696c0b0690 con 0x7f696c15c200 2023-07-14T11:54:14.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f696b7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f696c0afd80 0x7f696c0cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:14.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f696b7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f696c0afd80 0x7f696c0cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35114/0 (socket says 172.21.15.104:35114) 2023-07-14T11:54:14.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f696b7fe700 1 -- 172.21.15.104:0/4175109021 learned_addr learned my addr 172.21.15.104:0/4175109021 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:14.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f696affd700 1 --2- 172.21.15.104:0/4175109021 >> v2:172.21.15.121:3300/0 conn(0x7f696c15c200 0x7f696c0b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:14.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f696b7fe700 1 -- 172.21.15.104:0/4175109021 >> v2:172.21.15.121:3300/0 conn(0x7f696c15c200 msgr2=0x7f696c0b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:14.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f696b7fe700 1 --2- 172.21.15.104:0/4175109021 >> v2:172.21.15.121:3300/0 conn(0x7f696c15c200 0x7f696c0b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f696b7fe700 1 -- 172.21.15.104:0/4175109021 >> v1:172.21.15.121:6789/0 conn(0x7f696c0ad0f0 legacy=0x7f696c0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:14.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f696b7fe700 1 -- 172.21.15.104:0/4175109021 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696c0ad760 con 0x7f696c0afd80 2023-07-14T11:54:14.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f696b7fe700 1 --2- 172.21.15.104:0/4175109021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f696c0afd80 0x7f696c0cb890 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f696c0cbdd0 tx=0x7f695c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6aac32e2f63a4b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:14.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f696a7fc700 1 -- 172.21.15.104:0/4175109021 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c011830 con 0x7f696c0afd80 2023-07-14T11:54:14.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f696a7fc700 1 -- 172.21.15.104:0/4175109021 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c011990 con 0x7f696c0afd80 2023-07-14T11:54:14.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.644+0000 7f696a7fc700 1 -- 172.21.15.104:0/4175109021 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c011d20 con 0x7f696c0afd80 2023-07-14T11:54:14.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.648+0000 7f6971b62700 1 -- 172.21.15.104:0/4175109021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f696c0afd80 msgr2=0x7f696c0cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:14.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.648+0000 7f6971b62700 1 --2- 172.21.15.104:0/4175109021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f696c0afd80 0x7f696c0cb890 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f696c0cbdd0 tx=0x7f695c009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.648+0000 7f6971b62700 1 -- 172.21.15.104:0/4175109021 shutdown_connections 2023-07-14T11:54:14.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.648+0000 7f6971b62700 1 --2- 172.21.15.104:0/4175109021 >> v2:172.21.15.121:3300/0 conn(0x7f696c15c200 0x7f696c0b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.648+0000 7f6971b62700 1 --2- 172.21.15.104:0/4175109021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f696c0afd80 0x7f696c0cb890 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.648+0000 7f6971b62700 1 -- 172.21.15.104:0/4175109021 >> 172.21.15.104:0/4175109021 conn(0x7f696c0cad80 msgr2=0x7f696c0cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:14.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.648+0000 7f6971b62700 1 -- 172.21.15.104:0/4175109021 shutdown_connections 2023-07-14T11:54:14.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.648+0000 7f6971b62700 1 -- 172.21.15.104:0/4175109021 wait complete. 2023-07-14T11:54:14.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f6971b62700 1 Processor -- start 2023-07-14T11:54:14.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f6971b62700 1 -- start start 2023-07-14T11:54:14.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f6971b62700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f696c0ad0f0 0x7f696c0c4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:14.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f6971b62700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f696c0afd80 0x7f696c15c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:14.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f6971b62700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f696c15c200 0x7f696c15cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:14.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f6971b62700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c1f5930 con 0x7f696c15c200 2023-07-14T11:54:14.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f6971b62700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c15fe20 con 0x7f696c0afd80 2023-07-14T11:54:14.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f6971b62700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696c160100 con 0x7f696c0ad0f0 2023-07-14T11:54:14.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f696bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f696c15c200 0x7f696c15cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:14.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f696bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f696c15c200 0x7f696c15cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35130/0 (socket says 172.21.15.104:35130) 2023-07-14T11:54:14.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f696bfff700 1 -- 172.21.15.104:0/3603509605 learned_addr learned my addr 172.21.15.104:0/3603509605 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:14.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f696b7fe700 1 --2- 172.21.15.104:0/3603509605 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f696c0ad0f0 0x7f696c0c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:14.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f696affd700 1 --2- 172.21.15.104:0/3603509605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f696c0afd80 0x7f696c15c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:14.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f696bfff700 1 -- 172.21.15.104:0/3603509605 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f696c0ad0f0 msgr2=0x7f696c0c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:14.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f696bfff700 1 --2- 172.21.15.104:0/3603509605 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f696c0ad0f0 0x7f696c0c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f696bfff700 1 -- 172.21.15.104:0/3603509605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f696c0afd80 msgr2=0x7f696c15c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:14.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f696bfff700 1 --2- 172.21.15.104:0/3603509605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f696c0afd80 0x7f696c15c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f696bfff700 1 -- 172.21.15.104:0/3603509605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c009570 con 0x7f696c15c200 2023-07-14T11:54:14.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.652+0000 7f696bfff700 1 --2- 172.21.15.104:0/3603509605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f696c15c200 0x7f696c15cb40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f696400caf0 tx=0x7f696400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:14.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.656+0000 7f6953fff700 1 -- 172.21.15.104:0/3603509605 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6964003c20 con 0x7f696c15c200 2023-07-14T11:54:14.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.656+0000 7f6953fff700 1 -- 172.21.15.104:0/3603509605 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6964003d80 con 0x7f696c15c200 2023-07-14T11:54:14.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.656+0000 7f6971b62700 1 -- 172.21.15.104:0/3603509605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696c1603e0 con 0x7f696c15c200 2023-07-14T11:54:14.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.656+0000 7f6953fff700 1 -- 172.21.15.104:0/3603509605 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696400aa20 con 0x7f696c15c200 2023-07-14T11:54:14.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.656+0000 7f6971b62700 1 -- 172.21.15.104:0/3603509605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696c15d080 con 0x7f696c15c200 2023-07-14T11:54:14.641 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.656+0000 7f6971b62700 1 -- 172.21.15.104:0/3603509605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f696c0c2710 con 0x7f696c15c200 2023-07-14T11:54:14.642 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.656+0000 7f6953fff700 1 -- 172.21.15.104:0/3603509605 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f696400ab80 con 0x7f696c15c200 2023-07-14T11:54:14.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.660+0000 7f6953fff700 1 --2- 172.21.15.104:0/3603509605 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f69580662f0 0x7f69580687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:14.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.660+0000 7f6953fff700 1 -- 172.21.15.104:0/3603509605 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6964083ff0 con 0x7f696c15c200 2023-07-14T11:54:14.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.660+0000 7f696b7fe700 1 --2- 172.21.15.104:0/3603509605 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f69580662f0 0x7f69580687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:14.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.660+0000 7f696b7fe700 1 --2- 172.21.15.104:0/3603509605 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f69580662f0 0x7f69580687a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f695c000c00 tx=0x7f695c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:14.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.668+0000 7f6953fff700 1 -- 172.21.15.104:0/3603509605 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f696c0c2710 con 0x7f696c15c200 2023-07-14T11:54:14.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.844+0000 7f6971b62700 1 -- 172.21.15.104:0/3603509605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f696c0c2710 con 0x7f696c15c200 2023-07-14T11:54:14.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.844+0000 7f6953fff700 1 -- 172.21.15.104:0/3603509605 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f696c0c2710 con 0x7f696c15c200 2023-07-14T11:54:14.829 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:54:14.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.848+0000 7f6971b62700 1 -- 172.21.15.104:0/3603509605 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f69580662f0 msgr2=0x7f69580687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:14.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.848+0000 7f6971b62700 1 --2- 172.21.15.104:0/3603509605 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f69580662f0 0x7f69580687a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f695c000c00 tx=0x7f695c008030 comp rx=0 tx=0).stop 2023-07-14T11:54:14.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.848+0000 7f6971b62700 1 -- 172.21.15.104:0/3603509605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f696c15c200 msgr2=0x7f696c15cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:14.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.848+0000 7f6971b62700 1 --2- 172.21.15.104:0/3603509605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f696c15c200 0x7f696c15cb40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f696400caf0 tx=0x7f696400ce00 comp rx=0 tx=0).stop 2023-07-14T11:54:14.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.848+0000 7f6971b62700 1 -- 172.21.15.104:0/3603509605 shutdown_connections 2023-07-14T11:54:14.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.848+0000 7f6971b62700 1 --2- 172.21.15.104:0/3603509605 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f69580662f0 0x7f69580687a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.848+0000 7f6971b62700 1 --2- 172.21.15.104:0/3603509605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f696c15c200 0x7f696c15cb40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.848+0000 7f6971b62700 1 --2- 172.21.15.104:0/3603509605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f696c0afd80 0x7f696c15c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.848+0000 7f6971b62700 1 --2- 172.21.15.104:0/3603509605 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f696c0ad0f0 0x7f696c0c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:14.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.848+0000 7f6971b62700 1 -- 172.21.15.104:0/3603509605 >> 172.21.15.104:0/3603509605 conn(0x7f696c0cad80 msgr2=0x7f696c15a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:14.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.852+0000 7f6971b62700 1 -- 172.21.15.104:0/3603509605 shutdown_connections 2023-07-14T11:54:14.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:14.852+0000 7f6971b62700 1 -- 172.21.15.104:0/3603509605 wait complete. 2023-07-14T11:54:14.857 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:54:15.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.048+0000 7f1ec6a08700 1 Processor -- start 2023-07-14T11:54:15.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.048+0000 7f1ec6a08700 1 -- start start 2023-07-14T11:54:15.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.048+0000 7f1ec6a08700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1ec0160000 0x7f1ec006dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.048+0000 7f1ec6a08700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1ec00cd7f0 0x7f1ec00c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.048+0000 7f1ec6a08700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec00cdfb0 con 0x7f1ec01603d0 2023-07-14T11:54:15.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.048+0000 7f1ec6a08700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec0070e40 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.048+0000 7f1ec6a08700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec00cdcd0 con 0x7f1ec0160000 2023-07-14T11:54:15.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.048+0000 7f1ec6207700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1ec00cd7f0 0x7f1ec00c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.048+0000 7f1ec5205700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f1ec01603d0 0x7f1ec00c9e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:44846/0 (socket says 172.21.15.104:44846) 2023-07-14T11:54:15.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.048+0000 7f1ec5205700 1 -- 172.21.15.104:0/698302630 learned_addr learned my addr 172.21.15.104:0/698302630 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:15.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.048+0000 7f1ec5a06700 1 --2- 172.21.15.104:0/698302630 >> v2:172.21.15.121:3300/0 conn(0x7f1ec0160000 0x7f1ec006dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.048+0000 7f1ec6207700 1 -- 172.21.15.104:0/698302630 >> v2:172.21.15.121:3300/0 conn(0x7f1ec0160000 msgr2=0x7f1ec006dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:15.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.048+0000 7f1ec6207700 1 --2- 172.21.15.104:0/698302630 >> v2:172.21.15.121:3300/0 conn(0x7f1ec0160000 0x7f1ec006dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.048+0000 7f1ec6207700 1 -- 172.21.15.104:0/698302630 >> v1:172.21.15.104:6789/0 conn(0x7f1ec01603d0 legacy=0x7f1ec00c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.048+0000 7f1ec6207700 1 -- 172.21.15.104:0/698302630 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec0070780 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.052+0000 7f1ec6207700 1 --2- 172.21.15.104:0/698302630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00cd7f0 0x7f1ec00c0d50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1ec006b3b0 tx=0x7f1eb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a3d822ccf6cd6e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:15.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.052+0000 7f1ec4a04700 1 -- 172.21.15.104:0/698302630 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb800e070 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.052+0000 7f1ec4a04700 1 -- 172.21.15.104:0/698302630 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb800bb20 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.052+0000 7f1ec4a04700 1 -- 172.21.15.104:0/698302630 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8012430 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.052+0000 7f1ec6a08700 1 -- 172.21.15.104:0/698302630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00cd7f0 msgr2=0x7f1ec00c0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.052+0000 7f1ec6a08700 1 --2- 172.21.15.104:0/698302630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00cd7f0 0x7f1ec00c0d50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1ec006b3b0 tx=0x7f1eb800b330 comp rx=0 tx=0).stop 2023-07-14T11:54:15.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.052+0000 7f1ec6a08700 1 -- 172.21.15.104:0/698302630 shutdown_connections 2023-07-14T11:54:15.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.052+0000 7f1ec6a08700 1 --2- 172.21.15.104:0/698302630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00cd7f0 0x7f1ec00c0d50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.052+0000 7f1ec6a08700 1 --2- 172.21.15.104:0/698302630 >> v2:172.21.15.121:3300/0 conn(0x7f1ec0160000 0x7f1ec006dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.052+0000 7f1ec6a08700 1 -- 172.21.15.104:0/698302630 >> 172.21.15.104:0/698302630 conn(0x7f1ec00afc60 msgr2=0x7f1ec00b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:15.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.052+0000 7f1ec6a08700 1 -- 172.21.15.104:0/698302630 shutdown_connections 2023-07-14T11:54:15.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.052+0000 7f1ec6a08700 1 -- 172.21.15.104:0/698302630 wait complete. 2023-07-14T11:54:15.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec6a08700 1 Processor -- start 2023-07-14T11:54:15.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec6a08700 1 -- start start 2023-07-14T11:54:15.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec6a08700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00cd7f0 0x7f1ec015a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec6a08700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ec0160000 0x7f1ec00c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec6a08700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ec01603d0 0x7f1ec00aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec6a08700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00b2f30 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec6a08700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00b1fd0 con 0x7f1ec01603d0 2023-07-14T11:54:15.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec6a08700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00b22b0 con 0x7f1ec0160000 2023-07-14T11:54:15.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec5a06700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00cd7f0 0x7f1ec015a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec5a06700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00cd7f0 0x7f1ec015a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35144/0 (socket says 172.21.15.104:35144) 2023-07-14T11:54:15.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec5a06700 1 -- 172.21.15.104:0/2570875390 learned_addr learned my addr 172.21.15.104:0/2570875390 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:15.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec5205700 1 --2- 172.21.15.104:0/2570875390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ec0160000 0x7f1ec00c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec6207700 1 --2- 172.21.15.104:0/2570875390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ec01603d0 0x7f1ec00aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec5a06700 1 -- 172.21.15.104:0/2570875390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ec0160000 msgr2=0x7f1ec00c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec5a06700 1 --2- 172.21.15.104:0/2570875390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ec0160000 0x7f1ec00c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec5a06700 1 -- 172.21.15.104:0/2570875390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ec01603d0 msgr2=0x7f1ec00aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec5a06700 1 --2- 172.21.15.104:0/2570875390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ec01603d0 0x7f1ec00aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec5a06700 1 -- 172.21.15.104:0/2570875390 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb8009a70 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec5205700 1 --2- 172.21.15.104:0/2570875390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ec0160000 0x7f1ec00c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:15.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.056+0000 7f1ec5a06700 1 --2- 172.21.15.104:0/2570875390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00cd7f0 0x7f1ec015a7b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00e9d0 tx=0x7f1ebc00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:15.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.060+0000 7f1eb67fc700 1 -- 172.21.15.104:0/2570875390 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc009720 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.060+0000 7f1eb67fc700 1 -- 172.21.15.104:0/2570875390 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc00c850 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.060+0000 7f1ec6a08700 1 -- 172.21.15.104:0/2570875390 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec00b2590 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.060+0000 7f1eb67fc700 1 -- 172.21.15.104:0/2570875390 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc00cbc0 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.060+0000 7f1ec6a08700 1 -- 172.21.15.104:0/2570875390 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec00b2b50 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.060+0000 7f1ec6a08700 1 -- 172.21.15.104:0/2570875390 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec00c51e0 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.064+0000 7f1eb67fc700 1 -- 172.21.15.104:0/2570875390 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1ebc00cd20 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.064+0000 7f1eb67fc700 1 --2- 172.21.15.104:0/2570875390 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1ea40662f0 0x7f1ea40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.064+0000 7f1eb67fc700 1 -- 172.21.15.104:0/2570875390 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1ebc084140 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.064+0000 7f1ec5205700 1 --2- 172.21.15.104:0/2570875390 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1ea40662f0 0x7f1ea40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.064+0000 7f1ec5205700 1 --2- 172.21.15.104:0/2570875390 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1ea40662f0 0x7f1ea40687a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0007fd0 tx=0x7f1eb0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:15.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.068+0000 7f1eb67fc700 1 -- 172.21.15.104:0/2570875390 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f1ebc054d10 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.236+0000 7f1ec6a08700 1 -- 172.21.15.104:0/2570875390 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ec004e500 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.236+0000 7f1eb67fc700 1 -- 172.21.15.104:0/2570875390 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1ebc0548a0 con 0x7f1ec00cd7f0 2023-07-14T11:54:15.221 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:15.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.240+0000 7f1ec6a08700 1 -- 172.21.15.104:0/2570875390 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1ea40662f0 msgr2=0x7f1ea40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.240+0000 7f1ec6a08700 1 --2- 172.21.15.104:0/2570875390 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1ea40662f0 0x7f1ea40687a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0007fd0 tx=0x7f1eb0009040 comp rx=0 tx=0).stop 2023-07-14T11:54:15.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.240+0000 7f1ec6a08700 1 -- 172.21.15.104:0/2570875390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00cd7f0 msgr2=0x7f1ec015a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.240+0000 7f1ec6a08700 1 --2- 172.21.15.104:0/2570875390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00cd7f0 0x7f1ec015a7b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00e9d0 tx=0x7f1ebc00ed90 comp rx=0 tx=0).stop 2023-07-14T11:54:15.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.240+0000 7f1ec6a08700 1 -- 172.21.15.104:0/2570875390 shutdown_connections 2023-07-14T11:54:15.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.240+0000 7f1ec6a08700 1 --2- 172.21.15.104:0/2570875390 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1ea40662f0 0x7f1ea40687a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.240+0000 7f1ec6a08700 1 --2- 172.21.15.104:0/2570875390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ec01603d0 0x7f1ec00aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.240+0000 7f1ec6a08700 1 --2- 172.21.15.104:0/2570875390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ec0160000 0x7f1ec00c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.240+0000 7f1ec6a08700 1 --2- 172.21.15.104:0/2570875390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00cd7f0 0x7f1ec015a7b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.240+0000 7f1ec6a08700 1 -- 172.21.15.104:0/2570875390 >> 172.21.15.104:0/2570875390 conn(0x7f1ec00afc60 msgr2=0x7f1ec015f610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:15.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.240+0000 7f1ec6a08700 1 -- 172.21.15.104:0/2570875390 shutdown_connections 2023-07-14T11:54:15.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.240+0000 7f1ec6a08700 1 -- 172.21.15.104:0/2570875390 wait complete. 2023-07-14T11:54:15.248 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:54:15.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.432+0000 7f8cd5c78700 1 Processor -- start 2023-07-14T11:54:15.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.432+0000 7f8cd5c78700 1 -- start start 2023-07-14T11:54:15.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cd5c78700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8cd00d0210 0x7f8cd00d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cd5c78700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd0071d10 con 0x7f8cd00cf000 2023-07-14T11:54:15.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cd5c78700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd015c420 con 0x7f8cd00b1c50 2023-07-14T11:54:15.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cd5c78700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd0070840 con 0x7f8cd00d0210 2023-07-14T11:54:15.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8ccffff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f8cd00cf000 0x7f8cd00ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:44848/0 (socket says 172.21.15.104:44848) 2023-07-14T11:54:15.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8ccffff700 1 -- 172.21.15.104:0/3584737337 learned_addr learned my addr 172.21.15.104:0/3584737337 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:15.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cceffd700 1 --2- 172.21.15.104:0/3584737337 >> v2:172.21.15.104:3300/0 conn(0x7f8cd00d0210 0x7f8cd00d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cce7fc700 1 -- 172.21.15.104:0/3584737337 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667041808 0 0) 0x7f8cc8001490 con 0x7f8cd00cf000 2023-07-14T11:54:15.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cce7fc700 1 -- 172.21.15.104:0/3584737337 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cb80035d0 con 0x7f8cd00cf000 2023-07-14T11:54:15.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cce7fc700 1 -- 172.21.15.104:0/3584737337 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1594319799 0 0) 0x7f8cb80035d0 con 0x7f8cd00cf000 2023-07-14T11:54:15.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cce7fc700 1 -- 172.21.15.104:0/3584737337 >> v2:172.21.15.104:3300/0 conn(0x7f8cd00d0210 msgr2=0x7f8cd00d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:15.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cce7fc700 1 --2- 172.21.15.104:0/3584737337 >> v2:172.21.15.104:3300/0 conn(0x7f8cd00d0210 0x7f8cd00d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cce7fc700 1 -- 172.21.15.104:0/3584737337 >> v1:172.21.15.121:6789/0 conn(0x7f8cd00b1c50 legacy=0x7f8cd0071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cce7fc700 1 -- 172.21.15.104:0/3584737337 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd0070b20 con 0x7f8cd00cf000 2023-07-14T11:54:15.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cce7fc700 1 -- 172.21.15.104:0/3584737337 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f8cc8001b80 con 0x7f8cd00cf000 2023-07-14T11:54:15.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cceffd700 1 --2- 172.21.15.104:0/3584737337 >> v2:172.21.15.104:3300/0 conn(0x7f8cd00d0210 0x7f8cd00d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:15.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cce7fc700 1 -- 172.21.15.104:0/3584737337 >> v1:172.21.15.104:6789/0 conn(0x7f8cd00cf000 legacy=0x7f8cd00ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cce7fc700 1 --2- 172.21.15.104:0/3584737337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003a20 0x7f8cb8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cce7fc700 1 --2- 172.21.15.104:0/3584737337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cb80043c0 0x7f8cb8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cce7fc700 1 --2- 172.21.15.104:0/3584737337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cb8005110 0x7f8cb800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cce7fc700 1 -- 172.21.15.104:0/3584737337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd0070840 con 0x7f8cb8003a20 2023-07-14T11:54:15.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cce7fc700 1 -- 172.21.15.104:0/3584737337 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb800c050 con 0x7f8cb8005110 2023-07-14T11:54:15.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cce7fc700 1 -- 172.21.15.104:0/3584737337 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb800c210 con 0x7f8cb80043c0 2023-07-14T11:54:15.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8ccf7fe700 1 --2- 172.21.15.104:0/3584737337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003a20 0x7f8cb8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8cceffd700 1 --2- 172.21.15.104:0/3584737337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cb80043c0 0x7f8cb8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8ccffff700 1 --2- 172.21.15.104:0/3584737337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cb8005110 0x7f8cb800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8ccf7fe700 1 -- 172.21.15.104:0/3584737337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cb80043c0 msgr2=0x7f8cb8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:15.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8ccf7fe700 1 --2- 172.21.15.104:0/3584737337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cb80043c0 0x7f8cb8004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.436+0000 7f8ccf7fe700 1 -- 172.21.15.104:0/3584737337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cb8005110 msgr2=0x7f8cb800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:15.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.440+0000 7f8ccf7fe700 1 --2- 172.21.15.104:0/3584737337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cb8005110 0x7f8cb800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.440+0000 7f8ccffff700 1 -- 172.21.15.104:0/3584737337 reap_dead start 2023-07-14T11:54:15.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.440+0000 7f8ccf7fe700 1 -- 172.21.15.104:0/3584737337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb80035d0 con 0x7f8cb8003a20 2023-07-14T11:54:15.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.440+0000 7f8ccf7fe700 1 --2- 172.21.15.104:0/3584737337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003a20 0x7f8cb8003df0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00018f0 tx=0x7f8cb80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ed04d937ff283c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:15.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.440+0000 7f8cce7fc700 1 -- 172.21.15.104:0/3584737337 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0001390 con 0x7f8cb8003a20 2023-07-14T11:54:15.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.440+0000 7f8cce7fc700 1 -- 172.21.15.104:0/3584737337 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc00052d0 con 0x7f8cb8003a20 2023-07-14T11:54:15.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.440+0000 7f8cce7fc700 1 -- 172.21.15.104:0/3584737337 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00036a0 con 0x7f8cb8003a20 2023-07-14T11:54:15.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.440+0000 7f8cd5c78700 1 -- 172.21.15.104:0/3584737337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003a20 msgr2=0x7f8cb8003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.440+0000 7f8cd5c78700 1 --2- 172.21.15.104:0/3584737337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003a20 0x7f8cb8003df0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00018f0 tx=0x7f8cb80058a0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.440+0000 7f8cd5c78700 1 -- 172.21.15.104:0/3584737337 shutdown_connections 2023-07-14T11:54:15.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.440+0000 7f8cd5c78700 1 --2- 172.21.15.104:0/3584737337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003a20 0x7f8cb8003df0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.440+0000 7f8cd5c78700 1 -- 172.21.15.104:0/3584737337 >> 172.21.15.104:0/3584737337 conn(0x7f8cd009b9c0 msgr2=0x7f8cd00d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:15.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.440+0000 7f8cd5c78700 1 -- 172.21.15.104:0/3584737337 shutdown_connections 2023-07-14T11:54:15.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.440+0000 7f8cd5c78700 1 -- 172.21.15.104:0/3584737337 wait complete. 2023-07-14T11:54:15.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8cd5c78700 1 Processor -- start 2023-07-14T11:54:15.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8cd5c78700 1 -- start start 2023-07-14T11:54:15.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8cd5c78700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003a20 0x7f8cd015d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8cd5c78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cd01f5430 0x7f8cd015d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8cd5c78700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cd01541d0 0x7f8cd015eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8cd5c78700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd0158460 con 0x7f8cb8003a20 2023-07-14T11:54:15.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8cd5c78700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd015e1a0 con 0x7f8cd01f5430 2023-07-14T11:54:15.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8cd5c78700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd015dec0 con 0x7f8cd01541d0 2023-07-14T11:54:15.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8ccf7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003a20 0x7f8cd015d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8ccf7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003a20 0x7f8cd015d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35166/0 (socket says 172.21.15.104:35166) 2023-07-14T11:54:15.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8ccf7fe700 1 -- 172.21.15.104:0/2618240305 learned_addr learned my addr 172.21.15.104:0/2618240305 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:15.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8cceffd700 1 --2- 172.21.15.104:0/2618240305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cd01f5430 0x7f8cd015d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8ccffff700 1 --2- 172.21.15.104:0/2618240305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cd01541d0 0x7f8cd015eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8ccf7fe700 1 -- 172.21.15.104:0/2618240305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cd01541d0 msgr2=0x7f8cd015eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8ccf7fe700 1 --2- 172.21.15.104:0/2618240305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cd01541d0 0x7f8cd015eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8ccf7fe700 1 -- 172.21.15.104:0/2618240305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cd01f5430 msgr2=0x7f8cd015d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8ccf7fe700 1 --2- 172.21.15.104:0/2618240305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cd01f5430 0x7f8cd015d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8ccf7fe700 1 -- 172.21.15.104:0/2618240305 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd0157e10 con 0x7f8cb8003a20 2023-07-14T11:54:15.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.444+0000 7f8ccf7fe700 1 --2- 172.21.15.104:0/2618240305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003a20 0x7f8cd015d350 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800c020 tx=0x7f8cc0001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:15.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.448+0000 7f8cb7fff700 1 -- 172.21.15.104:0/2618240305 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0006880 con 0x7f8cb8003a20 2023-07-14T11:54:15.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.448+0000 7f8cb7fff700 1 -- 172.21.15.104:0/2618240305 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc0006a00 con 0x7f8cb8003a20 2023-07-14T11:54:15.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.448+0000 7f8cb7fff700 1 -- 172.21.15.104:0/2618240305 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0006db0 con 0x7f8cb8003a20 2023-07-14T11:54:15.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.448+0000 7f8cd5c78700 1 -- 172.21.15.104:0/2618240305 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd01580f0 con 0x7f8cb8003a20 2023-07-14T11:54:15.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.448+0000 7f8cd5c78700 1 -- 172.21.15.104:0/2618240305 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd015f050 con 0x7f8cb8003a20 2023-07-14T11:54:15.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.448+0000 7f8cd5c78700 1 -- 172.21.15.104:0/2618240305 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd0158280 con 0x7f8cb8003a20 2023-07-14T11:54:15.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.448+0000 7f8cb7fff700 1 -- 172.21.15.104:0/2618240305 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8cc0007030 con 0x7f8cb8003a20 2023-07-14T11:54:15.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.452+0000 7f8cb7fff700 1 --2- 172.21.15.104:0/2618240305 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cbc066320 0x7f8cbc0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.452+0000 7f8cb7fff700 1 -- 172.21.15.104:0/2618240305 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8cc0084e20 con 0x7f8cb8003a20 2023-07-14T11:54:15.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.452+0000 7f8cceffd700 1 --2- 172.21.15.104:0/2618240305 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cbc066320 0x7f8cbc0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.452+0000 7f8cceffd700 1 --2- 172.21.15.104:0/2618240305 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cbc066320 0x7f8cbc0687d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8cd00d1f00 tx=0x7f8cc4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:15.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.456+0000 7f8cb7fff700 1 -- 172.21.15.104:0/2618240305 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f8cc0052000 con 0x7f8cb8003a20 2023-07-14T11:54:15.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.628+0000 7f8cd5c78700 1 -- 172.21.15.104:0/2618240305 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8cd004dcb0 con 0x7f8cb8003a20 2023-07-14T11:54:15.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.628+0000 7f8cb7fff700 1 -- 172.21.15.104:0/2618240305 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8cc0055620 con 0x7f8cb8003a20 2023-07-14T11:54:15.614 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:54:15.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.632+0000 7f8cd5c78700 1 -- 172.21.15.104:0/2618240305 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cbc066320 msgr2=0x7f8cbc0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.632+0000 7f8cd5c78700 1 --2- 172.21.15.104:0/2618240305 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cbc066320 0x7f8cbc0687d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8cd00d1f00 tx=0x7f8cc4008040 comp rx=0 tx=0).stop 2023-07-14T11:54:15.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.632+0000 7f8cd5c78700 1 -- 172.21.15.104:0/2618240305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003a20 msgr2=0x7f8cd015d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.632+0000 7f8cd5c78700 1 --2- 172.21.15.104:0/2618240305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003a20 0x7f8cd015d350 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800c020 tx=0x7f8cc0001230 comp rx=0 tx=0).stop 2023-07-14T11:54:15.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.632+0000 7f8cd5c78700 1 -- 172.21.15.104:0/2618240305 shutdown_connections 2023-07-14T11:54:15.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.632+0000 7f8cd5c78700 1 --2- 172.21.15.104:0/2618240305 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cbc066320 0x7f8cbc0687d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.632+0000 7f8cd5c78700 1 --2- 172.21.15.104:0/2618240305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cd01541d0 0x7f8cd015eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.632+0000 7f8cd5c78700 1 --2- 172.21.15.104:0/2618240305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cd01f5430 0x7f8cd015d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.632+0000 7f8cd5c78700 1 --2- 172.21.15.104:0/2618240305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003a20 0x7f8cd015d350 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.632+0000 7f8cd5c78700 1 -- 172.21.15.104:0/2618240305 >> 172.21.15.104:0/2618240305 conn(0x7f8cd009b9c0 msgr2=0x7f8cd00b7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:15.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.632+0000 7f8cd5c78700 1 -- 172.21.15.104:0/2618240305 shutdown_connections 2023-07-14T11:54:15.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.632+0000 7f8cd5c78700 1 -- 172.21.15.104:0/2618240305 wait complete. 2023-07-14T11:54:15.641 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:54:15.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.832+0000 7fcd1bcc6700 1 Processor -- start 2023-07-14T11:54:15.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.832+0000 7fcd1bcc6700 1 -- start start 2023-07-14T11:54:15.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.836+0000 7fcd1bcc6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcd140b1c50 0x7fcd14071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.836+0000 7fcd1bcc6700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1415c420 con 0x7fcd140d0210 2023-07-14T11:54:15.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.836+0000 7fcd1bcc6700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd14070840 con 0x7fcd140cf710 2023-07-14T11:54:15.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.836+0000 7fcd1bcc6700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd14070b20 con 0x7fcd140b1c50 2023-07-14T11:54:15.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.836+0000 7fcd19261700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fcd140d0210 0x7fcd140cf000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:44862/0 (socket says 172.21.15.104:44862) 2023-07-14T11:54:15.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.836+0000 7fcd19261700 1 -- 172.21.15.104:0/2881487317 learned_addr learned my addr 172.21.15.104:0/2881487317 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:15.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.836+0000 7fcd19a62700 1 --2- 172.21.15.104:0/2881487317 >> v2:172.21.15.121:3300/0 conn(0x7fcd140b1c50 0x7fcd14071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.836+0000 7fcd18a60700 1 -- 172.21.15.104:0/2881487317 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912360514 0 0) 0x7fcd1415c420 con 0x7fcd140d0210 2023-07-14T11:54:15.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.836+0000 7fcd18a60700 1 -- 172.21.15.104:0/2881487317 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd000035d0 con 0x7fcd140d0210 2023-07-14T11:54:15.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.836+0000 7fcd19a62700 1 -- 172.21.15.104:0/2881487317 >> v1:172.21.15.121:6789/0 conn(0x7fcd140cf710 legacy=0x7fcd140ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.836+0000 7fcd19a62700 1 -- 172.21.15.104:0/2881487317 >> v1:172.21.15.104:6789/0 conn(0x7fcd140d0210 legacy=0x7fcd140cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.836+0000 7fcd19a62700 1 -- 172.21.15.104:0/2881487317 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd14070e00 con 0x7fcd140b1c50 2023-07-14T11:54:15.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.836+0000 7fcd19a62700 1 --2- 172.21.15.104:0/2881487317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b1c50 0x7fcd14071600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009850 tx=0x7fcd10009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33eafe40420938b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:15.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.840+0000 7fcd18a60700 1 -- 172.21.15.104:0/2881487317 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10011830 con 0x7fcd140b1c50 2023-07-14T11:54:15.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.840+0000 7fcd18a60700 1 -- 172.21.15.104:0/2881487317 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd10011990 con 0x7fcd140b1c50 2023-07-14T11:54:15.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.840+0000 7fcd18a60700 1 -- 172.21.15.104:0/2881487317 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10011d40 con 0x7fcd140b1c50 2023-07-14T11:54:15.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.840+0000 7fcd1bcc6700 1 -- 172.21.15.104:0/2881487317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b1c50 msgr2=0x7fcd14071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.840+0000 7fcd1bcc6700 1 --2- 172.21.15.104:0/2881487317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b1c50 0x7fcd14071600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009850 tx=0x7fcd10009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:15.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.840+0000 7fcd1bcc6700 1 -- 172.21.15.104:0/2881487317 shutdown_connections 2023-07-14T11:54:15.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.840+0000 7fcd1bcc6700 1 --2- 172.21.15.104:0/2881487317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b1c50 0x7fcd14071600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.840+0000 7fcd1bcc6700 1 -- 172.21.15.104:0/2881487317 >> 172.21.15.104:0/2881487317 conn(0x7fcd1409b9c0 msgr2=0x7fcd140d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:15.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.840+0000 7fcd1bcc6700 1 -- 172.21.15.104:0/2881487317 shutdown_connections 2023-07-14T11:54:15.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.840+0000 7fcd1bcc6700 1 -- 172.21.15.104:0/2881487317 wait complete. 2023-07-14T11:54:15.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.840+0000 7fcd1bcc6700 1 Processor -- start 2023-07-14T11:54:15.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.840+0000 7fcd1bcc6700 1 -- start start 2023-07-14T11:54:15.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1bcc6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b1c50 0x7fcd140b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1bcc6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd140cf710 0x7fcd140b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1bcc6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd140d0210 0x7fcd14154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1bcc6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140b2860 con 0x7fcd140d0210 2023-07-14T11:54:15.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1bcc6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140b4f30 con 0x7fcd140b1c50 2023-07-14T11:54:15.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1bcc6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140b77b0 con 0x7fcd140cf710 2023-07-14T11:54:15.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1a263700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd140d0210 0x7fcd14154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1a263700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd140d0210 0x7fcd14154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35168/0 (socket says 172.21.15.104:35168) 2023-07-14T11:54:15.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1a263700 1 -- 172.21.15.104:0/14245602 learned_addr learned my addr 172.21.15.104:0/14245602 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:15.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd19a62700 1 --2- 172.21.15.104:0/14245602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b1c50 0x7fcd140b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd19261700 1 --2- 172.21.15.104:0/14245602 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd140cf710 0x7fcd140b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1a263700 1 -- 172.21.15.104:0/14245602 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd140cf710 msgr2=0x7fcd140b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1a263700 1 --2- 172.21.15.104:0/14245602 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd140cf710 0x7fcd140b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1a263700 1 -- 172.21.15.104:0/14245602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b1c50 msgr2=0x7fcd140b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:15.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1a263700 1 --2- 172.21.15.104:0/14245602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b1c50 0x7fcd140b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:15.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1a263700 1 -- 172.21.15.104:0/14245602 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd100094b0 con 0x7fcd140d0210 2023-07-14T11:54:15.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1a263700 1 --2- 172.21.15.104:0/14245602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd140d0210 0x7fcd14154d40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00cbc0 tx=0x7fcd0c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:15.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd0a7fc700 1 -- 172.21.15.104:0/14245602 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c003c20 con 0x7fcd140d0210 2023-07-14T11:54:15.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd0a7fc700 1 -- 172.21.15.104:0/14245602 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c003d80 con 0x7fcd140d0210 2023-07-14T11:54:15.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd0a7fc700 1 -- 172.21.15.104:0/14245602 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c00aae0 con 0x7fcd140d0210 2023-07-14T11:54:15.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1bcc6700 1 -- 172.21.15.104:0/14245602 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd140b7a90 con 0x7fcd140d0210 2023-07-14T11:54:15.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.844+0000 7fcd1bcc6700 1 -- 172.21.15.104:0/14245602 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd14155280 con 0x7fcd140d0210 2023-07-14T11:54:15.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.848+0000 7fcd1bcc6700 1 -- 172.21.15.104:0/14245602 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1404dcb0 con 0x7fcd140d0210 2023-07-14T11:54:15.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.848+0000 7fcd0a7fc700 1 -- 172.21.15.104:0/14245602 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fcd0c00ac40 con 0x7fcd140d0210 2023-07-14T11:54:15.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.852+0000 7fcd0a7fc700 1 --2- 172.21.15.104:0/14245602 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fccf80662f0 0x7fccf80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:15.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.852+0000 7fcd0a7fc700 1 -- 172.21.15.104:0/14245602 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcd0c083ff0 con 0x7fcd140d0210 2023-07-14T11:54:15.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.852+0000 7fcd19a62700 1 --2- 172.21.15.104:0/14245602 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fccf80662f0 0x7fccf80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:15.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.852+0000 7fcd19a62700 1 --2- 172.21.15.104:0/14245602 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fccf80662f0 0x7fccf80687a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcd100090b0 tx=0x7fcd10008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:15.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:15.856+0000 7fcd0a7fc700 1 -- 172.21.15.104:0/14245602 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fcd0c051140 con 0x7fcd140d0210 2023-07-14T11:54:16.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.024+0000 7fcd1bcc6700 1 -- 172.21.15.104:0/14245602 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcd1408b400 con 0x7fcd140d0210 2023-07-14T11:54:16.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.028+0000 7fcd0a7fc700 1 -- 172.21.15.104:0/14245602 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcd0c054760 con 0x7fcd140d0210 2023-07-14T11:54:16.011 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:16.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.028+0000 7fcd1bcc6700 1 -- 172.21.15.104:0/14245602 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fccf80662f0 msgr2=0x7fccf80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:16.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.028+0000 7fcd1bcc6700 1 --2- 172.21.15.104:0/14245602 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fccf80662f0 0x7fccf80687a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcd100090b0 tx=0x7fcd10008030 comp rx=0 tx=0).stop 2023-07-14T11:54:16.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.028+0000 7fcd1bcc6700 1 -- 172.21.15.104:0/14245602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd140d0210 msgr2=0x7fcd14154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:16.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.028+0000 7fcd1bcc6700 1 --2- 172.21.15.104:0/14245602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd140d0210 0x7fcd14154d40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00cbc0 tx=0x7fcd0c00ced0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.032+0000 7fcd1bcc6700 1 -- 172.21.15.104:0/14245602 shutdown_connections 2023-07-14T11:54:16.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.032+0000 7fcd1bcc6700 1 --2- 172.21.15.104:0/14245602 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fccf80662f0 0x7fccf80687a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.032+0000 7fcd1bcc6700 1 --2- 172.21.15.104:0/14245602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd140d0210 0x7fcd14154d40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.032+0000 7fcd1bcc6700 1 --2- 172.21.15.104:0/14245602 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd140cf710 0x7fcd140b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.032+0000 7fcd1bcc6700 1 --2- 172.21.15.104:0/14245602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd140b1c50 0x7fcd140b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.032+0000 7fcd1bcc6700 1 -- 172.21.15.104:0/14245602 >> 172.21.15.104:0/14245602 conn(0x7fcd1409b9c0 msgr2=0x7fcd140b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:16.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.032+0000 7fcd1bcc6700 1 -- 172.21.15.104:0/14245602 shutdown_connections 2023-07-14T11:54:16.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.032+0000 7fcd1bcc6700 1 -- 172.21.15.104:0/14245602 wait complete. 2023-07-14T11:54:16.042 INFO:tasks.cephfs_test_runner:Starting test: test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-07-14T11:54:16.043 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2023-07-14T11:54:16.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.228+0000 7f5fb2aa2700 1 Processor -- start 2023-07-14T11:54:16.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.228+0000 7f5fb2aa2700 1 -- start start 2023-07-14T11:54:16.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.228+0000 7f5fb2aa2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fac15bf20 0x7f5fac0b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:16.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.228+0000 7f5fb2aa2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5fac0d1430 0x7f5fac0b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:16.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.228+0000 7f5fb2aa2700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fac15c2f0 con 0x7f5fac0b1530 2023-07-14T11:54:16.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.228+0000 7f5fb2aa2700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fac070830 con 0x7f5fac0d1430 2023-07-14T11:54:16.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.228+0000 7f5fb2aa2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac070b10 con 0x7f5fac15bf20 2023-07-14T11:54:16.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.228+0000 7f5fabfff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f5fac0b1530 0x7f5fac0d0d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:44872/0 (socket says 172.21.15.104:44872) 2023-07-14T11:54:16.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.228+0000 7f5fabfff700 1 -- 172.21.15.104:0/3818937108 learned_addr learned my addr 172.21.15.104:0/3818937108 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:16.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.228+0000 7f5fb083e700 1 --2- 172.21.15.104:0/3818937108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fac15bf20 0x7f5fac0b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:16.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fb103f700 1 --2- 172.21.15.104:0/3818937108 >> v2:172.21.15.121:3300/0 conn(0x7f5fac0d1430 0x7f5fac0b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:16.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 -- 172.21.15.104:0/3818937108 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057376069 0 0) 0x7f5fac15c2f0 con 0x7f5fac0b1530 2023-07-14T11:54:16.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 -- 172.21.15.104:0/3818937108 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f9c0035d0 con 0x7f5fac0b1530 2023-07-14T11:54:16.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 -- 172.21.15.104:0/3818937108 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1380288290 0 0) 0x7f5f9c0035d0 con 0x7f5fac0b1530 2023-07-14T11:54:16.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 -- 172.21.15.104:0/3818937108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fac15bf20 msgr2=0x7f5fac0b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:16.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 --2- 172.21.15.104:0/3818937108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fac15bf20 0x7f5fac0b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 -- 172.21.15.104:0/3818937108 >> v2:172.21.15.121:3300/0 conn(0x7f5fac0d1430 msgr2=0x7f5fac0b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:16.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 --2- 172.21.15.104:0/3818937108 >> v2:172.21.15.121:3300/0 conn(0x7f5fac0d1430 0x7f5fac0b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 -- 172.21.15.104:0/3818937108 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac0cccc0 con 0x7f5fac0b1530 2023-07-14T11:54:16.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 -- 172.21.15.104:0/3818937108 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f5f98001670 con 0x7f5fac0b1530 2023-07-14T11:54:16.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 -- 172.21.15.104:0/3818937108 >> v1:172.21.15.104:6789/0 conn(0x7f5fac0b1530 legacy=0x7f5fac0d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:16.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 --2- 172.21.15.104:0/3818937108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f9c003a20 0x7f5f9c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:16.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 --2- 172.21.15.104:0/3818937108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f9c004450 0x7f5f9c004910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:16.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 --2- 172.21.15.104:0/3818937108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fac15bf20 0x7f5f9c005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:16.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 -- 172.21.15.104:0/3818937108 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac070830 con 0x7f5f9c003a20 2023-07-14T11:54:16.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 -- 172.21.15.104:0/3818937108 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac070b10 con 0x7f5f9c004450 2023-07-14T11:54:16.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 -- 172.21.15.104:0/3818937108 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c00bab0 con 0x7f5fac15bf20 2023-07-14T11:54:16.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fb103f700 1 --2- 172.21.15.104:0/3818937108 >> v2:172.21.15.121:3300/0 conn(0x7f5fac0d1430 0x7f5fac0b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:16.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fb083e700 1 --2- 172.21.15.104:0/3818937108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f9c003a20 0x7f5f9c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:16.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fabfff700 1 --2- 172.21.15.104:0/3818937108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f9c004450 0x7f5f9c004910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:16.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fb083e700 1 -- 172.21.15.104:0/3818937108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fac15bf20 msgr2=0x7f5f9c005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:16.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fb083e700 1 --2- 172.21.15.104:0/3818937108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fac15bf20 0x7f5f9c005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fb083e700 1 -- 172.21.15.104:0/3818937108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f9c004450 msgr2=0x7f5f9c004910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:16.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fb083e700 1 --2- 172.21.15.104:0/3818937108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f9c004450 0x7f5f9c004910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fb083e700 1 -- 172.21.15.104:0/3818937108 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c0035d0 con 0x7f5f9c003a20 2023-07-14T11:54:16.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fb083e700 1 --2- 172.21.15.104:0/3818937108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f9c003a20 0x7f5f9c003e30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009b70 tx=0x7f5f9c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b3dc0a039a78b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:16.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 -- 172.21.15.104:0/3818937108 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa000e6c0 con 0x7f5f9c003a20 2023-07-14T11:54:16.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 -- 172.21.15.104:0/3818937108 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa000e820 con 0x7f5f9c003a20 2023-07-14T11:54:16.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fab7fe700 1 -- 172.21.15.104:0/3818937108 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa000eb90 con 0x7f5f9c003a20 2023-07-14T11:54:16.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fb2aa2700 1 -- 172.21.15.104:0/3818937108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f9c003a20 msgr2=0x7f5f9c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:16.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.232+0000 7f5fb2aa2700 1 --2- 172.21.15.104:0/3818937108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f9c003a20 0x7f5f9c003e30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009b70 tx=0x7f5f9c0058a0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.236+0000 7f5fb103f700 1 -- 172.21.15.104:0/3818937108 reap_dead start 2023-07-14T11:54:16.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.236+0000 7f5fb2aa2700 1 -- 172.21.15.104:0/3818937108 shutdown_connections 2023-07-14T11:54:16.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.236+0000 7f5fb2aa2700 1 -- 172.21.15.104:0/3818937108 >> 172.21.15.104:0/3818937108 conn(0x7f5fac06fc80 msgr2=0x7f5fac070080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:16.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.236+0000 7f5fb2aa2700 1 -- 172.21.15.104:0/3818937108 shutdown_connections 2023-07-14T11:54:16.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.236+0000 7f5fb2aa2700 1 -- 172.21.15.104:0/3818937108 wait complete. 2023-07-14T11:54:16.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.236+0000 7f5fb2aa2700 1 Processor -- start 2023-07-14T11:54:16.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.236+0000 7f5fb2aa2700 1 -- start start 2023-07-14T11:54:16.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb2aa2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fac14fbd0 0x7f5fac15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:16.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb2aa2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fac15ace0 0x7f5fac0ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:16.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb2aa2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fac0ca5f0 0x7f5fac0b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:16.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb2aa2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac15b150 con 0x7f5fac0ca5f0 2023-07-14T11:54:16.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb2aa2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac0cadc0 con 0x7f5fac14fbd0 2023-07-14T11:54:16.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb2aa2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac0cb0a0 con 0x7f5fac15ace0 2023-07-14T11:54:16.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb103f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fac0ca5f0 0x7f5fac0b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:16.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb103f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fac0ca5f0 0x7f5fac0b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35186/0 (socket says 172.21.15.104:35186) 2023-07-14T11:54:16.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb103f700 1 -- 172.21.15.104:0/3529756925 learned_addr learned my addr 172.21.15.104:0/3529756925 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:16.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb083e700 1 --2- 172.21.15.104:0/3529756925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fac14fbd0 0x7f5fac15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:16.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fabfff700 1 --2- 172.21.15.104:0/3529756925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fac15ace0 0x7f5fac0ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:16.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb103f700 1 -- 172.21.15.104:0/3529756925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fac15ace0 msgr2=0x7f5fac0ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:16.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb103f700 1 --2- 172.21.15.104:0/3529756925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fac15ace0 0x7f5fac0ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb103f700 1 -- 172.21.15.104:0/3529756925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fac14fbd0 msgr2=0x7f5fac15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:16.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb103f700 1 --2- 172.21.15.104:0/3529756925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fac14fbd0 0x7f5fac15a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb103f700 1 -- 172.21.15.104:0/3529756925 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac0cb380 con 0x7f5fac0ca5f0 2023-07-14T11:54:16.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb103f700 1 --2- 172.21.15.104:0/3529756925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fac0ca5f0 0x7f5fac0b1f30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0612e0 tx=0x7f5fa4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:16.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fa97fa700 1 -- 172.21.15.104:0/3529756925 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4003c40 con 0x7f5fac0ca5f0 2023-07-14T11:54:16.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fa97fa700 1 -- 172.21.15.104:0/3529756925 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa400f040 con 0x7f5fac0ca5f0 2023-07-14T11:54:16.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb2aa2700 1 -- 172.21.15.104:0/3529756925 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fac0cb660 con 0x7f5fac0ca5f0 2023-07-14T11:54:16.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fa97fa700 1 -- 172.21.15.104:0/3529756925 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa400d7c0 con 0x7f5fac0ca5f0 2023-07-14T11:54:16.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.240+0000 7f5fb2aa2700 1 -- 172.21.15.104:0/3529756925 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fac14ffa0 con 0x7f5fac0ca5f0 2023-07-14T11:54:16.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.244+0000 7f5fb2aa2700 1 -- 172.21.15.104:0/3529756925 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fac04dcb0 con 0x7f5fac0ca5f0 2023-07-14T11:54:16.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.244+0000 7f5fa97fa700 1 -- 172.21.15.104:0/3529756925 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5fa400d920 con 0x7f5fac0ca5f0 2023-07-14T11:54:16.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.244+0000 7f5fa97fa700 1 --2- 172.21.15.104:0/3529756925 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5f900662f0 0x7f5f900687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:16.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.244+0000 7f5fa97fa700 1 -- 172.21.15.104:0/3529756925 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5fa4083c90 con 0x7f5fac0ca5f0 2023-07-14T11:54:16.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.244+0000 7f5fb083e700 1 --2- 172.21.15.104:0/3529756925 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5f900662f0 0x7f5f900687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:16.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.248+0000 7f5fb083e700 1 --2- 172.21.15.104:0/3529756925 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5f900662f0 0x7f5f900687a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5fac068530 tx=0x7f5fa001f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:16.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.252+0000 7f5fa97fa700 1 -- 172.21.15.104:0/3529756925 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f5fa4050d40 con 0x7f5fac0ca5f0 2023-07-14T11:54:16.473 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.488+0000 7f5fb2aa2700 1 -- 172.21.15.104:0/3529756925 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f5fac150240 con 0x7f5fac0ca5f0 2023-07-14T11:54:16.730 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.744+0000 7f5fa97fa700 1 -- 172.21.15.104:0/3529756925 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v80) v1 ==== 150+0+0 (secure 0 0 0) 0x7f5fa4054360 con 0x7f5fac0ca5f0 2023-07-14T11:54:16.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.752+0000 7f5fb2aa2700 1 -- 172.21.15.104:0/3529756925 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5f900662f0 msgr2=0x7f5f900687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:16.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.752+0000 7f5fb2aa2700 1 --2- 172.21.15.104:0/3529756925 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5f900662f0 0x7f5f900687a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5fac068530 tx=0x7f5fa001f040 comp rx=0 tx=0).stop 2023-07-14T11:54:16.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.752+0000 7f5fb2aa2700 1 -- 172.21.15.104:0/3529756925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fac0ca5f0 msgr2=0x7f5fac0b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:16.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.752+0000 7f5fb2aa2700 1 --2- 172.21.15.104:0/3529756925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fac0ca5f0 0x7f5fac0b1f30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0612e0 tx=0x7f5fa4007f90 comp rx=0 tx=0).stop 2023-07-14T11:54:16.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.752+0000 7f5fb2aa2700 1 -- 172.21.15.104:0/3529756925 shutdown_connections 2023-07-14T11:54:16.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.752+0000 7f5fb2aa2700 1 --2- 172.21.15.104:0/3529756925 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5f900662f0 0x7f5f900687a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.752+0000 7f5fb2aa2700 1 --2- 172.21.15.104:0/3529756925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fac0ca5f0 0x7f5fac0b1f30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.752+0000 7f5fb2aa2700 1 --2- 172.21.15.104:0/3529756925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fac15ace0 0x7f5fac0ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.752+0000 7f5fb2aa2700 1 --2- 172.21.15.104:0/3529756925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fac14fbd0 0x7f5fac15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.752+0000 7f5fb2aa2700 1 -- 172.21.15.104:0/3529756925 >> 172.21.15.104:0/3529756925 conn(0x7f5fac06fc80 msgr2=0x7f5fac15cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:16.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.752+0000 7f5fb2aa2700 1 -- 172.21.15.104:0/3529756925 shutdown_connections 2023-07-14T11:54:16.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.752+0000 7f5fb2aa2700 1 -- 172.21.15.104:0/3529756925 wait complete. 2023-07-14T11:54:16.758 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-14T11:54:16.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.956+0000 7fcf76595700 1 Processor -- start 2023-07-14T11:54:16.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.956+0000 7fcf76595700 1 -- start start 2023-07-14T11:54:16.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.960+0000 7fcf76595700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcf70160450 0x7fcf700af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:16.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.960+0000 7fcf76595700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf700af270 0x7fcf700c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:16.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.960+0000 7fcf76595700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf700d10a0 con 0x7fcf700c3810 2023-07-14T11:54:16.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.960+0000 7fcf76595700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf700c3c90 con 0x7fcf70160450 2023-07-14T11:54:16.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.960+0000 7fcf76595700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf70070f40 con 0x7fcf700af270 2023-07-14T11:54:16.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.960+0000 7fcf75593700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcf70160450 0x7fcf700af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:16.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.960+0000 7fcf74d92700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf700af270 0x7fcf700c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:16.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.960+0000 7fcf75d94700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fcf700c3810 0x7fcf70069950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:53942/0 (socket says 172.21.15.104:53942) 2023-07-14T11:54:16.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.960+0000 7fcf75d94700 1 -- 172.21.15.104:0/2567318293 learned_addr learned my addr 172.21.15.104:0/2567318293 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:16.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.960+0000 7fcf67fff700 1 -- 172.21.15.104:0/2567318293 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288334569 0 0) 0x7fcf700d10a0 con 0x7fcf700c3810 2023-07-14T11:54:16.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.960+0000 7fcf67fff700 1 -- 172.21.15.104:0/2567318293 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf5c0035d0 con 0x7fcf700c3810 2023-07-14T11:54:16.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.960+0000 7fcf75593700 1 -- 172.21.15.104:0/2567318293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf700af270 msgr2=0x7fcf700c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:16.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.960+0000 7fcf75593700 1 --2- 172.21.15.104:0/2567318293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf700af270 0x7fcf700c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.960+0000 7fcf75593700 1 -- 172.21.15.104:0/2567318293 >> v1:172.21.15.121:6789/0 conn(0x7fcf700c3810 legacy=0x7fcf70069950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:16.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.960+0000 7fcf75593700 1 -- 172.21.15.104:0/2567318293 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf700711e0 con 0x7fcf70160450 2023-07-14T11:54:16.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.964+0000 7fcf75593700 1 --2- 172.21.15.104:0/2567318293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf70160450 0x7fcf700af7c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009840 tx=0x7fcf6c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98fd276044562bde server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:16.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.964+0000 7fcf67fff700 1 -- 172.21.15.104:0/2567318293 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c005e50 con 0x7fcf70160450 2023-07-14T11:54:16.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.964+0000 7fcf67fff700 1 -- 172.21.15.104:0/2567318293 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c006030 con 0x7fcf70160450 2023-07-14T11:54:16.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.964+0000 7fcf67fff700 1 -- 172.21.15.104:0/2567318293 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0063c0 con 0x7fcf70160450 2023-07-14T11:54:16.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.964+0000 7fcf76595700 1 -- 172.21.15.104:0/2567318293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf70160450 msgr2=0x7fcf700af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:16.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.964+0000 7fcf76595700 1 --2- 172.21.15.104:0/2567318293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf70160450 0x7fcf700af7c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009840 tx=0x7fcf6c009b50 comp rx=0 tx=0).stop 2023-07-14T11:54:16.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.964+0000 7fcf76595700 1 -- 172.21.15.104:0/2567318293 shutdown_connections 2023-07-14T11:54:16.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.964+0000 7fcf76595700 1 --2- 172.21.15.104:0/2567318293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf700af270 0x7fcf700c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.964+0000 7fcf76595700 1 --2- 172.21.15.104:0/2567318293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf70160450 0x7fcf700af7c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.964+0000 7fcf76595700 1 -- 172.21.15.104:0/2567318293 >> 172.21.15.104:0/2567318293 conn(0x7fcf700d0300 msgr2=0x7fcf700d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:16.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.964+0000 7fcf76595700 1 -- 172.21.15.104:0/2567318293 shutdown_connections 2023-07-14T11:54:16.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.964+0000 7fcf76595700 1 -- 172.21.15.104:0/2567318293 wait complete. 2023-07-14T11:54:16.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf76595700 1 Processor -- start 2023-07-14T11:54:16.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf76595700 1 -- start start 2023-07-14T11:54:16.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf76595700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf700af270 0x7fcf7015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:16.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf76595700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf700c3810 0x7fcf701e9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:16.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf76595700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf70160450 0x7fcf701e9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:16.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf76595700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7015ca80 con 0x7fcf70160450 2023-07-14T11:54:16.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf76595700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7015be90 con 0x7fcf700af270 2023-07-14T11:54:16.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf76595700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7015ccd0 con 0x7fcf700c3810 2023-07-14T11:54:16.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf75d94700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf70160450 0x7fcf701e9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:16.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf75d94700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf70160450 0x7fcf701e9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35188/0 (socket says 172.21.15.104:35188) 2023-07-14T11:54:16.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf75d94700 1 -- 172.21.15.104:0/2742602629 learned_addr learned my addr 172.21.15.104:0/2742602629 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:16.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf74d92700 1 --2- 172.21.15.104:0/2742602629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf700c3810 0x7fcf701e9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:16.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf75593700 1 --2- 172.21.15.104:0/2742602629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf700af270 0x7fcf7015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:16.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf75d94700 1 -- 172.21.15.104:0/2742602629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf700c3810 msgr2=0x7fcf701e9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:16.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf75d94700 1 --2- 172.21.15.104:0/2742602629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf700c3810 0x7fcf701e9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf75d94700 1 -- 172.21.15.104:0/2742602629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf700af270 msgr2=0x7fcf7015b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:16.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf75d94700 1 --2- 172.21.15.104:0/2742602629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf700af270 0x7fcf7015b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:16.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf75d94700 1 -- 172.21.15.104:0/2742602629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c009450 con 0x7fcf70160450 2023-07-14T11:54:16.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.968+0000 7fcf75d94700 1 --2- 172.21.15.104:0/2742602629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf70160450 0x7fcf701e9ff0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800cb40 tx=0x7fcf6800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:16.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.972+0000 7fcf65ffb700 1 -- 172.21.15.104:0/2742602629 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800a9e0 con 0x7fcf70160450 2023-07-14T11:54:16.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.972+0000 7fcf65ffb700 1 -- 172.21.15.104:0/2742602629 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6800ab40 con 0x7fcf70160450 2023-07-14T11:54:16.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.972+0000 7fcf76595700 1 -- 172.21.15.104:0/2742602629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7015cfb0 con 0x7fcf70160450 2023-07-14T11:54:16.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.972+0000 7fcf65ffb700 1 -- 172.21.15.104:0/2742602629 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800ce60 con 0x7fcf70160450 2023-07-14T11:54:16.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.972+0000 7fcf76595700 1 -- 172.21.15.104:0/2742602629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7015d570 con 0x7fcf70160450 2023-07-14T11:54:16.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.972+0000 7fcf76595700 1 -- 172.21.15.104:0/2742602629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7015d790 con 0x7fcf70160450 2023-07-14T11:54:16.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.972+0000 7fcf65ffb700 1 -- 172.21.15.104:0/2742602629 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fcf6801b3b0 con 0x7fcf70160450 2023-07-14T11:54:16.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.976+0000 7fcf65ffb700 1 --2- 172.21.15.104:0/2742602629 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcf540662f0 0x7fcf540687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:16.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.976+0000 7fcf65ffb700 1 -- 172.21.15.104:0/2742602629 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcf68083be0 con 0x7fcf70160450 2023-07-14T11:54:16.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.976+0000 7fcf75593700 1 --2- 172.21.15.104:0/2742602629 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcf540662f0 0x7fcf540687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:16.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.976+0000 7fcf75593700 1 --2- 172.21.15.104:0/2742602629 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcf540662f0 0x7fcf540687a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009b90 tx=0x7fcf6c008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:16.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:16.980+0000 7fcf65ffb700 1 -- 172.21.15.104:0/2742602629 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fcf68050c90 con 0x7fcf70160450 2023-07-14T11:54:17.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.160+0000 7fcf76595700 1 -- 172.21.15.104:0/2742602629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcf7004e500 con 0x7fcf70160450 2023-07-14T11:54:17.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.168+0000 7fcf65ffb700 1 -- 172.21.15.104:0/2742602629 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcf680542b0 con 0x7fcf70160450 2023-07-14T11:54:17.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.168+0000 7fcf76595700 1 -- 172.21.15.104:0/2742602629 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcf540662f0 msgr2=0x7fcf540687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:17.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.168+0000 7fcf76595700 1 --2- 172.21.15.104:0/2742602629 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcf540662f0 0x7fcf540687a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009b90 tx=0x7fcf6c008070 comp rx=0 tx=0).stop 2023-07-14T11:54:17.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.168+0000 7fcf76595700 1 -- 172.21.15.104:0/2742602629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf70160450 msgr2=0x7fcf701e9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:17.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.168+0000 7fcf76595700 1 --2- 172.21.15.104:0/2742602629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf70160450 0x7fcf701e9ff0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800cb40 tx=0x7fcf6800cc20 comp rx=0 tx=0).stop 2023-07-14T11:54:17.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.168+0000 7fcf76595700 1 -- 172.21.15.104:0/2742602629 shutdown_connections 2023-07-14T11:54:17.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.168+0000 7fcf76595700 1 --2- 172.21.15.104:0/2742602629 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcf540662f0 0x7fcf540687a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.168+0000 7fcf76595700 1 --2- 172.21.15.104:0/2742602629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf70160450 0x7fcf701e9ff0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.168+0000 7fcf76595700 1 --2- 172.21.15.104:0/2742602629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf700c3810 0x7fcf701e9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.168+0000 7fcf76595700 1 --2- 172.21.15.104:0/2742602629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf700af270 0x7fcf7015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.168+0000 7fcf76595700 1 -- 172.21.15.104:0/2742602629 >> 172.21.15.104:0/2742602629 conn(0x7fcf700d0300 msgr2=0x7fcf700b58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:17.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.168+0000 7fcf76595700 1 -- 172.21.15.104:0/2742602629 shutdown_connections 2023-07-14T11:54:17.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.168+0000 7fcf76595700 1 -- 172.21.15.104:0/2742602629 wait complete. 2023-07-14T11:54:17.173 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-14T11:54:17.174 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-14T11:54:17.216 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-14T11:54:17.260 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-14T11:54:17.261 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:54:17.261 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:54:17.280 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:54:17.281 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:54:17.340 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:17.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.572+0000 7f5c51524700 1 Processor -- start 2023-07-14T11:54:17.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.572+0000 7f5c51524700 1 -- start start 2023-07-14T11:54:17.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.572+0000 7f5c51524700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c4c06dc20 0x7f5c4c0cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:17.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.572+0000 7f5c51524700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5c4c0c9820 0x7f5c4c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:17.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.572+0000 7f5c51524700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4c159d60 con 0x7f5c4c158fe0 2023-07-14T11:54:17.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.572+0000 7f5c51524700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4c0cd250 con 0x7f5c4c0c9820 2023-07-14T11:54:17.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.572+0000 7f5c51524700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0c8a60 con 0x7f5c4c06dc20 2023-07-14T11:54:17.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.572+0000 7f5c4bfff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c4c06dc20 0x7f5c4c0cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:17.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.572+0000 7f5c437fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5c4c0c9820 0x7f5c4c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:17.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.572+0000 7f5c437fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5c4c0c9820 0x7f5c4c0ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:53360/0 (socket says 172.21.15.104:53360) 2023-07-14T11:54:17.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.572+0000 7f5c437fe700 1 -- 172.21.15.104:0/3279232421 learned_addr learned my addr 172.21.15.104:0/3279232421 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:17.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c437fe700 1 -- 172.21.15.104:0/3279232421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c4c06dc20 msgr2=0x7f5c4c0cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:17.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c437fe700 1 --2- 172.21.15.104:0/3279232421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c4c06dc20 0x7f5c4c0cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c437fe700 1 -- 172.21.15.104:0/3279232421 >> v1:172.21.15.121:6789/0 conn(0x7f5c4c158fe0 legacy=0x7f5c4c159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:17.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c437fe700 1 -- 172.21.15.104:0/3279232421 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c0c8d40 con 0x7f5c4c0c9820 2023-07-14T11:54:17.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c437fe700 1 --2- 172.21.15.104:0/3279232421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c4c0c9820 0x7f5c4c0ce410 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009e80 tx=0x7f5c3400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8ed55e0679085de server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:17.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c4b7fe700 1 -- 172.21.15.104:0/3279232421 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400e070 con 0x7f5c4c0c9820 2023-07-14T11:54:17.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c4b7fe700 1 -- 172.21.15.104:0/3279232421 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3400b8d0 con 0x7f5c4c0c9820 2023-07-14T11:54:17.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c4b7fe700 1 -- 172.21.15.104:0/3279232421 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400bc80 con 0x7f5c4c0c9820 2023-07-14T11:54:17.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c51524700 1 -- 172.21.15.104:0/3279232421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c4c0c9820 msgr2=0x7f5c4c0ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:17.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c51524700 1 --2- 172.21.15.104:0/3279232421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c4c0c9820 0x7f5c4c0ce410 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009e80 tx=0x7f5c3400b330 comp rx=0 tx=0).stop 2023-07-14T11:54:17.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c51524700 1 -- 172.21.15.104:0/3279232421 shutdown_connections 2023-07-14T11:54:17.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c51524700 1 --2- 172.21.15.104:0/3279232421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c4c0c9820 0x7f5c4c0ce410 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c51524700 1 --2- 172.21.15.104:0/3279232421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c4c06dc20 0x7f5c4c0cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c51524700 1 -- 172.21.15.104:0/3279232421 >> 172.21.15.104:0/3279232421 conn(0x7f5c4c06e960 msgr2=0x7f5c4c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:17.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c51524700 1 -- 172.21.15.104:0/3279232421 shutdown_connections 2023-07-14T11:54:17.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.576+0000 7f5c51524700 1 -- 172.21.15.104:0/3279232421 wait complete. 2023-07-14T11:54:17.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.580+0000 7f5c51524700 1 Processor -- start 2023-07-14T11:54:17.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.580+0000 7f5c51524700 1 -- start start 2023-07-14T11:54:17.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.580+0000 7f5c51524700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c4c06dc20 0x7f5c4c1f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:17.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.580+0000 7f5c51524700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c4c0c9820 0x7f5c4c0acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:17.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.580+0000 7f5c51524700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c4c158fe0 0x7f5c4c15b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:17.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.580+0000 7f5c51524700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c154ea0 con 0x7f5c4c06dc20 2023-07-14T11:54:17.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.580+0000 7f5c51524700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0b0900 con 0x7f5c4c158fe0 2023-07-14T11:54:17.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.580+0000 7f5c51524700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0afcf0 con 0x7f5c4c0c9820 2023-07-14T11:54:17.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c4bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c4c06dc20 0x7f5c4c1f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:17.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c4bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c4c06dc20 0x7f5c4c1f3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35192/0 (socket says 172.21.15.104:35192) 2023-07-14T11:54:17.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c4bfff700 1 -- 172.21.15.104:0/1477520322 learned_addr learned my addr 172.21.15.104:0/1477520322 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:17.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c50d23700 1 --2- 172.21.15.104:0/1477520322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c4c158fe0 0x7f5c4c15b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:17.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c437fe700 1 --2- 172.21.15.104:0/1477520322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c4c0c9820 0x7f5c4c0acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:17.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c4bfff700 1 -- 172.21.15.104:0/1477520322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c4c0c9820 msgr2=0x7f5c4c0acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:17.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c4bfff700 1 --2- 172.21.15.104:0/1477520322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c4c0c9820 0x7f5c4c0acd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c4bfff700 1 -- 172.21.15.104:0/1477520322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c4c158fe0 msgr2=0x7f5c4c15b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:17.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c4bfff700 1 --2- 172.21.15.104:0/1477520322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c4c158fe0 0x7f5c4c15b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c4bfff700 1 -- 172.21.15.104:0/1477520322 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c34009b30 con 0x7f5c4c06dc20 2023-07-14T11:54:17.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c50d23700 1 --2- 172.21.15.104:0/1477520322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c4c158fe0 0x7f5c4c15b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:17.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c4bfff700 1 --2- 172.21.15.104:0/1477520322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c4c06dc20 0x7f5c4c1f3e30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00e9d0 tx=0x7f5c3c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:17.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c497fa700 1 -- 172.21.15.104:0/1477520322 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c009720 con 0x7f5c4c06dc20 2023-07-14T11:54:17.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c497fa700 1 -- 172.21.15.104:0/1477520322 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c00c850 con 0x7f5c4c06dc20 2023-07-14T11:54:17.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c497fa700 1 -- 172.21.15.104:0/1477520322 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c00cbc0 con 0x7f5c4c06dc20 2023-07-14T11:54:17.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c51524700 1 -- 172.21.15.104:0/1477520322 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4c0affd0 con 0x7f5c4c06dc20 2023-07-14T11:54:17.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c51524700 1 -- 172.21.15.104:0/1477520322 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4c0b0590 con 0x7f5c4c06dc20 2023-07-14T11:54:17.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.584+0000 7f5c51524700 1 -- 172.21.15.104:0/1477520322 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4c04e500 con 0x7f5c4c06dc20 2023-07-14T11:54:17.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.588+0000 7f5c497fa700 1 -- 172.21.15.104:0/1477520322 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5c3c00cd20 con 0x7f5c4c06dc20 2023-07-14T11:54:17.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.588+0000 7f5c497fa700 1 --2- 172.21.15.104:0/1477520322 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5c2c066320 0x7f5c2c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:17.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.588+0000 7f5c497fa700 1 -- 172.21.15.104:0/1477520322 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5c3c084140 con 0x7f5c4c06dc20 2023-07-14T11:54:17.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.588+0000 7f5c437fe700 1 --2- 172.21.15.104:0/1477520322 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5c2c066320 0x7f5c2c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:17.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.592+0000 7f5c437fe700 1 --2- 172.21.15.104:0/1477520322 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5c2c066320 0x7f5c2c0687d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0c9c10 tx=0x7f5c34009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:17.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.596+0000 7f5c497fa700 1 -- 172.21.15.104:0/1477520322 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f5c3c054d10 con 0x7f5c4c06dc20 2023-07-14T11:54:17.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.772+0000 7f5c51524700 1 -- 172.21.15.104:0/1477520322 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c4c0cc020 con 0x7f5c4c06dc20 2023-07-14T11:54:17.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.772+0000 7f5c497fa700 1 -- 172.21.15.104:0/1477520322 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20251 (secure 0 0 0) 0x7f5c3c0548a0 con 0x7f5c4c06dc20 2023-07-14T11:54:17.759 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:17.759 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":30,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:55.601783+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:17.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.780+0000 7f5c51524700 1 -- 172.21.15.104:0/1477520322 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5c2c066320 msgr2=0x7f5c2c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:17.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.780+0000 7f5c51524700 1 --2- 172.21.15.104:0/1477520322 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5c2c066320 0x7f5c2c0687d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0c9c10 tx=0x7f5c34009000 comp rx=0 tx=0).stop 2023-07-14T11:54:17.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.780+0000 7f5c51524700 1 -- 172.21.15.104:0/1477520322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c4c06dc20 msgr2=0x7f5c4c1f3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:17.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.780+0000 7f5c51524700 1 --2- 172.21.15.104:0/1477520322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c4c06dc20 0x7f5c4c1f3e30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00e9d0 tx=0x7f5c3c00ed90 comp rx=0 tx=0).stop 2023-07-14T11:54:17.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.780+0000 7f5c51524700 1 -- 172.21.15.104:0/1477520322 shutdown_connections 2023-07-14T11:54:17.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.780+0000 7f5c51524700 1 --2- 172.21.15.104:0/1477520322 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5c2c066320 0x7f5c2c0687d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.780+0000 7f5c51524700 1 --2- 172.21.15.104:0/1477520322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c4c158fe0 0x7f5c4c15b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.780+0000 7f5c51524700 1 --2- 172.21.15.104:0/1477520322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c4c0c9820 0x7f5c4c0acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.780+0000 7f5c51524700 1 --2- 172.21.15.104:0/1477520322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c4c06dc20 0x7f5c4c1f3e30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.780+0000 7f5c51524700 1 -- 172.21.15.104:0/1477520322 >> 172.21.15.104:0/1477520322 conn(0x7f5c4c06e960 msgr2=0x7f5c4c0709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:17.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.780+0000 7f5c51524700 1 -- 172.21.15.104:0/1477520322 shutdown_connections 2023-07-14T11:54:17.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.780+0000 7f5c51524700 1 -- 172.21.15.104:0/1477520322 wait complete. 2023-07-14T11:54:17.785 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:54:17.786 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:54:17.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.972+0000 7fddbf59e700 1 Processor -- start 2023-07-14T11:54:17.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.972+0000 7fddbf59e700 1 -- start start 2023-07-14T11:54:17.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbf59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fddc01597a0 0x7fddc00c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:17.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbf59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddc0159b70 0x7fddc00705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:17.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbf59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fddc00cada0 0x7fddc00cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:17.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbf59e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc0071560 con 0x7fddc01597a0 2023-07-14T11:54:17.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbf59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc0070cc0 con 0x7fddc00cada0 2023-07-14T11:54:17.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbf59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc0070fa0 con 0x7fddc0159b70 2023-07-14T11:54:17.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbe59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fddc01597a0 0x7fddc00c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:17.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbe59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fddc01597a0 0x7fddc00c87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35206/0 (socket says 172.21.15.104:35206) 2023-07-14T11:54:17.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbe59c700 1 -- 172.21.15.104:0/3921907738 learned_addr learned my addr 172.21.15.104:0/3921907738 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:17.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddb5d9b700 1 --2- 172.21.15.104:0/3921907738 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddc0159b70 0x7fddc00705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:17.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbed9d700 1 --2- 172.21.15.104:0/3921907738 >> v2:172.21.15.121:3300/0 conn(0x7fddc00cada0 0x7fddc00cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:17.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbe59c700 1 -- 172.21.15.104:0/3921907738 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddc0159b70 msgr2=0x7fddc00705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:17.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbe59c700 1 --2- 172.21.15.104:0/3921907738 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddc0159b70 0x7fddc00705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbe59c700 1 -- 172.21.15.104:0/3921907738 >> v2:172.21.15.121:3300/0 conn(0x7fddc00cada0 msgr2=0x7fddc00cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:17.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbe59c700 1 --2- 172.21.15.104:0/3921907738 >> v2:172.21.15.121:3300/0 conn(0x7fddc00cada0 0x7fddc00cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbe59c700 1 -- 172.21.15.104:0/3921907738 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc0071280 con 0x7fddc01597a0 2023-07-14T11:54:17.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbe59c700 1 --2- 172.21.15.104:0/3921907738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddc01597a0 0x7fddc00c87a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fddc00cf220 tx=0x7fddb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fd24f9af1b04b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:17.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbdd9b700 1 -- 172.21.15.104:0/3921907738 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb000e070 con 0x7fddc01597a0 2023-07-14T11:54:17.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbdd9b700 1 -- 172.21.15.104:0/3921907738 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb000b9c0 con 0x7fddc01597a0 2023-07-14T11:54:17.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.976+0000 7fddbdd9b700 1 -- 172.21.15.104:0/3921907738 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb000bd70 con 0x7fddc01597a0 2023-07-14T11:54:17.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.980+0000 7fddbf59e700 1 -- 172.21.15.104:0/3921907738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddc01597a0 msgr2=0x7fddc00c87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:17.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.980+0000 7fddbf59e700 1 --2- 172.21.15.104:0/3921907738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddc01597a0 0x7fddc00c87a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fddc00cf220 tx=0x7fddb000b330 comp rx=0 tx=0).stop 2023-07-14T11:54:17.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.980+0000 7fddbf59e700 1 -- 172.21.15.104:0/3921907738 shutdown_connections 2023-07-14T11:54:17.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.980+0000 7fddbf59e700 1 --2- 172.21.15.104:0/3921907738 >> v2:172.21.15.121:3300/0 conn(0x7fddc00cada0 0x7fddc00cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.980+0000 7fddbf59e700 1 --2- 172.21.15.104:0/3921907738 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddc0159b70 0x7fddc00705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.980+0000 7fddbf59e700 1 --2- 172.21.15.104:0/3921907738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddc01597a0 0x7fddc00c87a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.980+0000 7fddbf59e700 1 -- 172.21.15.104:0/3921907738 >> 172.21.15.104:0/3921907738 conn(0x7fddc00afc60 msgr2=0x7fddc00b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:17.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.980+0000 7fddbf59e700 1 -- 172.21.15.104:0/3921907738 shutdown_connections 2023-07-14T11:54:17.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.980+0000 7fddbf59e700 1 -- 172.21.15.104:0/3921907738 wait complete. 2023-07-14T11:54:17.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbf59e700 1 Processor -- start 2023-07-14T11:54:17.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbf59e700 1 -- start start 2023-07-14T11:54:17.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbf59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddc00cada0 0x7fddc0155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:17.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbf59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddc01597a0 0x7fddc015ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:17.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbf59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddc0159b70 0x7fddc0156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:17.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbf59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc0155dc0 con 0x7fddc0159b70 2023-07-14T11:54:17.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbf59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc00aea10 con 0x7fddc01597a0 2023-07-14T11:54:17.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbf59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc00adc50 con 0x7fddc00cada0 2023-07-14T11:54:17.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbed9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddc0159b70 0x7fddc0156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:17.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbed9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddc0159b70 0x7fddc0156450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35216/0 (socket says 172.21.15.104:35216) 2023-07-14T11:54:17.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbed9d700 1 -- 172.21.15.104:0/3080846053 learned_addr learned my addr 172.21.15.104:0/3080846053 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:17.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddb5d9b700 1 --2- 172.21.15.104:0/3080846053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddc01597a0 0x7fddc015ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:17.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbe59c700 1 --2- 172.21.15.104:0/3080846053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddc00cada0 0x7fddc0155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:17.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbed9d700 1 -- 172.21.15.104:0/3080846053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddc00cada0 msgr2=0x7fddc0155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:17.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbed9d700 1 --2- 172.21.15.104:0/3080846053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddc00cada0 0x7fddc0155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbed9d700 1 -- 172.21.15.104:0/3080846053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddc01597a0 msgr2=0x7fddc015ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:17.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbed9d700 1 --2- 172.21.15.104:0/3080846053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddc01597a0 0x7fddc015ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:17.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbed9d700 1 -- 172.21.15.104:0/3080846053 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb0009b30 con 0x7fddc0159b70 2023-07-14T11:54:17.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddbed9d700 1 --2- 172.21.15.104:0/3080846053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddc0159b70 0x7fddc0156450 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fddb800e810 tx=0x7fddb800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:17.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddb77fe700 1 -- 172.21.15.104:0/3080846053 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8013610 con 0x7fddc0159b70 2023-07-14T11:54:17.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.984+0000 7fddb77fe700 1 -- 172.21.15.104:0/3080846053 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb8013790 con 0x7fddc0159b70 2023-07-14T11:54:17.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.988+0000 7fddbf59e700 1 -- 172.21.15.104:0/3080846053 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc00adf30 con 0x7fddc0159b70 2023-07-14T11:54:17.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.988+0000 7fddb77fe700 1 -- 172.21.15.104:0/3080846053 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8013b40 con 0x7fddc0159b70 2023-07-14T11:54:17.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.988+0000 7fddbf59e700 1 -- 172.21.15.104:0/3080846053 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc00ae4f0 con 0x7fddc0159b70 2023-07-14T11:54:17.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.988+0000 7fddbf59e700 1 -- 172.21.15.104:0/3080846053 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc004e500 con 0x7fddc0159b70 2023-07-14T11:54:17.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.988+0000 7fddb77fe700 1 -- 172.21.15.104:0/3080846053 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fddb8013ca0 con 0x7fddc0159b70 2023-07-14T11:54:17.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.992+0000 7fddb77fe700 1 --2- 172.21.15.104:0/3080846053 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdda00662f0 0x7fdda00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:17.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.992+0000 7fddb77fe700 1 -- 172.21.15.104:0/3080846053 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fddb8083c10 con 0x7fddc0159b70 2023-07-14T11:54:17.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.992+0000 7fddbe59c700 1 --2- 172.21.15.104:0/3080846053 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdda00662f0 0x7fdda00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:17.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.992+0000 7fddbe59c700 1 --2- 172.21.15.104:0/3080846053 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdda00662f0 0x7fdda00687a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fddb000b370 tx=0x7fddb0009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:17.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:17.996+0000 7fddb77fe700 1 -- 172.21.15.104:0/3080846053 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fddb8050d60 con 0x7fddc0159b70 2023-07-14T11:54:18.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.164+0000 7fddbf59e700 1 -- 172.21.15.104:0/3080846053 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fddc006dcf0 con 0x7fddc0159b70 2023-07-14T11:54:18.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.164+0000 7fddb77fe700 1 -- 172.21.15.104:0/3080846053 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fddb80210a0 con 0x7fddc0159b70 2023-07-14T11:54:18.150 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:54:18.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.168+0000 7fddbf59e700 1 -- 172.21.15.104:0/3080846053 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdda00662f0 msgr2=0x7fdda00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.168+0000 7fddbf59e700 1 --2- 172.21.15.104:0/3080846053 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdda00662f0 0x7fdda00687a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fddb000b370 tx=0x7fddb0009000 comp rx=0 tx=0).stop 2023-07-14T11:54:18.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.168+0000 7fddbf59e700 1 -- 172.21.15.104:0/3080846053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddc0159b70 msgr2=0x7fddc0156450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.168+0000 7fddbf59e700 1 --2- 172.21.15.104:0/3080846053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddc0159b70 0x7fddc0156450 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fddb800e810 tx=0x7fddb800ebd0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.168+0000 7fddbf59e700 1 -- 172.21.15.104:0/3080846053 shutdown_connections 2023-07-14T11:54:18.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.168+0000 7fddbf59e700 1 --2- 172.21.15.104:0/3080846053 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdda00662f0 0x7fdda00687a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.168+0000 7fddbf59e700 1 --2- 172.21.15.104:0/3080846053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddc0159b70 0x7fddc0156450 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.168+0000 7fddbf59e700 1 --2- 172.21.15.104:0/3080846053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddc01597a0 0x7fddc015ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.172+0000 7fddbf59e700 1 --2- 172.21.15.104:0/3080846053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddc00cada0 0x7fddc0155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.172+0000 7fddbf59e700 1 -- 172.21.15.104:0/3080846053 >> 172.21.15.104:0/3080846053 conn(0x7fddc00afc60 msgr2=0x7fddc00c9950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:18.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.172+0000 7fddbf59e700 1 -- 172.21.15.104:0/3080846053 shutdown_connections 2023-07-14T11:54:18.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.172+0000 7fddbf59e700 1 -- 172.21.15.104:0/3080846053 wait complete. 2023-07-14T11:54:18.174 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:54:18.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.368+0000 7f0e2db6b700 1 Processor -- start 2023-07-14T11:54:18.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.368+0000 7f0e2db6b700 1 -- start start 2023-07-14T11:54:18.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e2db6b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0e280b0ee0 0x7f0e280ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:18.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e2db6b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e280d07e0 0x7f0e280af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:18.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e2db6b700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e280cd4e0 con 0x7f0e280cd090 2023-07-14T11:54:18.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e2db6b700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e280d0c60 con 0x7f0e280b0ee0 2023-07-14T11:54:18.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e2db6b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e280d1a60 con 0x7f0e280d07e0 2023-07-14T11:54:18.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e27fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f0e280cd090 0x7f0e280d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:44874/0 (socket says 172.21.15.104:44874) 2023-07-14T11:54:18.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e27fff700 1 -- 172.21.15.104:0/3096263920 learned_addr learned my addr 172.21.15.104:0/3096263920 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:18.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e2cb69700 1 --2- 172.21.15.104:0/3096263920 >> v2:172.21.15.104:3300/0 conn(0x7f0e280b0ee0 0x7f0e280ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:18.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e2d36a700 1 --2- 172.21.15.104:0/3096263920 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e280d07e0 0x7f0e280af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:18.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e277fe700 1 -- 172.21.15.104:0/3096263920 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498515696 0 0) 0x7f0e14001110 con 0x7f0e280cd090 2023-07-14T11:54:18.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e277fe700 1 -- 172.21.15.104:0/3096263920 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e180035d0 con 0x7f0e280cd090 2023-07-14T11:54:18.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e2cb69700 1 -- 172.21.15.104:0/3096263920 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e280d07e0 msgr2=0x7f0e280af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:18.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e2cb69700 1 --2- 172.21.15.104:0/3096263920 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e280d07e0 0x7f0e280af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e2cb69700 1 -- 172.21.15.104:0/3096263920 >> v1:172.21.15.104:6789/0 conn(0x7f0e280cd090 legacy=0x7f0e280d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e2cb69700 1 -- 172.21.15.104:0/3096263920 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e280d1ef0 con 0x7f0e280b0ee0 2023-07-14T11:54:18.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e2cb69700 1 --2- 172.21.15.104:0/3096263920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e280b0ee0 0x7f0e280ccb50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009850 tx=0x7f0e1c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ad095887cf4da4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:18.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e277fe700 1 -- 172.21.15.104:0/3096263920 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c011830 con 0x7f0e280b0ee0 2023-07-14T11:54:18.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e277fe700 1 -- 172.21.15.104:0/3096263920 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c011990 con 0x7f0e280b0ee0 2023-07-14T11:54:18.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.372+0000 7f0e277fe700 1 -- 172.21.15.104:0/3096263920 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c011d40 con 0x7f0e280b0ee0 2023-07-14T11:54:18.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.376+0000 7f0e2db6b700 1 -- 172.21.15.104:0/3096263920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e280b0ee0 msgr2=0x7f0e280ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.376+0000 7f0e2db6b700 1 --2- 172.21.15.104:0/3096263920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e280b0ee0 0x7f0e280ccb50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009850 tx=0x7f0e1c009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:18.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.376+0000 7f0e2db6b700 1 -- 172.21.15.104:0/3096263920 shutdown_connections 2023-07-14T11:54:18.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.376+0000 7f0e2db6b700 1 --2- 172.21.15.104:0/3096263920 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e280d07e0 0x7f0e280af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.376+0000 7f0e2db6b700 1 --2- 172.21.15.104:0/3096263920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e280b0ee0 0x7f0e280ccb50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.376+0000 7f0e2db6b700 1 -- 172.21.15.104:0/3096263920 >> 172.21.15.104:0/3096263920 conn(0x7f0e28060c40 msgr2=0x7f0e2809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:18.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.376+0000 7f0e2db6b700 1 -- 172.21.15.104:0/3096263920 shutdown_connections 2023-07-14T11:54:18.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.376+0000 7f0e2db6b700 1 -- 172.21.15.104:0/3096263920 wait complete. 2023-07-14T11:54:18.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.376+0000 7f0e2db6b700 1 Processor -- start 2023-07-14T11:54:18.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.376+0000 7f0e2db6b700 1 -- start start 2023-07-14T11:54:18.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2db6b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e280b0ee0 0x7f0e280cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:18.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2db6b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e280cd090 0x7f0e2815adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:18.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2db6b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e280d07e0 0x7f0e280c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:18.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2db6b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e281f5530 con 0x7f0e280d07e0 2023-07-14T11:54:18.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2db6b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2815b2f0 con 0x7f0e280cd090 2023-07-14T11:54:18.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2db6b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e281f48b0 con 0x7f0e280b0ee0 2023-07-14T11:54:18.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2d36a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e280d07e0 0x7f0e280c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:18.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2d36a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e280d07e0 0x7f0e280c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35226/0 (socket says 172.21.15.104:35226) 2023-07-14T11:54:18.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2d36a700 1 -- 172.21.15.104:0/1680988851 learned_addr learned my addr 172.21.15.104:0/1680988851 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:18.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2cb69700 1 --2- 172.21.15.104:0/1680988851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e280b0ee0 0x7f0e280cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:18.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e27fff700 1 --2- 172.21.15.104:0/1680988851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e280cd090 0x7f0e2815adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:18.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2d36a700 1 -- 172.21.15.104:0/1680988851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e280b0ee0 msgr2=0x7f0e280cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2d36a700 1 --2- 172.21.15.104:0/1680988851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e280b0ee0 0x7f0e280cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2d36a700 1 -- 172.21.15.104:0/1680988851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e280cd090 msgr2=0x7f0e2815adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2d36a700 1 --2- 172.21.15.104:0/1680988851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e280cd090 0x7f0e2815adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2d36a700 1 -- 172.21.15.104:0/1680988851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1c0094b0 con 0x7f0e280d07e0 2023-07-14T11:54:18.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e27fff700 1 --2- 172.21.15.104:0/1680988851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e280cd090 0x7f0e2815adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:18.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2d36a700 1 --2- 172.21.15.104:0/1680988851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e280d07e0 0x7f0e280c95d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009dd0 tx=0x7f0e2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:18.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e257fa700 1 -- 172.21.15.104:0/1680988851 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20009c40 con 0x7f0e280d07e0 2023-07-14T11:54:18.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e257fa700 1 -- 172.21.15.104:0/1680988851 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2000f040 con 0x7f0e280d07e0 2023-07-14T11:54:18.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e257fa700 1 -- 172.21.15.104:0/1680988851 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20014480 con 0x7f0e280d07e0 2023-07-14T11:54:18.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2db6b700 1 -- 172.21.15.104:0/1680988851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e281f4b90 con 0x7f0e280d07e0 2023-07-14T11:54:18.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.380+0000 7f0e2db6b700 1 -- 172.21.15.104:0/1680988851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e281f5150 con 0x7f0e280d07e0 2023-07-14T11:54:18.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.384+0000 7f0e2db6b700 1 -- 172.21.15.104:0/1680988851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e281f5670 con 0x7f0e280d07e0 2023-07-14T11:54:18.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.384+0000 7f0e257fa700 1 -- 172.21.15.104:0/1680988851 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f0e200145e0 con 0x7f0e280d07e0 2023-07-14T11:54:18.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.384+0000 7f0e257fa700 1 --2- 172.21.15.104:0/1680988851 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0e0c0662f0 0x7f0e0c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:18.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.384+0000 7f0e257fa700 1 -- 172.21.15.104:0/1680988851 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0e20084970 con 0x7f0e280d07e0 2023-07-14T11:54:18.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.388+0000 7f0e2cb69700 1 --2- 172.21.15.104:0/1680988851 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0e0c0662f0 0x7f0e0c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:18.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.388+0000 7f0e2cb69700 1 --2- 172.21.15.104:0/1680988851 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0e0c0662f0 0x7f0e0c0687a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0090b0 tx=0x7f0e1c01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:18.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.388+0000 7f0e257fa700 1 -- 172.21.15.104:0/1680988851 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f0e20051a20 con 0x7f0e280d07e0 2023-07-14T11:54:18.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.556+0000 7f0e2db6b700 1 -- 172.21.15.104:0/1680988851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0e2804e500 con 0x7f0e280d07e0 2023-07-14T11:54:18.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.556+0000 7f0e257fa700 1 -- 172.21.15.104:0/1680988851 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0e20055040 con 0x7f0e280d07e0 2023-07-14T11:54:18.540 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:54:18.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.560+0000 7f0e2db6b700 1 -- 172.21.15.104:0/1680988851 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0e0c0662f0 msgr2=0x7f0e0c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.560+0000 7f0e2db6b700 1 --2- 172.21.15.104:0/1680988851 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0e0c0662f0 0x7f0e0c0687a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0090b0 tx=0x7f0e1c01c040 comp rx=0 tx=0).stop 2023-07-14T11:54:18.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.560+0000 7f0e2db6b700 1 -- 172.21.15.104:0/1680988851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e280d07e0 msgr2=0x7f0e280c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.560+0000 7f0e2db6b700 1 --2- 172.21.15.104:0/1680988851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e280d07e0 0x7f0e280c95d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009dd0 tx=0x7f0e2000c3a0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.560+0000 7f0e2db6b700 1 -- 172.21.15.104:0/1680988851 shutdown_connections 2023-07-14T11:54:18.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.560+0000 7f0e2db6b700 1 --2- 172.21.15.104:0/1680988851 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0e0c0662f0 0x7f0e0c0687a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.560+0000 7f0e2db6b700 1 --2- 172.21.15.104:0/1680988851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e280d07e0 0x7f0e280c95d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.560+0000 7f0e2db6b700 1 --2- 172.21.15.104:0/1680988851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e280cd090 0x7f0e2815adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.560+0000 7f0e2db6b700 1 --2- 172.21.15.104:0/1680988851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e280b0ee0 0x7f0e280cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.560+0000 7f0e2db6b700 1 -- 172.21.15.104:0/1680988851 >> 172.21.15.104:0/1680988851 conn(0x7f0e28060c40 msgr2=0x7f0e2806fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:18.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.560+0000 7f0e2db6b700 1 -- 172.21.15.104:0/1680988851 shutdown_connections 2023-07-14T11:54:18.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.560+0000 7f0e2db6b700 1 -- 172.21.15.104:0/1680988851 wait complete. 2023-07-14T11:54:18.566 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:54:18.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f2248900700 1 Processor -- start 2023-07-14T11:54:18.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f2248900700 1 -- start start 2023-07-14T11:54:18.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f2248900700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f224406a5c0 0x7f22440ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:18.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f2248900700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22440cebe0 con 0x7f22440ce7e0 2023-07-14T11:54:18.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f2248900700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f224415c030 con 0x7f224406e960 2023-07-14T11:54:18.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f2248900700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22440cee20 con 0x7f224406a5c0 2023-07-14T11:54:18.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f224259c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f224406a5c0 0x7f22440ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:18.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f224259c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f224406a5c0 0x7f22440ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35234/0 (socket says 172.21.15.104:35234) 2023-07-14T11:54:18.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f224259c700 1 -- 172.21.15.104:0/3168170352 learned_addr learned my addr 172.21.15.104:0/3168170352 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:18.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f224359e700 1 --1- 172.21.15.104:0/3168170352 >> v1:172.21.15.104:6789/0 conn(0x7f22440ce7e0 0x7f22440cb7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:44882/0 (socket says 172.21.15.104:44882) 2023-07-14T11:54:18.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f224259c700 1 -- 172.21.15.104:0/3168170352 >> v1:172.21.15.121:6789/0 conn(0x7f224406e960 legacy=0x7f224406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f224259c700 1 -- 172.21.15.104:0/3168170352 >> v1:172.21.15.104:6789/0 conn(0x7f22440ce7e0 legacy=0x7f22440cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f224259c700 1 -- 172.21.15.104:0/3168170352 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2244069800 con 0x7f224406a5c0 2023-07-14T11:54:18.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f224259c700 1 --2- 172.21.15.104:0/3168170352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f224406a5c0 0x7f22440ce2a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2238009910 tx=0x7f2238009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2057888824618e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:18.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f2241d9b700 1 -- 172.21.15.104:0/3168170352 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223800ba60 con 0x7f224406a5c0 2023-07-14T11:54:18.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f2241d9b700 1 -- 172.21.15.104:0/3168170352 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223800bbc0 con 0x7f224406a5c0 2023-07-14T11:54:18.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.764+0000 7f2241d9b700 1 -- 172.21.15.104:0/3168170352 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238009e60 con 0x7f224406a5c0 2023-07-14T11:54:18.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.768+0000 7f2248900700 1 -- 172.21.15.104:0/3168170352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f224406a5c0 msgr2=0x7f22440ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.768+0000 7f2248900700 1 --2- 172.21.15.104:0/3168170352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f224406a5c0 0x7f22440ce2a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2238009910 tx=0x7f2238009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:18.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.768+0000 7f2248900700 1 -- 172.21.15.104:0/3168170352 shutdown_connections 2023-07-14T11:54:18.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.768+0000 7f2248900700 1 --2- 172.21.15.104:0/3168170352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f224406a5c0 0x7f22440ce2a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.768+0000 7f2248900700 1 -- 172.21.15.104:0/3168170352 >> 172.21.15.104:0/3168170352 conn(0x7f224406fe60 msgr2=0x7f2244070260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:18.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.768+0000 7f2248900700 1 -- 172.21.15.104:0/3168170352 shutdown_connections 2023-07-14T11:54:18.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.768+0000 7f2248900700 1 -- 172.21.15.104:0/3168170352 wait complete. 2023-07-14T11:54:18.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.768+0000 7f2248900700 1 Processor -- start 2023-07-14T11:54:18.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.768+0000 7f2248900700 1 -- start start 2023-07-14T11:54:18.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2248900700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f224406a5c0 0x7f22441596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:18.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2248900700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f224406e960 0x7f22440aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:18.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2248900700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22440ce7e0 0x7f22441e6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:18.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2248900700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22440c4210 con 0x7f224406a5c0 2023-07-14T11:54:18.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2248900700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22440c44f0 con 0x7f22440ce7e0 2023-07-14T11:54:18.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2248900700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22440c47d0 con 0x7f224406e960 2023-07-14T11:54:18.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2242d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f224406a5c0 0x7f22441596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:18.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2242d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f224406a5c0 0x7f22441596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35248/0 (socket says 172.21.15.104:35248) 2023-07-14T11:54:18.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2242d9d700 1 -- 172.21.15.104:0/3273727087 learned_addr learned my addr 172.21.15.104:0/3273727087 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:18.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f224259c700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f224406e960 0x7f22440aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:18.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f224359e700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22440ce7e0 0x7f22441e6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:18.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2242d9d700 1 -- 172.21.15.104:0/3273727087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f224406e960 msgr2=0x7f22440aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2242d9d700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f224406e960 0x7f22440aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2242d9d700 1 -- 172.21.15.104:0/3273727087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22440ce7e0 msgr2=0x7f22441e6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2242d9d700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22440ce7e0 0x7f22441e6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2242d9d700 1 -- 172.21.15.104:0/3273727087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2238009570 con 0x7f224406a5c0 2023-07-14T11:54:18.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f224259c700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f224406e960 0x7f22440aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:18.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f224359e700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22440ce7e0 0x7f22441e6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:18.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2242d9d700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f224406a5c0 0x7f22441596e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f223400c500 tx=0x7f223400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:18.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f222b7fe700 1 -- 172.21.15.104:0/3273727087 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22340052d0 con 0x7f224406a5c0 2023-07-14T11:54:18.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f222b7fe700 1 -- 172.21.15.104:0/3273727087 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2234005430 con 0x7f224406a5c0 2023-07-14T11:54:18.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2248900700 1 -- 172.21.15.104:0/3273727087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22440c4ab0 con 0x7f224406a5c0 2023-07-14T11:54:18.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f222b7fe700 1 -- 172.21.15.104:0/3273727087 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22340055a0 con 0x7f224406a5c0 2023-07-14T11:54:18.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.772+0000 7f2248900700 1 -- 172.21.15.104:0/3273727087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22441e72c0 con 0x7f224406a5c0 2023-07-14T11:54:18.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.776+0000 7f2248900700 1 -- 172.21.15.104:0/3273727087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2244159f50 con 0x7f224406a5c0 2023-07-14T11:54:18.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.776+0000 7f222b7fe700 1 -- 172.21.15.104:0/3273727087 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f22340038e0 con 0x7f224406a5c0 2023-07-14T11:54:18.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.776+0000 7f222b7fe700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f22300662f0 0x7f22300687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:18.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.776+0000 7f222b7fe700 1 -- 172.21.15.104:0/3273727087 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2234083550 con 0x7f224406a5c0 2023-07-14T11:54:18.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.776+0000 7f224259c700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f22300662f0 0x7f22300687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:18.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.780+0000 7f224259c700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f22300662f0 0x7f22300687a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2238006f80 tx=0x7f2238008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:18.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.784+0000 7f222b7fe700 1 -- 172.21.15.104:0/3273727087 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f2234050600 con 0x7f224406a5c0 2023-07-14T11:54:18.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.956+0000 7f2248900700 1 -- 172.21.15.104:0/3273727087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f224406b090 con 0x7f224406a5c0 2023-07-14T11:54:18.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.956+0000 7f222b7fe700 1 -- 172.21.15.104:0/3273727087 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2234053c20 con 0x7f224406a5c0 2023-07-14T11:54:18.941 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:18.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.960+0000 7f2248900700 1 -- 172.21.15.104:0/3273727087 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f22300662f0 msgr2=0x7f22300687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.960+0000 7f2248900700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f22300662f0 0x7f22300687a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2238006f80 tx=0x7f2238008030 comp rx=0 tx=0).stop 2023-07-14T11:54:18.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.960+0000 7f2248900700 1 -- 172.21.15.104:0/3273727087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f224406a5c0 msgr2=0x7f22441596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.960+0000 7f2248900700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f224406a5c0 0x7f22441596e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f223400c500 tx=0x7f223400c8c0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.960+0000 7f2248900700 1 -- 172.21.15.104:0/3273727087 shutdown_connections 2023-07-14T11:54:18.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.960+0000 7f2248900700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f22300662f0 0x7f22300687a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.960+0000 7f2248900700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22440ce7e0 0x7f22441e6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.960+0000 7f2248900700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f224406e960 0x7f22440aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.960+0000 7f2248900700 1 --2- 172.21.15.104:0/3273727087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f224406a5c0 0x7f22441596e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.960+0000 7f2248900700 1 -- 172.21.15.104:0/3273727087 >> 172.21.15.104:0/3273727087 conn(0x7f224406fe60 msgr2=0x7f22440cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:18.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.960+0000 7f2248900700 1 -- 172.21.15.104:0/3273727087 shutdown_connections 2023-07-14T11:54:18.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:18.960+0000 7f2248900700 1 -- 172.21.15.104:0/3273727087 wait complete. 2023-07-14T11:54:18.963 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:54:19.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.156+0000 7f221ec20700 1 Processor -- start 2023-07-14T11:54:19.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.156+0000 7f221ec20700 1 -- start start 2023-07-14T11:54:19.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221ec20700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22180b0ee0 0x7f22180ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221ec20700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f22180cd090 0x7f22180d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221ec20700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f22180d0610 0x7f221815f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221ec20700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22180cd4e0 con 0x7f22180cd090 2023-07-14T11:54:19.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221ec20700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22180d1a60 con 0x7f22180d0610 2023-07-14T11:54:19.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221ec20700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22180d1ef0 con 0x7f22180b0ee0 2023-07-14T11:54:19.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221d41d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f22180cd090 0x7f22180d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221d41d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f22180cd090 0x7f22180d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35254/0 (socket says 172.21.15.104:35254) 2023-07-14T11:54:19.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221d41d700 1 -- 172.21.15.104:0/3176820622 learned_addr learned my addr 172.21.15.104:0/3176820622 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:19.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221e41f700 1 --2- 172.21.15.104:0/3176820622 >> v2:172.21.15.121:3300/0 conn(0x7f22180d0610 0x7f221815f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221dc1e700 1 --2- 172.21.15.104:0/3176820622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22180b0ee0 0x7f22180ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221d41d700 1 -- 172.21.15.104:0/3176820622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22180b0ee0 msgr2=0x7f22180ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:19.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221d41d700 1 --2- 172.21.15.104:0/3176820622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22180b0ee0 0x7f22180ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221d41d700 1 -- 172.21.15.104:0/3176820622 >> v2:172.21.15.121:3300/0 conn(0x7f22180d0610 msgr2=0x7f221815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:19.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221d41d700 1 --2- 172.21.15.104:0/3176820622 >> v2:172.21.15.121:3300/0 conn(0x7f22180d0610 0x7f221815f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221d41d700 1 -- 172.21.15.104:0/3176820622 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22180d0e50 con 0x7f22180cd090 2023-07-14T11:54:19.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221dc1e700 1 --2- 172.21.15.104:0/3176820622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22180b0ee0 0x7f22180ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:19.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.160+0000 7f221d41d700 1 --2- 172.21.15.104:0/3176820622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22180cd090 0x7f22180d00d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2208009910 tx=0x7f2208009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86114fbb9cd053c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:19.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.164+0000 7f221cc1c700 1 -- 172.21.15.104:0/3176820622 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220800ba20 con 0x7f22180cd090 2023-07-14T11:54:19.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.164+0000 7f221cc1c700 1 -- 172.21.15.104:0/3176820622 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220800bb80 con 0x7f22180cd090 2023-07-14T11:54:19.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.164+0000 7f221cc1c700 1 -- 172.21.15.104:0/3176820622 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208009e60 con 0x7f22180cd090 2023-07-14T11:54:19.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.164+0000 7f221ec20700 1 -- 172.21.15.104:0/3176820622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22180cd090 msgr2=0x7f22180d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:19.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.164+0000 7f221ec20700 1 --2- 172.21.15.104:0/3176820622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22180cd090 0x7f22180d00d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2208009910 tx=0x7f2208009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:19.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.164+0000 7f221ec20700 1 -- 172.21.15.104:0/3176820622 shutdown_connections 2023-07-14T11:54:19.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.164+0000 7f221ec20700 1 --2- 172.21.15.104:0/3176820622 >> v2:172.21.15.121:3300/0 conn(0x7f22180d0610 0x7f221815f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.164+0000 7f221ec20700 1 --2- 172.21.15.104:0/3176820622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22180cd090 0x7f22180d00d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.164+0000 7f221ec20700 1 --2- 172.21.15.104:0/3176820622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22180b0ee0 0x7f22180ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.164+0000 7f221ec20700 1 -- 172.21.15.104:0/3176820622 >> 172.21.15.104:0/3176820622 conn(0x7f2218060c40 msgr2=0x7f221809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:19.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.164+0000 7f221ec20700 1 -- 172.21.15.104:0/3176820622 shutdown_connections 2023-07-14T11:54:19.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.164+0000 7f221ec20700 1 -- 172.21.15.104:0/3176820622 wait complete. 2023-07-14T11:54:19.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.164+0000 7f221ec20700 1 Processor -- start 2023-07-14T11:54:19.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.164+0000 7f221ec20700 1 -- start start 2023-07-14T11:54:19.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221ec20700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22180b0ee0 0x7f22181f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221ec20700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22180cd090 0x7f221815f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221ec20700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22180d0610 0x7f221815fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221ec20700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22181f54f0 con 0x7f22180b0ee0 2023-07-14T11:54:19.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221ec20700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22180b81f0 con 0x7f22180d0610 2023-07-14T11:54:19.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221ec20700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22181f4bc0 con 0x7f22180cd090 2023-07-14T11:54:19.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221dc1e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22180b0ee0 0x7f22181f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221dc1e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22180b0ee0 0x7f22181f0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35256/0 (socket says 172.21.15.104:35256) 2023-07-14T11:54:19.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221dc1e700 1 -- 172.21.15.104:0/730886760 learned_addr learned my addr 172.21.15.104:0/730886760 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:19.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221d41d700 1 --2- 172.21.15.104:0/730886760 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22180cd090 0x7f221815f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221e41f700 1 --2- 172.21.15.104:0/730886760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22180d0610 0x7f221815fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221dc1e700 1 -- 172.21.15.104:0/730886760 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22180cd090 msgr2=0x7f221815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:19.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221dc1e700 1 --2- 172.21.15.104:0/730886760 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22180cd090 0x7f221815f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221dc1e700 1 -- 172.21.15.104:0/730886760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22180d0610 msgr2=0x7f221815fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:19.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221dc1e700 1 --2- 172.21.15.104:0/730886760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22180d0610 0x7f221815fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221dc1e700 1 -- 172.21.15.104:0/730886760 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2208009570 con 0x7f22180b0ee0 2023-07-14T11:54:19.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221dc1e700 1 --2- 172.21.15.104:0/730886760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22180b0ee0 0x7f22181f0d00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f221400c7f0 tx=0x7f221400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:19.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f220e7fc700 1 -- 172.21.15.104:0/730886760 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2214011800 con 0x7f22180b0ee0 2023-07-14T11:54:19.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f220e7fc700 1 -- 172.21.15.104:0/730886760 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2214011960 con 0x7f22180b0ee0 2023-07-14T11:54:19.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221ec20700 1 -- 172.21.15.104:0/730886760 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22181f4ea0 con 0x7f22180b0ee0 2023-07-14T11:54:19.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f220e7fc700 1 -- 172.21.15.104:0/730886760 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2214011d10 con 0x7f22180b0ee0 2023-07-14T11:54:19.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.168+0000 7f221ec20700 1 -- 172.21.15.104:0/730886760 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22181f5980 con 0x7f22180b0ee0 2023-07-14T11:54:19.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.172+0000 7f221ec20700 1 -- 172.21.15.104:0/730886760 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22181596d0 con 0x7f22180b0ee0 2023-07-14T11:54:19.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.172+0000 7f220e7fc700 1 -- 172.21.15.104:0/730886760 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2214018360 con 0x7f22180b0ee0 2023-07-14T11:54:19.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.176+0000 7f220e7fc700 1 --2- 172.21.15.104:0/730886760 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21fc0662f0 0x7f21fc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.176+0000 7f220e7fc700 1 -- 172.21.15.104:0/730886760 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2214084080 con 0x7f22180b0ee0 2023-07-14T11:54:19.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.176+0000 7f221d41d700 1 --2- 172.21.15.104:0/730886760 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21fc0662f0 0x7f21fc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.176+0000 7f221d41d700 1 --2- 172.21.15.104:0/730886760 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21fc0662f0 0x7f21fc0687a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2208000c00 tx=0x7f2208008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:19.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.180+0000 7f220e7fc700 1 -- 172.21.15.104:0/730886760 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f22140b4720 con 0x7f22180b0ee0 2023-07-14T11:54:19.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.352+0000 7f221ec20700 1 -- 172.21.15.104:0/730886760 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f221804e500 con 0x7f22180b0ee0 2023-07-14T11:54:19.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.352+0000 7f220e7fc700 1 -- 172.21.15.104:0/730886760 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f22181596d0 con 0x7f22180b0ee0 2023-07-14T11:54:19.338 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:54:19.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.356+0000 7f221ec20700 1 -- 172.21.15.104:0/730886760 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21fc0662f0 msgr2=0x7f21fc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:19.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.356+0000 7f221ec20700 1 --2- 172.21.15.104:0/730886760 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21fc0662f0 0x7f21fc0687a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2208000c00 tx=0x7f2208008030 comp rx=0 tx=0).stop 2023-07-14T11:54:19.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.356+0000 7f221ec20700 1 -- 172.21.15.104:0/730886760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22180b0ee0 msgr2=0x7f22181f0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:19.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.356+0000 7f221ec20700 1 --2- 172.21.15.104:0/730886760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22180b0ee0 0x7f22181f0d00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f221400c7f0 tx=0x7f221400cb00 comp rx=0 tx=0).stop 2023-07-14T11:54:19.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.356+0000 7f221ec20700 1 -- 172.21.15.104:0/730886760 shutdown_connections 2023-07-14T11:54:19.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.360+0000 7f221ec20700 1 --2- 172.21.15.104:0/730886760 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21fc0662f0 0x7f21fc0687a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.360+0000 7f221ec20700 1 --2- 172.21.15.104:0/730886760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22180d0610 0x7f221815fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.360+0000 7f221ec20700 1 --2- 172.21.15.104:0/730886760 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22180cd090 0x7f221815f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.360+0000 7f221ec20700 1 --2- 172.21.15.104:0/730886760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22180b0ee0 0x7f22181f0d00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.360+0000 7f221ec20700 1 -- 172.21.15.104:0/730886760 >> 172.21.15.104:0/730886760 conn(0x7f2218060c40 msgr2=0x7f221815b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:19.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.360+0000 7f221ec20700 1 -- 172.21.15.104:0/730886760 shutdown_connections 2023-07-14T11:54:19.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.360+0000 7f221ec20700 1 -- 172.21.15.104:0/730886760 wait complete. 2023-07-14T11:54:19.363 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:54:19.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.552+0000 7f934db9e700 1 Processor -- start 2023-07-14T11:54:19.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.552+0000 7f934db9e700 1 -- start start 2023-07-14T11:54:19.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f934db9e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93480b0ee0 0x7f93480ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f934db9e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f93480d07e0 0x7f93480af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f934db9e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93480cd4e0 con 0x7f93480cd090 2023-07-14T11:54:19.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f934db9e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93480d0c60 con 0x7f93480d07e0 2023-07-14T11:54:19.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f934db9e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93480d1a60 con 0x7f93480b0ee0 2023-07-14T11:54:19.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f934d39d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f93480d07e0 0x7f93480af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f934d39d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f93480d07e0 0x7f93480af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35262/0 (socket says 172.21.15.104:35262) 2023-07-14T11:54:19.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f934d39d700 1 -- 172.21.15.104:0/1703403704 learned_addr learned my addr 172.21.15.104:0/1703403704 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:19.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f934cb9c700 1 --2- 172.21.15.104:0/1703403704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93480b0ee0 0x7f93480ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f934d39d700 1 -- 172.21.15.104:0/1703403704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93480b0ee0 msgr2=0x7f93480ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:19.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f934d39d700 1 --2- 172.21.15.104:0/1703403704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93480b0ee0 0x7f93480ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f934d39d700 1 -- 172.21.15.104:0/1703403704 >> v1:172.21.15.121:6789/0 conn(0x7f93480cd090 legacy=0x7f93480d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:19.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f934d39d700 1 -- 172.21.15.104:0/1703403704 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93480d1ef0 con 0x7f93480d07e0 2023-07-14T11:54:19.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f934d39d700 1 --2- 172.21.15.104:0/1703403704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93480d07e0 0x7f93480af0c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9340009e80 tx=0x7f934000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5dfb3248b594464 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:19.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f93477fe700 1 -- 172.21.15.104:0/1703403704 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934000e070 con 0x7f93480d07e0 2023-07-14T11:54:19.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f93477fe700 1 -- 172.21.15.104:0/1703403704 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934000b980 con 0x7f93480d07e0 2023-07-14T11:54:19.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.556+0000 7f93477fe700 1 -- 172.21.15.104:0/1703403704 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934000bd30 con 0x7f93480d07e0 2023-07-14T11:54:19.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.560+0000 7f934db9e700 1 -- 172.21.15.104:0/1703403704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93480d07e0 msgr2=0x7f93480af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:19.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.560+0000 7f934db9e700 1 --2- 172.21.15.104:0/1703403704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93480d07e0 0x7f93480af0c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9340009e80 tx=0x7f934000b330 comp rx=0 tx=0).stop 2023-07-14T11:54:19.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.560+0000 7f934db9e700 1 -- 172.21.15.104:0/1703403704 shutdown_connections 2023-07-14T11:54:19.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.560+0000 7f934db9e700 1 --2- 172.21.15.104:0/1703403704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93480d07e0 0x7f93480af0c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.560+0000 7f934db9e700 1 --2- 172.21.15.104:0/1703403704 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93480b0ee0 0x7f93480ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.560+0000 7f934db9e700 1 -- 172.21.15.104:0/1703403704 >> 172.21.15.104:0/1703403704 conn(0x7f9348060c40 msgr2=0x7f934809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:19.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.560+0000 7f934db9e700 1 -- 172.21.15.104:0/1703403704 shutdown_connections 2023-07-14T11:54:19.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.560+0000 7f934db9e700 1 -- 172.21.15.104:0/1703403704 wait complete. 2023-07-14T11:54:19.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934db9e700 1 Processor -- start 2023-07-14T11:54:19.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934db9e700 1 -- start start 2023-07-14T11:54:19.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934db9e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93480b0ee0 0x7f93480b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934db9e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93480cd090 0x7f93480b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934db9e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93480d07e0 0x7f93480b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934db9e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93480b1e40 con 0x7f93480d07e0 2023-07-14T11:54:19.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934db9e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93481f2030 con 0x7f93480cd090 2023-07-14T11:54:19.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934db9e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93480b2050 con 0x7f93480b0ee0 2023-07-14T11:54:19.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934d39d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93480d07e0 0x7f93480b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934d39d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93480d07e0 0x7f93480b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35272/0 (socket says 172.21.15.104:35272) 2023-07-14T11:54:19.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934d39d700 1 -- 172.21.15.104:0/2314820655 learned_addr learned my addr 172.21.15.104:0/2314820655 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:19.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f9347fff700 1 --2- 172.21.15.104:0/2314820655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93480cd090 0x7f93480b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934cb9c700 1 --2- 172.21.15.104:0/2314820655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93480b0ee0 0x7f93480b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934d39d700 1 -- 172.21.15.104:0/2314820655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93480b0ee0 msgr2=0x7f93480b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:19.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934d39d700 1 --2- 172.21.15.104:0/2314820655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93480b0ee0 0x7f93480b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934d39d700 1 -- 172.21.15.104:0/2314820655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93480cd090 msgr2=0x7f93480b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:19.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934d39d700 1 --2- 172.21.15.104:0/2314820655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93480cd090 0x7f93480b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934d39d700 1 -- 172.21.15.104:0/2314820655 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9340009b30 con 0x7f93480d07e0 2023-07-14T11:54:19.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f9347fff700 1 --2- 172.21.15.104:0/2314820655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93480cd090 0x7f93480b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:19.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.564+0000 7f934d39d700 1 --2- 172.21.15.104:0/2314820655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93480d07e0 0x7f93480b48b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f934808adc0 tx=0x7f9340004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:19.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.568+0000 7f93457fa700 1 -- 172.21.15.104:0/2314820655 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340004960 con 0x7f93480d07e0 2023-07-14T11:54:19.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.568+0000 7f93457fa700 1 -- 172.21.15.104:0/2314820655 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93400099b0 con 0x7f93480d07e0 2023-07-14T11:54:19.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.568+0000 7f934db9e700 1 -- 172.21.15.104:0/2314820655 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93480b2330 con 0x7f93480d07e0 2023-07-14T11:54:19.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.568+0000 7f93457fa700 1 -- 172.21.15.104:0/2314820655 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934001c650 con 0x7f93480d07e0 2023-07-14T11:54:19.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.568+0000 7f934db9e700 1 -- 172.21.15.104:0/2314820655 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93480b28f0 con 0x7f93480d07e0 2023-07-14T11:54:19.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.568+0000 7f934db9e700 1 -- 172.21.15.104:0/2314820655 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934804e500 con 0x7f93480d07e0 2023-07-14T11:54:19.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.572+0000 7f93457fa700 1 -- 172.21.15.104:0/2314820655 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f934001c7b0 con 0x7f93480d07e0 2023-07-14T11:54:19.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.572+0000 7f93457fa700 1 --2- 172.21.15.104:0/2314820655 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f932c0662f0 0x7f932c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.572+0000 7f93457fa700 1 -- 172.21.15.104:0/2314820655 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9340085be0 con 0x7f93480d07e0 2023-07-14T11:54:19.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.572+0000 7f934cb9c700 1 --2- 172.21.15.104:0/2314820655 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f932c0662f0 0x7f932c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.572+0000 7f934cb9c700 1 --2- 172.21.15.104:0/2314820655 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f932c0662f0 0x7f932c0687a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f93480c8f60 tx=0x7f933c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:19.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.576+0000 7f93457fa700 1 -- 172.21.15.104:0/2314820655 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f9340052dc0 con 0x7f93480d07e0 2023-07-14T11:54:19.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.752+0000 7f934db9e700 1 -- 172.21.15.104:0/2314820655 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f93480b18d0 con 0x7f93480d07e0 2023-07-14T11:54:19.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.752+0000 7f93457fa700 1 -- 172.21.15.104:0/2314820655 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f93400563e0 con 0x7f93480d07e0 2023-07-14T11:54:19.738 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:19.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.756+0000 7f934db9e700 1 -- 172.21.15.104:0/2314820655 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f932c0662f0 msgr2=0x7f932c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:19.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.756+0000 7f934db9e700 1 --2- 172.21.15.104:0/2314820655 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f932c0662f0 0x7f932c0687a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f93480c8f60 tx=0x7f933c008040 comp rx=0 tx=0).stop 2023-07-14T11:54:19.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.756+0000 7f934db9e700 1 -- 172.21.15.104:0/2314820655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93480d07e0 msgr2=0x7f93480b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:19.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.756+0000 7f934db9e700 1 --2- 172.21.15.104:0/2314820655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93480d07e0 0x7f93480b48b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f934808adc0 tx=0x7f9340004af0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.760+0000 7f934db9e700 1 -- 172.21.15.104:0/2314820655 shutdown_connections 2023-07-14T11:54:19.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.760+0000 7f934db9e700 1 --2- 172.21.15.104:0/2314820655 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f932c0662f0 0x7f932c0687a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.760+0000 7f934db9e700 1 --2- 172.21.15.104:0/2314820655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93480d07e0 0x7f93480b48b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.760+0000 7f934db9e700 1 --2- 172.21.15.104:0/2314820655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93480cd090 0x7f93480b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.760+0000 7f934db9e700 1 --2- 172.21.15.104:0/2314820655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93480b0ee0 0x7f93480b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.760+0000 7f934db9e700 1 -- 172.21.15.104:0/2314820655 >> 172.21.15.104:0/2314820655 conn(0x7f9348060c40 msgr2=0x7f934815b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:19.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.760+0000 7f934db9e700 1 -- 172.21.15.104:0/2314820655 shutdown_connections 2023-07-14T11:54:19.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.760+0000 7f934db9e700 1 -- 172.21.15.104:0/2314820655 wait complete. 2023-07-14T11:54:19.765 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:54:19.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.952+0000 7fe365b5d700 1 Processor -- start 2023-07-14T11:54:19.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.952+0000 7fe365b5d700 1 -- start start 2023-07-14T11:54:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe365b5d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3601599d0 0x7fe360071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe365b5d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe3600caa20 0x7fe3600cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe365b5d700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3600cde10 con 0x7fe36015ccf0 2023-07-14T11:54:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe365b5d700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3600cb3c0 con 0x7fe3600caa20 2023-07-14T11:54:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe365b5d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3600cd520 con 0x7fe3601599d0 2023-07-14T11:54:19.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe35ffff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fe36015ccf0 0x7fe3600c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:44890/0 (socket says 172.21.15.104:44890) 2023-07-14T11:54:19.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe35ffff700 1 -- 172.21.15.104:0/828279493 learned_addr learned my addr 172.21.15.104:0/828279493 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:19.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe364b5b700 1 --2- 172.21.15.104:0/828279493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3601599d0 0x7fe360071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe36535c700 1 --2- 172.21.15.104:0/828279493 >> v2:172.21.15.121:3300/0 conn(0x7fe3600caa20 0x7fe3600cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe35f7fe700 1 -- 172.21.15.104:0/828279493 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148606136 0 0) 0x7fe3600cde10 con 0x7fe36015ccf0 2023-07-14T11:54:19.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe35f7fe700 1 -- 172.21.15.104:0/828279493 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3500035d0 con 0x7fe36015ccf0 2023-07-14T11:54:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe35f7fe700 1 -- 172.21.15.104:0/828279493 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 795853883 0 0) 0x7fe3500035d0 con 0x7fe36015ccf0 2023-07-14T11:54:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe364b5b700 1 -- 172.21.15.104:0/828279493 >> v2:172.21.15.121:3300/0 conn(0x7fe3600caa20 msgr2=0x7fe3600cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe364b5b700 1 --2- 172.21.15.104:0/828279493 >> v2:172.21.15.121:3300/0 conn(0x7fe3600caa20 0x7fe3600cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe364b5b700 1 -- 172.21.15.104:0/828279493 >> v1:172.21.15.104:6789/0 conn(0x7fe36015ccf0 legacy=0x7fe3600c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe364b5b700 1 -- 172.21.15.104:0/828279493 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3600715e0 con 0x7fe3601599d0 2023-07-14T11:54:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe36535c700 1 --2- 172.21.15.104:0/828279493 >> v2:172.21.15.121:3300/0 conn(0x7fe3600caa20 0x7fe3600cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:19.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe364b5b700 1 --2- 172.21.15.104:0/828279493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3601599d0 0x7fe360071000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe354009d90 tx=0x7fe35400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d784981926532a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:19.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe35f7fe700 1 -- 172.21.15.104:0/828279493 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35400e070 con 0x7fe3601599d0 2023-07-14T11:54:19.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe35f7fe700 1 -- 172.21.15.104:0/828279493 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe354003a40 con 0x7fe3601599d0 2023-07-14T11:54:19.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.956+0000 7fe35f7fe700 1 -- 172.21.15.104:0/828279493 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c001670 con 0x7fe3601599d0 2023-07-14T11:54:19.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.960+0000 7fe365b5d700 1 -- 172.21.15.104:0/828279493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3601599d0 msgr2=0x7fe360071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:19.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.960+0000 7fe365b5d700 1 --2- 172.21.15.104:0/828279493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3601599d0 0x7fe360071000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe354009d90 tx=0x7fe35400b330 comp rx=0 tx=0).stop 2023-07-14T11:54:19.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.960+0000 7fe365b5d700 1 -- 172.21.15.104:0/828279493 shutdown_connections 2023-07-14T11:54:19.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.960+0000 7fe365b5d700 1 --2- 172.21.15.104:0/828279493 >> v2:172.21.15.121:3300/0 conn(0x7fe3600caa20 0x7fe3600cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.960+0000 7fe365b5d700 1 --2- 172.21.15.104:0/828279493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3601599d0 0x7fe360071000 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.960+0000 7fe365b5d700 1 -- 172.21.15.104:0/828279493 >> 172.21.15.104:0/828279493 conn(0x7fe36006e960 msgr2=0x7fe36006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:19.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.960+0000 7fe365b5d700 1 -- 172.21.15.104:0/828279493 shutdown_connections 2023-07-14T11:54:19.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.960+0000 7fe365b5d700 1 -- 172.21.15.104:0/828279493 wait complete. 2023-07-14T11:54:19.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.960+0000 7fe365b5d700 1 Processor -- start 2023-07-14T11:54:19.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe365b5d700 1 -- start start 2023-07-14T11:54:19.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe365b5d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3600caa20 0x7fe3601f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe365b5d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3601599d0 0x7fe360147e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe365b5d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe36015ccf0 0x7fe3601460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe365b5d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3601586c0 con 0x7fe3600caa20 2023-07-14T11:54:19.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe365b5d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3600c7250 con 0x7fe3601599d0 2023-07-14T11:54:19.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe365b5d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3601f5a20 con 0x7fe36015ccf0 2023-07-14T11:54:19.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe364b5b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3600caa20 0x7fe3601f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe364b5b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3600caa20 0x7fe3601f54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35288/0 (socket says 172.21.15.104:35288) 2023-07-14T11:54:19.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe364b5b700 1 -- 172.21.15.104:0/1786751132 learned_addr learned my addr 172.21.15.104:0/1786751132 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:19.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe36535c700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe36015ccf0 0x7fe3601460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe35ffff700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3601599d0 0x7fe360147e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe364b5b700 1 -- 172.21.15.104:0/1786751132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe36015ccf0 msgr2=0x7fe3601460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:19.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe364b5b700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe36015ccf0 0x7fe3601460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe364b5b700 1 -- 172.21.15.104:0/1786751132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3601599d0 msgr2=0x7fe360147e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:19.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe364b5b700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3601599d0 0x7fe360147e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:19.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe364b5b700 1 -- 172.21.15.104:0/1786751132 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3580094b0 con 0x7fe3600caa20 2023-07-14T11:54:19.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe36535c700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe36015ccf0 0x7fe3601460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:19.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe35ffff700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3601599d0 0x7fe360147e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:19.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe364b5b700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3600caa20 0x7fe3601f54e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe36006f940 tx=0x7fe354004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:19.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe35d7fa700 1 -- 172.21.15.104:0/1786751132 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354004670 con 0x7fe3600caa20 2023-07-14T11:54:19.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe35d7fa700 1 -- 172.21.15.104:0/1786751132 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe354004810 con 0x7fe3600caa20 2023-07-14T11:54:19.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe35d7fa700 1 -- 172.21.15.104:0/1786751132 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354012670 con 0x7fe3600caa20 2023-07-14T11:54:19.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.964+0000 7fe365b5d700 1 -- 172.21.15.104:0/1786751132 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe354009a70 con 0x7fe3600caa20 2023-07-14T11:54:19.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.968+0000 7fe365b5d700 1 -- 172.21.15.104:0/1786751132 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3601583e0 con 0x7fe3600caa20 2023-07-14T11:54:19.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.968+0000 7fe365b5d700 1 -- 172.21.15.104:0/1786751132 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36015d140 con 0x7fe3600caa20 2023-07-14T11:54:19.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.968+0000 7fe35d7fa700 1 -- 172.21.15.104:0/1786751132 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe3540127d0 con 0x7fe3600caa20 2023-07-14T11:54:19.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.972+0000 7fe35d7fa700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe3440662f0 0x7fe3440687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:19.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.972+0000 7fe35d7fa700 1 -- 172.21.15.104:0/1786751132 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe354085a10 con 0x7fe3600caa20 2023-07-14T11:54:19.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.972+0000 7fe35ffff700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe3440662f0 0x7fe3440687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:19.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.972+0000 7fe35ffff700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe3440662f0 0x7fe3440687a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe34c001860 tx=0x7fe34c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:19.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:19.976+0000 7fe35d7fa700 1 -- 172.21.15.104:0/1786751132 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe36015d140 con 0x7fe3600caa20 2023-07-14T11:54:20.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.159+0000 7fe365b5d700 1 -- 172.21.15.104:0/1786751132 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe36015d140 con 0x7fe3600caa20 2023-07-14T11:54:20.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.159+0000 7fe35d7fa700 1 -- 172.21.15.104:0/1786751132 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe36015d140 con 0x7fe3600caa20 2023-07-14T11:54:20.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.163+0000 7fe365b5d700 1 -- 172.21.15.104:0/1786751132 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe3440662f0 msgr2=0x7fe3440687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:20.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.163+0000 7fe365b5d700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe3440662f0 0x7fe3440687a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe34c001860 tx=0x7fe34c009040 comp rx=0 tx=0).stop 2023-07-14T11:54:20.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.163+0000 7fe365b5d700 1 -- 172.21.15.104:0/1786751132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3600caa20 msgr2=0x7fe3601f54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:20.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.163+0000 7fe365b5d700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3600caa20 0x7fe3601f54e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe36006f940 tx=0x7fe354004af0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.163+0000 7fe365b5d700 1 -- 172.21.15.104:0/1786751132 shutdown_connections 2023-07-14T11:54:20.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.163+0000 7fe365b5d700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe3440662f0 0x7fe3440687a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.163+0000 7fe365b5d700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe36015ccf0 0x7fe3601460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.163+0000 7fe365b5d700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3601599d0 0x7fe360147e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.163+0000 7fe365b5d700 1 --2- 172.21.15.104:0/1786751132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3600caa20 0x7fe3601f54e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.163+0000 7fe365b5d700 1 -- 172.21.15.104:0/1786751132 >> 172.21.15.104:0/1786751132 conn(0x7fe36006e960 msgr2=0x7fe360154940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:20.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.163+0000 7fe365b5d700 1 -- 172.21.15.104:0/1786751132 shutdown_connections 2023-07-14T11:54:20.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.163+0000 7fe365b5d700 1 -- 172.21.15.104:0/1786751132 wait complete. 2023-07-14T11:54:20.152 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:54:20.171 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:54:20.171 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:54:20.186 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:20.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.415+0000 7fc42999e700 1 Processor -- start 2023-07-14T11:54:20.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.415+0000 7fc42999e700 1 -- start start 2023-07-14T11:54:20.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.415+0000 7fc42999e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc4240c9d90 0x7fc4240ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:20.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.415+0000 7fc42999e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc42415d7f0 0x7fc42415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:20.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.415+0000 7fc42999e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4240b0ef0 con 0x7fc424159f20 2023-07-14T11:54:20.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.415+0000 7fc42999e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4240ce9c0 con 0x7fc4240c9d90 2023-07-14T11:54:20.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.415+0000 7fc42999e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240cee50 con 0x7fc42415d7f0 2023-07-14T11:54:20.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.415+0000 7fc42899c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc4240c9d90 0x7fc4240ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:20.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42899c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc4240c9d90 0x7fc4240ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56862/0 (socket says 172.21.15.104:56862) 2023-07-14T11:54:20.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42899c700 1 -- 172.21.15.104:0/2075068162 learned_addr learned my addr 172.21.15.104:0/2075068162 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:20.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42919d700 1 --2- 172.21.15.104:0/2075068162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc42415d7f0 0x7fc42415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:20.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42899c700 1 -- 172.21.15.104:0/2075068162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc42415d7f0 msgr2=0x7fc42415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:20.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42899c700 1 --2- 172.21.15.104:0/2075068162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc42415d7f0 0x7fc42415dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42899c700 1 -- 172.21.15.104:0/2075068162 >> v1:172.21.15.121:6789/0 conn(0x7fc424159f20 legacy=0x7fc4240caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:20.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42899c700 1 -- 172.21.15.104:0/2075068162 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4240ca6a0 con 0x7fc4240c9d90 2023-07-14T11:54:20.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42899c700 1 --2- 172.21.15.104:0/2075068162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4240c9d90 0x7fc4240ca160 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc418009910 tx=0x7fc418009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471a1cc1ea44e59c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:20.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc4237fe700 1 -- 172.21.15.104:0/2075068162 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418011830 con 0x7fc4240c9d90 2023-07-14T11:54:20.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc4237fe700 1 -- 172.21.15.104:0/2075068162 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc418011990 con 0x7fc4240c9d90 2023-07-14T11:54:20.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc4237fe700 1 -- 172.21.15.104:0/2075068162 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418011d40 con 0x7fc4240c9d90 2023-07-14T11:54:20.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42999e700 1 -- 172.21.15.104:0/2075068162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4240c9d90 msgr2=0x7fc4240ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:20.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42999e700 1 --2- 172.21.15.104:0/2075068162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4240c9d90 0x7fc4240ca160 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc418009910 tx=0x7fc418009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:20.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42999e700 1 -- 172.21.15.104:0/2075068162 shutdown_connections 2023-07-14T11:54:20.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42999e700 1 --2- 172.21.15.104:0/2075068162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc42415d7f0 0x7fc42415dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42999e700 1 --2- 172.21.15.104:0/2075068162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4240c9d90 0x7fc4240ca160 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42999e700 1 -- 172.21.15.104:0/2075068162 >> 172.21.15.104:0/2075068162 conn(0x7fc42406e960 msgr2=0x7fc42406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:20.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42999e700 1 -- 172.21.15.104:0/2075068162 shutdown_connections 2023-07-14T11:54:20.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.419+0000 7fc42999e700 1 -- 172.21.15.104:0/2075068162 wait complete. 2023-07-14T11:54:20.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.423+0000 7fc42999e700 1 Processor -- start 2023-07-14T11:54:20.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.423+0000 7fc42999e700 1 -- start start 2023-07-14T11:54:20.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42999e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4240c9d90 0x7fc4241464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:20.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42999e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc424159f20 0x7fc4240c1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:20.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42999e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc42415d7f0 0x7fc4240cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:20.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42999e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4241f5b50 con 0x7fc4240c9d90 2023-07-14T11:54:20.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42999e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc424146a30 con 0x7fc42415d7f0 2023-07-14T11:54:20.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42999e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4241f4f40 con 0x7fc424159f20 2023-07-14T11:54:20.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42899c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4240c9d90 0x7fc4241464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:20.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42899c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4240c9d90 0x7fc4241464f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56866/0 (socket says 172.21.15.104:56866) 2023-07-14T11:54:20.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42899c700 1 -- 172.21.15.104:0/3320419674 learned_addr learned my addr 172.21.15.104:0/3320419674 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:20.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42919d700 1 --2- 172.21.15.104:0/3320419674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc42415d7f0 0x7fc4240cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:20.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc423fff700 1 --2- 172.21.15.104:0/3320419674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc424159f20 0x7fc4240c1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:20.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42899c700 1 -- 172.21.15.104:0/3320419674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc424159f20 msgr2=0x7fc4240c1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:20.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42899c700 1 --2- 172.21.15.104:0/3320419674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc424159f20 0x7fc4240c1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42899c700 1 -- 172.21.15.104:0/3320419674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc42415d7f0 msgr2=0x7fc4240cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:20.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42899c700 1 --2- 172.21.15.104:0/3320419674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc42415d7f0 0x7fc4240cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42899c700 1 -- 172.21.15.104:0/3320419674 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc418009570 con 0x7fc4240c9d90 2023-07-14T11:54:20.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42899c700 1 --2- 172.21.15.104:0/3320419674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4240c9d90 0x7fc4241464f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc418009f70 tx=0x7fc418004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:20.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc4217fa700 1 -- 172.21.15.104:0/3320419674 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418004510 con 0x7fc4240c9d90 2023-07-14T11:54:20.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc4217fa700 1 -- 172.21.15.104:0/3320419674 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41801e070 con 0x7fc4240c9d90 2023-07-14T11:54:20.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc4217fa700 1 -- 172.21.15.104:0/3320419674 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418003900 con 0x7fc4240c9d90 2023-07-14T11:54:20.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42999e700 1 -- 172.21.15.104:0/3320419674 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4241f5220 con 0x7fc4240c9d90 2023-07-14T11:54:20.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.427+0000 7fc42999e700 1 -- 172.21.15.104:0/3320419674 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4241f57e0 con 0x7fc4240c9d90 2023-07-14T11:54:20.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.431+0000 7fc42999e700 1 -- 172.21.15.104:0/3320419674 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc42404e500 con 0x7fc4240c9d90 2023-07-14T11:54:20.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.431+0000 7fc4217fa700 1 -- 172.21.15.104:0/3320419674 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc418003a60 con 0x7fc4240c9d90 2023-07-14T11:54:20.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.435+0000 7fc4217fa700 1 --2- 172.21.15.104:0/3320419674 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc4080662f0 0x7fc4080687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:20.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.435+0000 7fc4217fa700 1 -- 172.21.15.104:0/3320419674 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc418084c50 con 0x7fc4240c9d90 2023-07-14T11:54:20.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.435+0000 7fc423fff700 1 --2- 172.21.15.104:0/3320419674 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc4080662f0 0x7fc4080687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:20.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.435+0000 7fc423fff700 1 --2- 172.21.15.104:0/3320419674 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc4080662f0 0x7fc4080687a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc4240b0c80 tx=0x7fc410009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:20.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.439+0000 7fc4217fa700 1 -- 172.21.15.104:0/3320419674 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fc418051d90 con 0x7fc4240c9d90 2023-07-14T11:54:20.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.611+0000 7fc42999e700 1 -- 172.21.15.104:0/3320419674 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4240ce640 con 0x7fc4240c9d90 2023-07-14T11:54:20.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.615+0000 7fc4217fa700 1 -- 172.21.15.104:0/3320419674 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20251 (secure 0 0 0) 0x7fc418020090 con 0x7fc4240c9d90 2023-07-14T11:54:20.600 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:20.600 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":30,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:55.601783+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:20.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.615+0000 7fc42999e700 1 -- 172.21.15.104:0/3320419674 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc4080662f0 msgr2=0x7fc4080687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:20.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.615+0000 7fc42999e700 1 --2- 172.21.15.104:0/3320419674 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc4080662f0 0x7fc4080687a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc4240b0c80 tx=0x7fc410009040 comp rx=0 tx=0).stop 2023-07-14T11:54:20.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.615+0000 7fc42999e700 1 -- 172.21.15.104:0/3320419674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4240c9d90 msgr2=0x7fc4241464f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:20.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.615+0000 7fc42999e700 1 --2- 172.21.15.104:0/3320419674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4240c9d90 0x7fc4241464f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc418009f70 tx=0x7fc418004770 comp rx=0 tx=0).stop 2023-07-14T11:54:20.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.619+0000 7fc42999e700 1 -- 172.21.15.104:0/3320419674 shutdown_connections 2023-07-14T11:54:20.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.619+0000 7fc42999e700 1 --2- 172.21.15.104:0/3320419674 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc4080662f0 0x7fc4080687a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.619+0000 7fc42999e700 1 --2- 172.21.15.104:0/3320419674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc42415d7f0 0x7fc4240cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.619+0000 7fc42999e700 1 --2- 172.21.15.104:0/3320419674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc424159f20 0x7fc4240c1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.619+0000 7fc42999e700 1 --2- 172.21.15.104:0/3320419674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4240c9d90 0x7fc4241464f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.619+0000 7fc42999e700 1 -- 172.21.15.104:0/3320419674 >> 172.21.15.104:0/3320419674 conn(0x7fc42406e960 msgr2=0x7fc4241543e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:20.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.619+0000 7fc42999e700 1 -- 172.21.15.104:0/3320419674 shutdown_connections 2023-07-14T11:54:20.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.619+0000 7fc42999e700 1 -- 172.21.15.104:0/3320419674 wait complete. 2023-07-14T11:54:20.625 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:54:20.626 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:54:20.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.827+0000 7ffa29493700 1 Processor -- start 2023-07-14T11:54:20.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.827+0000 7ffa29493700 1 -- start start 2023-07-14T11:54:20.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa29493700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ffa240b0ee0 0x7ffa240ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:20.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa29493700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa240cd090 0x7ffa240d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:20.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa29493700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa240cd4e0 con 0x7ffa240d0610 2023-07-14T11:54:20.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa29493700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa240d1a60 con 0x7ffa240b0ee0 2023-07-14T11:54:20.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa29493700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa240d1ef0 con 0x7ffa240cd090 2023-07-14T11:54:20.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa23fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ffa240b0ee0 0x7ffa240ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:20.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa28c92700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ffa240d0610 0x7ffa2415f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53668/0 (socket says 172.21.15.104:53668) 2023-07-14T11:54:20.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa28c92700 1 -- 172.21.15.104:0/528285475 learned_addr learned my addr 172.21.15.104:0/528285475 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:20.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa237fe700 1 --2- 172.21.15.104:0/528285475 >> v2:172.21.15.121:3300/0 conn(0x7ffa240cd090 0x7ffa240d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:20.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa23fff700 1 -- 172.21.15.104:0/528285475 >> v2:172.21.15.121:3300/0 conn(0x7ffa240cd090 msgr2=0x7ffa240d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:20.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa23fff700 1 --2- 172.21.15.104:0/528285475 >> v2:172.21.15.121:3300/0 conn(0x7ffa240cd090 0x7ffa240d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa23fff700 1 -- 172.21.15.104:0/528285475 >> v1:172.21.15.104:6789/0 conn(0x7ffa240d0610 legacy=0x7ffa2415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:20.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa23fff700 1 -- 172.21.15.104:0/528285475 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa240d0e50 con 0x7ffa240b0ee0 2023-07-14T11:54:20.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa23fff700 1 --2- 172.21.15.104:0/528285475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa240b0ee0 0x7ffa240ccb50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffa240b5090 tx=0x7ffa14009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e1c90311235d43 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:20.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa22ffd700 1 -- 172.21.15.104:0/528285475 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa14011830 con 0x7ffa240b0ee0 2023-07-14T11:54:20.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa22ffd700 1 -- 172.21.15.104:0/528285475 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa14011990 con 0x7ffa240b0ee0 2023-07-14T11:54:20.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.831+0000 7ffa22ffd700 1 -- 172.21.15.104:0/528285475 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa14011d40 con 0x7ffa240b0ee0 2023-07-14T11:54:20.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.835+0000 7ffa29493700 1 -- 172.21.15.104:0/528285475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa240b0ee0 msgr2=0x7ffa240ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:20.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.835+0000 7ffa29493700 1 --2- 172.21.15.104:0/528285475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa240b0ee0 0x7ffa240ccb50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffa240b5090 tx=0x7ffa14009b30 comp rx=0 tx=0).stop 2023-07-14T11:54:20.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.835+0000 7ffa29493700 1 -- 172.21.15.104:0/528285475 shutdown_connections 2023-07-14T11:54:20.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.835+0000 7ffa29493700 1 --2- 172.21.15.104:0/528285475 >> v2:172.21.15.121:3300/0 conn(0x7ffa240cd090 0x7ffa240d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.835+0000 7ffa29493700 1 --2- 172.21.15.104:0/528285475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa240b0ee0 0x7ffa240ccb50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.835+0000 7ffa29493700 1 -- 172.21.15.104:0/528285475 >> 172.21.15.104:0/528285475 conn(0x7ffa24060c40 msgr2=0x7ffa2409b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:20.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.835+0000 7ffa29493700 1 -- 172.21.15.104:0/528285475 shutdown_connections 2023-07-14T11:54:20.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.835+0000 7ffa29493700 1 -- 172.21.15.104:0/528285475 wait complete. 2023-07-14T11:54:20.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.835+0000 7ffa29493700 1 Processor -- start 2023-07-14T11:54:20.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.835+0000 7ffa29493700 1 -- start start 2023-07-14T11:54:20.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa29493700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa240b0ee0 0x7ffa240c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:20.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa29493700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa240cd090 0x7ffa240c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:20.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa29493700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa240d0610 0x7ffa2415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:20.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa29493700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa240c6800 con 0x7ffa240cd090 2023-07-14T11:54:20.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa29493700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa240c7e70 con 0x7ffa240d0610 2023-07-14T11:54:20.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa29493700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa240ca400 con 0x7ffa240b0ee0 2023-07-14T11:54:20.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa237fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa240cd090 0x7ffa240c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:20.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa237fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa240cd090 0x7ffa240c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56878/0 (socket says 172.21.15.104:56878) 2023-07-14T11:54:20.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa237fe700 1 -- 172.21.15.104:0/1335701050 learned_addr learned my addr 172.21.15.104:0/1335701050 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:20.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa23fff700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa240b0ee0 0x7ffa240c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:20.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa28c92700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa240d0610 0x7ffa2415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:20.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa237fe700 1 -- 172.21.15.104:0/1335701050 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa240b0ee0 msgr2=0x7ffa240c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:20.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa237fe700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa240b0ee0 0x7ffa240c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa237fe700 1 -- 172.21.15.104:0/1335701050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa240d0610 msgr2=0x7ffa2415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:20.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa237fe700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa240d0610 0x7ffa2415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:20.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa237fe700 1 -- 172.21.15.104:0/1335701050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa140094b0 con 0x7ffa240cd090 2023-07-14T11:54:20.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa23fff700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa240b0ee0 0x7ffa240c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:20.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa28c92700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa240d0610 0x7ffa2415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:20.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa237fe700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa240cd090 0x7ffa240c7930 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800e760 tx=0x7ffa1800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:20.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa20ff9700 1 -- 172.21.15.104:0/1335701050 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18013610 con 0x7ffa240cd090 2023-07-14T11:54:20.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa20ff9700 1 -- 172.21.15.104:0/1335701050 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa18013770 con 0x7ffa240cd090 2023-07-14T11:54:20.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa29493700 1 -- 172.21.15.104:0/1335701050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa240c61b0 con 0x7ffa240cd090 2023-07-14T11:54:20.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa20ff9700 1 -- 172.21.15.104:0/1335701050 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18013ae0 con 0x7ffa240cd090 2023-07-14T11:54:20.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.839+0000 7ffa29493700 1 -- 172.21.15.104:0/1335701050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2415a720 con 0x7ffa240cd090 2023-07-14T11:54:20.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.843+0000 7ffa29493700 1 -- 172.21.15.104:0/1335701050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa240c6010 con 0x7ffa240cd090 2023-07-14T11:54:20.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.843+0000 7ffa20ff9700 1 -- 172.21.15.104:0/1335701050 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ffa18013c40 con 0x7ffa240cd090 2023-07-14T11:54:20.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.847+0000 7ffa20ff9700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa10066320 0x7ffa100687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:20.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.847+0000 7ffa20ff9700 1 -- 172.21.15.104:0/1335701050 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ffa18083d50 con 0x7ffa240cd090 2023-07-14T11:54:20.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.847+0000 7ffa23fff700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa10066320 0x7ffa100687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:20.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.847+0000 7ffa23fff700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa10066320 0x7ffa100687d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa24158ed0 tx=0x7ffa14008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:20.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:20.851+0000 7ffa20ff9700 1 -- 172.21.15.104:0/1335701050 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ffa240c6010 con 0x7ffa240cd090 2023-07-14T11:54:21.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.027+0000 7ffa29493700 1 -- 172.21.15.104:0/1335701050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa240c6010 con 0x7ffa240cd090 2023-07-14T11:54:21.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.027+0000 7ffa20ff9700 1 -- 172.21.15.104:0/1335701050 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa240c6010 con 0x7ffa240cd090 2023-07-14T11:54:21.014 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:54:21.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.031+0000 7ffa29493700 1 -- 172.21.15.104:0/1335701050 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa10066320 msgr2=0x7ffa100687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:21.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.031+0000 7ffa29493700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa10066320 0x7ffa100687d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa24158ed0 tx=0x7ffa14008030 comp rx=0 tx=0).stop 2023-07-14T11:54:21.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.031+0000 7ffa29493700 1 -- 172.21.15.104:0/1335701050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa240cd090 msgr2=0x7ffa240c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:21.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.031+0000 7ffa29493700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa240cd090 0x7ffa240c7930 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800e760 tx=0x7ffa1800eb20 comp rx=0 tx=0).stop 2023-07-14T11:54:21.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.031+0000 7ffa29493700 1 -- 172.21.15.104:0/1335701050 shutdown_connections 2023-07-14T11:54:21.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.031+0000 7ffa29493700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa10066320 0x7ffa100687d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.031+0000 7ffa29493700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa240cd090 0x7ffa240c7930 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.031+0000 7ffa29493700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa240d0610 0x7ffa2415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.031+0000 7ffa29493700 1 --2- 172.21.15.104:0/1335701050 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa240b0ee0 0x7ffa240c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.031+0000 7ffa29493700 1 -- 172.21.15.104:0/1335701050 >> 172.21.15.104:0/1335701050 conn(0x7ffa24060c40 msgr2=0x7ffa240ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:21.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.031+0000 7ffa29493700 1 -- 172.21.15.104:0/1335701050 shutdown_connections 2023-07-14T11:54:21.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.031+0000 7ffa29493700 1 -- 172.21.15.104:0/1335701050 wait complete. 2023-07-14T11:54:21.038 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:54:21.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.235+0000 7f7059973700 1 Processor -- start 2023-07-14T11:54:21.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.235+0000 7f7059973700 1 -- start start 2023-07-14T11:54:21.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f7059973700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f70540b0ee0 0x7f70540ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:21.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f7059973700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70540cd090 0x7f70540d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:21.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f7059973700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70540cd4e0 con 0x7f70540d0610 2023-07-14T11:54:21.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f7059973700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70540d1a60 con 0x7f70540b0ee0 2023-07-14T11:54:21.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f7059973700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70540d1ef0 con 0x7f70540cd090 2023-07-14T11:54:21.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f7059172700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f70540d0610 0x7f705415f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53684/0 (socket says 172.21.15.104:53684) 2023-07-14T11:54:21.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f7059172700 1 -- 172.21.15.104:0/1407069090 learned_addr learned my addr 172.21.15.104:0/1407069090 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:21.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f7053fff700 1 --2- 172.21.15.104:0/1407069090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70540cd090 0x7f70540d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:21.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f7058971700 1 --2- 172.21.15.104:0/1407069090 >> v2:172.21.15.121:3300/0 conn(0x7f70540b0ee0 0x7f70540ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:21.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f70537fe700 1 -- 172.21.15.104:0/1407069090 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051879730 0 0) 0x7f704c001490 con 0x7f70540d0610 2023-07-14T11:54:21.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f70537fe700 1 -- 172.21.15.104:0/1407069090 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70440035d0 con 0x7f70540d0610 2023-07-14T11:54:21.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f7053fff700 1 -- 172.21.15.104:0/1407069090 >> v2:172.21.15.121:3300/0 conn(0x7f70540b0ee0 msgr2=0x7f70540ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:21.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f7053fff700 1 --2- 172.21.15.104:0/1407069090 >> v2:172.21.15.121:3300/0 conn(0x7f70540b0ee0 0x7f70540ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f7053fff700 1 -- 172.21.15.104:0/1407069090 >> v1:172.21.15.104:6789/0 conn(0x7f70540d0610 legacy=0x7f705415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:21.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f7053fff700 1 -- 172.21.15.104:0/1407069090 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70540d0e50 con 0x7f70540cd090 2023-07-14T11:54:21.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f7058971700 1 --2- 172.21.15.104:0/1407069090 >> v2:172.21.15.121:3300/0 conn(0x7f70540b0ee0 0x7f70540ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:21.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.239+0000 7f7053fff700 1 --2- 172.21.15.104:0/1407069090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70540cd090 0x7f70540d00d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7040009880 tx=0x7f7040009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffdd28ffa1ecb297 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:21.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.243+0000 7f70537fe700 1 -- 172.21.15.104:0/1407069090 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040011830 con 0x7f70540cd090 2023-07-14T11:54:21.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.243+0000 7f70537fe700 1 -- 172.21.15.104:0/1407069090 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7040011990 con 0x7f70540cd090 2023-07-14T11:54:21.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.243+0000 7f70537fe700 1 -- 172.21.15.104:0/1407069090 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040011d20 con 0x7f70540cd090 2023-07-14T11:54:21.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.243+0000 7f7059973700 1 -- 172.21.15.104:0/1407069090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70540cd090 msgr2=0x7f70540d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:21.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.243+0000 7f7059973700 1 --2- 172.21.15.104:0/1407069090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70540cd090 0x7f70540d00d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7040009880 tx=0x7f7040009b90 comp rx=0 tx=0).stop 2023-07-14T11:54:21.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.243+0000 7f7059973700 1 -- 172.21.15.104:0/1407069090 shutdown_connections 2023-07-14T11:54:21.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.243+0000 7f7059973700 1 --2- 172.21.15.104:0/1407069090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70540cd090 0x7f70540d00d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.243+0000 7f7059973700 1 --2- 172.21.15.104:0/1407069090 >> v2:172.21.15.121:3300/0 conn(0x7f70540b0ee0 0x7f70540ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.243+0000 7f7059973700 1 -- 172.21.15.104:0/1407069090 >> 172.21.15.104:0/1407069090 conn(0x7f7054060c40 msgr2=0x7f705409b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:21.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.243+0000 7f7059973700 1 -- 172.21.15.104:0/1407069090 shutdown_connections 2023-07-14T11:54:21.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.243+0000 7f7059973700 1 -- 172.21.15.104:0/1407069090 wait complete. 2023-07-14T11:54:21.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7059973700 1 Processor -- start 2023-07-14T11:54:21.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7059973700 1 -- start start 2023-07-14T11:54:21.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7059973700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70540b0ee0 0x7f70540b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:21.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7059973700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70540cd090 0x7f70540cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:21.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7059973700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70540d0610 0x7f70541f0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:21.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7059973700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70540b4cd0 con 0x7f70540b0ee0 2023-07-14T11:54:21.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7059973700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70540cb8a0 con 0x7f70540cd090 2023-07-14T11:54:21.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7059973700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70540cbb50 con 0x7f70540d0610 2023-07-14T11:54:21.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7058971700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70540b0ee0 0x7f70540b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:21.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7058971700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70540b0ee0 0x7f70540b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56888/0 (socket says 172.21.15.104:56888) 2023-07-14T11:54:21.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7058971700 1 -- 172.21.15.104:0/1264677953 learned_addr learned my addr 172.21.15.104:0/1264677953 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:21.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7059172700 1 --2- 172.21.15.104:0/1264677953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70540d0610 0x7f70541f0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:21.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7053fff700 1 --2- 172.21.15.104:0/1264677953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70540cd090 0x7f70540cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:21.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7058971700 1 -- 172.21.15.104:0/1264677953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70540d0610 msgr2=0x7f70541f0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:21.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7058971700 1 --2- 172.21.15.104:0/1264677953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70540d0610 0x7f70541f0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7058971700 1 -- 172.21.15.104:0/1264677953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70540cd090 msgr2=0x7f70540cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:21.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7058971700 1 --2- 172.21.15.104:0/1264677953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70540cd090 0x7f70540cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7058971700 1 -- 172.21.15.104:0/1264677953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70480094b0 con 0x7f70540b0ee0 2023-07-14T11:54:21.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.247+0000 7f7053fff700 1 --2- 172.21.15.104:0/1264677953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70540cd090 0x7f70540cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:21.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.251+0000 7f7058971700 1 --2- 172.21.15.104:0/1264677953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70540b0ee0 0x7f70540b4790 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f704800ebe0 tx=0x7f704800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:21.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.251+0000 7f70517fa700 1 -- 172.21.15.104:0/1264677953 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704800c850 con 0x7f70540b0ee0 2023-07-14T11:54:21.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.251+0000 7f70517fa700 1 -- 172.21.15.104:0/1264677953 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704800c9b0 con 0x7f70540b0ee0 2023-07-14T11:54:21.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.251+0000 7f70517fa700 1 -- 172.21.15.104:0/1264677953 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704800cd40 con 0x7f70540b0ee0 2023-07-14T11:54:21.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.251+0000 7f7059973700 1 -- 172.21.15.104:0/1264677953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70400094b0 con 0x7f70540b0ee0 2023-07-14T11:54:21.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.251+0000 7f7059973700 1 -- 172.21.15.104:0/1264677953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70540cc110 con 0x7f70540b0ee0 2023-07-14T11:54:21.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.251+0000 7f7059973700 1 -- 172.21.15.104:0/1264677953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705404e500 con 0x7f70540b0ee0 2023-07-14T11:54:21.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.255+0000 7f70517fa700 1 -- 172.21.15.104:0/1264677953 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7048013470 con 0x7f70540b0ee0 2023-07-14T11:54:21.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.255+0000 7f70517fa700 1 --2- 172.21.15.104:0/1264677953 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f70380662f0 0x7f70380687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:21.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.255+0000 7f70517fa700 1 -- 172.21.15.104:0/1264677953 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7048084140 con 0x7f70540b0ee0 2023-07-14T11:54:21.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.255+0000 7f7053fff700 1 --2- 172.21.15.104:0/1264677953 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f70380662f0 0x7f70380687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:21.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.255+0000 7f7053fff700 1 --2- 172.21.15.104:0/1264677953 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f70380662f0 0x7f70380687a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7040009bd0 tx=0x7f7040008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:21.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.259+0000 7f70517fa700 1 -- 172.21.15.104:0/1264677953 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f70480511f0 con 0x7f70540b0ee0 2023-07-14T11:54:21.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.435+0000 7f7059973700 1 -- 172.21.15.104:0/1264677953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f70541f1450 con 0x7f70540b0ee0 2023-07-14T11:54:21.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.435+0000 7f70517fa700 1 -- 172.21.15.104:0/1264677953 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f704800cea0 con 0x7f70540b0ee0 2023-07-14T11:54:21.421 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:54:21.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.439+0000 7f7059973700 1 -- 172.21.15.104:0/1264677953 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f70380662f0 msgr2=0x7f70380687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:21.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.439+0000 7f7059973700 1 --2- 172.21.15.104:0/1264677953 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f70380662f0 0x7f70380687a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7040009bd0 tx=0x7f7040008030 comp rx=0 tx=0).stop 2023-07-14T11:54:21.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.439+0000 7f7059973700 1 -- 172.21.15.104:0/1264677953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70540b0ee0 msgr2=0x7f70540b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:21.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.439+0000 7f7059973700 1 --2- 172.21.15.104:0/1264677953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70540b0ee0 0x7f70540b4790 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f704800ebe0 tx=0x7f704800eef0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.439+0000 7f7059973700 1 -- 172.21.15.104:0/1264677953 shutdown_connections 2023-07-14T11:54:21.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.439+0000 7f7059973700 1 --2- 172.21.15.104:0/1264677953 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f70380662f0 0x7f70380687a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.439+0000 7f7059973700 1 --2- 172.21.15.104:0/1264677953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70540d0610 0x7f70541f0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.439+0000 7f7059973700 1 --2- 172.21.15.104:0/1264677953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70540cd090 0x7f70540cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.439+0000 7f7059973700 1 --2- 172.21.15.104:0/1264677953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70540b0ee0 0x7f70540b4790 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.439+0000 7f7059973700 1 -- 172.21.15.104:0/1264677953 >> 172.21.15.104:0/1264677953 conn(0x7f7054060c40 msgr2=0x7f70540cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:21.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.439+0000 7f7059973700 1 -- 172.21.15.104:0/1264677953 shutdown_connections 2023-07-14T11:54:21.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.439+0000 7f7059973700 1 -- 172.21.15.104:0/1264677953 wait complete. 2023-07-14T11:54:21.447 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:54:21.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.635+0000 7f3145400700 1 Processor -- start 2023-07-14T11:54:21.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.635+0000 7f3145400700 1 -- start start 2023-07-14T11:54:21.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f3145400700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3140158a20 0x7f31400cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:21.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f3145400700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f314006c220 0x7f31400cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:21.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f3145400700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31400cb820 0x7f314006d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:21.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f3145400700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314006d310 con 0x7f3140158a20 2023-07-14T11:54:21.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f3145400700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314006c700 con 0x7f314006c220 2023-07-14T11:54:21.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f3145400700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314006c9e0 con 0x7f31400cb820 2023-07-14T11:54:21.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f313effd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3140158a20 0x7f31400cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:21.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f313effd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3140158a20 0x7f31400cb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56896/0 (socket says 172.21.15.104:56896) 2023-07-14T11:54:21.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f313effd700 1 -- 172.21.15.104:0/4126980380 learned_addr learned my addr 172.21.15.104:0/4126980380 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:21.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f313f7fe700 1 --2- 172.21.15.104:0/4126980380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31400cb820 0x7f314006d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:21.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f31367fc700 1 --2- 172.21.15.104:0/4126980380 >> v2:172.21.15.121:3300/0 conn(0x7f314006c220 0x7f31400cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:21.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f313effd700 1 -- 172.21.15.104:0/4126980380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31400cb820 msgr2=0x7f314006d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:21.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f313effd700 1 --2- 172.21.15.104:0/4126980380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31400cb820 0x7f314006d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f313effd700 1 -- 172.21.15.104:0/4126980380 >> v2:172.21.15.121:3300/0 conn(0x7f314006c220 msgr2=0x7f31400cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:21.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f313effd700 1 --2- 172.21.15.104:0/4126980380 >> v2:172.21.15.121:3300/0 conn(0x7f314006c220 0x7f31400cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f313effd700 1 -- 172.21.15.104:0/4126980380 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314006ccc0 con 0x7f3140158a20 2023-07-14T11:54:21.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.639+0000 7f313f7fe700 1 --2- 172.21.15.104:0/4126980380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31400cb820 0x7f314006d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:21.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.643+0000 7f313effd700 1 --2- 172.21.15.104:0/4126980380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3140158a20 0x7f31400cb2e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f314006a630 tx=0x7f3130009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9ca24a825563c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:21.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.643+0000 7f313e7fc700 1 -- 172.21.15.104:0/4126980380 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3130011830 con 0x7f3140158a20 2023-07-14T11:54:21.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.643+0000 7f313e7fc700 1 -- 172.21.15.104:0/4126980380 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3130011990 con 0x7f3140158a20 2023-07-14T11:54:21.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.643+0000 7f313e7fc700 1 -- 172.21.15.104:0/4126980380 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3130011d20 con 0x7f3140158a20 2023-07-14T11:54:21.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.643+0000 7f3145400700 1 -- 172.21.15.104:0/4126980380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3140158a20 msgr2=0x7f31400cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:21.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.643+0000 7f3145400700 1 --2- 172.21.15.104:0/4126980380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3140158a20 0x7f31400cb2e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f314006a630 tx=0x7f3130009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.643+0000 7f3145400700 1 -- 172.21.15.104:0/4126980380 shutdown_connections 2023-07-14T11:54:21.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.643+0000 7f3145400700 1 --2- 172.21.15.104:0/4126980380 >> v2:172.21.15.121:3300/0 conn(0x7f314006c220 0x7f31400cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.643+0000 7f3145400700 1 --2- 172.21.15.104:0/4126980380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31400cb820 0x7f314006d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.643+0000 7f3145400700 1 --2- 172.21.15.104:0/4126980380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3140158a20 0x7f31400cb2e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.643+0000 7f3145400700 1 -- 172.21.15.104:0/4126980380 >> 172.21.15.104:0/4126980380 conn(0x7f31400afc60 msgr2=0x7f31400b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:21.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.643+0000 7f3145400700 1 -- 172.21.15.104:0/4126980380 shutdown_connections 2023-07-14T11:54:21.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.643+0000 7f3145400700 1 -- 172.21.15.104:0/4126980380 wait complete. 2023-07-14T11:54:21.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.647+0000 7f3145400700 1 Processor -- start 2023-07-14T11:54:21.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.647+0000 7f3145400700 1 -- start start 2023-07-14T11:54:21.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.647+0000 7f3145400700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314006c220 0x7f31400b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:21.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.647+0000 7f3145400700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31400cb820 0x7f31400b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:21.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f3145400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3140158a20 0x7f31400c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:21.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f3145400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31400b3f00 con 0x7f3140158a20 2023-07-14T11:54:21.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f3145400700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31400b2920 con 0x7f314006c220 2023-07-14T11:54:21.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f3145400700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31400b2c00 con 0x7f31400cb820 2023-07-14T11:54:21.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f313f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3140158a20 0x7f31400c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:21.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f313f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3140158a20 0x7f31400c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56910/0 (socket says 172.21.15.104:56910) 2023-07-14T11:54:21.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f313f7fe700 1 -- 172.21.15.104:0/3077866785 learned_addr learned my addr 172.21.15.104:0/3077866785 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:21.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f313effd700 1 --2- 172.21.15.104:0/3077866785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314006c220 0x7f31400b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:21.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f31367fc700 1 --2- 172.21.15.104:0/3077866785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31400cb820 0x7f31400b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:21.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f313effd700 1 -- 172.21.15.104:0/3077866785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31400cb820 msgr2=0x7f31400b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:21.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f313effd700 1 --2- 172.21.15.104:0/3077866785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31400cb820 0x7f31400b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f313effd700 1 -- 172.21.15.104:0/3077866785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3140158a20 msgr2=0x7f31400c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:21.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f313effd700 1 --2- 172.21.15.104:0/3077866785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3140158a20 0x7f31400c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f313effd700 1 -- 172.21.15.104:0/3077866785 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3130009570 con 0x7f314006c220 2023-07-14T11:54:21.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f313f7fe700 1 --2- 172.21.15.104:0/3077866785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3140158a20 0x7f31400c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:21.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f313effd700 1 --2- 172.21.15.104:0/3077866785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314006c220 0x7f31400b39c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f314006b0f0 tx=0x7f3130004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:21.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f3137fff700 1 -- 172.21.15.104:0/3077866785 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3130004450 con 0x7f314006c220 2023-07-14T11:54:21.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f3137fff700 1 -- 172.21.15.104:0/3077866785 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313001e070 con 0x7f314006c220 2023-07-14T11:54:21.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f3145400700 1 -- 172.21.15.104:0/3077866785 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31400b2ee0 con 0x7f314006c220 2023-07-14T11:54:21.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f3137fff700 1 -- 172.21.15.104:0/3077866785 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3130003900 con 0x7f314006c220 2023-07-14T11:54:21.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.651+0000 7f3145400700 1 -- 172.21.15.104:0/3077866785 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31400b34a0 con 0x7f314006c220 2023-07-14T11:54:21.641 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.655+0000 7f3145400700 1 -- 172.21.15.104:0/3077866785 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f314004e500 con 0x7f314006c220 2023-07-14T11:54:21.641 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.655+0000 7f3137fff700 1 -- 172.21.15.104:0/3077866785 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3130003a60 con 0x7f314006c220 2023-07-14T11:54:21.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.659+0000 7f3137fff700 1 --2- 172.21.15.104:0/3077866785 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31200662f0 0x7f31200687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:21.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.659+0000 7f3137fff700 1 -- 172.21.15.104:0/3077866785 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3130084a90 con 0x7f314006c220 2023-07-14T11:54:21.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.659+0000 7f31367fc700 1 --2- 172.21.15.104:0/3077866785 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31200662f0 0x7f31200687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:21.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.659+0000 7f31367fc700 1 --2- 172.21.15.104:0/3077866785 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31200662f0 0x7f31200687a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f31401f46b0 tx=0x7f312800d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:21.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.663+0000 7f3137fff700 1 -- 172.21.15.104:0/3077866785 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f3130051cf0 con 0x7f314006c220 2023-07-14T11:54:21.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.835+0000 7f3145400700 1 -- 172.21.15.104:0/3077866785 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31400b40d0 con 0x7f314006c220 2023-07-14T11:54:21.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.839+0000 7f3137fff700 1 -- 172.21.15.104:0/3077866785 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3130055310 con 0x7f314006c220 2023-07-14T11:54:21.823 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:21.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.843+0000 7f3145400700 1 -- 172.21.15.104:0/3077866785 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31200662f0 msgr2=0x7f31200687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:21.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.843+0000 7f3145400700 1 --2- 172.21.15.104:0/3077866785 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31200662f0 0x7f31200687a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f31401f46b0 tx=0x7f312800d040 comp rx=0 tx=0).stop 2023-07-14T11:54:21.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.843+0000 7f3145400700 1 -- 172.21.15.104:0/3077866785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314006c220 msgr2=0x7f31400b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:21.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.843+0000 7f3145400700 1 --2- 172.21.15.104:0/3077866785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314006c220 0x7f31400b39c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f314006b0f0 tx=0x7f3130004770 comp rx=0 tx=0).stop 2023-07-14T11:54:21.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.843+0000 7f3145400700 1 -- 172.21.15.104:0/3077866785 shutdown_connections 2023-07-14T11:54:21.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.843+0000 7f3145400700 1 --2- 172.21.15.104:0/3077866785 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31200662f0 0x7f31200687a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.843+0000 7f3145400700 1 --2- 172.21.15.104:0/3077866785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3140158a20 0x7f31400c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.843+0000 7f3145400700 1 --2- 172.21.15.104:0/3077866785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31400cb820 0x7f31400b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.843+0000 7f3145400700 1 --2- 172.21.15.104:0/3077866785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314006c220 0x7f31400b39c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:21.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.843+0000 7f3145400700 1 -- 172.21.15.104:0/3077866785 >> 172.21.15.104:0/3077866785 conn(0x7f31400afc60 msgr2=0x7f31400c4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:21.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.843+0000 7f3145400700 1 -- 172.21.15.104:0/3077866785 shutdown_connections 2023-07-14T11:54:21.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:21.843+0000 7f3145400700 1 -- 172.21.15.104:0/3077866785 wait complete. 2023-07-14T11:54:21.851 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:54:22.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.051+0000 7f8dc0c3b700 1 Processor -- start 2023-07-14T11:54:22.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.051+0000 7f8dc0c3b700 1 -- start start 2023-07-14T11:54:22.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.051+0000 7f8dc0c3b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8dbc0cf7a0 0x7f8dbc0d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.051+0000 7f8dc0c3b700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dbc0af2e0 con 0x7f8dbc0d21c0 2023-07-14T11:54:22.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.051+0000 7f8dc0c3b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dbc0af5c0 con 0x7f8dbc0b1390 2023-07-14T11:54:22.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.051+0000 7f8dc0c3b700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dbc0d0030 con 0x7f8dbc0cf7a0 2023-07-14T11:54:22.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.051+0000 7f8db9d9b700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f8dbc0d21c0 0x7f8dbc06fd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53692/0 (socket says 172.21.15.104:53692) 2023-07-14T11:54:22.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.051+0000 7f8db9d9b700 1 -- 172.21.15.104:0/3961478301 learned_addr learned my addr 172.21.15.104:0/3961478301 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:22.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.051+0000 7f8dba59c700 1 --2- 172.21.15.104:0/3961478301 >> v2:172.21.15.104:3300/0 conn(0x7f8dbc0cf7a0 0x7f8dbc0d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:22.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.051+0000 7f8db959a700 1 -- 172.21.15.104:0/3961478301 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531118324 0 0) 0x7f8dbc0af2e0 con 0x7f8dbc0d21c0 2023-07-14T11:54:22.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8db959a700 1 -- 172.21.15.104:0/3961478301 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8da40035d0 con 0x7f8dbc0d21c0 2023-07-14T11:54:22.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8db959a700 1 -- 172.21.15.104:0/3961478301 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 172434940 0 0) 0x7f8da40035d0 con 0x7f8dbc0d21c0 2023-07-14T11:54:22.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8dba59c700 1 -- 172.21.15.104:0/3961478301 >> v1:172.21.15.121:6789/0 conn(0x7f8dbc0b1390 legacy=0x7f8dbc0704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8dba59c700 1 -- 172.21.15.104:0/3961478301 >> v1:172.21.15.104:6789/0 conn(0x7f8dbc0d21c0 legacy=0x7f8dbc06fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8dba59c700 1 -- 172.21.15.104:0/3961478301 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc0af8a0 con 0x7f8dbc0cf7a0 2023-07-14T11:54:22.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8dba59c700 1 --2- 172.21.15.104:0/3961478301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0cf7a0 0x7f8dbc0d0210 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009850 tx=0x7f8dac009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95177486199f373 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:22.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8db959a700 1 -- 172.21.15.104:0/3961478301 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0001670 con 0x7f8dbc0cf7a0 2023-07-14T11:54:22.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8db959a700 1 -- 172.21.15.104:0/3961478301 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac004030 con 0x7f8dbc0cf7a0 2023-07-14T11:54:22.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8db959a700 1 -- 172.21.15.104:0/3961478301 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac0043a0 con 0x7f8dbc0cf7a0 2023-07-14T11:54:22.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8dc0c3b700 1 -- 172.21.15.104:0/3961478301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0cf7a0 msgr2=0x7f8dbc0d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8dc0c3b700 1 --2- 172.21.15.104:0/3961478301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0cf7a0 0x7f8dbc0d0210 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009850 tx=0x7f8dac009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:22.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8dc0c3b700 1 -- 172.21.15.104:0/3961478301 shutdown_connections 2023-07-14T11:54:22.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8dc0c3b700 1 --2- 172.21.15.104:0/3961478301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0cf7a0 0x7f8dbc0d0210 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8dc0c3b700 1 -- 172.21.15.104:0/3961478301 >> 172.21.15.104:0/3961478301 conn(0x7f8dbc09b9c0 msgr2=0x7f8dbc0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:22.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8dc0c3b700 1 -- 172.21.15.104:0/3961478301 shutdown_connections 2023-07-14T11:54:22.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.055+0000 7f8dc0c3b700 1 -- 172.21.15.104:0/3961478301 wait complete. 2023-07-14T11:54:22.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.059+0000 7f8dc0c3b700 1 Processor -- start 2023-07-14T11:54:22.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.059+0000 7f8dc0c3b700 1 -- start start 2023-07-14T11:54:22.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dc0c3b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0b1390 0x7f8dbc09cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dc0c3b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dbc0cf7a0 0x7f8dbc0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dc0c3b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dbc0d21c0 0x7f8dbc14a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dc0c3b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc09efc0 con 0x7f8dbc0b1390 2023-07-14T11:54:22.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dc0c3b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc15b330 con 0x7f8dbc0d21c0 2023-07-14T11:54:22.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dc0c3b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc09e970 con 0x7f8dbc0cf7a0 2023-07-14T11:54:22.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dba59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0b1390 0x7f8dbc09cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:22.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dba59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0b1390 0x7f8dbc09cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56934/0 (socket says 172.21.15.104:56934) 2023-07-14T11:54:22.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dba59c700 1 -- 172.21.15.104:0/1565347773 learned_addr learned my addr 172.21.15.104:0/1565347773 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:22.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dba59c700 1 -- 172.21.15.104:0/1565347773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dbc0cf7a0 msgr2=0x7f8dbc0b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dba59c700 1 --2- 172.21.15.104:0/1565347773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dbc0cf7a0 0x7f8dbc0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dba59c700 1 -- 172.21.15.104:0/1565347773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dbc0d21c0 msgr2=0x7f8dbc14a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dba59c700 1 --2- 172.21.15.104:0/1565347773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dbc0d21c0 0x7f8dbc14a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dba59c700 1 -- 172.21.15.104:0/1565347773 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac0094b0 con 0x7f8dbc0b1390 2023-07-14T11:54:22.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dba59c700 1 --2- 172.21.15.104:0/1565347773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0b1390 0x7f8dbc09cd20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0090b0 tx=0x7f8dac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:22.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8daaffd700 1 -- 172.21.15.104:0/1565347773 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac003ea0 con 0x7f8dbc0b1390 2023-07-14T11:54:22.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8daaffd700 1 -- 172.21.15.104:0/1565347773 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac01f070 con 0x7f8dbc0b1390 2023-07-14T11:54:22.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dc0c3b700 1 -- 172.21.15.104:0/1565347773 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dbc09ec50 con 0x7f8dbc0b1390 2023-07-14T11:54:22.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8daaffd700 1 -- 172.21.15.104:0/1565347773 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac011a90 con 0x7f8dbc0b1390 2023-07-14T11:54:22.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dc0c3b700 1 -- 172.21.15.104:0/1565347773 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dbc09e590 con 0x7f8dbc0b1390 2023-07-14T11:54:22.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.063+0000 7f8dc0c3b700 1 -- 172.21.15.104:0/1565347773 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dbc15b4f0 con 0x7f8dbc0b1390 2023-07-14T11:54:22.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.067+0000 7f8daaffd700 1 -- 172.21.15.104:0/1565347773 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8dac011bf0 con 0x7f8dbc0b1390 2023-07-14T11:54:22.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.067+0000 7f8daaffd700 1 --2- 172.21.15.104:0/1565347773 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8d9c0662f0 0x7f8d9c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.067+0000 7f8daaffd700 1 -- 172.21.15.104:0/1565347773 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8dac021070 con 0x7f8dbc0b1390 2023-07-14T11:54:22.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.067+0000 7f8db9d9b700 1 --2- 172.21.15.104:0/1565347773 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8d9c0662f0 0x7f8d9c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:22.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.067+0000 7f8db9d9b700 1 --2- 172.21.15.104:0/1565347773 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8d9c0662f0 0x7f8d9c0687a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8db0000c00 tx=0x7f8db00010e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:22.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.075+0000 7f8daaffd700 1 -- 172.21.15.104:0/1565347773 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f8dac052c30 con 0x7f8dbc0b1390 2023-07-14T11:54:22.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.243+0000 7f8dc0c3b700 1 -- 172.21.15.104:0/1565347773 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8dbc04dcb0 con 0x7f8dbc0b1390 2023-07-14T11:54:22.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.243+0000 7f8daaffd700 1 -- 172.21.15.104:0/1565347773 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8dac056250 con 0x7f8dbc0b1390 2023-07-14T11:54:22.229 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:54:22.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.247+0000 7f8dc0c3b700 1 -- 172.21.15.104:0/1565347773 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8d9c0662f0 msgr2=0x7f8d9c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.247+0000 7f8dc0c3b700 1 --2- 172.21.15.104:0/1565347773 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8d9c0662f0 0x7f8d9c0687a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8db0000c00 tx=0x7f8db00010e0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.247+0000 7f8dc0c3b700 1 -- 172.21.15.104:0/1565347773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0b1390 msgr2=0x7f8dbc09cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.247+0000 7f8dc0c3b700 1 --2- 172.21.15.104:0/1565347773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0b1390 0x7f8dbc09cd20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0090b0 tx=0x7f8dac004770 comp rx=0 tx=0).stop 2023-07-14T11:54:22.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.247+0000 7f8dc0c3b700 1 -- 172.21.15.104:0/1565347773 shutdown_connections 2023-07-14T11:54:22.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.247+0000 7f8dc0c3b700 1 --2- 172.21.15.104:0/1565347773 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8d9c0662f0 0x7f8d9c0687a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.247+0000 7f8dc0c3b700 1 --2- 172.21.15.104:0/1565347773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dbc0d21c0 0x7f8dbc14a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.247+0000 7f8dc0c3b700 1 --2- 172.21.15.104:0/1565347773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dbc0cf7a0 0x7f8dbc0b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.247+0000 7f8dc0c3b700 1 --2- 172.21.15.104:0/1565347773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0b1390 0x7f8dbc09cd20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.247+0000 7f8dc0c3b700 1 -- 172.21.15.104:0/1565347773 >> 172.21.15.104:0/1565347773 conn(0x7f8dbc09b9c0 msgr2=0x7f8dbc0b3610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:22.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.247+0000 7f8dc0c3b700 1 -- 172.21.15.104:0/1565347773 shutdown_connections 2023-07-14T11:54:22.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.247+0000 7f8dc0c3b700 1 -- 172.21.15.104:0/1565347773 wait complete. 2023-07-14T11:54:22.257 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:54:22.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.443+0000 7fe98359e700 1 Processor -- start 2023-07-14T11:54:22.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.443+0000 7fe98359e700 1 -- start start 2023-07-14T11:54:22.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.447+0000 7fe98359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe98406e960 0x7fe98406a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.447+0000 7fe98359e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe98406ed30 0x7fe9840ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.447+0000 7fe98359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe984069ff0 con 0x7fe9840ce7e0 2023-07-14T11:54:22.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.447+0000 7fe98359e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9840ceff0 con 0x7fe98406ed30 2023-07-14T11:54:22.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.447+0000 7fe98359e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840ced10 con 0x7fe98406e960 2023-07-14T11:54:22.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.447+0000 7fe981d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe98406ed30 0x7fe9840ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:22.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.447+0000 7fe981d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe98406ed30 0x7fe9840ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:43412/0 (socket says 172.21.15.104:43412) 2023-07-14T11:54:22.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.447+0000 7fe981d9b700 1 -- 172.21.15.104:0/2371781737 learned_addr learned my addr 172.21.15.104:0/2371781737 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:22.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.447+0000 7fe98259c700 1 --2- 172.21.15.104:0/2371781737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe98406e960 0x7fe98406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:22.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe981d9b700 1 -- 172.21.15.104:0/2371781737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe98406e960 msgr2=0x7fe98406a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:22.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe981d9b700 1 --2- 172.21.15.104:0/2371781737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe98406e960 0x7fe98406a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe981d9b700 1 -- 172.21.15.104:0/2371781737 >> v1:172.21.15.121:6789/0 conn(0x7fe9840ce7e0 legacy=0x7fe9840cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe981d9b700 1 -- 172.21.15.104:0/2371781737 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9840699a0 con 0x7fe98406ed30 2023-07-14T11:54:22.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe981d9b700 1 --2- 172.21.15.104:0/2371781737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98406ed30 0x7fe9840ce2a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe978009910 tx=0x7fe978009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6a87559e0c2771a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:22.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe98159a700 1 -- 172.21.15.104:0/2371781737 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978011830 con 0x7fe98406ed30 2023-07-14T11:54:22.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe98159a700 1 -- 172.21.15.104:0/2371781737 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe978011990 con 0x7fe98406ed30 2023-07-14T11:54:22.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe98159a700 1 -- 172.21.15.104:0/2371781737 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978011d40 con 0x7fe98406ed30 2023-07-14T11:54:22.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe98359e700 1 -- 172.21.15.104:0/2371781737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98406ed30 msgr2=0x7fe9840ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe98359e700 1 --2- 172.21.15.104:0/2371781737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98406ed30 0x7fe9840ce2a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe978009910 tx=0x7fe978009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:22.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe98359e700 1 -- 172.21.15.104:0/2371781737 shutdown_connections 2023-07-14T11:54:22.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe98359e700 1 --2- 172.21.15.104:0/2371781737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98406ed30 0x7fe9840ce2a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe98359e700 1 --2- 172.21.15.104:0/2371781737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe98406e960 0x7fe98406a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe98359e700 1 -- 172.21.15.104:0/2371781737 >> 172.21.15.104:0/2371781737 conn(0x7fe98406fe60 msgr2=0x7fe984070260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:22.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe98359e700 1 -- 172.21.15.104:0/2371781737 shutdown_connections 2023-07-14T11:54:22.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.451+0000 7fe98359e700 1 -- 172.21.15.104:0/2371781737 wait complete. 2023-07-14T11:54:22.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe98359e700 1 Processor -- start 2023-07-14T11:54:22.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe98359e700 1 -- start start 2023-07-14T11:54:22.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe98359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98406e960 0x7fe9841552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe98359e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe98406ed30 0x7fe9840bf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe98359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9840ce7e0 0x7fe9840bfb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe98359e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840beab0 con 0x7fe98406ed30 2023-07-14T11:54:22.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe98359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840bed90 con 0x7fe98406e960 2023-07-14T11:54:22.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe98359e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840c2dd0 con 0x7fe9840ce7e0 2023-07-14T11:54:22.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe981d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe98406ed30 0x7fe9840bf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:22.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe981d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe98406ed30 0x7fe9840bf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56948/0 (socket says 172.21.15.104:56948) 2023-07-14T11:54:22.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe981d9b700 1 -- 172.21.15.104:0/2507883780 learned_addr learned my addr 172.21.15.104:0/2507883780 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:22.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe98259c700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98406e960 0x7fe9841552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:22.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe982d9d700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9840ce7e0 0x7fe9840bfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:22.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe981d9b700 1 -- 172.21.15.104:0/2507883780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9840ce7e0 msgr2=0x7fe9840bfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe981d9b700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9840ce7e0 0x7fe9840bfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe981d9b700 1 -- 172.21.15.104:0/2507883780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98406e960 msgr2=0x7fe9841552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe981d9b700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98406e960 0x7fe9841552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe981d9b700 1 -- 172.21.15.104:0/2507883780 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe978009570 con 0x7fe98406ed30 2023-07-14T11:54:22.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe982d9d700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9840ce7e0 0x7fe9840bfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:22.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.455+0000 7fe98259c700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98406e960 0x7fe9841552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:22.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.459+0000 7fe981d9b700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe98406ed30 0x7fe9840bf5e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe978000c00 tx=0x7fe978004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:22.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.459+0000 7fe972ffd700 1 -- 172.21.15.104:0/2507883780 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978004610 con 0x7fe98406ed30 2023-07-14T11:54:22.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.459+0000 7fe972ffd700 1 -- 172.21.15.104:0/2507883780 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9780036c0 con 0x7fe98406ed30 2023-07-14T11:54:22.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.459+0000 7fe972ffd700 1 -- 172.21.15.104:0/2507883780 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978003a50 con 0x7fe98406ed30 2023-07-14T11:54:22.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.459+0000 7fe98359e700 1 -- 172.21.15.104:0/2507883780 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98406f300 con 0x7fe98406ed30 2023-07-14T11:54:22.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.459+0000 7fe98359e700 1 -- 172.21.15.104:0/2507883780 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9840c3390 con 0x7fe98406ed30 2023-07-14T11:54:22.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.459+0000 7fe98359e700 1 -- 172.21.15.104:0/2507883780 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98404e500 con 0x7fe98406ed30 2023-07-14T11:54:22.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.463+0000 7fe972ffd700 1 -- 172.21.15.104:0/2507883780 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe978003bb0 con 0x7fe98406ed30 2023-07-14T11:54:22.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.463+0000 7fe972ffd700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe9640662f0 0x7fe9640687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.463+0000 7fe972ffd700 1 -- 172.21.15.104:0/2507883780 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe978021030 con 0x7fe98406ed30 2023-07-14T11:54:22.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.463+0000 7fe98259c700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe9640662f0 0x7fe9640687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:22.449 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.463+0000 7fe98259c700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe9640662f0 0x7fe9640687a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe974009c40 tx=0x7fe974008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:22.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.467+0000 7fe972ffd700 1 -- 172.21.15.104:0/2507883780 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe978052920 con 0x7fe98406ed30 2023-07-14T11:54:22.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.639+0000 7fe98359e700 1 -- 172.21.15.104:0/2507883780 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe98406d5f0 con 0x7fe98406ed30 2023-07-14T11:54:22.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.639+0000 7fe972ffd700 1 -- 172.21.15.104:0/2507883780 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe978003e60 con 0x7fe98406ed30 2023-07-14T11:54:22.626 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:22.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.643+0000 7fe98359e700 1 -- 172.21.15.104:0/2507883780 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe9640662f0 msgr2=0x7fe9640687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.643+0000 7fe98359e700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe9640662f0 0x7fe9640687a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe974009c40 tx=0x7fe974008040 comp rx=0 tx=0).stop 2023-07-14T11:54:22.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.643+0000 7fe98359e700 1 -- 172.21.15.104:0/2507883780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe98406ed30 msgr2=0x7fe9840bf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.643+0000 7fe98359e700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe98406ed30 0x7fe9840bf5e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe978000c00 tx=0x7fe978004770 comp rx=0 tx=0).stop 2023-07-14T11:54:22.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.647+0000 7fe98359e700 1 -- 172.21.15.104:0/2507883780 shutdown_connections 2023-07-14T11:54:22.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.647+0000 7fe98359e700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe9640662f0 0x7fe9640687a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.647+0000 7fe98359e700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe98406ed30 0x7fe9840bf5e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.647+0000 7fe98359e700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9840ce7e0 0x7fe9840bfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.647+0000 7fe98359e700 1 --2- 172.21.15.104:0/2507883780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98406e960 0x7fe9841552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.647+0000 7fe98359e700 1 -- 172.21.15.104:0/2507883780 >> 172.21.15.104:0/2507883780 conn(0x7fe98406fe60 msgr2=0x7fe98415e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:22.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.647+0000 7fe98359e700 1 -- 172.21.15.104:0/2507883780 shutdown_connections 2023-07-14T11:54:22.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.647+0000 7fe98359e700 1 -- 172.21.15.104:0/2507883780 wait complete. 2023-07-14T11:54:22.652 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:54:22.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.839+0000 7f7e1548c700 1 Processor -- start 2023-07-14T11:54:22.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.839+0000 7f7e1548c700 1 -- start start 2023-07-14T11:54:22.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e1548c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7e100b1a50 0x7f7e101600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e1548c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7e100cca70 0x7f7e100cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e1548c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e100d0530 0x7f7e1015c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e1548c700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e100ce4f0 con 0x7f7e100b1a50 2023-07-14T11:54:22.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e1548c700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e100d0930 con 0x7f7e100cca70 2023-07-14T11:54:22.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e1548c700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e101605e0 con 0x7f7e100d0530 2023-07-14T11:54:22.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e0ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7e100b1a50 0x7f7e101600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:22.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e0ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7e100b1a50 0x7f7e101600a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56952/0 (socket says 172.21.15.104:56952) 2023-07-14T11:54:22.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e0ffff700 1 -- 172.21.15.104:0/536623022 learned_addr learned my addr 172.21.15.104:0/536623022 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:22.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e0ffff700 1 -- 172.21.15.104:0/536623022 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e100d0530 msgr2=0x7f7e1015c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:22.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e0ffff700 1 --2- 172.21.15.104:0/536623022 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e100d0530 0x7f7e1015c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e0ffff700 1 -- 172.21.15.104:0/536623022 >> v2:172.21.15.121:3300/0 conn(0x7f7e100cca70 msgr2=0x7f7e100cfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:22.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e0ffff700 1 --2- 172.21.15.104:0/536623022 >> v2:172.21.15.121:3300/0 conn(0x7f7e100cca70 0x7f7e100cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e0ffff700 1 -- 172.21.15.104:0/536623022 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e100cf7a0 con 0x7f7e100b1a50 2023-07-14T11:54:22.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e0ffff700 1 --2- 172.21.15.104:0/536623022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e100b1a50 0x7f7e101600a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7e100b4410 tx=0x7f7e00009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2edd41b4c62249 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:22.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e0f7fe700 1 -- 172.21.15.104:0/536623022 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0000b8a0 con 0x7f7e100b1a50 2023-07-14T11:54:22.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e0f7fe700 1 -- 172.21.15.104:0/536623022 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0000ba00 con 0x7f7e100b1a50 2023-07-14T11:54:22.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.843+0000 7f7e0f7fe700 1 -- 172.21.15.104:0/536623022 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0000bd90 con 0x7f7e100b1a50 2023-07-14T11:54:22.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.847+0000 7f7e1548c700 1 -- 172.21.15.104:0/536623022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e100b1a50 msgr2=0x7f7e101600a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.847+0000 7f7e1548c700 1 --2- 172.21.15.104:0/536623022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e100b1a50 0x7f7e101600a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7e100b4410 tx=0x7f7e00009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.847+0000 7f7e1548c700 1 -- 172.21.15.104:0/536623022 shutdown_connections 2023-07-14T11:54:22.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.847+0000 7f7e1548c700 1 --2- 172.21.15.104:0/536623022 >> v2:172.21.15.121:3300/0 conn(0x7f7e100cca70 0x7f7e100cfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.847+0000 7f7e1548c700 1 --2- 172.21.15.104:0/536623022 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e100d0530 0x7f7e1015c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.847+0000 7f7e1548c700 1 --2- 172.21.15.104:0/536623022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e100b1a50 0x7f7e101600a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.847+0000 7f7e1548c700 1 -- 172.21.15.104:0/536623022 >> 172.21.15.104:0/536623022 conn(0x7f7e10060320 msgr2=0x7f7e100606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:22.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.847+0000 7f7e1548c700 1 -- 172.21.15.104:0/536623022 shutdown_connections 2023-07-14T11:54:22.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.847+0000 7f7e1548c700 1 -- 172.21.15.104:0/536623022 wait complete. 2023-07-14T11:54:22.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.847+0000 7f7e1548c700 1 Processor -- start 2023-07-14T11:54:22.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e1548c700 1 -- start start 2023-07-14T11:54:22.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e1548c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e100b1a50 0x7f7e1015db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e1548c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e100cca70 0x7f7e101f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e1548c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e100d0530 0x7f7e101e72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e1548c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1015d760 con 0x7f7e100cca70 2023-07-14T11:54:22.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e1548c700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e101511c0 con 0x7f7e100b1a50 2023-07-14T11:54:22.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e1548c700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1015cae0 con 0x7f7e100d0530 2023-07-14T11:54:22.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e077fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e100cca70 0x7f7e101f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:22.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e077fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e100cca70 0x7f7e101f0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56960/0 (socket says 172.21.15.104:56960) 2023-07-14T11:54:22.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e077fe700 1 -- 172.21.15.104:0/3546357909 learned_addr learned my addr 172.21.15.104:0/3546357909 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:22.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e0ffff700 1 --2- 172.21.15.104:0/3546357909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e100b1a50 0x7f7e1015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:22.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e14c8b700 1 --2- 172.21.15.104:0/3546357909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e100d0530 0x7f7e101e72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:22.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e077fe700 1 -- 172.21.15.104:0/3546357909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e100d0530 msgr2=0x7f7e101e72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e077fe700 1 --2- 172.21.15.104:0/3546357909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e100d0530 0x7f7e101e72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e077fe700 1 -- 172.21.15.104:0/3546357909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e100b1a50 msgr2=0x7f7e1015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:22.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e077fe700 1 --2- 172.21.15.104:0/3546357909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e100b1a50 0x7f7e1015db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:22.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e077fe700 1 -- 172.21.15.104:0/3546357909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e00009570 con 0x7f7e100cca70 2023-07-14T11:54:22.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.851+0000 7f7e077fe700 1 --2- 172.21.15.104:0/3546357909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e100cca70 0x7f7e101f0d50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7df800b580 tx=0x7f7df800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:22.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.855+0000 7f7e0d7fa700 1 -- 172.21.15.104:0/3546357909 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8011830 con 0x7f7e100cca70 2023-07-14T11:54:22.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.855+0000 7f7e0d7fa700 1 -- 172.21.15.104:0/3546357909 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df800be90 con 0x7f7e100cca70 2023-07-14T11:54:22.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.855+0000 7f7e0d7fa700 1 -- 172.21.15.104:0/3546357909 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8011bd0 con 0x7f7e100cca70 2023-07-14T11:54:22.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.855+0000 7f7e1548c700 1 -- 172.21.15.104:0/3546357909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e1015cdc0 con 0x7f7e100cca70 2023-07-14T11:54:22.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.855+0000 7f7e1548c700 1 -- 172.21.15.104:0/3546357909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1015d380 con 0x7f7e100cca70 2023-07-14T11:54:22.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.855+0000 7f7e1548c700 1 -- 172.21.15.104:0/3546357909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e1015d8a0 con 0x7f7e100cca70 2023-07-14T11:54:22.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.855+0000 7f7e0d7fa700 1 -- 172.21.15.104:0/3546357909 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7df8011d30 con 0x7f7e100cca70 2023-07-14T11:54:22.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.859+0000 7f7e0d7fa700 1 --2- 172.21.15.104:0/3546357909 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7df00662f0 0x7f7df00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:22.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.859+0000 7f7e0d7fa700 1 -- 172.21.15.104:0/3546357909 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7df8083de0 con 0x7f7e100cca70 2023-07-14T11:54:22.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.859+0000 7f7e0ffff700 1 --2- 172.21.15.104:0/3546357909 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7df00662f0 0x7f7df00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:22.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.859+0000 7f7e0ffff700 1 --2- 172.21.15.104:0/3546357909 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7df00662f0 0x7f7df00687a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7e00000c00 tx=0x7f7e00008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:22.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:22.867+0000 7f7e0d7fa700 1 -- 172.21.15.104:0/3546357909 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f7e1015d8a0 con 0x7f7e100cca70 2023-07-14T11:54:23.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.051+0000 7f7e1548c700 1 -- 172.21.15.104:0/3546357909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7e1015d8a0 con 0x7f7e100cca70 2023-07-14T11:54:23.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.051+0000 7f7e0d7fa700 1 -- 172.21.15.104:0/3546357909 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7e1015d8a0 con 0x7f7e100cca70 2023-07-14T11:54:23.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.055+0000 7f7e1548c700 1 -- 172.21.15.104:0/3546357909 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7df00662f0 msgr2=0x7f7df00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:23.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.055+0000 7f7e1548c700 1 --2- 172.21.15.104:0/3546357909 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7df00662f0 0x7f7df00687a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7e00000c00 tx=0x7f7e00008030 comp rx=0 tx=0).stop 2023-07-14T11:54:23.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.055+0000 7f7e1548c700 1 -- 172.21.15.104:0/3546357909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e100cca70 msgr2=0x7f7e101f0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:23.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.055+0000 7f7e1548c700 1 --2- 172.21.15.104:0/3546357909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e100cca70 0x7f7e101f0d50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7df800b580 tx=0x7f7df800b890 comp rx=0 tx=0).stop 2023-07-14T11:54:23.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.055+0000 7f7e1548c700 1 -- 172.21.15.104:0/3546357909 shutdown_connections 2023-07-14T11:54:23.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.055+0000 7f7e1548c700 1 --2- 172.21.15.104:0/3546357909 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7df00662f0 0x7f7df00687a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.055+0000 7f7e1548c700 1 --2- 172.21.15.104:0/3546357909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e100cca70 0x7f7e101f0d50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.055+0000 7f7e1548c700 1 --2- 172.21.15.104:0/3546357909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e100d0530 0x7f7e101e72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.055+0000 7f7e1548c700 1 --2- 172.21.15.104:0/3546357909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e100b1a50 0x7f7e1015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.055+0000 7f7e1548c700 1 -- 172.21.15.104:0/3546357909 >> 172.21.15.104:0/3546357909 conn(0x7f7e10060320 msgr2=0x7f7e100b5250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:23.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.055+0000 7f7e1548c700 1 -- 172.21.15.104:0/3546357909 shutdown_connections 2023-07-14T11:54:23.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.055+0000 7f7e1548c700 1 -- 172.21.15.104:0/3546357909 wait complete. 2023-07-14T11:54:23.045 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:54:23.065 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi104.front.sepia.ceph.com... 2023-07-14T11:54:23.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:23.066 DEBUG:teuthology.orchestra.run.smithi104:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-14T11:54:23.100 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:54:23.100 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:54:23.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:23.101 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-14T11:54:23.166 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:54:23.166 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:54:23.185 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:54:23.186 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:54:23.246 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:23.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.439+0000 7fc2e8746700 1 Processor -- start 2023-07-14T11:54:23.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.439+0000 7fc2e8746700 1 -- start start 2023-07-14T11:54:23.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.439+0000 7fc2e8746700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc2e006d860 0x7fc2e00cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:23.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.439+0000 7fc2e8746700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc2e006be70 0x7fc2e006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:23.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.439+0000 7fc2e8746700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2e00c1e00 0x7fc2e00cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:23.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.439+0000 7fc2e8746700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e00cc1b0 con 0x7fc2e006be70 2023-07-14T11:54:23.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.439+0000 7fc2e8746700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e006e170 con 0x7fc2e006d860 2023-07-14T11:54:23.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.439+0000 7fc2e8746700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00cd690 con 0x7fc2e00c1e00 2023-07-14T11:54:23.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.439+0000 7fc2e5ce1700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc2e006be70 0x7fc2e006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:23.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.439+0000 7fc2e5ce1700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc2e006be70 0x7fc2e006dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56976/0 (socket says 172.21.15.104:56976) 2023-07-14T11:54:23.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.439+0000 7fc2e5ce1700 1 -- 172.21.15.104:0/1275394174 learned_addr learned my addr 172.21.15.104:0/1275394174 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:23.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.439+0000 7fc2e64e2700 1 --2- 172.21.15.104:0/1275394174 >> v2:172.21.15.121:3300/0 conn(0x7fc2e006d860 0x7fc2e00cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:23.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.439+0000 7fc2e6ce3700 1 --2- 172.21.15.104:0/1275394174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2e00c1e00 0x7fc2e00cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:23.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e5ce1700 1 -- 172.21.15.104:0/1275394174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2e00c1e00 msgr2=0x7fc2e00cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:23.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e5ce1700 1 --2- 172.21.15.104:0/1275394174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2e00c1e00 0x7fc2e00cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e5ce1700 1 -- 172.21.15.104:0/1275394174 >> v2:172.21.15.121:3300/0 conn(0x7fc2e006d860 msgr2=0x7fc2e00cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:23.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e5ce1700 1 --2- 172.21.15.104:0/1275394174 >> v2:172.21.15.121:3300/0 conn(0x7fc2e006d860 0x7fc2e00cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e5ce1700 1 -- 172.21.15.104:0/1275394174 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e00cd970 con 0x7fc2e006be70 2023-07-14T11:54:23.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e5ce1700 1 --2- 172.21.15.104:0/1275394174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2e006be70 0x7fc2e006dc30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc2e006c410 tx=0x7fc2d0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7222673518c3b1a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:23.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e54e0700 1 -- 172.21.15.104:0/1275394174 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0011830 con 0x7fc2e006be70 2023-07-14T11:54:23.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e54e0700 1 -- 172.21.15.104:0/1275394174 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d0011990 con 0x7fc2e006be70 2023-07-14T11:54:23.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e54e0700 1 -- 172.21.15.104:0/1275394174 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0011d20 con 0x7fc2e006be70 2023-07-14T11:54:23.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e8746700 1 -- 172.21.15.104:0/1275394174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2e006be70 msgr2=0x7fc2e006dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:23.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e8746700 1 --2- 172.21.15.104:0/1275394174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2e006be70 0x7fc2e006dc30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc2e006c410 tx=0x7fc2d0009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e8746700 1 -- 172.21.15.104:0/1275394174 shutdown_connections 2023-07-14T11:54:23.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e8746700 1 --2- 172.21.15.104:0/1275394174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2e00c1e00 0x7fc2e00cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e8746700 1 --2- 172.21.15.104:0/1275394174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2e006be70 0x7fc2e006dc30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e8746700 1 --2- 172.21.15.104:0/1275394174 >> v2:172.21.15.121:3300/0 conn(0x7fc2e006d860 0x7fc2e00cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e8746700 1 -- 172.21.15.104:0/1275394174 >> 172.21.15.104:0/1275394174 conn(0x7fc2e00cad30 msgr2=0x7fc2e00cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:23.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e8746700 1 -- 172.21.15.104:0/1275394174 shutdown_connections 2023-07-14T11:54:23.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.443+0000 7fc2e8746700 1 -- 172.21.15.104:0/1275394174 wait complete. 2023-07-14T11:54:23.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e8746700 1 Processor -- start 2023-07-14T11:54:23.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e8746700 1 -- start start 2023-07-14T11:54:23.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e8746700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2e006be70 0x7fc2e0156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:23.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e8746700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2e006d860 0x7fc2e01f1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:23.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e8746700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2e00c1e00 0x7fc2e015b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:23.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e8746700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e0156c80 con 0x7fc2e006be70 2023-07-14T11:54:23.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e8746700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e01f24f0 con 0x7fc2e00c1e00 2023-07-14T11:54:23.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e8746700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e01f1f30 con 0x7fc2e006d860 2023-07-14T11:54:23.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e64e2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2e006be70 0x7fc2e0156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:23.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e64e2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2e006be70 0x7fc2e0156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56978/0 (socket says 172.21.15.104:56978) 2023-07-14T11:54:23.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e64e2700 1 -- 172.21.15.104:0/2608945776 learned_addr learned my addr 172.21.15.104:0/2608945776 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:23.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e6ce3700 1 --2- 172.21.15.104:0/2608945776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2e00c1e00 0x7fc2e015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:23.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e5ce1700 1 --2- 172.21.15.104:0/2608945776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2e006d860 0x7fc2e01f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:23.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e64e2700 1 -- 172.21.15.104:0/2608945776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2e006d860 msgr2=0x7fc2e01f1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:23.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e64e2700 1 --2- 172.21.15.104:0/2608945776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2e006d860 0x7fc2e01f1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e64e2700 1 -- 172.21.15.104:0/2608945776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2e00c1e00 msgr2=0x7fc2e015b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:23.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e64e2700 1 --2- 172.21.15.104:0/2608945776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2e00c1e00 0x7fc2e015b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e64e2700 1 -- 172.21.15.104:0/2608945776 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d0009570 con 0x7fc2e006be70 2023-07-14T11:54:23.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.447+0000 7fc2e64e2700 1 --2- 172.21.15.104:0/2608945776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2e006be70 0x7fc2e0156740 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00e940 tx=0x7fc2dc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:23.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.451+0000 7fc2d6ffd700 1 -- 172.21.15.104:0/2608945776 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc013680 con 0x7fc2e006be70 2023-07-14T11:54:23.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.451+0000 7fc2d6ffd700 1 -- 172.21.15.104:0/2608945776 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2dc0137e0 con 0x7fc2e006be70 2023-07-14T11:54:23.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.451+0000 7fc2e8746700 1 -- 172.21.15.104:0/2608945776 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e01f2210 con 0x7fc2e006be70 2023-07-14T11:54:23.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.451+0000 7fc2d6ffd700 1 -- 172.21.15.104:0/2608945776 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc013b70 con 0x7fc2e006be70 2023-07-14T11:54:23.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.451+0000 7fc2e8746700 1 -- 172.21.15.104:0/2608945776 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e00c54d0 con 0x7fc2e006be70 2023-07-14T11:54:23.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.451+0000 7fc2e8746700 1 -- 172.21.15.104:0/2608945776 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e004dcb0 con 0x7fc2e006be70 2023-07-14T11:54:23.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.451+0000 7fc2d6ffd700 1 -- 172.21.15.104:0/2608945776 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc2dc013cd0 con 0x7fc2e006be70 2023-07-14T11:54:23.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.455+0000 7fc2d6ffd700 1 --2- 172.21.15.104:0/2608945776 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc2c40662f0 0x7fc2c40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:23.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.455+0000 7fc2d6ffd700 1 -- 172.21.15.104:0/2608945776 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc2dc083ff0 con 0x7fc2e006be70 2023-07-14T11:54:23.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.455+0000 7fc2e5ce1700 1 --2- 172.21.15.104:0/2608945776 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc2c40662f0 0x7fc2c40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:23.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.455+0000 7fc2e5ce1700 1 --2- 172.21.15.104:0/2608945776 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc2c40662f0 0x7fc2c40687a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009860 tx=0x7fc2d0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:23.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.459+0000 7fc2d6ffd700 1 -- 172.21.15.104:0/2608945776 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fc2dc054b90 con 0x7fc2e006be70 2023-07-14T11:54:23.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.631+0000 7fc2e8746700 1 -- 172.21.15.104:0/2608945776 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2e00c7240 con 0x7fc2e006be70 2023-07-14T11:54:23.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.635+0000 7fc2d6ffd700 1 -- 172.21.15.104:0/2608945776 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20251 (secure 0 0 0) 0x7fc2dc054720 con 0x7fc2e006be70 2023-07-14T11:54:23.621 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:23.621 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":30,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:55.601783+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:23.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.639+0000 7fc2e8746700 1 -- 172.21.15.104:0/2608945776 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc2c40662f0 msgr2=0x7fc2c40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:23.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.639+0000 7fc2e8746700 1 --2- 172.21.15.104:0/2608945776 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc2c40662f0 0x7fc2c40687a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009860 tx=0x7fc2d0008030 comp rx=0 tx=0).stop 2023-07-14T11:54:23.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.639+0000 7fc2e8746700 1 -- 172.21.15.104:0/2608945776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2e006be70 msgr2=0x7fc2e0156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:23.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.639+0000 7fc2e8746700 1 --2- 172.21.15.104:0/2608945776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2e006be70 0x7fc2e0156740 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00e940 tx=0x7fc2dc00ec50 comp rx=0 tx=0).stop 2023-07-14T11:54:23.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.639+0000 7fc2e8746700 1 -- 172.21.15.104:0/2608945776 shutdown_connections 2023-07-14T11:54:23.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.639+0000 7fc2e8746700 1 --2- 172.21.15.104:0/2608945776 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc2c40662f0 0x7fc2c40687a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.639+0000 7fc2e8746700 1 --2- 172.21.15.104:0/2608945776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2e00c1e00 0x7fc2e015b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.639+0000 7fc2e8746700 1 --2- 172.21.15.104:0/2608945776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2e006d860 0x7fc2e01f1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.639+0000 7fc2e8746700 1 --2- 172.21.15.104:0/2608945776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2e006be70 0x7fc2e0156740 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.639+0000 7fc2e8746700 1 -- 172.21.15.104:0/2608945776 >> 172.21.15.104:0/2608945776 conn(0x7fc2e00cad30 msgr2=0x7fc2e00ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:23.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.639+0000 7fc2e8746700 1 -- 172.21.15.104:0/2608945776 shutdown_connections 2023-07-14T11:54:23.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.639+0000 7fc2e8746700 1 -- 172.21.15.104:0/2608945776 wait complete. 2023-07-14T11:54:23.648 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:54:23.649 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:54:23.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.839+0000 7f5195871700 1 Processor -- start 2023-07-14T11:54:23.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.839+0000 7f5195871700 1 -- start start 2023-07-14T11:54:23.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f5195871700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f51901597a0 0x7f51900c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:23.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f5195871700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5190071560 con 0x7f5190159b70 2023-07-14T11:54:23.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f5195871700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5190070a60 con 0x7f51900705e0 2023-07-14T11:54:23.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f5195871700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51900cbbc0 con 0x7f51901597a0 2023-07-14T11:54:23.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f518e7fc700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f5190159b70 0x7f51900cada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53708/0 (socket says 172.21.15.104:53708) 2023-07-14T11:54:23.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f518e7fc700 1 -- 172.21.15.104:0/4225299335 learned_addr learned my addr 172.21.15.104:0/4225299335 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:23.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f518effd700 1 --2- 172.21.15.104:0/4225299335 >> v2:172.21.15.121:3300/0 conn(0x7f51901597a0 0x7f51900c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:23.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f518dffb700 1 -- 172.21.15.104:0/4225299335 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100768407 0 0) 0x7f5190071560 con 0x7f5190159b70 2023-07-14T11:54:23.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f518dffb700 1 -- 172.21.15.104:0/4225299335 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51780035d0 con 0x7f5190159b70 2023-07-14T11:54:23.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f518dffb700 1 -- 172.21.15.104:0/4225299335 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363510647 0 0) 0x7f5190070a60 con 0x7f51900705e0 2023-07-14T11:54:23.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f518dffb700 1 -- 172.21.15.104:0/4225299335 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5190071560 con 0x7f51900705e0 2023-07-14T11:54:23.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f518effd700 1 -- 172.21.15.104:0/4225299335 >> v1:172.21.15.121:6789/0 conn(0x7f51900705e0 legacy=0x7f51900cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:23.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f518effd700 1 -- 172.21.15.104:0/4225299335 >> v1:172.21.15.104:6789/0 conn(0x7f5190159b70 legacy=0x7f51900cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:23.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f518dffb700 1 -- 172.21.15.104:0/4225299335 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 544326851 0 0) 0x7f51780035d0 con 0x7f5190159b70 2023-07-14T11:54:23.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f518effd700 1 -- 172.21.15.104:0/4225299335 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5190070cc0 con 0x7f51901597a0 2023-07-14T11:54:23.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.843+0000 7f518effd700 1 --2- 172.21.15.104:0/4225299335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51901597a0 0x7f51900c87a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5180009850 tx=0x7f5180009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e690c6e9bba6baa2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:23.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.847+0000 7f518dffb700 1 -- 172.21.15.104:0/4225299335 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184001670 con 0x7f51901597a0 2023-07-14T11:54:23.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.847+0000 7f518dffb700 1 -- 172.21.15.104:0/4225299335 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5180004030 con 0x7f51901597a0 2023-07-14T11:54:23.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.847+0000 7f518dffb700 1 -- 172.21.15.104:0/4225299335 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51800043c0 con 0x7f51901597a0 2023-07-14T11:54:23.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.847+0000 7f5195871700 1 -- 172.21.15.104:0/4225299335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51901597a0 msgr2=0x7f51900c87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:23.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.847+0000 7f5195871700 1 --2- 172.21.15.104:0/4225299335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51901597a0 0x7f51900c87a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5180009850 tx=0x7f5180009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:23.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.847+0000 7f5195871700 1 -- 172.21.15.104:0/4225299335 shutdown_connections 2023-07-14T11:54:23.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.847+0000 7f5195871700 1 --2- 172.21.15.104:0/4225299335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51901597a0 0x7f51900c87a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.847+0000 7f5195871700 1 -- 172.21.15.104:0/4225299335 >> 172.21.15.104:0/4225299335 conn(0x7f51900afc60 msgr2=0x7f51900b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:23.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.847+0000 7f5195871700 1 -- 172.21.15.104:0/4225299335 shutdown_connections 2023-07-14T11:54:23.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.847+0000 7f5195871700 1 -- 172.21.15.104:0/4225299335 wait complete. 2023-07-14T11:54:23.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.847+0000 7f5195871700 1 Processor -- start 2023-07-14T11:54:23.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f5195871700 1 -- start start 2023-07-14T11:54:23.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f5195871700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51900705e0 0x7f519014d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:23.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f5195871700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51901597a0 0x7f519006d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:23.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f5195871700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5190159b70 0x7f51900ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:23.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f5195871700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51900c18e0 con 0x7f51901597a0 2023-07-14T11:54:23.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f5195871700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5190156ec0 con 0x7f51900705e0 2023-07-14T11:54:23.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f5195871700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519015d4f0 con 0x7f5190159b70 2023-07-14T11:54:23.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f518e7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51901597a0 0x7f519006d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:23.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f518e7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51901597a0 0x7f519006d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56986/0 (socket says 172.21.15.104:56986) 2023-07-14T11:54:23.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f518e7fc700 1 -- 172.21.15.104:0/4041401011 learned_addr learned my addr 172.21.15.104:0/4041401011 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:23.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f518effd700 1 --2- 172.21.15.104:0/4041401011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51900705e0 0x7f519014d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:23.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f518f7fe700 1 --2- 172.21.15.104:0/4041401011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5190159b70 0x7f51900ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:23.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f518e7fc700 1 -- 172.21.15.104:0/4041401011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5190159b70 msgr2=0x7f51900ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:23.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f518e7fc700 1 --2- 172.21.15.104:0/4041401011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5190159b70 0x7f51900ac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f518e7fc700 1 -- 172.21.15.104:0/4041401011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51900705e0 msgr2=0x7f519014d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:23.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f518e7fc700 1 --2- 172.21.15.104:0/4041401011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51900705e0 0x7f519014d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:23.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f518e7fc700 1 -- 172.21.15.104:0/4041401011 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800094b0 con 0x7f51901597a0 2023-07-14T11:54:23.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f518f7fe700 1 --2- 172.21.15.104:0/4041401011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5190159b70 0x7f51900ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:23.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f518e7fc700 1 --2- 172.21.15.104:0/4041401011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51901597a0 0x7f519006d0d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f518400c700 tx=0x7f518400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:23.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f5177fff700 1 -- 172.21.15.104:0/4041401011 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184004ca0 con 0x7f51901597a0 2023-07-14T11:54:23.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f5177fff700 1 -- 172.21.15.104:0/4041401011 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5184004e00 con 0x7f51901597a0 2023-07-14T11:54:23.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f5195871700 1 -- 172.21.15.104:0/4041401011 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519015d7d0 con 0x7f51901597a0 2023-07-14T11:54:23.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f5177fff700 1 -- 172.21.15.104:0/4041401011 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184007310 con 0x7f51901597a0 2023-07-14T11:54:23.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.851+0000 7f5195871700 1 -- 172.21.15.104:0/4041401011 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519015dd90 con 0x7f51901597a0 2023-07-14T11:54:23.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.855+0000 7f5195871700 1 -- 172.21.15.104:0/4041401011 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519004e500 con 0x7f51901597a0 2023-07-14T11:54:23.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.855+0000 7f5177fff700 1 -- 172.21.15.104:0/4041401011 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5184007470 con 0x7f51901597a0 2023-07-14T11:54:23.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.859+0000 7f5177fff700 1 --2- 172.21.15.104:0/4041401011 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f517c0662f0 0x7f517c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:23.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.859+0000 7f5177fff700 1 -- 172.21.15.104:0/4041401011 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5184083dc0 con 0x7f51901597a0 2023-07-14T11:54:23.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.859+0000 7f518effd700 1 --2- 172.21.15.104:0/4041401011 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f517c0662f0 0x7f517c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:23.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.859+0000 7f518effd700 1 --2- 172.21.15.104:0/4041401011 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f517c0662f0 0x7f517c0687a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51800090b0 tx=0x7f5180008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:23.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:23.863+0000 7f5177fff700 1 -- 172.21.15.104:0/4041401011 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f5184050e90 con 0x7f51901597a0 2023-07-14T11:54:24.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.035+0000 7f5195871700 1 -- 172.21.15.104:0/4041401011 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51900c9b90 con 0x7f51901597a0 2023-07-14T11:54:24.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.035+0000 7f5177fff700 1 -- 172.21.15.104:0/4041401011 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51840544b0 con 0x7f51901597a0 2023-07-14T11:54:24.022 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:54:24.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.039+0000 7f5195871700 1 -- 172.21.15.104:0/4041401011 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f517c0662f0 msgr2=0x7f517c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:24.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.039+0000 7f5195871700 1 --2- 172.21.15.104:0/4041401011 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f517c0662f0 0x7f517c0687a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51800090b0 tx=0x7f5180008030 comp rx=0 tx=0).stop 2023-07-14T11:54:24.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.039+0000 7f5195871700 1 -- 172.21.15.104:0/4041401011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51901597a0 msgr2=0x7f519006d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:24.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.039+0000 7f5195871700 1 --2- 172.21.15.104:0/4041401011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51901597a0 0x7f519006d0d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f518400c700 tx=0x7f518400ca10 comp rx=0 tx=0).stop 2023-07-14T11:54:24.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.039+0000 7f5195871700 1 -- 172.21.15.104:0/4041401011 shutdown_connections 2023-07-14T11:54:24.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.039+0000 7f5195871700 1 --2- 172.21.15.104:0/4041401011 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f517c0662f0 0x7f517c0687a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.039+0000 7f5195871700 1 --2- 172.21.15.104:0/4041401011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51901597a0 0x7f519006d0d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.039+0000 7f5195871700 1 --2- 172.21.15.104:0/4041401011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5190159b70 0x7f51900ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.039+0000 7f5195871700 1 --2- 172.21.15.104:0/4041401011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51900705e0 0x7f519014d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.039+0000 7f5195871700 1 -- 172.21.15.104:0/4041401011 >> 172.21.15.104:0/4041401011 conn(0x7f51900afc60 msgr2=0x7f519015cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:24.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.039+0000 7f5195871700 1 -- 172.21.15.104:0/4041401011 shutdown_connections 2023-07-14T11:54:24.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.039+0000 7f5195871700 1 -- 172.21.15.104:0/4041401011 wait complete. 2023-07-14T11:54:24.044 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:54:24.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.235+0000 7fb4d11d9700 1 Processor -- start 2023-07-14T11:54:24.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.235+0000 7fb4d11d9700 1 -- start start 2023-07-14T11:54:24.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.235+0000 7fb4d11d9700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb4cc1603d0 0x7fb4cc06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:24.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.235+0000 7fb4d11d9700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc0cdfb0 con 0x7fb4cc160000 2023-07-14T11:54:24.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.235+0000 7fb4d11d9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc0cdcd0 con 0x7fb4cc0cd7f0 2023-07-14T11:54:24.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.235+0000 7fb4d11d9700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc070e40 con 0x7fb4cc1603d0 2023-07-14T11:54:24.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.235+0000 7fb4cb7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb4cc1603d0 0x7fb4cc06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:24.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.235+0000 7fb4cbfff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb4cc160000 0x7fb4cc0c9e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53720/0 (socket says 172.21.15.104:53720) 2023-07-14T11:54:24.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4cbfff700 1 -- 172.21.15.104:0/1642984299 learned_addr learned my addr 172.21.15.104:0/1642984299 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:24.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4caffd700 1 -- 172.21.15.104:0/1642984299 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342833626 0 0) 0x7fb4bc001110 con 0x7fb4cc160000 2023-07-14T11:54:24.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4caffd700 1 -- 172.21.15.104:0/1642984299 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4b40035d0 con 0x7fb4cc160000 2023-07-14T11:54:24.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4caffd700 1 -- 172.21.15.104:0/1642984299 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153412436 0 0) 0x7fb4cc0cdcd0 con 0x7fb4cc0cd7f0 2023-07-14T11:54:24.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4cb7fe700 1 -- 172.21.15.104:0/1642984299 >> v1:172.21.15.121:6789/0 conn(0x7fb4cc0cd7f0 legacy=0x7fb4cc0c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:24.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4cb7fe700 1 -- 172.21.15.104:0/1642984299 >> v1:172.21.15.104:6789/0 conn(0x7fb4cc160000 legacy=0x7fb4cc0c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:24.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4cb7fe700 1 -- 172.21.15.104:0/1642984299 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc070780 con 0x7fb4cc1603d0 2023-07-14T11:54:24.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4cb7fe700 1 --2- 172.21.15.104:0/1642984299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4cc1603d0 0x7fb4cc06dd10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009850 tx=0x7fb4c0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5253bd3ea6494e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:24.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4caffd700 1 -- 172.21.15.104:0/1642984299 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0011830 con 0x7fb4cc1603d0 2023-07-14T11:54:24.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4caffd700 1 -- 172.21.15.104:0/1642984299 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c0011990 con 0x7fb4cc1603d0 2023-07-14T11:54:24.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4caffd700 1 -- 172.21.15.104:0/1642984299 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0011d00 con 0x7fb4cc1603d0 2023-07-14T11:54:24.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4d11d9700 1 -- 172.21.15.104:0/1642984299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4cc1603d0 msgr2=0x7fb4cc06dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:24.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4d11d9700 1 --2- 172.21.15.104:0/1642984299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4cc1603d0 0x7fb4cc06dd10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009850 tx=0x7fb4c0009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:24.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4d11d9700 1 -- 172.21.15.104:0/1642984299 shutdown_connections 2023-07-14T11:54:24.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4d11d9700 1 --2- 172.21.15.104:0/1642984299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4cc1603d0 0x7fb4cc06dd10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4d11d9700 1 -- 172.21.15.104:0/1642984299 >> 172.21.15.104:0/1642984299 conn(0x7fb4cc0afc60 msgr2=0x7fb4cc0b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:24.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4d11d9700 1 -- 172.21.15.104:0/1642984299 shutdown_connections 2023-07-14T11:54:24.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.239+0000 7fb4d11d9700 1 -- 172.21.15.104:0/1642984299 wait complete. 2023-07-14T11:54:24.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.243+0000 7fb4d11d9700 1 Processor -- start 2023-07-14T11:54:24.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.243+0000 7fb4d11d9700 1 -- start start 2023-07-14T11:54:24.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4d11d9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc0cd7f0 0x7fb4cc1f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:24.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4d11d9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4cc160000 0x7fb4cc151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:24.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4d11d9700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4cc1603d0 0x7fb4cc0c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:24.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4d11d9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc1524f0 con 0x7fb4cc160000 2023-07-14T11:54:24.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4d11d9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc160b80 con 0x7fb4cc0cd7f0 2023-07-14T11:54:24.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4d11d9700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc160e60 con 0x7fb4cc1603d0 2023-07-14T11:54:24.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4cb7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4cc160000 0x7fb4cc151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:24.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4cb7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4cc160000 0x7fb4cc151fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56998/0 (socket says 172.21.15.104:56998) 2023-07-14T11:54:24.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4cb7fe700 1 -- 172.21.15.104:0/2209333964 learned_addr learned my addr 172.21.15.104:0/2209333964 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:24.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4cbfff700 1 --2- 172.21.15.104:0/2209333964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc0cd7f0 0x7fb4cc1f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:24.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4d09d8700 1 --2- 172.21.15.104:0/2209333964 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4cc1603d0 0x7fb4cc0c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:24.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4cb7fe700 1 -- 172.21.15.104:0/2209333964 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4cc1603d0 msgr2=0x7fb4cc0c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:24.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4cb7fe700 1 --2- 172.21.15.104:0/2209333964 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4cc1603d0 0x7fb4cc0c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4cb7fe700 1 -- 172.21.15.104:0/2209333964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc0cd7f0 msgr2=0x7fb4cc1f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:24.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4cb7fe700 1 --2- 172.21.15.104:0/2209333964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc0cd7f0 0x7fb4cc1f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4cb7fe700 1 -- 172.21.15.104:0/2209333964 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c00094b0 con 0x7fb4cc160000 2023-07-14T11:54:24.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4cb7fe700 1 --2- 172.21.15.104:0/2209333964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4cc160000 0x7fb4cc151fb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00090b0 tx=0x7fb4c0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:24.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4c8ff9700 1 -- 172.21.15.104:0/2209333964 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00036c0 con 0x7fb4cc160000 2023-07-14T11:54:24.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4c8ff9700 1 -- 172.21.15.104:0/2209333964 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c001e070 con 0x7fb4cc160000 2023-07-14T11:54:24.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4d11d9700 1 -- 172.21.15.104:0/2209333964 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc161140 con 0x7fb4cc160000 2023-07-14T11:54:24.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4c8ff9700 1 -- 172.21.15.104:0/2209333964 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0003a20 con 0x7fb4cc160000 2023-07-14T11:54:24.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.247+0000 7fb4d11d9700 1 -- 172.21.15.104:0/2209333964 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc161700 con 0x7fb4cc160000 2023-07-14T11:54:24.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.251+0000 7fb4d11d9700 1 -- 172.21.15.104:0/2209333964 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc1618f0 con 0x7fb4cc160000 2023-07-14T11:54:24.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.251+0000 7fb4c8ff9700 1 -- 172.21.15.104:0/2209333964 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb4c0003b80 con 0x7fb4cc160000 2023-07-14T11:54:24.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.251+0000 7fb4c8ff9700 1 --2- 172.21.15.104:0/2209333964 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb4b80662f0 0x7fb4b80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:24.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.251+0000 7fb4c8ff9700 1 -- 172.21.15.104:0/2209333964 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb4c0084fe0 con 0x7fb4cc160000 2023-07-14T11:54:24.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.251+0000 7fb4cbfff700 1 --2- 172.21.15.104:0/2209333964 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb4b80662f0 0x7fb4b80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:24.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.255+0000 7fb4cbfff700 1 --2- 172.21.15.104:0/2209333964 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb4b80662f0 0x7fb4b80687a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc005930 tx=0x7fb4bc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:24.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.259+0000 7fb4c8ff9700 1 -- 172.21.15.104:0/2209333964 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb4c0052140 con 0x7fb4cc160000 2023-07-14T11:54:24.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.431+0000 7fb4d11d9700 1 -- 172.21.15.104:0/2209333964 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb4cc04e500 con 0x7fb4cc160000 2023-07-14T11:54:24.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.431+0000 7fb4c8ff9700 1 -- 172.21.15.104:0/2209333964 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb4c002b350 con 0x7fb4cc160000 2023-07-14T11:54:24.416 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:54:24.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.431+0000 7fb4d11d9700 1 -- 172.21.15.104:0/2209333964 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb4b80662f0 msgr2=0x7fb4b80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:24.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.431+0000 7fb4d11d9700 1 --2- 172.21.15.104:0/2209333964 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb4b80662f0 0x7fb4b80687a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc005930 tx=0x7fb4bc009040 comp rx=0 tx=0).stop 2023-07-14T11:54:24.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.431+0000 7fb4d11d9700 1 -- 172.21.15.104:0/2209333964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4cc160000 msgr2=0x7fb4cc151fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:24.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.431+0000 7fb4d11d9700 1 --2- 172.21.15.104:0/2209333964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4cc160000 0x7fb4cc151fb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00090b0 tx=0x7fb4c0004770 comp rx=0 tx=0).stop 2023-07-14T11:54:24.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.431+0000 7fb4d11d9700 1 -- 172.21.15.104:0/2209333964 shutdown_connections 2023-07-14T11:54:24.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.431+0000 7fb4d11d9700 1 --2- 172.21.15.104:0/2209333964 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb4b80662f0 0x7fb4b80687a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.431+0000 7fb4d11d9700 1 --2- 172.21.15.104:0/2209333964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4cc160000 0x7fb4cc151fb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.431+0000 7fb4d11d9700 1 --2- 172.21.15.104:0/2209333964 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4cc1603d0 0x7fb4cc0c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.431+0000 7fb4d11d9700 1 --2- 172.21.15.104:0/2209333964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc0cd7f0 0x7fb4cc1f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.431+0000 7fb4d11d9700 1 -- 172.21.15.104:0/2209333964 >> 172.21.15.104:0/2209333964 conn(0x7fb4cc0afc60 msgr2=0x7fb4cc0ade60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:24.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.435+0000 7fb4d11d9700 1 -- 172.21.15.104:0/2209333964 shutdown_connections 2023-07-14T11:54:24.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.435+0000 7fb4d11d9700 1 -- 172.21.15.104:0/2209333964 wait complete. 2023-07-14T11:54:24.439 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:54:24.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.627+0000 7fd580260700 1 Processor -- start 2023-07-14T11:54:24.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd580260700 1 -- start start 2023-07-14T11:54:24.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd580260700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fd5780cdcb0 0x7fd578071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:24.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd580260700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd5780b0680 0x7fd5780c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:24.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd580260700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd578158970 con 0x7fd5780c90a0 2023-07-14T11:54:24.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd580260700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5780ce080 con 0x7fd5780cdcb0 2023-07-14T11:54:24.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd580260700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5780c94a0 con 0x7fd5780b0680 2023-07-14T11:54:24.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd57dffc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fd5780cdcb0 0x7fd578071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:24.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd57dffc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fd5780cdcb0 0x7fd578071440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57002/0 (socket says 172.21.15.104:57002) 2023-07-14T11:54:24.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd57dffc700 1 -- 172.21.15.104:0/895725482 learned_addr learned my addr 172.21.15.104:0/895725482 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:24.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd57e7fd700 1 --1- 172.21.15.104:0/895725482 >> v1:172.21.15.104:6789/0 conn(0x7fd5780c90a0 0x7fd57815bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53722/0 (socket says 172.21.15.104:53722) 2023-07-14T11:54:24.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd57d7fb700 1 --2- 172.21.15.104:0/895725482 >> v2:172.21.15.121:3300/0 conn(0x7fd5780b0680 0x7fd5780c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:24.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd57dffc700 1 -- 172.21.15.104:0/895725482 >> v2:172.21.15.121:3300/0 conn(0x7fd5780b0680 msgr2=0x7fd5780c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:24.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd57dffc700 1 --2- 172.21.15.104:0/895725482 >> v2:172.21.15.121:3300/0 conn(0x7fd5780b0680 0x7fd5780c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd57dffc700 1 -- 172.21.15.104:0/895725482 >> v1:172.21.15.104:6789/0 conn(0x7fd5780c90a0 legacy=0x7fd57815bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:24.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd57dffc700 1 -- 172.21.15.104:0/895725482 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd578158bf0 con 0x7fd5780cdcb0 2023-07-14T11:54:24.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd57dffc700 1 --2- 172.21.15.104:0/895725482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5780cdcb0 0x7fd578071440 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd57806bef0 tx=0x7fd574009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11839fbc6ac9f59b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:24.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.631+0000 7fd57cffa700 1 -- 172.21.15.104:0/895725482 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57400b8a0 con 0x7fd5780cdcb0 2023-07-14T11:54:24.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.635+0000 7fd57cffa700 1 -- 172.21.15.104:0/895725482 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57400ba00 con 0x7fd5780cdcb0 2023-07-14T11:54:24.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.635+0000 7fd57cffa700 1 -- 172.21.15.104:0/895725482 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57400bd90 con 0x7fd5780cdcb0 2023-07-14T11:54:24.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.635+0000 7fd580260700 1 -- 172.21.15.104:0/895725482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5780cdcb0 msgr2=0x7fd578071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:24.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.635+0000 7fd580260700 1 --2- 172.21.15.104:0/895725482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5780cdcb0 0x7fd578071440 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd57806bef0 tx=0x7fd574009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.635+0000 7fd580260700 1 -- 172.21.15.104:0/895725482 shutdown_connections 2023-07-14T11:54:24.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.635+0000 7fd580260700 1 --2- 172.21.15.104:0/895725482 >> v2:172.21.15.121:3300/0 conn(0x7fd5780b0680 0x7fd5780c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.635+0000 7fd580260700 1 --2- 172.21.15.104:0/895725482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5780cdcb0 0x7fd578071440 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.635+0000 7fd580260700 1 -- 172.21.15.104:0/895725482 >> 172.21.15.104:0/895725482 conn(0x7fd5780cad80 msgr2=0x7fd5780cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:24.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.635+0000 7fd580260700 1 -- 172.21.15.104:0/895725482 shutdown_connections 2023-07-14T11:54:24.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.635+0000 7fd580260700 1 -- 172.21.15.104:0/895725482 wait complete. 2023-07-14T11:54:24.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.635+0000 7fd580260700 1 Processor -- start 2023-07-14T11:54:24.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.635+0000 7fd580260700 1 -- start start 2023-07-14T11:54:24.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd580260700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5780b0680 0x7fd5780b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:24.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd580260700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5780c90a0 0x7fd57815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:24.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd580260700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5780cdcb0 0x7fd57815ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:24.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd580260700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780b1ad0 con 0x7fd5780cdcb0 2023-07-14T11:54:24.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd580260700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5781e3da0 con 0x7fd5780b0680 2023-07-14T11:54:24.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd580260700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5781e2e10 con 0x7fd5780c90a0 2023-07-14T11:54:24.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd57e7fd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5780cdcb0 0x7fd57815ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:24.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd57e7fd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5780cdcb0 0x7fd57815ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57018/0 (socket says 172.21.15.104:57018) 2023-07-14T11:54:24.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd57e7fd700 1 -- 172.21.15.104:0/2342543034 learned_addr learned my addr 172.21.15.104:0/2342543034 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:24.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd57dffc700 1 --2- 172.21.15.104:0/2342543034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5780b0680 0x7fd5780b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:24.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd57d7fb700 1 --2- 172.21.15.104:0/2342543034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5780c90a0 0x7fd57815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:24.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd57e7fd700 1 -- 172.21.15.104:0/2342543034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5780c90a0 msgr2=0x7fd57815a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:24.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd57e7fd700 1 --2- 172.21.15.104:0/2342543034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5780c90a0 0x7fd57815a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd57e7fd700 1 -- 172.21.15.104:0/2342543034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5780b0680 msgr2=0x7fd5780b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:24.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd57e7fd700 1 --2- 172.21.15.104:0/2342543034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5780b0680 0x7fd5780b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd57e7fd700 1 -- 172.21.15.104:0/2342543034 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd574009570 con 0x7fd5780cdcb0 2023-07-14T11:54:24.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd57e7fd700 1 --2- 172.21.15.104:0/2342543034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5780cdcb0 0x7fd57815ace0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd57000cb50 tx=0x7fd57000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:24.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd56e7fc700 1 -- 172.21.15.104:0/2342543034 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000aa20 con 0x7fd5780cdcb0 2023-07-14T11:54:24.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd56e7fc700 1 -- 172.21.15.104:0/2342543034 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57000ab80 con 0x7fd5780cdcb0 2023-07-14T11:54:24.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd580260700 1 -- 172.21.15.104:0/2342543034 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5780b1130 con 0x7fd5780cdcb0 2023-07-14T11:54:24.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd56e7fc700 1 -- 172.21.15.104:0/2342543034 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570007070 con 0x7fd5780cdcb0 2023-07-14T11:54:24.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.639+0000 7fd580260700 1 -- 172.21.15.104:0/2342543034 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5780b16f0 con 0x7fd5780cdcb0 2023-07-14T11:54:24.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.643+0000 7fd580260700 1 -- 172.21.15.104:0/2342543034 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd578061060 con 0x7fd5780cdcb0 2023-07-14T11:54:24.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.643+0000 7fd56e7fc700 1 -- 172.21.15.104:0/2342543034 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd5700071d0 con 0x7fd5780cdcb0 2023-07-14T11:54:24.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.643+0000 7fd56e7fc700 1 --2- 172.21.15.104:0/2342543034 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd55c0662f0 0x7fd55c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:24.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.643+0000 7fd56e7fc700 1 -- 172.21.15.104:0/2342543034 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd570083e20 con 0x7fd5780cdcb0 2023-07-14T11:54:24.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.643+0000 7fd57dffc700 1 --2- 172.21.15.104:0/2342543034 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd55c0662f0 0x7fd55c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:24.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.647+0000 7fd57dffc700 1 --2- 172.21.15.104:0/2342543034 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd55c0662f0 0x7fd55c0687a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd574000c00 tx=0x7fd574008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:24.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.651+0000 7fd56e7fc700 1 -- 172.21.15.104:0/2342543034 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fd570050f60 con 0x7fd5780cdcb0 2023-07-14T11:54:24.806 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.819+0000 7fd580260700 1 -- 172.21.15.104:0/2342543034 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd5780b1c10 con 0x7fd5780cdcb0 2023-07-14T11:54:24.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.823+0000 7fd56e7fc700 1 -- 172.21.15.104:0/2342543034 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd5780b1c10 con 0x7fd5780cdcb0 2023-07-14T11:54:24.807 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:24.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.823+0000 7fd580260700 1 -- 172.21.15.104:0/2342543034 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd55c0662f0 msgr2=0x7fd55c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:24.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.823+0000 7fd580260700 1 --2- 172.21.15.104:0/2342543034 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd55c0662f0 0x7fd55c0687a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd574000c00 tx=0x7fd574008030 comp rx=0 tx=0).stop 2023-07-14T11:54:24.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.823+0000 7fd580260700 1 -- 172.21.15.104:0/2342543034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5780cdcb0 msgr2=0x7fd57815ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:24.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.823+0000 7fd580260700 1 --2- 172.21.15.104:0/2342543034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5780cdcb0 0x7fd57815ace0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd57000cb50 tx=0x7fd57000ce60 comp rx=0 tx=0).stop 2023-07-14T11:54:24.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.823+0000 7fd580260700 1 -- 172.21.15.104:0/2342543034 shutdown_connections 2023-07-14T11:54:24.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.823+0000 7fd580260700 1 --2- 172.21.15.104:0/2342543034 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd55c0662f0 0x7fd55c0687a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.823+0000 7fd580260700 1 --2- 172.21.15.104:0/2342543034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5780cdcb0 0x7fd57815ace0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.823+0000 7fd580260700 1 --2- 172.21.15.104:0/2342543034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5780c90a0 0x7fd57815a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.823+0000 7fd580260700 1 --2- 172.21.15.104:0/2342543034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5780b0680 0x7fd5780b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:24.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.823+0000 7fd580260700 1 -- 172.21.15.104:0/2342543034 >> 172.21.15.104:0/2342543034 conn(0x7fd5780cad80 msgr2=0x7fd5780ae040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:24.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.823+0000 7fd580260700 1 -- 172.21.15.104:0/2342543034 shutdown_connections 2023-07-14T11:54:24.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:24.823+0000 7fd580260700 1 -- 172.21.15.104:0/2342543034 wait complete. 2023-07-14T11:54:24.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:54:25.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.015+0000 7f9243fff700 1 Processor -- start 2023-07-14T11:54:25.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.015+0000 7f9243fff700 1 -- start start 2023-07-14T11:54:25.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.019+0000 7f9243fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9244160000 0x7f924406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.019+0000 7f9243fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f92441603d0 0x7f92440c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.019+0000 7f9243fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f92440cd7f0 0x7f92440ca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.019+0000 7f9243fff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9244070e40 con 0x7f9244160000 2023-07-14T11:54:25.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.019+0000 7f9243fff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92440cdfb0 con 0x7f92440cd7f0 2023-07-14T11:54:25.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.019+0000 7f9243fff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92440ca8c0 con 0x7f92441603d0 2023-07-14T11:54:25.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.019+0000 7f9242ffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9244160000 0x7f924406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.019+0000 7f9242ffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9244160000 0x7f924406dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57030/0 (socket says 172.21.15.104:57030) 2023-07-14T11:54:25.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.019+0000 7f9242ffd700 1 -- 172.21.15.104:0/1699375257 learned_addr learned my addr 172.21.15.104:0/1699375257 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:25.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.019+0000 7f92437fe700 1 --2- 172.21.15.104:0/1699375257 >> v2:172.21.15.121:3300/0 conn(0x7f92440cd7f0 0x7f92440ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.019+0000 7f92427fc700 1 --2- 172.21.15.104:0/1699375257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f92441603d0 0x7f92440c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9242ffd700 1 -- 172.21.15.104:0/1699375257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f92441603d0 msgr2=0x7f92440c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:25.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9242ffd700 1 --2- 172.21.15.104:0/1699375257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f92441603d0 0x7f92440c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9242ffd700 1 -- 172.21.15.104:0/1699375257 >> v2:172.21.15.121:3300/0 conn(0x7f92440cd7f0 msgr2=0x7f92440ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:25.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9242ffd700 1 --2- 172.21.15.104:0/1699375257 >> v2:172.21.15.121:3300/0 conn(0x7f92440cd7f0 0x7f92440ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9242ffd700 1 -- 172.21.15.104:0/1699375257 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92440cdcd0 con 0x7f9244160000 2023-07-14T11:54:25.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f92427fc700 1 --2- 172.21.15.104:0/1699375257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f92441603d0 0x7f92440c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:25.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9242ffd700 1 --2- 172.21.15.104:0/1699375257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9244160000 0x7f924406dd10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f924406bd60 tx=0x7f9234009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808fa53f2b1750a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:25.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9241ffb700 1 -- 172.21.15.104:0/1699375257 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234011830 con 0x7f9244160000 2023-07-14T11:54:25.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9241ffb700 1 -- 172.21.15.104:0/1699375257 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9234011990 con 0x7f9244160000 2023-07-14T11:54:25.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9241ffb700 1 -- 172.21.15.104:0/1699375257 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234011d20 con 0x7f9244160000 2023-07-14T11:54:25.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9243fff700 1 -- 172.21.15.104:0/1699375257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9244160000 msgr2=0x7f924406dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9243fff700 1 --2- 172.21.15.104:0/1699375257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9244160000 0x7f924406dd10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f924406bd60 tx=0x7f9234009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9243fff700 1 -- 172.21.15.104:0/1699375257 shutdown_connections 2023-07-14T11:54:25.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9243fff700 1 --2- 172.21.15.104:0/1699375257 >> v2:172.21.15.121:3300/0 conn(0x7f92440cd7f0 0x7f92440ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9243fff700 1 --2- 172.21.15.104:0/1699375257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f92441603d0 0x7f92440c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9243fff700 1 --2- 172.21.15.104:0/1699375257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9244160000 0x7f924406dd10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9243fff700 1 -- 172.21.15.104:0/1699375257 >> 172.21.15.104:0/1699375257 conn(0x7f92440afc60 msgr2=0x7f92440b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:25.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9243fff700 1 -- 172.21.15.104:0/1699375257 shutdown_connections 2023-07-14T11:54:25.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.023+0000 7f9243fff700 1 -- 172.21.15.104:0/1699375257 wait complete. 2023-07-14T11:54:25.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9243fff700 1 Processor -- start 2023-07-14T11:54:25.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9243fff700 1 -- start start 2023-07-14T11:54:25.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9243fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92440cd7f0 0x7f924415df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9243fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9244160000 0x7f924415e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9243fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92441603d0 0x7f92441f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9243fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92441f6d20 con 0x7f92440cd7f0 2023-07-14T11:54:25.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9243fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924414a6e0 con 0x7f92441603d0 2023-07-14T11:54:25.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9243fff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924414a9c0 con 0x7f9244160000 2023-07-14T11:54:25.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9242ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92440cd7f0 0x7f924415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9242ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92440cd7f0 0x7f924415df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57038/0 (socket says 172.21.15.104:57038) 2023-07-14T11:54:25.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9242ffd700 1 -- 172.21.15.104:0/2853869255 learned_addr learned my addr 172.21.15.104:0/2853869255 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:25.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f92437fe700 1 --2- 172.21.15.104:0/2853869255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92441603d0 0x7f92441f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f92427fc700 1 --2- 172.21.15.104:0/2853869255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9244160000 0x7f924415e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9242ffd700 1 -- 172.21.15.104:0/2853869255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9244160000 msgr2=0x7f924415e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9242ffd700 1 --2- 172.21.15.104:0/2853869255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9244160000 0x7f924415e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9242ffd700 1 -- 172.21.15.104:0/2853869255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92441603d0 msgr2=0x7f92441f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9242ffd700 1 --2- 172.21.15.104:0/2853869255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92441603d0 0x7f92441f5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f9242ffd700 1 -- 172.21.15.104:0/2853869255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9234009570 con 0x7f92440cd7f0 2023-07-14T11:54:25.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.027+0000 7f92427fc700 1 --2- 172.21.15.104:0/2853869255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9244160000 0x7f924415e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:25.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.031+0000 7f9242ffd700 1 --2- 172.21.15.104:0/2853869255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92440cd7f0 0x7f924415df70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f924406b5d0 tx=0x7f9234004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:25.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.031+0000 7f922bfff700 1 -- 172.21.15.104:0/2853869255 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234004240 con 0x7f92440cd7f0 2023-07-14T11:54:25.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.031+0000 7f922bfff700 1 -- 172.21.15.104:0/2853869255 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923401f070 con 0x7f92440cd7f0 2023-07-14T11:54:25.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.031+0000 7f9243fff700 1 -- 172.21.15.104:0/2853869255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924414aca0 con 0x7f92440cd7f0 2023-07-14T11:54:25.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.031+0000 7f922bfff700 1 -- 172.21.15.104:0/2853869255 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92340045e0 con 0x7f92440cd7f0 2023-07-14T11:54:25.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.031+0000 7f9243fff700 1 -- 172.21.15.104:0/2853869255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92441f6ea0 con 0x7f92440cd7f0 2023-07-14T11:54:25.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.031+0000 7f9243fff700 1 -- 172.21.15.104:0/2853869255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92441f70f0 con 0x7f92440cd7f0 2023-07-14T11:54:25.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.035+0000 7f922bfff700 1 -- 172.21.15.104:0/2853869255 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9234021030 con 0x7f92440cd7f0 2023-07-14T11:54:25.020 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.035+0000 7f922bfff700 1 --2- 172.21.15.104:0/2853869255 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f92300662f0 0x7f92300687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.035+0000 7f922bfff700 1 -- 172.21.15.104:0/2853869255 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f92340850c0 con 0x7f92440cd7f0 2023-07-14T11:54:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.035+0000 7f92427fc700 1 --2- 172.21.15.104:0/2853869255 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f92300662f0 0x7f92300687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.035+0000 7f92427fc700 1 --2- 172.21.15.104:0/2853869255 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f92300662f0 0x7f92300687a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f924415b160 tx=0x7f9238008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:25.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.039+0000 7f922bfff700 1 -- 172.21.15.104:0/2853869255 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f92340522c0 con 0x7f92440cd7f0 2023-07-14T11:54:25.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.211+0000 7f9243fff700 1 -- 172.21.15.104:0/2853869255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f924404e500 con 0x7f92440cd7f0 2023-07-14T11:54:25.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.211+0000 7f922bfff700 1 -- 172.21.15.104:0/2853869255 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f92340558e0 con 0x7f92440cd7f0 2023-07-14T11:54:25.198 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:54:25.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.215+0000 7f9243fff700 1 -- 172.21.15.104:0/2853869255 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f92300662f0 msgr2=0x7f92300687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.215+0000 7f9243fff700 1 --2- 172.21.15.104:0/2853869255 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f92300662f0 0x7f92300687a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f924415b160 tx=0x7f9238008040 comp rx=0 tx=0).stop 2023-07-14T11:54:25.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.215+0000 7f9243fff700 1 -- 172.21.15.104:0/2853869255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92440cd7f0 msgr2=0x7f924415df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.215+0000 7f9243fff700 1 --2- 172.21.15.104:0/2853869255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92440cd7f0 0x7f924415df70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f924406b5d0 tx=0x7f9234004770 comp rx=0 tx=0).stop 2023-07-14T11:54:25.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.215+0000 7f9243fff700 1 -- 172.21.15.104:0/2853869255 shutdown_connections 2023-07-14T11:54:25.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.215+0000 7f9243fff700 1 --2- 172.21.15.104:0/2853869255 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f92300662f0 0x7f92300687a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.215+0000 7f9243fff700 1 --2- 172.21.15.104:0/2853869255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92441603d0 0x7f92441f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.215+0000 7f9243fff700 1 --2- 172.21.15.104:0/2853869255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9244160000 0x7f924415e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.215+0000 7f9243fff700 1 --2- 172.21.15.104:0/2853869255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92440cd7f0 0x7f924415df70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.215+0000 7f9243fff700 1 -- 172.21.15.104:0/2853869255 >> 172.21.15.104:0/2853869255 conn(0x7f92440afc60 msgr2=0x7f924406bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:25.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.215+0000 7f9243fff700 1 -- 172.21.15.104:0/2853869255 shutdown_connections 2023-07-14T11:54:25.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.215+0000 7f9243fff700 1 -- 172.21.15.104:0/2853869255 wait complete. 2023-07-14T11:54:25.223 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:54:25.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.415+0000 7f70651e0700 1 Processor -- start 2023-07-14T11:54:25.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.415+0000 7f70651e0700 1 -- start start 2023-07-14T11:54:25.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f70651e0700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f70600d0210 0x7f70600d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f70651e0700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7060071d10 con 0x7f70600cf000 2023-07-14T11:54:25.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f70651e0700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706015c420 con 0x7f70600b1c50 2023-07-14T11:54:25.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f70651e0700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7060070840 con 0x7f70600d0210 2023-07-14T11:54:25.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f705e59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f70600d0210 0x7f70600d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f705e59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f70600d0210 0x7f70600d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57040/0 (socket says 172.21.15.104:57040) 2023-07-14T11:54:25.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f705e59c700 1 -- 172.21.15.104:0/3922401114 learned_addr learned my addr 172.21.15.104:0/3922401114 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:25.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f705f59e700 1 --1- 172.21.15.104:0/3922401114 >> v1:172.21.15.104:6789/0 conn(0x7f70600cf000 0x7f70600ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53730/0 (socket says 172.21.15.104:53730) 2023-07-14T11:54:25.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f705e59c700 1 -- 172.21.15.104:0/3922401114 >> v1:172.21.15.121:6789/0 conn(0x7f70600b1c50 legacy=0x7f7060071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:25.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f705e59c700 1 -- 172.21.15.104:0/3922401114 >> v1:172.21.15.104:6789/0 conn(0x7f70600cf000 legacy=0x7f70600ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f705e59c700 1 -- 172.21.15.104:0/3922401114 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7060070b20 con 0x7f70600d0210 2023-07-14T11:54:25.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f705e59c700 1 --2- 172.21.15.104:0/3922401114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70600d0210 0x7f70600d0640 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7054009910 tx=0x7f7054009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756fe91db6a57d61 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:25.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f705dd9b700 1 -- 172.21.15.104:0/3922401114 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054004230 con 0x7f70600d0210 2023-07-14T11:54:25.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f705dd9b700 1 -- 172.21.15.104:0/3922401114 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7054004390 con 0x7f70600d0210 2023-07-14T11:54:25.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f705dd9b700 1 -- 172.21.15.104:0/3922401114 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054009e60 con 0x7f70600d0210 2023-07-14T11:54:25.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f70651e0700 1 -- 172.21.15.104:0/3922401114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70600d0210 msgr2=0x7f70600d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f70651e0700 1 --2- 172.21.15.104:0/3922401114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70600d0210 0x7f70600d0640 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7054009910 tx=0x7f7054009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:25.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f70651e0700 1 -- 172.21.15.104:0/3922401114 shutdown_connections 2023-07-14T11:54:25.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f70651e0700 1 --2- 172.21.15.104:0/3922401114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70600d0210 0x7f70600d0640 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f70651e0700 1 -- 172.21.15.104:0/3922401114 >> 172.21.15.104:0/3922401114 conn(0x7f706009b9c0 msgr2=0x7f70600d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:25.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f70651e0700 1 -- 172.21.15.104:0/3922401114 shutdown_connections 2023-07-14T11:54:25.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.419+0000 7f70651e0700 1 -- 172.21.15.104:0/3922401114 wait complete. 2023-07-14T11:54:25.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.423+0000 7f70651e0700 1 Processor -- start 2023-07-14T11:54:25.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.423+0000 7f70651e0700 1 -- start start 2023-07-14T11:54:25.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f70651e0700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70600b1c50 0x7f70601f0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f70651e0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70600cf000 0x7f706009f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f70651e0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70600d0210 0x7f70600d2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f70651e0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70601f2100 con 0x7f70600cf000 2023-07-14T11:54:25.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f70651e0700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70601f1260 con 0x7f70600b1c50 2023-07-14T11:54:25.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f70651e0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706009fb80 con 0x7f70600d0210 2023-07-14T11:54:25.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f705e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70600cf000 0x7f706009f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f705e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70600cf000 0x7f706009f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57042/0 (socket says 172.21.15.104:57042) 2023-07-14T11:54:25.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f705e59c700 1 -- 172.21.15.104:0/928662344 learned_addr learned my addr 172.21.15.104:0/928662344 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:25.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f705f59e700 1 --2- 172.21.15.104:0/928662344 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70600d0210 0x7f70600d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f705ed9d700 1 --2- 172.21.15.104:0/928662344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70600b1c50 0x7f70601f0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f705e59c700 1 -- 172.21.15.104:0/928662344 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70600d0210 msgr2=0x7f70600d2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f705e59c700 1 --2- 172.21.15.104:0/928662344 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70600d0210 0x7f70600d2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f705e59c700 1 -- 172.21.15.104:0/928662344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70600b1c50 msgr2=0x7f70601f0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f705e59c700 1 --2- 172.21.15.104:0/928662344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70600b1c50 0x7f70601f0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f705e59c700 1 -- 172.21.15.104:0/928662344 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7054009570 con 0x7f70600cf000 2023-07-14T11:54:25.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f705e59c700 1 --2- 172.21.15.104:0/928662344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70600cf000 0x7f706009f640 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f70600d2610 tx=0x7f7054004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:25.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f70477fe700 1 -- 172.21.15.104:0/928662344 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70540036c0 con 0x7f70600cf000 2023-07-14T11:54:25.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f70477fe700 1 -- 172.21.15.104:0/928662344 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7054003840 con 0x7f70600cf000 2023-07-14T11:54:25.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f70477fe700 1 -- 172.21.15.104:0/928662344 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054003bf0 con 0x7f70600cf000 2023-07-14T11:54:25.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f70651e0700 1 -- 172.21.15.104:0/928662344 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70601f1480 con 0x7f70600cf000 2023-07-14T11:54:25.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.427+0000 7f70651e0700 1 -- 172.21.15.104:0/928662344 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70601f1a40 con 0x7f70600cf000 2023-07-14T11:54:25.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.431+0000 7f70651e0700 1 -- 172.21.15.104:0/928662344 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706004dcb0 con 0x7f70600cf000 2023-07-14T11:54:25.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.431+0000 7f70477fe700 1 -- 172.21.15.104:0/928662344 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7054003d50 con 0x7f70600cf000 2023-07-14T11:54:25.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.435+0000 7f70477fe700 1 --2- 172.21.15.104:0/928662344 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f704c066320 0x7f704c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.435+0000 7f70477fe700 1 -- 172.21.15.104:0/928662344 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7054085950 con 0x7f70600cf000 2023-07-14T11:54:25.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.435+0000 7f705ed9d700 1 --2- 172.21.15.104:0/928662344 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f704c066320 0x7f704c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.435+0000 7f705ed9d700 1 --2- 172.21.15.104:0/928662344 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f704c066320 0x7f704c0687d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7050004f70 tx=0x7f7050009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:25.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.439+0000 7f70477fe700 1 -- 172.21.15.104:0/928662344 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f7054052bb0 con 0x7f70600cf000 2023-07-14T11:54:25.591 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.607+0000 7f70651e0700 1 -- 172.21.15.104:0/928662344 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f706009fdb0 con 0x7f70600cf000 2023-07-14T11:54:25.592 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.607+0000 7f70477fe700 1 -- 172.21.15.104:0/928662344 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f70540561d0 con 0x7f70600cf000 2023-07-14T11:54:25.592 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:25.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.611+0000 7f70651e0700 1 -- 172.21.15.104:0/928662344 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f704c066320 msgr2=0x7f704c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.611+0000 7f70651e0700 1 --2- 172.21.15.104:0/928662344 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f704c066320 0x7f704c0687d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7050004f70 tx=0x7f7050009040 comp rx=0 tx=0).stop 2023-07-14T11:54:25.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.611+0000 7f70651e0700 1 -- 172.21.15.104:0/928662344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70600cf000 msgr2=0x7f706009f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.611+0000 7f70651e0700 1 --2- 172.21.15.104:0/928662344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70600cf000 0x7f706009f640 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f70600d2610 tx=0x7f7054004770 comp rx=0 tx=0).stop 2023-07-14T11:54:25.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.611+0000 7f70651e0700 1 -- 172.21.15.104:0/928662344 shutdown_connections 2023-07-14T11:54:25.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.611+0000 7f70651e0700 1 --2- 172.21.15.104:0/928662344 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f704c066320 0x7f704c0687d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.611+0000 7f70651e0700 1 --2- 172.21.15.104:0/928662344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70600cf000 0x7f706009f640 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.611+0000 7f70651e0700 1 --2- 172.21.15.104:0/928662344 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70600d0210 0x7f70600d2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.611+0000 7f70651e0700 1 --2- 172.21.15.104:0/928662344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70600b1c50 0x7f70601f0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.611+0000 7f70651e0700 1 -- 172.21.15.104:0/928662344 >> 172.21.15.104:0/928662344 conn(0x7f706009b9c0 msgr2=0x7f70600b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:25.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.611+0000 7f70651e0700 1 -- 172.21.15.104:0/928662344 shutdown_connections 2023-07-14T11:54:25.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.611+0000 7f70651e0700 1 -- 172.21.15.104:0/928662344 wait complete. 2023-07-14T11:54:25.615 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:54:25.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.803+0000 7f4149682700 1 Processor -- start 2023-07-14T11:54:25.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.803+0000 7f4149682700 1 -- start start 2023-07-14T11:54:25.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.803+0000 7f4149682700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f41440cb840 0x7f41440cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.803+0000 7f4149682700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41440cc1b0 con 0x7f414406be70 2023-07-14T11:54:25.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.803+0000 7f4149682700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41440cd690 con 0x7f414406d860 2023-07-14T11:54:25.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.803+0000 7f4149682700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41440c2510 con 0x7f41440cb840 2023-07-14T11:54:25.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.803+0000 7f41427fc700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f414406be70 0x7f41440c1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53736/0 (socket says 172.21.15.104:53736) 2023-07-14T11:54:25.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.803+0000 7f41427fc700 1 -- 172.21.15.104:0/3511143263 learned_addr learned my addr 172.21.15.104:0/3511143263 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:25.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.803+0000 7f41437fe700 1 --2- 172.21.15.104:0/3511143263 >> v2:172.21.15.104:3300/0 conn(0x7f41440cb840 0x7f41440cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.803+0000 7f41437fe700 1 -- 172.21.15.104:0/3511143263 >> v1:172.21.15.121:6789/0 conn(0x7f414406d860 legacy=0x7f414406dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.803+0000 7f41437fe700 1 -- 172.21.15.104:0/3511143263 >> v1:172.21.15.104:6789/0 conn(0x7f414406be70 legacy=0x7f41440c1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.803+0000 7f41437fe700 1 -- 172.21.15.104:0/3511143263 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41440cd970 con 0x7f41440cb840 2023-07-14T11:54:25.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.807+0000 7f41437fe700 1 --2- 172.21.15.104:0/3511143263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41440cb840 0x7f41440cc640 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f413c009d90 tx=0x7f413c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd25f67dba4280e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:25.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.807+0000 7f4141ffb700 1 -- 172.21.15.104:0/3511143263 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c00e070 con 0x7f41440cb840 2023-07-14T11:54:25.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.807+0000 7f4141ffb700 1 -- 172.21.15.104:0/3511143263 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413c00bbd0 con 0x7f41440cb840 2023-07-14T11:54:25.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.807+0000 7f4141ffb700 1 -- 172.21.15.104:0/3511143263 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c012430 con 0x7f41440cb840 2023-07-14T11:54:25.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.807+0000 7f4149682700 1 -- 172.21.15.104:0/3511143263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41440cb840 msgr2=0x7f41440cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.807+0000 7f4149682700 1 --2- 172.21.15.104:0/3511143263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41440cb840 0x7f41440cc640 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f413c009d90 tx=0x7f413c00b330 comp rx=0 tx=0).stop 2023-07-14T11:54:25.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.807+0000 7f4149682700 1 -- 172.21.15.104:0/3511143263 shutdown_connections 2023-07-14T11:54:25.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.807+0000 7f4149682700 1 --2- 172.21.15.104:0/3511143263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41440cb840 0x7f41440cc640 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.807+0000 7f4149682700 1 -- 172.21.15.104:0/3511143263 >> 172.21.15.104:0/3511143263 conn(0x7f41440cad30 msgr2=0x7f41440cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:25.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.807+0000 7f4149682700 1 -- 172.21.15.104:0/3511143263 shutdown_connections 2023-07-14T11:54:25.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.807+0000 7f4149682700 1 -- 172.21.15.104:0/3511143263 wait complete. 2023-07-14T11:54:25.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4149682700 1 Processor -- start 2023-07-14T11:54:25.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4149682700 1 -- start start 2023-07-14T11:54:25.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4149682700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f414406be70 0x7f414414b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4149682700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f414406d860 0x7f41440c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4149682700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41440cb840 0x7f41440aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4149682700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41440c75f0 con 0x7f414406be70 2023-07-14T11:54:25.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4149682700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41440c7310 con 0x7f414406d860 2023-07-14T11:54:25.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4149682700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41440b1210 con 0x7f41440cb840 2023-07-14T11:54:25.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4142ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f414406be70 0x7f414414b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.799 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4142ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f414406be70 0x7f414414b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57064/0 (socket says 172.21.15.104:57064) 2023-07-14T11:54:25.799 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4142ffd700 1 -- 172.21.15.104:0/2149254558 learned_addr learned my addr 172.21.15.104:0/2149254558 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:25.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f41437fe700 1 --2- 172.21.15.104:0/2149254558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41440cb840 0x7f41440aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.801 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f41427fc700 1 --2- 172.21.15.104:0/2149254558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f414406d860 0x7f41440c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.801 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4142ffd700 1 -- 172.21.15.104:0/2149254558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41440cb840 msgr2=0x7f41440aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.801 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4142ffd700 1 --2- 172.21.15.104:0/2149254558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41440cb840 0x7f41440aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.802 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4142ffd700 1 -- 172.21.15.104:0/2149254558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f414406d860 msgr2=0x7f41440c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.803 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4142ffd700 1 --2- 172.21.15.104:0/2149254558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f414406d860 0x7f41440c6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.803 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4142ffd700 1 -- 172.21.15.104:0/2149254558 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c009a70 con 0x7f414406be70 2023-07-14T11:54:25.803 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4142ffd700 1 --2- 172.21.15.104:0/2149254558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f414406be70 0x7f414414b0a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f413400c500 tx=0x7f413400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:25.803 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f412b7fe700 1 -- 172.21.15.104:0/2149254558 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41340052d0 con 0x7f414406be70 2023-07-14T11:54:25.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f412b7fe700 1 -- 172.21.15.104:0/2149254558 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4134005430 con 0x7f414406be70 2023-07-14T11:54:25.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4149682700 1 -- 172.21.15.104:0/2149254558 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41440b14f0 con 0x7f414406be70 2023-07-14T11:54:25.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f412b7fe700 1 -- 172.21.15.104:0/2149254558 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41340055a0 con 0x7f414406be70 2023-07-14T11:54:25.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.811+0000 7f4149682700 1 -- 172.21.15.104:0/2149254558 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414414b5e0 con 0x7f414406be70 2023-07-14T11:54:25.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.815+0000 7f4149682700 1 -- 172.21.15.104:0/2149254558 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41440b1a10 con 0x7f414406be70 2023-07-14T11:54:25.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.815+0000 7f412b7fe700 1 -- 172.21.15.104:0/2149254558 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f41340038e0 con 0x7f414406be70 2023-07-14T11:54:25.806 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.815+0000 7f412b7fe700 1 --2- 172.21.15.104:0/2149254558 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f41300662f0 0x7f41300687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:25.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.815+0000 7f412b7fe700 1 -- 172.21.15.104:0/2149254558 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f41340835e0 con 0x7f414406be70 2023-07-14T11:54:25.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.815+0000 7f41427fc700 1 --2- 172.21.15.104:0/2149254558 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f41300662f0 0x7f41300687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:25.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.819+0000 7f41427fc700 1 --2- 172.21.15.104:0/2149254558 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f41300662f0 0x7f41300687a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f41440b2160 tx=0x7f4138009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:25.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:25.823+0000 7f412b7fe700 1 -- 172.21.15.104:0/2149254558 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f41440b1a10 con 0x7f414406be70 2023-07-14T11:54:25.993 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.007+0000 7f4149682700 1 -- 172.21.15.104:0/2149254558 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f41440cdc20 con 0x7f414406be70 2023-07-14T11:54:25.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.007+0000 7f412b7fe700 1 -- 172.21.15.104:0/2149254558 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f41340506a0 con 0x7f414406be70 2023-07-14T11:54:25.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.011+0000 7f4149682700 1 -- 172.21.15.104:0/2149254558 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f41300662f0 msgr2=0x7f41300687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.011+0000 7f4149682700 1 --2- 172.21.15.104:0/2149254558 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f41300662f0 0x7f41300687a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f41440b2160 tx=0x7f4138009040 comp rx=0 tx=0).stop 2023-07-14T11:54:25.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.011+0000 7f4149682700 1 -- 172.21.15.104:0/2149254558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f414406be70 msgr2=0x7f414414b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:25.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.011+0000 7f4149682700 1 --2- 172.21.15.104:0/2149254558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f414406be70 0x7f414414b0a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f413400c500 tx=0x7f413400c8c0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.011+0000 7f4149682700 1 -- 172.21.15.104:0/2149254558 shutdown_connections 2023-07-14T11:54:25.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.011+0000 7f4149682700 1 --2- 172.21.15.104:0/2149254558 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f41300662f0 0x7f41300687a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.011+0000 7f4149682700 1 --2- 172.21.15.104:0/2149254558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f41440cb840 0x7f41440aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.011+0000 7f4149682700 1 --2- 172.21.15.104:0/2149254558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f414406d860 0x7f41440c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.011+0000 7f4149682700 1 --2- 172.21.15.104:0/2149254558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f414406be70 0x7f414414b0a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:25.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.011+0000 7f4149682700 1 -- 172.21.15.104:0/2149254558 >> 172.21.15.104:0/2149254558 conn(0x7f41440cad30 msgr2=0x7f41440cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:25.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.011+0000 7f4149682700 1 -- 172.21.15.104:0/2149254558 shutdown_connections 2023-07-14T11:54:25.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.011+0000 7f4149682700 1 -- 172.21.15.104:0/2149254558 wait complete. 2023-07-14T11:54:26.000 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:54:26.020 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:54:26.020 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:54:26.039 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:26.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.223+0000 7f310b09f700 1 Processor -- start 2023-07-14T11:54:26.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.223+0000 7f310b09f700 1 -- start start 2023-07-14T11:54:26.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.227+0000 7f310b09f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31040cd240 0x7f31040ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:26.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.227+0000 7f310b09f700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31040af0a0 con 0x7f31040ceff0 2023-07-14T11:54:26.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.227+0000 7f310b09f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31040cf470 con 0x7f310415bff0 2023-07-14T11:54:26.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.227+0000 7f310b09f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31040cf720 con 0x7f31040cd240 2023-07-14T11:54:26.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.227+0000 7f310a89e700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f31040ceff0 0x7f31040afdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53750/0 (socket says 172.21.15.104:53750) 2023-07-14T11:54:26.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.227+0000 7f310a89e700 1 -- 172.21.15.104:0/3348195513 learned_addr learned my addr 172.21.15.104:0/3348195513 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:26.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.227+0000 7f310989c700 1 --2- 172.21.15.104:0/3348195513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31040cd240 0x7f31040ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:26.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.227+0000 7f310909b700 1 -- 172.21.15.104:0/3348195513 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 796215748 0 0) 0x7f31040cf470 con 0x7f310415bff0 2023-07-14T11:54:26.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.227+0000 7f310909b700 1 -- 172.21.15.104:0/3348195513 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30f00035d0 con 0x7f310415bff0 2023-07-14T11:54:26.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.227+0000 7f310909b700 1 -- 172.21.15.104:0/3348195513 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288578542 0 0) 0x7f31040af0a0 con 0x7f31040ceff0 2023-07-14T11:54:26.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.227+0000 7f310909b700 1 -- 172.21.15.104:0/3348195513 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31040cf470 con 0x7f31040ceff0 2023-07-14T11:54:26.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.227+0000 7f310989c700 1 -- 172.21.15.104:0/3348195513 >> v1:172.21.15.121:6789/0 conn(0x7f310415bff0 legacy=0x7f31040ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:26.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.227+0000 7f310989c700 1 -- 172.21.15.104:0/3348195513 >> v1:172.21.15.104:6789/0 conn(0x7f31040ceff0 legacy=0x7f31040afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:26.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.227+0000 7f310989c700 1 -- 172.21.15.104:0/3348195513 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31040af2f0 con 0x7f31040cd240 2023-07-14T11:54:26.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.227+0000 7f310989c700 1 --2- 172.21.15.104:0/3348195513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31040cd240 0x7f31040ceab0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009880 tx=0x7f30f4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de7f525777f40882 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:26.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.231+0000 7f310909b700 1 -- 172.21.15.104:0/3348195513 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f4011830 con 0x7f31040cd240 2023-07-14T11:54:26.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.231+0000 7f310909b700 1 -- 172.21.15.104:0/3348195513 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f4011990 con 0x7f31040cd240 2023-07-14T11:54:26.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.231+0000 7f310909b700 1 -- 172.21.15.104:0/3348195513 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f4011d20 con 0x7f31040cd240 2023-07-14T11:54:26.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.231+0000 7f310b09f700 1 -- 172.21.15.104:0/3348195513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31040cd240 msgr2=0x7f31040ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:26.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.231+0000 7f310b09f700 1 --2- 172.21.15.104:0/3348195513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31040cd240 0x7f31040ceab0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009880 tx=0x7f30f4009b90 comp rx=0 tx=0).stop 2023-07-14T11:54:26.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.231+0000 7f310b09f700 1 -- 172.21.15.104:0/3348195513 shutdown_connections 2023-07-14T11:54:26.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.231+0000 7f310b09f700 1 --2- 172.21.15.104:0/3348195513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31040cd240 0x7f31040ceab0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.231+0000 7f310b09f700 1 -- 172.21.15.104:0/3348195513 >> 172.21.15.104:0/3348195513 conn(0x7f3104060c40 msgr2=0x7f310409b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:26.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.231+0000 7f310b09f700 1 -- 172.21.15.104:0/3348195513 shutdown_connections 2023-07-14T11:54:26.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.231+0000 7f310b09f700 1 -- 172.21.15.104:0/3348195513 wait complete. 2023-07-14T11:54:26.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310b09f700 1 Processor -- start 2023-07-14T11:54:26.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310b09f700 1 -- start start 2023-07-14T11:54:26.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310b09f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31040cd240 0x7f31040b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:26.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310b09f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31040ceff0 0x7f310415c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:26.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310b09f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f310415bff0 0x7f3104156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:26.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310b09f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310415c990 con 0x7f31040cd240 2023-07-14T11:54:26.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310b09f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310415cc30 con 0x7f31040ceff0 2023-07-14T11:54:26.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310b09f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310415cf10 con 0x7f310415bff0 2023-07-14T11:54:26.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310a09d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31040cd240 0x7f31040b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:26.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310a09d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31040cd240 0x7f31040b7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57076/0 (socket says 172.21.15.104:57076) 2023-07-14T11:54:26.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310a09d700 1 -- 172.21.15.104:0/2277773680 learned_addr learned my addr 172.21.15.104:0/2277773680 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:26.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310989c700 1 --2- 172.21.15.104:0/2277773680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31040ceff0 0x7f310415c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:26.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310a89e700 1 --2- 172.21.15.104:0/2277773680 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f310415bff0 0x7f3104156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:26.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310a09d700 1 -- 172.21.15.104:0/2277773680 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f310415bff0 msgr2=0x7f3104156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:26.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310a09d700 1 --2- 172.21.15.104:0/2277773680 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f310415bff0 0x7f3104156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310a09d700 1 -- 172.21.15.104:0/2277773680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31040ceff0 msgr2=0x7f310415c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:26.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310a09d700 1 --2- 172.21.15.104:0/2277773680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31040ceff0 0x7f310415c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.235+0000 7f310a09d700 1 -- 172.21.15.104:0/2277773680 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f40094b0 con 0x7f31040cd240 2023-07-14T11:54:26.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.239+0000 7f310a09d700 1 --2- 172.21.15.104:0/2277773680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31040cd240 0x7f31040b7c60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f310000cc00 tx=0x7f310000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:26.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.239+0000 7f30faffd700 1 -- 172.21.15.104:0/2277773680 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3100003c20 con 0x7f31040cd240 2023-07-14T11:54:26.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.239+0000 7f30faffd700 1 -- 172.21.15.104:0/2277773680 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3100003d80 con 0x7f31040cd240 2023-07-14T11:54:26.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.239+0000 7f30faffd700 1 -- 172.21.15.104:0/2277773680 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310000ab70 con 0x7f31040cd240 2023-07-14T11:54:26.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.239+0000 7f310b09f700 1 -- 172.21.15.104:0/2277773680 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310415d1f0 con 0x7f31040cd240 2023-07-14T11:54:26.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.239+0000 7f310b09f700 1 -- 172.21.15.104:0/2277773680 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310415d7b0 con 0x7f31040cd240 2023-07-14T11:54:26.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.239+0000 7f310b09f700 1 -- 172.21.15.104:0/2277773680 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310415d9a0 con 0x7f31040cd240 2023-07-14T11:54:26.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.239+0000 7f30faffd700 1 -- 172.21.15.104:0/2277773680 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f310000acd0 con 0x7f31040cd240 2023-07-14T11:54:26.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.243+0000 7f30faffd700 1 --2- 172.21.15.104:0/2277773680 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f30e80662f0 0x7f30e80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:26.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.243+0000 7f30faffd700 1 -- 172.21.15.104:0/2277773680 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f31000841b0 con 0x7f31040cd240 2023-07-14T11:54:26.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.243+0000 7f310989c700 1 --2- 172.21.15.104:0/2277773680 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f30e80662f0 0x7f30e80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:26.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.243+0000 7f310989c700 1 --2- 172.21.15.104:0/2277773680 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f30e80662f0 0x7f30e80687a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f31040cd630 tx=0x7f30f401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:26.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.247+0000 7f30faffd700 1 -- 172.21.15.104:0/2277773680 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f3100051260 con 0x7f31040cd240 2023-07-14T11:54:26.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.423+0000 7f310b09f700 1 -- 172.21.15.104:0/2277773680 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31040af5a0 con 0x7f31040cd240 2023-07-14T11:54:26.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.427+0000 7f30faffd700 1 -- 172.21.15.104:0/2277773680 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20251 (secure 0 0 0) 0x7f3100054880 con 0x7f31040cd240 2023-07-14T11:54:26.412 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:26.412 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":30,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:55.601783+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:26.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.427+0000 7f310b09f700 1 -- 172.21.15.104:0/2277773680 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f30e80662f0 msgr2=0x7f30e80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:26.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.427+0000 7f310b09f700 1 --2- 172.21.15.104:0/2277773680 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f30e80662f0 0x7f30e80687a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f31040cd630 tx=0x7f30f401c040 comp rx=0 tx=0).stop 2023-07-14T11:54:26.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.427+0000 7f310b09f700 1 -- 172.21.15.104:0/2277773680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31040cd240 msgr2=0x7f31040b7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:26.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.427+0000 7f310b09f700 1 --2- 172.21.15.104:0/2277773680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31040cd240 0x7f31040b7c60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f310000cc00 tx=0x7f310000cf10 comp rx=0 tx=0).stop 2023-07-14T11:54:26.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.427+0000 7f310b09f700 1 -- 172.21.15.104:0/2277773680 shutdown_connections 2023-07-14T11:54:26.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.427+0000 7f310b09f700 1 --2- 172.21.15.104:0/2277773680 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f30e80662f0 0x7f30e80687a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.427+0000 7f310b09f700 1 --2- 172.21.15.104:0/2277773680 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f310415bff0 0x7f3104156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.427+0000 7f310b09f700 1 --2- 172.21.15.104:0/2277773680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31040ceff0 0x7f310415c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.427+0000 7f310b09f700 1 --2- 172.21.15.104:0/2277773680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31040cd240 0x7f31040b7c60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.427+0000 7f310b09f700 1 -- 172.21.15.104:0/2277773680 >> 172.21.15.104:0/2277773680 conn(0x7f3104060c40 msgr2=0x7f31040704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:26.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.427+0000 7f310b09f700 1 -- 172.21.15.104:0/2277773680 shutdown_connections 2023-07-14T11:54:26.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.427+0000 7f310b09f700 1 -- 172.21.15.104:0/2277773680 wait complete. 2023-07-14T11:54:26.434 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:54:26.435 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:54:26.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.627+0000 7fe6aa6cd700 1 Processor -- start 2023-07-14T11:54:26.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.627+0000 7fe6aa6cd700 1 -- start start 2023-07-14T11:54:26.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6aa6cd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe6a40b1b80 0x7fe6a40b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:26.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6aa6cd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6a40ce050 0x7fe6a40ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:26.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6aa6cd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe6a40caf20 0x7fe6a40cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:26.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6aa6cd700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a40d1410 con 0x7fe6a40caf20 2023-07-14T11:54:26.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6aa6cd700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a40b5710 con 0x7fe6a40b1b80 2023-07-14T11:54:26.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6aa6cd700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40caad0 con 0x7fe6a40ce050 2023-07-14T11:54:26.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6a3fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe6a40b1b80 0x7fe6a40b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:26.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6a37fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6a40ce050 0x7fe6a40ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:26.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6a3fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe6a40b1b80 0x7fe6a40b51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:43546/0 (socket says 172.21.15.104:43546) 2023-07-14T11:54:26.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6a3fff700 1 -- 172.21.15.104:0/2126321472 learned_addr learned my addr 172.21.15.104:0/2126321472 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:26.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6a8c6a700 1 --2- 172.21.15.104:0/2126321472 >> v2:172.21.15.104:3300/0 conn(0x7fe6a40caf20 0x7fe6a40cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:26.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6a3fff700 1 -- 172.21.15.104:0/2126321472 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6a40ce050 msgr2=0x7fe6a40ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:26.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6a3fff700 1 --2- 172.21.15.104:0/2126321472 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6a40ce050 0x7fe6a40ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6a3fff700 1 -- 172.21.15.104:0/2126321472 >> v2:172.21.15.104:3300/0 conn(0x7fe6a40caf20 msgr2=0x7fe6a40cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:26.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6a3fff700 1 --2- 172.21.15.104:0/2126321472 >> v2:172.21.15.104:3300/0 conn(0x7fe6a40caf20 0x7fe6a40cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6a3fff700 1 -- 172.21.15.104:0/2126321472 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a40ce980 con 0x7fe6a40b1b80 2023-07-14T11:54:26.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6a3fff700 1 --2- 172.21.15.104:0/2126321472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6a40b1b80 0x7fe6a40b51d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe6a406d2c0 tx=0x7fe694009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17f988231ecb9d5c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:26.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6a2ffd700 1 -- 172.21.15.104:0/2126321472 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694005e50 con 0x7fe6a40b1b80 2023-07-14T11:54:26.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6a2ffd700 1 -- 172.21.15.104:0/2126321472 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe694006030 con 0x7fe6a40b1b80 2023-07-14T11:54:26.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.631+0000 7fe6a2ffd700 1 -- 172.21.15.104:0/2126321472 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6940063c0 con 0x7fe6a40b1b80 2023-07-14T11:54:26.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.635+0000 7fe6aa6cd700 1 -- 172.21.15.104:0/2126321472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6a40b1b80 msgr2=0x7fe6a40b51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:26.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.635+0000 7fe6aa6cd700 1 --2- 172.21.15.104:0/2126321472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6a40b1b80 0x7fe6a40b51d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe6a406d2c0 tx=0x7fe694009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.635+0000 7fe6aa6cd700 1 -- 172.21.15.104:0/2126321472 shutdown_connections 2023-07-14T11:54:26.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.635+0000 7fe6aa6cd700 1 --2- 172.21.15.104:0/2126321472 >> v2:172.21.15.104:3300/0 conn(0x7fe6a40caf20 0x7fe6a40cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.635+0000 7fe6aa6cd700 1 --2- 172.21.15.104:0/2126321472 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6a40ce050 0x7fe6a40ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.635+0000 7fe6aa6cd700 1 --2- 172.21.15.104:0/2126321472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6a40b1b80 0x7fe6a40b51d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.635+0000 7fe6aa6cd700 1 -- 172.21.15.104:0/2126321472 >> 172.21.15.104:0/2126321472 conn(0x7fe6a408a000 msgr2=0x7fe6a40d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:26.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.635+0000 7fe6aa6cd700 1 -- 172.21.15.104:0/2126321472 shutdown_connections 2023-07-14T11:54:26.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.635+0000 7fe6aa6cd700 1 -- 172.21.15.104:0/2126321472 wait complete. 2023-07-14T11:54:26.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.635+0000 7fe6aa6cd700 1 Processor -- start 2023-07-14T11:54:26.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.635+0000 7fe6aa6cd700 1 -- start start 2023-07-14T11:54:26.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6aa6cd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6a40b1b80 0x7fe6a4157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:26.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6aa6cd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6a40caf20 0x7fe6a40c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:26.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6aa6cd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6a40ce050 0x7fe6a415a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:26.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6aa6cd700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40c5510 con 0x7fe6a40ce050 2023-07-14T11:54:26.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6aa6cd700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40c59a0 con 0x7fe6a40caf20 2023-07-14T11:54:26.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6aa6cd700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40c4900 con 0x7fe6a40b1b80 2023-07-14T11:54:26.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6a8c6a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6a40ce050 0x7fe6a415a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:26.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6a8c6a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6a40ce050 0x7fe6a415a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57082/0 (socket says 172.21.15.104:57082) 2023-07-14T11:54:26.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6a8c6a700 1 -- 172.21.15.104:0/874338081 learned_addr learned my addr 172.21.15.104:0/874338081 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:26.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6a37fe700 1 --2- 172.21.15.104:0/874338081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6a40caf20 0x7fe6a40c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:26.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6a3fff700 1 --2- 172.21.15.104:0/874338081 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6a40b1b80 0x7fe6a4157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:26.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6a8c6a700 1 -- 172.21.15.104:0/874338081 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6a40b1b80 msgr2=0x7fe6a4157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:26.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6a8c6a700 1 --2- 172.21.15.104:0/874338081 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6a40b1b80 0x7fe6a4157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6a8c6a700 1 -- 172.21.15.104:0/874338081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6a40caf20 msgr2=0x7fe6a40c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:26.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6a8c6a700 1 --2- 172.21.15.104:0/874338081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6a40caf20 0x7fe6a40c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6a8c6a700 1 -- 172.21.15.104:0/874338081 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe694009520 con 0x7fe6a40ce050 2023-07-14T11:54:26.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6a37fe700 1 --2- 172.21.15.104:0/874338081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6a40caf20 0x7fe6a40c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:26.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6a8c6a700 1 --2- 172.21.15.104:0/874338081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6a40ce050 0x7fe6a415a2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00efc0 tx=0x7fe69c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:26.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6a0ff9700 1 -- 172.21.15.104:0/874338081 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c009ea0 con 0x7fe6a40ce050 2023-07-14T11:54:26.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6a0ff9700 1 -- 172.21.15.104:0/874338081 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c00f040 con 0x7fe6a40ce050 2023-07-14T11:54:26.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6aa6cd700 1 -- 172.21.15.104:0/874338081 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a40c4be0 con 0x7fe6a40ce050 2023-07-14T11:54:26.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6a0ff9700 1 -- 172.21.15.104:0/874338081 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c014480 con 0x7fe6a40ce050 2023-07-14T11:54:26.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.639+0000 7fe6aa6cd700 1 -- 172.21.15.104:0/874338081 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a40c51a0 con 0x7fe6a40ce050 2023-07-14T11:54:26.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.643+0000 7fe6aa6cd700 1 -- 172.21.15.104:0/874338081 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a40c56c0 con 0x7fe6a40ce050 2023-07-14T11:54:26.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.643+0000 7fe6a0ff9700 1 -- 172.21.15.104:0/874338081 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe69c0145e0 con 0x7fe6a40ce050 2023-07-14T11:54:26.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.647+0000 7fe6a0ff9700 1 --2- 172.21.15.104:0/874338081 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6900662f0 0x7fe6900687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:26.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.647+0000 7fe6a0ff9700 1 -- 172.21.15.104:0/874338081 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe69c084cc0 con 0x7fe6a40ce050 2023-07-14T11:54:26.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.647+0000 7fe6a3fff700 1 --2- 172.21.15.104:0/874338081 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6900662f0 0x7fe6900687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:26.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.647+0000 7fe6a3fff700 1 --2- 172.21.15.104:0/874338081 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6900662f0 0x7fe6900687a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe694009c30 tx=0x7fe694008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:26.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.651+0000 7fe6a0ff9700 1 -- 172.21.15.104:0/874338081 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe69c08b080 con 0x7fe6a40ce050 2023-07-14T11:54:26.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.823+0000 7fe6aa6cd700 1 -- 172.21.15.104:0/874338081 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6a404dcb0 con 0x7fe6a40ce050 2023-07-14T11:54:26.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.823+0000 7fe6a0ff9700 1 -- 172.21.15.104:0/874338081 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe69c051cf0 con 0x7fe6a40ce050 2023-07-14T11:54:26.811 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:54:26.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.831+0000 7fe6aa6cd700 1 -- 172.21.15.104:0/874338081 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6900662f0 msgr2=0x7fe6900687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:26.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.831+0000 7fe6aa6cd700 1 --2- 172.21.15.104:0/874338081 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6900662f0 0x7fe6900687a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe694009c30 tx=0x7fe694008070 comp rx=0 tx=0).stop 2023-07-14T11:54:26.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.831+0000 7fe6aa6cd700 1 -- 172.21.15.104:0/874338081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6a40ce050 msgr2=0x7fe6a415a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:26.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.831+0000 7fe6aa6cd700 1 --2- 172.21.15.104:0/874338081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6a40ce050 0x7fe6a415a2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00efc0 tx=0x7fe69c00c3a0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.831+0000 7fe6aa6cd700 1 -- 172.21.15.104:0/874338081 shutdown_connections 2023-07-14T11:54:26.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.831+0000 7fe6aa6cd700 1 --2- 172.21.15.104:0/874338081 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6900662f0 0x7fe6900687a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.831+0000 7fe6aa6cd700 1 --2- 172.21.15.104:0/874338081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6a40ce050 0x7fe6a415a2f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.831+0000 7fe6aa6cd700 1 --2- 172.21.15.104:0/874338081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6a40caf20 0x7fe6a40c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.831+0000 7fe6aa6cd700 1 --2- 172.21.15.104:0/874338081 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6a40b1b80 0x7fe6a4157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:26.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.831+0000 7fe6aa6cd700 1 -- 172.21.15.104:0/874338081 >> 172.21.15.104:0/874338081 conn(0x7fe6a408a000 msgr2=0x7fe6a40d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:26.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.831+0000 7fe6aa6cd700 1 -- 172.21.15.104:0/874338081 shutdown_connections 2023-07-14T11:54:26.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:26.831+0000 7fe6aa6cd700 1 -- 172.21.15.104:0/874338081 wait complete. 2023-07-14T11:54:26.837 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:54:27.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.023+0000 7f66dcfd2700 1 Processor -- start 2023-07-14T11:54:27.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.027+0000 7f66dcfd2700 1 -- start start 2023-07-14T11:54:27.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.027+0000 7f66dcfd2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f66d80b0ee0 0x7f66d80ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.027+0000 7f66dcfd2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f66d80cd090 0x7f66d80d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.027+0000 7f66dcfd2700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d80cd4e0 con 0x7f66d80d0610 2023-07-14T11:54:27.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.027+0000 7f66dcfd2700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d80d1a60 con 0x7f66d80b0ee0 2023-07-14T11:54:27.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.027+0000 7f66dcfd2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d80d1ef0 con 0x7f66d80cd090 2023-07-14T11:54:27.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.027+0000 7f66d6ffd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f66d80cd090 0x7f66d80d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.027+0000 7f66d6ffd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f66d80cd090 0x7f66d80d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:55718/0 (socket says 172.21.15.104:55718) 2023-07-14T11:54:27.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.027+0000 7f66d6ffd700 1 -- 172.21.15.104:0/3050915896 learned_addr learned my addr 172.21.15.104:0/3050915896 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:27.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.027+0000 7f66d77fe700 1 --2- 172.21.15.104:0/3050915896 >> v2:172.21.15.121:3300/0 conn(0x7f66d80b0ee0 0x7f66d80ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66d6ffd700 1 -- 172.21.15.104:0/3050915896 >> v2:172.21.15.121:3300/0 conn(0x7f66d80b0ee0 msgr2=0x7f66d80ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:27.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66d6ffd700 1 --2- 172.21.15.104:0/3050915896 >> v2:172.21.15.121:3300/0 conn(0x7f66d80b0ee0 0x7f66d80ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66d6ffd700 1 -- 172.21.15.104:0/3050915896 >> v1:172.21.15.121:6789/0 conn(0x7f66d80d0610 legacy=0x7f66d815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66d6ffd700 1 -- 172.21.15.104:0/3050915896 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d80d0e50 con 0x7f66d80cd090 2023-07-14T11:54:27.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66d6ffd700 1 --2- 172.21.15.104:0/3050915896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f66d80cd090 0x7f66d80d00d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009940 tx=0x7f66cc009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3207f000861c5843 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:27.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66d67fc700 1 -- 172.21.15.104:0/3050915896 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc011830 con 0x7f66d80cd090 2023-07-14T11:54:27.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66d67fc700 1 -- 172.21.15.104:0/3050915896 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66cc011990 con 0x7f66d80cd090 2023-07-14T11:54:27.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66d67fc700 1 -- 172.21.15.104:0/3050915896 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc011d20 con 0x7f66d80cd090 2023-07-14T11:54:27.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66dcfd2700 1 -- 172.21.15.104:0/3050915896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f66d80cd090 msgr2=0x7f66d80d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66dcfd2700 1 --2- 172.21.15.104:0/3050915896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f66d80cd090 0x7f66d80d00d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009940 tx=0x7f66cc009c50 comp rx=0 tx=0).stop 2023-07-14T11:54:27.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66dcfd2700 1 -- 172.21.15.104:0/3050915896 shutdown_connections 2023-07-14T11:54:27.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66dcfd2700 1 --2- 172.21.15.104:0/3050915896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f66d80cd090 0x7f66d80d00d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66dcfd2700 1 --2- 172.21.15.104:0/3050915896 >> v2:172.21.15.121:3300/0 conn(0x7f66d80b0ee0 0x7f66d80ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66dcfd2700 1 -- 172.21.15.104:0/3050915896 >> 172.21.15.104:0/3050915896 conn(0x7f66d8060c40 msgr2=0x7f66d809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:27.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66dcfd2700 1 -- 172.21.15.104:0/3050915896 shutdown_connections 2023-07-14T11:54:27.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.031+0000 7f66dcfd2700 1 -- 172.21.15.104:0/3050915896 wait complete. 2023-07-14T11:54:27.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.035+0000 7f66dcfd2700 1 Processor -- start 2023-07-14T11:54:27.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.035+0000 7f66dcfd2700 1 -- start start 2023-07-14T11:54:27.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66dcfd2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f66d80b0ee0 0x7f66d8156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66dcfd2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66d80cd090 0x7f66d80b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66dcfd2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66d80d0610 0x7f66d80703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66dcfd2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8156fd0 con 0x7f66d80cd090 2023-07-14T11:54:27.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66dcfd2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d80b33e0 con 0x7f66d80d0610 2023-07-14T11:54:27.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66dcfd2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d80cacf0 con 0x7f66d80b0ee0 2023-07-14T11:54:27.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66d6ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66d80cd090 0x7f66d80b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66d6ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66d80cd090 0x7f66d80b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57094/0 (socket says 172.21.15.104:57094) 2023-07-14T11:54:27.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66d6ffd700 1 -- 172.21.15.104:0/3359968383 learned_addr learned my addr 172.21.15.104:0/3359968383 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:27.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66d77fe700 1 --2- 172.21.15.104:0/3359968383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f66d80b0ee0 0x7f66d8156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66d7fff700 1 --2- 172.21.15.104:0/3359968383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66d80d0610 0x7f66d80703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66d6ffd700 1 -- 172.21.15.104:0/3359968383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f66d80b0ee0 msgr2=0x7f66d8156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66d6ffd700 1 --2- 172.21.15.104:0/3359968383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f66d80b0ee0 0x7f66d8156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66d6ffd700 1 -- 172.21.15.104:0/3359968383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66d80d0610 msgr2=0x7f66d80703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66d6ffd700 1 --2- 172.21.15.104:0/3359968383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66d80d0610 0x7f66d80703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66d6ffd700 1 -- 172.21.15.104:0/3359968383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc009570 con 0x7f66d80cd090 2023-07-14T11:54:27.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66d7fff700 1 --2- 172.21.15.104:0/3359968383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66d80d0610 0x7f66d80703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:27.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66d6ffd700 1 --2- 172.21.15.104:0/3359968383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66d80cd090 0x7f66d80b73e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009fd0 tx=0x7f66cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:27.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66bffff700 1 -- 172.21.15.104:0/3359968383 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc004450 con 0x7f66d80cd090 2023-07-14T11:54:27.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66bffff700 1 -- 172.21.15.104:0/3359968383 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66cc0045b0 con 0x7f66d80cd090 2023-07-14T11:54:27.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66dcfd2700 1 -- 172.21.15.104:0/3359968383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d80cafd0 con 0x7f66d80cd090 2023-07-14T11:54:27.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66bffff700 1 -- 172.21.15.104:0/3359968383 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0038c0 con 0x7f66d80cd090 2023-07-14T11:54:27.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.039+0000 7f66dcfd2700 1 -- 172.21.15.104:0/3359968383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d80cb590 con 0x7f66d80cd090 2023-07-14T11:54:27.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.043+0000 7f66dcfd2700 1 -- 172.21.15.104:0/3359968383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d804e500 con 0x7f66d80cd090 2023-07-14T11:54:27.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.043+0000 7f66bffff700 1 -- 172.21.15.104:0/3359968383 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f66cc003a20 con 0x7f66d80cd090 2023-07-14T11:54:27.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.047+0000 7f66bffff700 1 --2- 172.21.15.104:0/3359968383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f66c40662f0 0x7f66c40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.047+0000 7f66bffff700 1 -- 172.21.15.104:0/3359968383 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f66cc0853e0 con 0x7f66d80cd090 2023-07-14T11:54:27.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.047+0000 7f66d77fe700 1 --2- 172.21.15.104:0/3359968383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f66c40662f0 0x7f66c40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.047+0000 7f66d77fe700 1 --2- 172.21.15.104:0/3359968383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f66c40662f0 0x7f66c40687a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009b50 tx=0x7f66c8009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:27.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.051+0000 7f66bffff700 1 -- 172.21.15.104:0/3359968383 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f66cc052660 con 0x7f66d80cd090 2023-07-14T11:54:27.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.219+0000 7f66dcfd2700 1 -- 172.21.15.104:0/3359968383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f66d80b18d0 con 0x7f66d80cd090 2023-07-14T11:54:27.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.219+0000 7f66bffff700 1 -- 172.21.15.104:0/3359968383 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f66cc055c80 con 0x7f66d80cd090 2023-07-14T11:54:27.205 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:54:27.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.223+0000 7f66dcfd2700 1 -- 172.21.15.104:0/3359968383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f66c40662f0 msgr2=0x7f66c40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.223+0000 7f66dcfd2700 1 --2- 172.21.15.104:0/3359968383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f66c40662f0 0x7f66c40687a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009b50 tx=0x7f66c8009240 comp rx=0 tx=0).stop 2023-07-14T11:54:27.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.223+0000 7f66dcfd2700 1 -- 172.21.15.104:0/3359968383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66d80cd090 msgr2=0x7f66d80b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.223+0000 7f66dcfd2700 1 --2- 172.21.15.104:0/3359968383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66d80cd090 0x7f66d80b73e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009fd0 tx=0x7f66cc004770 comp rx=0 tx=0).stop 2023-07-14T11:54:27.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.223+0000 7f66dcfd2700 1 -- 172.21.15.104:0/3359968383 shutdown_connections 2023-07-14T11:54:27.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.223+0000 7f66dcfd2700 1 --2- 172.21.15.104:0/3359968383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f66c40662f0 0x7f66c40687a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.223+0000 7f66dcfd2700 1 --2- 172.21.15.104:0/3359968383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66d80cd090 0x7f66d80b73e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.223+0000 7f66dcfd2700 1 --2- 172.21.15.104:0/3359968383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66d80d0610 0x7f66d80703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.223+0000 7f66dcfd2700 1 --2- 172.21.15.104:0/3359968383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f66d80b0ee0 0x7f66d8156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.223+0000 7f66dcfd2700 1 -- 172.21.15.104:0/3359968383 >> 172.21.15.104:0/3359968383 conn(0x7f66d8060c40 msgr2=0x7f66d806fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:27.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.223+0000 7f66dcfd2700 1 -- 172.21.15.104:0/3359968383 shutdown_connections 2023-07-14T11:54:27.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.223+0000 7f66dcfd2700 1 -- 172.21.15.104:0/3359968383 wait complete. 2023-07-14T11:54:27.229 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:54:27.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.415+0000 7f70299a7700 1 Processor -- start 2023-07-14T11:54:27.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.415+0000 7f70299a7700 1 -- start start 2023-07-14T11:54:27.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f70299a7700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f70240afd80 0x7f702415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f70299a7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70240c8950 0x7f70240c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f70299a7700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70240c9290 con 0x7f702415df10 2023-07-14T11:54:27.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f70299a7700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f702415e830 con 0x7f70240afd80 2023-07-14T11:54:27.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f70299a7700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7024070a30 con 0x7f70240c8950 2023-07-14T11:54:27.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f70227fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f70240afd80 0x7f702415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f70227fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f70240afd80 0x7f702415e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57100/0 (socket says 172.21.15.104:57100) 2023-07-14T11:54:27.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f70227fc700 1 -- 172.21.15.104:0/551171470 learned_addr learned my addr 172.21.15.104:0/551171470 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:27.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f70237fe700 1 --2- 172.21.15.104:0/551171470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70240c8950 0x7f70240c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f70237fe700 1 -- 172.21.15.104:0/551171470 >> v2:172.21.15.104:3300/0 conn(0x7f70240afd80 msgr2=0x7f702415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:27.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f70237fe700 1 --2- 172.21.15.104:0/551171470 >> v2:172.21.15.104:3300/0 conn(0x7f70240afd80 0x7f702415e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f70237fe700 1 -- 172.21.15.104:0/551171470 >> v1:172.21.15.121:6789/0 conn(0x7f702415df10 legacy=0x7f70240711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f70237fe700 1 -- 172.21.15.104:0/551171470 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70240b0210 con 0x7f70240c8950 2023-07-14T11:54:27.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f70237fe700 1 --2- 172.21.15.104:0/551171470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70240c8950 0x7f70240c8d50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f701c009d90 tx=0x7f701c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=260f2c36440313f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:27.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f7021ffb700 1 -- 172.21.15.104:0/551171470 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c00e070 con 0x7f70240c8950 2023-07-14T11:54:27.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f7021ffb700 1 -- 172.21.15.104:0/551171470 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701c00bb20 con 0x7f70240c8950 2023-07-14T11:54:27.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.419+0000 7f7021ffb700 1 -- 172.21.15.104:0/551171470 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c012430 con 0x7f70240c8950 2023-07-14T11:54:27.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.423+0000 7f70299a7700 1 -- 172.21.15.104:0/551171470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70240c8950 msgr2=0x7f70240c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.423+0000 7f70299a7700 1 --2- 172.21.15.104:0/551171470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70240c8950 0x7f70240c8d50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f701c009d90 tx=0x7f701c00b330 comp rx=0 tx=0).stop 2023-07-14T11:54:27.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.423+0000 7f70299a7700 1 -- 172.21.15.104:0/551171470 shutdown_connections 2023-07-14T11:54:27.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.423+0000 7f70299a7700 1 --2- 172.21.15.104:0/551171470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70240c8950 0x7f70240c8d50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.423+0000 7f70299a7700 1 --2- 172.21.15.104:0/551171470 >> v2:172.21.15.104:3300/0 conn(0x7f70240afd80 0x7f702415e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.423+0000 7f70299a7700 1 -- 172.21.15.104:0/551171470 >> 172.21.15.104:0/551171470 conn(0x7f70240cad80 msgr2=0x7f70240cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:27.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.423+0000 7f70299a7700 1 -- 172.21.15.104:0/551171470 shutdown_connections 2023-07-14T11:54:27.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.423+0000 7f70299a7700 1 -- 172.21.15.104:0/551171470 wait complete. 2023-07-14T11:54:27.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.423+0000 7f70299a7700 1 Processor -- start 2023-07-14T11:54:27.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f70299a7700 1 -- start start 2023-07-14T11:54:27.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f70299a7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70240afd80 0x7f7024158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f70299a7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70240c8950 0x7f702406d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f70299a7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f702406dc80 0x7f70241ef400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f70299a7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70241f0470 con 0x7f70240afd80 2023-07-14T11:54:27.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f70299a7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702406e100 con 0x7f70240c8950 2023-07-14T11:54:27.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f70299a7700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70241f0dd0 con 0x7f702406dc80 2023-07-14T11:54:27.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f7022ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70240afd80 0x7f7024158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f7022ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70240afd80 0x7f7024158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57102/0 (socket says 172.21.15.104:57102) 2023-07-14T11:54:27.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f7022ffd700 1 -- 172.21.15.104:0/3522792645 learned_addr learned my addr 172.21.15.104:0/3522792645 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:27.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f70227fc700 1 --2- 172.21.15.104:0/3522792645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70240c8950 0x7f702406d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f70237fe700 1 --2- 172.21.15.104:0/3522792645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f702406dc80 0x7f70241ef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f7022ffd700 1 -- 172.21.15.104:0/3522792645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f702406dc80 msgr2=0x7f70241ef400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f7022ffd700 1 --2- 172.21.15.104:0/3522792645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f702406dc80 0x7f70241ef400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f7022ffd700 1 -- 172.21.15.104:0/3522792645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70240c8950 msgr2=0x7f702406d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f7022ffd700 1 --2- 172.21.15.104:0/3522792645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70240c8950 0x7f702406d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f7022ffd700 1 -- 172.21.15.104:0/3522792645 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c009a70 con 0x7f70240afd80 2023-07-14T11:54:27.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f7022ffd700 1 --2- 172.21.15.104:0/3522792645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70240afd80 0x7f7024158a60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f701400c500 tx=0x7f701400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:27.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f700bfff700 1 -- 172.21.15.104:0/3522792645 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70140052d0 con 0x7f70240afd80 2023-07-14T11:54:27.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f700bfff700 1 -- 172.21.15.104:0/3522792645 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7014005430 con 0x7f70240afd80 2023-07-14T11:54:27.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.427+0000 7f700bfff700 1 -- 172.21.15.104:0/3522792645 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70140055a0 con 0x7f70240afd80 2023-07-14T11:54:27.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.431+0000 7f70299a7700 1 -- 172.21.15.104:0/3522792645 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7024158fa0 con 0x7f70240afd80 2023-07-14T11:54:27.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.431+0000 7f70299a7700 1 -- 172.21.15.104:0/3522792645 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70241f0780 con 0x7f70240afd80 2023-07-14T11:54:27.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.431+0000 7f70299a7700 1 -- 172.21.15.104:0/3522792645 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70241f05e0 con 0x7f70240afd80 2023-07-14T11:54:27.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.431+0000 7f700bfff700 1 -- 172.21.15.104:0/3522792645 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f70140038e0 con 0x7f70240afd80 2023-07-14T11:54:27.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.435+0000 7f700bfff700 1 --2- 172.21.15.104:0/3522792645 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7010066320 0x7f70100687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.435+0000 7f700bfff700 1 -- 172.21.15.104:0/3522792645 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f70140833d0 con 0x7f70240afd80 2023-07-14T11:54:27.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.435+0000 7f70227fc700 1 --2- 172.21.15.104:0/3522792645 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7010066320 0x7f70100687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.435+0000 7f70227fc700 1 --2- 172.21.15.104:0/3522792645 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7010066320 0x7f70100687d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f70240cb810 tx=0x7f7018008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:27.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.439+0000 7f700bfff700 1 -- 172.21.15.104:0/3522792645 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f70241f05e0 con 0x7f70240afd80 2023-07-14T11:54:27.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.611+0000 7f70299a7700 1 -- 172.21.15.104:0/3522792645 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f70241f05e0 con 0x7f70240afd80 2023-07-14T11:54:27.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.615+0000 7f700bfff700 1 -- 172.21.15.104:0/3522792645 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f70241f05e0 con 0x7f70240afd80 2023-07-14T11:54:27.599 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:27.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.615+0000 7f70299a7700 1 -- 172.21.15.104:0/3522792645 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7010066320 msgr2=0x7f70100687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.615+0000 7f70299a7700 1 --2- 172.21.15.104:0/3522792645 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7010066320 0x7f70100687d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f70240cb810 tx=0x7f7018008040 comp rx=0 tx=0).stop 2023-07-14T11:54:27.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.615+0000 7f70299a7700 1 -- 172.21.15.104:0/3522792645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70240afd80 msgr2=0x7f7024158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.615+0000 7f70299a7700 1 --2- 172.21.15.104:0/3522792645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70240afd80 0x7f7024158a60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f701400c500 tx=0x7f701400c8c0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.615+0000 7f70299a7700 1 -- 172.21.15.104:0/3522792645 shutdown_connections 2023-07-14T11:54:27.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.615+0000 7f70299a7700 1 --2- 172.21.15.104:0/3522792645 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7010066320 0x7f70100687d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.615+0000 7f70299a7700 1 --2- 172.21.15.104:0/3522792645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f702406dc80 0x7f70241ef400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.615+0000 7f70299a7700 1 --2- 172.21.15.104:0/3522792645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70240c8950 0x7f702406d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.615+0000 7f70299a7700 1 --2- 172.21.15.104:0/3522792645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70240afd80 0x7f7024158a60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.615+0000 7f70299a7700 1 -- 172.21.15.104:0/3522792645 >> 172.21.15.104:0/3522792645 conn(0x7f70240cad80 msgr2=0x7f7024159870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:27.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.619+0000 7f70299a7700 1 -- 172.21.15.104:0/3522792645 shutdown_connections 2023-07-14T11:54:27.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.619+0000 7f70299a7700 1 -- 172.21.15.104:0/3522792645 wait complete. 2023-07-14T11:54:27.625 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:54:27.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.819+0000 7f46c69c1700 1 Processor -- start 2023-07-14T11:54:27.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.819+0000 7f46c69c1700 1 -- start start 2023-07-14T11:54:27.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c69c1700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46c00cdcb0 0x7f46c0071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c69c1700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f46c00c9270 0x7f46c00cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c69c1700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c0158970 con 0x7f46c00b0680 2023-07-14T11:54:27.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c69c1700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c00ce080 con 0x7f46c00c9270 2023-07-14T11:54:27.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c69c1700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c0158bf0 con 0x7f46c00cdcb0 2023-07-14T11:54:27.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46bf7fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f46c00b0680 0x7f46c00c8b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53758/0 (socket says 172.21.15.104:53758) 2023-07-14T11:54:27.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46bf7fe700 1 -- 172.21.15.104:0/926271719 learned_addr learned my addr 172.21.15.104:0/926271719 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:27.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c4f5e700 1 --2- 172.21.15.104:0/926271719 >> v2:172.21.15.104:3300/0 conn(0x7f46c00c9270 0x7f46c00cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46beffd700 1 -- 172.21.15.104:0/926271719 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3278360103 0 0) 0x7f46c0158970 con 0x7f46c00b0680 2023-07-14T11:54:27.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46bffff700 1 --2- 172.21.15.104:0/926271719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46c00cdcb0 0x7f46c0071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46beffd700 1 -- 172.21.15.104:0/926271719 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46a80034f0 con 0x7f46c00b0680 2023-07-14T11:54:27.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c4f5e700 1 -- 172.21.15.104:0/926271719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46c00cdcb0 msgr2=0x7f46c0071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:27.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c4f5e700 1 --2- 172.21.15.104:0/926271719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46c00cdcb0 0x7f46c0071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c4f5e700 1 -- 172.21.15.104:0/926271719 >> v1:172.21.15.104:6789/0 conn(0x7f46c00b0680 legacy=0x7f46c00c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c4f5e700 1 -- 172.21.15.104:0/926271719 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c0158ed0 con 0x7f46c00c9270 2023-07-14T11:54:27.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46bffff700 1 --2- 172.21.15.104:0/926271719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46c00cdcb0 0x7f46c0071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:27.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c4f5e700 1 --2- 172.21.15.104:0/926271719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c00c9270 0x7f46c00cc080 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009d90 tx=0x7f46b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8dce2c2582359c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:27.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46beffd700 1 -- 172.21.15.104:0/926271719 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b800e070 con 0x7f46c00c9270 2023-07-14T11:54:27.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46beffd700 1 -- 172.21.15.104:0/926271719 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b800bb20 con 0x7f46c00c9270 2023-07-14T11:54:27.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46beffd700 1 -- 172.21.15.104:0/926271719 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b800be90 con 0x7f46c00c9270 2023-07-14T11:54:27.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c69c1700 1 -- 172.21.15.104:0/926271719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c00c9270 msgr2=0x7f46c00cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c69c1700 1 --2- 172.21.15.104:0/926271719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c00c9270 0x7f46c00cc080 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009d90 tx=0x7f46b800b330 comp rx=0 tx=0).stop 2023-07-14T11:54:27.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c69c1700 1 -- 172.21.15.104:0/926271719 shutdown_connections 2023-07-14T11:54:27.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c69c1700 1 --2- 172.21.15.104:0/926271719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c00c9270 0x7f46c00cc080 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c69c1700 1 --2- 172.21.15.104:0/926271719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46c00cdcb0 0x7f46c0071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c69c1700 1 -- 172.21.15.104:0/926271719 >> 172.21.15.104:0/926271719 conn(0x7f46c00cad80 msgr2=0x7f46c00cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:27.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c69c1700 1 -- 172.21.15.104:0/926271719 shutdown_connections 2023-07-14T11:54:27.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.823+0000 7f46c69c1700 1 -- 172.21.15.104:0/926271719 wait complete. 2023-07-14T11:54:27.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.827+0000 7f46c69c1700 1 Processor -- start 2023-07-14T11:54:27.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.827+0000 7f46c69c1700 1 -- start start 2023-07-14T11:54:27.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46c69c1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c00b0680 0x7f46c0151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46c69c1700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46c00c9270 0x7f46c00b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46c69c1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46c00cdcb0 0x7f46c00b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46c69c1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c00c5700 con 0x7f46c00b0680 2023-07-14T11:54:27.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46c69c1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c01540b0 con 0x7f46c00cdcb0 2023-07-14T11:54:27.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46c69c1700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c0154390 con 0x7f46c00c9270 2023-07-14T11:54:27.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46bffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c00b0680 0x7f46c0151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46bffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c00b0680 0x7f46c0151380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57120/0 (socket says 172.21.15.104:57120) 2023-07-14T11:54:27.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46bffff700 1 -- 172.21.15.104:0/352040521 learned_addr learned my addr 172.21.15.104:0/352040521 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:27.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46c4f5e700 1 --2- 172.21.15.104:0/352040521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46c00cdcb0 0x7f46c00b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46bf7fe700 1 --2- 172.21.15.104:0/352040521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46c00c9270 0x7f46c00b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46bffff700 1 -- 172.21.15.104:0/352040521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46c00c9270 msgr2=0x7f46c00b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46bffff700 1 --2- 172.21.15.104:0/352040521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46c00c9270 0x7f46c00b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46bffff700 1 -- 172.21.15.104:0/352040521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46c00cdcb0 msgr2=0x7f46c00b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:27.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46bffff700 1 --2- 172.21.15.104:0/352040521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46c00cdcb0 0x7f46c00b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:27.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46bffff700 1 -- 172.21.15.104:0/352040521 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b8009a70 con 0x7f46c00b0680 2023-07-14T11:54:27.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46bf7fe700 1 --2- 172.21.15.104:0/352040521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46c00c9270 0x7f46c00b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:27.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46bffff700 1 --2- 172.21.15.104:0/352040521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c00b0680 0x7f46c0151380 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f46b000c770 tx=0x7f46b000ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:27.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46bcff9700 1 -- 172.21.15.104:0/352040521 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b0011800 con 0x7f46c00b0680 2023-07-14T11:54:27.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46bcff9700 1 -- 172.21.15.104:0/352040521 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b0011960 con 0x7f46c00b0680 2023-07-14T11:54:27.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46c69c1700 1 -- 172.21.15.104:0/352040521 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c0154670 con 0x7f46c00b0680 2023-07-14T11:54:27.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46bcff9700 1 -- 172.21.15.104:0/352040521 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b0011d10 con 0x7f46c00b0680 2023-07-14T11:54:27.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.831+0000 7f46c69c1700 1 -- 172.21.15.104:0/352040521 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c0154950 con 0x7f46c00b0680 2023-07-14T11:54:27.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.835+0000 7f46c69c1700 1 -- 172.21.15.104:0/352040521 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c0061060 con 0x7f46c00b0680 2023-07-14T11:54:27.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.835+0000 7f46bcff9700 1 -- 172.21.15.104:0/352040521 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f46b0018360 con 0x7f46c00b0680 2023-07-14T11:54:27.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.839+0000 7f46bcff9700 1 --2- 172.21.15.104:0/352040521 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46ac0662f0 0x7f46ac0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:27.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.839+0000 7f46bcff9700 1 -- 172.21.15.104:0/352040521 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f46b0084110 con 0x7f46c00b0680 2023-07-14T11:54:27.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.839+0000 7f46bf7fe700 1 --2- 172.21.15.104:0/352040521 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46ac0662f0 0x7f46ac0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:27.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.839+0000 7f46bf7fe700 1 --2- 172.21.15.104:0/352040521 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46ac0662f0 0x7f46ac0687a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f46c00ca260 tx=0x7f46b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:27.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:27.847+0000 7f46bcff9700 1 -- 172.21.15.104:0/352040521 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f46b00511d0 con 0x7f46c00b0680 2023-07-14T11:54:28.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.019+0000 7f46c69c1700 1 -- 172.21.15.104:0/352040521 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f46c004dcb0 con 0x7f46c00b0680 2023-07-14T11:54:28.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.023+0000 7f46bcff9700 1 -- 172.21.15.104:0/352040521 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f46b00547f0 con 0x7f46c00b0680 2023-07-14T11:54:28.007 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:54:28.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.023+0000 7f46c69c1700 1 -- 172.21.15.104:0/352040521 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46ac0662f0 msgr2=0x7f46ac0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:28.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.023+0000 7f46c69c1700 1 --2- 172.21.15.104:0/352040521 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46ac0662f0 0x7f46ac0687a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f46c00ca260 tx=0x7f46b4009040 comp rx=0 tx=0).stop 2023-07-14T11:54:28.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.023+0000 7f46c69c1700 1 -- 172.21.15.104:0/352040521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c00b0680 msgr2=0x7f46c0151380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:28.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.023+0000 7f46c69c1700 1 --2- 172.21.15.104:0/352040521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c00b0680 0x7f46c0151380 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f46b000c770 tx=0x7f46b000ca80 comp rx=0 tx=0).stop 2023-07-14T11:54:28.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.023+0000 7f46c69c1700 1 -- 172.21.15.104:0/352040521 shutdown_connections 2023-07-14T11:54:28.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.023+0000 7f46c69c1700 1 --2- 172.21.15.104:0/352040521 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46ac0662f0 0x7f46ac0687a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.023+0000 7f46c69c1700 1 --2- 172.21.15.104:0/352040521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46c00cdcb0 0x7f46c00b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.023+0000 7f46c69c1700 1 --2- 172.21.15.104:0/352040521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46c00c9270 0x7f46c00b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.023+0000 7f46c69c1700 1 --2- 172.21.15.104:0/352040521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46c00b0680 0x7f46c0151380 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.023+0000 7f46c69c1700 1 -- 172.21.15.104:0/352040521 >> 172.21.15.104:0/352040521 conn(0x7f46c00cad80 msgr2=0x7f46c0152210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:28.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.023+0000 7f46c69c1700 1 -- 172.21.15.104:0/352040521 shutdown_connections 2023-07-14T11:54:28.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.023+0000 7f46c69c1700 1 -- 172.21.15.104:0/352040521 wait complete. 2023-07-14T11:54:28.030 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:54:28.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.219+0000 7f0b1a39e700 1 Processor -- start 2023-07-14T11:54:28.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.219+0000 7f0b1a39e700 1 -- start start 2023-07-14T11:54:28.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b1a39e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0b14160000 0x7f0b1406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:28.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b1a39e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b141603d0 0x7f0b140c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:28.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b1a39e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b140cdfb0 con 0x7f0b140cd7f0 2023-07-14T11:54:28.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b1a39e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b14070e40 con 0x7f0b14160000 2023-07-14T11:54:28.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b1a39e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140cdcd0 con 0x7f0b141603d0 2023-07-14T11:54:28.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b1939c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0b14160000 0x7f0b1406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:28.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b19b9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f0b140cd7f0 0x7f0b140c0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53762/0 (socket says 172.21.15.104:53762) 2023-07-14T11:54:28.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b19b9d700 1 -- 172.21.15.104:0/3142836295 learned_addr learned my addr 172.21.15.104:0/3142836295 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:28.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b18b9b700 1 --2- 172.21.15.104:0/3142836295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b141603d0 0x7f0b140c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:28.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b0bfff700 1 -- 172.21.15.104:0/3142836295 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181438978 0 0) 0x7f0b0c001110 con 0x7f0b140cd7f0 2023-07-14T11:54:28.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b0bfff700 1 -- 172.21.15.104:0/3142836295 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b000035d0 con 0x7f0b140cd7f0 2023-07-14T11:54:28.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b1939c700 1 -- 172.21.15.104:0/3142836295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b141603d0 msgr2=0x7f0b140c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:28.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b1939c700 1 --2- 172.21.15.104:0/3142836295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b141603d0 0x7f0b140c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b1939c700 1 -- 172.21.15.104:0/3142836295 >> v1:172.21.15.104:6789/0 conn(0x7f0b140cd7f0 legacy=0x7f0b140c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:28.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b1939c700 1 -- 172.21.15.104:0/3142836295 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b14070780 con 0x7f0b14160000 2023-07-14T11:54:28.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b1939c700 1 --2- 172.21.15.104:0/3142836295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b14160000 0x7f0b1406dd10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009d90 tx=0x7f0b1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d1dc6c2876bb4a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:28.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b0bfff700 1 -- 172.21.15.104:0/3142836295 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1000e070 con 0x7f0b14160000 2023-07-14T11:54:28.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b0bfff700 1 -- 172.21.15.104:0/3142836295 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1000bb20 con 0x7f0b14160000 2023-07-14T11:54:28.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.223+0000 7f0b0bfff700 1 -- 172.21.15.104:0/3142836295 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10012430 con 0x7f0b14160000 2023-07-14T11:54:28.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.227+0000 7f0b1a39e700 1 -- 172.21.15.104:0/3142836295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b14160000 msgr2=0x7f0b1406dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:28.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.227+0000 7f0b1a39e700 1 --2- 172.21.15.104:0/3142836295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b14160000 0x7f0b1406dd10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009d90 tx=0x7f0b1000b330 comp rx=0 tx=0).stop 2023-07-14T11:54:28.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.227+0000 7f0b1a39e700 1 -- 172.21.15.104:0/3142836295 shutdown_connections 2023-07-14T11:54:28.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.227+0000 7f0b1a39e700 1 --2- 172.21.15.104:0/3142836295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b141603d0 0x7f0b140c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.227+0000 7f0b1a39e700 1 --2- 172.21.15.104:0/3142836295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b14160000 0x7f0b1406dd10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.227+0000 7f0b1a39e700 1 -- 172.21.15.104:0/3142836295 >> 172.21.15.104:0/3142836295 conn(0x7f0b140afc60 msgr2=0x7f0b140b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:28.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.227+0000 7f0b1a39e700 1 -- 172.21.15.104:0/3142836295 shutdown_connections 2023-07-14T11:54:28.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.227+0000 7f0b1a39e700 1 -- 172.21.15.104:0/3142836295 wait complete. 2023-07-14T11:54:28.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.227+0000 7f0b1a39e700 1 Processor -- start 2023-07-14T11:54:28.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.227+0000 7f0b1a39e700 1 -- start start 2023-07-14T11:54:28.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b1a39e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b140cd7f0 0x7f0b141546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:28.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b1a39e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b14160000 0x7f0b140ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:28.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b1a39e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b141603d0 0x7f0b140cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:28.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b1a39e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140b2750 con 0x7f0b141603d0 2023-07-14T11:54:28.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b1a39e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b14154c20 con 0x7f0b14160000 2023-07-14T11:54:28.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b1a39e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140c75e0 con 0x7f0b140cd7f0 2023-07-14T11:54:28.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b19b9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b141603d0 0x7f0b140cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:28.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b19b9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b141603d0 0x7f0b140cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57144/0 (socket says 172.21.15.104:57144) 2023-07-14T11:54:28.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b19b9d700 1 -- 172.21.15.104:0/4281911008 learned_addr learned my addr 172.21.15.104:0/4281911008 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:28.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b1939c700 1 --2- 172.21.15.104:0/4281911008 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b140cd7f0 0x7f0b141546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:28.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b19b9d700 1 -- 172.21.15.104:0/4281911008 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b140cd7f0 msgr2=0x7f0b141546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:28.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b18b9b700 1 --2- 172.21.15.104:0/4281911008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b14160000 0x7f0b140ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:28.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b19b9d700 1 --2- 172.21.15.104:0/4281911008 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b140cd7f0 0x7f0b141546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b19b9d700 1 -- 172.21.15.104:0/4281911008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b14160000 msgr2=0x7f0b140ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:28.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b19b9d700 1 --2- 172.21.15.104:0/4281911008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b14160000 0x7f0b140ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b19b9d700 1 -- 172.21.15.104:0/4281911008 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b10009a70 con 0x7f0b141603d0 2023-07-14T11:54:28.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b18b9b700 1 --2- 172.21.15.104:0/4281911008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b14160000 0x7f0b140ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:28.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b19b9d700 1 --2- 172.21.15.104:0/4281911008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b141603d0 0x7f0b140cac30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00c730 tx=0x7f0b0c00caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:28.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b09ffb700 1 -- 172.21.15.104:0/4281911008 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c005d90 con 0x7f0b141603d0 2023-07-14T11:54:28.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b09ffb700 1 -- 172.21.15.104:0/4281911008 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c0070d0 con 0x7f0b141603d0 2023-07-14T11:54:28.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b09ffb700 1 -- 172.21.15.104:0/4281911008 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c007460 con 0x7f0b141603d0 2023-07-14T11:54:28.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b1a39e700 1 -- 172.21.15.104:0/4281911008 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b140b2100 con 0x7f0b141603d0 2023-07-14T11:54:28.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.231+0000 7f0b1a39e700 1 -- 172.21.15.104:0/4281911008 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b140c78c0 con 0x7f0b141603d0 2023-07-14T11:54:28.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.235+0000 7f0b1a39e700 1 -- 172.21.15.104:0/4281911008 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b14071240 con 0x7f0b141603d0 2023-07-14T11:54:28.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.235+0000 7f0b09ffb700 1 -- 172.21.15.104:0/4281911008 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f0b0c0075c0 con 0x7f0b141603d0 2023-07-14T11:54:28.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.235+0000 7f0b09ffb700 1 --2- 172.21.15.104:0/4281911008 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0af80662f0 0x7f0af80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:28.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.235+0000 7f0b09ffb700 1 -- 172.21.15.104:0/4281911008 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0b0c083370 con 0x7f0b141603d0 2023-07-14T11:54:28.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.235+0000 7f0b1939c700 1 --2- 172.21.15.104:0/4281911008 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0af80662f0 0x7f0af80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:28.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.239+0000 7f0b1939c700 1 --2- 172.21.15.104:0/4281911008 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0af80662f0 0x7f0af80687a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0b10014040 tx=0x7f0b10011000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:28.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.239+0000 7f0b09ffb700 1 -- 172.21.15.104:0/4281911008 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f0b14071240 con 0x7f0b141603d0 2023-07-14T11:54:28.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.411+0000 7f0b1a39e700 1 -- 172.21.15.104:0/4281911008 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0b140cdfb0 con 0x7f0b141603d0 2023-07-14T11:54:28.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.411+0000 7f0b09ffb700 1 -- 172.21.15.104:0/4281911008 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0b140cdfb0 con 0x7f0b141603d0 2023-07-14T11:54:28.398 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:28.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.419+0000 7f0b1a39e700 1 -- 172.21.15.104:0/4281911008 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0af80662f0 msgr2=0x7f0af80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:28.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.419+0000 7f0b1a39e700 1 --2- 172.21.15.104:0/4281911008 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0af80662f0 0x7f0af80687a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0b10014040 tx=0x7f0b10011000 comp rx=0 tx=0).stop 2023-07-14T11:54:28.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.419+0000 7f0b1a39e700 1 -- 172.21.15.104:0/4281911008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b141603d0 msgr2=0x7f0b140cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:28.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.419+0000 7f0b1a39e700 1 --2- 172.21.15.104:0/4281911008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b141603d0 0x7f0b140cac30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00c730 tx=0x7f0b0c00caf0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.419+0000 7f0b1a39e700 1 -- 172.21.15.104:0/4281911008 shutdown_connections 2023-07-14T11:54:28.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.419+0000 7f0b1a39e700 1 --2- 172.21.15.104:0/4281911008 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0af80662f0 0x7f0af80687a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.419+0000 7f0b1a39e700 1 --2- 172.21.15.104:0/4281911008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b141603d0 0x7f0b140cac30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.419+0000 7f0b1a39e700 1 --2- 172.21.15.104:0/4281911008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b14160000 0x7f0b140ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.419+0000 7f0b1a39e700 1 --2- 172.21.15.104:0/4281911008 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b140cd7f0 0x7f0b141546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.419+0000 7f0b1a39e700 1 -- 172.21.15.104:0/4281911008 >> 172.21.15.104:0/4281911008 conn(0x7f0b140afc60 msgr2=0x7f0b140c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:28.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.419+0000 7f0b1a39e700 1 -- 172.21.15.104:0/4281911008 shutdown_connections 2023-07-14T11:54:28.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.419+0000 7f0b1a39e700 1 -- 172.21.15.104:0/4281911008 wait complete. 2023-07-14T11:54:28.425 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:54:28.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.611+0000 7f21809c0700 1 Processor -- start 2023-07-14T11:54:28.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.615+0000 7f21809c0700 1 -- start start 2023-07-14T11:54:28.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.615+0000 7f21809c0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f217c0b0de0 0x7f217c1669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:28.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.615+0000 7f21809c0700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c0af0a0 con 0x7f217c0cd240 2023-07-14T11:54:28.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.615+0000 7f21809c0700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c0cf720 con 0x7f217c15bff0 2023-07-14T11:54:28.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.615+0000 7f21809c0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0af2f0 con 0x7f217c0b0de0 2023-07-14T11:54:28.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.615+0000 7f217a59c700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f217c0cd240 0x7f217c0ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53778/0 (socket says 172.21.15.104:53778) 2023-07-14T11:54:28.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.615+0000 7f217a59c700 1 -- 172.21.15.104:0/950641558 learned_addr learned my addr 172.21.15.104:0/950641558 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:28.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f217b59e700 1 --2- 172.21.15.104:0/950641558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f217c0b0de0 0x7f217c1669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:28.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896294861 0 0) 0x7f217c0af0a0 con 0x7f217c0cd240 2023-07-14T11:54:28.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21640035d0 con 0x7f217c0cd240 2023-07-14T11:54:28.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122845337 0 0) 0x7f217c0cf720 con 0x7f217c15bff0 2023-07-14T11:54:28.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f217c0af0a0 con 0x7f217c15bff0 2023-07-14T11:54:28.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3847414663 0 0) 0x7f21640035d0 con 0x7f217c0cd240 2023-07-14T11:54:28.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f217c0b0de0 msgr2=0x7f217c1669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:28.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 --2- 172.21.15.104:0/950641558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f217c0b0de0 0x7f217c1669d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 >> v1:172.21.15.121:6789/0 conn(0x7f217c15bff0 legacy=0x7f217c0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:28.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c0af5d0 con 0x7f217c0cd240 2023-07-14T11:54:28.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f2170001670 con 0x7f217c0cd240 2023-07-14T11:54:28.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 >> v1:172.21.15.104:6789/0 conn(0x7f217c0cd240 legacy=0x7f217c0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:28.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 --2- 172.21.15.104:0/950641558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2164004c40 0x7f2164005030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:28.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 --2- 172.21.15.104:0/950641558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2164003810 0x7f2164003c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:28.606 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 --2- 172.21.15.104:0/950641558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21640041c0 0x7f216400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:28.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0af2f0 con 0x7f21640041c0 2023-07-14T11:54:28.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216400bff0 con 0x7f2164003810 2023-07-14T11:54:28.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216400c1b0 con 0x7f2164004c40 2023-07-14T11:54:28.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f217b59e700 1 --2- 172.21.15.104:0/950641558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f217c0b0de0 0x7f217c1669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:28.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f217b59e700 1 --2- 172.21.15.104:0/950641558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21640041c0 0x7f216400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:28.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f217a59c700 1 --2- 172.21.15.104:0/950641558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2164003810 0x7f2164003c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:28.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f217ad9d700 1 --2- 172.21.15.104:0/950641558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2164004c40 0x7f2164005030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:28.609 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f217b59e700 1 -- 172.21.15.104:0/950641558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2164004c40 msgr2=0x7f2164005030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:28.609 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f217b59e700 1 --2- 172.21.15.104:0/950641558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2164004c40 0x7f2164005030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.609 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f217b59e700 1 -- 172.21.15.104:0/950641558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2164003810 msgr2=0x7f2164003c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:28.609 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f217b59e700 1 --2- 172.21.15.104:0/950641558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2164003810 0x7f2164003c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.609 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f217b59e700 1 -- 172.21.15.104:0/950641558 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21640035d0 con 0x7f21640041c0 2023-07-14T11:54:28.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.619+0000 7f217a59c700 1 --2- 172.21.15.104:0/950641558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2164003810 0x7f2164003c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:28.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.623+0000 7f217b59e700 1 --2- 172.21.15.104:0/950641558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21640041c0 0x7f216400ba30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f217c0b11b0 tx=0x7f21640058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6414822dec8c988d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:28.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.623+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174003c40 con 0x7f21640041c0 2023-07-14T11:54:28.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.623+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217400f040 con 0x7f21640041c0 2023-07-14T11:54:28.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.623+0000 7f2179d9b700 1 -- 172.21.15.104:0/950641558 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217400ea60 con 0x7f21640041c0 2023-07-14T11:54:28.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.623+0000 7f21809c0700 1 -- 172.21.15.104:0/950641558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21640041c0 msgr2=0x7f216400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:28.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.623+0000 7f21809c0700 1 --2- 172.21.15.104:0/950641558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21640041c0 0x7f216400ba30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f217c0b11b0 tx=0x7f21640058a0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.623+0000 7f217b59e700 1 -- 172.21.15.104:0/950641558 reap_dead start 2023-07-14T11:54:28.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.623+0000 7f21809c0700 1 -- 172.21.15.104:0/950641558 shutdown_connections 2023-07-14T11:54:28.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.623+0000 7f21809c0700 1 -- 172.21.15.104:0/950641558 >> 172.21.15.104:0/950641558 conn(0x7f217c060c40 msgr2=0x7f217c09b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:28.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.623+0000 7f21809c0700 1 -- 172.21.15.104:0/950641558 shutdown_connections 2023-07-14T11:54:28.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.623+0000 7f21809c0700 1 -- 172.21.15.104:0/950641558 wait complete. 2023-07-14T11:54:28.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f21809c0700 1 Processor -- start 2023-07-14T11:54:28.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f21809c0700 1 -- start start 2023-07-14T11:54:28.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f21809c0700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21640041c0 0x7f217c1555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:28.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f21809c0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f217c0b5310 0x7f217c0d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:28.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f21809c0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f217c0b3f30 0x7f217c0d1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:28.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f21809c0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0b5880 con 0x7f217c0b3f30 2023-07-14T11:54:28.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f21809c0700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0b5b60 con 0x7f21640041c0 2023-07-14T11:54:28.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f21809c0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0b5e40 con 0x7f217c0b5310 2023-07-14T11:54:28.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f217b59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f217c0b3f30 0x7f217c0d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:28.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f217b59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f217c0b3f30 0x7f217c0d1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57162/0 (socket says 172.21.15.104:57162) 2023-07-14T11:54:28.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f217b59e700 1 -- 172.21.15.104:0/3082371577 learned_addr learned my addr 172.21.15.104:0/3082371577 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:28.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f217b59e700 1 -- 172.21.15.104:0/3082371577 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f217c0b5310 msgr2=0x7f217c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:28.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f217a59c700 1 --2- 172.21.15.104:0/3082371577 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f217c0b5310 0x7f217c0d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:28.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f217ad9d700 1 --2- 172.21.15.104:0/3082371577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21640041c0 0x7f217c1555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:28.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f217b59e700 1 --2- 172.21.15.104:0/3082371577 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f217c0b5310 0x7f217c0d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f217b59e700 1 -- 172.21.15.104:0/3082371577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21640041c0 msgr2=0x7f217c1555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:28.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f217b59e700 1 --2- 172.21.15.104:0/3082371577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21640041c0 0x7f217c1555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f217b59e700 1 -- 172.21.15.104:0/3082371577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c0b6120 con 0x7f217c0b3f30 2023-07-14T11:54:28.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.627+0000 7f217a59c700 1 --2- 172.21.15.104:0/3082371577 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f217c0b5310 0x7f217c0d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:28.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.631+0000 7f217b59e700 1 --2- 172.21.15.104:0/3082371577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f217c0b3f30 0x7f217c0d1050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2174015000 tx=0x7f2174009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:28.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.631+0000 7f21637fe700 1 -- 172.21.15.104:0/3082371577 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174004230 con 0x7f217c0b3f30 2023-07-14T11:54:28.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.631+0000 7f21637fe700 1 -- 172.21.15.104:0/3082371577 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2174022070 con 0x7f217c0b3f30 2023-07-14T11:54:28.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.631+0000 7f21809c0700 1 -- 172.21.15.104:0/3082371577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c0b6400 con 0x7f217c0b3f30 2023-07-14T11:54:28.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.631+0000 7f21637fe700 1 -- 172.21.15.104:0/3082371577 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21740045d0 con 0x7f217c0b3f30 2023-07-14T11:54:28.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.631+0000 7f21809c0700 1 -- 172.21.15.104:0/3082371577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c0b7d50 con 0x7f217c0b3f30 2023-07-14T11:54:28.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.631+0000 7f21809c0700 1 -- 172.21.15.104:0/3082371577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c04e500 con 0x7f217c0b3f30 2023-07-14T11:54:28.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.631+0000 7f21637fe700 1 -- 172.21.15.104:0/3082371577 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2174024030 con 0x7f217c0b3f30 2023-07-14T11:54:28.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.635+0000 7f21637fe700 1 --2- 172.21.15.104:0/3082371577 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21680662f0 0x7f21680687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:28.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.635+0000 7f21637fe700 1 -- 172.21.15.104:0/3082371577 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2174085890 con 0x7f217c0b3f30 2023-07-14T11:54:28.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.635+0000 7f217ad9d700 1 --2- 172.21.15.104:0/3082371577 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21680662f0 0x7f21680687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:28.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.635+0000 7f217ad9d700 1 --2- 172.21.15.104:0/3082371577 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21680662f0 0x7f21680687a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f216c007fd0 tx=0x7f216c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:28.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.639+0000 7f21637fe700 1 -- 172.21.15.104:0/3082371577 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f2174052a70 con 0x7f217c0b3f30 2023-07-14T11:54:28.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.819+0000 7f21809c0700 1 -- 172.21.15.104:0/3082371577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f217c061010 con 0x7f217c0b3f30 2023-07-14T11:54:28.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.823+0000 7f21637fe700 1 -- 172.21.15.104:0/3082371577 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2174009d40 con 0x7f217c0b3f30 2023-07-14T11:54:28.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.827+0000 7f21809c0700 1 -- 172.21.15.104:0/3082371577 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21680662f0 msgr2=0x7f21680687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:28.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.827+0000 7f21809c0700 1 --2- 172.21.15.104:0/3082371577 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21680662f0 0x7f21680687a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f216c007fd0 tx=0x7f216c009040 comp rx=0 tx=0).stop 2023-07-14T11:54:28.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.827+0000 7f21809c0700 1 -- 172.21.15.104:0/3082371577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f217c0b3f30 msgr2=0x7f217c0d1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:28.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.827+0000 7f21809c0700 1 --2- 172.21.15.104:0/3082371577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f217c0b3f30 0x7f217c0d1050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2174015000 tx=0x7f2174009750 comp rx=0 tx=0).stop 2023-07-14T11:54:28.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.827+0000 7f21809c0700 1 -- 172.21.15.104:0/3082371577 shutdown_connections 2023-07-14T11:54:28.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.827+0000 7f21809c0700 1 --2- 172.21.15.104:0/3082371577 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21680662f0 0x7f21680687a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.827+0000 7f21809c0700 1 --2- 172.21.15.104:0/3082371577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f217c0b3f30 0x7f217c0d1050 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.827+0000 7f21809c0700 1 --2- 172.21.15.104:0/3082371577 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f217c0b5310 0x7f217c0d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.827+0000 7f21809c0700 1 --2- 172.21.15.104:0/3082371577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21640041c0 0x7f217c1555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:28.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.827+0000 7f21809c0700 1 -- 172.21.15.104:0/3082371577 >> 172.21.15.104:0/3082371577 conn(0x7f217c060c40 msgr2=0x7f217c0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:28.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.827+0000 7f21809c0700 1 -- 172.21.15.104:0/3082371577 shutdown_connections 2023-07-14T11:54:28.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:28.827+0000 7f21809c0700 1 -- 172.21.15.104:0/3082371577 wait complete. 2023-07-14T11:54:28.815 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:54:28.835 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi121.front.sepia.ceph.com... 2023-07-14T11:54:28.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:28.836 DEBUG:teuthology.orchestra.run.smithi121:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-14T11:54:28.885 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:54:28.886 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:54:28.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:28.886 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-14T11:54:28.912 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:54:28.913 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:54:28.932 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:54:28.932 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:54:28.995 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:29.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.231+0000 7f016e66b700 1 Processor -- start 2023-07-14T11:54:29.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.231+0000 7f016e66b700 1 -- start start 2023-07-14T11:54:29.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f016e66b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0168160370 0x7f01680caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:29.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f016e66b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01680cafb0 0x7f01680ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:29.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f016e66b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01680ca8a0 con 0x7f016815ffa0 2023-07-14T11:54:29.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f016e66b700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01680cb3f0 con 0x7f0168160370 2023-07-14T11:54:29.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f016e66b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01680cbcd0 con 0x7f01680cafb0 2023-07-14T11:54:29.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f016ce68700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0168160370 0x7f01680caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:29.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f016ce68700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0168160370 0x7f01680caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57172/0 (socket says 172.21.15.104:57172) 2023-07-14T11:54:29.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f016ce68700 1 -- 172.21.15.104:0/1218256166 learned_addr learned my addr 172.21.15.104:0/1218256166 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:29.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f016de6a700 1 --2- 172.21.15.104:0/1218256166 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01680cafb0 0x7f01680ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:29.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f016ce68700 1 -- 172.21.15.104:0/1218256166 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01680cafb0 msgr2=0x7f01680ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:29.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f016ce68700 1 --2- 172.21.15.104:0/1218256166 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01680cafb0 0x7f01680ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f016ce68700 1 -- 172.21.15.104:0/1218256166 >> v1:172.21.15.121:6789/0 conn(0x7f016815ffa0 legacy=0x7f016806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:29.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f016ce68700 1 -- 172.21.15.104:0/1218256166 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01680cbf70 con 0x7f0168160370 2023-07-14T11:54:29.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f016ce68700 1 --2- 172.21.15.104:0/1218256166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0168160370 0x7f01680caa70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0158009910 tx=0x7f0158009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6421d63c426a504c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:29.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f015ffff700 1 -- 172.21.15.104:0/1218256166 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0158011830 con 0x7f0168160370 2023-07-14T11:54:29.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f015ffff700 1 -- 172.21.15.104:0/1218256166 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0158011990 con 0x7f0168160370 2023-07-14T11:54:29.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.235+0000 7f015ffff700 1 -- 172.21.15.104:0/1218256166 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0158011d40 con 0x7f0168160370 2023-07-14T11:54:29.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.239+0000 7f016e66b700 1 -- 172.21.15.104:0/1218256166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0168160370 msgr2=0x7f01680caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:29.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.239+0000 7f016e66b700 1 --2- 172.21.15.104:0/1218256166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0168160370 0x7f01680caa70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0158009910 tx=0x7f0158009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:29.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.239+0000 7f016e66b700 1 -- 172.21.15.104:0/1218256166 shutdown_connections 2023-07-14T11:54:29.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.239+0000 7f016e66b700 1 --2- 172.21.15.104:0/1218256166 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01680cafb0 0x7f01680ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.239+0000 7f016e66b700 1 --2- 172.21.15.104:0/1218256166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0168160370 0x7f01680caa70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.239+0000 7f016e66b700 1 -- 172.21.15.104:0/1218256166 >> 172.21.15.104:0/1218256166 conn(0x7f016806e960 msgr2=0x7f016806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:29.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.239+0000 7f016e66b700 1 -- 172.21.15.104:0/1218256166 shutdown_connections 2023-07-14T11:54:29.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.239+0000 7f016e66b700 1 -- 172.21.15.104:0/1218256166 wait complete. 2023-07-14T11:54:29.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.239+0000 7f016e66b700 1 Processor -- start 2023-07-14T11:54:29.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.239+0000 7f016e66b700 1 -- start start 2023-07-14T11:54:29.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016e66b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01680cafb0 0x7f016814ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:29.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016e66b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f016815ffa0 0x7f016814c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:29.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016e66b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0168160370 0x7f016814a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:29.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016e66b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016815e6f0 con 0x7f016815ffa0 2023-07-14T11:54:29.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016e66b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016814b370 con 0x7f01680cafb0 2023-07-14T11:54:29.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016e66b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016814b650 con 0x7f0168160370 2023-07-14T11:54:29.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016ce68700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f016815ffa0 0x7f016814c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:29.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016ce68700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f016815ffa0 0x7f016814c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57184/0 (socket says 172.21.15.104:57184) 2023-07-14T11:54:29.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016ce68700 1 -- 172.21.15.104:0/2680148442 learned_addr learned my addr 172.21.15.104:0/2680148442 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:29.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016d669700 1 --2- 172.21.15.104:0/2680148442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01680cafb0 0x7f016814ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:29.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016de6a700 1 --2- 172.21.15.104:0/2680148442 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0168160370 0x7f016814a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:29.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016ce68700 1 -- 172.21.15.104:0/2680148442 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0168160370 msgr2=0x7f016814a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:29.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016ce68700 1 --2- 172.21.15.104:0/2680148442 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0168160370 0x7f016814a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016ce68700 1 -- 172.21.15.104:0/2680148442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01680cafb0 msgr2=0x7f016814ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:29.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016ce68700 1 --2- 172.21.15.104:0/2680148442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01680cafb0 0x7f016814ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016ce68700 1 -- 172.21.15.104:0/2680148442 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0158009570 con 0x7f016815ffa0 2023-07-14T11:54:29.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016ce68700 1 --2- 172.21.15.104:0/2680148442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f016815ffa0 0x7f016814c410 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f016806a810 tx=0x7f0158004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:29.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f015dffb700 1 -- 172.21.15.104:0/2680148442 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0158004590 con 0x7f016815ffa0 2023-07-14T11:54:29.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f015dffb700 1 -- 172.21.15.104:0/2680148442 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015801e070 con 0x7f016815ffa0 2023-07-14T11:54:29.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016e66b700 1 -- 172.21.15.104:0/2680148442 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016814b930 con 0x7f016815ffa0 2023-07-14T11:54:29.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f015dffb700 1 -- 172.21.15.104:0/2680148442 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0158003900 con 0x7f016815ffa0 2023-07-14T11:54:29.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.243+0000 7f016e66b700 1 -- 172.21.15.104:0/2680148442 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016814bef0 con 0x7f016815ffa0 2023-07-14T11:54:29.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.247+0000 7f016e66b700 1 -- 172.21.15.104:0/2680148442 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f016814c0e0 con 0x7f016815ffa0 2023-07-14T11:54:29.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.247+0000 7f015dffb700 1 -- 172.21.15.104:0/2680148442 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f0158003a60 con 0x7f016815ffa0 2023-07-14T11:54:29.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.247+0000 7f015dffb700 1 --2- 172.21.15.104:0/2680148442 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f014c0662f0 0x7f014c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:29.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.247+0000 7f015dffb700 1 -- 172.21.15.104:0/2680148442 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0158084d40 con 0x7f016815ffa0 2023-07-14T11:54:29.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.251+0000 7f016d669700 1 --2- 172.21.15.104:0/2680148442 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f014c0662f0 0x7f014c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:29.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.251+0000 7f016d669700 1 --2- 172.21.15.104:0/2680148442 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f014c0662f0 0x7f014c0687a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0164000f60 tx=0x7f0164009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:29.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.251+0000 7f015dffb700 1 -- 172.21.15.104:0/2680148442 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f0158051f20 con 0x7f016815ffa0 2023-07-14T11:54:29.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.423+0000 7f016e66b700 1 -- 172.21.15.104:0/2680148442 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f016804e500 con 0x7f016815ffa0 2023-07-14T11:54:29.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.427+0000 7f015dffb700 1 -- 172.21.15.104:0/2680148442 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20251 (secure 0 0 0) 0x7f0158020090 con 0x7f016815ffa0 2023-07-14T11:54:29.412 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:29.413 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":30,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:55.601783+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:29.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.431+0000 7f016e66b700 1 -- 172.21.15.104:0/2680148442 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f014c0662f0 msgr2=0x7f014c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:29.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.431+0000 7f016e66b700 1 --2- 172.21.15.104:0/2680148442 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f014c0662f0 0x7f014c0687a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0164000f60 tx=0x7f0164009040 comp rx=0 tx=0).stop 2023-07-14T11:54:29.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.431+0000 7f016e66b700 1 -- 172.21.15.104:0/2680148442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f016815ffa0 msgr2=0x7f016814c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:29.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.431+0000 7f016e66b700 1 --2- 172.21.15.104:0/2680148442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f016815ffa0 0x7f016814c410 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f016806a810 tx=0x7f0158004770 comp rx=0 tx=0).stop 2023-07-14T11:54:29.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.431+0000 7f016e66b700 1 -- 172.21.15.104:0/2680148442 shutdown_connections 2023-07-14T11:54:29.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.431+0000 7f016e66b700 1 --2- 172.21.15.104:0/2680148442 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f014c0662f0 0x7f014c0687a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.431+0000 7f016e66b700 1 --2- 172.21.15.104:0/2680148442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f016815ffa0 0x7f016814c410 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.431+0000 7f016e66b700 1 --2- 172.21.15.104:0/2680148442 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0168160370 0x7f016814a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.431+0000 7f016e66b700 1 --2- 172.21.15.104:0/2680148442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01680cafb0 0x7f016814ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.431+0000 7f016e66b700 1 -- 172.21.15.104:0/2680148442 >> 172.21.15.104:0/2680148442 conn(0x7f016806e960 msgr2=0x7f01680cde90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:29.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.431+0000 7f016e66b700 1 -- 172.21.15.104:0/2680148442 shutdown_connections 2023-07-14T11:54:29.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.431+0000 7f016e66b700 1 -- 172.21.15.104:0/2680148442 wait complete. 2023-07-14T11:54:29.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:54:29.438 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:54:29.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.631+0000 7f0a01984700 1 Processor -- start 2023-07-14T11:54:29.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.631+0000 7f0a01984700 1 -- start start 2023-07-14T11:54:29.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.631+0000 7f0a01984700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09fc0b0ee0 0x7f09fc0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:29.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.631+0000 7f0a01984700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f09fc0cd090 0x7f09fc0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:29.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.631+0000 7f0a01984700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f09fc0d0610 0x7f09fc15f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:29.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.631+0000 7f0a01984700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09fc0cd4e0 con 0x7f09fc0cd090 2023-07-14T11:54:29.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.631+0000 7f0a01984700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09fc0d1a60 con 0x7f09fc0d0610 2023-07-14T11:54:29.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.631+0000 7f0a01984700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc0d1ef0 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.631+0000 7f09fbfff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f09fc0cd090 0x7f09fc0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:29.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f0a00982700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09fc0b0ee0 0x7f09fc0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:29.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f0a00982700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09fc0b0ee0 0x7f09fc0ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:55824/0 (socket says 172.21.15.104:55824) 2023-07-14T11:54:29.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f0a00982700 1 -- 172.21.15.104:0/3204530144 learned_addr learned my addr 172.21.15.104:0/3204530144 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:29.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f09fbfff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f09fc0cd090 0x7f09fc0d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57196/0 (socket says 172.21.15.104:57196) 2023-07-14T11:54:29.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f0a01183700 1 --2- 172.21.15.104:0/3204530144 >> v2:172.21.15.121:3300/0 conn(0x7f09fc0d0610 0x7f09fc15f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:29.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f0a00982700 1 -- 172.21.15.104:0/3204530144 >> v2:172.21.15.121:3300/0 conn(0x7f09fc0d0610 msgr2=0x7f09fc15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:29.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f0a00982700 1 --2- 172.21.15.104:0/3204530144 >> v2:172.21.15.121:3300/0 conn(0x7f09fc0d0610 0x7f09fc15f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f0a00982700 1 -- 172.21.15.104:0/3204530144 >> v2:172.21.15.104:3300/0 conn(0x7f09fc0cd090 msgr2=0x7f09fc0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:29.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f0a00982700 1 --2- 172.21.15.104:0/3204530144 >> v2:172.21.15.104:3300/0 conn(0x7f09fc0cd090 0x7f09fc0d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f0a00982700 1 -- 172.21.15.104:0/3204530144 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc0d0e50 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f09fbfff700 1 --2- 172.21.15.104:0/3204530144 >> v2:172.21.15.104:3300/0 conn(0x7f09fc0cd090 0x7f09fc0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:29.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f0a00982700 1 --2- 172.21.15.104:0/3204530144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09fc0b0ee0 0x7f09fc0ccb50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009910 tx=0x7f09f0009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc587219b3f81e7f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:29.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f09fb7fe700 1 -- 172.21.15.104:0/3204530144 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f0011830 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f09fb7fe700 1 -- 172.21.15.104:0/3204530144 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f0011990 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f09fb7fe700 1 -- 172.21.15.104:0/3204530144 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f0011d40 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f0a01984700 1 -- 172.21.15.104:0/3204530144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09fc0b0ee0 msgr2=0x7f09fc0ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:29.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.635+0000 7f0a01984700 1 --2- 172.21.15.104:0/3204530144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09fc0b0ee0 0x7f09fc0ccb50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009910 tx=0x7f09f0009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:29.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.639+0000 7f0a01984700 1 -- 172.21.15.104:0/3204530144 shutdown_connections 2023-07-14T11:54:29.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.639+0000 7f0a01984700 1 --2- 172.21.15.104:0/3204530144 >> v2:172.21.15.121:3300/0 conn(0x7f09fc0d0610 0x7f09fc15f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.639+0000 7f0a01984700 1 --2- 172.21.15.104:0/3204530144 >> v2:172.21.15.104:3300/0 conn(0x7f09fc0cd090 0x7f09fc0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.639+0000 7f0a01984700 1 --2- 172.21.15.104:0/3204530144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09fc0b0ee0 0x7f09fc0ccb50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.639+0000 7f0a01984700 1 -- 172.21.15.104:0/3204530144 >> 172.21.15.104:0/3204530144 conn(0x7f09fc060c40 msgr2=0x7f09fc09b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:29.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.639+0000 7f0a01984700 1 -- 172.21.15.104:0/3204530144 shutdown_connections 2023-07-14T11:54:29.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.639+0000 7f0a01984700 1 -- 172.21.15.104:0/3204530144 wait complete. 2023-07-14T11:54:29.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.639+0000 7f0a01984700 1 Processor -- start 2023-07-14T11:54:29.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.639+0000 7f0a01984700 1 -- start start 2023-07-14T11:54:29.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a01984700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09fc0b0ee0 0x7f09fc0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:29.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a01984700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09fc0cd090 0x7f09fc158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:29.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a01984700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09fc0d0610 0x7f09fc0cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:29.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a01984700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc0cb0d0 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a01984700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc09d5f0 con 0x7f09fc0cd090 2023-07-14T11:54:29.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a01984700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc0ca550 con 0x7f09fc0d0610 2023-07-14T11:54:29.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a00982700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09fc0b0ee0 0x7f09fc0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:29.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a00982700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09fc0b0ee0 0x7f09fc0b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57198/0 (socket says 172.21.15.104:57198) 2023-07-14T11:54:29.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a00982700 1 -- 172.21.15.104:0/1827790062 learned_addr learned my addr 172.21.15.104:0/1827790062 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:29.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a01183700 1 --2- 172.21.15.104:0/1827790062 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09fc0d0610 0x7f09fc0cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:29.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f09fbfff700 1 --2- 172.21.15.104:0/1827790062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09fc0cd090 0x7f09fc158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:29.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a00982700 1 -- 172.21.15.104:0/1827790062 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09fc0d0610 msgr2=0x7f09fc0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:29.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a00982700 1 --2- 172.21.15.104:0/1827790062 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09fc0d0610 0x7f09fc0cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a00982700 1 -- 172.21.15.104:0/1827790062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09fc0cd090 msgr2=0x7f09fc158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:29.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a00982700 1 --2- 172.21.15.104:0/1827790062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09fc0cd090 0x7f09fc158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a00982700 1 -- 172.21.15.104:0/1827790062 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e80094b0 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a00982700 1 --2- 172.21.15.104:0/1827790062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09fc0b0ee0 0x7f09fc0b4140 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f09f0000c00 tx=0x7f09f0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:29.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f09f97fa700 1 -- 172.21.15.104:0/1827790062 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f0004450 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f09f97fa700 1 -- 172.21.15.104:0/1827790062 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f001e070 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f09f97fa700 1 -- 172.21.15.104:0/1827790062 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f0003900 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a01984700 1 -- 172.21.15.104:0/1827790062 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f0009570 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.643+0000 7f0a01984700 1 -- 172.21.15.104:0/1827790062 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc0cab10 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.647+0000 7f0a01984700 1 -- 172.21.15.104:0/1827790062 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc04e500 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.647+0000 7f09f97fa700 1 -- 172.21.15.104:0/1827790062 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f09f0006c80 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.651+0000 7f09f97fa700 1 --2- 172.21.15.104:0/1827790062 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f09e00662f0 0x7f09e00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:29.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.651+0000 7f09f97fa700 1 -- 172.21.15.104:0/1827790062 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f09f0084de0 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.651+0000 7f09fbfff700 1 --2- 172.21.15.104:0/1827790062 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f09e00662f0 0x7f09e00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:29.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.651+0000 7f09fbfff700 1 --2- 172.21.15.104:0/1827790062 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f09e00662f0 0x7f09e00687a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f09fc160910 tx=0x7f09e8009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:29.641 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.655+0000 7f09f97fa700 1 -- 172.21.15.104:0/1827790062 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f09f0052080 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.827+0000 7f0a01984700 1 -- 172.21.15.104:0/1827790062 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09fc0b18d0 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.827+0000 7f09f97fa700 1 -- 172.21.15.104:0/1827790062 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09f00556a0 con 0x7f09fc0b0ee0 2023-07-14T11:54:29.814 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:54:29.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.831+0000 7f0a01984700 1 -- 172.21.15.104:0/1827790062 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f09e00662f0 msgr2=0x7f09e00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:29.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.831+0000 7f0a01984700 1 --2- 172.21.15.104:0/1827790062 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f09e00662f0 0x7f09e00687a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f09fc160910 tx=0x7f09e8009240 comp rx=0 tx=0).stop 2023-07-14T11:54:29.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.831+0000 7f0a01984700 1 -- 172.21.15.104:0/1827790062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09fc0b0ee0 msgr2=0x7f09fc0b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:29.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.831+0000 7f0a01984700 1 --2- 172.21.15.104:0/1827790062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09fc0b0ee0 0x7f09fc0b4140 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f09f0000c00 tx=0x7f09f0004770 comp rx=0 tx=0).stop 2023-07-14T11:54:29.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.831+0000 7f0a01984700 1 -- 172.21.15.104:0/1827790062 shutdown_connections 2023-07-14T11:54:29.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.831+0000 7f0a01984700 1 --2- 172.21.15.104:0/1827790062 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f09e00662f0 0x7f09e00687a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.831+0000 7f0a01984700 1 --2- 172.21.15.104:0/1827790062 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09fc0d0610 0x7f09fc0cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.831+0000 7f0a01984700 1 --2- 172.21.15.104:0/1827790062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09fc0cd090 0x7f09fc158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.831+0000 7f0a01984700 1 --2- 172.21.15.104:0/1827790062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09fc0b0ee0 0x7f09fc0b4140 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:29.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.831+0000 7f0a01984700 1 -- 172.21.15.104:0/1827790062 >> 172.21.15.104:0/1827790062 conn(0x7f09fc060c40 msgr2=0x7f09fc0b7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:29.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.831+0000 7f0a01984700 1 -- 172.21.15.104:0/1827790062 shutdown_connections 2023-07-14T11:54:29.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:29.831+0000 7f0a01984700 1 -- 172.21.15.104:0/1827790062 wait complete. 2023-07-14T11:54:29.838 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:54:30.020 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.035+0000 7f1f6a071700 1 Processor -- start 2023-07-14T11:54:30.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.035+0000 7f1f6a071700 1 -- start start 2023-07-14T11:54:30.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f6a071700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1f640b1c50 0x7f1f64071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f6a071700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1f640cf710 0x7f1f640d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f6a071700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6415c420 con 0x7f1f640d0210 2023-07-14T11:54:30.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f6a071700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f64070840 con 0x7f1f640b1c50 2023-07-14T11:54:30.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f6a071700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f64070b20 con 0x7f1f640cf710 2023-07-14T11:54:30.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f637fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1f640b1c50 0x7f1f64071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f637fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1f640b1c50 0x7f1f64071600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57200/0 (socket says 172.21.15.104:57200) 2023-07-14T11:54:30.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f637fe700 1 -- 172.21.15.104:0/3621017202 learned_addr learned my addr 172.21.15.104:0/3621017202 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:30.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f63fff700 1 --2- 172.21.15.104:0/3621017202 >> v2:172.21.15.121:3300/0 conn(0x7f1f640cf710 0x7f1f640d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f637fe700 1 -- 172.21.15.104:0/3621017202 >> v2:172.21.15.121:3300/0 conn(0x7f1f640cf710 msgr2=0x7f1f640d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:30.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f637fe700 1 --2- 172.21.15.104:0/3621017202 >> v2:172.21.15.121:3300/0 conn(0x7f1f640cf710 0x7f1f640d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f637fe700 1 -- 172.21.15.104:0/3621017202 >> v1:172.21.15.104:6789/0 conn(0x7f1f640d0210 legacy=0x7f1f640cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:30.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f637fe700 1 -- 172.21.15.104:0/3621017202 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f64070e00 con 0x7f1f640b1c50 2023-07-14T11:54:30.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f637fe700 1 --2- 172.21.15.104:0/3621017202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f640b1c50 0x7f1f64071600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1f6408cb80 tx=0x7f1f54009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104d360838f808af server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:30.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f627fc700 1 -- 172.21.15.104:0/3621017202 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5400b8a0 con 0x7f1f640b1c50 2023-07-14T11:54:30.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f627fc700 1 -- 172.21.15.104:0/3621017202 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5400ba00 con 0x7f1f640b1c50 2023-07-14T11:54:30.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f627fc700 1 -- 172.21.15.104:0/3621017202 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5400bd90 con 0x7f1f640b1c50 2023-07-14T11:54:30.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f6a071700 1 -- 172.21.15.104:0/3621017202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f640b1c50 msgr2=0x7f1f64071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:30.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f6a071700 1 --2- 172.21.15.104:0/3621017202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f640b1c50 0x7f1f64071600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1f6408cb80 tx=0x7f1f54009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f6a071700 1 -- 172.21.15.104:0/3621017202 shutdown_connections 2023-07-14T11:54:30.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f6a071700 1 --2- 172.21.15.104:0/3621017202 >> v2:172.21.15.121:3300/0 conn(0x7f1f640cf710 0x7f1f640d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f6a071700 1 --2- 172.21.15.104:0/3621017202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f640b1c50 0x7f1f64071600 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f6a071700 1 -- 172.21.15.104:0/3621017202 >> 172.21.15.104:0/3621017202 conn(0x7f1f6409b9c0 msgr2=0x7f1f640d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:30.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f6a071700 1 -- 172.21.15.104:0/3621017202 shutdown_connections 2023-07-14T11:54:30.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.039+0000 7f1f6a071700 1 -- 172.21.15.104:0/3621017202 wait complete. 2023-07-14T11:54:30.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.043+0000 7f1f6a071700 1 Processor -- start 2023-07-14T11:54:30.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.043+0000 7f1f6a071700 1 -- start start 2023-07-14T11:54:30.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.043+0000 7f1f6a071700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f640b1c50 0x7f1f6409c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.043+0000 7f1f6a071700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f640cf710 0x7f1f641e5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.043+0000 7f1f6a071700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f640d0210 0x7f1f6414a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.043+0000 7f1f6a071700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f641e64a0 con 0x7f1f640cf710 2023-07-14T11:54:30.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.043+0000 7f1f6a071700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f641e4920 con 0x7f1f640d0210 2023-07-14T11:54:30.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.043+0000 7f1f6a071700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f641e4c00 con 0x7f1f640b1c50 2023-07-14T11:54:30.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.043+0000 7f1f62ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f640cf710 0x7f1f641e5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.043+0000 7f1f62ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f640cf710 0x7f1f641e5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57214/0 (socket says 172.21.15.104:57214) 2023-07-14T11:54:30.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.043+0000 7f1f62ffd700 1 -- 172.21.15.104:0/3320278459 learned_addr learned my addr 172.21.15.104:0/3320278459 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:30.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.043+0000 7f1f63fff700 1 --2- 172.21.15.104:0/3320278459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f640d0210 0x7f1f6414a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.043+0000 7f1f637fe700 1 --2- 172.21.15.104:0/3320278459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f640b1c50 0x7f1f6409c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.047+0000 7f1f62ffd700 1 -- 172.21.15.104:0/3320278459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f640b1c50 msgr2=0x7f1f6409c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:30.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.047+0000 7f1f62ffd700 1 --2- 172.21.15.104:0/3320278459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f640b1c50 0x7f1f6409c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.047+0000 7f1f62ffd700 1 -- 172.21.15.104:0/3320278459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f640d0210 msgr2=0x7f1f6414a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:30.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.047+0000 7f1f62ffd700 1 --2- 172.21.15.104:0/3320278459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f640d0210 0x7f1f6414a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.047+0000 7f1f62ffd700 1 -- 172.21.15.104:0/3320278459 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f54009570 con 0x7f1f640cf710 2023-07-14T11:54:30.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.047+0000 7f1f62ffd700 1 --2- 172.21.15.104:0/3320278459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f640cf710 0x7f1f641e5f60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800c410 tx=0x7f1f5800c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:30.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.047+0000 7f1f4bfff700 1 -- 172.21.15.104:0/3320278459 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f580052d0 con 0x7f1f640cf710 2023-07-14T11:54:30.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.047+0000 7f1f4bfff700 1 -- 172.21.15.104:0/3320278459 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f58005430 con 0x7f1f640cf710 2023-07-14T11:54:30.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.047+0000 7f1f4bfff700 1 -- 172.21.15.104:0/3320278459 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f580055a0 con 0x7f1f640cf710 2023-07-14T11:54:30.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.047+0000 7f1f6a071700 1 -- 172.21.15.104:0/3320278459 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f641e4ee0 con 0x7f1f640cf710 2023-07-14T11:54:30.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.047+0000 7f1f6a071700 1 -- 172.21.15.104:0/3320278459 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6414aea0 con 0x7f1f640cf710 2023-07-14T11:54:30.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.047+0000 7f1f6a071700 1 -- 172.21.15.104:0/3320278459 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f641e5400 con 0x7f1f640cf710 2023-07-14T11:54:30.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.051+0000 7f1f4bfff700 1 -- 172.21.15.104:0/3320278459 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1f580036a0 con 0x7f1f640cf710 2023-07-14T11:54:30.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.051+0000 7f1f4bfff700 1 --2- 172.21.15.104:0/3320278459 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1f500662f0 0x7f1f500687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.051+0000 7f1f4bfff700 1 -- 172.21.15.104:0/3320278459 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1f580831f0 con 0x7f1f640cf710 2023-07-14T11:54:30.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.055+0000 7f1f637fe700 1 --2- 172.21.15.104:0/3320278459 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1f500662f0 0x7f1f500687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.055+0000 7f1f637fe700 1 --2- 172.21.15.104:0/3320278459 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1f500662f0 0x7f1f500687a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1f54000c00 tx=0x7f1f54008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:30.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.059+0000 7f1f4bfff700 1 -- 172.21.15.104:0/3320278459 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f1f641e5400 con 0x7f1f640cf710 2023-07-14T11:54:30.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.231+0000 7f1f6a071700 1 -- 172.21.15.104:0/3320278459 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1f641e5400 con 0x7f1f640cf710 2023-07-14T11:54:30.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.231+0000 7f1f4bfff700 1 -- 172.21.15.104:0/3320278459 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1f641e5400 con 0x7f1f640cf710 2023-07-14T11:54:30.216 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:54:30.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.235+0000 7f1f6a071700 1 -- 172.21.15.104:0/3320278459 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1f500662f0 msgr2=0x7f1f500687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:30.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.235+0000 7f1f6a071700 1 --2- 172.21.15.104:0/3320278459 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1f500662f0 0x7f1f500687a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1f54000c00 tx=0x7f1f54008030 comp rx=0 tx=0).stop 2023-07-14T11:54:30.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.235+0000 7f1f6a071700 1 -- 172.21.15.104:0/3320278459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f640cf710 msgr2=0x7f1f641e5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:30.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.235+0000 7f1f6a071700 1 --2- 172.21.15.104:0/3320278459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f640cf710 0x7f1f641e5f60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800c410 tx=0x7f1f5800c720 comp rx=0 tx=0).stop 2023-07-14T11:54:30.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.235+0000 7f1f6a071700 1 -- 172.21.15.104:0/3320278459 shutdown_connections 2023-07-14T11:54:30.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.235+0000 7f1f6a071700 1 --2- 172.21.15.104:0/3320278459 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1f500662f0 0x7f1f500687a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.235+0000 7f1f6a071700 1 --2- 172.21.15.104:0/3320278459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f640cf710 0x7f1f641e5f60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.235+0000 7f1f6a071700 1 --2- 172.21.15.104:0/3320278459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f640d0210 0x7f1f6414a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.235+0000 7f1f6a071700 1 --2- 172.21.15.104:0/3320278459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1f640b1c50 0x7f1f6409c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.235+0000 7f1f6a071700 1 -- 172.21.15.104:0/3320278459 >> 172.21.15.104:0/3320278459 conn(0x7f1f6409b9c0 msgr2=0x7f1f640b7680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:30.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.235+0000 7f1f6a071700 1 -- 172.21.15.104:0/3320278459 shutdown_connections 2023-07-14T11:54:30.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.235+0000 7f1f6a071700 1 -- 172.21.15.104:0/3320278459 wait complete. 2023-07-14T11:54:30.240 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:54:30.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.427+0000 7fa778078700 1 Processor -- start 2023-07-14T11:54:30.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.427+0000 7fa778078700 1 -- start start 2023-07-14T11:54:30.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.431+0000 7fa778078700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa77015df10 0x7fa77015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.431+0000 7fa778078700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa7700c8950 0x7fa7700c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.431+0000 7fa778078700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa770070a30 con 0x7fa7700afd80 2023-07-14T11:54:30.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.431+0000 7fa778078700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7700c92f0 con 0x7fa7700c8950 2023-07-14T11:54:30.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.431+0000 7fa778078700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77015e830 con 0x7fa77015df10 2023-07-14T11:54:30.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.431+0000 7fa776615700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa7700c8950 0x7fa7700c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.431+0000 7fa776615700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa7700c8950 0x7fa7700c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43744/0 (socket says 172.21.15.104:43744) 2023-07-14T11:54:30.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.431+0000 7fa776615700 1 -- 172.21.15.104:0/564537055 learned_addr learned my addr 172.21.15.104:0/564537055 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:30.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.431+0000 7fa775e14700 1 --2- 172.21.15.104:0/564537055 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa77015df10 0x7fa77015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.431+0000 7fa776615700 1 -- 172.21.15.104:0/564537055 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa77015df10 msgr2=0x7fa77015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:30.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.431+0000 7fa776615700 1 --2- 172.21.15.104:0/564537055 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa77015df10 0x7fa77015e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.431+0000 7fa776615700 1 -- 172.21.15.104:0/564537055 >> v1:172.21.15.121:6789/0 conn(0x7fa7700afd80 legacy=0x7fa7700711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:30.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.431+0000 7fa776615700 1 -- 172.21.15.104:0/564537055 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7700b0210 con 0x7fa7700c8950 2023-07-14T11:54:30.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.431+0000 7fa776615700 1 --2- 172.21.15.104:0/564537055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7700c8950 0x7fa7700c8db0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa768009e50 tx=0x7fa76800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4086ed84b28cbe96 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:30.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.435+0000 7fa774e12700 1 -- 172.21.15.104:0/564537055 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76800e070 con 0x7fa7700c8950 2023-07-14T11:54:30.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.435+0000 7fa774e12700 1 -- 172.21.15.104:0/564537055 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76800b9c0 con 0x7fa7700c8950 2023-07-14T11:54:30.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.435+0000 7fa774e12700 1 -- 172.21.15.104:0/564537055 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76800bd70 con 0x7fa7700c8950 2023-07-14T11:54:30.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.435+0000 7fa778078700 1 -- 172.21.15.104:0/564537055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7700c8950 msgr2=0x7fa7700c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:30.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.435+0000 7fa778078700 1 --2- 172.21.15.104:0/564537055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7700c8950 0x7fa7700c8db0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa768009e50 tx=0x7fa76800b330 comp rx=0 tx=0).stop 2023-07-14T11:54:30.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.435+0000 7fa778078700 1 -- 172.21.15.104:0/564537055 shutdown_connections 2023-07-14T11:54:30.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.435+0000 7fa778078700 1 --2- 172.21.15.104:0/564537055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7700c8950 0x7fa7700c8db0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.435+0000 7fa778078700 1 --2- 172.21.15.104:0/564537055 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa77015df10 0x7fa77015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.435+0000 7fa778078700 1 -- 172.21.15.104:0/564537055 >> 172.21.15.104:0/564537055 conn(0x7fa7700cad80 msgr2=0x7fa7700cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:30.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.435+0000 7fa778078700 1 -- 172.21.15.104:0/564537055 shutdown_connections 2023-07-14T11:54:30.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.435+0000 7fa778078700 1 -- 172.21.15.104:0/564537055 wait complete. 2023-07-14T11:54:30.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.439+0000 7fa778078700 1 Processor -- start 2023-07-14T11:54:30.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.439+0000 7fa778078700 1 -- start start 2023-07-14T11:54:30.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.439+0000 7fa778078700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7700afd80 0x7fa7700c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.439+0000 7fa778078700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7700c8950 0x7fa7701561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.439+0000 7fa778078700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7700c7c10 0x7fa7700c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.439+0000 7fa778078700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7701566e0 con 0x7fa7700afd80 2023-07-14T11:54:30.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.439+0000 7fa778078700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700c53d0 con 0x7fa7700c8950 2023-07-14T11:54:30.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.439+0000 7fa778078700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700c5690 con 0x7fa7700c7c10 2023-07-14T11:54:30.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.439+0000 7fa775e14700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7700afd80 0x7fa7700c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.439+0000 7fa775e14700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7700afd80 0x7fa7700c6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43756/0 (socket says 172.21.15.104:43756) 2023-07-14T11:54:30.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.439+0000 7fa775e14700 1 -- 172.21.15.104:0/3600008513 learned_addr learned my addr 172.21.15.104:0/3600008513 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:30.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.443+0000 7fa775613700 1 --2- 172.21.15.104:0/3600008513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7700c8950 0x7fa7701561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.443+0000 7fa776615700 1 --2- 172.21.15.104:0/3600008513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7700c7c10 0x7fa7700c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.443+0000 7fa775e14700 1 -- 172.21.15.104:0/3600008513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7700c7c10 msgr2=0x7fa7700c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:30.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.443+0000 7fa775e14700 1 --2- 172.21.15.104:0/3600008513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7700c7c10 0x7fa7700c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.443+0000 7fa775e14700 1 -- 172.21.15.104:0/3600008513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7700c8950 msgr2=0x7fa7701561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:30.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.443+0000 7fa775e14700 1 --2- 172.21.15.104:0/3600008513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7700c8950 0x7fa7701561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.443+0000 7fa775e14700 1 -- 172.21.15.104:0/3600008513 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa768009b30 con 0x7fa7700afd80 2023-07-14T11:54:30.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.443+0000 7fa775e14700 1 --2- 172.21.15.104:0/3600008513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7700afd80 0x7fa7700c6450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00d800 tx=0x7fa76c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:30.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.443+0000 7fa7667fc700 1 -- 172.21.15.104:0/3600008513 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c0125f0 con 0x7fa7700afd80 2023-07-14T11:54:30.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.443+0000 7fa7667fc700 1 -- 172.21.15.104:0/3600008513 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c012750 con 0x7fa7700afd80 2023-07-14T11:54:30.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.443+0000 7fa778078700 1 -- 172.21.15.104:0/3600008513 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7700c5970 con 0x7fa7700afd80 2023-07-14T11:54:30.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.443+0000 7fa7667fc700 1 -- 172.21.15.104:0/3600008513 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c012ac0 con 0x7fa7700afd80 2023-07-14T11:54:30.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.443+0000 7fa778078700 1 -- 172.21.15.104:0/3600008513 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7700c5f30 con 0x7fa7700afd80 2023-07-14T11:54:30.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.443+0000 7fa778078700 1 -- 172.21.15.104:0/3600008513 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77004dcb0 con 0x7fa7700afd80 2023-07-14T11:54:30.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.447+0000 7fa7667fc700 1 -- 172.21.15.104:0/3600008513 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa76c012c20 con 0x7fa7700afd80 2023-07-14T11:54:30.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.447+0000 7fa7667fc700 1 --2- 172.21.15.104:0/3600008513 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7540662f0 0x7fa7540687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.447+0000 7fa7667fc700 1 -- 172.21.15.104:0/3600008513 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa76c084120 con 0x7fa7700afd80 2023-07-14T11:54:30.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.447+0000 7fa775613700 1 --2- 172.21.15.104:0/3600008513 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7540662f0 0x7fa7540687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.447+0000 7fa775613700 1 --2- 172.21.15.104:0/3600008513 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7540662f0 0x7fa7540687a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa7700c8e00 tx=0x7fa760009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:30.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.455+0000 7fa7667fc700 1 -- 172.21.15.104:0/3600008513 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fa76c0511d0 con 0x7fa7700afd80 2023-07-14T11:54:30.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.631+0000 7fa778078700 1 -- 172.21.15.104:0/3600008513 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa77006bb50 con 0x7fa7700afd80 2023-07-14T11:54:30.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.631+0000 7fa7667fc700 1 -- 172.21.15.104:0/3600008513 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa76c0547f0 con 0x7fa7700afd80 2023-07-14T11:54:30.618 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:30.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.635+0000 7fa778078700 1 -- 172.21.15.104:0/3600008513 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7540662f0 msgr2=0x7fa7540687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:30.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.635+0000 7fa778078700 1 --2- 172.21.15.104:0/3600008513 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7540662f0 0x7fa7540687a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa7700c8e00 tx=0x7fa760009040 comp rx=0 tx=0).stop 2023-07-14T11:54:30.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.635+0000 7fa778078700 1 -- 172.21.15.104:0/3600008513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7700afd80 msgr2=0x7fa7700c6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:30.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.635+0000 7fa778078700 1 --2- 172.21.15.104:0/3600008513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7700afd80 0x7fa7700c6450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00d800 tx=0x7fa76c00db10 comp rx=0 tx=0).stop 2023-07-14T11:54:30.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.635+0000 7fa778078700 1 -- 172.21.15.104:0/3600008513 shutdown_connections 2023-07-14T11:54:30.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.635+0000 7fa778078700 1 --2- 172.21.15.104:0/3600008513 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7540662f0 0x7fa7540687a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.635+0000 7fa778078700 1 --2- 172.21.15.104:0/3600008513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7700c7c10 0x7fa7700c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.635+0000 7fa778078700 1 --2- 172.21.15.104:0/3600008513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7700c8950 0x7fa7701561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.635+0000 7fa778078700 1 --2- 172.21.15.104:0/3600008513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7700afd80 0x7fa7700c6450 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.635+0000 7fa778078700 1 -- 172.21.15.104:0/3600008513 >> 172.21.15.104:0/3600008513 conn(0x7fa7700cad80 msgr2=0x7fa7700cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:30.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.635+0000 7fa778078700 1 -- 172.21.15.104:0/3600008513 shutdown_connections 2023-07-14T11:54:30.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.635+0000 7fa778078700 1 -- 172.21.15.104:0/3600008513 wait complete. 2023-07-14T11:54:30.643 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:54:30.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.839+0000 7f4688ee7700 1 Processor -- start 2023-07-14T11:54:30.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.839+0000 7f4688ee7700 1 -- start start 2023-07-14T11:54:30.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4688ee7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f468406fa70 0x7f468406fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4688ee7700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f46840d0050 0x7f46840cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4688ee7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f46840cfb20 0x7f46840cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4688ee7700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46840cf930 con 0x7f46840d0050 2023-07-14T11:54:30.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4688ee7700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46840cea60 con 0x7f46840cfb20 2023-07-14T11:54:30.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4688ee7700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4684070380 con 0x7f468406fa70 2023-07-14T11:54:30.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4681d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f46840d0050 0x7f46840cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4681d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f46840d0050 0x7f46840cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43758/0 (socket says 172.21.15.104:43758) 2023-07-14T11:54:30.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4681d9b700 1 -- 172.21.15.104:0/2410414148 learned_addr learned my addr 172.21.15.104:0/2410414148 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:30.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4682d9d700 1 --2- 172.21.15.104:0/2410414148 >> v2:172.21.15.121:3300/0 conn(0x7f46840cfb20 0x7f46840cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f468259c700 1 --2- 172.21.15.104:0/2410414148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f468406fa70 0x7f468406fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4682d9d700 1 -- 172.21.15.104:0/2410414148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f468406fa70 msgr2=0x7f468406fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:30.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4682d9d700 1 --2- 172.21.15.104:0/2410414148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f468406fa70 0x7f468406fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4682d9d700 1 -- 172.21.15.104:0/2410414148 >> v2:172.21.15.104:3300/0 conn(0x7f46840d0050 msgr2=0x7f46840cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:30.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4682d9d700 1 --2- 172.21.15.104:0/2410414148 >> v2:172.21.15.104:3300/0 conn(0x7f46840d0050 0x7f46840cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4682d9d700 1 -- 172.21.15.104:0/2410414148 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46840cecb0 con 0x7f46840cfb20 2023-07-14T11:54:30.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4681d9b700 1 --2- 172.21.15.104:0/2410414148 >> v2:172.21.15.104:3300/0 conn(0x7f46840d0050 0x7f46840cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:30.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f4682d9d700 1 --2- 172.21.15.104:0/2410414148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46840cfb20 0x7f46840cd3a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f468415fa90 tx=0x7f467c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c69eefd43eae2b81 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:30.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f468159a700 1 -- 172.21.15.104:0/2410414148 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c00e070 con 0x7f46840cfb20 2023-07-14T11:54:30.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f468159a700 1 -- 172.21.15.104:0/2410414148 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c00bb20 con 0x7f46840cfb20 2023-07-14T11:54:30.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.843+0000 7f468159a700 1 -- 172.21.15.104:0/2410414148 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c012430 con 0x7f46840cfb20 2023-07-14T11:54:30.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.847+0000 7f4688ee7700 1 -- 172.21.15.104:0/2410414148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46840cfb20 msgr2=0x7f46840cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:30.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.847+0000 7f4688ee7700 1 --2- 172.21.15.104:0/2410414148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46840cfb20 0x7f46840cd3a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f468415fa90 tx=0x7f467c00b330 comp rx=0 tx=0).stop 2023-07-14T11:54:30.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.847+0000 7f4688ee7700 1 -- 172.21.15.104:0/2410414148 shutdown_connections 2023-07-14T11:54:30.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.847+0000 7f4688ee7700 1 --2- 172.21.15.104:0/2410414148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46840cfb20 0x7f46840cd3a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.847+0000 7f4688ee7700 1 --2- 172.21.15.104:0/2410414148 >> v2:172.21.15.104:3300/0 conn(0x7f46840d0050 0x7f46840cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.847+0000 7f4688ee7700 1 --2- 172.21.15.104:0/2410414148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f468406fa70 0x7f468406fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.847+0000 7f4688ee7700 1 -- 172.21.15.104:0/2410414148 >> 172.21.15.104:0/2410414148 conn(0x7f468408a000 msgr2=0x7f46840d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.847+0000 7f4688ee7700 1 -- 172.21.15.104:0/2410414148 shutdown_connections 2023-07-14T11:54:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.847+0000 7f4688ee7700 1 -- 172.21.15.104:0/2410414148 wait complete. 2023-07-14T11:54:30.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.847+0000 7f4688ee7700 1 Processor -- start 2023-07-14T11:54:30.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.847+0000 7f4688ee7700 1 -- start start 2023-07-14T11:54:30.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f4688ee7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f468406fa70 0x7f46840b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f4688ee7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46840cfb20 0x7f46840c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f4688ee7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46840d0050 0x7f46840c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f4688ee7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46840b3be0 con 0x7f468406fa70 2023-07-14T11:54:30.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f4688ee7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46840c95a0 con 0x7f46840cfb20 2023-07-14T11:54:30.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f4688ee7700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46840c9840 con 0x7f46840d0050 2023-07-14T11:54:30.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f468259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f468406fa70 0x7f46840b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f468259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f468406fa70 0x7f46840b5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43770/0 (socket says 172.21.15.104:43770) 2023-07-14T11:54:30.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f468259c700 1 -- 172.21.15.104:0/884324889 learned_addr learned my addr 172.21.15.104:0/884324889 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:30.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f4681d9b700 1 --2- 172.21.15.104:0/884324889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46840cfb20 0x7f46840c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f4682d9d700 1 --2- 172.21.15.104:0/884324889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46840d0050 0x7f46840c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f468259c700 1 -- 172.21.15.104:0/884324889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46840d0050 msgr2=0x7f46840c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:30.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f468259c700 1 --2- 172.21.15.104:0/884324889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46840d0050 0x7f46840c64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f468259c700 1 -- 172.21.15.104:0/884324889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46840cfb20 msgr2=0x7f46840c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:30.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f468259c700 1 --2- 172.21.15.104:0/884324889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46840cfb20 0x7f46840c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:30.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f468259c700 1 -- 172.21.15.104:0/884324889 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4678009570 con 0x7f468406fa70 2023-07-14T11:54:30.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f468259c700 1 --2- 172.21.15.104:0/884324889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f468406fa70 0x7f46840b5d30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f467400e890 tx=0x7f467400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:30.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f4672ffd700 1 -- 172.21.15.104:0/884324889 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4674005a40 con 0x7f468406fa70 2023-07-14T11:54:30.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f4672ffd700 1 -- 172.21.15.104:0/884324889 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4674005ba0 con 0x7f468406fa70 2023-07-14T11:54:30.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f4672ffd700 1 -- 172.21.15.104:0/884324889 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4674013680 con 0x7f468406fa70 2023-07-14T11:54:30.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f4688ee7700 1 -- 172.21.15.104:0/884324889 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c009a70 con 0x7f468406fa70 2023-07-14T11:54:30.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.851+0000 7f4688ee7700 1 -- 172.21.15.104:0/884324889 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46840b3590 con 0x7f468406fa70 2023-07-14T11:54:30.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.855+0000 7f4688ee7700 1 -- 172.21.15.104:0/884324889 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f468404dcb0 con 0x7f468406fa70 2023-07-14T11:54:30.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.855+0000 7f4672ffd700 1 -- 172.21.15.104:0/884324889 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f46740137e0 con 0x7f468406fa70 2023-07-14T11:54:30.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.855+0000 7f4672ffd700 1 --2- 172.21.15.104:0/884324889 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46640662f0 0x7f46640687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:30.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.855+0000 7f4672ffd700 1 -- 172.21.15.104:0/884324889 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f46740840b0 con 0x7f468406fa70 2023-07-14T11:54:30.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.859+0000 7f4681d9b700 1 --2- 172.21.15.104:0/884324889 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46640662f0 0x7f46640687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:30.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.859+0000 7f4681d9b700 1 --2- 172.21.15.104:0/884324889 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46640662f0 0x7f46640687a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f46840ca5e0 tx=0x7f46780092e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:30.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:30.863+0000 7f4672ffd700 1 -- 172.21.15.104:0/884324889 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f4674051200 con 0x7f468406fa70 2023-07-14T11:54:31.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.035+0000 7f4688ee7700 1 -- 172.21.15.104:0/884324889 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4684071100 con 0x7f468406fa70 2023-07-14T11:54:31.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.035+0000 7f4672ffd700 1 -- 172.21.15.104:0/884324889 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f46740200a0 con 0x7f468406fa70 2023-07-14T11:54:31.021 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:54:31.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.039+0000 7f4688ee7700 1 -- 172.21.15.104:0/884324889 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46640662f0 msgr2=0x7f46640687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:31.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.039+0000 7f4688ee7700 1 --2- 172.21.15.104:0/884324889 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46640662f0 0x7f46640687a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f46840ca5e0 tx=0x7f46780092e0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.039+0000 7f4688ee7700 1 -- 172.21.15.104:0/884324889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f468406fa70 msgr2=0x7f46840b5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:31.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.039+0000 7f4688ee7700 1 --2- 172.21.15.104:0/884324889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f468406fa70 0x7f46840b5d30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f467400e890 tx=0x7f467400eba0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.039+0000 7f4688ee7700 1 -- 172.21.15.104:0/884324889 shutdown_connections 2023-07-14T11:54:31.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.039+0000 7f4688ee7700 1 --2- 172.21.15.104:0/884324889 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46640662f0 0x7f46640687a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.039+0000 7f4688ee7700 1 --2- 172.21.15.104:0/884324889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46840d0050 0x7f46840c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.039+0000 7f4688ee7700 1 --2- 172.21.15.104:0/884324889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46840cfb20 0x7f46840c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.039+0000 7f4688ee7700 1 --2- 172.21.15.104:0/884324889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f468406fa70 0x7f46840b5d30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.039+0000 7f4688ee7700 1 -- 172.21.15.104:0/884324889 >> 172.21.15.104:0/884324889 conn(0x7f468408a000 msgr2=0x7f468406ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:31.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.039+0000 7f4688ee7700 1 -- 172.21.15.104:0/884324889 shutdown_connections 2023-07-14T11:54:31.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.039+0000 7f4688ee7700 1 -- 172.21.15.104:0/884324889 wait complete. 2023-07-14T11:54:31.047 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:54:31.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.243+0000 7fb420af2700 1 Processor -- start 2023-07-14T11:54:31.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.243+0000 7fb420af2700 1 -- start start 2023-07-14T11:54:31.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb420af2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb41c0cd260 0x7fb41c0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:31.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb420af2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb41c0d0610 0x7fb41c0af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:31.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb420af2700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41c0d1a60 con 0x7fb41c0b0ee0 2023-07-14T11:54:31.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb420af2700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41c0d1ef0 con 0x7fb41c0d0610 2023-07-14T11:54:31.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb420af2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0d0e50 con 0x7fb41c0cd260 2023-07-14T11:54:31.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb413fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb41c0cd260 0x7fb41c0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:31.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb41bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb41c0d0610 0x7fb41c0af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:31.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb413fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb41c0cd260 0x7fb41c0d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:38670/0 (socket says 172.21.15.104:38670) 2023-07-14T11:54:31.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb413fff700 1 -- 172.21.15.104:0/1037549449 learned_addr learned my addr 172.21.15.104:0/1037549449 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:31.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb413fff700 1 -- 172.21.15.104:0/1037549449 >> v2:172.21.15.121:3300/0 conn(0x7fb41c0d0610 msgr2=0x7fb41c0af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:31.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb413fff700 1 --2- 172.21.15.104:0/1037549449 >> v2:172.21.15.121:3300/0 conn(0x7fb41c0d0610 0x7fb41c0af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb413fff700 1 -- 172.21.15.104:0/1037549449 >> v1:172.21.15.121:6789/0 conn(0x7fb41c0b0ee0 legacy=0x7fb41c0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:31.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb413fff700 1 -- 172.21.15.104:0/1037549449 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c0d1130 con 0x7fb41c0cd260 2023-07-14T11:54:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb41bfff700 1 --2- 172.21.15.104:0/1037549449 >> v2:172.21.15.121:3300/0 conn(0x7fb41c0d0610 0x7fb41c0af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb413fff700 1 --2- 172.21.15.104:0/1037549449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb41c0cd260 0x7fb41c0d00d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb404009910 tx=0x7fb404009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4221f939805bc18e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.247+0000 7fb41affd700 1 -- 172.21.15.104:0/1037549449 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404011830 con 0x7fb41c0cd260 2023-07-14T11:54:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.251+0000 7fb41affd700 1 -- 172.21.15.104:0/1037549449 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb404011990 con 0x7fb41c0cd260 2023-07-14T11:54:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.251+0000 7fb41affd700 1 -- 172.21.15.104:0/1037549449 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404011d40 con 0x7fb41c0cd260 2023-07-14T11:54:31.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.251+0000 7fb420af2700 1 -- 172.21.15.104:0/1037549449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb41c0cd260 msgr2=0x7fb41c0d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:31.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.251+0000 7fb420af2700 1 --2- 172.21.15.104:0/1037549449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb41c0cd260 0x7fb41c0d00d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb404009910 tx=0x7fb404009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:31.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.251+0000 7fb420af2700 1 -- 172.21.15.104:0/1037549449 shutdown_connections 2023-07-14T11:54:31.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.251+0000 7fb420af2700 1 --2- 172.21.15.104:0/1037549449 >> v2:172.21.15.121:3300/0 conn(0x7fb41c0d0610 0x7fb41c0af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.251+0000 7fb420af2700 1 --2- 172.21.15.104:0/1037549449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb41c0cd260 0x7fb41c0d00d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.251+0000 7fb420af2700 1 -- 172.21.15.104:0/1037549449 >> 172.21.15.104:0/1037549449 conn(0x7fb41c060c40 msgr2=0x7fb41c09b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:31.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.251+0000 7fb420af2700 1 -- 172.21.15.104:0/1037549449 shutdown_connections 2023-07-14T11:54:31.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.251+0000 7fb420af2700 1 -- 172.21.15.104:0/1037549449 wait complete. 2023-07-14T11:54:31.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.251+0000 7fb420af2700 1 Processor -- start 2023-07-14T11:54:31.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.251+0000 7fb420af2700 1 -- start start 2023-07-14T11:54:31.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb420af2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb41c0b0ee0 0x7fb41c158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:31.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb420af2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb41c0cd260 0x7fb41c0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:31.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb420af2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb41c0d0610 0x7fb41c0af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:31.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb420af2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c158f40 con 0x7fb41c0b0ee0 2023-07-14T11:54:31.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb420af2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0b3de0 con 0x7fb41c0cd260 2023-07-14T11:54:31.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb420af2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0b40c0 con 0x7fb41c0d0610 2023-07-14T11:54:31.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb41b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb41c0b0ee0 0x7fb41c158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:31.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb41b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb41c0b0ee0 0x7fb41c158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43784/0 (socket says 172.21.15.104:43784) 2023-07-14T11:54:31.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb41b7fe700 1 -- 172.21.15.104:0/2515330985 learned_addr learned my addr 172.21.15.104:0/2515330985 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:31.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb413fff700 1 --2- 172.21.15.104:0/2515330985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb41c0cd260 0x7fb41c0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:31.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb41b7fe700 1 -- 172.21.15.104:0/2515330985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb41c0d0610 msgr2=0x7fb41c0af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:31.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb41b7fe700 1 --2- 172.21.15.104:0/2515330985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb41c0d0610 0x7fb41c0af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb41b7fe700 1 -- 172.21.15.104:0/2515330985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb41c0cd260 msgr2=0x7fb41c0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:31.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb41b7fe700 1 --2- 172.21.15.104:0/2515330985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb41c0cd260 0x7fb41c0b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb41b7fe700 1 -- 172.21.15.104:0/2515330985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb414009a70 con 0x7fb41c0b0ee0 2023-07-14T11:54:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb413fff700 1 --2- 172.21.15.104:0/2515330985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb41c0cd260 0x7fb41c0b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb41b7fe700 1 --2- 172.21.15.104:0/2515330985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb41c0b0ee0 0x7fb41c158a00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00c500 tx=0x7fb40c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb418ff9700 1 -- 172.21.15.104:0/2515330985 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c0052d0 con 0x7fb41c0b0ee0 2023-07-14T11:54:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb418ff9700 1 -- 172.21.15.104:0/2515330985 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40c005430 con 0x7fb41c0b0ee0 2023-07-14T11:54:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb418ff9700 1 -- 172.21.15.104:0/2515330985 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c0055a0 con 0x7fb41c0b0ee0 2023-07-14T11:54:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb420af2700 1 -- 172.21.15.104:0/2515330985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb404009570 con 0x7fb41c0b0ee0 2023-07-14T11:54:31.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.255+0000 7fb420af2700 1 -- 172.21.15.104:0/2515330985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41c0b4680 con 0x7fb41c0b0ee0 2023-07-14T11:54:31.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.259+0000 7fb420af2700 1 -- 172.21.15.104:0/2515330985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41c04e500 con 0x7fb41c0b0ee0 2023-07-14T11:54:31.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.259+0000 7fb418ff9700 1 -- 172.21.15.104:0/2515330985 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb40c0038e0 con 0x7fb41c0b0ee0 2023-07-14T11:54:31.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.259+0000 7fb418ff9700 1 --2- 172.21.15.104:0/2515330985 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb3fc0662f0 0x7fb3fc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:31.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.263+0000 7fb418ff9700 1 -- 172.21.15.104:0/2515330985 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb40c083500 con 0x7fb41c0b0ee0 2023-07-14T11:54:31.247 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.263+0000 7fb413fff700 1 --2- 172.21.15.104:0/2515330985 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb3fc0662f0 0x7fb3fc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:31.247 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.263+0000 7fb413fff700 1 --2- 172.21.15.104:0/2515330985 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb3fc0662f0 0x7fb3fc0687a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0cd650 tx=0x7fb404008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:31.250 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.267+0000 7fb418ff9700 1 -- 172.21.15.104:0/2515330985 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb40c050520 con 0x7fb41c0b0ee0 2023-07-14T11:54:31.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.443+0000 7fb420af2700 1 -- 172.21.15.104:0/2515330985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb41c0b18d0 con 0x7fb41c0b0ee0 2023-07-14T11:54:31.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.443+0000 7fb418ff9700 1 -- 172.21.15.104:0/2515330985 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb40c053b40 con 0x7fb41c0b0ee0 2023-07-14T11:54:31.429 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:31.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.447+0000 7fb420af2700 1 -- 172.21.15.104:0/2515330985 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb3fc0662f0 msgr2=0x7fb3fc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:31.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.447+0000 7fb420af2700 1 --2- 172.21.15.104:0/2515330985 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb3fc0662f0 0x7fb3fc0687a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0cd650 tx=0x7fb404008030 comp rx=0 tx=0).stop 2023-07-14T11:54:31.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.447+0000 7fb420af2700 1 -- 172.21.15.104:0/2515330985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb41c0b0ee0 msgr2=0x7fb41c158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:31.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.447+0000 7fb420af2700 1 --2- 172.21.15.104:0/2515330985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb41c0b0ee0 0x7fb41c158a00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00c500 tx=0x7fb40c00c8c0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.447+0000 7fb420af2700 1 -- 172.21.15.104:0/2515330985 shutdown_connections 2023-07-14T11:54:31.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.447+0000 7fb420af2700 1 --2- 172.21.15.104:0/2515330985 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb3fc0662f0 0x7fb3fc0687a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.447+0000 7fb420af2700 1 --2- 172.21.15.104:0/2515330985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb41c0d0610 0x7fb41c0af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.447+0000 7fb420af2700 1 --2- 172.21.15.104:0/2515330985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb41c0cd260 0x7fb41c0b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.447+0000 7fb420af2700 1 --2- 172.21.15.104:0/2515330985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb41c0b0ee0 0x7fb41c158a00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.447+0000 7fb420af2700 1 -- 172.21.15.104:0/2515330985 >> 172.21.15.104:0/2515330985 conn(0x7fb41c060c40 msgr2=0x7fb41c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:31.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.451+0000 7fb420af2700 1 -- 172.21.15.104:0/2515330985 shutdown_connections 2023-07-14T11:54:31.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.451+0000 7fb420af2700 1 -- 172.21.15.104:0/2515330985 wait complete. 2023-07-14T11:54:31.458 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:54:31.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.639+0000 7f3219dca700 1 Processor -- start 2023-07-14T11:54:31.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.639+0000 7f3219dca700 1 -- start start 2023-07-14T11:54:31.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.639+0000 7f3219dca700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f32140c86b0 0x7f32140c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:31.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3219dca700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3214158370 0x7f321415bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:31.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3219dca700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32140cbf80 con 0x7f32140cb100 2023-07-14T11:54:31.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3219dca700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32140cbc30 con 0x7f32140c86b0 2023-07-14T11:54:31.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3219dca700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32140c8fc0 con 0x7f3214158370 2023-07-14T11:54:31.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3218dc8700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f32140c86b0 0x7f32140c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:31.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3218dc8700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f32140c86b0 0x7f32140c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43798/0 (socket says 172.21.15.104:43798) 2023-07-14T11:54:31.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3218dc8700 1 -- 172.21.15.104:0/3234031515 learned_addr learned my addr 172.21.15.104:0/3234031515 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:31.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3213fff700 1 --2- 172.21.15.104:0/3234031515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3214158370 0x7f321415bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:31.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3218dc8700 1 -- 172.21.15.104:0/3234031515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3214158370 msgr2=0x7f321415bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:31.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3218dc8700 1 --2- 172.21.15.104:0/3234031515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3214158370 0x7f321415bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3218dc8700 1 -- 172.21.15.104:0/3234031515 >> v1:172.21.15.121:6789/0 conn(0x7f32140cb100 legacy=0x7f321406bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:31.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3218dc8700 1 -- 172.21.15.104:0/3234031515 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321415c4d0 con 0x7f32140c86b0 2023-07-14T11:54:31.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3218dc8700 1 --2- 172.21.15.104:0/3234031515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32140c86b0 0x7f32140c8a80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f320800bbe0 tx=0x7f320800bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc0a5a3e2758ee0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:31.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f32137fe700 1 -- 172.21.15.104:0/3234031515 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320800dcf0 con 0x7f32140c86b0 2023-07-14T11:54:31.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f32137fe700 1 -- 172.21.15.104:0/3234031515 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320800de50 con 0x7f32140c86b0 2023-07-14T11:54:31.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f32137fe700 1 -- 172.21.15.104:0/3234031515 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208010690 con 0x7f32140c86b0 2023-07-14T11:54:31.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3219dca700 1 -- 172.21.15.104:0/3234031515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32140c86b0 msgr2=0x7f32140c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:31.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3219dca700 1 --2- 172.21.15.104:0/3234031515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32140c86b0 0x7f32140c8a80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f320800bbe0 tx=0x7f320800bef0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3219dca700 1 -- 172.21.15.104:0/3234031515 shutdown_connections 2023-07-14T11:54:31.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3219dca700 1 --2- 172.21.15.104:0/3234031515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3214158370 0x7f321415bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3219dca700 1 --2- 172.21.15.104:0/3234031515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32140c86b0 0x7f32140c8a80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3219dca700 1 -- 172.21.15.104:0/3234031515 >> 172.21.15.104:0/3234031515 conn(0x7f321406e960 msgr2=0x7f321406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:31.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3219dca700 1 -- 172.21.15.104:0/3234031515 shutdown_connections 2023-07-14T11:54:31.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.643+0000 7f3219dca700 1 -- 172.21.15.104:0/3234031515 wait complete. 2023-07-14T11:54:31.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.647+0000 7f3219dca700 1 Processor -- start 2023-07-14T11:54:31.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.647+0000 7f3219dca700 1 -- start start 2023-07-14T11:54:31.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3219dca700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32140c86b0 0x7f321414b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:31.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3219dca700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32140cb100 0x7f32140b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:31.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3219dca700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3214158370 0x7f32140b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:31.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3219dca700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32140b16b0 con 0x7f32140cb100 2023-07-14T11:54:31.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3219dca700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321414bd30 con 0x7f3214158370 2023-07-14T11:54:31.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3219dca700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3214159c90 con 0x7f32140c86b0 2023-07-14T11:54:31.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3213fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32140cb100 0x7f32140b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:31.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3218dc8700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32140c86b0 0x7f321414b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:31.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3213fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32140cb100 0x7f32140b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43810/0 (socket says 172.21.15.104:43810) 2023-07-14T11:54:31.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3218dc8700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32140c86b0 0x7f321414b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:38698/0 (socket says 172.21.15.104:38698) 2023-07-14T11:54:31.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3213fff700 1 -- 172.21.15.104:0/3465288004 learned_addr learned my addr 172.21.15.104:0/3465288004 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:31.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f32195c9700 1 --2- 172.21.15.104:0/3465288004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3214158370 0x7f32140b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:31.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3218dc8700 1 -- 172.21.15.104:0/3465288004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3214158370 msgr2=0x7f32140b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:31.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3218dc8700 1 --2- 172.21.15.104:0/3465288004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3214158370 0x7f32140b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3218dc8700 1 -- 172.21.15.104:0/3465288004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32140cb100 msgr2=0x7f32140b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:31.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3218dc8700 1 --2- 172.21.15.104:0/3465288004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32140cb100 0x7f32140b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3218dc8700 1 -- 172.21.15.104:0/3465288004 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320800b810 con 0x7f32140c86b0 2023-07-14T11:54:31.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3218dc8700 1 --2- 172.21.15.104:0/3465288004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32140c86b0 0x7f321414b7f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3208018000 tx=0x7f3208004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:31.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f32117fa700 1 -- 172.21.15.104:0/3465288004 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208003c90 con 0x7f32140c86b0 2023-07-14T11:54:31.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f32117fa700 1 -- 172.21.15.104:0/3465288004 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3208003df0 con 0x7f32140c86b0 2023-07-14T11:54:31.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f32117fa700 1 -- 172.21.15.104:0/3465288004 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208010690 con 0x7f32140c86b0 2023-07-14T11:54:31.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.651+0000 7f3219dca700 1 -- 172.21.15.104:0/3465288004 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32140b13d0 con 0x7f32140c86b0 2023-07-14T11:54:31.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.655+0000 7f3219dca700 1 -- 172.21.15.104:0/3465288004 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3214159920 con 0x7f32140c86b0 2023-07-14T11:54:31.641 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.655+0000 7f3219dca700 1 -- 172.21.15.104:0/3465288004 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32141594a0 con 0x7f32140c86b0 2023-07-14T11:54:31.642 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.655+0000 7f32117fa700 1 -- 172.21.15.104:0/3465288004 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f32080107f0 con 0x7f32140c86b0 2023-07-14T11:54:31.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.659+0000 7f32117fa700 1 --2- 172.21.15.104:0/3465288004 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31f8066320 0x7f31f80687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:31.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.659+0000 7f32117fa700 1 -- 172.21.15.104:0/3465288004 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3208085c70 con 0x7f32140c86b0 2023-07-14T11:54:31.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.659+0000 7f3213fff700 1 --2- 172.21.15.104:0/3465288004 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31f8066320 0x7f31f80687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:31.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.659+0000 7f3213fff700 1 --2- 172.21.15.104:0/3465288004 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31f8066320 0x7f31f80687d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3214159440 tx=0x7f3200009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:31.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.667+0000 7f32117fa700 1 -- 172.21.15.104:0/3465288004 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f3208052f30 con 0x7f32140c86b0 2023-07-14T11:54:31.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.855+0000 7f3219dca700 1 -- 172.21.15.104:0/3465288004 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f32141594a0 con 0x7f32140c86b0 2023-07-14T11:54:31.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.855+0000 7f32117fa700 1 -- 172.21.15.104:0/3465288004 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f32141594a0 con 0x7f32140c86b0 2023-07-14T11:54:31.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.859+0000 7f3219dca700 1 -- 172.21.15.104:0/3465288004 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31f8066320 msgr2=0x7f31f80687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:31.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.859+0000 7f3219dca700 1 --2- 172.21.15.104:0/3465288004 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31f8066320 0x7f31f80687d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3214159440 tx=0x7f3200009240 comp rx=0 tx=0).stop 2023-07-14T11:54:31.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.859+0000 7f3219dca700 1 -- 172.21.15.104:0/3465288004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32140c86b0 msgr2=0x7f321414b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:31.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.859+0000 7f3219dca700 1 --2- 172.21.15.104:0/3465288004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32140c86b0 0x7f321414b7f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3208018000 tx=0x7f3208004770 comp rx=0 tx=0).stop 2023-07-14T11:54:31.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.859+0000 7f3219dca700 1 -- 172.21.15.104:0/3465288004 shutdown_connections 2023-07-14T11:54:31.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.859+0000 7f3219dca700 1 --2- 172.21.15.104:0/3465288004 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31f8066320 0x7f31f80687d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.859+0000 7f3219dca700 1 --2- 172.21.15.104:0/3465288004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32140cb100 0x7f32140b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.859+0000 7f3219dca700 1 --2- 172.21.15.104:0/3465288004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3214158370 0x7f32140b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.859+0000 7f3219dca700 1 --2- 172.21.15.104:0/3465288004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32140c86b0 0x7f321414b7f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:31.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.859+0000 7f3219dca700 1 -- 172.21.15.104:0/3465288004 >> 172.21.15.104:0/3465288004 conn(0x7f321406e960 msgr2=0x7f32140cded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:31.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.859+0000 7f3219dca700 1 -- 172.21.15.104:0/3465288004 shutdown_connections 2023-07-14T11:54:31.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:31.859+0000 7f3219dca700 1 -- 172.21.15.104:0/3465288004 wait complete. 2023-07-14T11:54:31.846 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:54:31.864 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:54:31.865 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:54:31.880 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:32.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.111+0000 7f5193fff700 1 Processor -- start 2023-07-14T11:54:32.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.111+0000 7f5193fff700 1 -- start start 2023-07-14T11:54:32.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f5193fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f519415bff0 0x7f51940ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f5193fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f51940cd070 0x7f51940ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f5193fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51940ceff0 0x7f51940d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f5193fff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51940cd4f0 con 0x7f51940cd070 2023-07-14T11:54:32.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f5193fff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51940af0a0 con 0x7f519415bff0 2023-07-14T11:54:32.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f5193fff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51940cf470 con 0x7f51940ceff0 2023-07-14T11:54:32.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f51927fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f51940cd070 0x7f51940ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f51927fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f51940cd070 0x7f51940ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43816/0 (socket says 172.21.15.104:43816) 2023-07-14T11:54:32.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f51927fc700 1 -- 172.21.15.104:0/1207856585 learned_addr learned my addr 172.21.15.104:0/1207856585 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:32.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f51937fe700 1 --2- 172.21.15.104:0/1207856585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51940ceff0 0x7f51940d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f5192ffd700 1 --2- 172.21.15.104:0/1207856585 >> v2:172.21.15.121:3300/0 conn(0x7f519415bff0 0x7f51940ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f51927fc700 1 -- 172.21.15.104:0/1207856585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51940ceff0 msgr2=0x7f51940d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:32.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f51927fc700 1 --2- 172.21.15.104:0/1207856585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51940ceff0 0x7f51940d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f51927fc700 1 -- 172.21.15.104:0/1207856585 >> v2:172.21.15.121:3300/0 conn(0x7f519415bff0 msgr2=0x7f51940ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:32.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f51927fc700 1 --2- 172.21.15.104:0/1207856585 >> v2:172.21.15.121:3300/0 conn(0x7f519415bff0 0x7f51940ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f51927fc700 1 -- 172.21.15.104:0/1207856585 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51940cf720 con 0x7f51940cd070 2023-07-14T11:54:32.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f51927fc700 1 --2- 172.21.15.104:0/1207856585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51940cd070 0x7f51940ceab0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f519408ac70 tx=0x7f5188009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fb0c9914049668 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:32.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f5191ffb700 1 -- 172.21.15.104:0/1207856585 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518800b8a0 con 0x7f51940cd070 2023-07-14T11:54:32.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f5191ffb700 1 -- 172.21.15.104:0/1207856585 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518800ba00 con 0x7f51940cd070 2023-07-14T11:54:32.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f5191ffb700 1 -- 172.21.15.104:0/1207856585 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518800bd90 con 0x7f51940cd070 2023-07-14T11:54:32.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f5193fff700 1 -- 172.21.15.104:0/1207856585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51940cd070 msgr2=0x7f51940ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:32.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.115+0000 7f5193fff700 1 --2- 172.21.15.104:0/1207856585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51940cd070 0x7f51940ceab0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f519408ac70 tx=0x7f5188009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.119+0000 7f5193fff700 1 -- 172.21.15.104:0/1207856585 shutdown_connections 2023-07-14T11:54:32.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.119+0000 7f5193fff700 1 --2- 172.21.15.104:0/1207856585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51940ceff0 0x7f51940d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.119+0000 7f5193fff700 1 --2- 172.21.15.104:0/1207856585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51940cd070 0x7f51940ceab0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.119+0000 7f5193fff700 1 --2- 172.21.15.104:0/1207856585 >> v2:172.21.15.121:3300/0 conn(0x7f519415bff0 0x7f51940ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.119+0000 7f5193fff700 1 -- 172.21.15.104:0/1207856585 >> 172.21.15.104:0/1207856585 conn(0x7f5194060c40 msgr2=0x7f519409b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:32.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.119+0000 7f5193fff700 1 -- 172.21.15.104:0/1207856585 shutdown_connections 2023-07-14T11:54:32.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.119+0000 7f5193fff700 1 -- 172.21.15.104:0/1207856585 wait complete. 2023-07-14T11:54:32.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.119+0000 7f5193fff700 1 Processor -- start 2023-07-14T11:54:32.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.119+0000 7f5193fff700 1 -- start start 2023-07-14T11:54:32.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.119+0000 7f5193fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51940cd070 0x7f51940b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.119+0000 7f5193fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51940ceff0 0x7f51940b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.119+0000 7f5193fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f519415bff0 0x7f51940cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.119+0000 7f5193fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51940c9640 con 0x7f51940cd070 2023-07-14T11:54:32.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.119+0000 7f5193fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519414ed90 con 0x7f51940ceff0 2023-07-14T11:54:32.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.119+0000 7f5193fff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51940b69f0 con 0x7f519415bff0 2023-07-14T11:54:32.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f5192ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51940cd070 0x7f51940b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f5192ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51940cd070 0x7f51940b7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43826/0 (socket says 172.21.15.104:43826) 2023-07-14T11:54:32.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f5192ffd700 1 -- 172.21.15.104:0/1685947069 learned_addr learned my addr 172.21.15.104:0/1685947069 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:32.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f51937fe700 1 --2- 172.21.15.104:0/1685947069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f519415bff0 0x7f51940cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f51927fc700 1 --2- 172.21.15.104:0/1685947069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51940ceff0 0x7f51940b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f5192ffd700 1 -- 172.21.15.104:0/1685947069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f519415bff0 msgr2=0x7f51940cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:32.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f5192ffd700 1 --2- 172.21.15.104:0/1685947069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f519415bff0 0x7f51940cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f5192ffd700 1 -- 172.21.15.104:0/1685947069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51940ceff0 msgr2=0x7f51940b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:32.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f5192ffd700 1 --2- 172.21.15.104:0/1685947069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51940ceff0 0x7f51940b6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f5192ffd700 1 -- 172.21.15.104:0/1685947069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5188009570 con 0x7f51940cd070 2023-07-14T11:54:32.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f51927fc700 1 --2- 172.21.15.104:0/1685947069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51940ceff0 0x7f51940b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:32.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f5192ffd700 1 --2- 172.21.15.104:0/1685947069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51940cd070 0x7f51940b7f00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f518400d830 tx=0x7f518400dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:32.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f517b7fe700 1 -- 172.21.15.104:0/1685947069 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51840125f0 con 0x7f51940cd070 2023-07-14T11:54:32.110 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f517b7fe700 1 -- 172.21.15.104:0/1685947069 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5184012750 con 0x7f51940cd070 2023-07-14T11:54:32.110 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f5193fff700 1 -- 172.21.15.104:0/1685947069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51940b6cd0 con 0x7f51940cd070 2023-07-14T11:54:32.110 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f517b7fe700 1 -- 172.21.15.104:0/1685947069 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184012ac0 con 0x7f51940cd070 2023-07-14T11:54:32.110 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f5193fff700 1 -- 172.21.15.104:0/1685947069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51940b7290 con 0x7f51940cd070 2023-07-14T11:54:32.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.123+0000 7f5193fff700 1 -- 172.21.15.104:0/1685947069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519404e500 con 0x7f51940cd070 2023-07-14T11:54:32.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.127+0000 7f517b7fe700 1 -- 172.21.15.104:0/1685947069 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5184012c20 con 0x7f51940cd070 2023-07-14T11:54:32.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.127+0000 7f517b7fe700 1 --2- 172.21.15.104:0/1685947069 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f51800662f0 0x7f51800687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.127+0000 7f517b7fe700 1 -- 172.21.15.104:0/1685947069 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5184084490 con 0x7f51940cd070 2023-07-14T11:54:32.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.127+0000 7f51927fc700 1 --2- 172.21.15.104:0/1685947069 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f51800662f0 0x7f51800687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.127+0000 7f51927fc700 1 --2- 172.21.15.104:0/1685947069 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f51800662f0 0x7f51800687a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f51940a02a0 tx=0x7f5188008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:32.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.135+0000 7f517b7fe700 1 -- 172.21.15.104:0/1685947069 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f5184054ff0 con 0x7f51940cd070 2023-07-14T11:54:32.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.307+0000 7f5193fff700 1 -- 172.21.15.104:0/1685947069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f519415a330 con 0x7f51940cd070 2023-07-14T11:54:32.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.311+0000 7f517b7fe700 1 -- 172.21.15.104:0/1685947069 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20251 (secure 0 0 0) 0x7f5184012ed0 con 0x7f51940cd070 2023-07-14T11:54:32.296 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:32.297 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":30,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:55.601783+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:32.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.311+0000 7f5193fff700 1 -- 172.21.15.104:0/1685947069 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f51800662f0 msgr2=0x7f51800687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:32.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.311+0000 7f5193fff700 1 --2- 172.21.15.104:0/1685947069 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f51800662f0 0x7f51800687a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f51940a02a0 tx=0x7f5188008030 comp rx=0 tx=0).stop 2023-07-14T11:54:32.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.311+0000 7f5193fff700 1 -- 172.21.15.104:0/1685947069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51940cd070 msgr2=0x7f51940b7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:32.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.311+0000 7f5193fff700 1 --2- 172.21.15.104:0/1685947069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51940cd070 0x7f51940b7f00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f518400d830 tx=0x7f518400dbf0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.311+0000 7f5193fff700 1 -- 172.21.15.104:0/1685947069 shutdown_connections 2023-07-14T11:54:32.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.311+0000 7f5193fff700 1 --2- 172.21.15.104:0/1685947069 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f51800662f0 0x7f51800687a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.311+0000 7f5193fff700 1 --2- 172.21.15.104:0/1685947069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f519415bff0 0x7f51940cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.311+0000 7f5193fff700 1 --2- 172.21.15.104:0/1685947069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51940ceff0 0x7f51940b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.311+0000 7f5193fff700 1 --2- 172.21.15.104:0/1685947069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51940cd070 0x7f51940b7f00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.311+0000 7f5193fff700 1 -- 172.21.15.104:0/1685947069 >> 172.21.15.104:0/1685947069 conn(0x7f5194060c40 msgr2=0x7f51940b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:32.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.315+0000 7f5193fff700 1 -- 172.21.15.104:0/1685947069 shutdown_connections 2023-07-14T11:54:32.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.315+0000 7f5193fff700 1 -- 172.21.15.104:0/1685947069 wait complete. 2023-07-14T11:54:32.323 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:54:32.324 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:54:32.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.511+0000 7fde76a4f700 1 Processor -- start 2023-07-14T11:54:32.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.511+0000 7fde76a4f700 1 -- start start 2023-07-14T11:54:32.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.515+0000 7fde76a4f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fde70160000 0x7fde7006dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.515+0000 7fde76a4f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde701603d0 0x7fde700c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.515+0000 7fde76a4f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde700cdfb0 con 0x7fde700cd7f0 2023-07-14T11:54:32.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.515+0000 7fde76a4f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde70070e40 con 0x7fde70160000 2023-07-14T11:54:32.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.515+0000 7fde76a4f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde700cdcd0 con 0x7fde701603d0 2023-07-14T11:54:32.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.515+0000 7fde7524c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde701603d0 0x7fde700c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.515+0000 7fde75a4d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fde70160000 0x7fde7006dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.515+0000 7fde75a4d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fde70160000 0x7fde7006dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:36982/0 (socket says 172.21.15.104:36982) 2023-07-14T11:54:32.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.515+0000 7fde75a4d700 1 -- 172.21.15.104:0/1084262913 learned_addr learned my addr 172.21.15.104:0/1084262913 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:32.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde75a4d700 1 -- 172.21.15.104:0/1084262913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde701603d0 msgr2=0x7fde700c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:32.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde75a4d700 1 --2- 172.21.15.104:0/1084262913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde701603d0 0x7fde700c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde75a4d700 1 -- 172.21.15.104:0/1084262913 >> v1:172.21.15.121:6789/0 conn(0x7fde700cd7f0 legacy=0x7fde700c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:32.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde74a4b700 1 -- 172.21.15.104:0/1084262913 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 902859551 0 0) 0x7fde68001490 con 0x7fde700cd7f0 2023-07-14T11:54:32.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde75a4d700 1 -- 172.21.15.104:0/1084262913 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde70070780 con 0x7fde70160000 2023-07-14T11:54:32.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde7524c700 1 --2- 172.21.15.104:0/1084262913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde701603d0 0x7fde700c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:32.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde75a4d700 1 --2- 172.21.15.104:0/1084262913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde70160000 0x7fde7006dd10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009910 tx=0x7fde6c009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb70c77e64b33a1e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:32.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde74a4b700 1 -- 172.21.15.104:0/1084262913 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c011830 con 0x7fde70160000 2023-07-14T11:54:32.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde74a4b700 1 -- 172.21.15.104:0/1084262913 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6c011990 con 0x7fde70160000 2023-07-14T11:54:32.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde74a4b700 1 -- 172.21.15.104:0/1084262913 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c011d40 con 0x7fde70160000 2023-07-14T11:54:32.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde76a4f700 1 -- 172.21.15.104:0/1084262913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde70160000 msgr2=0x7fde7006dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:32.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde76a4f700 1 --2- 172.21.15.104:0/1084262913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde70160000 0x7fde7006dd10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009910 tx=0x7fde6c009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:32.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde76a4f700 1 -- 172.21.15.104:0/1084262913 shutdown_connections 2023-07-14T11:54:32.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde76a4f700 1 --2- 172.21.15.104:0/1084262913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde701603d0 0x7fde700c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde76a4f700 1 --2- 172.21.15.104:0/1084262913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde70160000 0x7fde7006dd10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde76a4f700 1 -- 172.21.15.104:0/1084262913 >> 172.21.15.104:0/1084262913 conn(0x7fde700afc60 msgr2=0x7fde700b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:32.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde76a4f700 1 -- 172.21.15.104:0/1084262913 shutdown_connections 2023-07-14T11:54:32.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.519+0000 7fde76a4f700 1 -- 172.21.15.104:0/1084262913 wait complete. 2023-07-14T11:54:32.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.523+0000 7fde76a4f700 1 Processor -- start 2023-07-14T11:54:32.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.523+0000 7fde76a4f700 1 -- start start 2023-07-14T11:54:32.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.523+0000 7fde76a4f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde700cd7f0 0x7fde70163f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.523+0000 7fde76a4f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde70160000 0x7fde700aca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.523+0000 7fde76a4f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde701603d0 0x7fde700acf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.523+0000 7fde76a4f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde701f9930 con 0x7fde701603d0 2023-07-14T11:54:32.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.523+0000 7fde76a4f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde701f8cb0 con 0x7fde70160000 2023-07-14T11:54:32.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.523+0000 7fde76a4f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde701f8f90 con 0x7fde700cd7f0 2023-07-14T11:54:32.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.523+0000 7fde7624e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde701603d0 0x7fde700acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.523+0000 7fde7624e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde701603d0 0x7fde700acf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43840/0 (socket says 172.21.15.104:43840) 2023-07-14T11:54:32.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.523+0000 7fde7624e700 1 -- 172.21.15.104:0/2599055730 learned_addr learned my addr 172.21.15.104:0/2599055730 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:32.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.523+0000 7fde7524c700 1 --2- 172.21.15.104:0/2599055730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde70160000 0x7fde700aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.523+0000 7fde75a4d700 1 --2- 172.21.15.104:0/2599055730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde700cd7f0 0x7fde70163f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.527+0000 7fde7624e700 1 -- 172.21.15.104:0/2599055730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde700cd7f0 msgr2=0x7fde70163f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:32.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.527+0000 7fde7624e700 1 --2- 172.21.15.104:0/2599055730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde700cd7f0 0x7fde70163f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.527+0000 7fde7624e700 1 -- 172.21.15.104:0/2599055730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde70160000 msgr2=0x7fde700aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:32.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.527+0000 7fde7624e700 1 --2- 172.21.15.104:0/2599055730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde70160000 0x7fde700aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.527+0000 7fde7624e700 1 -- 172.21.15.104:0/2599055730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde600094b0 con 0x7fde701603d0 2023-07-14T11:54:32.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.527+0000 7fde7524c700 1 --2- 172.21.15.104:0/2599055730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde70160000 0x7fde700aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:32.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.527+0000 7fde7624e700 1 --2- 172.21.15.104:0/2599055730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde701603d0 0x7fde700acf50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fde6800cca0 tx=0x7fde6800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:32.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.527+0000 7fde667fc700 1 -- 172.21.15.104:0/2599055730 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68003e50 con 0x7fde701603d0 2023-07-14T11:54:32.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.527+0000 7fde76a4f700 1 -- 172.21.15.104:0/2599055730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6c009570 con 0x7fde701603d0 2023-07-14T11:54:32.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.527+0000 7fde667fc700 1 -- 172.21.15.104:0/2599055730 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6800a700 con 0x7fde701603d0 2023-07-14T11:54:32.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.527+0000 7fde667fc700 1 -- 172.21.15.104:0/2599055730 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6800aab0 con 0x7fde701603d0 2023-07-14T11:54:32.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.527+0000 7fde76a4f700 1 -- 172.21.15.104:0/2599055730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde701f9550 con 0x7fde701603d0 2023-07-14T11:54:32.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.527+0000 7fde76a4f700 1 -- 172.21.15.104:0/2599055730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde70071240 con 0x7fde701603d0 2023-07-14T11:54:32.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.531+0000 7fde667fc700 1 -- 172.21.15.104:0/2599055730 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fde6800ac10 con 0x7fde701603d0 2023-07-14T11:54:32.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.531+0000 7fde667fc700 1 --2- 172.21.15.104:0/2599055730 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fde540662f0 0x7fde540687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.531+0000 7fde667fc700 1 -- 172.21.15.104:0/2599055730 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fde680839e0 con 0x7fde701603d0 2023-07-14T11:54:32.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.531+0000 7fde75a4d700 1 --2- 172.21.15.104:0/2599055730 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fde540662f0 0x7fde540687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.531+0000 7fde75a4d700 1 --2- 172.21.15.104:0/2599055730 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fde540662f0 0x7fde540687a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fde6c000c00 tx=0x7fde6c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:32.523 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.539+0000 7fde667fc700 1 -- 172.21.15.104:0/2599055730 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fde70071240 con 0x7fde701603d0 2023-07-14T11:54:32.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.711+0000 7fde76a4f700 1 -- 172.21.15.104:0/2599055730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde701f9a70 con 0x7fde701603d0 2023-07-14T11:54:32.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.711+0000 7fde667fc700 1 -- 172.21.15.104:0/2599055730 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde701f9a70 con 0x7fde701603d0 2023-07-14T11:54:32.696 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:54:32.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.711+0000 7fde76a4f700 1 -- 172.21.15.104:0/2599055730 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fde540662f0 msgr2=0x7fde540687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:32.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.711+0000 7fde76a4f700 1 --2- 172.21.15.104:0/2599055730 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fde540662f0 0x7fde540687a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fde6c000c00 tx=0x7fde6c008030 comp rx=0 tx=0).stop 2023-07-14T11:54:32.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.711+0000 7fde76a4f700 1 -- 172.21.15.104:0/2599055730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde701603d0 msgr2=0x7fde700acf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:32.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.711+0000 7fde76a4f700 1 --2- 172.21.15.104:0/2599055730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde701603d0 0x7fde700acf50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fde6800cca0 tx=0x7fde6800a060 comp rx=0 tx=0).stop 2023-07-14T11:54:32.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.715+0000 7fde76a4f700 1 -- 172.21.15.104:0/2599055730 shutdown_connections 2023-07-14T11:54:32.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.715+0000 7fde76a4f700 1 --2- 172.21.15.104:0/2599055730 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fde540662f0 0x7fde540687a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.715+0000 7fde76a4f700 1 --2- 172.21.15.104:0/2599055730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde701603d0 0x7fde700acf50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.715+0000 7fde76a4f700 1 --2- 172.21.15.104:0/2599055730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde70160000 0x7fde700aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.715+0000 7fde76a4f700 1 --2- 172.21.15.104:0/2599055730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde700cd7f0 0x7fde70163f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.715+0000 7fde76a4f700 1 -- 172.21.15.104:0/2599055730 >> 172.21.15.104:0/2599055730 conn(0x7fde700afc60 msgr2=0x7fde700aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:32.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.715+0000 7fde76a4f700 1 -- 172.21.15.104:0/2599055730 shutdown_connections 2023-07-14T11:54:32.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.715+0000 7fde76a4f700 1 -- 172.21.15.104:0/2599055730 wait complete. 2023-07-14T11:54:32.718 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:54:32.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.907+0000 7ff6cb04a700 1 Processor -- start 2023-07-14T11:54:32.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.907+0000 7ff6cb04a700 1 -- start start 2023-07-14T11:54:32.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.911+0000 7ff6cb04a700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff6c415df10 0x7ff6c415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.911+0000 7ff6cb04a700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff6c40c8950 0x7ff6c40c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.911+0000 7ff6cb04a700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6c40c9290 con 0x7ff6c40afd80 2023-07-14T11:54:32.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.911+0000 7ff6cb04a700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c415e830 con 0x7ff6c415df10 2023-07-14T11:54:32.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.911+0000 7ff6cb04a700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c4070a30 con 0x7ff6c40c8950 2023-07-14T11:54:32.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.911+0000 7ff6c8de6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff6c415df10 0x7ff6c415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.911+0000 7ff6c8de6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff6c415df10 0x7ff6c415e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43854/0 (socket says 172.21.15.104:43854) 2023-07-14T11:54:32.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.911+0000 7ff6c8de6700 1 -- 172.21.15.104:0/1964780798 learned_addr learned my addr 172.21.15.104:0/1964780798 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:32.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.911+0000 7ff6c95e7700 1 --2- 172.21.15.104:0/1964780798 >> v2:172.21.15.121:3300/0 conn(0x7ff6c40c8950 0x7ff6c40c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.911+0000 7ff6c8de6700 1 -- 172.21.15.104:0/1964780798 >> v2:172.21.15.121:3300/0 conn(0x7ff6c40c8950 msgr2=0x7ff6c40c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:32.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.911+0000 7ff6c8de6700 1 --2- 172.21.15.104:0/1964780798 >> v2:172.21.15.121:3300/0 conn(0x7ff6c40c8950 0x7ff6c40c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.911+0000 7ff6c8de6700 1 -- 172.21.15.104:0/1964780798 >> v1:172.21.15.104:6789/0 conn(0x7ff6c40afd80 legacy=0x7ff6c40711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:32.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.911+0000 7ff6c8de6700 1 -- 172.21.15.104:0/1964780798 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c40b0210 con 0x7ff6c415df10 2023-07-14T11:54:32.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.911+0000 7ff6c8de6700 1 --2- 172.21.15.104:0/1964780798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c415df10 0x7ff6c415e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40c88e0 tx=0x7ff6b8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124e335c4ee382ab server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:32.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.915+0000 7ff6c37fe700 1 -- 172.21.15.104:0/1964780798 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b800b8a0 con 0x7ff6c415df10 2023-07-14T11:54:32.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.915+0000 7ff6c37fe700 1 -- 172.21.15.104:0/1964780798 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b800ba00 con 0x7ff6c415df10 2023-07-14T11:54:32.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.915+0000 7ff6c37fe700 1 -- 172.21.15.104:0/1964780798 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b800bd90 con 0x7ff6c415df10 2023-07-14T11:54:32.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.915+0000 7ff6cb04a700 1 -- 172.21.15.104:0/1964780798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c415df10 msgr2=0x7ff6c415e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:32.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.915+0000 7ff6cb04a700 1 --2- 172.21.15.104:0/1964780798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c415df10 0x7ff6c415e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40c88e0 tx=0x7ff6b8009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.915+0000 7ff6cb04a700 1 -- 172.21.15.104:0/1964780798 shutdown_connections 2023-07-14T11:54:32.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.915+0000 7ff6cb04a700 1 --2- 172.21.15.104:0/1964780798 >> v2:172.21.15.121:3300/0 conn(0x7ff6c40c8950 0x7ff6c40c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.915+0000 7ff6cb04a700 1 --2- 172.21.15.104:0/1964780798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c415df10 0x7ff6c415e2f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.915+0000 7ff6cb04a700 1 -- 172.21.15.104:0/1964780798 >> 172.21.15.104:0/1964780798 conn(0x7ff6c40cad80 msgr2=0x7ff6c40cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:32.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.915+0000 7ff6cb04a700 1 -- 172.21.15.104:0/1964780798 shutdown_connections 2023-07-14T11:54:32.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.915+0000 7ff6cb04a700 1 -- 172.21.15.104:0/1964780798 wait complete. 2023-07-14T11:54:32.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.915+0000 7ff6cb04a700 1 Processor -- start 2023-07-14T11:54:32.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.915+0000 7ff6cb04a700 1 -- start start 2023-07-14T11:54:32.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6cb04a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c40afd80 0x7ff6c40b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6cb04a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c40c8950 0x7ff6c40b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6cb04a700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c4152070 0x7ff6c4150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6cb04a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c40b23d0 con 0x7ff6c40afd80 2023-07-14T11:54:32.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6cb04a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c40b2da0 con 0x7ff6c40c8950 2023-07-14T11:54:32.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6cb04a700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c40b17c0 con 0x7ff6c4152070 2023-07-14T11:54:32.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6c8de6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c40afd80 0x7ff6c40b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6c95e7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c4152070 0x7ff6c4150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6c8de6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c40afd80 0x7ff6c40b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43862/0 (socket says 172.21.15.104:43862) 2023-07-14T11:54:32.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6c8de6700 1 -- 172.21.15.104:0/3540316910 learned_addr learned my addr 172.21.15.104:0/3540316910 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:32.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6c3fff700 1 --2- 172.21.15.104:0/3540316910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c40c8950 0x7ff6c40b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6c8de6700 1 -- 172.21.15.104:0/3540316910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c4152070 msgr2=0x7ff6c4150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:32.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6c8de6700 1 --2- 172.21.15.104:0/3540316910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c4152070 0x7ff6c4150780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6c8de6700 1 -- 172.21.15.104:0/3540316910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c40c8950 msgr2=0x7ff6c40b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:32.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6c8de6700 1 --2- 172.21.15.104:0/3540316910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c40c8950 0x7ff6c40b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:32.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6c8de6700 1 -- 172.21.15.104:0/3540316910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b8009570 con 0x7ff6c40afd80 2023-07-14T11:54:32.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6c3fff700 1 --2- 172.21.15.104:0/3540316910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c40c8950 0x7ff6c40b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:32.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6c8de6700 1 --2- 172.21.15.104:0/3540316910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c40afd80 0x7ff6c40b1130 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff6c406e410 tx=0x7ff6b8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:32.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6c17fa700 1 -- 172.21.15.104:0/3540316910 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b80036c0 con 0x7ff6c40afd80 2023-07-14T11:54:32.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6c17fa700 1 -- 172.21.15.104:0/3540316910 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b801f070 con 0x7ff6c40afd80 2023-07-14T11:54:32.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6cb04a700 1 -- 172.21.15.104:0/3540316910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c40b1aa0 con 0x7ff6c40afd80 2023-07-14T11:54:32.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6c17fa700 1 -- 172.21.15.104:0/3540316910 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8003a80 con 0x7ff6c40afd80 2023-07-14T11:54:32.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.919+0000 7ff6cb04a700 1 -- 172.21.15.104:0/3540316910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c40b2060 con 0x7ff6c40afd80 2023-07-14T11:54:32.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.923+0000 7ff6cb04a700 1 -- 172.21.15.104:0/3540316910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c406bbb0 con 0x7ff6c40afd80 2023-07-14T11:54:32.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.923+0000 7ff6c17fa700 1 -- 172.21.15.104:0/3540316910 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff6b8021030 con 0x7ff6c40afd80 2023-07-14T11:54:32.911 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.927+0000 7ff6c17fa700 1 --2- 172.21.15.104:0/3540316910 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6a80662f0 0x7ff6a80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:32.911 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.927+0000 7ff6c17fa700 1 -- 172.21.15.104:0/3540316910 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff6b80856b0 con 0x7ff6c40afd80 2023-07-14T11:54:32.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.927+0000 7ff6c3fff700 1 --2- 172.21.15.104:0/3540316910 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6a80662f0 0x7ff6a80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:32.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.927+0000 7ff6c3fff700 1 --2- 172.21.15.104:0/3540316910 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6a80662f0 0x7ff6a80687a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff6c41511f0 tx=0x7ff6b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:32.916 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:32.931+0000 7ff6c17fa700 1 -- 172.21.15.104:0/3540316910 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff6b8052910 con 0x7ff6c40afd80 2023-07-14T11:54:33.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.103+0000 7ff6cb04a700 1 -- 172.21.15.104:0/3540316910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff6c404dcb0 con 0x7ff6c40afd80 2023-07-14T11:54:33.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.103+0000 7ff6c17fa700 1 -- 172.21.15.104:0/3540316910 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff6b8055f30 con 0x7ff6c40afd80 2023-07-14T11:54:33.088 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:54:33.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.103+0000 7ff6cb04a700 1 -- 172.21.15.104:0/3540316910 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6a80662f0 msgr2=0x7ff6a80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.103+0000 7ff6cb04a700 1 --2- 172.21.15.104:0/3540316910 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6a80662f0 0x7ff6a80687a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff6c41511f0 tx=0x7ff6b0009040 comp rx=0 tx=0).stop 2023-07-14T11:54:33.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.103+0000 7ff6cb04a700 1 -- 172.21.15.104:0/3540316910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c40afd80 msgr2=0x7ff6c40b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.103+0000 7ff6cb04a700 1 --2- 172.21.15.104:0/3540316910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c40afd80 0x7ff6c40b1130 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff6c406e410 tx=0x7ff6b8004770 comp rx=0 tx=0).stop 2023-07-14T11:54:33.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.107+0000 7ff6cb04a700 1 -- 172.21.15.104:0/3540316910 shutdown_connections 2023-07-14T11:54:33.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.107+0000 7ff6cb04a700 1 --2- 172.21.15.104:0/3540316910 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6a80662f0 0x7ff6a80687a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.107+0000 7ff6cb04a700 1 --2- 172.21.15.104:0/3540316910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6c4152070 0x7ff6c4150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.107+0000 7ff6cb04a700 1 --2- 172.21.15.104:0/3540316910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6c40c8950 0x7ff6c40b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.107+0000 7ff6cb04a700 1 --2- 172.21.15.104:0/3540316910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c40afd80 0x7ff6c40b1130 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.107+0000 7ff6cb04a700 1 -- 172.21.15.104:0/3540316910 >> 172.21.15.104:0/3540316910 conn(0x7ff6c40cad80 msgr2=0x7ff6c40ca180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:33.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.107+0000 7ff6cb04a700 1 -- 172.21.15.104:0/3540316910 shutdown_connections 2023-07-14T11:54:33.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.107+0000 7ff6cb04a700 1 -- 172.21.15.104:0/3540316910 wait complete. 2023-07-14T11:54:33.111 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:54:33.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.303+0000 7f9be49a0700 1 Processor -- start 2023-07-14T11:54:33.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.303+0000 7f9be49a0700 1 -- start start 2023-07-14T11:54:33.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9be49a0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9be00b1c50 0x7f9be0071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9be49a0700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9be00cf710 0x7f9be00d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9be49a0700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be015c420 con 0x7f9be00d0210 2023-07-14T11:54:33.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9be49a0700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be0070840 con 0x7f9be00cf710 2023-07-14T11:54:33.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9be49a0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be0070b20 con 0x7f9be00b1c50 2023-07-14T11:54:33.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9bded9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9be00cf710 0x7f9be00d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9bde59c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9be00b1c50 0x7f9be0071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9bded9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9be00cf710 0x7f9be00d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:37002/0 (socket says 172.21.15.104:37002) 2023-07-14T11:54:33.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9bded9d700 1 -- 172.21.15.104:0/299804016 learned_addr learned my addr 172.21.15.104:0/299804016 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:33.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9bded9d700 1 -- 172.21.15.104:0/299804016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9be00b1c50 msgr2=0x7f9be0071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:33.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9bded9d700 1 --2- 172.21.15.104:0/299804016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9be00b1c50 0x7f9be0071600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9bded9d700 1 -- 172.21.15.104:0/299804016 >> v1:172.21.15.121:6789/0 conn(0x7f9be00d0210 legacy=0x7f9be00cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9bded9d700 1 -- 172.21.15.104:0/299804016 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be0070e00 con 0x7f9be00cf710 2023-07-14T11:54:33.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9bde59c700 1 --2- 172.21.15.104:0/299804016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9be00b1c50 0x7f9be0071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:33.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9bded9d700 1 --2- 172.21.15.104:0/299804016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9be00cf710 0x7f9be00d0640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009e50 tx=0x7f9bd800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6061268212cb52af server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:33.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9bdd59a700 1 -- 172.21.15.104:0/299804016 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd800e070 con 0x7f9be00cf710 2023-07-14T11:54:33.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9bdd59a700 1 -- 172.21.15.104:0/299804016 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd800b910 con 0x7f9be00cf710 2023-07-14T11:54:33.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.307+0000 7f9bdd59a700 1 -- 172.21.15.104:0/299804016 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd800bcc0 con 0x7f9be00cf710 2023-07-14T11:54:33.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.311+0000 7f9be49a0700 1 -- 172.21.15.104:0/299804016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9be00cf710 msgr2=0x7f9be00d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.311+0000 7f9be49a0700 1 --2- 172.21.15.104:0/299804016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9be00cf710 0x7f9be00d0640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009e50 tx=0x7f9bd800b330 comp rx=0 tx=0).stop 2023-07-14T11:54:33.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.311+0000 7f9be49a0700 1 -- 172.21.15.104:0/299804016 shutdown_connections 2023-07-14T11:54:33.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.311+0000 7f9be49a0700 1 --2- 172.21.15.104:0/299804016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9be00cf710 0x7f9be00d0640 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.311+0000 7f9be49a0700 1 --2- 172.21.15.104:0/299804016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9be00b1c50 0x7f9be0071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.311+0000 7f9be49a0700 1 -- 172.21.15.104:0/299804016 >> 172.21.15.104:0/299804016 conn(0x7f9be009b9c0 msgr2=0x7f9be00d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:33.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.311+0000 7f9be49a0700 1 -- 172.21.15.104:0/299804016 shutdown_connections 2023-07-14T11:54:33.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.311+0000 7f9be49a0700 1 -- 172.21.15.104:0/299804016 wait complete. 2023-07-14T11:54:33.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9be49a0700 1 Processor -- start 2023-07-14T11:54:33.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9be49a0700 1 -- start start 2023-07-14T11:54:33.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9be49a0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9be00b1c50 0x7f9be01f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9be49a0700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9be00cf710 0x7f9be015b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9be49a0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9be00d0210 0x7f9be015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9be49a0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be015bf10 con 0x7f9be00d0210 2023-07-14T11:54:33.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9be49a0700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be01f67d0 con 0x7f9be00cf710 2023-07-14T11:54:33.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9be49a0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00c71f0 con 0x7f9be00b1c50 2023-07-14T11:54:33.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9bded9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9be00d0210 0x7f9be015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9bded9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9be00d0210 0x7f9be015b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43866/0 (socket says 172.21.15.104:43866) 2023-07-14T11:54:33.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9bded9d700 1 -- 172.21.15.104:0/2897899421 learned_addr learned my addr 172.21.15.104:0/2897899421 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:33.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9bde59c700 1 --2- 172.21.15.104:0/2897899421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9be00b1c50 0x7f9be01f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9bddd9b700 1 --2- 172.21.15.104:0/2897899421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9be00cf710 0x7f9be015b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9bded9d700 1 -- 172.21.15.104:0/2897899421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9be00b1c50 msgr2=0x7f9be01f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9bded9d700 1 --2- 172.21.15.104:0/2897899421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9be00b1c50 0x7f9be01f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9bded9d700 1 -- 172.21.15.104:0/2897899421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9be00cf710 msgr2=0x7f9be015b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9bded9d700 1 --2- 172.21.15.104:0/2897899421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9be00cf710 0x7f9be015b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9bded9d700 1 -- 172.21.15.104:0/2897899421 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd00094b0 con 0x7f9be00d0210 2023-07-14T11:54:33.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9bded9d700 1 --2- 172.21.15.104:0/2897899421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9be00d0210 0x7f9be015b9d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8012000 tx=0x7f9bd8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:33.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.315+0000 7f9bceffd700 1 -- 172.21.15.104:0/2897899421 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8004960 con 0x7f9be00d0210 2023-07-14T11:54:33.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.319+0000 7f9bceffd700 1 -- 172.21.15.104:0/2897899421 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd80099b0 con 0x7f9be00d0210 2023-07-14T11:54:33.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.319+0000 7f9be49a0700 1 -- 172.21.15.104:0/2897899421 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd8009b30 con 0x7f9be00d0210 2023-07-14T11:54:33.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.319+0000 7f9bceffd700 1 -- 172.21.15.104:0/2897899421 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd801c680 con 0x7f9be00d0210 2023-07-14T11:54:33.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.319+0000 7f9be49a0700 1 -- 172.21.15.104:0/2897899421 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be00c77b0 con 0x7f9be00d0210 2023-07-14T11:54:33.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.319+0000 7f9be49a0700 1 -- 172.21.15.104:0/2897899421 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be004dcb0 con 0x7f9be00d0210 2023-07-14T11:54:33.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.319+0000 7f9bceffd700 1 -- 172.21.15.104:0/2897899421 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9bd801c7e0 con 0x7f9be00d0210 2023-07-14T11:54:33.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.323+0000 7f9bceffd700 1 --2- 172.21.15.104:0/2897899421 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9bc00662f0 0x7f9bc00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.323+0000 7f9bceffd700 1 -- 172.21.15.104:0/2897899421 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9bd8085d10 con 0x7f9be00d0210 2023-07-14T11:54:33.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.323+0000 7f9bde59c700 1 --2- 172.21.15.104:0/2897899421 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9bc00662f0 0x7f9bc00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.309 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.323+0000 7f9bde59c700 1 --2- 172.21.15.104:0/2897899421 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9bc00662f0 0x7f9bc00687a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009d10 tx=0x7f9bd0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:33.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.327+0000 7f9bceffd700 1 -- 172.21.15.104:0/2897899421 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f9bd8052ef0 con 0x7f9be00d0210 2023-07-14T11:54:33.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.499+0000 7f9be49a0700 1 -- 172.21.15.104:0/2897899421 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9be00b7fc0 con 0x7f9be00d0210 2023-07-14T11:54:33.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.499+0000 7f9bceffd700 1 -- 172.21.15.104:0/2897899421 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9bd8056510 con 0x7f9be00d0210 2023-07-14T11:54:33.486 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:33.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.507+0000 7f9be49a0700 1 -- 172.21.15.104:0/2897899421 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9bc00662f0 msgr2=0x7f9bc00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.507+0000 7f9be49a0700 1 --2- 172.21.15.104:0/2897899421 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9bc00662f0 0x7f9bc00687a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009d10 tx=0x7f9bd0008040 comp rx=0 tx=0).stop 2023-07-14T11:54:33.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.507+0000 7f9be49a0700 1 -- 172.21.15.104:0/2897899421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9be00d0210 msgr2=0x7f9be015b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.507+0000 7f9be49a0700 1 --2- 172.21.15.104:0/2897899421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9be00d0210 0x7f9be015b9d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8012000 tx=0x7f9bd8004af0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.507+0000 7f9be49a0700 1 -- 172.21.15.104:0/2897899421 shutdown_connections 2023-07-14T11:54:33.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.507+0000 7f9be49a0700 1 --2- 172.21.15.104:0/2897899421 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9bc00662f0 0x7f9bc00687a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.507+0000 7f9be49a0700 1 --2- 172.21.15.104:0/2897899421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9be00d0210 0x7f9be015b9d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.507+0000 7f9be49a0700 1 --2- 172.21.15.104:0/2897899421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9be00cf710 0x7f9be015b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.507+0000 7f9be49a0700 1 --2- 172.21.15.104:0/2897899421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9be00b1c50 0x7f9be01f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.507+0000 7f9be49a0700 1 -- 172.21.15.104:0/2897899421 >> 172.21.15.104:0/2897899421 conn(0x7f9be009b9c0 msgr2=0x7f9be00b6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:33.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.507+0000 7f9be49a0700 1 -- 172.21.15.104:0/2897899421 shutdown_connections 2023-07-14T11:54:33.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.507+0000 7f9be49a0700 1 -- 172.21.15.104:0/2897899421 wait complete. 2023-07-14T11:54:33.515 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:54:33.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.699+0000 7f2e85399700 1 Processor -- start 2023-07-14T11:54:33.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.699+0000 7f2e85399700 1 -- start start 2023-07-14T11:54:33.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e85399700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e8015df10 0x7f2e8015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e85399700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2e800afd80 0x7f2e800711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e85399700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e80070a30 con 0x7f2e800716e0 2023-07-14T11:54:33.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e85399700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8015e830 con 0x7f2e800afd80 2023-07-14T11:54:33.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e85399700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800b0210 con 0x7f2e8015df10 2023-07-14T11:54:33.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e77fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2e800afd80 0x7f2e800711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e7f7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f2e800716e0 0x7f2e800c8950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:41606/0 (socket says 172.21.15.104:41606) 2023-07-14T11:54:33.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e7f7fe700 1 -- 172.21.15.104:0/900040800 learned_addr learned my addr 172.21.15.104:0/900040800 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:33.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e7effd700 1 --2- 172.21.15.104:0/900040800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e8015df10 0x7f2e8015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e7e7fc700 1 -- 172.21.15.104:0/900040800 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435890847 0 0) 0x7f2e80070a30 con 0x7f2e800716e0 2023-07-14T11:54:33.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e7e7fc700 1 -- 172.21.15.104:0/900040800 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e6c0035d0 con 0x7f2e800716e0 2023-07-14T11:54:33.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e7effd700 1 -- 172.21.15.104:0/900040800 >> v2:172.21.15.104:3300/0 conn(0x7f2e800afd80 msgr2=0x7f2e800711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:33.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e7effd700 1 --2- 172.21.15.104:0/900040800 >> v2:172.21.15.104:3300/0 conn(0x7f2e800afd80 0x7f2e800711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e7effd700 1 -- 172.21.15.104:0/900040800 >> v1:172.21.15.121:6789/0 conn(0x7f2e800716e0 legacy=0x7f2e800c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e7effd700 1 -- 172.21.15.104:0/900040800 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e800b04f0 con 0x7f2e8015df10 2023-07-14T11:54:33.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.703+0000 7f2e7effd700 1 --2- 172.21.15.104:0/900040800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e8015df10 0x7f2e8015e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009850 tx=0x7f2e70009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c1762c3164f02ab server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:33.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.707+0000 7f2e7e7fc700 1 -- 172.21.15.104:0/900040800 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70011830 con 0x7f2e8015df10 2023-07-14T11:54:33.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.707+0000 7f2e7e7fc700 1 -- 172.21.15.104:0/900040800 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e70011990 con 0x7f2e8015df10 2023-07-14T11:54:33.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.707+0000 7f2e7e7fc700 1 -- 172.21.15.104:0/900040800 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70011d40 con 0x7f2e8015df10 2023-07-14T11:54:33.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.707+0000 7f2e85399700 1 -- 172.21.15.104:0/900040800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e8015df10 msgr2=0x7f2e8015e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.707+0000 7f2e85399700 1 --2- 172.21.15.104:0/900040800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e8015df10 0x7f2e8015e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009850 tx=0x7f2e70009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:33.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.707+0000 7f2e85399700 1 -- 172.21.15.104:0/900040800 shutdown_connections 2023-07-14T11:54:33.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.707+0000 7f2e85399700 1 --2- 172.21.15.104:0/900040800 >> v2:172.21.15.104:3300/0 conn(0x7f2e800afd80 0x7f2e800711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.707+0000 7f2e85399700 1 --2- 172.21.15.104:0/900040800 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e8015df10 0x7f2e8015e2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.707+0000 7f2e85399700 1 -- 172.21.15.104:0/900040800 >> 172.21.15.104:0/900040800 conn(0x7f2e800cad80 msgr2=0x7f2e800cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:33.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.707+0000 7f2e85399700 1 -- 172.21.15.104:0/900040800 shutdown_connections 2023-07-14T11:54:33.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.707+0000 7f2e85399700 1 -- 172.21.15.104:0/900040800 wait complete. 2023-07-14T11:54:33.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e85399700 1 Processor -- start 2023-07-14T11:54:33.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e85399700 1 -- start start 2023-07-14T11:54:33.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e85399700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e800716e0 0x7f2e801f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e85399700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e800afd80 0x7f2e800c5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e85399700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e8015bd10 0x7f2e8014c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e85399700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e801f24e0 con 0x7f2e8015bd10 2023-07-14T11:54:33.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e85399700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e801f48f0 con 0x7f2e800afd80 2023-07-14T11:54:33.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e85399700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e801f4b90 con 0x7f2e800716e0 2023-07-14T11:54:33.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e7f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e8015bd10 0x7f2e8014c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e7f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e8015bd10 0x7f2e8014c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43884/0 (socket says 172.21.15.104:43884) 2023-07-14T11:54:33.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e7f7fe700 1 -- 172.21.15.104:0/2397928011 learned_addr learned my addr 172.21.15.104:0/2397928011 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:33.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e7effd700 1 --2- 172.21.15.104:0/2397928011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e800716e0 0x7f2e801f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e77fff700 1 --2- 172.21.15.104:0/2397928011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e800afd80 0x7f2e800c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e7f7fe700 1 -- 172.21.15.104:0/2397928011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e800716e0 msgr2=0x7f2e801f1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e7f7fe700 1 --2- 172.21.15.104:0/2397928011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e800716e0 0x7f2e801f1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e7f7fe700 1 -- 172.21.15.104:0/2397928011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e800afd80 msgr2=0x7f2e800c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e7f7fe700 1 --2- 172.21.15.104:0/2397928011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e800afd80 0x7f2e800c5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e7f7fe700 1 -- 172.21.15.104:0/2397928011 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e700094b0 con 0x7f2e8015bd10 2023-07-14T11:54:33.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.711+0000 7f2e7f7fe700 1 --2- 172.21.15.104:0/2397928011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e8015bd10 0x7f2e8014c2e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800cb40 tx=0x7f2e7800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:33.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.715+0000 7f2e777fe700 1 -- 172.21.15.104:0/2397928011 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78007500 con 0x7f2e8015bd10 2023-07-14T11:54:33.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.715+0000 7f2e777fe700 1 -- 172.21.15.104:0/2397928011 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e78007660 con 0x7f2e8015bd10 2023-07-14T11:54:33.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.715+0000 7f2e85399700 1 -- 172.21.15.104:0/2397928011 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e801f4e70 con 0x7f2e8015bd10 2023-07-14T11:54:33.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.715+0000 7f2e777fe700 1 -- 172.21.15.104:0/2397928011 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e780079f0 con 0x7f2e8015bd10 2023-07-14T11:54:33.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.715+0000 7f2e85399700 1 -- 172.21.15.104:0/2397928011 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e801f5430 con 0x7f2e8015bd10 2023-07-14T11:54:33.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.715+0000 7f2e85399700 1 -- 172.21.15.104:0/2397928011 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8015d860 con 0x7f2e8015bd10 2023-07-14T11:54:33.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.719+0000 7f2e777fe700 1 -- 172.21.15.104:0/2397928011 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2e7801b3b0 con 0x7f2e8015bd10 2023-07-14T11:54:33.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.719+0000 7f2e777fe700 1 --2- 172.21.15.104:0/2397928011 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2e600662f0 0x7f2e600687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:33.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.719+0000 7f2e777fe700 1 -- 172.21.15.104:0/2397928011 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2e78083aa0 con 0x7f2e8015bd10 2023-07-14T11:54:33.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.719+0000 7f2e7effd700 1 --2- 172.21.15.104:0/2397928011 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2e600662f0 0x7f2e600687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:33.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.719+0000 7f2e7effd700 1 --2- 172.21.15.104:0/2397928011 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2e600662f0 0x7f2e600687a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e700090b0 tx=0x7f2e7001c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:33.712 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.723+0000 7f2e777fe700 1 -- 172.21.15.104:0/2397928011 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f2e78050be0 con 0x7f2e8015bd10 2023-07-14T11:54:33.879 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.895+0000 7f2e85399700 1 -- 172.21.15.104:0/2397928011 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2e801f5620 con 0x7f2e8015bd10 2023-07-14T11:54:33.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.895+0000 7f2e777fe700 1 -- 172.21.15.104:0/2397928011 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2e78017020 con 0x7f2e8015bd10 2023-07-14T11:54:33.880 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:54:33.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.899+0000 7f2e85399700 1 -- 172.21.15.104:0/2397928011 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2e600662f0 msgr2=0x7f2e600687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.899+0000 7f2e85399700 1 --2- 172.21.15.104:0/2397928011 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2e600662f0 0x7f2e600687a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e700090b0 tx=0x7f2e7001c040 comp rx=0 tx=0).stop 2023-07-14T11:54:33.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.899+0000 7f2e85399700 1 -- 172.21.15.104:0/2397928011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e8015bd10 msgr2=0x7f2e8014c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:33.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.899+0000 7f2e85399700 1 --2- 172.21.15.104:0/2397928011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e8015bd10 0x7f2e8014c2e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800cb40 tx=0x7f2e7800cb70 comp rx=0 tx=0).stop 2023-07-14T11:54:33.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.899+0000 7f2e85399700 1 -- 172.21.15.104:0/2397928011 shutdown_connections 2023-07-14T11:54:33.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.899+0000 7f2e85399700 1 --2- 172.21.15.104:0/2397928011 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2e600662f0 0x7f2e600687a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.899+0000 7f2e85399700 1 --2- 172.21.15.104:0/2397928011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e8015bd10 0x7f2e8014c2e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.899+0000 7f2e85399700 1 --2- 172.21.15.104:0/2397928011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e800afd80 0x7f2e800c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.899+0000 7f2e85399700 1 --2- 172.21.15.104:0/2397928011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e800716e0 0x7f2e801f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:33.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.899+0000 7f2e85399700 1 -- 172.21.15.104:0/2397928011 >> 172.21.15.104:0/2397928011 conn(0x7f2e800cad80 msgr2=0x7f2e8006c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:33.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.899+0000 7f2e85399700 1 -- 172.21.15.104:0/2397928011 shutdown_connections 2023-07-14T11:54:33.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:33.899+0000 7f2e85399700 1 -- 172.21.15.104:0/2397928011 wait complete. 2023-07-14T11:54:33.907 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:54:34.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.103+0000 7f3b8a74c700 1 Processor -- start 2023-07-14T11:54:34.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.103+0000 7f3b8a74c700 1 -- start start 2023-07-14T11:54:34.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b8a74c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b840b0ee0 0x7f3b840ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b8a74c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3b840cd090 0x7f3b840d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b8a74c700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b840cd4e0 con 0x7f3b840d0610 2023-07-14T11:54:34.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b8a74c700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b840d1a60 con 0x7f3b840cd090 2023-07-14T11:54:34.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b8a74c700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840d1ef0 con 0x7f3b840b0ee0 2023-07-14T11:54:34.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b88f49700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3b840cd090 0x7f3b840d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b89f4b700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f3b840d0610 0x7f3b8415f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:45666/0 (socket says 172.21.15.104:45666) 2023-07-14T11:54:34.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b89f4b700 1 -- 172.21.15.104:0/745454231 learned_addr learned my addr 172.21.15.104:0/745454231 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:34.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b8974a700 1 --2- 172.21.15.104:0/745454231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b840b0ee0 0x7f3b840ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b88f49700 1 -- 172.21.15.104:0/745454231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b840b0ee0 msgr2=0x7f3b840ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:34.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b88f49700 1 --2- 172.21.15.104:0/745454231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b840b0ee0 0x7f3b840ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b88f49700 1 -- 172.21.15.104:0/745454231 >> v1:172.21.15.104:6789/0 conn(0x7f3b840d0610 legacy=0x7f3b8415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b88f49700 1 -- 172.21.15.104:0/745454231 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b840d0e50 con 0x7f3b840cd090 2023-07-14T11:54:34.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b88f49700 1 --2- 172.21.15.104:0/745454231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b840cd090 0x7f3b840d00d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009880 tx=0x7f3b74009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d780c27b2edc482 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:34.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b7bfff700 1 -- 172.21.15.104:0/745454231 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b74011830 con 0x7f3b840cd090 2023-07-14T11:54:34.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b7bfff700 1 -- 172.21.15.104:0/745454231 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b74011990 con 0x7f3b840cd090 2023-07-14T11:54:34.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.107+0000 7f3b7bfff700 1 -- 172.21.15.104:0/745454231 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b74011d20 con 0x7f3b840cd090 2023-07-14T11:54:34.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.111+0000 7f3b8a74c700 1 -- 172.21.15.104:0/745454231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b840cd090 msgr2=0x7f3b840d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.111+0000 7f3b8a74c700 1 --2- 172.21.15.104:0/745454231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b840cd090 0x7f3b840d00d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009880 tx=0x7f3b74009b90 comp rx=0 tx=0).stop 2023-07-14T11:54:34.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.111+0000 7f3b8a74c700 1 -- 172.21.15.104:0/745454231 shutdown_connections 2023-07-14T11:54:34.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.111+0000 7f3b8a74c700 1 --2- 172.21.15.104:0/745454231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b840cd090 0x7f3b840d00d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.111+0000 7f3b8a74c700 1 --2- 172.21.15.104:0/745454231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b840b0ee0 0x7f3b840ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.111+0000 7f3b8a74c700 1 -- 172.21.15.104:0/745454231 >> 172.21.15.104:0/745454231 conn(0x7f3b84060c40 msgr2=0x7f3b8409b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:34.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.111+0000 7f3b8a74c700 1 -- 172.21.15.104:0/745454231 shutdown_connections 2023-07-14T11:54:34.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.111+0000 7f3b8a74c700 1 -- 172.21.15.104:0/745454231 wait complete. 2023-07-14T11:54:34.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.111+0000 7f3b8a74c700 1 Processor -- start 2023-07-14T11:54:34.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.111+0000 7f3b8a74c700 1 -- start start 2023-07-14T11:54:34.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8a74c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b840b0ee0 0x7f3b840b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8a74c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b840cd090 0x7f3b840cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8a74c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b840d0610 0x7f3b840b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8a74c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b841f1170 con 0x7f3b840b0ee0 2023-07-14T11:54:34.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8a74c700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840b1e90 con 0x7f3b840d0610 2023-07-14T11:54:34.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8a74c700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840b2430 con 0x7f3b840cd090 2023-07-14T11:54:34.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8974a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b840b0ee0 0x7f3b840b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8974a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b840b0ee0 0x7f3b840b6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43894/0 (socket says 172.21.15.104:43894) 2023-07-14T11:54:34.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8974a700 1 -- 172.21.15.104:0/2281327498 learned_addr learned my addr 172.21.15.104:0/2281327498 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:34.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b88f49700 1 --2- 172.21.15.104:0/2281327498 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b840cd090 0x7f3b840cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b89f4b700 1 --2- 172.21.15.104:0/2281327498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b840d0610 0x7f3b840b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8974a700 1 -- 172.21.15.104:0/2281327498 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b840cd090 msgr2=0x7f3b840cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8974a700 1 --2- 172.21.15.104:0/2281327498 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b840cd090 0x7f3b840cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8974a700 1 -- 172.21.15.104:0/2281327498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b840d0610 msgr2=0x7f3b840b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8974a700 1 --2- 172.21.15.104:0/2281327498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b840d0610 0x7f3b840b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8974a700 1 -- 172.21.15.104:0/2281327498 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b740094b0 con 0x7f3b840b0ee0 2023-07-14T11:54:34.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b89f4b700 1 --2- 172.21.15.104:0/2281327498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b840d0610 0x7f3b840b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:34.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8974a700 1 --2- 172.21.15.104:0/2281327498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b840b0ee0 0x7f3b840b6380 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000e840 tx=0x7f3b8000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:34.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b79ffb700 1 -- 172.21.15.104:0/2281327498 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b800095a0 con 0x7f3b840b0ee0 2023-07-14T11:54:34.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b79ffb700 1 -- 172.21.15.104:0/2281327498 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8000c690 con 0x7f3b840b0ee0 2023-07-14T11:54:34.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8a74c700 1 -- 172.21.15.104:0/2281327498 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b840cc140 con 0x7f3b840b0ee0 2023-07-14T11:54:34.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b79ffb700 1 -- 172.21.15.104:0/2281327498 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8000ca00 con 0x7f3b840b0ee0 2023-07-14T11:54:34.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.115+0000 7f3b8a74c700 1 -- 172.21.15.104:0/2281327498 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b840b20c0 con 0x7f3b840b0ee0 2023-07-14T11:54:34.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.119+0000 7f3b8a74c700 1 -- 172.21.15.104:0/2281327498 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8404e500 con 0x7f3b840b0ee0 2023-07-14T11:54:34.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.119+0000 7f3b79ffb700 1 -- 172.21.15.104:0/2281327498 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3b8000cb60 con 0x7f3b840b0ee0 2023-07-14T11:54:34.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.123+0000 7f3b79ffb700 1 --2- 172.21.15.104:0/2281327498 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3b68066320 0x7f3b680687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.123+0000 7f3b79ffb700 1 -- 172.21.15.104:0/2281327498 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3b80083fa0 con 0x7f3b840b0ee0 2023-07-14T11:54:34.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.123+0000 7f3b88f49700 1 --2- 172.21.15.104:0/2281327498 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3b68066320 0x7f3b680687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.123+0000 7f3b88f49700 1 --2- 172.21.15.104:0/2281327498 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3b68066320 0x7f3b680687d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009bd0 tx=0x7f3b74008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:34.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.127+0000 7f3b79ffb700 1 -- 172.21.15.104:0/2281327498 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f3b800b4720 con 0x7f3b840b0ee0 2023-07-14T11:54:34.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.295+0000 7f3b8a74c700 1 -- 172.21.15.104:0/2281327498 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3b84153f20 con 0x7f3b840b0ee0 2023-07-14T11:54:34.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.295+0000 7f3b79ffb700 1 -- 172.21.15.104:0/2281327498 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3b80054b80 con 0x7f3b840b0ee0 2023-07-14T11:54:34.283 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:34.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.299+0000 7f3b8a74c700 1 -- 172.21.15.104:0/2281327498 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3b68066320 msgr2=0x7f3b680687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.299+0000 7f3b8a74c700 1 --2- 172.21.15.104:0/2281327498 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3b68066320 0x7f3b680687d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009bd0 tx=0x7f3b74008030 comp rx=0 tx=0).stop 2023-07-14T11:54:34.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.299+0000 7f3b8a74c700 1 -- 172.21.15.104:0/2281327498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b840b0ee0 msgr2=0x7f3b840b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.299+0000 7f3b8a74c700 1 --2- 172.21.15.104:0/2281327498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b840b0ee0 0x7f3b840b6380 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000e840 tx=0x7f3b8000ec00 comp rx=0 tx=0).stop 2023-07-14T11:54:34.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.299+0000 7f3b8a74c700 1 -- 172.21.15.104:0/2281327498 shutdown_connections 2023-07-14T11:54:34.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.299+0000 7f3b8a74c700 1 --2- 172.21.15.104:0/2281327498 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3b68066320 0x7f3b680687d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.299+0000 7f3b8a74c700 1 --2- 172.21.15.104:0/2281327498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b840d0610 0x7f3b840b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.299+0000 7f3b8a74c700 1 --2- 172.21.15.104:0/2281327498 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b840cd090 0x7f3b840cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.299+0000 7f3b8a74c700 1 --2- 172.21.15.104:0/2281327498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b840b0ee0 0x7f3b840b6380 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.299+0000 7f3b8a74c700 1 -- 172.21.15.104:0/2281327498 >> 172.21.15.104:0/2281327498 conn(0x7f3b84060c40 msgr2=0x7f3b8415b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:34.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.299+0000 7f3b8a74c700 1 -- 172.21.15.104:0/2281327498 shutdown_connections 2023-07-14T11:54:34.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.299+0000 7f3b8a74c700 1 -- 172.21.15.104:0/2281327498 wait complete. 2023-07-14T11:54:34.306 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:54:34.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.495+0000 7f8511fe0700 1 Processor -- start 2023-07-14T11:54:34.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.495+0000 7f8511fe0700 1 -- start start 2023-07-14T11:54:34.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f8511fe0700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f850c071570 0x7f850c1589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f8511fe0700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850c158ef0 con 0x7f850c070ab0 2023-07-14T11:54:34.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f8511fe0700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850c0cc3e0 con 0x7f850c155ee0 2023-07-14T11:54:34.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f8511fe0700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850c1599c0 con 0x7f850c071570 2023-07-14T11:54:34.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f850bfff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f850c071570 0x7f850c1589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f850bfff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f850c071570 0x7f850c1589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43904/0 (socket says 172.21.15.104:43904) 2023-07-14T11:54:34.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f850bfff700 1 -- 172.21.15.104:0/3426551571 learned_addr learned my addr 172.21.15.104:0/3426551571 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:34.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f850bfff700 1 -- 172.21.15.104:0/3426551571 >> v1:172.21.15.121:6789/0 conn(0x7f850c155ee0 legacy=0x7f850c0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f850bfff700 1 -- 172.21.15.104:0/3426551571 >> v1:172.21.15.104:6789/0 conn(0x7f850c070ab0 legacy=0x7f850c0cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f850bfff700 1 -- 172.21.15.104:0/3426551571 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c159120 con 0x7f850c071570 2023-07-14T11:54:34.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f850bfff700 1 --2- 172.21.15.104:0/3426551571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f850c071570 0x7f850c1589b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f850400d150 tx=0x7f850400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3de08155c54c087 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:34.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f850a7fc700 1 -- 172.21.15.104:0/3426551571 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504010070 con 0x7f850c071570 2023-07-14T11:54:34.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f850a7fc700 1 -- 172.21.15.104:0/3426551571 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850400dbb0 con 0x7f850c071570 2023-07-14T11:54:34.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f850a7fc700 1 -- 172.21.15.104:0/3426551571 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504016400 con 0x7f850c071570 2023-07-14T11:54:34.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f8511fe0700 1 -- 172.21.15.104:0/3426551571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f850c071570 msgr2=0x7f850c1589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f8511fe0700 1 --2- 172.21.15.104:0/3426551571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f850c071570 0x7f850c1589b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f850400d150 tx=0x7f850400d460 comp rx=0 tx=0).stop 2023-07-14T11:54:34.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f8511fe0700 1 -- 172.21.15.104:0/3426551571 shutdown_connections 2023-07-14T11:54:34.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f8511fe0700 1 --2- 172.21.15.104:0/3426551571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f850c071570 0x7f850c1589b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f8511fe0700 1 -- 172.21.15.104:0/3426551571 >> 172.21.15.104:0/3426551571 conn(0x7f850c0cad30 msgr2=0x7f850c0cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:34.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f8511fe0700 1 -- 172.21.15.104:0/3426551571 shutdown_connections 2023-07-14T11:54:34.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.499+0000 7f8511fe0700 1 -- 172.21.15.104:0/3426551571 wait complete. 2023-07-14T11:54:34.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.503+0000 7f8511fe0700 1 Processor -- start 2023-07-14T11:54:34.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.503+0000 7f8511fe0700 1 -- start start 2023-07-14T11:54:34.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f8511fe0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f850c070ab0 0x7f850c153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f8511fe0700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f850c071570 0x7f850c144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f8511fe0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f850c155ee0 0x7f850c144720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f8511fe0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c144c60 con 0x7f850c070ab0 2023-07-14T11:54:34.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f8511fe0700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c147db0 con 0x7f850c071570 2023-07-14T11:54:34.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f8511fe0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850c0ae2e0 con 0x7f850c155ee0 2023-07-14T11:54:34.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f850b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f850c070ab0 0x7f850c153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f850b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f850c070ab0 0x7f850c153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43910/0 (socket says 172.21.15.104:43910) 2023-07-14T11:54:34.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f850b7fe700 1 -- 172.21.15.104:0/3294486925 learned_addr learned my addr 172.21.15.104:0/3294486925 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:34.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f850bfff700 1 --2- 172.21.15.104:0/3294486925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f850c155ee0 0x7f850c144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f850affd700 1 --2- 172.21.15.104:0/3294486925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f850c071570 0x7f850c144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f850b7fe700 1 -- 172.21.15.104:0/3294486925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f850c155ee0 msgr2=0x7f850c144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f850b7fe700 1 --2- 172.21.15.104:0/3294486925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f850c155ee0 0x7f850c144720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f850b7fe700 1 -- 172.21.15.104:0/3294486925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f850c071570 msgr2=0x7f850c144ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f850b7fe700 1 --2- 172.21.15.104:0/3294486925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f850c071570 0x7f850c144ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f850b7fe700 1 -- 172.21.15.104:0/3294486925 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850400bd40 con 0x7f850c070ab0 2023-07-14T11:54:34.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f850b7fe700 1 --2- 172.21.15.104:0/3294486925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f850c070ab0 0x7f850c153ab0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c500 tx=0x7f84fc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:34.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f84f3fff700 1 -- 172.21.15.104:0/3294486925 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc0052d0 con 0x7f850c070ab0 2023-07-14T11:54:34.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f84f3fff700 1 -- 172.21.15.104:0/3294486925 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc005430 con 0x7f850c070ab0 2023-07-14T11:54:34.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f8511fe0700 1 -- 172.21.15.104:0/3294486925 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850c0ae580 con 0x7f850c070ab0 2023-07-14T11:54:34.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f84f3fff700 1 -- 172.21.15.104:0/3294486925 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc0055a0 con 0x7f850c070ab0 2023-07-14T11:54:34.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.507+0000 7f8511fe0700 1 -- 172.21.15.104:0/3294486925 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850c0aeb40 con 0x7f850c070ab0 2023-07-14T11:54:34.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.511+0000 7f8511fe0700 1 -- 172.21.15.104:0/3294486925 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850c04dcb0 con 0x7f850c070ab0 2023-07-14T11:54:34.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.511+0000 7f84f3fff700 1 -- 172.21.15.104:0/3294486925 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f84fc0038e0 con 0x7f850c070ab0 2023-07-14T11:54:34.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.515+0000 7f84f3fff700 1 --2- 172.21.15.104:0/3294486925 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f84f80662f0 0x7f84f80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:34.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.515+0000 7f84f3fff700 1 -- 172.21.15.104:0/3294486925 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f84fc083340 con 0x7f850c070ab0 2023-07-14T11:54:34.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.515+0000 7f850affd700 1 --2- 172.21.15.104:0/3294486925 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f84f80662f0 0x7f84f80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:34.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.515+0000 7f850affd700 1 --2- 172.21.15.104:0/3294486925 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f84f80662f0 0x7f84f80687a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f850c147a10 tx=0x7f8500009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:34.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.519+0000 7f84f3fff700 1 -- 172.21.15.104:0/3294486925 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f84fc050520 con 0x7f850c070ab0 2023-07-14T11:54:34.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.711+0000 7f8511fe0700 1 -- 172.21.15.104:0/3294486925 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f850c0b07e0 con 0x7f850c070ab0 2023-07-14T11:54:34.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.711+0000 7f84f3fff700 1 -- 172.21.15.104:0/3294486925 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f84fc053b40 con 0x7f850c070ab0 2023-07-14T11:54:34.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.711+0000 7f8511fe0700 1 -- 172.21.15.104:0/3294486925 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f84f80662f0 msgr2=0x7f84f80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.711+0000 7f8511fe0700 1 --2- 172.21.15.104:0/3294486925 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f84f80662f0 0x7f84f80687a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f850c147a10 tx=0x7f8500009040 comp rx=0 tx=0).stop 2023-07-14T11:54:34.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.711+0000 7f8511fe0700 1 -- 172.21.15.104:0/3294486925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f850c070ab0 msgr2=0x7f850c153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:34.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.711+0000 7f8511fe0700 1 --2- 172.21.15.104:0/3294486925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f850c070ab0 0x7f850c153ab0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c500 tx=0x7f84fc00c810 comp rx=0 tx=0).stop 2023-07-14T11:54:34.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.711+0000 7f8511fe0700 1 -- 172.21.15.104:0/3294486925 shutdown_connections 2023-07-14T11:54:34.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.711+0000 7f8511fe0700 1 --2- 172.21.15.104:0/3294486925 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f84f80662f0 0x7f84f80687a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.711+0000 7f8511fe0700 1 --2- 172.21.15.104:0/3294486925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f850c155ee0 0x7f850c144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.711+0000 7f8511fe0700 1 --2- 172.21.15.104:0/3294486925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f850c071570 0x7f850c144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.711+0000 7f8511fe0700 1 --2- 172.21.15.104:0/3294486925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f850c070ab0 0x7f850c153ab0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:34.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.711+0000 7f8511fe0700 1 -- 172.21.15.104:0/3294486925 >> 172.21.15.104:0/3294486925 conn(0x7f850c0cad30 msgr2=0x7f850c1556d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:34.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.711+0000 7f8511fe0700 1 -- 172.21.15.104:0/3294486925 shutdown_connections 2023-07-14T11:54:34.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:34.711+0000 7f8511fe0700 1 -- 172.21.15.104:0/3294486925 wait complete. 2023-07-14T11:54:34.701 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:54:34.719 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi104.front.sepia.ceph.com... 2023-07-14T11:54:34.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:34.719 DEBUG:teuthology.orchestra.run.smithi104:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-07-14T11:54:34.754 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:54:34.754 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:54:34.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:34.755 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-14T11:54:34.819 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:54:34.819 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:54:34.838 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:54:34.838 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:54:34.902 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:35.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.095+0000 7f353f59e700 1 Processor -- start 2023-07-14T11:54:35.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.095+0000 7f353f59e700 1 -- start start 2023-07-14T11:54:35.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353f59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f35381599d0 0x7f3538071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353f59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f353815ccf0 0x7f35380c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353f59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f35380caa20 0x7f35380cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353f59e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35380cde10 con 0x7f35381599d0 2023-07-14T11:54:35.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353f59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35380cb3c0 con 0x7f35380caa20 2023-07-14T11:54:35.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353f59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35380c8bf0 con 0x7f353815ccf0 2023-07-14T11:54:35.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353e59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f35381599d0 0x7f3538071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353e59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f35381599d0 0x7f3538071000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43912/0 (socket says 172.21.15.104:43912) 2023-07-14T11:54:35.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353e59c700 1 -- 172.21.15.104:0/3434212769 learned_addr learned my addr 172.21.15.104:0/3434212769 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:35.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353ed9d700 1 --2- 172.21.15.104:0/3434212769 >> v2:172.21.15.121:3300/0 conn(0x7f35380caa20 0x7f35380cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353dd9b700 1 --2- 172.21.15.104:0/3434212769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f353815ccf0 0x7f35380c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353e59c700 1 -- 172.21.15.104:0/3434212769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f353815ccf0 msgr2=0x7f35380c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:35.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353e59c700 1 --2- 172.21.15.104:0/3434212769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f353815ccf0 0x7f35380c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353e59c700 1 -- 172.21.15.104:0/3434212769 >> v2:172.21.15.121:3300/0 conn(0x7f35380caa20 msgr2=0x7f35380cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:35.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353e59c700 1 --2- 172.21.15.104:0/3434212769 >> v2:172.21.15.121:3300/0 conn(0x7f35380caa20 0x7f35380cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353e59c700 1 -- 172.21.15.104:0/3434212769 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35380cd520 con 0x7f35381599d0 2023-07-14T11:54:35.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353ed9d700 1 --2- 172.21.15.104:0/3434212769 >> v2:172.21.15.121:3300/0 conn(0x7f35380caa20 0x7f35380cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:35.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353e59c700 1 --2- 172.21.15.104:0/3434212769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35381599d0 0x7f3538071000 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f35380720b0 tx=0x7f352c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e9e94dbe02bed7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:35.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353d59a700 1 -- 172.21.15.104:0/3434212769 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c00b8a0 con 0x7f35381599d0 2023-07-14T11:54:35.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353d59a700 1 -- 172.21.15.104:0/3434212769 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352c00ba00 con 0x7f35381599d0 2023-07-14T11:54:35.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353d59a700 1 -- 172.21.15.104:0/3434212769 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c00bd90 con 0x7f35381599d0 2023-07-14T11:54:35.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353f59e700 1 -- 172.21.15.104:0/3434212769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35381599d0 msgr2=0x7f3538071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353f59e700 1 --2- 172.21.15.104:0/3434212769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35381599d0 0x7f3538071000 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f35380720b0 tx=0x7f352c009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353f59e700 1 -- 172.21.15.104:0/3434212769 shutdown_connections 2023-07-14T11:54:35.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353f59e700 1 --2- 172.21.15.104:0/3434212769 >> v2:172.21.15.121:3300/0 conn(0x7f35380caa20 0x7f35380cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353f59e700 1 --2- 172.21.15.104:0/3434212769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f353815ccf0 0x7f35380c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353f59e700 1 --2- 172.21.15.104:0/3434212769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35381599d0 0x7f3538071000 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353f59e700 1 -- 172.21.15.104:0/3434212769 >> 172.21.15.104:0/3434212769 conn(0x7f353806e960 msgr2=0x7f353806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:35.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353f59e700 1 -- 172.21.15.104:0/3434212769 shutdown_connections 2023-07-14T11:54:35.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.099+0000 7f353f59e700 1 -- 172.21.15.104:0/3434212769 wait complete. 2023-07-14T11:54:35.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.103+0000 7f353f59e700 1 Processor -- start 2023-07-14T11:54:35.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.103+0000 7f353f59e700 1 -- start start 2023-07-14T11:54:35.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35380caa20 0x7f35381f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353f59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35381599d0 0x7f35381f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353f59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f353815ccf0 0x7f353815d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353f59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35381f5cc0 con 0x7f35381599d0 2023-07-14T11:54:35.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353f59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35381f2730 con 0x7f35380caa20 2023-07-14T11:54:35.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353f59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35381f1c30 con 0x7f353815ccf0 2023-07-14T11:54:35.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35381599d0 0x7f35381f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35381599d0 0x7f35381f16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43924/0 (socket says 172.21.15.104:43924) 2023-07-14T11:54:35.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353dd9b700 1 -- 172.21.15.104:0/3210238253 learned_addr learned my addr 172.21.15.104:0/3210238253 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:35.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353ed9d700 1 --2- 172.21.15.104:0/3210238253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f353815ccf0 0x7f353815d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353e59c700 1 --2- 172.21.15.104:0/3210238253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35380caa20 0x7f35381f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353dd9b700 1 -- 172.21.15.104:0/3210238253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f353815ccf0 msgr2=0x7f353815d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353dd9b700 1 --2- 172.21.15.104:0/3210238253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f353815ccf0 0x7f353815d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353dd9b700 1 -- 172.21.15.104:0/3210238253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35380caa20 msgr2=0x7f35381f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353dd9b700 1 --2- 172.21.15.104:0/3210238253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35380caa20 0x7f35381f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353dd9b700 1 -- 172.21.15.104:0/3210238253 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c009570 con 0x7f35381599d0 2023-07-14T11:54:35.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353dd9b700 1 --2- 172.21.15.104:0/3210238253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35381599d0 0x7f35381f16f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f353000e810 tx=0x7f353000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:35.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f352affd700 1 -- 172.21.15.104:0/3210238253 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3530013610 con 0x7f35381599d0 2023-07-14T11:54:35.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f352affd700 1 -- 172.21.15.104:0/3210238253 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3530013770 con 0x7f35381599d0 2023-07-14T11:54:35.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353f59e700 1 -- 172.21.15.104:0/3210238253 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35381f1e90 con 0x7f35381599d0 2023-07-14T11:54:35.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f352affd700 1 -- 172.21.15.104:0/3210238253 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3530013b20 con 0x7f35381599d0 2023-07-14T11:54:35.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.107+0000 7f353f59e700 1 -- 172.21.15.104:0/3210238253 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35381f2450 con 0x7f35381599d0 2023-07-14T11:54:35.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.111+0000 7f353f59e700 1 -- 172.21.15.104:0/3210238253 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f353804e500 con 0x7f35381599d0 2023-07-14T11:54:35.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.111+0000 7f352affd700 1 -- 172.21.15.104:0/3210238253 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3530013c80 con 0x7f35381599d0 2023-07-14T11:54:35.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.111+0000 7f352affd700 1 --2- 172.21.15.104:0/3210238253 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f351c0662f0 0x7f351c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.111+0000 7f352affd700 1 -- 172.21.15.104:0/3210238253 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3530083e60 con 0x7f35381599d0 2023-07-14T11:54:35.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.115+0000 7f353e59c700 1 --2- 172.21.15.104:0/3210238253 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f351c0662f0 0x7f351c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.115+0000 7f353e59c700 1 --2- 172.21.15.104:0/3210238253 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f351c0662f0 0x7f351c0687a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f352c000c00 tx=0x7f352c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:35.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.119+0000 7f352affd700 1 -- 172.21.15.104:0/3210238253 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f3530050f20 con 0x7f35381599d0 2023-07-14T11:54:35.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.287+0000 7f353f59e700 1 -- 172.21.15.104:0/3210238253 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3538070de0 con 0x7f35381599d0 2023-07-14T11:54:35.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.291+0000 7f352affd700 1 -- 172.21.15.104:0/3210238253 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20251 (secure 0 0 0) 0x7f3530054540 con 0x7f35381599d0 2023-07-14T11:54:35.278 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:35.278 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":30,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:55.601783+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:35.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.295+0000 7f353f59e700 1 -- 172.21.15.104:0/3210238253 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f351c0662f0 msgr2=0x7f351c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.295+0000 7f353f59e700 1 --2- 172.21.15.104:0/3210238253 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f351c0662f0 0x7f351c0687a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f352c000c00 tx=0x7f352c008030 comp rx=0 tx=0).stop 2023-07-14T11:54:35.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.295+0000 7f353f59e700 1 -- 172.21.15.104:0/3210238253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35381599d0 msgr2=0x7f35381f16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.295+0000 7f353f59e700 1 --2- 172.21.15.104:0/3210238253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35381599d0 0x7f35381f16f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f353000e810 tx=0x7f353000ebd0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.295+0000 7f353f59e700 1 -- 172.21.15.104:0/3210238253 shutdown_connections 2023-07-14T11:54:35.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.295+0000 7f353f59e700 1 --2- 172.21.15.104:0/3210238253 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f351c0662f0 0x7f351c0687a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.295+0000 7f353f59e700 1 --2- 172.21.15.104:0/3210238253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35381599d0 0x7f35381f16f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.295+0000 7f353f59e700 1 --2- 172.21.15.104:0/3210238253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f353815ccf0 0x7f353815d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.295+0000 7f353f59e700 1 --2- 172.21.15.104:0/3210238253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35380caa20 0x7f35381f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.295+0000 7f353f59e700 1 -- 172.21.15.104:0/3210238253 >> 172.21.15.104:0/3210238253 conn(0x7f353806e960 msgr2=0x7f353806c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:35.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.295+0000 7f353f59e700 1 -- 172.21.15.104:0/3210238253 shutdown_connections 2023-07-14T11:54:35.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.295+0000 7f353f59e700 1 -- 172.21.15.104:0/3210238253 wait complete. 2023-07-14T11:54:35.302 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:54:35.303 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:54:35.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.491+0000 7f47022d8700 1 Processor -- start 2023-07-14T11:54:35.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.495+0000 7f47022d8700 1 -- start start 2023-07-14T11:54:35.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.495+0000 7f47022d8700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f46fc160000 0x7f46fc06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.495+0000 7f47022d8700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46fc0cd7f0 0x7f46fc0c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.495+0000 7f47022d8700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46fc0cdfb0 con 0x7f46fc1603d0 2023-07-14T11:54:35.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.495+0000 7f47022d8700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46fc070e40 con 0x7f46fc160000 2023-07-14T11:54:35.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.495+0000 7f47022d8700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0cdcd0 con 0x7f46fc0cd7f0 2023-07-14T11:54:35.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.495+0000 7f47012d6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f46fc160000 0x7f46fc06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.495+0000 7f47012d6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f46fc160000 0x7f46fc06dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43932/0 (socket says 172.21.15.104:43932) 2023-07-14T11:54:35.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.495+0000 7f47012d6700 1 -- 172.21.15.104:0/3727354018 learned_addr learned my addr 172.21.15.104:0/3727354018 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:35.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.495+0000 7f4701ad7700 1 --2- 172.21.15.104:0/3727354018 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46fc0cd7f0 0x7f46fc0c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.495+0000 7f47012d6700 1 -- 172.21.15.104:0/3727354018 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46fc0cd7f0 msgr2=0x7f46fc0c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:35.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.495+0000 7f47012d6700 1 --2- 172.21.15.104:0/3727354018 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46fc0cd7f0 0x7f46fc0c0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.495+0000 7f47012d6700 1 -- 172.21.15.104:0/3727354018 >> v1:172.21.15.121:6789/0 conn(0x7f46fc1603d0 legacy=0x7f46fc0c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.495+0000 7f47012d6700 1 -- 172.21.15.104:0/3727354018 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc070780 con 0x7f46fc160000 2023-07-14T11:54:35.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.495+0000 7f47012d6700 1 --2- 172.21.15.104:0/3727354018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46fc160000 0x7f46fc06dd10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009910 tx=0x7f46f8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cd7a971d2203df server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:35.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.499+0000 7f46f3fff700 1 -- 172.21.15.104:0/3727354018 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8011830 con 0x7f46fc160000 2023-07-14T11:54:35.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.499+0000 7f46f3fff700 1 -- 172.21.15.104:0/3727354018 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f8011990 con 0x7f46fc160000 2023-07-14T11:54:35.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.499+0000 7f46f3fff700 1 -- 172.21.15.104:0/3727354018 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8011d40 con 0x7f46fc160000 2023-07-14T11:54:35.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.499+0000 7f47022d8700 1 -- 172.21.15.104:0/3727354018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46fc160000 msgr2=0x7f46fc06dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.499+0000 7f47022d8700 1 --2- 172.21.15.104:0/3727354018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46fc160000 0x7f46fc06dd10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009910 tx=0x7f46f8009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:35.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.499+0000 7f47022d8700 1 -- 172.21.15.104:0/3727354018 shutdown_connections 2023-07-14T11:54:35.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.499+0000 7f47022d8700 1 --2- 172.21.15.104:0/3727354018 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46fc0cd7f0 0x7f46fc0c0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.499+0000 7f47022d8700 1 --2- 172.21.15.104:0/3727354018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46fc160000 0x7f46fc06dd10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.499+0000 7f47022d8700 1 -- 172.21.15.104:0/3727354018 >> 172.21.15.104:0/3727354018 conn(0x7f46fc0afc60 msgr2=0x7f46fc0b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:35.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.499+0000 7f47022d8700 1 -- 172.21.15.104:0/3727354018 shutdown_connections 2023-07-14T11:54:35.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.499+0000 7f47022d8700 1 -- 172.21.15.104:0/3727354018 wait complete. 2023-07-14T11:54:35.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47022d8700 1 Processor -- start 2023-07-14T11:54:35.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47022d8700 1 -- start start 2023-07-14T11:54:35.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47022d8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46fc0cd7f0 0x7f46fc157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47022d8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46fc160000 0x7f46fc0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47022d8700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46fc1603d0 0x7f46fc1613d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47022d8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc15b040 con 0x7f46fc0cd7f0 2023-07-14T11:54:35.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47022d8700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0c7fa0 con 0x7f46fc160000 2023-07-14T11:54:35.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47022d8700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc157540 con 0x7f46fc1603d0 2023-07-14T11:54:35.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47012d6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46fc0cd7f0 0x7f46fc157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47012d6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46fc0cd7f0 0x7f46fc157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43942/0 (socket says 172.21.15.104:43942) 2023-07-14T11:54:35.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47012d6700 1 -- 172.21.15.104:0/4071499971 learned_addr learned my addr 172.21.15.104:0/4071499971 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:35.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f4701ad7700 1 --2- 172.21.15.104:0/4071499971 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46fc1603d0 0x7f46fc1613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f4700ad5700 1 --2- 172.21.15.104:0/4071499971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46fc160000 0x7f46fc0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47012d6700 1 -- 172.21.15.104:0/4071499971 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46fc1603d0 msgr2=0x7f46fc1613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47012d6700 1 --2- 172.21.15.104:0/4071499971 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46fc1603d0 0x7f46fc1613d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47012d6700 1 -- 172.21.15.104:0/4071499971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46fc160000 msgr2=0x7f46fc0c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47012d6700 1 --2- 172.21.15.104:0/4071499971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46fc160000 0x7f46fc0c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47012d6700 1 -- 172.21.15.104:0/4071499971 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f8009570 con 0x7f46fc0cd7f0 2023-07-14T11:54:35.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f4700ad5700 1 --2- 172.21.15.104:0/4071499971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46fc160000 0x7f46fc0c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:35.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.503+0000 7f47012d6700 1 --2- 172.21.15.104:0/4071499971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46fc0cd7f0 0x7f46fc157000 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f46f8000c00 tx=0x7f46f8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:35.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.507+0000 7f46f1ffb700 1 -- 172.21.15.104:0/4071499971 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8004610 con 0x7f46fc0cd7f0 2023-07-14T11:54:35.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.507+0000 7f46f1ffb700 1 -- 172.21.15.104:0/4071499971 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f80036c0 con 0x7f46fc0cd7f0 2023-07-14T11:54:35.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.507+0000 7f46f1ffb700 1 -- 172.21.15.104:0/4071499971 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8003a50 con 0x7f46fc0cd7f0 2023-07-14T11:54:35.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.507+0000 7f47022d8700 1 -- 172.21.15.104:0/4071499971 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc0b3ce0 con 0x7f46fc0cd7f0 2023-07-14T11:54:35.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.507+0000 7f47022d8700 1 -- 172.21.15.104:0/4071499971 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc1608f0 con 0x7f46fc0cd7f0 2023-07-14T11:54:35.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.507+0000 7f47022d8700 1 -- 172.21.15.104:0/4071499971 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46fc04e500 con 0x7f46fc0cd7f0 2023-07-14T11:54:35.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.511+0000 7f46f1ffb700 1 -- 172.21.15.104:0/4071499971 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f46f8003bb0 con 0x7f46fc0cd7f0 2023-07-14T11:54:35.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.511+0000 7f46f1ffb700 1 --2- 172.21.15.104:0/4071499971 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46e00662f0 0x7f46e00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.511+0000 7f46f1ffb700 1 -- 172.21.15.104:0/4071499971 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f46f8021030 con 0x7f46fc0cd7f0 2023-07-14T11:54:35.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.511+0000 7f4700ad5700 1 --2- 172.21.15.104:0/4071499971 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46e00662f0 0x7f46e00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.511+0000 7f4700ad5700 1 --2- 172.21.15.104:0/4071499971 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46e00662f0 0x7f46e00687a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f46fc15e050 tx=0x7f46ec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:35.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.515+0000 7f46f1ffb700 1 -- 172.21.15.104:0/4071499971 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f46f8056430 con 0x7f46fc0cd7f0 2023-07-14T11:54:35.680 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.695+0000 7f47022d8700 1 -- 172.21.15.104:0/4071499971 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f46fc0b40d0 con 0x7f46fc0cd7f0 2023-07-14T11:54:35.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.695+0000 7f46f1ffb700 1 -- 172.21.15.104:0/4071499971 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f46f8003e60 con 0x7f46fc0cd7f0 2023-07-14T11:54:35.681 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:54:35.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.699+0000 7f47022d8700 1 -- 172.21.15.104:0/4071499971 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46e00662f0 msgr2=0x7f46e00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.699+0000 7f47022d8700 1 --2- 172.21.15.104:0/4071499971 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46e00662f0 0x7f46e00687a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f46fc15e050 tx=0x7f46ec009040 comp rx=0 tx=0).stop 2023-07-14T11:54:35.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.699+0000 7f47022d8700 1 -- 172.21.15.104:0/4071499971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46fc0cd7f0 msgr2=0x7f46fc157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.699+0000 7f47022d8700 1 --2- 172.21.15.104:0/4071499971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46fc0cd7f0 0x7f46fc157000 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f46f8000c00 tx=0x7f46f8004770 comp rx=0 tx=0).stop 2023-07-14T11:54:35.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.699+0000 7f47022d8700 1 -- 172.21.15.104:0/4071499971 shutdown_connections 2023-07-14T11:54:35.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.699+0000 7f47022d8700 1 --2- 172.21.15.104:0/4071499971 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f46e00662f0 0x7f46e00687a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.699+0000 7f47022d8700 1 --2- 172.21.15.104:0/4071499971 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46fc1603d0 0x7f46fc1613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.699+0000 7f47022d8700 1 --2- 172.21.15.104:0/4071499971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46fc160000 0x7f46fc0c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.699+0000 7f47022d8700 1 --2- 172.21.15.104:0/4071499971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46fc0cd7f0 0x7f46fc157000 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.699+0000 7f47022d8700 1 -- 172.21.15.104:0/4071499971 >> 172.21.15.104:0/4071499971 conn(0x7f46fc0afc60 msgr2=0x7f46fc0c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:35.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.699+0000 7f47022d8700 1 -- 172.21.15.104:0/4071499971 shutdown_connections 2023-07-14T11:54:35.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.699+0000 7f47022d8700 1 -- 172.21.15.104:0/4071499971 wait complete. 2023-07-14T11:54:35.708 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:54:35.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.899+0000 7f1e54e03700 1 Processor -- start 2023-07-14T11:54:35.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.899+0000 7f1e54e03700 1 -- start start 2023-07-14T11:54:35.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.903+0000 7f1e54e03700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1e500b0ee0 0x7f1e500ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.903+0000 7f1e54e03700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e500d07e0 0x7f1e500af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.903+0000 7f1e54e03700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e500cd4e0 con 0x7f1e500cd090 2023-07-14T11:54:35.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.903+0000 7f1e54e03700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e500d0c60 con 0x7f1e500b0ee0 2023-07-14T11:54:35.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.903+0000 7f1e54e03700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e500d1a60 con 0x7f1e500d07e0 2023-07-14T11:54:35.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.903+0000 7f1e4ffff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e500d07e0 0x7f1e500af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.903+0000 7f1e4ffff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e500d07e0 0x7f1e500af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:38876/0 (socket says 172.21.15.104:38876) 2023-07-14T11:54:35.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.903+0000 7f1e4ffff700 1 -- 172.21.15.104:0/524094150 learned_addr learned my addr 172.21.15.104:0/524094150 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:35.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.903+0000 7f1e4f7fe700 1 --2- 172.21.15.104:0/524094150 >> v2:172.21.15.121:3300/0 conn(0x7f1e500b0ee0 0x7f1e500ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.903+0000 7f1e4effd700 1 --1- 172.21.15.104:0/524094150 >> v1:172.21.15.121:6789/0 conn(0x7f1e500cd090 0x7f1e500d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:41636/0 (socket says 172.21.15.104:41636) 2023-07-14T11:54:35.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.903+0000 7f1e4ffff700 1 -- 172.21.15.104:0/524094150 >> v2:172.21.15.121:3300/0 conn(0x7f1e500b0ee0 msgr2=0x7f1e500ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:35.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.903+0000 7f1e4ffff700 1 --2- 172.21.15.104:0/524094150 >> v2:172.21.15.121:3300/0 conn(0x7f1e500b0ee0 0x7f1e500ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.903+0000 7f1e4ffff700 1 -- 172.21.15.104:0/524094150 >> v1:172.21.15.121:6789/0 conn(0x7f1e500cd090 legacy=0x7f1e500d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.903+0000 7f1e4ffff700 1 -- 172.21.15.104:0/524094150 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e500d1ef0 con 0x7f1e500d07e0 2023-07-14T11:54:35.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.907+0000 7f1e4ffff700 1 --2- 172.21.15.104:0/524094150 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e500d07e0 0x7f1e500af0c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009e50 tx=0x7f1e4800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=128af16575b10c83 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:35.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.907+0000 7f1e4e7fc700 1 -- 172.21.15.104:0/524094150 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4800e070 con 0x7f1e500d07e0 2023-07-14T11:54:35.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.907+0000 7f1e4e7fc700 1 -- 172.21.15.104:0/524094150 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4800b910 con 0x7f1e500d07e0 2023-07-14T11:54:35.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.907+0000 7f1e4e7fc700 1 -- 172.21.15.104:0/524094150 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4800bcc0 con 0x7f1e500d07e0 2023-07-14T11:54:35.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.907+0000 7f1e54e03700 1 -- 172.21.15.104:0/524094150 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e500d07e0 msgr2=0x7f1e500af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.907+0000 7f1e54e03700 1 --2- 172.21.15.104:0/524094150 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e500d07e0 0x7f1e500af0c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009e50 tx=0x7f1e4800b330 comp rx=0 tx=0).stop 2023-07-14T11:54:35.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.907+0000 7f1e54e03700 1 -- 172.21.15.104:0/524094150 shutdown_connections 2023-07-14T11:54:35.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.907+0000 7f1e54e03700 1 --2- 172.21.15.104:0/524094150 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e500d07e0 0x7f1e500af0c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.907+0000 7f1e54e03700 1 --2- 172.21.15.104:0/524094150 >> v2:172.21.15.121:3300/0 conn(0x7f1e500b0ee0 0x7f1e500ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.907+0000 7f1e54e03700 1 -- 172.21.15.104:0/524094150 >> 172.21.15.104:0/524094150 conn(0x7f1e50060c40 msgr2=0x7f1e5009b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:35.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.907+0000 7f1e54e03700 1 -- 172.21.15.104:0/524094150 shutdown_connections 2023-07-14T11:54:35.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.907+0000 7f1e54e03700 1 -- 172.21.15.104:0/524094150 wait complete. 2023-07-14T11:54:35.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.911+0000 7f1e54e03700 1 Processor -- start 2023-07-14T11:54:35.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.911+0000 7f1e54e03700 1 -- start start 2023-07-14T11:54:35.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e54e03700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e500b0ee0 0x7f1e500c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e54e03700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e500cd090 0x7f1e5015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e54e03700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e500d07e0 0x7f1e501f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e54e03700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5015b360 con 0x7f1e500cd090 2023-07-14T11:54:35.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e54e03700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e500cae30 con 0x7f1e500d07e0 2023-07-14T11:54:35.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e54e03700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e501f59a0 con 0x7f1e500b0ee0 2023-07-14T11:54:35.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e4effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e500cd090 0x7f1e5015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e4effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e500cd090 0x7f1e5015ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43954/0 (socket says 172.21.15.104:43954) 2023-07-14T11:54:35.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e4effd700 1 -- 172.21.15.104:0/629761320 learned_addr learned my addr 172.21.15.104:0/629761320 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:35.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e4f7fe700 1 --2- 172.21.15.104:0/629761320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e500b0ee0 0x7f1e500c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e4ffff700 1 --2- 172.21.15.104:0/629761320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e500d07e0 0x7f1e501f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e4effd700 1 -- 172.21.15.104:0/629761320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e500b0ee0 msgr2=0x7f1e500c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e4effd700 1 --2- 172.21.15.104:0/629761320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e500b0ee0 0x7f1e500c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e4effd700 1 -- 172.21.15.104:0/629761320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e500d07e0 msgr2=0x7f1e501f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:35.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e4effd700 1 --2- 172.21.15.104:0/629761320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e500d07e0 0x7f1e501f5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:35.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e4effd700 1 -- 172.21.15.104:0/629761320 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e48009b30 con 0x7f1e500cd090 2023-07-14T11:54:35.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e4effd700 1 --2- 172.21.15.104:0/629761320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e500cd090 0x7f1e5015ae20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400c730 tx=0x7f1e4400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:35.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e37fff700 1 -- 172.21.15.104:0/629761320 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44005cd0 con 0x7f1e500cd090 2023-07-14T11:54:35.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e37fff700 1 -- 172.21.15.104:0/629761320 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e44005e30 con 0x7f1e500cd090 2023-07-14T11:54:35.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e54e03700 1 -- 172.21.15.104:0/629761320 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e500c9bb0 con 0x7f1e500cd090 2023-07-14T11:54:35.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e37fff700 1 -- 172.21.15.104:0/629761320 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e440072d0 con 0x7f1e500cd090 2023-07-14T11:54:35.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.915+0000 7f1e54e03700 1 -- 172.21.15.104:0/629761320 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e500ca170 con 0x7f1e500cd090 2023-07-14T11:54:35.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.919+0000 7f1e54e03700 1 -- 172.21.15.104:0/629761320 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e500cafe0 con 0x7f1e500cd090 2023-07-14T11:54:35.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.919+0000 7f1e37fff700 1 -- 172.21.15.104:0/629761320 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1e44007430 con 0x7f1e500cd090 2023-07-14T11:54:35.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.919+0000 7f1e37fff700 1 --2- 172.21.15.104:0/629761320 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1e3c0662f0 0x7f1e3c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:35.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.919+0000 7f1e37fff700 1 -- 172.21.15.104:0/629761320 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1e44083120 con 0x7f1e500cd090 2023-07-14T11:54:35.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.923+0000 7f1e4f7fe700 1 --2- 172.21.15.104:0/629761320 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1e3c0662f0 0x7f1e3c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:35.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.923+0000 7f1e4f7fe700 1 --2- 172.21.15.104:0/629761320 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1e3c0662f0 0x7f1e3c0687a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009b50 tx=0x7f1e40009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:35.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:35.927+0000 7f1e37fff700 1 -- 172.21.15.104:0/629761320 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f1e500cafe0 con 0x7f1e500cd090 2023-07-14T11:54:36.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.099+0000 7f1e54e03700 1 -- 172.21.15.104:0/629761320 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1e500cd4e0 con 0x7f1e500cd090 2023-07-14T11:54:36.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.099+0000 7f1e37fff700 1 -- 172.21.15.104:0/629761320 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1e44017020 con 0x7f1e500cd090 2023-07-14T11:54:36.086 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:54:36.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.103+0000 7f1e54e03700 1 -- 172.21.15.104:0/629761320 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1e3c0662f0 msgr2=0x7f1e3c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.103+0000 7f1e54e03700 1 --2- 172.21.15.104:0/629761320 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1e3c0662f0 0x7f1e3c0687a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009b50 tx=0x7f1e40009240 comp rx=0 tx=0).stop 2023-07-14T11:54:36.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.103+0000 7f1e54e03700 1 -- 172.21.15.104:0/629761320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e500cd090 msgr2=0x7f1e5015ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.103+0000 7f1e54e03700 1 --2- 172.21.15.104:0/629761320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e500cd090 0x7f1e5015ae20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400c730 tx=0x7f1e4400ca40 comp rx=0 tx=0).stop 2023-07-14T11:54:36.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.103+0000 7f1e54e03700 1 -- 172.21.15.104:0/629761320 shutdown_connections 2023-07-14T11:54:36.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.103+0000 7f1e54e03700 1 --2- 172.21.15.104:0/629761320 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1e3c0662f0 0x7f1e3c0687a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.103+0000 7f1e54e03700 1 --2- 172.21.15.104:0/629761320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e500cd090 0x7f1e5015ae20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.103+0000 7f1e54e03700 1 --2- 172.21.15.104:0/629761320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e500d07e0 0x7f1e501f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.103+0000 7f1e54e03700 1 --2- 172.21.15.104:0/629761320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e500b0ee0 0x7f1e500c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.103+0000 7f1e54e03700 1 -- 172.21.15.104:0/629761320 >> 172.21.15.104:0/629761320 conn(0x7f1e50060c40 msgr2=0x7f1e5015f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:36.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.103+0000 7f1e54e03700 1 -- 172.21.15.104:0/629761320 shutdown_connections 2023-07-14T11:54:36.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.103+0000 7f1e54e03700 1 -- 172.21.15.104:0/629761320 wait complete. 2023-07-14T11:54:36.111 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:54:36.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.295+0000 7f62a5e2f700 1 Processor -- start 2023-07-14T11:54:36.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.295+0000 7f62a5e2f700 1 -- start start 2023-07-14T11:54:36.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.299+0000 7f62a5e2f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a00cb2d0 0x7f62a00b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.299+0000 7f62a5e2f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f62a00cb6a0 0x7f62a006cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.299+0000 7f62a5e2f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f62a006d040 0x7f62a006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.299+0000 7f62a5e2f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a006f130 con 0x7f62a006d040 2023-07-14T11:54:36.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.299+0000 7f62a5e2f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a006ee30 con 0x7f62a00cb6a0 2023-07-14T11:54:36.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.299+0000 7f62a5e2f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a00b3010 con 0x7f62a00cb2d0 2023-07-14T11:54:36.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.299+0000 7f62a562e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f62a006d040 0x7f62a006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.299+0000 7f62a562e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f62a006d040 0x7f62a006f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43970/0 (socket says 172.21.15.104:43970) 2023-07-14T11:54:36.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.299+0000 7f62a562e700 1 -- 172.21.15.104:0/4158528809 learned_addr learned my addr 172.21.15.104:0/4158528809 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:36.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.299+0000 7f62a4e2d700 1 --2- 172.21.15.104:0/4158528809 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a00cb2d0 0x7f62a00b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.299+0000 7f629ffff700 1 --2- 172.21.15.104:0/4158528809 >> v2:172.21.15.121:3300/0 conn(0x7f62a00cb6a0 0x7f62a006cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f62a562e700 1 -- 172.21.15.104:0/4158528809 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a00cb2d0 msgr2=0x7f62a00b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:36.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f62a562e700 1 --2- 172.21.15.104:0/4158528809 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a00cb2d0 0x7f62a00b2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f62a562e700 1 -- 172.21.15.104:0/4158528809 >> v2:172.21.15.121:3300/0 conn(0x7f62a00cb6a0 msgr2=0x7f62a006cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:36.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f62a562e700 1 --2- 172.21.15.104:0/4158528809 >> v2:172.21.15.121:3300/0 conn(0x7f62a00cb6a0 0x7f62a006cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f62a562e700 1 -- 172.21.15.104:0/4158528809 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a006d4e0 con 0x7f62a006d040 2023-07-14T11:54:36.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f62a562e700 1 --2- 172.21.15.104:0/4158528809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a006d040 0x7f62a006f5c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6298009e50 tx=0x7f629800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d292a545e91d644 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:36.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f629f7fe700 1 -- 172.21.15.104:0/4158528809 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629800e070 con 0x7f62a006d040 2023-07-14T11:54:36.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f629f7fe700 1 -- 172.21.15.104:0/4158528809 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629800b8d0 con 0x7f62a006d040 2023-07-14T11:54:36.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f629f7fe700 1 -- 172.21.15.104:0/4158528809 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629800bc80 con 0x7f62a006d040 2023-07-14T11:54:36.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f62a5e2f700 1 -- 172.21.15.104:0/4158528809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a006d040 msgr2=0x7f62a006f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f62a5e2f700 1 --2- 172.21.15.104:0/4158528809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a006d040 0x7f62a006f5c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6298009e50 tx=0x7f629800b330 comp rx=0 tx=0).stop 2023-07-14T11:54:36.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f62a5e2f700 1 -- 172.21.15.104:0/4158528809 shutdown_connections 2023-07-14T11:54:36.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f62a5e2f700 1 --2- 172.21.15.104:0/4158528809 >> v2:172.21.15.121:3300/0 conn(0x7f62a00cb6a0 0x7f62a006cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f62a5e2f700 1 --2- 172.21.15.104:0/4158528809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a006d040 0x7f62a006f5c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f62a5e2f700 1 --2- 172.21.15.104:0/4158528809 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a00cb2d0 0x7f62a00b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f62a5e2f700 1 -- 172.21.15.104:0/4158528809 >> 172.21.15.104:0/4158528809 conn(0x7f62a00b04e0 msgr2=0x7f62a00b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:36.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f62a5e2f700 1 -- 172.21.15.104:0/4158528809 shutdown_connections 2023-07-14T11:54:36.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.303+0000 7f62a5e2f700 1 -- 172.21.15.104:0/4158528809 wait complete. 2023-07-14T11:54:36.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a5e2f700 1 Processor -- start 2023-07-14T11:54:36.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a5e2f700 1 -- start start 2023-07-14T11:54:36.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a5e2f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a006d040 0x7f62a014ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a5e2f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a00cb2d0 0x7f62a00ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a5e2f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a00cb6a0 0x7f62a00c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a5e2f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a00adf00 con 0x7f62a006d040 2023-07-14T11:54:36.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a5e2f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a00ae4a0 con 0x7f62a00cb6a0 2023-07-14T11:54:36.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a5e2f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a014d3d0 con 0x7f62a00cb2d0 2023-07-14T11:54:36.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a4e2d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a006d040 0x7f62a014ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a4e2d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a006d040 0x7f62a014ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43984/0 (socket says 172.21.15.104:43984) 2023-07-14T11:54:36.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a4e2d700 1 -- 172.21.15.104:0/7083007 learned_addr learned my addr 172.21.15.104:0/7083007 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:36.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a562e700 1 --2- 172.21.15.104:0/7083007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a00cb6a0 0x7f62a00c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f629ffff700 1 --2- 172.21.15.104:0/7083007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a00cb2d0 0x7f62a00ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a4e2d700 1 -- 172.21.15.104:0/7083007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a00cb2d0 msgr2=0x7f62a00ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a4e2d700 1 --2- 172.21.15.104:0/7083007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a00cb2d0 0x7f62a00ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a4e2d700 1 -- 172.21.15.104:0/7083007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a00cb6a0 msgr2=0x7f62a00c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a4e2d700 1 --2- 172.21.15.104:0/7083007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a00cb6a0 0x7f62a00c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f62a4e2d700 1 -- 172.21.15.104:0/7083007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6298009b30 con 0x7f62a006d040 2023-07-14T11:54:36.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.307+0000 7f629ffff700 1 --2- 172.21.15.104:0/7083007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a00cb2d0 0x7f62a00ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:36.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.311+0000 7f62a4e2d700 1 --2- 172.21.15.104:0/7083007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a006d040 0x7f62a014ce90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f629400e940 tx=0x7f629400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:36.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.311+0000 7f629d7fa700 1 -- 172.21.15.104:0/7083007 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62940096a0 con 0x7f62a006d040 2023-07-14T11:54:36.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.311+0000 7f629d7fa700 1 -- 172.21.15.104:0/7083007 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629400c850 con 0x7f62a006d040 2023-07-14T11:54:36.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.311+0000 7f62a5e2f700 1 -- 172.21.15.104:0/7083007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a00ac410 con 0x7f62a006d040 2023-07-14T11:54:36.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.311+0000 7f629d7fa700 1 -- 172.21.15.104:0/7083007 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629400cbe0 con 0x7f62a006d040 2023-07-14T11:54:36.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.311+0000 7f62a5e2f700 1 -- 172.21.15.104:0/7083007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a00ae130 con 0x7f62a006d040 2023-07-14T11:54:36.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.311+0000 7f62a5e2f700 1 -- 172.21.15.104:0/7083007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a0073350 con 0x7f62a006d040 2023-07-14T11:54:36.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.315+0000 7f629d7fa700 1 -- 172.21.15.104:0/7083007 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f629400cd40 con 0x7f62a006d040 2023-07-14T11:54:36.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.315+0000 7f629d7fa700 1 --2- 172.21.15.104:0/7083007 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f62840662f0 0x7f62840687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.315+0000 7f629d7fa700 1 -- 172.21.15.104:0/7083007 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6294084130 con 0x7f62a006d040 2023-07-14T11:54:36.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.315+0000 7f629ffff700 1 --2- 172.21.15.104:0/7083007 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f62840662f0 0x7f62840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.315+0000 7f629ffff700 1 --2- 172.21.15.104:0/7083007 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f62840662f0 0x7f62840687a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f62a0150b20 tx=0x7f628c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:36.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.323+0000 7f629d7fa700 1 -- 172.21.15.104:0/7083007 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f6294054d00 con 0x7f62a006d040 2023-07-14T11:54:36.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.491+0000 7f62a5e2f700 1 -- 172.21.15.104:0/7083007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62a004e500 con 0x7f62a006d040 2023-07-14T11:54:36.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.495+0000 7f629d7fa700 1 -- 172.21.15.104:0/7083007 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6294054890 con 0x7f62a006d040 2023-07-14T11:54:36.480 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:36.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.499+0000 7f62a5e2f700 1 -- 172.21.15.104:0/7083007 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f62840662f0 msgr2=0x7f62840687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.499+0000 7f62a5e2f700 1 --2- 172.21.15.104:0/7083007 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f62840662f0 0x7f62840687a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f62a0150b20 tx=0x7f628c008040 comp rx=0 tx=0).stop 2023-07-14T11:54:36.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.499+0000 7f62a5e2f700 1 -- 172.21.15.104:0/7083007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a006d040 msgr2=0x7f62a014ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.499+0000 7f62a5e2f700 1 --2- 172.21.15.104:0/7083007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a006d040 0x7f62a014ce90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f629400e940 tx=0x7f629400ed00 comp rx=0 tx=0).stop 2023-07-14T11:54:36.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.499+0000 7f62a5e2f700 1 -- 172.21.15.104:0/7083007 shutdown_connections 2023-07-14T11:54:36.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.499+0000 7f62a5e2f700 1 --2- 172.21.15.104:0/7083007 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f62840662f0 0x7f62840687a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.499+0000 7f62a5e2f700 1 --2- 172.21.15.104:0/7083007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a00cb6a0 0x7f62a00c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.499+0000 7f62a5e2f700 1 --2- 172.21.15.104:0/7083007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a00cb2d0 0x7f62a00ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.499+0000 7f62a5e2f700 1 --2- 172.21.15.104:0/7083007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a006d040 0x7f62a014ce90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.499+0000 7f62a5e2f700 1 -- 172.21.15.104:0/7083007 >> 172.21.15.104:0/7083007 conn(0x7f62a00b04e0 msgr2=0x7f62a00c79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:36.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.499+0000 7f62a5e2f700 1 -- 172.21.15.104:0/7083007 shutdown_connections 2023-07-14T11:54:36.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.499+0000 7f62a5e2f700 1 -- 172.21.15.104:0/7083007 wait complete. 2023-07-14T11:54:36.504 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:54:36.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.699+0000 7f9fda9d9700 1 Processor -- start 2023-07-14T11:54:36.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.699+0000 7f9fda9d9700 1 -- start start 2023-07-14T11:54:36.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.699+0000 7f9fda9d9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9fd4160000 0x7f9fd406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.699+0000 7f9fda9d9700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd41603d0 0x7f9fd40c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.699+0000 7f9fda9d9700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd40cdfb0 con 0x7f9fd40cd7f0 2023-07-14T11:54:36.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.699+0000 7f9fda9d9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd4070e40 con 0x7f9fd4160000 2023-07-14T11:54:36.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.699+0000 7f9fda9d9700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd40cdcd0 con 0x7f9fd41603d0 2023-07-14T11:54:36.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fd99d7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9fd4160000 0x7f9fd406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fd91d6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd41603d0 0x7f9fd40c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fd99d7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9fd4160000 0x7f9fd406dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:37122/0 (socket says 172.21.15.104:37122) 2023-07-14T11:54:36.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fd99d7700 1 -- 172.21.15.104:0/3173315381 learned_addr learned my addr 172.21.15.104:0/3173315381 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:36.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fda1d8700 1 --1- 172.21.15.104:0/3173315381 >> v1:172.21.15.104:6789/0 conn(0x7f9fd40cd7f0 0x7f9fd40c0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:45684/0 (socket says 172.21.15.104:45684) 2023-07-14T11:54:36.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fd99d7700 1 -- 172.21.15.104:0/3173315381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd41603d0 msgr2=0x7f9fd40c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:36.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fd99d7700 1 --2- 172.21.15.104:0/3173315381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd41603d0 0x7f9fd40c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fd99d7700 1 -- 172.21.15.104:0/3173315381 >> v1:172.21.15.104:6789/0 conn(0x7f9fd40cd7f0 legacy=0x7f9fd40c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fd99d7700 1 -- 172.21.15.104:0/3173315381 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd4070780 con 0x7f9fd4160000 2023-07-14T11:54:36.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fd99d7700 1 --2- 172.21.15.104:0/3173315381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fd4160000 0x7f9fd406dd10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009910 tx=0x7f9fd0009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7c8c86bbe9cb176 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:36.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fd89d5700 1 -- 172.21.15.104:0/3173315381 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0011830 con 0x7f9fd4160000 2023-07-14T11:54:36.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fd89d5700 1 -- 172.21.15.104:0/3173315381 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd0011990 con 0x7f9fd4160000 2023-07-14T11:54:36.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fd89d5700 1 -- 172.21.15.104:0/3173315381 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0011d40 con 0x7f9fd4160000 2023-07-14T11:54:36.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fda9d9700 1 -- 172.21.15.104:0/3173315381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fd4160000 msgr2=0x7f9fd406dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fda9d9700 1 --2- 172.21.15.104:0/3173315381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fd4160000 0x7f9fd406dd10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009910 tx=0x7f9fd0009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:36.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fda9d9700 1 -- 172.21.15.104:0/3173315381 shutdown_connections 2023-07-14T11:54:36.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fda9d9700 1 --2- 172.21.15.104:0/3173315381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd41603d0 0x7f9fd40c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fda9d9700 1 --2- 172.21.15.104:0/3173315381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fd4160000 0x7f9fd406dd10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.703+0000 7f9fda9d9700 1 -- 172.21.15.104:0/3173315381 >> 172.21.15.104:0/3173315381 conn(0x7f9fd40afc60 msgr2=0x7f9fd40b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:36.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.707+0000 7f9fda9d9700 1 -- 172.21.15.104:0/3173315381 shutdown_connections 2023-07-14T11:54:36.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.707+0000 7f9fda9d9700 1 -- 172.21.15.104:0/3173315381 wait complete. 2023-07-14T11:54:36.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.707+0000 7f9fda9d9700 1 Processor -- start 2023-07-14T11:54:36.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.707+0000 7f9fda9d9700 1 -- start start 2023-07-14T11:54:36.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.707+0000 7f9fda9d9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fd40cd7f0 0x7f9fd4154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.707+0000 7f9fda9d9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd4160000 0x7f9fd40c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.707+0000 7f9fda9d9700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd41603d0 0x7f9fd40b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.707+0000 7f9fda9d9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd4154c90 con 0x7f9fd4160000 2023-07-14T11:54:36.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.707+0000 7f9fda9d9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd40b2020 con 0x7f9fd40cd7f0 2023-07-14T11:54:36.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.707+0000 7f9fda9d9700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd40b1410 con 0x7f9fd41603d0 2023-07-14T11:54:36.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fd91d6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd4160000 0x7f9fd40c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fd91d6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd4160000 0x7f9fd40c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43988/0 (socket says 172.21.15.104:43988) 2023-07-14T11:54:36.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fd91d6700 1 -- 172.21.15.104:0/2858099741 learned_addr learned my addr 172.21.15.104:0/2858099741 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:36.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fda1d8700 1 --2- 172.21.15.104:0/2858099741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd41603d0 0x7f9fd40b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fd99d7700 1 --2- 172.21.15.104:0/2858099741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fd40cd7f0 0x7f9fd4154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fd91d6700 1 -- 172.21.15.104:0/2858099741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd41603d0 msgr2=0x7f9fd40b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fd91d6700 1 --2- 172.21.15.104:0/2858099741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd41603d0 0x7f9fd40b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fd91d6700 1 -- 172.21.15.104:0/2858099741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fd40cd7f0 msgr2=0x7f9fd4154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fd91d6700 1 --2- 172.21.15.104:0/2858099741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fd40cd7f0 0x7f9fd4154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fd91d6700 1 -- 172.21.15.104:0/2858099741 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd0009570 con 0x7f9fd4160000 2023-07-14T11:54:36.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fd91d6700 1 --2- 172.21.15.104:0/2858099741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd4160000 0x7f9fd40c5e30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400e840 tx=0x7f9fc400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:36.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fca7fc700 1 -- 172.21.15.104:0/2858099741 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40095a0 con 0x7f9fd4160000 2023-07-14T11:54:36.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fca7fc700 1 -- 172.21.15.104:0/2858099741 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc4013680 con 0x7f9fd4160000 2023-07-14T11:54:36.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fda9d9700 1 -- 172.21.15.104:0/2858099741 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd40b16f0 con 0x7f9fd4160000 2023-07-14T11:54:36.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fca7fc700 1 -- 172.21.15.104:0/2858099741 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40139f0 con 0x7f9fd4160000 2023-07-14T11:54:36.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fda9d9700 1 -- 172.21.15.104:0/2858099741 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd40b1a80 con 0x7f9fd4160000 2023-07-14T11:54:36.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.711+0000 7f9fda9d9700 1 -- 172.21.15.104:0/2858099741 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd4071240 con 0x7f9fd4160000 2023-07-14T11:54:36.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.715+0000 7f9fca7fc700 1 -- 172.21.15.104:0/2858099741 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9fc4013b50 con 0x7f9fd4160000 2023-07-14T11:54:36.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.715+0000 7f9fca7fc700 1 --2- 172.21.15.104:0/2858099741 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9fb80662f0 0x7f9fb80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:36.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.715+0000 7f9fca7fc700 1 -- 172.21.15.104:0/2858099741 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9fc4083f20 con 0x7f9fd4160000 2023-07-14T11:54:36.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.715+0000 7f9fd99d7700 1 --2- 172.21.15.104:0/2858099741 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9fb80662f0 0x7f9fb80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:36.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.715+0000 7f9fd99d7700 1 --2- 172.21.15.104:0/2858099741 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9fb80662f0 0x7f9fb80687a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0000c00 tx=0x7f9fd0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:36.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.723+0000 7f9fca7fc700 1 -- 172.21.15.104:0/2858099741 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f9fd4071240 con 0x7f9fd4160000 2023-07-14T11:54:36.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.899+0000 7f9fda9d9700 1 -- 172.21.15.104:0/2858099741 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9fd4071240 con 0x7f9fd4160000 2023-07-14T11:54:36.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.899+0000 7f9fca7fc700 1 -- 172.21.15.104:0/2858099741 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9fc4017020 con 0x7f9fd4160000 2023-07-14T11:54:36.886 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:54:36.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.903+0000 7f9fda9d9700 1 -- 172.21.15.104:0/2858099741 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9fb80662f0 msgr2=0x7f9fb80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.903+0000 7f9fda9d9700 1 --2- 172.21.15.104:0/2858099741 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9fb80662f0 0x7f9fb80687a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0000c00 tx=0x7f9fd0008030 comp rx=0 tx=0).stop 2023-07-14T11:54:36.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.903+0000 7f9fda9d9700 1 -- 172.21.15.104:0/2858099741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd4160000 msgr2=0x7f9fd40c5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:36.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.903+0000 7f9fda9d9700 1 --2- 172.21.15.104:0/2858099741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd4160000 0x7f9fd40c5e30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400e840 tx=0x7f9fc400eb50 comp rx=0 tx=0).stop 2023-07-14T11:54:36.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.903+0000 7f9fda9d9700 1 -- 172.21.15.104:0/2858099741 shutdown_connections 2023-07-14T11:54:36.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.903+0000 7f9fda9d9700 1 --2- 172.21.15.104:0/2858099741 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9fb80662f0 0x7f9fb80687a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.903+0000 7f9fda9d9700 1 --2- 172.21.15.104:0/2858099741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd4160000 0x7f9fd40c5e30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.903+0000 7f9fda9d9700 1 --2- 172.21.15.104:0/2858099741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fd41603d0 0x7f9fd40b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.903+0000 7f9fda9d9700 1 --2- 172.21.15.104:0/2858099741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fd40cd7f0 0x7f9fd4154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:36.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.903+0000 7f9fda9d9700 1 -- 172.21.15.104:0/2858099741 >> 172.21.15.104:0/2858099741 conn(0x7f9fd40afc60 msgr2=0x7f9fd40c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:36.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.907+0000 7f9fda9d9700 1 -- 172.21.15.104:0/2858099741 shutdown_connections 2023-07-14T11:54:36.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:36.907+0000 7f9fda9d9700 1 -- 172.21.15.104:0/2858099741 wait complete. 2023-07-14T11:54:36.912 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:54:37.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.111+0000 7f68ec786700 1 Processor -- start 2023-07-14T11:54:37.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.111+0000 7f68ec786700 1 -- start start 2023-07-14T11:54:37.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68ec786700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f68e40d0210 0x7f68e40d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68ec786700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68e40cf000 0x7f68e40cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68ec786700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e4071d10 con 0x7f68e40b1c50 2023-07-14T11:54:37.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68ec786700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e40cf9c0 con 0x7f68e40d0210 2023-07-14T11:54:37.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68ec786700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e415c420 con 0x7f68e40cf000 2023-07-14T11:54:37.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68e9d21700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f68e40d0210 0x7f68e40d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68e9d21700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f68e40d0210 0x7f68e40d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:43996/0 (socket says 172.21.15.104:43996) 2023-07-14T11:54:37.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68e9d21700 1 -- 172.21.15.104:0/662748813 learned_addr learned my addr 172.21.15.104:0/662748813 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:37.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68ead23700 1 --2- 172.21.15.104:0/662748813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68e40cf000 0x7f68e40cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68ead23700 1 -- 172.21.15.104:0/662748813 >> v2:172.21.15.104:3300/0 conn(0x7f68e40d0210 msgr2=0x7f68e40d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:37.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68ead23700 1 --2- 172.21.15.104:0/662748813 >> v2:172.21.15.104:3300/0 conn(0x7f68e40d0210 0x7f68e40d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68ead23700 1 -- 172.21.15.104:0/662748813 >> v1:172.21.15.121:6789/0 conn(0x7f68e40b1c50 legacy=0x7f68e4071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68ead23700 1 -- 172.21.15.104:0/662748813 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e4070840 con 0x7f68e40cf000 2023-07-14T11:54:37.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68e9d21700 1 --2- 172.21.15.104:0/662748813 >> v2:172.21.15.104:3300/0 conn(0x7f68e40d0210 0x7f68e40d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:37.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68ead23700 1 --2- 172.21.15.104:0/662748813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68e40cf000 0x7f68e40cf480 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009d90 tx=0x7f68dc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d12a5af622cd9426 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:37.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68e9520700 1 -- 172.21.15.104:0/662748813 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc00e070 con 0x7f68e40cf000 2023-07-14T11:54:37.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68e9520700 1 -- 172.21.15.104:0/662748813 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68dc00bb20 con 0x7f68e40cf000 2023-07-14T11:54:37.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68e9520700 1 -- 172.21.15.104:0/662748813 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc012430 con 0x7f68e40cf000 2023-07-14T11:54:37.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68ec786700 1 -- 172.21.15.104:0/662748813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68e40cf000 msgr2=0x7f68e40cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.115+0000 7f68ec786700 1 --2- 172.21.15.104:0/662748813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68e40cf000 0x7f68e40cf480 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009d90 tx=0x7f68dc00b330 comp rx=0 tx=0).stop 2023-07-14T11:54:37.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ec786700 1 -- 172.21.15.104:0/662748813 shutdown_connections 2023-07-14T11:54:37.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ec786700 1 --2- 172.21.15.104:0/662748813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68e40cf000 0x7f68e40cf480 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ec786700 1 --2- 172.21.15.104:0/662748813 >> v2:172.21.15.104:3300/0 conn(0x7f68e40d0210 0x7f68e40d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ec786700 1 -- 172.21.15.104:0/662748813 >> 172.21.15.104:0/662748813 conn(0x7f68e409b9c0 msgr2=0x7f68e40d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:37.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ec786700 1 -- 172.21.15.104:0/662748813 shutdown_connections 2023-07-14T11:54:37.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ec786700 1 -- 172.21.15.104:0/662748813 wait complete. 2023-07-14T11:54:37.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ec786700 1 Processor -- start 2023-07-14T11:54:37.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ec786700 1 -- start start 2023-07-14T11:54:37.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ec786700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68e40b1c50 0x7f68e41e53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ec786700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68e40cf000 0x7f68e4159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ec786700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68e40d0210 0x7f68e40d2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ec786700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e4159fb0 con 0x7f68e40cf000 2023-07-14T11:54:37.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ec786700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e41e5930 con 0x7f68e40b1c50 2023-07-14T11:54:37.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ec786700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e41f5360 con 0x7f68e40d0210 2023-07-14T11:54:37.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68e9d21700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68e40cf000 0x7f68e4159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68e9d21700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68e40cf000 0x7f68e4159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44010/0 (socket says 172.21.15.104:44010) 2023-07-14T11:54:37.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68e9d21700 1 -- 172.21.15.104:0/3871411192 learned_addr learned my addr 172.21.15.104:0/3871411192 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:37.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ead23700 1 --2- 172.21.15.104:0/3871411192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68e40d0210 0x7f68e40d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.108 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.119+0000 7f68ea522700 1 --2- 172.21.15.104:0/3871411192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68e40b1c50 0x7f68e41e53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.123+0000 7f68e9d21700 1 -- 172.21.15.104:0/3871411192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68e40d0210 msgr2=0x7f68e40d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.123+0000 7f68e9d21700 1 --2- 172.21.15.104:0/3871411192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68e40d0210 0x7f68e40d2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.123+0000 7f68e9d21700 1 -- 172.21.15.104:0/3871411192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68e40b1c50 msgr2=0x7f68e41e53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.123+0000 7f68e9d21700 1 --2- 172.21.15.104:0/3871411192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68e40b1c50 0x7f68e41e53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.123+0000 7f68e9d21700 1 -- 172.21.15.104:0/3871411192 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d4009570 con 0x7f68e40cf000 2023-07-14T11:54:37.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.123+0000 7f68e9d21700 1 --2- 172.21.15.104:0/3871411192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68e40cf000 0x7f68e4159790 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f68d400e9d0 tx=0x7f68d400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:37.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.123+0000 7f68daffd700 1 -- 172.21.15.104:0/3871411192 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d400ee70 con 0x7f68e40cf000 2023-07-14T11:54:37.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.123+0000 7f68daffd700 1 -- 172.21.15.104:0/3871411192 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d400c850 con 0x7f68e40cf000 2023-07-14T11:54:37.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.123+0000 7f68daffd700 1 -- 172.21.15.104:0/3871411192 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d400cbc0 con 0x7f68e40cf000 2023-07-14T11:54:37.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.123+0000 7f68ec786700 1 -- 172.21.15.104:0/3871411192 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68dc009a70 con 0x7f68e40cf000 2023-07-14T11:54:37.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.123+0000 7f68ec786700 1 -- 172.21.15.104:0/3871411192 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e41f5920 con 0x7f68e40cf000 2023-07-14T11:54:37.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.123+0000 7f68ec786700 1 -- 172.21.15.104:0/3871411192 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e404dcb0 con 0x7f68e40cf000 2023-07-14T11:54:37.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.127+0000 7f68daffd700 1 -- 172.21.15.104:0/3871411192 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f68d400cd20 con 0x7f68e40cf000 2023-07-14T11:54:37.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.127+0000 7f68daffd700 1 --2- 172.21.15.104:0/3871411192 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f68c80662f0 0x7f68c80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.127+0000 7f68daffd700 1 -- 172.21.15.104:0/3871411192 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f68d4083cf0 con 0x7f68e40cf000 2023-07-14T11:54:37.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.127+0000 7f68ea522700 1 --2- 172.21.15.104:0/3871411192 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f68c80662f0 0x7f68c80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.127+0000 7f68ea522700 1 --2- 172.21.15.104:0/3871411192 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f68c80662f0 0x7f68c80687a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f68e0000f60 tx=0x7f68e0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:37.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.135+0000 7f68daffd700 1 -- 172.21.15.104:0/3871411192 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f68d4050da0 con 0x7f68e40cf000 2023-07-14T11:54:37.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.303+0000 7f68ec786700 1 -- 172.21.15.104:0/3871411192 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f68e40b6160 con 0x7f68e40cf000 2023-07-14T11:54:37.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.303+0000 7f68daffd700 1 -- 172.21.15.104:0/3871411192 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f68d40543c0 con 0x7f68e40cf000 2023-07-14T11:54:37.289 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:37.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.307+0000 7f68ec786700 1 -- 172.21.15.104:0/3871411192 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f68c80662f0 msgr2=0x7f68c80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.307+0000 7f68ec786700 1 --2- 172.21.15.104:0/3871411192 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f68c80662f0 0x7f68c80687a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f68e0000f60 tx=0x7f68e0009040 comp rx=0 tx=0).stop 2023-07-14T11:54:37.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.307+0000 7f68ec786700 1 -- 172.21.15.104:0/3871411192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68e40cf000 msgr2=0x7f68e4159790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.307+0000 7f68ec786700 1 --2- 172.21.15.104:0/3871411192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68e40cf000 0x7f68e4159790 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f68d400e9d0 tx=0x7f68d400ece0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.307+0000 7f68ec786700 1 -- 172.21.15.104:0/3871411192 shutdown_connections 2023-07-14T11:54:37.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.307+0000 7f68ec786700 1 --2- 172.21.15.104:0/3871411192 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f68c80662f0 0x7f68c80687a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.307+0000 7f68ec786700 1 --2- 172.21.15.104:0/3871411192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68e40cf000 0x7f68e4159790 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.307+0000 7f68ec786700 1 --2- 172.21.15.104:0/3871411192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68e40d0210 0x7f68e40d2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.307+0000 7f68ec786700 1 --2- 172.21.15.104:0/3871411192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68e40b1c50 0x7f68e41e53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.307+0000 7f68ec786700 1 -- 172.21.15.104:0/3871411192 >> 172.21.15.104:0/3871411192 conn(0x7f68e409b9c0 msgr2=0x7f68e40b5750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:37.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.307+0000 7f68ec786700 1 -- 172.21.15.104:0/3871411192 shutdown_connections 2023-07-14T11:54:37.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.307+0000 7f68ec786700 1 -- 172.21.15.104:0/3871411192 wait complete. 2023-07-14T11:54:37.312 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:54:37.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.495+0000 7f5e8ac20700 1 Processor -- start 2023-07-14T11:54:37.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.499+0000 7f5e8ac20700 1 -- start start 2023-07-14T11:54:37.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.499+0000 7f5e8ac20700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5e8406d860 0x7f5e840cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.499+0000 7f5e8ac20700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e840c1e00 0x7f5e840cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.499+0000 7f5e8ac20700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e840cc1b0 con 0x7f5e8406be70 2023-07-14T11:54:37.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.499+0000 7f5e8ac20700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e840cd690 con 0x7f5e8406d860 2023-07-14T11:54:37.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.499+0000 7f5e8ac20700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e840cd970 con 0x7f5e840c1e00 2023-07-14T11:54:37.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.499+0000 7f5e889bc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5e8406d860 0x7f5e840cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.499+0000 7f5e891bd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e840c1e00 0x7f5e840cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.499+0000 7f5e889bc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5e8406d860 0x7f5e840cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:37166/0 (socket says 172.21.15.104:37166) 2023-07-14T11:54:37.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.499+0000 7f5e889bc700 1 -- 172.21.15.104:0/2275084059 learned_addr learned my addr 172.21.15.104:0/2275084059 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:37.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e889bc700 1 -- 172.21.15.104:0/2275084059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e840c1e00 msgr2=0x7f5e840cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:37.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e889bc700 1 --2- 172.21.15.104:0/2275084059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e840c1e00 0x7f5e840cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e889bc700 1 -- 172.21.15.104:0/2275084059 >> v1:172.21.15.121:6789/0 conn(0x7f5e8406be70 legacy=0x7f5e8406dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e889bc700 1 -- 172.21.15.104:0/2275084059 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e840cdc50 con 0x7f5e8406d860 2023-07-14T11:54:37.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e889bc700 1 --2- 172.21.15.104:0/2275084059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e8406d860 0x7f5e840cb840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009910 tx=0x7f5e78009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4765c991b9930864 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:37.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e837fe700 1 -- 172.21.15.104:0/2275084059 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78011830 con 0x7f5e8406d860 2023-07-14T11:54:37.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e837fe700 1 -- 172.21.15.104:0/2275084059 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e78011990 con 0x7f5e8406d860 2023-07-14T11:54:37.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e837fe700 1 -- 172.21.15.104:0/2275084059 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78011d40 con 0x7f5e8406d860 2023-07-14T11:54:37.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e8ac20700 1 -- 172.21.15.104:0/2275084059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e8406d860 msgr2=0x7f5e840cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e8ac20700 1 --2- 172.21.15.104:0/2275084059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e8406d860 0x7f5e840cb840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009910 tx=0x7f5e78009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:37.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e8ac20700 1 -- 172.21.15.104:0/2275084059 shutdown_connections 2023-07-14T11:54:37.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e8ac20700 1 --2- 172.21.15.104:0/2275084059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e840c1e00 0x7f5e840cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e8ac20700 1 --2- 172.21.15.104:0/2275084059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e8406d860 0x7f5e840cb840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e8ac20700 1 -- 172.21.15.104:0/2275084059 >> 172.21.15.104:0/2275084059 conn(0x7f5e840cad30 msgr2=0x7f5e840cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:37.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e8ac20700 1 -- 172.21.15.104:0/2275084059 shutdown_connections 2023-07-14T11:54:37.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.503+0000 7f5e8ac20700 1 -- 172.21.15.104:0/2275084059 wait complete. 2023-07-14T11:54:37.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.507+0000 7f5e8ac20700 1 Processor -- start 2023-07-14T11:54:37.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.507+0000 7f5e8ac20700 1 -- start start 2023-07-14T11:54:37.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e8ac20700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8406be70 0x7f5e841f1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e8ac20700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e8406d860 0x7f5e8415c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e8ac20700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e840c1e00 0x7f5e841f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e8ac20700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8415a5e0 con 0x7f5e8406be70 2023-07-14T11:54:37.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e8ac20700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8415c7b0 con 0x7f5e8406d860 2023-07-14T11:54:37.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e8ac20700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e841f2a80 con 0x7f5e840c1e00 2023-07-14T11:54:37.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e889bc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8406be70 0x7f5e841f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e889bc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8406be70 0x7f5e841f1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44020/0 (socket says 172.21.15.104:44020) 2023-07-14T11:54:37.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e889bc700 1 -- 172.21.15.104:0/2479619959 learned_addr learned my addr 172.21.15.104:0/2479619959 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:37.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e83fff700 1 --2- 172.21.15.104:0/2479619959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e8406d860 0x7f5e8415c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e891bd700 1 --2- 172.21.15.104:0/2479619959 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e840c1e00 0x7f5e841f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e889bc700 1 -- 172.21.15.104:0/2479619959 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e840c1e00 msgr2=0x7f5e841f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e889bc700 1 --2- 172.21.15.104:0/2479619959 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e840c1e00 0x7f5e841f2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e889bc700 1 -- 172.21.15.104:0/2479619959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e8406d860 msgr2=0x7f5e8415c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e889bc700 1 --2- 172.21.15.104:0/2479619959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e8406d860 0x7f5e8415c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e889bc700 1 -- 172.21.15.104:0/2479619959 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e78009570 con 0x7f5e8406be70 2023-07-14T11:54:37.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e83fff700 1 --2- 172.21.15.104:0/2479619959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e8406d860 0x7f5e8415c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:37.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e889bc700 1 --2- 172.21.15.104:0/2479619959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8406be70 0x7f5e841f1380 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5e78000c00 tx=0x7f5e78004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:37.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e817fa700 1 -- 172.21.15.104:0/2479619959 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78004450 con 0x7f5e8406be70 2023-07-14T11:54:37.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e817fa700 1 -- 172.21.15.104:0/2479619959 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e780045d0 con 0x7f5e8406be70 2023-07-14T11:54:37.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e817fa700 1 -- 172.21.15.104:0/2479619959 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78003900 con 0x7f5e8406be70 2023-07-14T11:54:37.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e8ac20700 1 -- 172.21.15.104:0/2479619959 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e841f19b0 con 0x7f5e8406be70 2023-07-14T11:54:37.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.511+0000 7f5e8ac20700 1 -- 172.21.15.104:0/2479619959 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e841f1f70 con 0x7f5e8406be70 2023-07-14T11:54:37.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.515+0000 7f5e8ac20700 1 -- 172.21.15.104:0/2479619959 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8415a720 con 0x7f5e8406be70 2023-07-14T11:54:37.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.515+0000 7f5e817fa700 1 -- 172.21.15.104:0/2479619959 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5e78006c80 con 0x7f5e8406be70 2023-07-14T11:54:37.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.519+0000 7f5e817fa700 1 --2- 172.21.15.104:0/2479619959 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5e680662f0 0x7f5e680687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.519+0000 7f5e817fa700 1 -- 172.21.15.104:0/2479619959 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5e78021030 con 0x7f5e8406be70 2023-07-14T11:54:37.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.519+0000 7f5e83fff700 1 --2- 172.21.15.104:0/2479619959 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5e680662f0 0x7f5e680687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.519+0000 7f5e83fff700 1 --2- 172.21.15.104:0/2479619959 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5e680662f0 0x7f5e680687a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5e84156860 tx=0x7f5e70009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:37.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.523+0000 7f5e817fa700 1 -- 172.21.15.104:0/2479619959 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f5e8415a720 con 0x7f5e8406be70 2023-07-14T11:54:37.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.707+0000 7f5e8ac20700 1 -- 172.21.15.104:0/2479619959 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5e8415a720 con 0x7f5e8406be70 2023-07-14T11:54:37.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.707+0000 7f5e817fa700 1 -- 172.21.15.104:0/2479619959 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5e8415a720 con 0x7f5e8406be70 2023-07-14T11:54:37.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.715+0000 7f5e8ac20700 1 -- 172.21.15.104:0/2479619959 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5e680662f0 msgr2=0x7f5e680687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.715+0000 7f5e8ac20700 1 --2- 172.21.15.104:0/2479619959 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5e680662f0 0x7f5e680687a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5e84156860 tx=0x7f5e70009040 comp rx=0 tx=0).stop 2023-07-14T11:54:37.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.715+0000 7f5e8ac20700 1 -- 172.21.15.104:0/2479619959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8406be70 msgr2=0x7f5e841f1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.715+0000 7f5e8ac20700 1 --2- 172.21.15.104:0/2479619959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8406be70 0x7f5e841f1380 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5e78000c00 tx=0x7f5e78004770 comp rx=0 tx=0).stop 2023-07-14T11:54:37.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.715+0000 7f5e8ac20700 1 -- 172.21.15.104:0/2479619959 shutdown_connections 2023-07-14T11:54:37.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.715+0000 7f5e8ac20700 1 --2- 172.21.15.104:0/2479619959 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5e680662f0 0x7f5e680687a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.715+0000 7f5e8ac20700 1 --2- 172.21.15.104:0/2479619959 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e840c1e00 0x7f5e841f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.715+0000 7f5e8ac20700 1 --2- 172.21.15.104:0/2479619959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e8406d860 0x7f5e8415c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.715+0000 7f5e8ac20700 1 --2- 172.21.15.104:0/2479619959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8406be70 0x7f5e841f1380 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.715+0000 7f5e8ac20700 1 -- 172.21.15.104:0/2479619959 >> 172.21.15.104:0/2479619959 conn(0x7f5e840cad30 msgr2=0x7f5e840ab8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:37.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.715+0000 7f5e8ac20700 1 -- 172.21.15.104:0/2479619959 shutdown_connections 2023-07-14T11:54:37.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.715+0000 7f5e8ac20700 1 -- 172.21.15.104:0/2479619959 wait complete. 2023-07-14T11:54:37.701 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:54:37.721 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:54:37.721 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:54:37.743 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:37.796 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:54:37.801 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:54:37.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8344ac6700 1 Processor -- start 2023-07-14T11:54:37.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8344ac6700 1 -- start start 2023-07-14T11:54:37.916 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8344ac6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f834015bf20 0x7f83400b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8344ac6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f83400b1530 0x7f83400d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8344ac6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f83400d1260 0x7f83400b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8344ac6700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f834015c2f0 con 0x7f83400b1530 2023-07-14T11:54:37.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8344ac6700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8340070830 con 0x7f83400d1260 2023-07-14T11:54:37.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8344ac6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8340070b10 con 0x7f834015bf20 2023-07-14T11:54:37.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8337fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f83400b1530 0x7f83400d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.920 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f833ed9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f83400d1260 0x7f83400b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.920 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f833e59c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f834015bf20 0x7f83400b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.921 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8337fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f83400b1530 0x7f83400d0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44030/0 (socket says 172.21.15.104:44030) 2023-07-14T11:54:37.921 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8337fff700 1 -- 172.21.15.104:0/2844483905 learned_addr learned my addr 172.21.15.104:0/2844483905 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:37.921 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8337fff700 1 -- 172.21.15.104:0/2844483905 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f834015bf20 msgr2=0x7f83400b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:37.921 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8337fff700 1 --2- 172.21.15.104:0/2844483905 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f834015bf20 0x7f83400b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8337fff700 1 -- 172.21.15.104:0/2844483905 >> v2:172.21.15.121:3300/0 conn(0x7f83400d1260 msgr2=0x7f83400b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:37.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8337fff700 1 --2- 172.21.15.104:0/2844483905 >> v2:172.21.15.121:3300/0 conn(0x7f83400d1260 0x7f83400b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.931+0000 7f8337fff700 1 -- 172.21.15.104:0/2844483905 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83400cccc0 con 0x7f83400b1530 2023-07-14T11:54:37.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.935+0000 7f8337fff700 1 --2- 172.21.15.104:0/2844483905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83400b1530 0x7f83400d0d20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f83400b1fc0 tx=0x7f8328009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ce1c461a0dc15e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:37.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.935+0000 7f833dd9b700 1 -- 172.21.15.104:0/2844483905 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328011830 con 0x7f83400b1530 2023-07-14T11:54:37.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.935+0000 7f833dd9b700 1 -- 172.21.15.104:0/2844483905 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8328011990 con 0x7f83400b1530 2023-07-14T11:54:37.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.935+0000 7f833dd9b700 1 -- 172.21.15.104:0/2844483905 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328011d20 con 0x7f83400b1530 2023-07-14T11:54:37.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.935+0000 7f8344ac6700 1 -- 172.21.15.104:0/2844483905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83400b1530 msgr2=0x7f83400d0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.935+0000 7f8344ac6700 1 --2- 172.21.15.104:0/2844483905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83400b1530 0x7f83400d0d20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f83400b1fc0 tx=0x7f8328009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.935+0000 7f8344ac6700 1 -- 172.21.15.104:0/2844483905 shutdown_connections 2023-07-14T11:54:37.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.935+0000 7f8344ac6700 1 --2- 172.21.15.104:0/2844483905 >> v2:172.21.15.121:3300/0 conn(0x7f83400d1260 0x7f83400b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.935+0000 7f8344ac6700 1 --2- 172.21.15.104:0/2844483905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83400b1530 0x7f83400d0d20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.935+0000 7f8344ac6700 1 --2- 172.21.15.104:0/2844483905 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f834015bf20 0x7f83400b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.935+0000 7f8344ac6700 1 -- 172.21.15.104:0/2844483905 >> 172.21.15.104:0/2844483905 conn(0x7f834006fc80 msgr2=0x7f8340070080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:37.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.935+0000 7f8344ac6700 1 -- 172.21.15.104:0/2844483905 shutdown_connections 2023-07-14T11:54:37.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.935+0000 7f8344ac6700 1 -- 172.21.15.104:0/2844483905 wait complete. 2023-07-14T11:54:37.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.939+0000 7f8344ac6700 1 Processor -- start 2023-07-14T11:54:37.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.939+0000 7f8344ac6700 1 -- start start 2023-07-14T11:54:37.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f8344ac6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83400b1530 0x7f834014eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f8344ac6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83400d1260 0x7f83400b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f8344ac6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f834015bf20 0x7f83400b6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f8344ac6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83400b6fc0 con 0x7f83400b1530 2023-07-14T11:54:37.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f8344ac6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83400b72a0 con 0x7f83400d1260 2023-07-14T11:54:37.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f8344ac6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83400b7580 con 0x7f834015bf20 2023-07-14T11:54:37.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f833e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83400b1530 0x7f834014eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f833e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83400b1530 0x7f834014eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44036/0 (socket says 172.21.15.104:44036) 2023-07-14T11:54:37.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f833e59c700 1 -- 172.21.15.104:0/3063226797 learned_addr learned my addr 172.21.15.104:0/3063226797 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:37.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f833e59c700 1 -- 172.21.15.104:0/3063226797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f834015bf20 msgr2=0x7f83400b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f833e59c700 1 --2- 172.21.15.104:0/3063226797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f834015bf20 0x7f83400b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f833e59c700 1 -- 172.21.15.104:0/3063226797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83400d1260 msgr2=0x7f83400b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:37.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f8337fff700 1 --2- 172.21.15.104:0/3063226797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83400d1260 0x7f83400b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f833e59c700 1 --2- 172.21.15.104:0/3063226797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83400d1260 0x7f83400b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:37.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f833e59c700 1 -- 172.21.15.104:0/3063226797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8328009570 con 0x7f83400b1530 2023-07-14T11:54:37.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f8337fff700 1 --2- 172.21.15.104:0/3063226797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83400d1260 0x7f83400b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:37.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f833e59c700 1 --2- 172.21.15.104:0/3063226797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83400b1530 0x7f834014eac0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f833000eb60 tx=0x7f833000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:37.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f8336ffd700 1 -- 172.21.15.104:0/3063226797 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330009740 con 0x7f83400b1530 2023-07-14T11:54:37.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f8336ffd700 1 -- 172.21.15.104:0/3063226797 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833000c850 con 0x7f83400b1530 2023-07-14T11:54:37.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f8344ac6700 1 -- 172.21.15.104:0/3063226797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83400b7860 con 0x7f83400b1530 2023-07-14T11:54:37.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f8336ffd700 1 -- 172.21.15.104:0/3063226797 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833000cbe0 con 0x7f83400b1530 2023-07-14T11:54:37.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f8344ac6700 1 -- 172.21.15.104:0/3063226797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834014f000 con 0x7f83400b1530 2023-07-14T11:54:37.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.943+0000 7f8344ac6700 1 -- 172.21.15.104:0/3063226797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83400b7d80 con 0x7f83400b1530 2023-07-14T11:54:37.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.947+0000 7f8336ffd700 1 -- 172.21.15.104:0/3063226797 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f833000cd40 con 0x7f83400b1530 2023-07-14T11:54:37.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.947+0000 7f8336ffd700 1 --2- 172.21.15.104:0/3063226797 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83200662f0 0x7f83200687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:37.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.947+0000 7f8336ffd700 1 -- 172.21.15.104:0/3063226797 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8330084110 con 0x7f83400b1530 2023-07-14T11:54:37.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.947+0000 7f8337fff700 1 --2- 172.21.15.104:0/3063226797 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83200662f0 0x7f83200687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:37.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.951+0000 7f8337fff700 1 --2- 172.21.15.104:0/3063226797 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83200662f0 0x7f83200687a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f83400c5c70 tx=0x7f8328008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:37.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:37.955+0000 7f8336ffd700 1 -- 172.21.15.104:0/3063226797 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f83400b7d80 con 0x7f83400b1530 2023-07-14T11:54:38.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.127+0000 7f8344ac6700 1 -- 172.21.15.104:0/3063226797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83400b7d80 con 0x7f83400b1530 2023-07-14T11:54:38.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.131+0000 7f8336ffd700 1 -- 172.21.15.104:0/3063226797 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20251 (secure 0 0 0) 0x7f83400b7d80 con 0x7f83400b1530 2023-07-14T11:54:38.116 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:38.116 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":30,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:55.601783+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:38.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.131+0000 7f8344ac6700 1 -- 172.21.15.104:0/3063226797 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83200662f0 msgr2=0x7f83200687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.131+0000 7f8344ac6700 1 --2- 172.21.15.104:0/3063226797 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83200662f0 0x7f83200687a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f83400c5c70 tx=0x7f8328008030 comp rx=0 tx=0).stop 2023-07-14T11:54:38.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.131+0000 7f8344ac6700 1 -- 172.21.15.104:0/3063226797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83400b1530 msgr2=0x7f834014eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.131+0000 7f8344ac6700 1 --2- 172.21.15.104:0/3063226797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83400b1530 0x7f834014eac0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f833000eb60 tx=0x7f833000ef20 comp rx=0 tx=0).stop 2023-07-14T11:54:38.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.131+0000 7f8344ac6700 1 -- 172.21.15.104:0/3063226797 shutdown_connections 2023-07-14T11:54:38.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.131+0000 7f8344ac6700 1 --2- 172.21.15.104:0/3063226797 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f83200662f0 0x7f83200687a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.131+0000 7f8344ac6700 1 --2- 172.21.15.104:0/3063226797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f834015bf20 0x7f83400b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.131+0000 7f8344ac6700 1 --2- 172.21.15.104:0/3063226797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83400d1260 0x7f83400b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.131+0000 7f8344ac6700 1 --2- 172.21.15.104:0/3063226797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83400b1530 0x7f834014eac0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.131+0000 7f8344ac6700 1 -- 172.21.15.104:0/3063226797 >> 172.21.15.104:0/3063226797 conn(0x7f834006fc80 msgr2=0x7f83400b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:38.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.135+0000 7f8344ac6700 1 -- 172.21.15.104:0/3063226797 shutdown_connections 2023-07-14T11:54:38.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.135+0000 7f8344ac6700 1 -- 172.21.15.104:0/3063226797 wait complete. 2023-07-14T11:54:38.140 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:54:38.141 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:54:38.325 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.339+0000 7f5078908700 1 Processor -- start 2023-07-14T11:54:38.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.339+0000 7f5078908700 1 -- start start 2023-07-14T11:54:38.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.343+0000 7f5078908700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f507415c410 0x7f50740c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.329 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.343+0000 7f5078908700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f50740c8d90 0x7f50740afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.329 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.343+0000 7f5078908700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507406d860 con 0x7f507406c700 2023-07-14T11:54:38.329 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.343+0000 7f5078908700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50740ca710 con 0x7f50740c8d90 2023-07-14T11:54:38.329 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.343+0000 7f5078908700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507406dcf0 con 0x7f507415c410 2023-07-14T11:54:38.330 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.343+0000 7f5072d9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f507406c700 0x7f507415c970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:45690/0 (socket says 172.21.15.104:45690) 2023-07-14T11:54:38.330 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.343+0000 7f5072d9d700 1 -- 172.21.15.104:0/3064506488 learned_addr learned my addr 172.21.15.104:0/3064506488 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:38.330 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.343+0000 7f5071d9b700 1 --2- 172.21.15.104:0/3064506488 >> v2:172.21.15.121:3300/0 conn(0x7f50740c8d90 0x7f50740afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.331 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.343+0000 7f507259c700 1 --2- 172.21.15.104:0/3064506488 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f507415c410 0x7f50740c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.343+0000 7f5071d9b700 1 -- 172.21.15.104:0/3064506488 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f507415c410 msgr2=0x7f50740c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:38.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.343+0000 7f5071d9b700 1 --2- 172.21.15.104:0/3064506488 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f507415c410 0x7f50740c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.343+0000 7f5071d9b700 1 -- 172.21.15.104:0/3064506488 >> v1:172.21.15.104:6789/0 conn(0x7f507406c700 legacy=0x7f507415c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.343+0000 7f5071d9b700 1 -- 172.21.15.104:0/3064506488 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507406cc50 con 0x7f50740c8d90 2023-07-14T11:54:38.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.343+0000 7f507159a700 1 -- 172.21.15.104:0/3064506488 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198318434 0 0) 0x7f507406d860 con 0x7f507406c700 2023-07-14T11:54:38.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.347+0000 7f5071d9b700 1 --2- 172.21.15.104:0/3064506488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50740c8d90 0x7f50740afd80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5068009850 tx=0x7f5068009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a352656e5fc48be server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:38.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.347+0000 7f507159a700 1 -- 172.21.15.104:0/3064506488 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068011830 con 0x7f50740c8d90 2023-07-14T11:54:38.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.347+0000 7f507159a700 1 -- 172.21.15.104:0/3064506488 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5068011990 con 0x7f50740c8d90 2023-07-14T11:54:38.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.347+0000 7f507159a700 1 -- 172.21.15.104:0/3064506488 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068011d40 con 0x7f50740c8d90 2023-07-14T11:54:38.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.347+0000 7f5078908700 1 -- 172.21.15.104:0/3064506488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50740c8d90 msgr2=0x7f50740afd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.347+0000 7f5078908700 1 --2- 172.21.15.104:0/3064506488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50740c8d90 0x7f50740afd80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5068009850 tx=0x7f5068009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:38.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.347+0000 7f5078908700 1 -- 172.21.15.104:0/3064506488 shutdown_connections 2023-07-14T11:54:38.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.347+0000 7f5078908700 1 --2- 172.21.15.104:0/3064506488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50740c8d90 0x7f50740afd80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.347+0000 7f5078908700 1 --2- 172.21.15.104:0/3064506488 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f507415c410 0x7f50740c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.347+0000 7f5078908700 1 -- 172.21.15.104:0/3064506488 >> 172.21.15.104:0/3064506488 conn(0x7f50740cad80 msgr2=0x7f50740cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:38.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.347+0000 7f5078908700 1 -- 172.21.15.104:0/3064506488 shutdown_connections 2023-07-14T11:54:38.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.347+0000 7f5078908700 1 -- 172.21.15.104:0/3064506488 wait complete. 2023-07-14T11:54:38.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.347+0000 7f5078908700 1 Processor -- start 2023-07-14T11:54:38.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5078908700 1 -- start start 2023-07-14T11:54:38.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5078908700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f507406c700 0x7f5074158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5078908700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50740c8d90 0x7f50740cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5078908700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f507415c410 0x7f50740cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5078908700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5074159d90 con 0x7f507415c410 2023-07-14T11:54:38.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5078908700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5074156f10 con 0x7f507406c700 2023-07-14T11:54:38.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5078908700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5074159180 con 0x7f50740c8d90 2023-07-14T11:54:38.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5072d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f507415c410 0x7f50740cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5072d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f507415c410 0x7f50740cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44040/0 (socket says 172.21.15.104:44040) 2023-07-14T11:54:38.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5072d9d700 1 -- 172.21.15.104:0/678081992 learned_addr learned my addr 172.21.15.104:0/678081992 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:38.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5071d9b700 1 --2- 172.21.15.104:0/678081992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50740c8d90 0x7f50740cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f507259c700 1 --2- 172.21.15.104:0/678081992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f507406c700 0x7f5074158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5072d9d700 1 -- 172.21.15.104:0/678081992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50740c8d90 msgr2=0x7f50740cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5072d9d700 1 --2- 172.21.15.104:0/678081992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50740c8d90 0x7f50740cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5072d9d700 1 -- 172.21.15.104:0/678081992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f507406c700 msgr2=0x7f5074158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5072d9d700 1 --2- 172.21.15.104:0/678081992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f507406c700 0x7f5074158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5072d9d700 1 -- 172.21.15.104:0/678081992 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50680094b0 con 0x7f507415c410 2023-07-14T11:54:38.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5072d9d700 1 --2- 172.21.15.104:0/678081992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f507415c410 0x7f50740cc510 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f506c00cc70 tx=0x7f506c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:38.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5062ffd700 1 -- 172.21.15.104:0/678081992 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c00ab50 con 0x7f507415c410 2023-07-14T11:54:38.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5062ffd700 1 -- 172.21.15.104:0/678081992 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c00acd0 con 0x7f507415c410 2023-07-14T11:54:38.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5078908700 1 -- 172.21.15.104:0/678081992 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5074159460 con 0x7f507415c410 2023-07-14T11:54:38.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5062ffd700 1 -- 172.21.15.104:0/678081992 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c007310 con 0x7f507415c410 2023-07-14T11:54:38.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.351+0000 7f5078908700 1 -- 172.21.15.104:0/678081992 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5074159a20 con 0x7f507415c410 2023-07-14T11:54:38.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.355+0000 7f5078908700 1 -- 172.21.15.104:0/678081992 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50740cbce0 con 0x7f507415c410 2023-07-14T11:54:38.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.355+0000 7f5062ffd700 1 -- 172.21.15.104:0/678081992 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f506c007470 con 0x7f507415c410 2023-07-14T11:54:38.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.355+0000 7f5062ffd700 1 --2- 172.21.15.104:0/678081992 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f50540662f0 0x7f50540687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.359+0000 7f5062ffd700 1 -- 172.21.15.104:0/678081992 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f506c083eb0 con 0x7f507415c410 2023-07-14T11:54:38.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.359+0000 7f507259c700 1 --2- 172.21.15.104:0/678081992 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f50540662f0 0x7f50540687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.359+0000 7f507259c700 1 --2- 172.21.15.104:0/678081992 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f50540662f0 0x7f50540687a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5064009ae0 tx=0x7f5064008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:38.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.363+0000 7f5062ffd700 1 -- 172.21.15.104:0/678081992 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f506c050ff0 con 0x7f507415c410 2023-07-14T11:54:38.524 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.539+0000 7f5078908700 1 -- 172.21.15.104:0/678081992 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f507404dcb0 con 0x7f507415c410 2023-07-14T11:54:38.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.539+0000 7f5062ffd700 1 -- 172.21.15.104:0/678081992 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f506c0180a0 con 0x7f507415c410 2023-07-14T11:54:38.525 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:54:38.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.543+0000 7f5078908700 1 -- 172.21.15.104:0/678081992 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f50540662f0 msgr2=0x7f50540687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.543+0000 7f5078908700 1 --2- 172.21.15.104:0/678081992 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f50540662f0 0x7f50540687a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5064009ae0 tx=0x7f5064008040 comp rx=0 tx=0).stop 2023-07-14T11:54:38.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.543+0000 7f5078908700 1 -- 172.21.15.104:0/678081992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f507415c410 msgr2=0x7f50740cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.543+0000 7f5078908700 1 --2- 172.21.15.104:0/678081992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f507415c410 0x7f50740cc510 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f506c00cc70 tx=0x7f506c00cf80 comp rx=0 tx=0).stop 2023-07-14T11:54:38.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.543+0000 7f5078908700 1 -- 172.21.15.104:0/678081992 shutdown_connections 2023-07-14T11:54:38.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.543+0000 7f5078908700 1 --2- 172.21.15.104:0/678081992 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f50540662f0 0x7f50540687a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.543+0000 7f5078908700 1 --2- 172.21.15.104:0/678081992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f507415c410 0x7f50740cc510 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.543+0000 7f5078908700 1 --2- 172.21.15.104:0/678081992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50740c8d90 0x7f50740cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.543+0000 7f5078908700 1 --2- 172.21.15.104:0/678081992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f507406c700 0x7f5074158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.543+0000 7f5078908700 1 -- 172.21.15.104:0/678081992 >> 172.21.15.104:0/678081992 conn(0x7f50740cad80 msgr2=0x7f50740c9730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:38.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.543+0000 7f5078908700 1 -- 172.21.15.104:0/678081992 shutdown_connections 2023-07-14T11:54:38.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.543+0000 7f5078908700 1 -- 172.21.15.104:0/678081992 wait complete. 2023-07-14T11:54:38.549 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:54:38.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.739+0000 7fd5a7307700 1 Processor -- start 2023-07-14T11:54:38.727 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.743+0000 7fd5a7307700 1 -- start start 2023-07-14T11:54:38.729 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.743+0000 7fd5a7307700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd5a0159c60 0x7fd5a006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.729 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.743+0000 7fd5a7307700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00cab00 0x7fd5a00caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.729 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.743+0000 7fd5a7307700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a00cb4c0 con 0x7fd5a00b0770 2023-07-14T11:54:38.729 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.743+0000 7fd5a7307700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a00714f0 con 0x7fd5a0159c60 2023-07-14T11:54:38.729 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.743+0000 7fd5a7307700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a006da00 con 0x7fd5a00cab00 2023-07-14T11:54:38.731 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.743+0000 7fd5a5b04700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fd5a00b0770 0x7fd5a0070de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:45702/0 (socket says 172.21.15.104:45702) 2023-07-14T11:54:38.732 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.743+0000 7fd5a5b04700 1 -- 172.21.15.104:0/835826597 learned_addr learned my addr 172.21.15.104:0/835826597 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:38.732 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.743+0000 7fd5a6305700 1 --2- 172.21.15.104:0/835826597 >> v2:172.21.15.121:3300/0 conn(0x7fd5a0159c60 0x7fd5a006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.732 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.743+0000 7fd5a6b06700 1 --2- 172.21.15.104:0/835826597 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00cab00 0x7fd5a00caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.733 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.743+0000 7fd5a5303700 1 -- 172.21.15.104:0/835826597 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 998361509 0 0) 0x7fd5a00cb4c0 con 0x7fd5a00b0770 2023-07-14T11:54:38.733 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.743+0000 7fd5a5303700 1 -- 172.21.15.104:0/835826597 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd58c0035d0 con 0x7fd5a00b0770 2023-07-14T11:54:38.733 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 -- 172.21.15.104:0/835826597 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3606825121 0 0) 0x7fd58c0035d0 con 0x7fd5a00b0770 2023-07-14T11:54:38.734 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 -- 172.21.15.104:0/835826597 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00cab00 msgr2=0x7fd5a00caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:38.734 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 --2- 172.21.15.104:0/835826597 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00cab00 0x7fd5a00caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.734 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 -- 172.21.15.104:0/835826597 >> v2:172.21.15.121:3300/0 conn(0x7fd5a0159c60 msgr2=0x7fd5a006d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:38.734 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 --2- 172.21.15.104:0/835826597 >> v2:172.21.15.121:3300/0 conn(0x7fd5a0159c60 0x7fd5a006d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 -- 172.21.15.104:0/835826597 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a006dce0 con 0x7fd5a00b0770 2023-07-14T11:54:38.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 -- 172.21.15.104:0/835826597 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7fd590001670 con 0x7fd5a00b0770 2023-07-14T11:54:38.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 -- 172.21.15.104:0/835826597 >> v1:172.21.15.104:6789/0 conn(0x7fd5a00b0770 legacy=0x7fd5a0070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 --2- 172.21.15.104:0/835826597 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00cab00 0x7fd58c003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.736 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 --2- 172.21.15.104:0/835826597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd58c003fa0 0x7fd58c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.736 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 --2- 172.21.15.104:0/835826597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd58c004db0 0x7fd58c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.736 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 -- 172.21.15.104:0/835826597 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a00714f0 con 0x7fd58c004db0 2023-07-14T11:54:38.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 -- 172.21.15.104:0/835826597 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a006da00 con 0x7fd58c003fa0 2023-07-14T11:54:38.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 -- 172.21.15.104:0/835826597 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c00fe40 con 0x7fd5a00cab00 2023-07-14T11:54:38.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a6b06700 1 --2- 172.21.15.104:0/835826597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd58c004db0 0x7fd58c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a6305700 1 --2- 172.21.15.104:0/835826597 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00cab00 0x7fd58c003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5b04700 1 --2- 172.21.15.104:0/835826597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd58c003fa0 0x7fd58c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a6b06700 1 -- 172.21.15.104:0/835826597 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00cab00 msgr2=0x7fd58c003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:38.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a6b06700 1 --2- 172.21.15.104:0/835826597 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00cab00 0x7fd58c003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a6b06700 1 -- 172.21.15.104:0/835826597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd58c003fa0 msgr2=0x7fd58c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:38.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a6b06700 1 --2- 172.21.15.104:0/835826597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd58c003fa0 0x7fd58c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a6b06700 1 -- 172.21.15.104:0/835826597 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c0035d0 con 0x7fd58c004db0 2023-07-14T11:54:38.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5b04700 1 --2- 172.21.15.104:0/835826597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd58c003fa0 0x7fd58c004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:38.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a6b06700 1 --2- 172.21.15.104:0/835826597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd58c004db0 0x7fd58c005220 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd598009fc0 tx=0x7fd58c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7dde660b30b04d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:38.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 -- 172.21.15.104:0/835826597 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59800ec30 con 0x7fd58c004db0 2023-07-14T11:54:38.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 -- 172.21.15.104:0/835826597 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59800f040 con 0x7fd58c004db0 2023-07-14T11:54:38.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a5303700 1 -- 172.21.15.104:0/835826597 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598014480 con 0x7fd58c004db0 2023-07-14T11:54:38.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a7307700 1 -- 172.21.15.104:0/835826597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd58c004db0 msgr2=0x7fd58c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a7307700 1 --2- 172.21.15.104:0/835826597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd58c004db0 0x7fd58c005220 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd598009fc0 tx=0x7fd58c0058a0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a6b06700 1 -- 172.21.15.104:0/835826597 reap_dead start 2023-07-14T11:54:38.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a7307700 1 -- 172.21.15.104:0/835826597 shutdown_connections 2023-07-14T11:54:38.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a7307700 1 -- 172.21.15.104:0/835826597 >> 172.21.15.104:0/835826597 conn(0x7fd5a00afc60 msgr2=0x7fd5a00b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:38.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.747+0000 7fd5a7307700 1 -- 172.21.15.104:0/835826597 shutdown_connections 2023-07-14T11:54:38.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.751+0000 7fd5a7307700 1 -- 172.21.15.104:0/835826597 wait complete. 2023-07-14T11:54:38.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.751+0000 7fd5a7307700 1 Processor -- start 2023-07-14T11:54:38.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.751+0000 7fd5a7307700 1 -- start start 2023-07-14T11:54:38.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.751+0000 7fd5a7307700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd58c004db0 0x7fd5a00c2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.751+0000 7fd5a7307700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00c6c80 0x7fd5a00b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.751+0000 7fd5a7307700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5a00b3bc0 0x7fd5a00c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.751+0000 7fd5a7307700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a00c2740 con 0x7fd5a00b3bc0 2023-07-14T11:54:38.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.751+0000 7fd5a7307700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a00c7240 con 0x7fd58c004db0 2023-07-14T11:54:38.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.751+0000 7fd5a7307700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a00c7520 con 0x7fd5a00c6c80 2023-07-14T11:54:38.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.751+0000 7fd5a6b06700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5a00b3bc0 0x7fd5a00c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.751+0000 7fd5a6b06700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5a00b3bc0 0x7fd5a00c3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44052/0 (socket says 172.21.15.104:44052) 2023-07-14T11:54:38.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.751+0000 7fd5a6b06700 1 -- 172.21.15.104:0/1222583862 learned_addr learned my addr 172.21.15.104:0/1222583862 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:38.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.755+0000 7fd5a6305700 1 --2- 172.21.15.104:0/1222583862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd58c004db0 0x7fd5a00c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.755+0000 7fd5a5b04700 1 --2- 172.21.15.104:0/1222583862 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00c6c80 0x7fd5a00b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.755+0000 7fd5a6b06700 1 -- 172.21.15.104:0/1222583862 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00c6c80 msgr2=0x7fd5a00b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.755+0000 7fd5a6b06700 1 --2- 172.21.15.104:0/1222583862 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00c6c80 0x7fd5a00b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.755+0000 7fd5a6b06700 1 -- 172.21.15.104:0/1222583862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd58c004db0 msgr2=0x7fd5a00c2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.755+0000 7fd5a6b06700 1 --2- 172.21.15.104:0/1222583862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd58c004db0 0x7fd5a00c2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.755+0000 7fd5a6b06700 1 -- 172.21.15.104:0/1222583862 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a00c7800 con 0x7fd5a00b3bc0 2023-07-14T11:54:38.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.755+0000 7fd5a6b06700 1 --2- 172.21.15.104:0/1222583862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5a00b3bc0 0x7fd5a00c3c90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd598016000 tx=0x7fd598009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:38.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.755+0000 7fd596ffd700 1 -- 172.21.15.104:0/1222583862 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598007000 con 0x7fd5a00b3bc0 2023-07-14T11:54:38.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.755+0000 7fd596ffd700 1 -- 172.21.15.104:0/1222583862 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd598022070 con 0x7fd5a00b3bc0 2023-07-14T11:54:38.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.755+0000 7fd5a7307700 1 -- 172.21.15.104:0/1222583862 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a00c7ae0 con 0x7fd5a00b3bc0 2023-07-14T11:54:38.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.755+0000 7fd596ffd700 1 -- 172.21.15.104:0/1222583862 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598029480 con 0x7fd5a00b3bc0 2023-07-14T11:54:38.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.755+0000 7fd5a7307700 1 -- 172.21.15.104:0/1222583862 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a00c41d0 con 0x7fd5a00b3bc0 2023-07-14T11:54:38.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.755+0000 7fd5a7307700 1 -- 172.21.15.104:0/1222583862 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a00708a0 con 0x7fd5a00b3bc0 2023-07-14T11:54:38.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.759+0000 7fd596ffd700 1 -- 172.21.15.104:0/1222583862 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd5980295e0 con 0x7fd5a00b3bc0 2023-07-14T11:54:38.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.759+0000 7fd596ffd700 1 --2- 172.21.15.104:0/1222583862 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd584066320 0x7fd5840687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:38.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.759+0000 7fd596ffd700 1 -- 172.21.15.104:0/1222583862 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd598024070 con 0x7fd5a00b3bc0 2023-07-14T11:54:38.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.759+0000 7fd5a6305700 1 --2- 172.21.15.104:0/1222583862 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd584066320 0x7fd5840687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:38.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.759+0000 7fd5a6305700 1 --2- 172.21.15.104:0/1222583862 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd584066320 0x7fd5840687d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009b40 tx=0x7fd59c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:38.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.767+0000 7fd596ffd700 1 -- 172.21.15.104:0/1222583862 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fd5a00708a0 con 0x7fd5a00b3bc0 2023-07-14T11:54:38.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.931+0000 7fd5a7307700 1 -- 172.21.15.104:0/1222583862 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd5a00708a0 con 0x7fd5a00b3bc0 2023-07-14T11:54:38.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.935+0000 7fd596ffd700 1 -- 172.21.15.104:0/1222583862 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd5a00708a0 con 0x7fd5a00b3bc0 2023-07-14T11:54:38.920 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:54:38.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.935+0000 7fd5a7307700 1 -- 172.21.15.104:0/1222583862 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd584066320 msgr2=0x7fd5840687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.935+0000 7fd5a7307700 1 --2- 172.21.15.104:0/1222583862 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd584066320 0x7fd5840687d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009b40 tx=0x7fd59c008040 comp rx=0 tx=0).stop 2023-07-14T11:54:38.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.935+0000 7fd5a7307700 1 -- 172.21.15.104:0/1222583862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5a00b3bc0 msgr2=0x7fd5a00c3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:38.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.935+0000 7fd5a7307700 1 --2- 172.21.15.104:0/1222583862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5a00b3bc0 0x7fd5a00c3c90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd598016000 tx=0x7fd598009750 comp rx=0 tx=0).stop 2023-07-14T11:54:38.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.935+0000 7fd5a7307700 1 -- 172.21.15.104:0/1222583862 shutdown_connections 2023-07-14T11:54:38.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.935+0000 7fd5a7307700 1 --2- 172.21.15.104:0/1222583862 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd584066320 0x7fd5840687d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.935+0000 7fd5a7307700 1 --2- 172.21.15.104:0/1222583862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5a00b3bc0 0x7fd5a00c3c90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.935+0000 7fd5a7307700 1 --2- 172.21.15.104:0/1222583862 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00c6c80 0x7fd5a00b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.935+0000 7fd5a7307700 1 --2- 172.21.15.104:0/1222583862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd58c004db0 0x7fd5a00c2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:38.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.935+0000 7fd5a7307700 1 -- 172.21.15.104:0/1222583862 >> 172.21.15.104:0/1222583862 conn(0x7fd5a00afc60 msgr2=0x7fd5a00aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:38.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.939+0000 7fd5a7307700 1 -- 172.21.15.104:0/1222583862 shutdown_connections 2023-07-14T11:54:38.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:38.939+0000 7fd5a7307700 1 -- 172.21.15.104:0/1222583862 wait complete. 2023-07-14T11:54:38.942 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:54:39.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.131+0000 7f499f264700 1 Processor -- start 2023-07-14T11:54:39.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.131+0000 7f499f264700 1 -- start start 2023-07-14T11:54:39.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.135+0000 7f499f264700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f499806c220 0x7f49980cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.135+0000 7f499f264700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499806d310 con 0x7f4998158a20 2023-07-14T11:54:39.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.135+0000 7f499f264700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499806c700 con 0x7f49980cc260 2023-07-14T11:54:39.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.135+0000 7f499f264700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499806c9e0 con 0x7f499806c220 2023-07-14T11:54:39.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.135+0000 7f499e262700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f4998158a20 0x7f49980cb2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:45708/0 (socket says 172.21.15.104:45708) 2023-07-14T11:54:39.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.135+0000 7f499e262700 1 -- 172.21.15.104:0/2678353518 learned_addr learned my addr 172.21.15.104:0/2678353518 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:39.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.135+0000 7f499da61700 1 --2- 172.21.15.104:0/2678353518 >> v2:172.21.15.121:3300/0 conn(0x7f499806c220 0x7f49980cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.135+0000 7f499d260700 1 -- 172.21.15.104:0/2678353518 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2829380616 0 0) 0x7f499806d310 con 0x7f4998158a20 2023-07-14T11:54:39.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.135+0000 7f499d260700 1 -- 172.21.15.104:0/2678353518 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49840035d0 con 0x7f4998158a20 2023-07-14T11:54:39.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.135+0000 7f499da61700 1 -- 172.21.15.104:0/2678353518 >> v1:172.21.15.121:6789/0 conn(0x7f49980cc260 legacy=0x7f499806d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.135+0000 7f499da61700 1 -- 172.21.15.104:0/2678353518 >> v1:172.21.15.104:6789/0 conn(0x7f4998158a20 legacy=0x7f49980cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.135+0000 7f499da61700 1 -- 172.21.15.104:0/2678353518 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499806ccc0 con 0x7f499806c220 2023-07-14T11:54:39.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.139+0000 7f499da61700 1 --2- 172.21.15.104:0/2678353518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f499806c220 0x7f49980cbd20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4988009850 tx=0x7f4988009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13313aa8bd3fc531 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:39.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.139+0000 7f499d260700 1 -- 172.21.15.104:0/2678353518 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988011830 con 0x7f499806c220 2023-07-14T11:54:39.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.139+0000 7f499d260700 1 -- 172.21.15.104:0/2678353518 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4988011990 con 0x7f499806c220 2023-07-14T11:54:39.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.139+0000 7f499d260700 1 -- 172.21.15.104:0/2678353518 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988011d40 con 0x7f499806c220 2023-07-14T11:54:39.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.139+0000 7f499f264700 1 -- 172.21.15.104:0/2678353518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f499806c220 msgr2=0x7f49980cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.139+0000 7f499f264700 1 --2- 172.21.15.104:0/2678353518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f499806c220 0x7f49980cbd20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4988009850 tx=0x7f4988009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:39.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.139+0000 7f499f264700 1 -- 172.21.15.104:0/2678353518 shutdown_connections 2023-07-14T11:54:39.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.139+0000 7f499f264700 1 --2- 172.21.15.104:0/2678353518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f499806c220 0x7f49980cbd20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.139+0000 7f499f264700 1 -- 172.21.15.104:0/2678353518 >> 172.21.15.104:0/2678353518 conn(0x7f49980afc60 msgr2=0x7f49980b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:39.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.139+0000 7f499f264700 1 -- 172.21.15.104:0/2678353518 shutdown_connections 2023-07-14T11:54:39.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.139+0000 7f499f264700 1 -- 172.21.15.104:0/2678353518 wait complete. 2023-07-14T11:54:39.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.143+0000 7f499f264700 1 Processor -- start 2023-07-14T11:54:39.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.143+0000 7f499f264700 1 -- start start 2023-07-14T11:54:39.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.143+0000 7f499f264700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499806c220 0x7f499814b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.143+0000 7f499f264700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49980cc260 0x7f49980ac770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.143+0000 7f499f264700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4998158a20 0x7f49980b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.143+0000 7f499f264700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49980b2b00 con 0x7f499806c220 2023-07-14T11:54:39.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.143+0000 7f499f264700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49980ad210 con 0x7f4998158a20 2023-07-14T11:54:39.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.143+0000 7f499f264700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49980b1d40 con 0x7f49980cc260 2023-07-14T11:54:39.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.143+0000 7f499e262700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499806c220 0x7f499814b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.143+0000 7f499e262700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499806c220 0x7f499814b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44068/0 (socket says 172.21.15.104:44068) 2023-07-14T11:54:39.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.143+0000 7f499e262700 1 -- 172.21.15.104:0/1307671910 learned_addr learned my addr 172.21.15.104:0/1307671910 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:39.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.143+0000 7f499ea63700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4998158a20 0x7f49980b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.143+0000 7f499da61700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49980cc260 0x7f49980ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.147+0000 7f499e262700 1 -- 172.21.15.104:0/1307671910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49980cc260 msgr2=0x7f49980ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.147+0000 7f499e262700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49980cc260 0x7f49980ac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.147+0000 7f499e262700 1 -- 172.21.15.104:0/1307671910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4998158a20 msgr2=0x7f49980b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.147+0000 7f499e262700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4998158a20 0x7f49980b0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.147+0000 7f499e262700 1 -- 172.21.15.104:0/1307671910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49880094b0 con 0x7f499806c220 2023-07-14T11:54:39.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.147+0000 7f499ea63700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4998158a20 0x7f49980b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:39.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.147+0000 7f499da61700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49980cc260 0x7f49980ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:39.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.147+0000 7f499e262700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499806c220 0x7f499814b5c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f499400c4d0 tx=0x7f499400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:39.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.147+0000 7f498effd700 1 -- 172.21.15.104:0/1307671910 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994005d90 con 0x7f499806c220 2023-07-14T11:54:39.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.147+0000 7f498effd700 1 -- 172.21.15.104:0/1307671910 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49940070d0 con 0x7f499806c220 2023-07-14T11:54:39.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.147+0000 7f498effd700 1 -- 172.21.15.104:0/1307671910 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994007480 con 0x7f499806c220 2023-07-14T11:54:39.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.147+0000 7f499f264700 1 -- 172.21.15.104:0/1307671910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49980b2020 con 0x7f499806c220 2023-07-14T11:54:39.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.147+0000 7f499f264700 1 -- 172.21.15.104:0/1307671910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49980b25e0 con 0x7f499806c220 2023-07-14T11:54:39.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.147+0000 7f499f264700 1 -- 172.21.15.104:0/1307671910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49980b27d0 con 0x7f499806c220 2023-07-14T11:54:39.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.151+0000 7f498effd700 1 -- 172.21.15.104:0/1307671910 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f49940075e0 con 0x7f499806c220 2023-07-14T11:54:39.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.151+0000 7f498effd700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f497c0662f0 0x7f497c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.151+0000 7f498effd700 1 -- 172.21.15.104:0/1307671910 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f49940836b0 con 0x7f499806c220 2023-07-14T11:54:39.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.151+0000 7f499da61700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f497c0662f0 0x7f497c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.151+0000 7f499da61700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f497c0662f0 0x7f497c0687a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f499806c630 tx=0x7f4988008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:39.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.159+0000 7f498effd700 1 -- 172.21.15.104:0/1307671910 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f49940506d0 con 0x7f499806c220 2023-07-14T11:54:39.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.339+0000 7f499f264700 1 -- 172.21.15.104:0/1307671910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f499806cf70 con 0x7f499806c220 2023-07-14T11:54:39.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.339+0000 7f498effd700 1 -- 172.21.15.104:0/1307671910 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4994053cf0 con 0x7f499806c220 2023-07-14T11:54:39.324 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:39.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.339+0000 7f499f264700 1 -- 172.21.15.104:0/1307671910 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f497c0662f0 msgr2=0x7f497c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.339+0000 7f499f264700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f497c0662f0 0x7f497c0687a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f499806c630 tx=0x7f4988008030 comp rx=0 tx=0).stop 2023-07-14T11:54:39.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.339+0000 7f499f264700 1 -- 172.21.15.104:0/1307671910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499806c220 msgr2=0x7f499814b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.339+0000 7f499f264700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499806c220 0x7f499814b5c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f499400c4d0 tx=0x7f499400c890 comp rx=0 tx=0).stop 2023-07-14T11:54:39.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.339+0000 7f499f264700 1 -- 172.21.15.104:0/1307671910 shutdown_connections 2023-07-14T11:54:39.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.339+0000 7f499f264700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f497c0662f0 0x7f497c0687a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.339+0000 7f499f264700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4998158a20 0x7f49980b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.339+0000 7f499f264700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49980cc260 0x7f49980ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.339+0000 7f499f264700 1 --2- 172.21.15.104:0/1307671910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499806c220 0x7f499814b5c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.339+0000 7f499f264700 1 -- 172.21.15.104:0/1307671910 >> 172.21.15.104:0/1307671910 conn(0x7f49980afc60 msgr2=0x7f49980c94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:39.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.339+0000 7f499f264700 1 -- 172.21.15.104:0/1307671910 shutdown_connections 2023-07-14T11:54:39.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.339+0000 7f499f264700 1 -- 172.21.15.104:0/1307671910 wait complete. 2023-07-14T11:54:39.346 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:54:39.523 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.539+0000 7fac38b19700 1 Processor -- start 2023-07-14T11:54:39.524 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.539+0000 7fac38b19700 1 -- start start 2023-07-14T11:54:39.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.539+0000 7fac38b19700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fac340afd80 0x7fac3415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.539+0000 7fac38b19700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac340c8950 0x7fac340c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.539+0000 7fac38b19700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac340c9290 con 0x7fac3415df10 2023-07-14T11:54:39.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.539+0000 7fac38b19700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac3415e830 con 0x7fac340afd80 2023-07-14T11:54:39.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.539+0000 7fac38b19700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac34070a30 con 0x7fac340c8950 2023-07-14T11:54:39.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac31d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fac340afd80 0x7fac3415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac31d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fac340afd80 0x7fac3415e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44082/0 (socket says 172.21.15.104:44082) 2023-07-14T11:54:39.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac31d9b700 1 -- 172.21.15.104:0/2966916228 learned_addr learned my addr 172.21.15.104:0/2966916228 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:39.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac3259c700 1 --1- 172.21.15.104:0/2966916228 >> v1:172.21.15.121:6789/0 conn(0x7fac3415df10 0x7fac340711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:41674/0 (socket says 172.21.15.104:41674) 2023-07-14T11:54:39.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac32d9d700 1 --2- 172.21.15.104:0/2966916228 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac340c8950 0x7fac340c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac31d9b700 1 -- 172.21.15.104:0/2966916228 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac340c8950 msgr2=0x7fac340c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:39.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac31d9b700 1 --2- 172.21.15.104:0/2966916228 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac340c8950 0x7fac340c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac31d9b700 1 -- 172.21.15.104:0/2966916228 >> v1:172.21.15.121:6789/0 conn(0x7fac3415df10 legacy=0x7fac340711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac31d9b700 1 -- 172.21.15.104:0/2966916228 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac340b0210 con 0x7fac340afd80 2023-07-14T11:54:39.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac3159a700 1 -- 172.21.15.104:0/2966916228 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351419266 0 0) 0x7fac24001110 con 0x7fac3415df10 2023-07-14T11:54:39.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac31d9b700 1 --2- 172.21.15.104:0/2966916228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac340afd80 0x7fac3415e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fac28009910 tx=0x7fac28009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300571f02ae173ee server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:39.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac3159a700 1 -- 172.21.15.104:0/2966916228 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac28011830 con 0x7fac340afd80 2023-07-14T11:54:39.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac3159a700 1 -- 172.21.15.104:0/2966916228 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac28011990 con 0x7fac340afd80 2023-07-14T11:54:39.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac3159a700 1 -- 172.21.15.104:0/2966916228 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac28011d40 con 0x7fac340afd80 2023-07-14T11:54:39.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac38b19700 1 -- 172.21.15.104:0/2966916228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac340afd80 msgr2=0x7fac3415e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac38b19700 1 --2- 172.21.15.104:0/2966916228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac340afd80 0x7fac3415e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fac28009910 tx=0x7fac28009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:39.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac38b19700 1 -- 172.21.15.104:0/2966916228 shutdown_connections 2023-07-14T11:54:39.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac38b19700 1 --2- 172.21.15.104:0/2966916228 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac340c8950 0x7fac340c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac38b19700 1 --2- 172.21.15.104:0/2966916228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac340afd80 0x7fac3415e2f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac38b19700 1 -- 172.21.15.104:0/2966916228 >> 172.21.15.104:0/2966916228 conn(0x7fac340cad80 msgr2=0x7fac340cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:39.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac38b19700 1 -- 172.21.15.104:0/2966916228 shutdown_connections 2023-07-14T11:54:39.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.543+0000 7fac38b19700 1 -- 172.21.15.104:0/2966916228 wait complete. 2023-07-14T11:54:39.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.547+0000 7fac38b19700 1 Processor -- start 2023-07-14T11:54:39.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.547+0000 7fac38b19700 1 -- start start 2023-07-14T11:54:39.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac38b19700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac340afd80 0x7fac340b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac38b19700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac340c8950 0x7fac3414a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac38b19700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac340ae4a0 0x7fac341506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac38b19700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac341515d0 con 0x7fac340c8950 2023-07-14T11:54:39.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac38b19700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3414f520 con 0x7fac340afd80 2023-07-14T11:54:39.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac38b19700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac3414e630 con 0x7fac340ae4a0 2023-07-14T11:54:39.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac31d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac340c8950 0x7fac3414a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac31d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac340c8950 0x7fac3414a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44088/0 (socket says 172.21.15.104:44088) 2023-07-14T11:54:39.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac31d9b700 1 -- 172.21.15.104:0/4020952717 learned_addr learned my addr 172.21.15.104:0/4020952717 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:39.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac32d9d700 1 --2- 172.21.15.104:0/4020952717 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac340ae4a0 0x7fac341506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac3259c700 1 --2- 172.21.15.104:0/4020952717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac340afd80 0x7fac340b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac31d9b700 1 -- 172.21.15.104:0/4020952717 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac340ae4a0 msgr2=0x7fac341506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac31d9b700 1 --2- 172.21.15.104:0/4020952717 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac340ae4a0 0x7fac341506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac31d9b700 1 -- 172.21.15.104:0/4020952717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac340afd80 msgr2=0x7fac340b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac31d9b700 1 --2- 172.21.15.104:0/4020952717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac340afd80 0x7fac340b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac31d9b700 1 -- 172.21.15.104:0/4020952717 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac28009570 con 0x7fac340c8950 2023-07-14T11:54:39.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac31d9b700 1 --2- 172.21.15.104:0/4020952717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac340c8950 0x7fac3414a860 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fac28014040 tx=0x7fac28004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:39.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac22ffd700 1 -- 172.21.15.104:0/4020952717 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2800ba70 con 0x7fac340c8950 2023-07-14T11:54:39.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac22ffd700 1 -- 172.21.15.104:0/4020952717 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2801e070 con 0x7fac340c8950 2023-07-14T11:54:39.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac22ffd700 1 -- 172.21.15.104:0/4020952717 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2800be50 con 0x7fac340c8950 2023-07-14T11:54:39.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac38b19700 1 -- 172.21.15.104:0/4020952717 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac3414e910 con 0x7fac340c8950 2023-07-14T11:54:39.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.551+0000 7fac38b19700 1 -- 172.21.15.104:0/4020952717 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3414eed0 con 0x7fac340c8950 2023-07-14T11:54:39.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.555+0000 7fac38b19700 1 -- 172.21.15.104:0/4020952717 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac3414f6d0 con 0x7fac340c8950 2023-07-14T11:54:39.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.555+0000 7fac22ffd700 1 -- 172.21.15.104:0/4020952717 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fac2801a420 con 0x7fac340c8950 2023-07-14T11:54:39.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.555+0000 7fac22ffd700 1 --2- 172.21.15.104:0/4020952717 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fac140662f0 0x7fac140687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.555+0000 7fac22ffd700 1 -- 172.21.15.104:0/4020952717 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fac28085020 con 0x7fac340c8950 2023-07-14T11:54:39.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.555+0000 7fac3259c700 1 --2- 172.21.15.104:0/4020952717 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fac140662f0 0x7fac140687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.555+0000 7fac3259c700 1 --2- 172.21.15.104:0/4020952717 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fac140662f0 0x7fac140687a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fac24000f60 tx=0x7fac24007a60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:39.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.563+0000 7fac22ffd700 1 -- 172.21.15.104:0/4020952717 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fac3414f6d0 con 0x7fac340c8950 2023-07-14T11:54:39.724 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.739+0000 7fac38b19700 1 -- 172.21.15.104:0/4020952717 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac3414f690 con 0x7fac340c8950 2023-07-14T11:54:39.724 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.739+0000 7fac22ffd700 1 -- 172.21.15.104:0/4020952717 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac280200a0 con 0x7fac340c8950 2023-07-14T11:54:39.725 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:54:39.727 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.743+0000 7fac38b19700 1 -- 172.21.15.104:0/4020952717 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fac140662f0 msgr2=0x7fac140687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.727 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.743+0000 7fac38b19700 1 --2- 172.21.15.104:0/4020952717 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fac140662f0 0x7fac140687a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fac24000f60 tx=0x7fac24007a60 comp rx=0 tx=0).stop 2023-07-14T11:54:39.727 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.743+0000 7fac38b19700 1 -- 172.21.15.104:0/4020952717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac340c8950 msgr2=0x7fac3414a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.727 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.743+0000 7fac38b19700 1 --2- 172.21.15.104:0/4020952717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac340c8950 0x7fac3414a860 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fac28014040 tx=0x7fac28004770 comp rx=0 tx=0).stop 2023-07-14T11:54:39.728 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.743+0000 7fac38b19700 1 -- 172.21.15.104:0/4020952717 shutdown_connections 2023-07-14T11:54:39.728 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.743+0000 7fac38b19700 1 --2- 172.21.15.104:0/4020952717 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fac140662f0 0x7fac140687a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.728 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.743+0000 7fac38b19700 1 --2- 172.21.15.104:0/4020952717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac340c8950 0x7fac3414a860 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.728 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.743+0000 7fac38b19700 1 --2- 172.21.15.104:0/4020952717 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac340ae4a0 0x7fac341506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.729 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.743+0000 7fac38b19700 1 --2- 172.21.15.104:0/4020952717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac340afd80 0x7fac340b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.729 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.743+0000 7fac38b19700 1 -- 172.21.15.104:0/4020952717 >> 172.21.15.104:0/4020952717 conn(0x7fac340cad80 msgr2=0x7fac340c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:39.729 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.743+0000 7fac38b19700 1 -- 172.21.15.104:0/4020952717 shutdown_connections 2023-07-14T11:54:39.730 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.743+0000 7fac38b19700 1 -- 172.21.15.104:0/4020952717 wait complete. 2023-07-14T11:54:39.745 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:54:39.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.939+0000 7f52db54d700 1 Processor -- start 2023-07-14T11:54:39.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.939+0000 7f52db54d700 1 -- start start 2023-07-14T11:54:39.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.939+0000 7f52db54d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52d40b1c50 0x7f52d4071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.939+0000 7f52db54d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f52d40d0210 0x7f52d40d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.939+0000 7f52db54d700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d415c420 con 0x7f52d40cf000 2023-07-14T11:54:39.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.939+0000 7f52db54d700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d4070840 con 0x7f52d40d0210 2023-07-14T11:54:39.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.939+0000 7f52db54d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d4070b20 con 0x7f52d40b1c50 2023-07-14T11:54:39.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.939+0000 7f52d92e9700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52d40b1c50 0x7f52d4071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.939+0000 7f52d8ae8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f52d40d0210 0x7f52d40d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52d92e9700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52d40b1c50 0x7f52d4071600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:39010/0 (socket says 172.21.15.104:39010) 2023-07-14T11:54:39.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52d9aea700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f52d40cf000 0x7f52d40ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:41676/0 (socket says 172.21.15.104:41676) 2023-07-14T11:54:39.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52d92e9700 1 -- 172.21.15.104:0/933181287 learned_addr learned my addr 172.21.15.104:0/933181287 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:39.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52cbfff700 1 -- 172.21.15.104:0/933181287 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2240778882 0 0) 0x7f52d415c420 con 0x7f52d40cf000 2023-07-14T11:54:39.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52cbfff700 1 -- 172.21.15.104:0/933181287 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52c00035d0 con 0x7f52d40cf000 2023-07-14T11:54:39.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52d8ae8700 1 -- 172.21.15.104:0/933181287 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52d40b1c50 msgr2=0x7f52d4071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:39.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52d8ae8700 1 --2- 172.21.15.104:0/933181287 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52d40b1c50 0x7f52d4071600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52d8ae8700 1 -- 172.21.15.104:0/933181287 >> v1:172.21.15.121:6789/0 conn(0x7f52d40cf000 legacy=0x7f52d40ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52d8ae8700 1 -- 172.21.15.104:0/933181287 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d4070e00 con 0x7f52d40d0210 2023-07-14T11:54:39.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52d8ae8700 1 --2- 172.21.15.104:0/933181287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52d40d0210 0x7f52d40d0640 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009850 tx=0x7f52c4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85585a6841cfae62 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:39.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52cbfff700 1 -- 172.21.15.104:0/933181287 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4011830 con 0x7f52d40d0210 2023-07-14T11:54:39.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52cbfff700 1 -- 172.21.15.104:0/933181287 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c4011990 con 0x7f52d40d0210 2023-07-14T11:54:39.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52cbfff700 1 -- 172.21.15.104:0/933181287 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4011d40 con 0x7f52d40d0210 2023-07-14T11:54:39.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52db54d700 1 -- 172.21.15.104:0/933181287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52d40d0210 msgr2=0x7f52d40d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52db54d700 1 --2- 172.21.15.104:0/933181287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52d40d0210 0x7f52d40d0640 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009850 tx=0x7f52c4009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:39.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52db54d700 1 -- 172.21.15.104:0/933181287 shutdown_connections 2023-07-14T11:54:39.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52db54d700 1 --2- 172.21.15.104:0/933181287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52d40d0210 0x7f52d40d0640 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52db54d700 1 --2- 172.21.15.104:0/933181287 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52d40b1c50 0x7f52d4071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52db54d700 1 -- 172.21.15.104:0/933181287 >> 172.21.15.104:0/933181287 conn(0x7f52d409b9c0 msgr2=0x7f52d40d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:39.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52db54d700 1 -- 172.21.15.104:0/933181287 shutdown_connections 2023-07-14T11:54:39.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.943+0000 7f52db54d700 1 -- 172.21.15.104:0/933181287 wait complete. 2023-07-14T11:54:39.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.947+0000 7f52db54d700 1 Processor -- start 2023-07-14T11:54:39.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.947+0000 7f52db54d700 1 -- start start 2023-07-14T11:54:39.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.947+0000 7f52db54d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52d40b1c50 0x7f52d409f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.947+0000 7f52db54d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52d40cf000 0x7f52d40b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52db54d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52d40d0210 0x7f52d40b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52db54d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41498c0 con 0x7f52d40b1c50 2023-07-14T11:54:39.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52db54d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40a0200 con 0x7f52d40d0210 2023-07-14T11:54:39.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52db54d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41f1580 con 0x7f52d40cf000 2023-07-14T11:54:39.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52d92e9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52d40b1c50 0x7f52d409f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52d92e9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52d40b1c50 0x7f52d409f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44104/0 (socket says 172.21.15.104:44104) 2023-07-14T11:54:39.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52d92e9700 1 -- 172.21.15.104:0/444833178 learned_addr learned my addr 172.21.15.104:0/444833178 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:39.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52d8ae8700 1 --2- 172.21.15.104:0/444833178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52d40cf000 0x7f52d40b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52d9aea700 1 --2- 172.21.15.104:0/444833178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52d40d0210 0x7f52d40b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52d92e9700 1 -- 172.21.15.104:0/444833178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52d40cf000 msgr2=0x7f52d40b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52d92e9700 1 --2- 172.21.15.104:0/444833178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52d40cf000 0x7f52d40b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52d92e9700 1 -- 172.21.15.104:0/444833178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52d40d0210 msgr2=0x7f52d40b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52d92e9700 1 --2- 172.21.15.104:0/444833178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52d40d0210 0x7f52d40b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52d92e9700 1 -- 172.21.15.104:0/444833178 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c40094b0 con 0x7f52d40b1c50 2023-07-14T11:54:39.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52d92e9700 1 --2- 172.21.15.104:0/444833178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52d40b1c50 0x7f52d409f180 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f52d000e930 tx=0x7f52d000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:39.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52c9ffb700 1 -- 172.21.15.104:0/444833178 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00096c0 con 0x7f52d40b1c50 2023-07-14T11:54:39.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52c9ffb700 1 -- 172.21.15.104:0/444833178 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d000ee80 con 0x7f52d40b1c50 2023-07-14T11:54:39.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52db54d700 1 -- 172.21.15.104:0/444833178 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d409f6c0 con 0x7f52d40b1c50 2023-07-14T11:54:39.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52c9ffb700 1 -- 172.21.15.104:0/444833178 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d000ca90 con 0x7f52d40b1c50 2023-07-14T11:54:39.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52db54d700 1 -- 172.21.15.104:0/444833178 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d409fc40 con 0x7f52d40b1c50 2023-07-14T11:54:39.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.951+0000 7f52db54d700 1 -- 172.21.15.104:0/444833178 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d40b7260 con 0x7f52d40b1c50 2023-07-14T11:54:39.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.955+0000 7f52c9ffb700 1 -- 172.21.15.104:0/444833178 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f52d000cbf0 con 0x7f52d40b1c50 2023-07-14T11:54:39.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.955+0000 7f52c9ffb700 1 --2- 172.21.15.104:0/444833178 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52b80662f0 0x7f52b80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:39.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.955+0000 7f52c9ffb700 1 -- 172.21.15.104:0/444833178 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f52d0083fe0 con 0x7f52d40b1c50 2023-07-14T11:54:39.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.955+0000 7f52d8ae8700 1 --2- 172.21.15.104:0/444833178 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52b80662f0 0x7f52b80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:39.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.955+0000 7f52d8ae8700 1 --2- 172.21.15.104:0/444833178 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52b80662f0 0x7f52b80687a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f52c40090b0 tx=0x7f52c401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:39.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:39.963+0000 7f52c9ffb700 1 -- 172.21.15.104:0/444833178 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f52d0054bd0 con 0x7f52d40b1c50 2023-07-14T11:54:40.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.131+0000 7f52db54d700 1 -- 172.21.15.104:0/444833178 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f52d4149b30 con 0x7f52d40b1c50 2023-07-14T11:54:40.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.131+0000 7f52c9ffb700 1 -- 172.21.15.104:0/444833178 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f52d00170a0 con 0x7f52d40b1c50 2023-07-14T11:54:40.119 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:40.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.135+0000 7f52db54d700 1 -- 172.21.15.104:0/444833178 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52b80662f0 msgr2=0x7f52b80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.135+0000 7f52db54d700 1 --2- 172.21.15.104:0/444833178 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52b80662f0 0x7f52b80687a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f52c40090b0 tx=0x7f52c401c040 comp rx=0 tx=0).stop 2023-07-14T11:54:40.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.135+0000 7f52db54d700 1 -- 172.21.15.104:0/444833178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52d40b1c50 msgr2=0x7f52d409f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.135+0000 7f52db54d700 1 --2- 172.21.15.104:0/444833178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52d40b1c50 0x7f52d409f180 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f52d000e930 tx=0x7f52d000ec40 comp rx=0 tx=0).stop 2023-07-14T11:54:40.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.135+0000 7f52db54d700 1 -- 172.21.15.104:0/444833178 shutdown_connections 2023-07-14T11:54:40.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.135+0000 7f52db54d700 1 --2- 172.21.15.104:0/444833178 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52b80662f0 0x7f52b80687a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.135+0000 7f52db54d700 1 --2- 172.21.15.104:0/444833178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52d40d0210 0x7f52d40b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.135+0000 7f52db54d700 1 --2- 172.21.15.104:0/444833178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52d40cf000 0x7f52d40b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.135+0000 7f52db54d700 1 --2- 172.21.15.104:0/444833178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52d40b1c50 0x7f52d409f180 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.135+0000 7f52db54d700 1 -- 172.21.15.104:0/444833178 >> 172.21.15.104:0/444833178 conn(0x7f52d409b9c0 msgr2=0x7f52d415ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:40.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.135+0000 7f52db54d700 1 -- 172.21.15.104:0/444833178 shutdown_connections 2023-07-14T11:54:40.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.135+0000 7f52db54d700 1 -- 172.21.15.104:0/444833178 wait complete. 2023-07-14T11:54:40.142 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:54:40.319 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.331+0000 7fcd4f59e700 1 Processor -- start 2023-07-14T11:54:40.319 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.335+0000 7fcd4f59e700 1 -- start start 2023-07-14T11:54:40.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.335+0000 7fcd4f59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcd50071000 0x7fcd5015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:40.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.335+0000 7fcd4f59e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5015d230 con 0x7fcd501599d0 2023-07-14T11:54:40.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.335+0000 7fcd4f59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd500cde10 con 0x7fcd500caa20 2023-07-14T11:54:40.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.335+0000 7fcd4f59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd500cd520 con 0x7fcd50071000 2023-07-14T11:54:40.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.335+0000 7fcd4e59c700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fcd501599d0 0x7fcd500c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:36796/0 (socket says 172.21.15.104:36796) 2023-07-14T11:54:40.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.335+0000 7fcd4e59c700 1 -- 172.21.15.104:0/3282310931 learned_addr learned my addr 172.21.15.104:0/3282310931 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:40.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.335+0000 7fcd47fff700 1 --2- 172.21.15.104:0/3282310931 >> v2:172.21.15.121:3300/0 conn(0x7fcd50071000 0x7fcd5015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:40.324 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd4dd9b700 1 -- 172.21.15.104:0/3282310931 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918112945 0 0) 0x7fcd5015d230 con 0x7fcd501599d0 2023-07-14T11:54:40.325 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd4dd9b700 1 -- 172.21.15.104:0/3282310931 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd3c0035d0 con 0x7fcd501599d0 2023-07-14T11:54:40.325 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd4dd9b700 1 -- 172.21.15.104:0/3282310931 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 862398266 0 0) 0x7fcd500cde10 con 0x7fcd500caa20 2023-07-14T11:54:40.325 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd4dd9b700 1 -- 172.21.15.104:0/3282310931 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd5015d230 con 0x7fcd500caa20 2023-07-14T11:54:40.325 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd47fff700 1 -- 172.21.15.104:0/3282310931 >> v1:172.21.15.121:6789/0 conn(0x7fcd500caa20 legacy=0x7fcd500cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.325 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd47fff700 1 -- 172.21.15.104:0/3282310931 >> v1:172.21.15.104:6789/0 conn(0x7fcd501599d0 legacy=0x7fcd500c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.325 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd47fff700 1 -- 172.21.15.104:0/3282310931 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd500715e0 con 0x7fcd50071000 2023-07-14T11:54:40.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd47fff700 1 --2- 172.21.15.104:0/3282310931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd50071000 0x7fcd5015ccf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009850 tx=0x7fcd40009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41ca7f11a38ff9a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:40.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd4dd9b700 1 -- 172.21.15.104:0/3282310931 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40011830 con 0x7fcd50071000 2023-07-14T11:54:40.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd4dd9b700 1 -- 172.21.15.104:0/3282310931 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd40011990 con 0x7fcd50071000 2023-07-14T11:54:40.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd4dd9b700 1 -- 172.21.15.104:0/3282310931 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40011d40 con 0x7fcd50071000 2023-07-14T11:54:40.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd4f59e700 1 -- 172.21.15.104:0/3282310931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd50071000 msgr2=0x7fcd5015ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd4f59e700 1 --2- 172.21.15.104:0/3282310931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd50071000 0x7fcd5015ccf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009850 tx=0x7fcd40009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:40.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd4f59e700 1 -- 172.21.15.104:0/3282310931 shutdown_connections 2023-07-14T11:54:40.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd4f59e700 1 --2- 172.21.15.104:0/3282310931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd50071000 0x7fcd5015ccf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd4f59e700 1 -- 172.21.15.104:0/3282310931 >> 172.21.15.104:0/3282310931 conn(0x7fcd5006e960 msgr2=0x7fcd5006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:40.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd4f59e700 1 -- 172.21.15.104:0/3282310931 shutdown_connections 2023-07-14T11:54:40.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.339+0000 7fcd4f59e700 1 -- 172.21.15.104:0/3282310931 wait complete. 2023-07-14T11:54:40.329 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.343+0000 7fcd4f59e700 1 Processor -- start 2023-07-14T11:54:40.329 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.343+0000 7fcd4f59e700 1 -- start start 2023-07-14T11:54:40.330 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.343+0000 7fcd4f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd50071000 0x7fcd5014af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:40.330 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.343+0000 7fcd4f59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd500caa20 0x7fcd5015d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:40.331 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.343+0000 7fcd4f59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd501599d0 0x7fcd5014ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:40.331 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.343+0000 7fcd4f59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd501602c0 con 0x7fcd500caa20 2023-07-14T11:54:40.331 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.343+0000 7fcd4f59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5014ad90 con 0x7fcd50071000 2023-07-14T11:54:40.331 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.343+0000 7fcd4f59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500add90 con 0x7fcd501599d0 2023-07-14T11:54:40.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.343+0000 7fcd47fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd500caa20 0x7fcd5015d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:40.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.343+0000 7fcd47fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd500caa20 0x7fcd5015d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59228/0 (socket says 172.21.15.104:59228) 2023-07-14T11:54:40.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.343+0000 7fcd47fff700 1 -- 172.21.15.104:0/2734384972 learned_addr learned my addr 172.21.15.104:0/2734384972 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:40.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.343+0000 7fcd4e59c700 1 --2- 172.21.15.104:0/2734384972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd50071000 0x7fcd5014af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:40.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.347+0000 7fcd4ed9d700 1 --2- 172.21.15.104:0/2734384972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd501599d0 0x7fcd5014ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:40.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.347+0000 7fcd47fff700 1 -- 172.21.15.104:0/2734384972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd501599d0 msgr2=0x7fcd5014ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.347+0000 7fcd47fff700 1 --2- 172.21.15.104:0/2734384972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd501599d0 0x7fcd5014ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.347+0000 7fcd47fff700 1 -- 172.21.15.104:0/2734384972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd50071000 msgr2=0x7fcd5014af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.347+0000 7fcd47fff700 1 --2- 172.21.15.104:0/2734384972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd50071000 0x7fcd5014af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.347+0000 7fcd47fff700 1 -- 172.21.15.104:0/2734384972 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd400094b0 con 0x7fcd500caa20 2023-07-14T11:54:40.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.347+0000 7fcd47fff700 1 --2- 172.21.15.104:0/2734384972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd500caa20 0x7fcd5015d5f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009eb0 tx=0x7fcd40004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:40.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.347+0000 7fcd46ffd700 1 -- 172.21.15.104:0/2734384972 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4001a990 con 0x7fcd500caa20 2023-07-14T11:54:40.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.347+0000 7fcd46ffd700 1 -- 172.21.15.104:0/2734384972 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4001f070 con 0x7fcd500caa20 2023-07-14T11:54:40.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.347+0000 7fcd4f59e700 1 -- 172.21.15.104:0/2734384972 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd500ae070 con 0x7fcd500caa20 2023-07-14T11:54:40.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.347+0000 7fcd46ffd700 1 -- 172.21.15.104:0/2734384972 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4001ad30 con 0x7fcd500caa20 2023-07-14T11:54:40.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.347+0000 7fcd4f59e700 1 -- 172.21.15.104:0/2734384972 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd500ae630 con 0x7fcd500caa20 2023-07-14T11:54:40.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.347+0000 7fcd4f59e700 1 -- 172.21.15.104:0/2734384972 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd500ae820 con 0x7fcd500caa20 2023-07-14T11:54:40.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.351+0000 7fcd46ffd700 1 -- 172.21.15.104:0/2734384972 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fcd4001b430 con 0x7fcd500caa20 2023-07-14T11:54:40.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.351+0000 7fcd46ffd700 1 --2- 172.21.15.104:0/2734384972 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcd300662f0 0x7fcd300687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:40.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.351+0000 7fcd46ffd700 1 -- 172.21.15.104:0/2734384972 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcd40084f40 con 0x7fcd500caa20 2023-07-14T11:54:40.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.351+0000 7fcd4e59c700 1 --2- 172.21.15.104:0/2734384972 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcd300662f0 0x7fcd300687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:40.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.351+0000 7fcd4e59c700 1 --2- 172.21.15.104:0/2734384972 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcd300662f0 0x7fcd300687a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcd38000f60 tx=0x7fcd38007e20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:40.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.355+0000 7fcd46ffd700 1 -- 172.21.15.104:0/2734384972 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fcd40052120 con 0x7fcd500caa20 2023-07-14T11:54:40.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.539+0000 7fcd4f59e700 1 -- 172.21.15.104:0/2734384972 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcd5004e500 con 0x7fcd500caa20 2023-07-14T11:54:40.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.539+0000 7fcd46ffd700 1 -- 172.21.15.104:0/2734384972 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcd40015080 con 0x7fcd500caa20 2023-07-14T11:54:40.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.543+0000 7fcd4f59e700 1 -- 172.21.15.104:0/2734384972 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcd300662f0 msgr2=0x7fcd300687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.543+0000 7fcd4f59e700 1 --2- 172.21.15.104:0/2734384972 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcd300662f0 0x7fcd300687a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcd38000f60 tx=0x7fcd38007e20 comp rx=0 tx=0).stop 2023-07-14T11:54:40.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.543+0000 7fcd4f59e700 1 -- 172.21.15.104:0/2734384972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd500caa20 msgr2=0x7fcd5015d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.543+0000 7fcd4f59e700 1 --2- 172.21.15.104:0/2734384972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd500caa20 0x7fcd5015d5f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009eb0 tx=0x7fcd40004770 comp rx=0 tx=0).stop 2023-07-14T11:54:40.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.547+0000 7fcd4f59e700 1 -- 172.21.15.104:0/2734384972 shutdown_connections 2023-07-14T11:54:40.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.547+0000 7fcd4f59e700 1 --2- 172.21.15.104:0/2734384972 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcd300662f0 0x7fcd300687a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.547+0000 7fcd4f59e700 1 --2- 172.21.15.104:0/2734384972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd500caa20 0x7fcd5015d5f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.547+0000 7fcd4f59e700 1 --2- 172.21.15.104:0/2734384972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd501599d0 0x7fcd5014ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.547+0000 7fcd4f59e700 1 --2- 172.21.15.104:0/2734384972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd50071000 0x7fcd5014af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.547+0000 7fcd4f59e700 1 -- 172.21.15.104:0/2734384972 >> 172.21.15.104:0/2734384972 conn(0x7fcd5006e960 msgr2=0x7fcd5006d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:40.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.547+0000 7fcd4f59e700 1 -- 172.21.15.104:0/2734384972 shutdown_connections 2023-07-14T11:54:40.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.547+0000 7fcd4f59e700 1 -- 172.21.15.104:0/2734384972 wait complete. 2023-07-14T11:54:40.535 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:54:40.552 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi121.front.sepia.ceph.com... 2023-07-14T11:54:40.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:40.554 DEBUG:teuthology.orchestra.run.smithi121:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-07-14T11:54:40.590 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:54:40.590 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:54:40.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:54:40.591 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-14T11:54:40.652 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-14T11:54:40.653 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:54:40.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.847+0000 7f22bb59e700 1 Processor -- start 2023-07-14T11:54:40.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.847+0000 7f22bb59e700 1 -- start start 2023-07-14T11:54:40.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22bb59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22bc0c9d90 0x7f22bc0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:40.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22bb59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f22bc15d7f0 0x7f22bc15dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:40.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22bb59e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22bc0b0ef0 con 0x7f22bc159f20 2023-07-14T11:54:40.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22bb59e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22bc0ce9c0 con 0x7f22bc15d7f0 2023-07-14T11:54:40.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22bb59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0cee50 con 0x7f22bc0c9d90 2023-07-14T11:54:40.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22b9d9b700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f22bc159f20 0x7f22bc0caa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:36798/0 (socket says 172.21.15.104:36798) 2023-07-14T11:54:40.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22b9d9b700 1 -- 172.21.15.104:0/679262949 learned_addr learned my addr 172.21.15.104:0/679262949 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:40.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22bad9d700 1 --2- 172.21.15.104:0/679262949 >> v2:172.21.15.104:3300/0 conn(0x7f22bc15d7f0 0x7f22bc15dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:40.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22ba59c700 1 --2- 172.21.15.104:0/679262949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22bc0c9d90 0x7f22bc0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:40.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22b959a700 1 -- 172.21.15.104:0/679262949 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4179012062 0 0) 0x7f22bc0b0ef0 con 0x7f22bc159f20 2023-07-14T11:54:40.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22b959a700 1 -- 172.21.15.104:0/679262949 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22a40035d0 con 0x7f22bc159f20 2023-07-14T11:54:40.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22bad9d700 1 -- 172.21.15.104:0/679262949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22bc0c9d90 msgr2=0x7f22bc0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:40.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22bad9d700 1 --2- 172.21.15.104:0/679262949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22bc0c9d90 0x7f22bc0ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22bad9d700 1 -- 172.21.15.104:0/679262949 >> v1:172.21.15.104:6789/0 conn(0x7f22bc159f20 legacy=0x7f22bc0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22bad9d700 1 -- 172.21.15.104:0/679262949 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc0ca6a0 con 0x7f22bc15d7f0 2023-07-14T11:54:40.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22ba59c700 1 --2- 172.21.15.104:0/679262949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22bc0c9d90 0x7f22bc0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:40.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22bad9d700 1 --2- 172.21.15.104:0/679262949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc15d7f0 0x7f22bc15dc50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009d90 tx=0x7f22b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45305d7cbfd3090e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:40.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.851+0000 7f22b959a700 1 -- 172.21.15.104:0/679262949 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b400e070 con 0x7f22bc15d7f0 2023-07-14T11:54:40.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.855+0000 7f22b959a700 1 -- 172.21.15.104:0/679262949 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b400bb20 con 0x7f22bc15d7f0 2023-07-14T11:54:40.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.855+0000 7f22b959a700 1 -- 172.21.15.104:0/679262949 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b4012430 con 0x7f22bc15d7f0 2023-07-14T11:54:40.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.855+0000 7f22bb59e700 1 -- 172.21.15.104:0/679262949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc15d7f0 msgr2=0x7f22bc15dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.855+0000 7f22bb59e700 1 --2- 172.21.15.104:0/679262949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc15d7f0 0x7f22bc15dc50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009d90 tx=0x7f22b400b330 comp rx=0 tx=0).stop 2023-07-14T11:54:40.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.855+0000 7f22bb59e700 1 -- 172.21.15.104:0/679262949 shutdown_connections 2023-07-14T11:54:40.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.855+0000 7f22bb59e700 1 --2- 172.21.15.104:0/679262949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc15d7f0 0x7f22bc15dc50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.855+0000 7f22bb59e700 1 --2- 172.21.15.104:0/679262949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22bc0c9d90 0x7f22bc0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.855+0000 7f22bb59e700 1 -- 172.21.15.104:0/679262949 >> 172.21.15.104:0/679262949 conn(0x7f22bc06e960 msgr2=0x7f22bc06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:40.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.855+0000 7f22bb59e700 1 -- 172.21.15.104:0/679262949 shutdown_connections 2023-07-14T11:54:40.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.855+0000 7f22bb59e700 1 -- 172.21.15.104:0/679262949 wait complete. 2023-07-14T11:54:40.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22bb59e700 1 Processor -- start 2023-07-14T11:54:40.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22bb59e700 1 -- start start 2023-07-14T11:54:40.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22bb59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22bc0c9d90 0x7f22bc0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:40.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22bb59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22bc159f20 0x7f22bc0c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:40.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22bb59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc15d7f0 0x7f22bc0c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:40.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22bb59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0c85f0 con 0x7f22bc15d7f0 2023-07-14T11:54:40.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22bb59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0c88d0 con 0x7f22bc0c9d90 2023-07-14T11:54:40.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22bb59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0c8bb0 con 0x7f22bc159f20 2023-07-14T11:54:40.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22b9d9b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22bc159f20 0x7f22bc0c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:40.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22bad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc15d7f0 0x7f22bc0c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:40.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22b9d9b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22bc159f20 0x7f22bc0c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:53688/0 (socket says 172.21.15.104:53688) 2023-07-14T11:54:40.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22b9d9b700 1 -- 172.21.15.104:0/3556805897 learned_addr learned my addr 172.21.15.104:0/3556805897 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:40.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22ba59c700 1 --2- 172.21.15.104:0/3556805897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22bc0c9d90 0x7f22bc0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:40.847 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22b9d9b700 1 -- 172.21.15.104:0/3556805897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22bc0c9d90 msgr2=0x7f22bc0c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22b9d9b700 1 --2- 172.21.15.104:0/3556805897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22bc0c9d90 0x7f22bc0c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22b9d9b700 1 -- 172.21.15.104:0/3556805897 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc15d7f0 msgr2=0x7f22bc0c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:40.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22b9d9b700 1 --2- 172.21.15.104:0/3556805897 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc15d7f0 0x7f22bc0c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:40.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22b9d9b700 1 -- 172.21.15.104:0/3556805897 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac0094b0 con 0x7f22bc159f20 2023-07-14T11:54:40.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22b9d9b700 1 --2- 172.21.15.104:0/3556805897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22bc159f20 0x7f22bc0c4670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f22b000c700 tx=0x7f22b000ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:40.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22aaffd700 1 -- 172.21.15.104:0/3556805897 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0005d90 con 0x7f22bc159f20 2023-07-14T11:54:40.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22aaffd700 1 -- 172.21.15.104:0/3556805897 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b00070d0 con 0x7f22bc159f20 2023-07-14T11:54:40.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22bb59e700 1 -- 172.21.15.104:0/3556805897 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b4009a70 con 0x7f22bc159f20 2023-07-14T11:54:40.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.859+0000 7f22aaffd700 1 -- 172.21.15.104:0/3556805897 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0007480 con 0x7f22bc159f20 2023-07-14T11:54:40.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.863+0000 7f22bb59e700 1 -- 172.21.15.104:0/3556805897 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22bc0c8300 con 0x7f22bc159f20 2023-07-14T11:54:40.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.863+0000 7f22bb59e700 1 -- 172.21.15.104:0/3556805897 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22bc0c8dd0 con 0x7f22bc159f20 2023-07-14T11:54:40.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.863+0000 7f22aaffd700 1 -- 172.21.15.104:0/3556805897 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f22b00075e0 con 0x7f22bc159f20 2023-07-14T11:54:40.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.867+0000 7f22aaffd700 1 --2- 172.21.15.104:0/3556805897 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f229c0662f0 0x7f229c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:40.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.867+0000 7f22aaffd700 1 -- 172.21.15.104:0/3556805897 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f22b0083540 con 0x7f22bc159f20 2023-07-14T11:54:40.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.867+0000 7f22ba59c700 1 --2- 172.21.15.104:0/3556805897 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f229c0662f0 0x7f229c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:40.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.867+0000 7f22ba59c700 1 --2- 172.21.15.104:0/3556805897 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f229c0662f0 0x7f229c0687a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009ce0 tx=0x7f22ac009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:40.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:40.871+0000 7f22aaffd700 1 -- 172.21.15.104:0/3556805897 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f22b00505f0 con 0x7f22bc159f20 2023-07-14T11:54:41.074 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.087+0000 7f22bb59e700 1 -- 172.21.15.104:0/3556805897 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22bc04e500 con 0x7f22bc159f20 2023-07-14T11:54:41.075 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.091+0000 7f22aaffd700 1 -- 172.21.15.104:0/3556805897 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4717 (secure 0 0 0) 0x7f22b0053c10 con 0x7f22bc159f20 2023-07-14T11:54:41.076 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:41.076 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2103358277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2103358277},{"type":"v1","addr":"172.21.15.121:6833","nonce":2103358277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1841065287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1841065287},{"type":"v1","addr":"172.21.15.121:6835","nonce":1841065287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3342053518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3342053518},{"type":"v1","addr":"172.21.15.104:6835","nonce":3342053518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:53:15.029045+0000","modified":"2023-07-14T11:53:41.785898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/2666329866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2666329866},{"type":"v1","addr":"172.21.15.104:6837","nonce":2666329866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:54:41.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.095+0000 7f22bb59e700 1 -- 172.21.15.104:0/3556805897 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f229c0662f0 msgr2=0x7f229c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.095+0000 7f22bb59e700 1 --2- 172.21.15.104:0/3556805897 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f229c0662f0 0x7f229c0687a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009ce0 tx=0x7f22ac009240 comp rx=0 tx=0).stop 2023-07-14T11:54:41.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.095+0000 7f22bb59e700 1 -- 172.21.15.104:0/3556805897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22bc159f20 msgr2=0x7f22bc0c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.095+0000 7f22bb59e700 1 --2- 172.21.15.104:0/3556805897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22bc159f20 0x7f22bc0c4670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f22b000c700 tx=0x7f22b000ca10 comp rx=0 tx=0).stop 2023-07-14T11:54:41.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.095+0000 7f22bb59e700 1 -- 172.21.15.104:0/3556805897 shutdown_connections 2023-07-14T11:54:41.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.095+0000 7f22bb59e700 1 --2- 172.21.15.104:0/3556805897 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f229c0662f0 0x7f229c0687a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.095+0000 7f22bb59e700 1 --2- 172.21.15.104:0/3556805897 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc15d7f0 0x7f22bc0c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.095+0000 7f22bb59e700 1 --2- 172.21.15.104:0/3556805897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22bc159f20 0x7f22bc0c4670 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.095+0000 7f22bb59e700 1 --2- 172.21.15.104:0/3556805897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22bc0c9d90 0x7f22bc0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.095+0000 7f22bb59e700 1 -- 172.21.15.104:0/3556805897 >> 172.21.15.104:0/3556805897 conn(0x7f22bc06e960 msgr2=0x7f22bc15ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:41.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.095+0000 7f22bb59e700 1 -- 172.21.15.104:0/3556805897 shutdown_connections 2023-07-14T11:54:41.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.095+0000 7f22bb59e700 1 -- 172.21.15.104:0/3556805897 wait complete. 2023-07-14T11:54:41.084 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 9 2023-07-14T11:54:41.103 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:41.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.283+0000 7efee32b5700 1 Processor -- start 2023-07-14T11:54:41.271 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.287+0000 7efee32b5700 1 -- start start 2023-07-14T11:54:41.272 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.287+0000 7efee32b5700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efedc15c070 0x7efedc0ad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.272 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.287+0000 7efee32b5700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efedc0ad8b0 0x7efedc0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.273 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.287+0000 7efee32b5700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efedc0cd530 con 0x7efedc0b0270 2023-07-14T11:54:41.273 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.287+0000 7efee32b5700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efedc0b06f0 con 0x7efedc0ad8b0 2023-07-14T11:54:41.273 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.287+0000 7efee32b5700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc15c440 con 0x7efedc15c070 2023-07-14T11:54:41.273 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.287+0000 7efee1852700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7efedc0b0270 0x7efedc15fb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:36806/0 (socket says 172.21.15.104:36806) 2023-07-14T11:54:41.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.287+0000 7efee1852700 1 -- 172.21.15.104:0/1906305506 learned_addr learned my addr 172.21.15.104:0/1906305506 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:41.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.287+0000 7efee0850700 1 --2- 172.21.15.104:0/1906305506 >> v2:172.21.15.121:3300/0 conn(0x7efedc0ad8b0 0x7efedc0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.287+0000 7efee1051700 1 --2- 172.21.15.104:0/1906305506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efedc15c070 0x7efedc0ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.287+0000 7efed3fff700 1 -- 172.21.15.104:0/1906305506 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418298654 0 0) 0x7efedc0cd530 con 0x7efedc0b0270 2023-07-14T11:54:41.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.287+0000 7efed3fff700 1 -- 172.21.15.104:0/1906305506 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efec80035d0 con 0x7efedc0b0270 2023-07-14T11:54:41.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.287+0000 7efee0850700 1 -- 172.21.15.104:0/1906305506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efedc15c070 msgr2=0x7efedc0ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:41.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.287+0000 7efee0850700 1 --2- 172.21.15.104:0/1906305506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efedc15c070 0x7efedc0ad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.291+0000 7efee0850700 1 -- 172.21.15.104:0/1906305506 >> v1:172.21.15.104:6789/0 conn(0x7efedc0b0270 legacy=0x7efedc15fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.291+0000 7efed3fff700 1 -- 172.21.15.104:0/1906305506 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3718492575 0 0) 0x7efec80035d0 con 0x7efedc0b0270 2023-07-14T11:54:41.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.291+0000 7efee0850700 1 -- 172.21.15.104:0/1906305506 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc0cc1b0 con 0x7efedc0ad8b0 2023-07-14T11:54:41.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.291+0000 7efee0850700 1 --2- 172.21.15.104:0/1906305506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efedc0ad8b0 0x7efedc0afd30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efecc009850 tx=0x7efecc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=477f554ed3772420 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:41.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.291+0000 7efed3fff700 1 -- 172.21.15.104:0/1906305506 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed40019f0 con 0x7efedc0ad8b0 2023-07-14T11:54:41.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.291+0000 7efed3fff700 1 -- 172.21.15.104:0/1906305506 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc004030 con 0x7efedc0ad8b0 2023-07-14T11:54:41.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.291+0000 7efed3fff700 1 -- 172.21.15.104:0/1906305506 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc0043e0 con 0x7efedc0ad8b0 2023-07-14T11:54:41.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.291+0000 7efee32b5700 1 -- 172.21.15.104:0/1906305506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efedc0ad8b0 msgr2=0x7efedc0afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.291+0000 7efee32b5700 1 --2- 172.21.15.104:0/1906305506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efedc0ad8b0 0x7efedc0afd30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efecc009850 tx=0x7efecc009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:41.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.291+0000 7efee32b5700 1 -- 172.21.15.104:0/1906305506 shutdown_connections 2023-07-14T11:54:41.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.291+0000 7efee32b5700 1 --2- 172.21.15.104:0/1906305506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efedc0ad8b0 0x7efedc0afd30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.291+0000 7efee32b5700 1 --2- 172.21.15.104:0/1906305506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efedc15c070 0x7efedc0ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.291+0000 7efee32b5700 1 -- 172.21.15.104:0/1906305506 >> 172.21.15.104:0/1906305506 conn(0x7efedc0cad30 msgr2=0x7efedc0cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:41.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.291+0000 7efee32b5700 1 -- 172.21.15.104:0/1906305506 shutdown_connections 2023-07-14T11:54:41.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.291+0000 7efee32b5700 1 -- 172.21.15.104:0/1906305506 wait complete. 2023-07-14T11:54:41.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee32b5700 1 Processor -- start 2023-07-14T11:54:41.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee32b5700 1 -- start start 2023-07-14T11:54:41.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee32b5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efedc0ad8b0 0x7efedc0b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee32b5700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efedc0b0270 0x7efedc0b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee32b5700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efedc15c070 0x7efedc0b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee32b5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc15c9e0 con 0x7efedc0ad8b0 2023-07-14T11:54:41.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee32b5700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc15d280 con 0x7efedc0b0270 2023-07-14T11:54:41.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee32b5700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc0af490 con 0x7efedc15c070 2023-07-14T11:54:41.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee1051700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efedc0ad8b0 0x7efedc0b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee1051700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efedc0ad8b0 0x7efedc0b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59250/0 (socket says 172.21.15.104:59250) 2023-07-14T11:54:41.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee1051700 1 -- 172.21.15.104:0/2933154963 learned_addr learned my addr 172.21.15.104:0/2933154963 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:41.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee1852700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efedc15c070 0x7efedc0b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee0850700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efedc0b0270 0x7efedc0b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee1051700 1 -- 172.21.15.104:0/2933154963 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efedc15c070 msgr2=0x7efedc0b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee1051700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efedc15c070 0x7efedc0b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee1051700 1 -- 172.21.15.104:0/2933154963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efedc0b0270 msgr2=0x7efedc0b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee1051700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efedc0b0270 0x7efedc0b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee1051700 1 -- 172.21.15.104:0/2933154963 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0094b0 con 0x7efedc0ad8b0 2023-07-14T11:54:41.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee1852700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efedc15c070 0x7efedc0b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:41.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee0850700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efedc0b0270 0x7efedc0b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:41.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.295+0000 7efee1051700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efedc0ad8b0 0x7efedc0b3cb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efed800e870 tx=0x7efed800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:41.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.299+0000 7efed1ffb700 1 -- 172.21.15.104:0/2933154963 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed80095a0 con 0x7efedc0ad8b0 2023-07-14T11:54:41.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.299+0000 7efed1ffb700 1 -- 172.21.15.104:0/2933154963 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed800ee70 con 0x7efedc0ad8b0 2023-07-14T11:54:41.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.299+0000 7efed1ffb700 1 -- 172.21.15.104:0/2933154963 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800c9f0 con 0x7efedc0ad8b0 2023-07-14T11:54:41.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.299+0000 7efee32b5700 1 -- 172.21.15.104:0/2933154963 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efedc15cc30 con 0x7efedc0ad8b0 2023-07-14T11:54:41.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.299+0000 7efee32b5700 1 -- 172.21.15.104:0/2933154963 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc0b1850 con 0x7efedc0ad8b0 2023-07-14T11:54:41.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.299+0000 7efee32b5700 1 -- 172.21.15.104:0/2933154963 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efedc0acf50 con 0x7efedc0ad8b0 2023-07-14T11:54:41.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.303+0000 7efed1ffb700 1 -- 172.21.15.104:0/2933154963 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7efed800cb50 con 0x7efedc0ad8b0 2023-07-14T11:54:41.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.303+0000 7efed1ffb700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efec00662f0 0x7efec00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.303+0000 7efed1ffb700 1 -- 172.21.15.104:0/2933154963 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efed8084090 con 0x7efedc0ad8b0 2023-07-14T11:54:41.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.303+0000 7efee0850700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efec00662f0 0x7efec00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.303+0000 7efee0850700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efec00662f0 0x7efec00687a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efedc15f610 tx=0x7efecc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:41.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.311+0000 7efed1ffb700 1 -- 172.21.15.104:0/2933154963 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7efed8054bd0 con 0x7efedc0ad8b0 2023-07-14T11:54:41.470 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.483+0000 7efee32b5700 1 -- 172.21.15.104:0/2933154963 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efedc04dcb0 con 0x7efedc0ad8b0 2023-07-14T11:54:41.473 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.487+0000 7efed1ffb700 1 -- 172.21.15.104:0/2933154963 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20251 (secure 0 0 0) 0x7efed8054760 con 0x7efedc0ad8b0 2023-07-14T11:54:41.474 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:41.474 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":30,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:55.601783+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:41.477 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.491+0000 7efee32b5700 1 -- 172.21.15.104:0/2933154963 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efec00662f0 msgr2=0x7efec00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.477 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.491+0000 7efee32b5700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efec00662f0 0x7efec00687a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efedc15f610 tx=0x7efecc008030 comp rx=0 tx=0).stop 2023-07-14T11:54:41.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.491+0000 7efee32b5700 1 -- 172.21.15.104:0/2933154963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efedc0ad8b0 msgr2=0x7efedc0b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.491+0000 7efee32b5700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efedc0ad8b0 0x7efedc0b3cb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efed800e870 tx=0x7efed800ec30 comp rx=0 tx=0).stop 2023-07-14T11:54:41.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.491+0000 7efee32b5700 1 -- 172.21.15.104:0/2933154963 shutdown_connections 2023-07-14T11:54:41.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.491+0000 7efee32b5700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efec00662f0 0x7efec00687a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.491+0000 7efee32b5700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efedc15c070 0x7efedc0b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.491+0000 7efee32b5700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efedc0b0270 0x7efedc0b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.491+0000 7efee32b5700 1 --2- 172.21.15.104:0/2933154963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efedc0ad8b0 0x7efedc0b3cb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.491+0000 7efee32b5700 1 -- 172.21.15.104:0/2933154963 >> 172.21.15.104:0/2933154963 conn(0x7efedc0cad30 msgr2=0x7efedc06d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:41.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.491+0000 7efee32b5700 1 -- 172.21.15.104:0/2933154963 shutdown_connections 2023-07-14T11:54:41.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.491+0000 7efee32b5700 1 -- 172.21.15.104:0/2933154963 wait complete. 2023-07-14T11:54:41.498 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:53:12.845106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:53:14.031806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:53:17.678874+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T11:54:41.499 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:54:41.673 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.687+0000 7f9dcfbcd700 1 Processor -- start 2023-07-14T11:54:41.674 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.687+0000 7f9dcfbcd700 1 -- start start 2023-07-14T11:54:41.677 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.691+0000 7f9dcfbcd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc8070890 0x7f9dc80b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.677 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.691+0000 7f9dcfbcd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9dc80b45a0 0x7f9dc80d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.677 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.691+0000 7f9dcfbcd700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc80cce40 con 0x7f9dc815bba0 2023-07-14T11:54:41.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.691+0000 7f9dcfbcd700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc80ccc30 con 0x7f9dc80b45a0 2023-07-14T11:54:41.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.691+0000 7f9dcfbcd700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80d0370 con 0x7f9dc8070890 2023-07-14T11:54:41.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.691+0000 7f9dcd969700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f9dc815bba0 0x7f9dc80cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:36814/0 (socket says 172.21.15.104:36814) 2023-07-14T11:54:41.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.691+0000 7f9dcd969700 1 -- 172.21.15.104:0/4171975907 learned_addr learned my addr 172.21.15.104:0/4171975907 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:41.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.691+0000 7f9dce16a700 1 --2- 172.21.15.104:0/4171975907 >> v2:172.21.15.104:3300/0 conn(0x7f9dc80b45a0 0x7f9dc80d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.691+0000 7f9dcd168700 1 --2- 172.21.15.104:0/4171975907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc8070890 0x7f9dc80b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.691+0000 7f9dcc967700 1 -- 172.21.15.104:0/4171975907 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406643035 0 0) 0x7f9dc80cce40 con 0x7f9dc815bba0 2023-07-14T11:54:41.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.691+0000 7f9dcc967700 1 -- 172.21.15.104:0/4171975907 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9db40035d0 con 0x7f9dc815bba0 2023-07-14T11:54:41.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dce16a700 1 -- 172.21.15.104:0/4171975907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc8070890 msgr2=0x7f9dc80b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:41.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dce16a700 1 --2- 172.21.15.104:0/4171975907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc8070890 0x7f9dc80b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dce16a700 1 -- 172.21.15.104:0/4171975907 >> v1:172.21.15.104:6789/0 conn(0x7f9dc815bba0 legacy=0x7f9dc80cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dce16a700 1 -- 172.21.15.104:0/4171975907 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc80cd120 con 0x7f9dc80b45a0 2023-07-14T11:54:41.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dce16a700 1 --2- 172.21.15.104:0/4171975907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc80b45a0 0x7f9dc80d07c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0009d90 tx=0x7f9dc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453a7af2efd330da server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:41.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dcc967700 1 -- 172.21.15.104:0/4171975907 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc000e070 con 0x7f9dc80b45a0 2023-07-14T11:54:41.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dcc967700 1 -- 172.21.15.104:0/4171975907 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc000bb20 con 0x7f9dc80b45a0 2023-07-14T11:54:41.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dcc967700 1 -- 172.21.15.104:0/4171975907 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0012430 con 0x7f9dc80b45a0 2023-07-14T11:54:41.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dcfbcd700 1 -- 172.21.15.104:0/4171975907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc80b45a0 msgr2=0x7f9dc80d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dcfbcd700 1 --2- 172.21.15.104:0/4171975907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc80b45a0 0x7f9dc80d07c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0009d90 tx=0x7f9dc000b330 comp rx=0 tx=0).stop 2023-07-14T11:54:41.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dcfbcd700 1 -- 172.21.15.104:0/4171975907 shutdown_connections 2023-07-14T11:54:41.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dcfbcd700 1 --2- 172.21.15.104:0/4171975907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc80b45a0 0x7f9dc80d07c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dcfbcd700 1 --2- 172.21.15.104:0/4171975907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc8070890 0x7f9dc80b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dcfbcd700 1 -- 172.21.15.104:0/4171975907 >> 172.21.15.104:0/4171975907 conn(0x7f9dc806fcf0 msgr2=0x7f9dc80700f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:41.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dcfbcd700 1 -- 172.21.15.104:0/4171975907 shutdown_connections 2023-07-14T11:54:41.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.695+0000 7f9dcfbcd700 1 -- 172.21.15.104:0/4171975907 wait complete. 2023-07-14T11:54:41.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dcfbcd700 1 Processor -- start 2023-07-14T11:54:41.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dcfbcd700 1 -- start start 2023-07-14T11:54:41.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dcfbcd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc8070890 0x7f9dc815dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dcfbcd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dc80b45a0 0x7f9dc815c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dcfbcd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc815bba0 0x7f9dc814f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dcfbcd700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc815f460 con 0x7f9dc815bba0 2023-07-14T11:54:41.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dcfbcd700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc815c890 con 0x7f9dc80b45a0 2023-07-14T11:54:41.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dcfbcd700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc815e3c0 con 0x7f9dc8070890 2023-07-14T11:54:41.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dce16a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc815bba0 0x7f9dc814f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dce16a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc815bba0 0x7f9dc814f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59268/0 (socket says 172.21.15.104:59268) 2023-07-14T11:54:41.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dce16a700 1 -- 172.21.15.104:0/663499060 learned_addr learned my addr 172.21.15.104:0/663499060 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:41.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dcd969700 1 --2- 172.21.15.104:0/663499060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc8070890 0x7f9dc815dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dcd168700 1 --2- 172.21.15.104:0/663499060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dc80b45a0 0x7f9dc815c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dce16a700 1 -- 172.21.15.104:0/663499060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc8070890 msgr2=0x7f9dc815dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dce16a700 1 --2- 172.21.15.104:0/663499060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc8070890 0x7f9dc815dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dce16a700 1 -- 172.21.15.104:0/663499060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dc80b45a0 msgr2=0x7f9dc815c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dce16a700 1 --2- 172.21.15.104:0/663499060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dc80b45a0 0x7f9dc815c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.699+0000 7f9dce16a700 1 -- 172.21.15.104:0/663499060 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc0009a70 con 0x7f9dc815bba0 2023-07-14T11:54:41.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.703+0000 7f9dce16a700 1 --2- 172.21.15.104:0/663499060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc815bba0 0x7f9dc814f160 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000b370 tx=0x7f9dc0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:41.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.703+0000 7f9dbe7fc700 1 -- 172.21.15.104:0/663499060 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0012dd0 con 0x7f9dc815bba0 2023-07-14T11:54:41.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.703+0000 7f9dbe7fc700 1 -- 172.21.15.104:0/663499060 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc0020070 con 0x7f9dc815bba0 2023-07-14T11:54:41.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.703+0000 7f9dcfbcd700 1 -- 172.21.15.104:0/663499060 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc815e6a0 con 0x7f9dc815bba0 2023-07-14T11:54:41.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.703+0000 7f9dbe7fc700 1 -- 172.21.15.104:0/663499060 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0004280 con 0x7f9dc815bba0 2023-07-14T11:54:41.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.703+0000 7f9dcfbcd700 1 -- 172.21.15.104:0/663499060 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc815ec60 con 0x7f9dc815bba0 2023-07-14T11:54:41.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.703+0000 7f9dcfbcd700 1 -- 172.21.15.104:0/663499060 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc804dcb0 con 0x7f9dc815bba0 2023-07-14T11:54:41.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.703+0000 7f9dbe7fc700 1 -- 172.21.15.104:0/663499060 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9dc00043e0 con 0x7f9dc815bba0 2023-07-14T11:54:41.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.707+0000 7f9dbe7fc700 1 --2- 172.21.15.104:0/663499060 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9dac0662f0 0x7f9dac0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:41.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.707+0000 7f9dbe7fc700 1 -- 172.21.15.104:0/663499060 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9dc0022070 con 0x7f9dc815bba0 2023-07-14T11:54:41.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.707+0000 7f9dcd969700 1 --2- 172.21.15.104:0/663499060 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9dac0662f0 0x7f9dac0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:41.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.707+0000 7f9dcd969700 1 --2- 172.21.15.104:0/663499060 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9dac0662f0 0x7f9dac0687a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4007fd0 tx=0x7f9dc4007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:41.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.711+0000 7f9dbe7fc700 1 -- 172.21.15.104:0/663499060 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f9dc0052de0 con 0x7f9dc815bba0 2023-07-14T11:54:41.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.883+0000 7f9dcfbcd700 1 -- 172.21.15.104:0/663499060 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9dc806f9d0 con 0x7f9dc815bba0 2023-07-14T11:54:41.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.883+0000 7f9dbe7fc700 1 -- 172.21.15.104:0/663499060 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9dc002c020 con 0x7f9dc815bba0 2023-07-14T11:54:41.871 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:54:41.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.887+0000 7f9dcfbcd700 1 -- 172.21.15.104:0/663499060 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9dac0662f0 msgr2=0x7f9dac0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.887+0000 7f9dcfbcd700 1 --2- 172.21.15.104:0/663499060 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9dac0662f0 0x7f9dac0687a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4007fd0 tx=0x7f9dc4007de0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.887+0000 7f9dcfbcd700 1 -- 172.21.15.104:0/663499060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc815bba0 msgr2=0x7f9dc814f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:41.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.887+0000 7f9dcfbcd700 1 --2- 172.21.15.104:0/663499060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc815bba0 0x7f9dc814f160 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000b370 tx=0x7f9dc0004b40 comp rx=0 tx=0).stop 2023-07-14T11:54:41.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.887+0000 7f9dcfbcd700 1 -- 172.21.15.104:0/663499060 shutdown_connections 2023-07-14T11:54:41.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.887+0000 7f9dcfbcd700 1 --2- 172.21.15.104:0/663499060 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9dac0662f0 0x7f9dac0687a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.887+0000 7f9dcfbcd700 1 --2- 172.21.15.104:0/663499060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc815bba0 0x7f9dc814f160 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.887+0000 7f9dcfbcd700 1 --2- 172.21.15.104:0/663499060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dc80b45a0 0x7f9dc815c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.887+0000 7f9dcfbcd700 1 --2- 172.21.15.104:0/663499060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc8070890 0x7f9dc815dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:41.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.887+0000 7f9dcfbcd700 1 -- 172.21.15.104:0/663499060 >> 172.21.15.104:0/663499060 conn(0x7f9dc806fcf0 msgr2=0x7f9dc80b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:41.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.887+0000 7f9dcfbcd700 1 -- 172.21.15.104:0/663499060 shutdown_connections 2023-07-14T11:54:41.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:41.887+0000 7f9dcfbcd700 1 -- 172.21.15.104:0/663499060 wait complete. 2023-07-14T11:54:41.894 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:54:42.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.091+0000 7fe52ac4a700 1 Processor -- start 2023-07-14T11:54:42.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.091+0000 7fe52ac4a700 1 -- start start 2023-07-14T11:54:42.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe52ac4a700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe52406fa70 0x7fe52406fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe52ac4a700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe5240d0050 0x7fe5240cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe52ac4a700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5240cfb20 0x7fe5240cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe52ac4a700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5240cf930 con 0x7fe5240d0050 2023-07-14T11:54:42.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe52ac4a700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5240cea60 con 0x7fe52406fa70 2023-07-14T11:54:42.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe52ac4a700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe524070380 con 0x7fe5240cfb20 2023-07-14T11:54:42.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe523fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe5240d0050 0x7fe5240cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe523fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe5240d0050 0x7fe5240cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59272/0 (socket says 172.21.15.104:59272) 2023-07-14T11:54:42.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe523fff700 1 -- 172.21.15.104:0/2707931665 learned_addr learned my addr 172.21.15.104:0/2707931665 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:42.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe5291e7700 1 --2- 172.21.15.104:0/2707931665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5240cfb20 0x7fe5240cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe5289e6700 1 --2- 172.21.15.104:0/2707931665 >> v2:172.21.15.121:3300/0 conn(0x7fe52406fa70 0x7fe52406fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe523fff700 1 -- 172.21.15.104:0/2707931665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5240cfb20 msgr2=0x7fe5240cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:42.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe523fff700 1 --2- 172.21.15.104:0/2707931665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5240cfb20 0x7fe5240cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe523fff700 1 -- 172.21.15.104:0/2707931665 >> v2:172.21.15.121:3300/0 conn(0x7fe52406fa70 msgr2=0x7fe52406fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:42.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe523fff700 1 --2- 172.21.15.104:0/2707931665 >> v2:172.21.15.121:3300/0 conn(0x7fe52406fa70 0x7fe52406fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe523fff700 1 -- 172.21.15.104:0/2707931665 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5240cecb0 con 0x7fe5240d0050 2023-07-14T11:54:42.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe523fff700 1 --2- 172.21.15.104:0/2707931665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5240d0050 0x7fe5240cce60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe52406a0b0 tx=0x7fe510009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a8c87fab6536f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:42.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe5237fe700 1 -- 172.21.15.104:0/2707931665 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510011830 con 0x7fe5240d0050 2023-07-14T11:54:42.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe5237fe700 1 -- 172.21.15.104:0/2707931665 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe510011990 con 0x7fe5240d0050 2023-07-14T11:54:42.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe5237fe700 1 -- 172.21.15.104:0/2707931665 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510011d20 con 0x7fe5240d0050 2023-07-14T11:54:42.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe52ac4a700 1 -- 172.21.15.104:0/2707931665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5240d0050 msgr2=0x7fe5240cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe52ac4a700 1 --2- 172.21.15.104:0/2707931665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5240d0050 0x7fe5240cce60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe52406a0b0 tx=0x7fe510009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe52ac4a700 1 -- 172.21.15.104:0/2707931665 shutdown_connections 2023-07-14T11:54:42.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe52ac4a700 1 --2- 172.21.15.104:0/2707931665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5240cfb20 0x7fe5240cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe52ac4a700 1 --2- 172.21.15.104:0/2707931665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5240d0050 0x7fe5240cce60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe52ac4a700 1 --2- 172.21.15.104:0/2707931665 >> v2:172.21.15.121:3300/0 conn(0x7fe52406fa70 0x7fe52406fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.095+0000 7fe52ac4a700 1 -- 172.21.15.104:0/2707931665 >> 172.21.15.104:0/2707931665 conn(0x7fe52408a000 msgr2=0x7fe5240d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:42.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.099+0000 7fe52ac4a700 1 -- 172.21.15.104:0/2707931665 shutdown_connections 2023-07-14T11:54:42.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.099+0000 7fe52ac4a700 1 -- 172.21.15.104:0/2707931665 wait complete. 2023-07-14T11:54:42.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.099+0000 7fe52ac4a700 1 Processor -- start 2023-07-14T11:54:42.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.099+0000 7fe52ac4a700 1 -- start start 2023-07-14T11:54:42.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.099+0000 7fe52ac4a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe52406fa70 0x7fe52415d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.099+0000 7fe52ac4a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5240cfb20 0x7fe524154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.099+0000 7fe52ac4a700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5240d0050 0x7fe524151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.099+0000 7fe52ac4a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52415e310 con 0x7fe5240cfb20 2023-07-14T11:54:42.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.099+0000 7fe52ac4a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52415d700 con 0x7fe52406fa70 2023-07-14T11:54:42.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.099+0000 7fe52ac4a700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52415d9e0 con 0x7fe5240d0050 2023-07-14T11:54:42.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe523fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5240cfb20 0x7fe524154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe523fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5240cfb20 0x7fe524154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59284/0 (socket says 172.21.15.104:59284) 2023-07-14T11:54:42.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe523fff700 1 -- 172.21.15.104:0/3657695385 learned_addr learned my addr 172.21.15.104:0/3657695385 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:42.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe5291e7700 1 --2- 172.21.15.104:0/3657695385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5240d0050 0x7fe524151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe5289e6700 1 --2- 172.21.15.104:0/3657695385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe52406fa70 0x7fe52415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe523fff700 1 -- 172.21.15.104:0/3657695385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5240d0050 msgr2=0x7fe524151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe523fff700 1 --2- 172.21.15.104:0/3657695385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5240d0050 0x7fe524151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe523fff700 1 -- 172.21.15.104:0/3657695385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe52406fa70 msgr2=0x7fe52415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe523fff700 1 --2- 172.21.15.104:0/3657695385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe52406fa70 0x7fe52415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe523fff700 1 -- 172.21.15.104:0/3657695385 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe510009570 con 0x7fe5240cfb20 2023-07-14T11:54:42.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe5289e6700 1 --2- 172.21.15.104:0/3657695385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe52406fa70 0x7fe52415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:42.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe523fff700 1 --2- 172.21.15.104:0/3657695385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5240cfb20 0x7fe524154e50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe510000c00 tx=0x7fe510004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:42.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe5217fa700 1 -- 172.21.15.104:0/3657695385 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510004610 con 0x7fe5240cfb20 2023-07-14T11:54:42.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe5217fa700 1 -- 172.21.15.104:0/3657695385 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51001e070 con 0x7fe5240cfb20 2023-07-14T11:54:42.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe52ac4a700 1 -- 172.21.15.104:0/3657695385 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe52415dcc0 con 0x7fe5240cfb20 2023-07-14T11:54:42.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe5217fa700 1 -- 172.21.15.104:0/3657695385 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5100038e0 con 0x7fe5240cfb20 2023-07-14T11:54:42.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe52ac4a700 1 -- 172.21.15.104:0/3657695385 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe524152100 con 0x7fe5240cfb20 2023-07-14T11:54:42.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.103+0000 7fe52ac4a700 1 -- 172.21.15.104:0/3657695385 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe52415e480 con 0x7fe5240cfb20 2023-07-14T11:54:42.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.107+0000 7fe5217fa700 1 -- 172.21.15.104:0/3657695385 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe510003a40 con 0x7fe5240cfb20 2023-07-14T11:54:42.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.107+0000 7fe5217fa700 1 --2- 172.21.15.104:0/3657695385 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe5080662f0 0x7fe5080687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.107+0000 7fe5217fa700 1 -- 172.21.15.104:0/3657695385 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe510084b10 con 0x7fe5240cfb20 2023-07-14T11:54:42.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.107+0000 7fe5289e6700 1 --2- 172.21.15.104:0/3657695385 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe5080662f0 0x7fe5080687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.107+0000 7fe5289e6700 1 --2- 172.21.15.104:0/3657695385 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe5080662f0 0x7fe5080687a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe518006fd0 tx=0x7fe518008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:42.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.111+0000 7fe5217fa700 1 -- 172.21.15.104:0/3657695385 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe510051e10 con 0x7fe5240cfb20 2023-07-14T11:54:42.272 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.287+0000 7fe52ac4a700 1 -- 172.21.15.104:0/3657695385 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe52404dcb0 con 0x7fe5240cfb20 2023-07-14T11:54:42.273 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.287+0000 7fe5217fa700 1 -- 172.21.15.104:0/3657695385 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe510020020 con 0x7fe5240cfb20 2023-07-14T11:54:42.274 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:54:42.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.291+0000 7fe52ac4a700 1 -- 172.21.15.104:0/3657695385 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe5080662f0 msgr2=0x7fe5080687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.291+0000 7fe52ac4a700 1 --2- 172.21.15.104:0/3657695385 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe5080662f0 0x7fe5080687a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe518006fd0 tx=0x7fe518008040 comp rx=0 tx=0).stop 2023-07-14T11:54:42.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.291+0000 7fe52ac4a700 1 -- 172.21.15.104:0/3657695385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5240cfb20 msgr2=0x7fe524154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.291+0000 7fe52ac4a700 1 --2- 172.21.15.104:0/3657695385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5240cfb20 0x7fe524154e50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe510000c00 tx=0x7fe510004770 comp rx=0 tx=0).stop 2023-07-14T11:54:42.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.291+0000 7fe52ac4a700 1 -- 172.21.15.104:0/3657695385 shutdown_connections 2023-07-14T11:54:42.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.291+0000 7fe52ac4a700 1 --2- 172.21.15.104:0/3657695385 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe5080662f0 0x7fe5080687a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.291+0000 7fe52ac4a700 1 --2- 172.21.15.104:0/3657695385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5240cfb20 0x7fe524154e50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.291+0000 7fe52ac4a700 1 --2- 172.21.15.104:0/3657695385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5240d0050 0x7fe524151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.291+0000 7fe52ac4a700 1 --2- 172.21.15.104:0/3657695385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe52406fa70 0x7fe52415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.291+0000 7fe52ac4a700 1 -- 172.21.15.104:0/3657695385 >> 172.21.15.104:0/3657695385 conn(0x7fe52408a000 msgr2=0x7fe5240ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:42.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.291+0000 7fe52ac4a700 1 -- 172.21.15.104:0/3657695385 shutdown_connections 2023-07-14T11:54:42.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.291+0000 7fe52ac4a700 1 -- 172.21.15.104:0/3657695385 wait complete. 2023-07-14T11:54:42.298 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:54:42.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.495+0000 7ff6fa592700 1 Processor -- start 2023-07-14T11:54:42.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.495+0000 7ff6fa592700 1 -- start start 2023-07-14T11:54:42.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.495+0000 7ff6fa592700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6f40c9270 0x7ff6f4071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.495+0000 7ff6fa592700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff6f40b0680 0x7ff6f40cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.495+0000 7ff6fa592700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f4158970 con 0x7ff6f40cdcb0 2023-07-14T11:54:42.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.495+0000 7ff6fa592700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f40ce080 con 0x7ff6f40b0680 2023-07-14T11:54:42.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6fa592700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f4158bf0 con 0x7ff6f40c9270 2023-07-14T11:54:42.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f3fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ff6f40cdcb0 0x7ff6f40c8b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:36826/0 (socket says 172.21.15.104:36826) 2023-07-14T11:54:42.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f3fff700 1 -- 172.21.15.104:0/2428371537 learned_addr learned my addr 172.21.15.104:0/2428371537 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:42.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f8b2f700 1 --2- 172.21.15.104:0/2428371537 >> v2:172.21.15.104:3300/0 conn(0x7ff6f40b0680 0x7ff6f40cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6eb7fe700 1 --2- 172.21.15.104:0/2428371537 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6f40c9270 0x7ff6f4071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 -- 172.21.15.104:0/2428371537 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692353822 0 0) 0x7ff6f4158970 con 0x7ff6f40cdcb0 2023-07-14T11:54:42.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 -- 172.21.15.104:0/2428371537 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6e00035d0 con 0x7ff6f40cdcb0 2023-07-14T11:54:42.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 -- 172.21.15.104:0/2428371537 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2849561384 0 0) 0x7ff6e00035d0 con 0x7ff6f40cdcb0 2023-07-14T11:54:42.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 -- 172.21.15.104:0/2428371537 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6f40c9270 msgr2=0x7ff6f4071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:42.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 --2- 172.21.15.104:0/2428371537 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6f40c9270 0x7ff6f4071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 -- 172.21.15.104:0/2428371537 >> v2:172.21.15.104:3300/0 conn(0x7ff6f40b0680 msgr2=0x7ff6f40cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:42.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 --2- 172.21.15.104:0/2428371537 >> v2:172.21.15.104:3300/0 conn(0x7ff6f40b0680 0x7ff6f40cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 -- 172.21.15.104:0/2428371537 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f4158ed0 con 0x7ff6f40cdcb0 2023-07-14T11:54:42.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 -- 172.21.15.104:0/2428371537 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7ff6e4001670 con 0x7ff6f40cdcb0 2023-07-14T11:54:42.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f8b2f700 1 --2- 172.21.15.104:0/2428371537 >> v2:172.21.15.104:3300/0 conn(0x7ff6f40b0680 0x7ff6f40cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:42.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 -- 172.21.15.104:0/2428371537 >> v1:172.21.15.104:6789/0 conn(0x7ff6f40cdcb0 legacy=0x7ff6f40c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 --2- 172.21.15.104:0/2428371537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6e0003a20 0x7ff6e0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 --2- 172.21.15.104:0/2428371537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6e0004450 0x7ff6e0004910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 --2- 172.21.15.104:0/2428371537 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6f40c9270 0x7ff6e0005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 -- 172.21.15.104:0/2428371537 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40ce080 con 0x7ff6e0003a20 2023-07-14T11:54:42.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 -- 172.21.15.104:0/2428371537 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f4158bf0 con 0x7ff6e0004450 2023-07-14T11:54:42.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 -- 172.21.15.104:0/2428371537 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e000dc70 con 0x7ff6f40c9270 2023-07-14T11:54:42.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f3fff700 1 --2- 172.21.15.104:0/2428371537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6e0003a20 0x7ff6e0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f8b2f700 1 --2- 172.21.15.104:0/2428371537 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6f40c9270 0x7ff6e0005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6eb7fe700 1 --2- 172.21.15.104:0/2428371537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6e0004450 0x7ff6e0004910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f3fff700 1 -- 172.21.15.104:0/2428371537 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6f40c9270 msgr2=0x7ff6e0005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:42.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f3fff700 1 --2- 172.21.15.104:0/2428371537 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6f40c9270 0x7ff6e0005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f3fff700 1 -- 172.21.15.104:0/2428371537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6e0004450 msgr2=0x7ff6e0004910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:42.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f3fff700 1 --2- 172.21.15.104:0/2428371537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6e0004450 0x7ff6e0004910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f3fff700 1 -- 172.21.15.104:0/2428371537 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e00035d0 con 0x7ff6e0003a20 2023-07-14T11:54:42.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f3fff700 1 --2- 172.21.15.104:0/2428371537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6e0003a20 0x7ff6e0003e30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400c2f0 tx=0x7ff6e00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2608e3cf0b257126 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:42.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.499+0000 7ff6f37fe700 1 -- 172.21.15.104:0/2428371537 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4007c80 con 0x7ff6e0003a20 2023-07-14T11:54:42.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.503+0000 7ff6f37fe700 1 -- 172.21.15.104:0/2428371537 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e400ce30 con 0x7ff6e0003a20 2023-07-14T11:54:42.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.503+0000 7ff6f37fe700 1 -- 172.21.15.104:0/2428371537 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4005bc0 con 0x7ff6e0003a20 2023-07-14T11:54:42.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.503+0000 7ff6fa592700 1 -- 172.21.15.104:0/2428371537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6e0003a20 msgr2=0x7ff6e0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.503+0000 7ff6fa592700 1 --2- 172.21.15.104:0/2428371537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6e0003a20 0x7ff6e0003e30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400c2f0 tx=0x7ff6e00058a0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.503+0000 7ff6f8b2f700 1 -- 172.21.15.104:0/2428371537 reap_dead start 2023-07-14T11:54:42.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.503+0000 7ff6fa592700 1 -- 172.21.15.104:0/2428371537 shutdown_connections 2023-07-14T11:54:42.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.503+0000 7ff6fa592700 1 -- 172.21.15.104:0/2428371537 >> 172.21.15.104:0/2428371537 conn(0x7ff6f40cad80 msgr2=0x7ff6f40cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:42.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.503+0000 7ff6fa592700 1 -- 172.21.15.104:0/2428371537 shutdown_connections 2023-07-14T11:54:42.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.503+0000 7ff6fa592700 1 -- 172.21.15.104:0/2428371537 wait complete. 2023-07-14T11:54:42.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6fa592700 1 Processor -- start 2023-07-14T11:54:42.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6fa592700 1 -- start start 2023-07-14T11:54:42.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6fa592700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6e0003a20 0x7ff6f414b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6fa592700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6f41500d0 0x7ff6f4150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6fa592700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6f40c77d0 0x7ff6f40c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6fa592700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f4150a80 con 0x7ff6e0003a20 2023-07-14T11:54:42.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6fa592700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40b2450 con 0x7ff6f41500d0 2023-07-14T11:54:42.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6fa592700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40c5f70 con 0x7ff6f40c77d0 2023-07-14T11:54:42.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6f3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6e0003a20 0x7ff6f414b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6f3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6e0003a20 0x7ff6f414b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59318/0 (socket says 172.21.15.104:59318) 2023-07-14T11:54:42.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6f3fff700 1 -- 172.21.15.104:0/2829081355 learned_addr learned my addr 172.21.15.104:0/2829081355 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:42.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6f8b2f700 1 --2- 172.21.15.104:0/2829081355 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6f40c77d0 0x7ff6f40c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6eb7fe700 1 --2- 172.21.15.104:0/2829081355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6f41500d0 0x7ff6f4150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6f3fff700 1 -- 172.21.15.104:0/2829081355 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6f40c77d0 msgr2=0x7ff6f40c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6f3fff700 1 --2- 172.21.15.104:0/2829081355 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6f40c77d0 0x7ff6f40c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6f3fff700 1 -- 172.21.15.104:0/2829081355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6f41500d0 msgr2=0x7ff6f4150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6f3fff700 1 --2- 172.21.15.104:0/2829081355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6f41500d0 0x7ff6f4150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6f3fff700 1 -- 172.21.15.104:0/2829081355 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f4150cb0 con 0x7ff6e0003a20 2023-07-14T11:54:42.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.507+0000 7ff6eb7fe700 1 --2- 172.21.15.104:0/2829081355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6f41500d0 0x7ff6f4150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:42.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.511+0000 7ff6f3fff700 1 --2- 172.21.15.104:0/2829081355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6e0003a20 0x7ff6f414b900 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4000c00 tx=0x7ff6e4000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:42.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.511+0000 7ff6f17fa700 1 -- 172.21.15.104:0/2829081355 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e40070d0 con 0x7ff6e0003a20 2023-07-14T11:54:42.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.511+0000 7ff6f17fa700 1 -- 172.21.15.104:0/2829081355 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e4007230 con 0x7ff6e0003a20 2023-07-14T11:54:42.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.511+0000 7ff6f17fa700 1 -- 172.21.15.104:0/2829081355 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e40075e0 con 0x7ff6e0003a20 2023-07-14T11:54:42.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.511+0000 7ff6fa592700 1 -- 172.21.15.104:0/2829081355 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f4150f90 con 0x7ff6e0003a20 2023-07-14T11:54:42.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.511+0000 7ff6fa592700 1 -- 172.21.15.104:0/2829081355 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f40b20e0 con 0x7ff6e0003a20 2023-07-14T11:54:42.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.511+0000 7ff6fa592700 1 -- 172.21.15.104:0/2829081355 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f40b22a0 con 0x7ff6e0003a20 2023-07-14T11:54:42.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.511+0000 7ff6f17fa700 1 -- 172.21.15.104:0/2829081355 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff6e4019030 con 0x7ff6e0003a20 2023-07-14T11:54:42.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.515+0000 7ff6f17fa700 1 --2- 172.21.15.104:0/2829081355 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6d4066320 0x7ff6d40687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.515+0000 7ff6f17fa700 1 -- 172.21.15.104:0/2829081355 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff6e4084f20 con 0x7ff6e0003a20 2023-07-14T11:54:42.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.515+0000 7ff6eb7fe700 1 --2- 172.21.15.104:0/2829081355 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6d4066320 0x7ff6d40687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.515+0000 7ff6eb7fe700 1 --2- 172.21.15.104:0/2829081355 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6d4066320 0x7ff6d40687d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40c91d0 tx=0x7ff6dc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:42.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.519+0000 7ff6f17fa700 1 -- 172.21.15.104:0/2829081355 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff6e4052220 con 0x7ff6e0003a20 2023-07-14T11:54:42.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.699+0000 7ff6fa592700 1 -- 172.21.15.104:0/2829081355 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff6f4061060 con 0x7ff6e0003a20 2023-07-14T11:54:42.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.699+0000 7ff6f17fa700 1 -- 172.21.15.104:0/2829081355 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff6e4055840 con 0x7ff6e0003a20 2023-07-14T11:54:42.687 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:42.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.707+0000 7ff6fa592700 1 -- 172.21.15.104:0/2829081355 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6d4066320 msgr2=0x7ff6d40687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.707+0000 7ff6fa592700 1 --2- 172.21.15.104:0/2829081355 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6d4066320 0x7ff6d40687d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40c91d0 tx=0x7ff6dc008040 comp rx=0 tx=0).stop 2023-07-14T11:54:42.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.707+0000 7ff6fa592700 1 -- 172.21.15.104:0/2829081355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6e0003a20 msgr2=0x7ff6f414b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.707+0000 7ff6fa592700 1 --2- 172.21.15.104:0/2829081355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6e0003a20 0x7ff6f414b900 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4000c00 tx=0x7ff6e4000fe0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.707+0000 7ff6fa592700 1 -- 172.21.15.104:0/2829081355 shutdown_connections 2023-07-14T11:54:42.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.707+0000 7ff6fa592700 1 --2- 172.21.15.104:0/2829081355 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6d4066320 0x7ff6d40687d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.707+0000 7ff6fa592700 1 --2- 172.21.15.104:0/2829081355 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6f40c77d0 0x7ff6f40c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.707+0000 7ff6fa592700 1 --2- 172.21.15.104:0/2829081355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6f41500d0 0x7ff6f4150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.707+0000 7ff6fa592700 1 --2- 172.21.15.104:0/2829081355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6e0003a20 0x7ff6f414b900 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.707+0000 7ff6fa592700 1 -- 172.21.15.104:0/2829081355 >> 172.21.15.104:0/2829081355 conn(0x7ff6f40cad80 msgr2=0x7ff6f40cbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:42.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.707+0000 7ff6fa592700 1 -- 172.21.15.104:0/2829081355 shutdown_connections 2023-07-14T11:54:42.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.707+0000 7ff6fa592700 1 -- 172.21.15.104:0/2829081355 wait complete. 2023-07-14T11:54:42.710 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:54:42.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.903+0000 7f788d693700 1 Processor -- start 2023-07-14T11:54:42.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.903+0000 7f788d693700 1 -- start start 2023-07-14T11:54:42.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f788d693700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f78880b0ee0 0x7f78880ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f788d693700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880d07e0 0x7f78880af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f788d693700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78880cd4e0 con 0x7f78880cd090 2023-07-14T11:54:42.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f788d693700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78880d0c60 con 0x7f78880b0ee0 2023-07-14T11:54:42.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f788d693700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78880d1a60 con 0x7f78880d07e0 2023-07-14T11:54:42.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f7887fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f78880b0ee0 0x7f78880ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f788ce92700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880d07e0 0x7f78880af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f7887fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f78880b0ee0 0x7f78880ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:43398/0 (socket says 172.21.15.104:43398) 2023-07-14T11:54:42.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f788ce92700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880d07e0 0x7f78880af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:53792/0 (socket says 172.21.15.104:53792) 2023-07-14T11:54:42.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f7887fff700 1 -- 172.21.15.104:0/2247076330 learned_addr learned my addr 172.21.15.104:0/2247076330 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:42.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f7887fff700 1 -- 172.21.15.104:0/2247076330 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880d07e0 msgr2=0x7f78880af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:42.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f7887fff700 1 --2- 172.21.15.104:0/2247076330 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880d07e0 0x7f78880af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f7887fff700 1 -- 172.21.15.104:0/2247076330 >> v1:172.21.15.121:6789/0 conn(0x7f78880cd090 legacy=0x7f78880d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f7887fff700 1 -- 172.21.15.104:0/2247076330 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78880d1ef0 con 0x7f78880b0ee0 2023-07-14T11:54:42.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f788ce92700 1 --2- 172.21.15.104:0/2247076330 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880d07e0 0x7f78880af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:42.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f7887fff700 1 --2- 172.21.15.104:0/2247076330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b0ee0 0x7f78880ccb50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7878009910 tx=0x7f7878009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8505a49d56e0405f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:42.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f7886ffd700 1 -- 172.21.15.104:0/2247076330 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878011830 con 0x7f78880b0ee0 2023-07-14T11:54:42.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f7886ffd700 1 -- 172.21.15.104:0/2247076330 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7878011990 con 0x7f78880b0ee0 2023-07-14T11:54:42.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.907+0000 7f7886ffd700 1 -- 172.21.15.104:0/2247076330 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878011d40 con 0x7f78880b0ee0 2023-07-14T11:54:42.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.911+0000 7f788d693700 1 -- 172.21.15.104:0/2247076330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b0ee0 msgr2=0x7f78880ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.911+0000 7f788d693700 1 --2- 172.21.15.104:0/2247076330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b0ee0 0x7f78880ccb50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7878009910 tx=0x7f7878009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:42.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.911+0000 7f788d693700 1 -- 172.21.15.104:0/2247076330 shutdown_connections 2023-07-14T11:54:42.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.911+0000 7f788d693700 1 --2- 172.21.15.104:0/2247076330 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880d07e0 0x7f78880af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.911+0000 7f788d693700 1 --2- 172.21.15.104:0/2247076330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b0ee0 0x7f78880ccb50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.911+0000 7f788d693700 1 -- 172.21.15.104:0/2247076330 >> 172.21.15.104:0/2247076330 conn(0x7f7888060c40 msgr2=0x7f788809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:42.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.911+0000 7f788d693700 1 -- 172.21.15.104:0/2247076330 shutdown_connections 2023-07-14T11:54:42.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.911+0000 7f788d693700 1 -- 172.21.15.104:0/2247076330 wait complete. 2023-07-14T11:54:42.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.911+0000 7f788d693700 1 Processor -- start 2023-07-14T11:54:42.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.911+0000 7f788d693700 1 -- start start 2023-07-14T11:54:42.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f788d693700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b0ee0 0x7f788814bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f788d693700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78880cd090 0x7f78880708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f788d693700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880d07e0 0x7f78881f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f788d693700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78880cb770 con 0x7f78880cd090 2023-07-14T11:54:42.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f788d693700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78880c89f0 con 0x7f78880b0ee0 2023-07-14T11:54:42.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f788d693700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788814c0b0 con 0x7f78880d07e0 2023-07-14T11:54:42.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f78877fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78880cd090 0x7f78880708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f7887fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b0ee0 0x7f788814bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f7887fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b0ee0 0x7f788814bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:43402/0 (socket says 172.21.15.104:43402) 2023-07-14T11:54:42.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f7887fff700 1 -- 172.21.15.104:0/841325010 learned_addr learned my addr 172.21.15.104:0/841325010 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:42.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f788ce92700 1 --2- 172.21.15.104:0/841325010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880d07e0 0x7f78881f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f7887fff700 1 -- 172.21.15.104:0/841325010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880d07e0 msgr2=0x7f78881f0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f7887fff700 1 --2- 172.21.15.104:0/841325010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880d07e0 0x7f78881f0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f7887fff700 1 -- 172.21.15.104:0/841325010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78880cd090 msgr2=0x7f78880708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:42.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f7887fff700 1 --2- 172.21.15.104:0/841325010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78880cd090 0x7f78880708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:42.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f7887fff700 1 -- 172.21.15.104:0/841325010 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7880009a70 con 0x7f78880b0ee0 2023-07-14T11:54:42.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f788ce92700 1 --2- 172.21.15.104:0/841325010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880d07e0 0x7f78881f0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:42.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f7887fff700 1 --2- 172.21.15.104:0/841325010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b0ee0 0x7f788814bb70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7878000c00 tx=0x7f7878004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:42.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f7884ff9700 1 -- 172.21.15.104:0/841325010 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878004590 con 0x7f78880b0ee0 2023-07-14T11:54:42.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f7884ff9700 1 -- 172.21.15.104:0/841325010 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78780036c0 con 0x7f78880b0ee0 2023-07-14T11:54:42.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f7884ff9700 1 -- 172.21.15.104:0/841325010 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878003a70 con 0x7f78880b0ee0 2023-07-14T11:54:42.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f788d693700 1 -- 172.21.15.104:0/841325010 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7878009570 con 0x7f78880b0ee0 2023-07-14T11:54:42.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.915+0000 7f788d693700 1 -- 172.21.15.104:0/841325010 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78880cae40 con 0x7f78880b0ee0 2023-07-14T11:54:42.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.919+0000 7f788d693700 1 -- 172.21.15.104:0/841325010 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788804e500 con 0x7f78880b0ee0 2023-07-14T11:54:42.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.919+0000 7f7884ff9700 1 -- 172.21.15.104:0/841325010 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7878003bd0 con 0x7f78880b0ee0 2023-07-14T11:54:42.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.923+0000 7f7884ff9700 1 --2- 172.21.15.104:0/841325010 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7874066340 0x7f78740687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:42.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.923+0000 7f7884ff9700 1 -- 172.21.15.104:0/841325010 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7878021030 con 0x7f78880b0ee0 2023-07-14T11:54:42.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.923+0000 7f78877fe700 1 --2- 172.21.15.104:0/841325010 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7874066340 0x7f78740687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:42.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.923+0000 7f78877fe700 1 --2- 172.21.15.104:0/841325010 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7874066340 0x7f78740687f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f78881f4a30 tx=0x7f787c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:42.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:42.927+0000 7f7884ff9700 1 -- 172.21.15.104:0/841325010 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f78780528a0 con 0x7f78880b0ee0 2023-07-14T11:54:43.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.099+0000 7f788d693700 1 -- 172.21.15.104:0/841325010 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78880b18d0 con 0x7f78880b0ee0 2023-07-14T11:54:43.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.099+0000 7f7884ff9700 1 -- 172.21.15.104:0/841325010 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7878055ec0 con 0x7f78880b0ee0 2023-07-14T11:54:43.085 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:54:43.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.103+0000 7f788d693700 1 -- 172.21.15.104:0/841325010 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7874066340 msgr2=0x7f78740687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.103+0000 7f788d693700 1 --2- 172.21.15.104:0/841325010 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7874066340 0x7f78740687f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f78881f4a30 tx=0x7f787c009040 comp rx=0 tx=0).stop 2023-07-14T11:54:43.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.103+0000 7f788d693700 1 -- 172.21.15.104:0/841325010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b0ee0 msgr2=0x7f788814bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.103+0000 7f788d693700 1 --2- 172.21.15.104:0/841325010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b0ee0 0x7f788814bb70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7878000c00 tx=0x7f7878004770 comp rx=0 tx=0).stop 2023-07-14T11:54:43.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.103+0000 7f788d693700 1 -- 172.21.15.104:0/841325010 shutdown_connections 2023-07-14T11:54:43.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.103+0000 7f788d693700 1 --2- 172.21.15.104:0/841325010 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7874066340 0x7f78740687f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.103+0000 7f788d693700 1 --2- 172.21.15.104:0/841325010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78880cd090 0x7f78880708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.103+0000 7f788d693700 1 --2- 172.21.15.104:0/841325010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880d07e0 0x7f78881f0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.103+0000 7f788d693700 1 --2- 172.21.15.104:0/841325010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b0ee0 0x7f788814bb70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.103+0000 7f788d693700 1 -- 172.21.15.104:0/841325010 >> 172.21.15.104:0/841325010 conn(0x7f7888060c40 msgr2=0x7f78880b7670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:43.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.103+0000 7f788d693700 1 -- 172.21.15.104:0/841325010 shutdown_connections 2023-07-14T11:54:43.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.103+0000 7f788d693700 1 -- 172.21.15.104:0/841325010 wait complete. 2023-07-14T11:54:43.110 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T11:54:43.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.303+0000 7f9cc8cb1700 1 Processor -- start 2023-07-14T11:54:43.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.303+0000 7f9cc8cb1700 1 -- start start 2023-07-14T11:54:43.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.307+0000 7f9cc8cb1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9cc415bac0 0x7f9cc406c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:43.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.307+0000 7f9cc8cb1700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9cc415be90 0x7f9cc406f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:43.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.307+0000 7f9cc8cb1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc406f600 con 0x7f9cc40b4240 2023-07-14T11:54:43.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.307+0000 7f9cc8cb1700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc406db90 con 0x7f9cc415be90 2023-07-14T11:54:43.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.307+0000 7f9cc8cb1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc406d010 con 0x7f9cc415bac0 2023-07-14T11:54:43.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.307+0000 7f9cc2ffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9cc415be90 0x7f9cc406f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:43.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.307+0000 7f9cc2ffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9cc415be90 0x7f9cc406f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59326/0 (socket says 172.21.15.104:59326) 2023-07-14T11:54:43.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.307+0000 7f9cc37fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9cc415bac0 0x7f9cc406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:43.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.307+0000 7f9cc2ffd700 1 -- 172.21.15.104:0/2835938143 learned_addr learned my addr 172.21.15.104:0/2835938143 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:43.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc2ffd700 1 -- 172.21.15.104:0/2835938143 >> v2:172.21.15.121:3300/0 conn(0x7f9cc415bac0 msgr2=0x7f9cc406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:43.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc2ffd700 1 --2- 172.21.15.104:0/2835938143 >> v2:172.21.15.121:3300/0 conn(0x7f9cc415bac0 0x7f9cc406c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc2ffd700 1 -- 172.21.15.104:0/2835938143 >> v1:172.21.15.121:6789/0 conn(0x7f9cc40b4240 legacy=0x7f9cc4070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc2ffd700 1 -- 172.21.15.104:0/2835938143 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc406d2f0 con 0x7f9cc415be90 2023-07-14T11:54:43.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc2ffd700 1 --2- 172.21.15.104:0/2835938143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cc415be90 0x7f9cc406f7e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4060710 tx=0x7f9cb8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a04a7d0473944a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:43.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc27fc700 1 -- 172.21.15.104:0/2835938143 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8011830 con 0x7f9cc415be90 2023-07-14T11:54:43.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc27fc700 1 -- 172.21.15.104:0/2835938143 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb8011990 con 0x7f9cc415be90 2023-07-14T11:54:43.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc27fc700 1 -- 172.21.15.104:0/2835938143 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8011d20 con 0x7f9cc415be90 2023-07-14T11:54:43.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc8cb1700 1 -- 172.21.15.104:0/2835938143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cc415be90 msgr2=0x7f9cc406f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc8cb1700 1 --2- 172.21.15.104:0/2835938143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cc415be90 0x7f9cc406f7e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4060710 tx=0x7f9cb8009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc8cb1700 1 -- 172.21.15.104:0/2835938143 shutdown_connections 2023-07-14T11:54:43.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc8cb1700 1 --2- 172.21.15.104:0/2835938143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cc415be90 0x7f9cc406f7e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc8cb1700 1 --2- 172.21.15.104:0/2835938143 >> v2:172.21.15.121:3300/0 conn(0x7f9cc415bac0 0x7f9cc406c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc8cb1700 1 -- 172.21.15.104:0/2835938143 >> 172.21.15.104:0/2835938143 conn(0x7f9cc40b04e0 msgr2=0x7f9cc40b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:43.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc8cb1700 1 -- 172.21.15.104:0/2835938143 shutdown_connections 2023-07-14T11:54:43.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.311+0000 7f9cc8cb1700 1 -- 172.21.15.104:0/2835938143 wait complete. 2023-07-14T11:54:43.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.315+0000 7f9cc8cb1700 1 Processor -- start 2023-07-14T11:54:43.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.315+0000 7f9cc8cb1700 1 -- start start 2023-07-14T11:54:43.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.315+0000 7f9cc8cb1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cc40b4240 0x7f9cc41542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:43.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.315+0000 7f9cc8cb1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cc415bac0 0x7f9cc41f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:43.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.315+0000 7f9cc8cb1700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cc415be90 0x7f9cc41599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:43.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.315+0000 7f9cc8cb1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc4155800 con 0x7f9cc415bac0 2023-07-14T11:54:43.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.315+0000 7f9cc8cb1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc41f5980 con 0x7f9cc40b4240 2023-07-14T11:54:43.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.315+0000 7f9cc8cb1700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc415c5e0 con 0x7f9cc415be90 2023-07-14T11:54:43.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cc2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cc415bac0 0x7f9cc41f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:43.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cc2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cc415bac0 0x7f9cc41f5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59332/0 (socket says 172.21.15.104:59332) 2023-07-14T11:54:43.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cc2ffd700 1 -- 172.21.15.104:0/4294716578 learned_addr learned my addr 172.21.15.104:0/4294716578 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:43.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cc37fe700 1 --2- 172.21.15.104:0/4294716578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cc40b4240 0x7f9cc41542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:43.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cc3fff700 1 --2- 172.21.15.104:0/4294716578 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cc415be90 0x7f9cc41599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:43.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cc2ffd700 1 -- 172.21.15.104:0/4294716578 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cc415be90 msgr2=0x7f9cc41599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cc2ffd700 1 --2- 172.21.15.104:0/4294716578 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cc415be90 0x7f9cc41599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cc2ffd700 1 -- 172.21.15.104:0/4294716578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cc40b4240 msgr2=0x7f9cc41542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cc2ffd700 1 --2- 172.21.15.104:0/4294716578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cc40b4240 0x7f9cc41542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cc2ffd700 1 -- 172.21.15.104:0/4294716578 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb8009570 con 0x7f9cc415bac0 2023-07-14T11:54:43.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cc3fff700 1 --2- 172.21.15.104:0/4294716578 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cc415be90 0x7f9cc41599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:43.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cc2ffd700 1 --2- 172.21.15.104:0/4294716578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cc415bac0 0x7f9cc41f5440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8013000 tx=0x7f9cb8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:43.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cabfff700 1 -- 172.21.15.104:0/4294716578 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8004590 con 0x7f9cc415bac0 2023-07-14T11:54:43.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cabfff700 1 -- 172.21.15.104:0/4294716578 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb801e070 con 0x7f9cc415bac0 2023-07-14T11:54:43.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cc8cb1700 1 -- 172.21.15.104:0/4294716578 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc4156d60 con 0x7f9cc415bac0 2023-07-14T11:54:43.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cabfff700 1 -- 172.21.15.104:0/4294716578 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8003900 con 0x7f9cc415bac0 2023-07-14T11:54:43.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.319+0000 7f9cc8cb1700 1 -- 172.21.15.104:0/4294716578 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc4156430 con 0x7f9cc415bac0 2023-07-14T11:54:43.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.323+0000 7f9cc8cb1700 1 -- 172.21.15.104:0/4294716578 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc404e500 con 0x7f9cc415bac0 2023-07-14T11:54:43.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.323+0000 7f9cabfff700 1 -- 172.21.15.104:0/4294716578 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9cb8003a60 con 0x7f9cc415bac0 2023-07-14T11:54:43.309 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.323+0000 7f9cabfff700 1 --2- 172.21.15.104:0/4294716578 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cb00662f0 0x7f9cb00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:43.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.323+0000 7f9cabfff700 1 -- 172.21.15.104:0/4294716578 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9cb8085130 con 0x7f9cc415bac0 2023-07-14T11:54:43.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.323+0000 7f9cc37fe700 1 --2- 172.21.15.104:0/4294716578 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cb00662f0 0x7f9cb00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:43.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.327+0000 7f9cc37fe700 1 --2- 172.21.15.104:0/4294716578 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cb00662f0 0x7f9cb00687a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009bf0 tx=0x7f9cb4009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:43.318 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.331+0000 7f9cabfff700 1 -- 172.21.15.104:0/4294716578 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f9cb8052310 con 0x7f9cc415bac0 2023-07-14T11:54:43.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.503+0000 7f9cc8cb1700 1 -- 172.21.15.104:0/4294716578 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9cc40afed0 con 0x7f9cc415bac0 2023-07-14T11:54:43.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.503+0000 7f9cabfff700 1 -- 172.21.15.104:0/4294716578 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9cb8055930 con 0x7f9cc415bac0 2023-07-14T11:54:43.490 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:54:43.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.507+0000 7f9cc8cb1700 1 -- 172.21.15.104:0/4294716578 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cb00662f0 msgr2=0x7f9cb00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.507+0000 7f9cc8cb1700 1 --2- 172.21.15.104:0/4294716578 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cb00662f0 0x7f9cb00687a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009bf0 tx=0x7f9cb4009240 comp rx=0 tx=0).stop 2023-07-14T11:54:43.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.507+0000 7f9cc8cb1700 1 -- 172.21.15.104:0/4294716578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cc415bac0 msgr2=0x7f9cc41f5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.507+0000 7f9cc8cb1700 1 --2- 172.21.15.104:0/4294716578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cc415bac0 0x7f9cc41f5440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8013000 tx=0x7f9cb8004770 comp rx=0 tx=0).stop 2023-07-14T11:54:43.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.507+0000 7f9cc8cb1700 1 -- 172.21.15.104:0/4294716578 shutdown_connections 2023-07-14T11:54:43.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.507+0000 7f9cc8cb1700 1 --2- 172.21.15.104:0/4294716578 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cb00662f0 0x7f9cb00687a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.507+0000 7f9cc8cb1700 1 --2- 172.21.15.104:0/4294716578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cc415bac0 0x7f9cc41f5440 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.507+0000 7f9cc8cb1700 1 --2- 172.21.15.104:0/4294716578 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9cc415be90 0x7f9cc41599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.507+0000 7f9cc8cb1700 1 --2- 172.21.15.104:0/4294716578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cc40b4240 0x7f9cc41542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.507+0000 7f9cc8cb1700 1 -- 172.21.15.104:0/4294716578 >> 172.21.15.104:0/4294716578 conn(0x7f9cc40b04e0 msgr2=0x7f9cc40ca560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:43.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.507+0000 7f9cc8cb1700 1 -- 172.21.15.104:0/4294716578 shutdown_connections 2023-07-14T11:54:43.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.507+0000 7f9cc8cb1700 1 -- 172.21.15.104:0/4294716578 wait complete. 2023-07-14T11:54:43.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:54:43.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c2259e700 1 Processor -- start 2023-07-14T11:54:43.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c2259e700 1 -- start start 2023-07-14T11:54:43.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c2259e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7c1c0c9d90 0x7f7c1c0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:43.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c2259e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c1c159f20 0x7f7c1c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:43.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c2259e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c1c0b0ef0 con 0x7f7c1c0caf70 2023-07-14T11:54:43.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c2259e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c1c0ce9c0 con 0x7f7c1c0c9d90 2023-07-14T11:54:43.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c2259e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c1c0cee50 con 0x7f7c1c159f20 2023-07-14T11:54:43.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c2159c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7c1c0c9d90 0x7f7c1c0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:43.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c2159c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7c1c0c9d90 0x7f7c1c0ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59334/0 (socket says 172.21.15.104:59334) 2023-07-14T11:54:43.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c2159c700 1 -- 172.21.15.104:0/94179905 learned_addr learned my addr 172.21.15.104:0/94179905 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:43.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c20d9b700 1 --2- 172.21.15.104:0/94179905 >> v2:172.21.15.121:3300/0 conn(0x7f7c1c159f20 0x7f7c1c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:43.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c2159c700 1 -- 172.21.15.104:0/94179905 >> v2:172.21.15.121:3300/0 conn(0x7f7c1c159f20 msgr2=0x7f7c1c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:43.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c2159c700 1 --2- 172.21.15.104:0/94179905 >> v2:172.21.15.121:3300/0 conn(0x7f7c1c159f20 0x7f7c1c0caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c2159c700 1 -- 172.21.15.104:0/94179905 >> v1:172.21.15.104:6789/0 conn(0x7f7c1c0caf70 legacy=0x7f7c1c0c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c2159c700 1 -- 172.21.15.104:0/94179905 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c0ca6a0 con 0x7f7c1c0c9d90 2023-07-14T11:54:43.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.703+0000 7f7c2159c700 1 --2- 172.21.15.104:0/94179905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c1c0c9d90 0x7f7c1c0ca160 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c06aa00 tx=0x7f7c18009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438f80497621fda5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:43.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.707+0000 7f7c13fff700 1 -- 172.21.15.104:0/94179905 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1800b8a0 con 0x7f7c1c0c9d90 2023-07-14T11:54:43.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.707+0000 7f7c13fff700 1 -- 172.21.15.104:0/94179905 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1800ba00 con 0x7f7c1c0c9d90 2023-07-14T11:54:43.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.707+0000 7f7c13fff700 1 -- 172.21.15.104:0/94179905 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1800bd90 con 0x7f7c1c0c9d90 2023-07-14T11:54:43.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.707+0000 7f7c2259e700 1 -- 172.21.15.104:0/94179905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c1c0c9d90 msgr2=0x7f7c1c0ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.707+0000 7f7c2259e700 1 --2- 172.21.15.104:0/94179905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c1c0c9d90 0x7f7c1c0ca160 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c06aa00 tx=0x7f7c18009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.707+0000 7f7c2259e700 1 -- 172.21.15.104:0/94179905 shutdown_connections 2023-07-14T11:54:43.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.707+0000 7f7c2259e700 1 --2- 172.21.15.104:0/94179905 >> v2:172.21.15.121:3300/0 conn(0x7f7c1c159f20 0x7f7c1c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.707+0000 7f7c2259e700 1 --2- 172.21.15.104:0/94179905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c1c0c9d90 0x7f7c1c0ca160 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.707+0000 7f7c2259e700 1 -- 172.21.15.104:0/94179905 >> 172.21.15.104:0/94179905 conn(0x7f7c1c06e960 msgr2=0x7f7c1c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:43.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.707+0000 7f7c2259e700 1 -- 172.21.15.104:0/94179905 shutdown_connections 2023-07-14T11:54:43.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.707+0000 7f7c2259e700 1 -- 172.21.15.104:0/94179905 wait complete. 2023-07-14T11:54:43.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.711+0000 7f7c2259e700 1 Processor -- start 2023-07-14T11:54:43.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.711+0000 7f7c2259e700 1 -- start start 2023-07-14T11:54:43.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.711+0000 7f7c2259e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c1c0c9d90 0x7f7c1c0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:43.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.711+0000 7f7c2259e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c1c0caf70 0x7f7c1c0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:43.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.711+0000 7f7c2259e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c1c159f20 0x7f7c1c0c3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:43.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.711+0000 7f7c2259e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c150b90 con 0x7f7c1c0caf70 2023-07-14T11:54:43.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.711+0000 7f7c2259e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c0b15c0 con 0x7f7c1c0c9d90 2023-07-14T11:54:43.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.711+0000 7f7c2259e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c0b1860 con 0x7f7c1c159f20 2023-07-14T11:54:43.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.711+0000 7f7c20d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c1c0caf70 0x7f7c1c0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:43.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.711+0000 7f7c20d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c1c0caf70 0x7f7c1c0b1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59342/0 (socket says 172.21.15.104:59342) 2023-07-14T11:54:43.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.711+0000 7f7c20d9b700 1 -- 172.21.15.104:0/1468440759 learned_addr learned my addr 172.21.15.104:0/1468440759 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:43.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.711+0000 7f7c2159c700 1 --2- 172.21.15.104:0/1468440759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c1c0c9d90 0x7f7c1c0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:43.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.711+0000 7f7c21d9d700 1 --2- 172.21.15.104:0/1468440759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c1c159f20 0x7f7c1c0c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:43.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.715+0000 7f7c20d9b700 1 -- 172.21.15.104:0/1468440759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c1c159f20 msgr2=0x7f7c1c0c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.715+0000 7f7c20d9b700 1 --2- 172.21.15.104:0/1468440759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c1c159f20 0x7f7c1c0c3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.715+0000 7f7c20d9b700 1 -- 172.21.15.104:0/1468440759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c1c0c9d90 msgr2=0x7f7c1c0cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.715+0000 7f7c20d9b700 1 --2- 172.21.15.104:0/1468440759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c1c0c9d90 0x7f7c1c0cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.715+0000 7f7c20d9b700 1 -- 172.21.15.104:0/1468440759 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c18009570 con 0x7f7c1c0caf70 2023-07-14T11:54:43.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.715+0000 7f7c2159c700 1 --2- 172.21.15.104:0/1468440759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c1c0c9d90 0x7f7c1c0cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:43.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.715+0000 7f7c20d9b700 1 --2- 172.21.15.104:0/1468440759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c1c0caf70 0x7f7c1c0b1080 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00e760 tx=0x7f7c0c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:43.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.715+0000 7f7c11ffb700 1 -- 172.21.15.104:0/1468440759 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c013610 con 0x7f7c1c0caf70 2023-07-14T11:54:43.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.715+0000 7f7c11ffb700 1 -- 172.21.15.104:0/1468440759 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0c013770 con 0x7f7c1c0caf70 2023-07-14T11:54:43.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.715+0000 7f7c2259e700 1 -- 172.21.15.104:0/1468440759 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1c0b1b40 con 0x7f7c1c0caf70 2023-07-14T11:54:43.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.715+0000 7f7c11ffb700 1 -- 172.21.15.104:0/1468440759 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c013ae0 con 0x7f7c1c0caf70 2023-07-14T11:54:43.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.715+0000 7f7c2259e700 1 -- 172.21.15.104:0/1468440759 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1c0b2100 con 0x7f7c1c0caf70 2023-07-14T11:54:43.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.715+0000 7f7c2259e700 1 -- 172.21.15.104:0/1468440759 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c1c150cd0 con 0x7f7c1c0caf70 2023-07-14T11:54:43.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.719+0000 7f7c11ffb700 1 -- 172.21.15.104:0/1468440759 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7c0c013c40 con 0x7f7c1c0caf70 2023-07-14T11:54:43.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.719+0000 7f7c11ffb700 1 --2- 172.21.15.104:0/1468440759 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c00066320 0x7f7c000687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:43.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.719+0000 7f7c11ffb700 1 -- 172.21.15.104:0/1468440759 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7c0c083d10 con 0x7f7c1c0caf70 2023-07-14T11:54:43.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.719+0000 7f7c2159c700 1 --2- 172.21.15.104:0/1468440759 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c00066320 0x7f7c000687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:43.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.719+0000 7f7c2159c700 1 --2- 172.21.15.104:0/1468440759 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c00066320 0x7f7c000687d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7c18000c00 tx=0x7f7c18008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:43.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.727+0000 7f7c11ffb700 1 -- 172.21.15.104:0/1468440759 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f7c1c150cd0 con 0x7f7c1c0caf70 2023-07-14T11:54:43.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.931+0000 7f7c2259e700 1 -- 172.21.15.104:0/1468440759 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c1c150cd0 con 0x7f7c1c0caf70 2023-07-14T11:54:43.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.935+0000 7f7c11ffb700 1 -- 172.21.15.104:0/1468440759 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4717 (secure 0 0 0) 0x7f7c1c150cd0 con 0x7f7c1c0caf70 2023-07-14T11:54:43.920 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:43.920 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2103358277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2103358277},{"type":"v1","addr":"172.21.15.121:6833","nonce":2103358277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1841065287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1841065287},{"type":"v1","addr":"172.21.15.121:6835","nonce":1841065287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3342053518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3342053518},{"type":"v1","addr":"172.21.15.104:6835","nonce":3342053518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:53:15.029045+0000","modified":"2023-07-14T11:53:41.785898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/2666329866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2666329866},{"type":"v1","addr":"172.21.15.104:6837","nonce":2666329866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:54:43.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.935+0000 7f7c2259e700 1 -- 172.21.15.104:0/1468440759 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c00066320 msgr2=0x7f7c000687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.935+0000 7f7c2259e700 1 --2- 172.21.15.104:0/1468440759 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c00066320 0x7f7c000687d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7c18000c00 tx=0x7f7c18008030 comp rx=0 tx=0).stop 2023-07-14T11:54:43.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.935+0000 7f7c2259e700 1 -- 172.21.15.104:0/1468440759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c1c0caf70 msgr2=0x7f7c1c0b1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:43.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.935+0000 7f7c2259e700 1 --2- 172.21.15.104:0/1468440759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c1c0caf70 0x7f7c1c0b1080 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00e760 tx=0x7f7c0c00eb20 comp rx=0 tx=0).stop 2023-07-14T11:54:43.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.939+0000 7f7c2259e700 1 -- 172.21.15.104:0/1468440759 shutdown_connections 2023-07-14T11:54:43.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.939+0000 7f7c2259e700 1 --2- 172.21.15.104:0/1468440759 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c00066320 0x7f7c000687d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.939+0000 7f7c2259e700 1 --2- 172.21.15.104:0/1468440759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c1c0caf70 0x7f7c1c0b1080 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.939+0000 7f7c2259e700 1 --2- 172.21.15.104:0/1468440759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c1c159f20 0x7f7c1c0c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.939+0000 7f7c2259e700 1 --2- 172.21.15.104:0/1468440759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c1c0c9d90 0x7f7c1c0cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:43.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.939+0000 7f7c2259e700 1 -- 172.21.15.104:0/1468440759 >> 172.21.15.104:0/1468440759 conn(0x7f7c1c06e960 msgr2=0x7f7c1c15bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:43.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.939+0000 7f7c2259e700 1 -- 172.21.15.104:0/1468440759 shutdown_connections 2023-07-14T11:54:43.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:43.939+0000 7f7c2259e700 1 -- 172.21.15.104:0/1468440759 wait complete. 2023-07-14T11:54:43.925 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 9 2023-07-14T11:54:43.944 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:44.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.127+0000 7f2ab759e700 1 Processor -- start 2023-07-14T11:54:44.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.127+0000 7f2ab759e700 1 -- start start 2023-07-14T11:54:44.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.127+0000 7f2ab759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2ab815ffa0 0x7f2ab806d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.127+0000 7f2ab759e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2ab806dc50 0x7f2ab80caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.127+0000 7f2ab759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ab8160370 0x7f2ab80cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.127+0000 7f2ab759e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab80ca8a0 con 0x7f2ab806dc50 2023-07-14T11:54:44.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.127+0000 7f2ab759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab80cbcd0 con 0x7f2ab815ffa0 2023-07-14T11:54:44.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.127+0000 7f2ab759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab80cbf70 con 0x7f2ab8160370 2023-07-14T11:54:44.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab5d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2ab806dc50 0x7f2ab80caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab5d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2ab806dc50 0x7f2ab80caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59354/0 (socket says 172.21.15.104:59354) 2023-07-14T11:54:44.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab5d9b700 1 -- 172.21.15.104:0/3901986880 learned_addr learned my addr 172.21.15.104:0/3901986880 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:44.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab659c700 1 --2- 172.21.15.104:0/3901986880 >> v2:172.21.15.121:3300/0 conn(0x7f2ab815ffa0 0x7f2ab806d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab6d9d700 1 --2- 172.21.15.104:0/3901986880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ab8160370 0x7f2ab80cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab5d9b700 1 -- 172.21.15.104:0/3901986880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ab8160370 msgr2=0x7f2ab80cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:44.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab5d9b700 1 --2- 172.21.15.104:0/3901986880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ab8160370 0x7f2ab80cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab5d9b700 1 -- 172.21.15.104:0/3901986880 >> v2:172.21.15.121:3300/0 conn(0x7f2ab815ffa0 msgr2=0x7f2ab806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:44.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab5d9b700 1 --2- 172.21.15.104:0/3901986880 >> v2:172.21.15.121:3300/0 conn(0x7f2ab815ffa0 0x7f2ab806d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab5d9b700 1 -- 172.21.15.104:0/3901986880 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab80cc250 con 0x7f2ab806dc50 2023-07-14T11:54:44.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab6d9d700 1 --2- 172.21.15.104:0/3901986880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ab8160370 0x7f2ab80cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:44.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab5d9b700 1 --2- 172.21.15.104:0/3901986880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ab806dc50 0x7f2ab80caa70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80caa40 tx=0x7f2aa0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf56e8ae086db67 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:44.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab559a700 1 -- 172.21.15.104:0/3901986880 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0011830 con 0x7f2ab806dc50 2023-07-14T11:54:44.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab559a700 1 -- 172.21.15.104:0/3901986880 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa0011990 con 0x7f2ab806dc50 2023-07-14T11:54:44.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab559a700 1 -- 172.21.15.104:0/3901986880 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0011d20 con 0x7f2ab806dc50 2023-07-14T11:54:44.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab759e700 1 -- 172.21.15.104:0/3901986880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ab806dc50 msgr2=0x7f2ab80caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab759e700 1 --2- 172.21.15.104:0/3901986880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ab806dc50 0x7f2ab80caa70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80caa40 tx=0x7f2aa0009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab759e700 1 -- 172.21.15.104:0/3901986880 shutdown_connections 2023-07-14T11:54:44.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab759e700 1 --2- 172.21.15.104:0/3901986880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ab8160370 0x7f2ab80cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab759e700 1 --2- 172.21.15.104:0/3901986880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ab806dc50 0x7f2ab80caa70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab759e700 1 --2- 172.21.15.104:0/3901986880 >> v2:172.21.15.121:3300/0 conn(0x7f2ab815ffa0 0x7f2ab806d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab759e700 1 -- 172.21.15.104:0/3901986880 >> 172.21.15.104:0/3901986880 conn(0x7f2ab806e960 msgr2=0x7f2ab806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:44.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.131+0000 7f2ab759e700 1 -- 172.21.15.104:0/3901986880 shutdown_connections 2023-07-14T11:54:44.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.135+0000 7f2ab759e700 1 -- 172.21.15.104:0/3901986880 wait complete. 2023-07-14T11:54:44.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab759e700 1 Processor -- start 2023-07-14T11:54:44.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab759e700 1 -- start start 2023-07-14T11:54:44.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ab806dc50 0x7f2ab80c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab759e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ab815ffa0 0x7f2ab8154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ab8160370 0x7f2ab81517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab759e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab80c7f80 con 0x7f2ab815ffa0 2023-07-14T11:54:44.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab8155340 con 0x7f2ab806dc50 2023-07-14T11:54:44.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab81f9d50 con 0x7f2ab8160370 2023-07-14T11:54:44.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab5d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ab815ffa0 0x7f2ab8154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab5d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ab815ffa0 0x7f2ab8154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59358/0 (socket says 172.21.15.104:59358) 2023-07-14T11:54:44.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab5d9b700 1 -- 172.21.15.104:0/803062679 learned_addr learned my addr 172.21.15.104:0/803062679 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:44.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab659c700 1 --2- 172.21.15.104:0/803062679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ab806dc50 0x7f2ab80c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab6d9d700 1 --2- 172.21.15.104:0/803062679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ab8160370 0x7f2ab81517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab5d9b700 1 -- 172.21.15.104:0/803062679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ab8160370 msgr2=0x7f2ab81517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab5d9b700 1 --2- 172.21.15.104:0/803062679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ab8160370 0x7f2ab81517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab5d9b700 1 -- 172.21.15.104:0/803062679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ab806dc50 msgr2=0x7f2ab80c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab5d9b700 1 --2- 172.21.15.104:0/803062679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ab806dc50 0x7f2ab80c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab5d9b700 1 -- 172.21.15.104:0/803062679 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa0009570 con 0x7f2ab815ffa0 2023-07-14T11:54:44.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab659c700 1 --2- 172.21.15.104:0/803062679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ab806dc50 0x7f2ab80c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:44.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab5d9b700 1 --2- 172.21.15.104:0/803062679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ab815ffa0 0x7f2ab8154e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000c00 tx=0x7f2aa0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:44.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2aaeffd700 1 -- 172.21.15.104:0/803062679 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0004450 con 0x7f2ab815ffa0 2023-07-14T11:54:44.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2aaeffd700 1 -- 172.21.15.104:0/803062679 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa001e070 con 0x7f2ab815ffa0 2023-07-14T11:54:44.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2aaeffd700 1 -- 172.21.15.104:0/803062679 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0003900 con 0x7f2ab815ffa0 2023-07-14T11:54:44.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.139+0000 7f2ab759e700 1 -- 172.21.15.104:0/803062679 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab81f8f90 con 0x7f2ab815ffa0 2023-07-14T11:54:44.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.143+0000 7f2ab759e700 1 -- 172.21.15.104:0/803062679 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab81f9550 con 0x7f2ab815ffa0 2023-07-14T11:54:44.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.143+0000 7f2ab759e700 1 -- 172.21.15.104:0/803062679 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab8071620 con 0x7f2ab815ffa0 2023-07-14T11:54:44.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.143+0000 7f2aaeffd700 1 -- 172.21.15.104:0/803062679 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2aa0003a60 con 0x7f2ab815ffa0 2023-07-14T11:54:44.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.143+0000 7f2aaeffd700 1 --2- 172.21.15.104:0/803062679 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2a980662f0 0x7f2a980687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.147+0000 7f2aaeffd700 1 -- 172.21.15.104:0/803062679 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2aa00849f0 con 0x7f2ab815ffa0 2023-07-14T11:54:44.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.147+0000 7f2ab659c700 1 --2- 172.21.15.104:0/803062679 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2a980662f0 0x7f2a980687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.147+0000 7f2ab659c700 1 --2- 172.21.15.104:0/803062679 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2a980662f0 0x7f2a980687a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8163920 tx=0x7f2aa8009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:44.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.151+0000 7f2aaeffd700 1 -- 172.21.15.104:0/803062679 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f2aa0051cf0 con 0x7f2ab815ffa0 2023-07-14T11:54:44.309 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.323+0000 7f2ab759e700 1 -- 172.21.15.104:0/803062679 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ab804e500 con 0x7f2ab815ffa0 2023-07-14T11:54:44.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.327+0000 7f2aaeffd700 1 -- 172.21.15.104:0/803062679 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20251 (secure 0 0 0) 0x7f2aa00045d0 con 0x7f2ab815ffa0 2023-07-14T11:54:44.313 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:44.313 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":30,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:55.601783+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:44.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.331+0000 7f2ab759e700 1 -- 172.21.15.104:0/803062679 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2a980662f0 msgr2=0x7f2a980687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.331+0000 7f2ab759e700 1 --2- 172.21.15.104:0/803062679 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2a980662f0 0x7f2a980687a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8163920 tx=0x7f2aa8009240 comp rx=0 tx=0).stop 2023-07-14T11:54:44.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.331+0000 7f2ab759e700 1 -- 172.21.15.104:0/803062679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ab815ffa0 msgr2=0x7f2ab8154e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.331+0000 7f2ab759e700 1 --2- 172.21.15.104:0/803062679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ab815ffa0 0x7f2ab8154e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000c00 tx=0x7f2aa0004770 comp rx=0 tx=0).stop 2023-07-14T11:54:44.316 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.331+0000 7f2ab759e700 1 -- 172.21.15.104:0/803062679 shutdown_connections 2023-07-14T11:54:44.316 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.331+0000 7f2ab759e700 1 --2- 172.21.15.104:0/803062679 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2a980662f0 0x7f2a980687a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.316 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.331+0000 7f2ab759e700 1 --2- 172.21.15.104:0/803062679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ab815ffa0 0x7f2ab8154e00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.316 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.331+0000 7f2ab759e700 1 --2- 172.21.15.104:0/803062679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2ab8160370 0x7f2ab81517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.317 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.331+0000 7f2ab759e700 1 --2- 172.21.15.104:0/803062679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ab806dc50 0x7f2ab80c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.317 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.331+0000 7f2ab759e700 1 -- 172.21.15.104:0/803062679 >> 172.21.15.104:0/803062679 conn(0x7f2ab806e960 msgr2=0x7f2ab80cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:44.317 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.331+0000 7f2ab759e700 1 -- 172.21.15.104:0/803062679 shutdown_connections 2023-07-14T11:54:44.317 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.331+0000 7f2ab759e700 1 -- 172.21.15.104:0/803062679 wait complete. 2023-07-14T11:54:44.336 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-14T11:54:44.337 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:54:44.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.527+0000 7f68586df700 1 Processor -- start 2023-07-14T11:54:44.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.527+0000 7f68586df700 1 -- start start 2023-07-14T11:54:44.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.527+0000 7f68586df700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f68500d1430 0x7f68500b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.527+0000 7f68586df700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685015c2f0 con 0x7f685015bf20 2023-07-14T11:54:44.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.527+0000 7f68586df700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6850070830 con 0x7f68500b1530 2023-07-14T11:54:44.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.527+0000 7f68586df700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68500d1830 con 0x7f68500d1430 2023-07-14T11:54:44.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.527+0000 7f685647b700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f685015bf20 0x7f68500b5420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:36848/0 (socket says 172.21.15.104:36848) 2023-07-14T11:54:44.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.527+0000 7f685647b700 1 -- 172.21.15.104:0/2736036100 learned_addr learned my addr 172.21.15.104:0/2736036100 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:44.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6856c7c700 1 --2- 172.21.15.104:0/2736036100 >> v2:172.21.15.121:3300/0 conn(0x7f68500d1430 0x7f68500b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 -- 172.21.15.104:0/2736036100 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720682013 0 0) 0x7f685015c2f0 con 0x7f685015bf20 2023-07-14T11:54:44.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 -- 172.21.15.104:0/2736036100 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f683c0035d0 con 0x7f685015bf20 2023-07-14T11:54:44.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 -- 172.21.15.104:0/2736036100 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3199423939 0 0) 0x7f683c0035d0 con 0x7f685015bf20 2023-07-14T11:54:44.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 -- 172.21.15.104:0/2736036100 >> v2:172.21.15.121:3300/0 conn(0x7f68500d1430 msgr2=0x7f68500b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:44.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 --2- 172.21.15.104:0/2736036100 >> v2:172.21.15.121:3300/0 conn(0x7f68500d1430 0x7f68500b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 -- 172.21.15.104:0/2736036100 >> v1:172.21.15.121:6789/0 conn(0x7f68500b1530 legacy=0x7f68500d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 -- 172.21.15.104:0/2736036100 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6850070b10 con 0x7f685015bf20 2023-07-14T11:54:44.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 -- 172.21.15.104:0/2736036100 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f684c001670 con 0x7f685015bf20 2023-07-14T11:54:44.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 -- 172.21.15.104:0/2736036100 >> v1:172.21.15.104:6789/0 conn(0x7f685015bf20 legacy=0x7f68500b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 --2- 172.21.15.104:0/2736036100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c003a20 0x7f683c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 --2- 172.21.15.104:0/2736036100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f683c004870 0x7f683c004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 --2- 172.21.15.104:0/2736036100 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f683c005340 0x7f683c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 -- 172.21.15.104:0/2736036100 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c00c1d0 con 0x7f683c003a20 2023-07-14T11:54:44.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 -- 172.21.15.104:0/2736036100 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c00c340 con 0x7f683c004870 2023-07-14T11:54:44.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 -- 172.21.15.104:0/2736036100 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c00c500 con 0x7f683c005340 2023-07-14T11:54:44.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f685647b700 1 --2- 172.21.15.104:0/2736036100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c003a20 0x7f683c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6856c7c700 1 --2- 172.21.15.104:0/2736036100 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f683c005340 0x7f683c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855c7a700 1 --2- 172.21.15.104:0/2736036100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f683c004870 0x7f683c004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f685647b700 1 -- 172.21.15.104:0/2736036100 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f683c005340 msgr2=0x7f683c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:44.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f685647b700 1 --2- 172.21.15.104:0/2736036100 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f683c005340 0x7f683c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f685647b700 1 -- 172.21.15.104:0/2736036100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f683c004870 msgr2=0x7f683c004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:44.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f685647b700 1 --2- 172.21.15.104:0/2736036100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f683c004870 0x7f683c004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f685647b700 1 -- 172.21.15.104:0/2736036100 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c0035d0 con 0x7f683c003a20 2023-07-14T11:54:44.521 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6856c7c700 1 -- 172.21.15.104:0/2736036100 reap_dead start 2023-07-14T11:54:44.521 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f685647b700 1 --2- 172.21.15.104:0/2736036100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c003a20 0x7f683c003e30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c320 tx=0x7f683c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c50eab0f7a0e0f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:44.521 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 -- 172.21.15.104:0/2736036100 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c007c30 con 0x7f683c003a20 2023-07-14T11:54:44.521 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 -- 172.21.15.104:0/2736036100 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c00ce80 con 0x7f683c003a20 2023-07-14T11:54:44.521 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.531+0000 7f6855479700 1 -- 172.21.15.104:0/2736036100 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c005bc0 con 0x7f683c003a20 2023-07-14T11:54:44.521 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.535+0000 7f68586df700 1 -- 172.21.15.104:0/2736036100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c003a20 msgr2=0x7f683c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.521 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.535+0000 7f68586df700 1 --2- 172.21.15.104:0/2736036100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c003a20 0x7f683c003e30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c320 tx=0x7f683c0058a0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.521 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.535+0000 7f68586df700 1 -- 172.21.15.104:0/2736036100 shutdown_connections 2023-07-14T11:54:44.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.535+0000 7f68586df700 1 --2- 172.21.15.104:0/2736036100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c003a20 0x7f683c003e30 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.535+0000 7f68586df700 1 -- 172.21.15.104:0/2736036100 >> 172.21.15.104:0/2736036100 conn(0x7f685006fc80 msgr2=0x7f6850070080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:44.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.535+0000 7f68586df700 1 -- 172.21.15.104:0/2736036100 shutdown_connections 2023-07-14T11:54:44.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.535+0000 7f68586df700 1 -- 172.21.15.104:0/2736036100 wait complete. 2023-07-14T11:54:44.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f68586df700 1 Processor -- start 2023-07-14T11:54:44.523 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f68586df700 1 -- start start 2023-07-14T11:54:44.524 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f68586df700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c003a20 0x7f68500c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.524 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f68586df700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6850159c60 0x7f685015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.524 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f68586df700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68500c8440 0x7f68500c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f68586df700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500b7a50 con 0x7f683c003a20 2023-07-14T11:54:44.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f68586df700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685015a610 con 0x7f68500c8440 2023-07-14T11:54:44.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f68586df700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500c9170 con 0x7f6850159c60 2023-07-14T11:54:44.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f685647b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c003a20 0x7f68500c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f685647b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c003a20 0x7f68500c6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59380/0 (socket says 172.21.15.104:59380) 2023-07-14T11:54:44.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f685647b700 1 -- 172.21.15.104:0/3363138942 learned_addr learned my addr 172.21.15.104:0/3363138942 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:44.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f6856c7c700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68500c8440 0x7f68500c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f6855c7a700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6850159c60 0x7f685015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f685647b700 1 -- 172.21.15.104:0/3363138942 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6850159c60 msgr2=0x7f685015a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f685647b700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6850159c60 0x7f685015a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f685647b700 1 -- 172.21.15.104:0/3363138942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68500c8440 msgr2=0x7f68500c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f685647b700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68500c8440 0x7f68500c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f685647b700 1 -- 172.21.15.104:0/3363138942 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685015c610 con 0x7f683c003a20 2023-07-14T11:54:44.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f6855c7a700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6850159c60 0x7f685015a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:44.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f6856c7c700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68500c8440 0x7f68500c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:44.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f685647b700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c003a20 0x7f68500c6a40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c910 tx=0x7f684c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:44.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f6846ffd700 1 -- 172.21.15.104:0/3363138942 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c004ca0 con 0x7f683c003a20 2023-07-14T11:54:44.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f6846ffd700 1 -- 172.21.15.104:0/3363138942 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c004e00 con 0x7f683c003a20 2023-07-14T11:54:44.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f6846ffd700 1 -- 172.21.15.104:0/3363138942 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c007310 con 0x7f683c003a20 2023-07-14T11:54:44.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f68586df700 1 -- 172.21.15.104:0/3363138942 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685015c8f0 con 0x7f683c003a20 2023-07-14T11:54:44.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.539+0000 7f68586df700 1 -- 172.21.15.104:0/3363138942 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685015ceb0 con 0x7f683c003a20 2023-07-14T11:54:44.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.543+0000 7f68586df700 1 -- 172.21.15.104:0/3363138942 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685015d040 con 0x7f683c003a20 2023-07-14T11:54:44.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.543+0000 7f6846ffd700 1 -- 172.21.15.104:0/3363138942 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f684c007470 con 0x7f683c003a20 2023-07-14T11:54:44.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.543+0000 7f6846ffd700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f68340662f0 0x7f68340687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.543+0000 7f6846ffd700 1 -- 172.21.15.104:0/3363138942 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f684c085000 con 0x7f683c003a20 2023-07-14T11:54:44.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.543+0000 7f6855c7a700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f68340662f0 0x7f68340687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.547+0000 7f6855c7a700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f68340662f0 0x7f68340687a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6840000c00 tx=0x7f6840009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:44.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.551+0000 7f6846ffd700 1 -- 172.21.15.104:0/3363138942 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f684c0521e0 con 0x7f683c003a20 2023-07-14T11:54:44.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.755+0000 7f68586df700 1 -- 172.21.15.104:0/3363138942 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f685004dcb0 con 0x7f683c003a20 2023-07-14T11:54:44.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.755+0000 7f6846ffd700 1 -- 172.21.15.104:0/3363138942 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4717 (secure 0 0 0) 0x7f684c007d90 con 0x7f683c003a20 2023-07-14T11:54:44.742 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:44.743 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2103358277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2103358277},{"type":"v1","addr":"172.21.15.121:6833","nonce":2103358277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1841065287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1841065287},{"type":"v1","addr":"172.21.15.121:6835","nonce":1841065287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3342053518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3342053518},{"type":"v1","addr":"172.21.15.104:6835","nonce":3342053518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:53:15.029045+0000","modified":"2023-07-14T11:53:41.785898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/2666329866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2666329866},{"type":"v1","addr":"172.21.15.104:6837","nonce":2666329866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:54:44.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.759+0000 7f68586df700 1 -- 172.21.15.104:0/3363138942 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f68340662f0 msgr2=0x7f68340687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.759+0000 7f68586df700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f68340662f0 0x7f68340687a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6840000c00 tx=0x7f6840009040 comp rx=0 tx=0).stop 2023-07-14T11:54:44.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.759+0000 7f68586df700 1 -- 172.21.15.104:0/3363138942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c003a20 msgr2=0x7f68500c6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.759+0000 7f68586df700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c003a20 0x7f68500c6a40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c910 tx=0x7f684c001a00 comp rx=0 tx=0).stop 2023-07-14T11:54:44.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.759+0000 7f68586df700 1 -- 172.21.15.104:0/3363138942 shutdown_connections 2023-07-14T11:54:44.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.759+0000 7f68586df700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f68340662f0 0x7f68340687a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.759+0000 7f68586df700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68500c8440 0x7f68500c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.759+0000 7f68586df700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6850159c60 0x7f685015a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.759+0000 7f68586df700 1 --2- 172.21.15.104:0/3363138942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c003a20 0x7f68500c6a40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.759+0000 7f68586df700 1 -- 172.21.15.104:0/3363138942 >> 172.21.15.104:0/3363138942 conn(0x7f685006fc80 msgr2=0x7f68500b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:44.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.759+0000 7f68586df700 1 -- 172.21.15.104:0/3363138942 shutdown_connections 2023-07-14T11:54:44.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.759+0000 7f68586df700 1 -- 172.21.15.104:0/3363138942 wait complete. 2023-07-14T11:54:44.747 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 9 2023-07-14T11:54:44.767 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:54:44.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.955+0000 7f319efe1700 1 Processor -- start 2023-07-14T11:54:44.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.955+0000 7f319efe1700 1 -- start start 2023-07-14T11:54:44.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.955+0000 7f319efe1700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31981599d0 0x7f3198071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.955+0000 7f319efe1700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31980cde10 con 0x7f319815ccf0 2023-07-14T11:54:44.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.955+0000 7f319efe1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31980cd520 con 0x7f31980caa20 2023-07-14T11:54:44.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.955+0000 7f319efe1700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31980715e0 con 0x7f31981599d0 2023-07-14T11:54:44.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319d7de700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f319815ccf0 0x7f31980c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:36862/0 (socket says 172.21.15.104:36862) 2023-07-14T11:54:44.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319d7de700 1 -- 172.21.15.104:0/308324496 learned_addr learned my addr 172.21.15.104:0/308324496 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:44.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319dfdf700 1 --2- 172.21.15.104:0/308324496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31981599d0 0x7f3198071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319cfdd700 1 -- 172.21.15.104:0/308324496 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437341026 0 0) 0x7f31980cd520 con 0x7f31980caa20 2023-07-14T11:54:44.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319cfdd700 1 -- 172.21.15.104:0/308324496 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31840035d0 con 0x7f31980caa20 2023-07-14T11:54:44.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319cfdd700 1 -- 172.21.15.104:0/308324496 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199859472 0 0) 0x7f31980cde10 con 0x7f319815ccf0 2023-07-14T11:54:44.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319cfdd700 1 -- 172.21.15.104:0/308324496 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31980cd520 con 0x7f319815ccf0 2023-07-14T11:54:44.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319dfdf700 1 -- 172.21.15.104:0/308324496 >> v1:172.21.15.121:6789/0 conn(0x7f31980caa20 legacy=0x7f31980ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319dfdf700 1 -- 172.21.15.104:0/308324496 >> v1:172.21.15.104:6789/0 conn(0x7f319815ccf0 legacy=0x7f31980c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319dfdf700 1 -- 172.21.15.104:0/308324496 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31980cd7c0 con 0x7f31981599d0 2023-07-14T11:54:44.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319dfdf700 1 --2- 172.21.15.104:0/308324496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31981599d0 0x7f3198071000 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3194009dc0 tx=0x7f319400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b84146b8aa354a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:44.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319cfdd700 1 -- 172.21.15.104:0/308324496 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319400e070 con 0x7f31981599d0 2023-07-14T11:54:44.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319cfdd700 1 -- 172.21.15.104:0/308324496 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319400bb20 con 0x7f31981599d0 2023-07-14T11:54:44.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319cfdd700 1 -- 172.21.15.104:0/308324496 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194012430 con 0x7f31981599d0 2023-07-14T11:54:44.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319efe1700 1 -- 172.21.15.104:0/308324496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31981599d0 msgr2=0x7f3198071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319efe1700 1 --2- 172.21.15.104:0/308324496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31981599d0 0x7f3198071000 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3194009dc0 tx=0x7f319400b330 comp rx=0 tx=0).stop 2023-07-14T11:54:44.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319efe1700 1 -- 172.21.15.104:0/308324496 shutdown_connections 2023-07-14T11:54:44.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319efe1700 1 --2- 172.21.15.104:0/308324496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31981599d0 0x7f3198071000 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319efe1700 1 -- 172.21.15.104:0/308324496 >> 172.21.15.104:0/308324496 conn(0x7f319806e960 msgr2=0x7f319806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:44.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319efe1700 1 -- 172.21.15.104:0/308324496 shutdown_connections 2023-07-14T11:54:44.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.959+0000 7f319efe1700 1 -- 172.21.15.104:0/308324496 wait complete. 2023-07-14T11:54:44.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.963+0000 7f319efe1700 1 Processor -- start 2023-07-14T11:54:44.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.963+0000 7f319efe1700 1 -- start start 2023-07-14T11:54:44.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.963+0000 7f319efe1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31980caa20 0x7f31980c18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.963+0000 7f319efe1700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31981599d0 0x7f3198145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.963+0000 7f319efe1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f319815ccf0 0x7f31980c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.963+0000 7f319efe1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31980c3320 con 0x7f31980caa20 2023-07-14T11:54:44.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.963+0000 7f319efe1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31980b1640 con 0x7f319815ccf0 2023-07-14T11:54:44.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.963+0000 7f319efe1700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31980c1de0 con 0x7f31981599d0 2023-07-14T11:54:44.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.963+0000 7f319dfdf700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31980caa20 0x7f31980c18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.963+0000 7f319dfdf700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31980caa20 0x7f31980c18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59386/0 (socket says 172.21.15.104:59386) 2023-07-14T11:54:44.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.963+0000 7f319dfdf700 1 -- 172.21.15.104:0/636907646 learned_addr learned my addr 172.21.15.104:0/636907646 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:44.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.967+0000 7f319e7e0700 1 --2- 172.21.15.104:0/636907646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f319815ccf0 0x7f31980c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.967+0000 7f319d7de700 1 --2- 172.21.15.104:0/636907646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31981599d0 0x7f3198145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.967+0000 7f319dfdf700 1 -- 172.21.15.104:0/636907646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31981599d0 msgr2=0x7f3198145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.967+0000 7f319dfdf700 1 --2- 172.21.15.104:0/636907646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31981599d0 0x7f3198145470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.967+0000 7f319dfdf700 1 -- 172.21.15.104:0/636907646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f319815ccf0 msgr2=0x7f31980c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.967+0000 7f319dfdf700 1 --2- 172.21.15.104:0/636907646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f319815ccf0 0x7f31980c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.967+0000 7f319dfdf700 1 -- 172.21.15.104:0/636907646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3194009a70 con 0x7f31980caa20 2023-07-14T11:54:44.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.967+0000 7f319dfdf700 1 --2- 172.21.15.104:0/636907646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31980caa20 0x7f31980c18a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3194014040 tx=0x7f3194004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:44.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.967+0000 7f318e7fc700 1 -- 172.21.15.104:0/636907646 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319401b810 con 0x7f31980caa20 2023-07-14T11:54:44.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.967+0000 7f318e7fc700 1 -- 172.21.15.104:0/636907646 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3194020070 con 0x7f31980caa20 2023-07-14T11:54:44.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.967+0000 7f318e7fc700 1 -- 172.21.15.104:0/636907646 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319401bbd0 con 0x7f31980caa20 2023-07-14T11:54:44.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.967+0000 7f319efe1700 1 -- 172.21.15.104:0/636907646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31981459b0 con 0x7f31980caa20 2023-07-14T11:54:44.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.967+0000 7f319efe1700 1 -- 172.21.15.104:0/636907646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31980c3550 con 0x7f31980caa20 2023-07-14T11:54:44.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.967+0000 7f319efe1700 1 -- 172.21.15.104:0/636907646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319804e500 con 0x7f31980caa20 2023-07-14T11:54:44.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.971+0000 7f318e7fc700 1 -- 172.21.15.104:0/636907646 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f319401bd30 con 0x7f31980caa20 2023-07-14T11:54:44.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.971+0000 7f318e7fc700 1 --2- 172.21.15.104:0/636907646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f317c066320 0x7f317c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.971+0000 7f318e7fc700 1 -- 172.21.15.104:0/636907646 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3194022070 con 0x7f31980caa20 2023-07-14T11:54:44.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.971+0000 7f319d7de700 1 --2- 172.21.15.104:0/636907646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f317c066320 0x7f317c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.975+0000 7f319d7de700 1 --2- 172.21.15.104:0/636907646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f317c066320 0x7f317c0687d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f31981f4730 tx=0x7f3188009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:44.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:44.979+0000 7f318e7fc700 1 -- 172.21.15.104:0/636907646 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f3194053580 con 0x7f31980caa20 2023-07-14T11:54:45.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.183+0000 7f319efe1700 1 -- 172.21.15.104:0/636907646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3198070de0 con 0x7f31980caa20 2023-07-14T11:54:45.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.183+0000 7f318e7fc700 1 -- 172.21.15.104:0/636907646 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4717 (secure 0 0 0) 0x7f3194056ba0 con 0x7f31980caa20 2023-07-14T11:54:45.170 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:45.170 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2103358277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2103358277},{"type":"v1","addr":"172.21.15.121:6833","nonce":2103358277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1841065287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1841065287},{"type":"v1","addr":"172.21.15.121:6835","nonce":1841065287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3342053518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3342053518},{"type":"v1","addr":"172.21.15.104:6835","nonce":3342053518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:53:15.029045+0000","modified":"2023-07-14T11:53:41.785898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/2666329866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2666329866},{"type":"v1","addr":"172.21.15.104:6837","nonce":2666329866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T11:54:45.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.187+0000 7f319efe1700 1 -- 172.21.15.104:0/636907646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f317c066320 msgr2=0x7f317c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:45.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.187+0000 7f319efe1700 1 --2- 172.21.15.104:0/636907646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f317c066320 0x7f317c0687d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f31981f4730 tx=0x7f3188009040 comp rx=0 tx=0).stop 2023-07-14T11:54:45.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.187+0000 7f319efe1700 1 -- 172.21.15.104:0/636907646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31980caa20 msgr2=0x7f31980c18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:45.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.187+0000 7f319efe1700 1 --2- 172.21.15.104:0/636907646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31980caa20 0x7f31980c18a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3194014040 tx=0x7f3194004af0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.187+0000 7f319efe1700 1 -- 172.21.15.104:0/636907646 shutdown_connections 2023-07-14T11:54:45.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.187+0000 7f319efe1700 1 --2- 172.21.15.104:0/636907646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f317c066320 0x7f317c0687d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.187+0000 7f319efe1700 1 --2- 172.21.15.104:0/636907646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f319815ccf0 0x7f31980c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.187+0000 7f319efe1700 1 --2- 172.21.15.104:0/636907646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31981599d0 0x7f3198145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.187+0000 7f319efe1700 1 --2- 172.21.15.104:0/636907646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31980caa20 0x7f31980c18a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.187+0000 7f319efe1700 1 -- 172.21.15.104:0/636907646 >> 172.21.15.104:0/636907646 conn(0x7f319806e960 msgr2=0x7f319815bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:45.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.187+0000 7f319efe1700 1 -- 172.21.15.104:0/636907646 shutdown_connections 2023-07-14T11:54:45.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.187+0000 7f319efe1700 1 -- 172.21.15.104:0/636907646 wait complete. 2023-07-14T11:54:45.175 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 9 2023-07-14T11:54:45.193 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:45.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.383+0000 7f6c71040700 1 Processor -- start 2023-07-14T11:54:45.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.383+0000 7f6c71040700 1 -- start start 2023-07-14T11:54:45.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c71040700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6c6c15bff0 0x7f6c6c0ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c71040700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c0af0a0 con 0x7f6c6c0cd070 2023-07-14T11:54:45.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c71040700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c0cf720 con 0x7f6c6c0cf1c0 2023-07-14T11:54:45.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6b7fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f6c6c0cd070 0x7f6c6c0ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:36868/0 (socket says 172.21.15.104:36868) 2023-07-14T11:54:45.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6b7fe700 1 -- 172.21.15.104:0/387063898 learned_addr learned my addr 172.21.15.104:0/387063898 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:45.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c71040700 1 -- 172.21.15.104:0/387063898 --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6c0cd4f0 con 0x7f6c6c15bff0 2023-07-14T11:54:45.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6bfff700 1 --2- 172.21.15.104:0/387063898 >> v2:172.21.15.121:3300/0 conn(0x7f6c6c15bff0 0x7f6c6c0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6affd700 1 -- 172.21.15.104:0/387063898 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1226123361 0 0) 0x7f6c6c0af0a0 con 0x7f6c6c0cd070 2023-07-14T11:54:45.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6affd700 1 -- 172.21.15.104:0/387063898 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c580035d0 con 0x7f6c6c0cd070 2023-07-14T11:54:45.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6affd700 1 -- 172.21.15.104:0/387063898 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1545068766 0 0) 0x7f6c580035d0 con 0x7f6c6c0cd070 2023-07-14T11:54:45.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6affd700 1 -- 172.21.15.104:0/387063898 >> v2:172.21.15.121:3300/0 conn(0x7f6c6c15bff0 msgr2=0x7f6c6c0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:45.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6affd700 1 --2- 172.21.15.104:0/387063898 >> v2:172.21.15.121:3300/0 conn(0x7f6c6c15bff0 0x7f6c6c0ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6affd700 1 -- 172.21.15.104:0/387063898 >> v1:172.21.15.121:6789/0 conn(0x7f6c6c0cf1c0 legacy=0x7f6c6c0afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:45.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6affd700 1 -- 172.21.15.104:0/387063898 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c0af2f0 con 0x7f6c6c0cd070 2023-07-14T11:54:45.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6affd700 1 -- 172.21.15.104:0/387063898 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f6c5c001670 con 0x7f6c6c0cd070 2023-07-14T11:54:45.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6affd700 1 -- 172.21.15.104:0/387063898 >> v1:172.21.15.104:6789/0 conn(0x7f6c6c0cd070 legacy=0x7f6c6c0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:45.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6affd700 1 --2- 172.21.15.104:0/387063898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c58003a20 0x7f6c58003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6affd700 1 --2- 172.21.15.104:0/387063898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c58004870 0x7f6c58004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6affd700 1 --2- 172.21.15.104:0/387063898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c58005220 0x7f6c5800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6affd700 1 -- 172.21.15.104:0/387063898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c0cd4f0 con 0x7f6c58003a20 2023-07-14T11:54:45.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6affd700 1 -- 172.21.15.104:0/387063898 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5800c0a0 con 0x7f6c58005220 2023-07-14T11:54:45.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6affd700 1 -- 172.21.15.104:0/387063898 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5800c260 con 0x7f6c58004870 2023-07-14T11:54:45.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6bfff700 1 --2- 172.21.15.104:0/387063898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c58003a20 0x7f6c58003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c7083f700 1 --2- 172.21.15.104:0/387063898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c58005220 0x7f6c5800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6b7fe700 1 --2- 172.21.15.104:0/387063898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c58004870 0x7f6c58004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6bfff700 1 -- 172.21.15.104:0/387063898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c58004870 msgr2=0x7f6c58004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:45.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6bfff700 1 --2- 172.21.15.104:0/387063898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c58004870 0x7f6c58004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6bfff700 1 -- 172.21.15.104:0/387063898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c58005220 msgr2=0x7f6c5800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:45.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6bfff700 1 --2- 172.21.15.104:0/387063898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c58005220 0x7f6c5800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c6bfff700 1 -- 172.21.15.104:0/387063898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c580035d0 con 0x7f6c58003a20 2023-07-14T11:54:45.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.387+0000 7f6c7083f700 1 -- 172.21.15.104:0/387063898 reap_dead start 2023-07-14T11:54:45.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.391+0000 7f6c6bfff700 1 --2- 172.21.15.104:0/387063898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c58003a20 0x7f6c58003e30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009fc0 tx=0x7f6c580058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0aea33e7007f495 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:45.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.391+0000 7f6c6affd700 1 -- 172.21.15.104:0/387063898 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6400ec30 con 0x7f6c58003a20 2023-07-14T11:54:45.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.391+0000 7f6c6affd700 1 -- 172.21.15.104:0/387063898 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6400f040 con 0x7f6c58003a20 2023-07-14T11:54:45.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.391+0000 7f6c6affd700 1 -- 172.21.15.104:0/387063898 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c64014480 con 0x7f6c58003a20 2023-07-14T11:54:45.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.391+0000 7f6c71040700 1 -- 172.21.15.104:0/387063898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c58003a20 msgr2=0x7f6c58003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:45.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.391+0000 7f6c71040700 1 --2- 172.21.15.104:0/387063898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c58003a20 0x7f6c58003e30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009fc0 tx=0x7f6c580058a0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.391+0000 7f6c71040700 1 -- 172.21.15.104:0/387063898 shutdown_connections 2023-07-14T11:54:45.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.391+0000 7f6c71040700 1 --2- 172.21.15.104:0/387063898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c58003a20 0x7f6c58003e30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.391+0000 7f6c71040700 1 -- 172.21.15.104:0/387063898 >> 172.21.15.104:0/387063898 conn(0x7f6c6c060c40 msgr2=0x7f6c6c09b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:45.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.391+0000 7f6c71040700 1 -- 172.21.15.104:0/387063898 shutdown_connections 2023-07-14T11:54:45.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.391+0000 7f6c71040700 1 -- 172.21.15.104:0/387063898 wait complete. 2023-07-14T11:54:45.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.395+0000 7f6c71040700 1 Processor -- start 2023-07-14T11:54:45.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.395+0000 7f6c71040700 1 -- start start 2023-07-14T11:54:45.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.395+0000 7f6c71040700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c58003a20 0x7f6c6c1f1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.395+0000 7f6c71040700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c6c0b7420 0x7f6c6c157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.395+0000 7f6c71040700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c6c0b7890 0x7f6c6c1582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.395+0000 7f6c71040700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c1f1ed0 con 0x7f6c6c0b7890 2023-07-14T11:54:45.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.395+0000 7f6c71040700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c0b56f0 con 0x7f6c58003a20 2023-07-14T11:54:45.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.395+0000 7f6c71040700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c0b59d0 con 0x7f6c6c0b7420 2023-07-14T11:54:45.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.395+0000 7f6c7083f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c6c0b7890 0x7f6c6c1582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c7083f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c6c0b7890 0x7f6c6c1582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59410/0 (socket says 172.21.15.104:59410) 2023-07-14T11:54:45.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c7083f700 1 -- 172.21.15.104:0/3966713329 learned_addr learned my addr 172.21.15.104:0/3966713329 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:45.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c6bfff700 1 --2- 172.21.15.104:0/3966713329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c58003a20 0x7f6c6c1f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c6b7fe700 1 --2- 172.21.15.104:0/3966713329 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c6c0b7420 0x7f6c6c157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c7083f700 1 -- 172.21.15.104:0/3966713329 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c6c0b7420 msgr2=0x7f6c6c157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:45.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c7083f700 1 --2- 172.21.15.104:0/3966713329 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c6c0b7420 0x7f6c6c157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c7083f700 1 -- 172.21.15.104:0/3966713329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c58003a20 msgr2=0x7f6c6c1f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:45.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c7083f700 1 --2- 172.21.15.104:0/3966713329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c58003a20 0x7f6c6c1f1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c7083f700 1 -- 172.21.15.104:0/3966713329 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c0b5cb0 con 0x7f6c6c0b7890 2023-07-14T11:54:45.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c7083f700 1 --2- 172.21.15.104:0/3966713329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c6c0b7890 0x7f6c6c1582f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c08b750 tx=0x7f6c5400e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:45.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c68ff9700 1 -- 172.21.15.104:0/3966713329 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5400edb0 con 0x7f6c6c0b7890 2023-07-14T11:54:45.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c68ff9700 1 -- 172.21.15.104:0/3966713329 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c540050d0 con 0x7f6c6c0b7890 2023-07-14T11:54:45.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c68ff9700 1 -- 172.21.15.104:0/3966713329 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54005460 con 0x7f6c6c0b7890 2023-07-14T11:54:45.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c71040700 1 -- 172.21.15.104:0/3966713329 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6c0b5f90 con 0x7f6c6c0b7890 2023-07-14T11:54:45.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c71040700 1 -- 172.21.15.104:0/3966713329 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6c15ee50 con 0x7f6c6c0b7890 2023-07-14T11:54:45.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c71040700 1 -- 172.21.15.104:0/3966713329 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6c0b64b0 con 0x7f6c6c0b7890 2023-07-14T11:54:45.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.399+0000 7f6c68ff9700 1 -- 172.21.15.104:0/3966713329 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6c540036a0 con 0x7f6c6c0b7890 2023-07-14T11:54:45.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.403+0000 7f6c68ff9700 1 --2- 172.21.15.104:0/3966713329 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c4c0662f0 0x7f6c4c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.403+0000 7f6c68ff9700 1 -- 172.21.15.104:0/3966713329 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6c540833e0 con 0x7f6c6c0b7890 2023-07-14T11:54:45.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.403+0000 7f6c6bfff700 1 --2- 172.21.15.104:0/3966713329 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c4c0662f0 0x7f6c4c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.403+0000 7f6c6bfff700 1 --2- 172.21.15.104:0/3966713329 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c4c0662f0 0x7f6c4c0687a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6c64000f80 tx=0x7f6c64013000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:45.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.407+0000 7f6c68ff9700 1 -- 172.21.15.104:0/3966713329 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f6c6c0b64b0 con 0x7f6c6c0b7890 2023-07-14T11:54:45.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.579+0000 7f6c71040700 1 -- 172.21.15.104:0/3966713329 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c6c0b64b0 con 0x7f6c6c0b7890 2023-07-14T11:54:45.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.583+0000 7f6c68ff9700 1 -- 172.21.15.104:0/3966713329 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20251 (secure 0 0 0) 0x7f6c6c0b64b0 con 0x7f6c6c0b7890 2023-07-14T11:54:45.569 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:45.569 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":30,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:53:55.601783+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:53:12.845106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-14T11:53:14.031806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:53:17.678874+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:45.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.587+0000 7f6c71040700 1 -- 172.21.15.104:0/3966713329 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c4c0662f0 msgr2=0x7f6c4c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:45.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.587+0000 7f6c71040700 1 --2- 172.21.15.104:0/3966713329 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c4c0662f0 0x7f6c4c0687a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6c64000f80 tx=0x7f6c64013000 comp rx=0 tx=0).stop 2023-07-14T11:54:45.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.587+0000 7f6c71040700 1 -- 172.21.15.104:0/3966713329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c6c0b7890 msgr2=0x7f6c6c1582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:45.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.587+0000 7f6c71040700 1 --2- 172.21.15.104:0/3966713329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c6c0b7890 0x7f6c6c1582f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c08b750 tx=0x7f6c5400e670 comp rx=0 tx=0).stop 2023-07-14T11:54:45.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.587+0000 7f6c71040700 1 -- 172.21.15.104:0/3966713329 shutdown_connections 2023-07-14T11:54:45.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.587+0000 7f6c71040700 1 --2- 172.21.15.104:0/3966713329 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c4c0662f0 0x7f6c4c0687a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.587+0000 7f6c71040700 1 --2- 172.21.15.104:0/3966713329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c6c0b7890 0x7f6c6c1582f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.587+0000 7f6c71040700 1 --2- 172.21.15.104:0/3966713329 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c6c0b7420 0x7f6c6c157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.587+0000 7f6c71040700 1 --2- 172.21.15.104:0/3966713329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c58003a20 0x7f6c6c1f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.587+0000 7f6c71040700 1 -- 172.21.15.104:0/3966713329 >> 172.21.15.104:0/3966713329 conn(0x7f6c6c060c40 msgr2=0x7f6c6c0712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:45.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.587+0000 7f6c71040700 1 -- 172.21.15.104:0/3966713329 shutdown_connections 2023-07-14T11:54:45.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.587+0000 7f6c71040700 1 -- 172.21.15.104:0/3966713329 wait complete. 2023-07-14T11:54:45.595 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-14T11:54:45.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.779+0000 7f71db471700 1 Processor -- start 2023-07-14T11:54:45.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71db471700 1 -- start start 2023-07-14T11:54:45.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71db471700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71d40afd30 0x7f71d40b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71db471700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d40cd530 con 0x7f71d40703a0 2023-07-14T11:54:45.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71db471700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d415c440 con 0x7f71d415c070 2023-07-14T11:54:45.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71db471700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d40b06e0 con 0x7f71d40afd30 2023-07-14T11:54:45.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71d9a0e700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f71d40703a0 0x7f71d4070820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:36882/0 (socket says 172.21.15.104:36882) 2023-07-14T11:54:45.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71d9a0e700 1 -- 172.21.15.104:0/1869611583 learned_addr learned my addr 172.21.15.104:0/1869611583 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:45.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71d8a0c700 1 --2- 172.21.15.104:0/1869611583 >> v2:172.21.15.121:3300/0 conn(0x7f71d40afd30 0x7f71d40b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71cbfff700 1 -- 172.21.15.104:0/1869611583 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678714872 0 0) 0x7f71d40cd530 con 0x7f71d40703a0 2023-07-14T11:54:45.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71cbfff700 1 -- 172.21.15.104:0/1869611583 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71c00035d0 con 0x7f71d40703a0 2023-07-14T11:54:45.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71cbfff700 1 -- 172.21.15.104:0/1869611583 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2476568364 0 0) 0x7f71c00035d0 con 0x7f71d40703a0 2023-07-14T11:54:45.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71cbfff700 1 -- 172.21.15.104:0/1869611583 >> v2:172.21.15.121:3300/0 conn(0x7f71d40afd30 msgr2=0x7f71d40b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:45.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71cbfff700 1 --2- 172.21.15.104:0/1869611583 >> v2:172.21.15.121:3300/0 conn(0x7f71d40afd30 0x7f71d40b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71cbfff700 1 -- 172.21.15.104:0/1869611583 >> v1:172.21.15.121:6789/0 conn(0x7f71d415c070 legacy=0x7f71d40ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:45.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71cbfff700 1 -- 172.21.15.104:0/1869611583 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d40cc1b0 con 0x7f71d40703a0 2023-07-14T11:54:45.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71cbfff700 1 -- 172.21.15.104:0/1869611583 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f71cc0019f0 con 0x7f71d40703a0 2023-07-14T11:54:45.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71cbfff700 1 -- 172.21.15.104:0/1869611583 >> v1:172.21.15.104:6789/0 conn(0x7f71d40703a0 legacy=0x7f71d4070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:45.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71cbfff700 1 --2- 172.21.15.104:0/1869611583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71c0003a20 0x7f71c0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71cbfff700 1 --2- 172.21.15.104:0/1869611583 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71c0004870 0x7f71c0004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71cbfff700 1 --2- 172.21.15.104:0/1869611583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c0005220 0x7f71c000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71cbfff700 1 -- 172.21.15.104:0/1869611583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40b06e0 con 0x7f71c0003a20 2023-07-14T11:54:45.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71cbfff700 1 -- 172.21.15.104:0/1869611583 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c000c0a0 con 0x7f71c0005220 2023-07-14T11:54:45.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.783+0000 7f71cbfff700 1 -- 172.21.15.104:0/1869611583 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c000c260 con 0x7f71c0004870 2023-07-14T11:54:45.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71d920d700 1 --2- 172.21.15.104:0/1869611583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71c0003a20 0x7f71c0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71d9a0e700 1 --2- 172.21.15.104:0/1869611583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c0005220 0x7f71c000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71d8a0c700 1 --2- 172.21.15.104:0/1869611583 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71c0004870 0x7f71c0004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71d920d700 1 -- 172.21.15.104:0/1869611583 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71c0004870 msgr2=0x7f71c0004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:45.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71d920d700 1 --2- 172.21.15.104:0/1869611583 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71c0004870 0x7f71c0004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71d920d700 1 -- 172.21.15.104:0/1869611583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c0005220 msgr2=0x7f71c000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:45.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71d920d700 1 --2- 172.21.15.104:0/1869611583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c0005220 0x7f71c000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71d920d700 1 -- 172.21.15.104:0/1869611583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c00035d0 con 0x7f71c0003a20 2023-07-14T11:54:45.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71d9a0e700 1 -- 172.21.15.104:0/1869611583 reap_dead start 2023-07-14T11:54:45.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71d920d700 1 --2- 172.21.15.104:0/1869611583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71c0003a20 0x7f71c0003e30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f71d00017e0 tx=0x7f71c00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42af3253ed6691a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:45.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71cbfff700 1 -- 172.21.15.104:0/1869611583 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d00052d0 con 0x7f71c0003a20 2023-07-14T11:54:45.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71cbfff700 1 -- 172.21.15.104:0/1869611583 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d0005450 con 0x7f71c0003a20 2023-07-14T11:54:45.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71cbfff700 1 -- 172.21.15.104:0/1869611583 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d00038e0 con 0x7f71c0003a20 2023-07-14T11:54:45.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71db471700 1 -- 172.21.15.104:0/1869611583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71c0003a20 msgr2=0x7f71c0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:45.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71db471700 1 --2- 172.21.15.104:0/1869611583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71c0003a20 0x7f71c0003e30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f71d00017e0 tx=0x7f71c00058a0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71db471700 1 -- 172.21.15.104:0/1869611583 shutdown_connections 2023-07-14T11:54:45.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71db471700 1 --2- 172.21.15.104:0/1869611583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71c0003a20 0x7f71c0003e30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71db471700 1 -- 172.21.15.104:0/1869611583 >> 172.21.15.104:0/1869611583 conn(0x7f71d40cad30 msgr2=0x7f71d40cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:45.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71db471700 1 -- 172.21.15.104:0/1869611583 shutdown_connections 2023-07-14T11:54:45.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.787+0000 7f71db471700 1 -- 172.21.15.104:0/1869611583 wait complete. 2023-07-14T11:54:45.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71db471700 1 Processor -- start 2023-07-14T11:54:45.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71db471700 1 -- start start 2023-07-14T11:54:45.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71db471700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c0003a20 0x7f71d4156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71db471700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71d40c4360 0x7f71d40c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71db471700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71d415c890 0x7f71d415cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71db471700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40c4d10 con 0x7f71d415c890 2023-07-14T11:54:45.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71db471700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d41f2820 con 0x7f71c0003a20 2023-07-14T11:54:45.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71db471700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d41f1c10 con 0x7f71d40c4360 2023-07-14T11:54:45.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71d9a0e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71d415c890 0x7f71d415cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71d9a0e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71d415c890 0x7f71d415cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59422/0 (socket says 172.21.15.104:59422) 2023-07-14T11:54:45.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71d9a0e700 1 -- 172.21.15.104:0/564882379 learned_addr learned my addr 172.21.15.104:0/564882379 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:45.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71d8a0c700 1 --2- 172.21.15.104:0/564882379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71d40c4360 0x7f71d40c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71d920d700 1 --2- 172.21.15.104:0/564882379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c0003a20 0x7f71d4156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71d9a0e700 1 -- 172.21.15.104:0/564882379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71d40c4360 msgr2=0x7f71d40c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:45.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71d9a0e700 1 --2- 172.21.15.104:0/564882379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71d40c4360 0x7f71d40c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71d9a0e700 1 -- 172.21.15.104:0/564882379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c0003a20 msgr2=0x7f71d4156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:45.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71d9a0e700 1 --2- 172.21.15.104:0/564882379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c0003a20 0x7f71d4156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:45.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.791+0000 7f71d9a0e700 1 -- 172.21.15.104:0/564882379 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d41f1ef0 con 0x7f71d415c890 2023-07-14T11:54:45.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.795+0000 7f71d9a0e700 1 --2- 172.21.15.104:0/564882379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71d415c890 0x7f71d415cd00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f71d40cf2c0 tx=0x7f71cc001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:45.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.795+0000 7f71c9ffb700 1 -- 172.21.15.104:0/564882379 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc00aa80 con 0x7f71d415c890 2023-07-14T11:54:45.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.795+0000 7f71c9ffb700 1 -- 172.21.15.104:0/564882379 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc00abe0 con 0x7f71d415c890 2023-07-14T11:54:45.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.795+0000 7f71db471700 1 -- 172.21.15.104:0/564882379 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d41f21d0 con 0x7f71d415c890 2023-07-14T11:54:45.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.795+0000 7f71c9ffb700 1 -- 172.21.15.104:0/564882379 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc0133f0 con 0x7f71d415c890 2023-07-14T11:54:45.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.795+0000 7f71db471700 1 -- 172.21.15.104:0/564882379 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d415d240 con 0x7f71d415c890 2023-07-14T11:54:45.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.795+0000 7f71db471700 1 -- 172.21.15.104:0/564882379 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d404dcb0 con 0x7f71d415c890 2023-07-14T11:54:45.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.795+0000 7f71c9ffb700 1 -- 172.21.15.104:0/564882379 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f71cc013550 con 0x7f71d415c890 2023-07-14T11:54:45.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.799+0000 7f71c9ffb700 1 --2- 172.21.15.104:0/564882379 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f71b80662f0 0x7f71b80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:45.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.799+0000 7f71c9ffb700 1 -- 172.21.15.104:0/564882379 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f71cc0846a0 con 0x7f71d415c890 2023-07-14T11:54:45.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.799+0000 7f71d920d700 1 --2- 172.21.15.104:0/564882379 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f71b80662f0 0x7f71b80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:45.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.799+0000 7f71d920d700 1 --2- 172.21.15.104:0/564882379 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f71b80662f0 0x7f71b80687a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f71d0005040 tx=0x7f71d0009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:45.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:45.803+0000 7f71c9ffb700 1 -- 172.21.15.104:0/564882379 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f71cc051750 con 0x7f71d415c890 2023-07-14T11:54:45.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.011+0000 7f71db471700 1 -- 172.21.15.104:0/564882379 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f71d40c85e0 con 0x7f71d415c890 2023-07-14T11:54:45.997 INFO:tasks.ceph.mon.a.smithi104.stderr:2023-07-14T11:54:46.011+0000 7fbde97f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T11:54:46.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.027+0000 7f71c9ffb700 1 -- 172.21.15.104:0/564882379 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f71cc054d70 con 0x7f71d415c890 2023-07-14T11:54:46.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.027+0000 7f71db471700 1 -- 172.21.15.104:0/564882379 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f71b80662f0 msgr2=0x7f71b80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:46.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.027+0000 7f71db471700 1 --2- 172.21.15.104:0/564882379 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f71b80662f0 0x7f71b80687a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f71d0005040 tx=0x7f71d0009030 comp rx=0 tx=0).stop 2023-07-14T11:54:46.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.027+0000 7f71db471700 1 -- 172.21.15.104:0/564882379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71d415c890 msgr2=0x7f71d415cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:46.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.027+0000 7f71db471700 1 --2- 172.21.15.104:0/564882379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71d415c890 0x7f71d415cd00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f71d40cf2c0 tx=0x7f71cc001ad0 comp rx=0 tx=0).stop 2023-07-14T11:54:46.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.027+0000 7f71db471700 1 -- 172.21.15.104:0/564882379 shutdown_connections 2023-07-14T11:54:46.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.027+0000 7f71db471700 1 --2- 172.21.15.104:0/564882379 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f71b80662f0 0x7f71b80687a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:46.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.027+0000 7f71db471700 1 --2- 172.21.15.104:0/564882379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71d415c890 0x7f71d415cd00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:46.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.027+0000 7f71db471700 1 --2- 172.21.15.104:0/564882379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71d40c4360 0x7f71d40c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:46.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.027+0000 7f71db471700 1 --2- 172.21.15.104:0/564882379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c0003a20 0x7f71d4156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:46.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.027+0000 7f71db471700 1 -- 172.21.15.104:0/564882379 >> 172.21.15.104:0/564882379 conn(0x7f71d40cad30 msgr2=0x7f71d40ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:46.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.027+0000 7f71db471700 1 -- 172.21.15.104:0/564882379 shutdown_connections 2023-07-14T11:54:46.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.031+0000 7f71db471700 1 -- 172.21.15.104:0/564882379 wait complete. 2023-07-14T11:54:46.017 INFO:teuthology.orchestra.run.smithi104.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-14T11:54:46.022 INFO:tasks.ceph.mds.c.smithi104.stderr: -4893> 2023-07-14T11:53:21.309+0000 7f5b769da700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-14T11:54:46.033 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-14T11:54:46.078 INFO:tasks.ceph.mds.c.smithi104.stdout:starting mds.c at 2023-07-14T11:54:46.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.223+0000 7fd51fec6700 1 Processor -- start 2023-07-14T11:54:46.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.227+0000 7fd51fec6700 1 -- start start 2023-07-14T11:54:46.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.227+0000 7fd51fec6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd51815bff0 0x7fd51806bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:46.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.227+0000 7fd51fec6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5180cd570 0x7fd5180cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:46.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.227+0000 7fd51fec6700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd51815c3c0 con 0x7fd51806c550 2023-07-14T11:54:46.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.227+0000 7fd51fec6700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51815fa60 con 0x7fd51815bff0 2023-07-14T11:54:46.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.227+0000 7fd51fec6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51806c2f0 con 0x7fd5180cd570 2023-07-14T11:54:46.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.227+0000 7fd51d461700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fd51806c550 0x7fd5180cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:36904/0 (socket says 172.21.15.104:36904) 2023-07-14T11:54:46.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.227+0000 7fd51d461700 1 -- 172.21.15.104:0/3289459840 learned_addr learned my addr 172.21.15.104:0/3289459840 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:46.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.227+0000 7fd51e463700 1 --2- 172.21.15.104:0/3289459840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5180cd570 0x7fd5180cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:46.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.227+0000 7fd51dc62700 1 --2- 172.21.15.104:0/3289459840 >> v2:172.21.15.121:3300/0 conn(0x7fd51815bff0 0x7fd51806bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:46.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51cc60700 1 -- 172.21.15.104:0/3289459840 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79966835 0 0) 0x7fd508001110 con 0x7fd51806c550 2023-07-14T11:54:46.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51cc60700 1 -- 172.21.15.104:0/3289459840 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5040035d0 con 0x7fd51806c550 2023-07-14T11:54:46.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51e463700 1 -- 172.21.15.104:0/3289459840 >> v2:172.21.15.121:3300/0 conn(0x7fd51815bff0 msgr2=0x7fd51806bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:46.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51e463700 1 --2- 172.21.15.104:0/3289459840 >> v2:172.21.15.121:3300/0 conn(0x7fd51815bff0 0x7fd51806bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:46.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51cc60700 1 -- 172.21.15.104:0/3289459840 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1780364397 0 0) 0x7fd5040035d0 con 0x7fd51806c550 2023-07-14T11:54:46.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51e463700 1 -- 172.21.15.104:0/3289459840 >> v1:172.21.15.104:6789/0 conn(0x7fd51806c550 legacy=0x7fd5180cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:46.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51e463700 1 -- 172.21.15.104:0/3289459840 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd518160850 con 0x7fd5180cd570 2023-07-14T11:54:46.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51e463700 1 --2- 172.21.15.104:0/3289459840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5180cd570 0x7fd5180cac10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd510009d90 tx=0x7fd51000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bef4a192e167be8e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:46.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51cc60700 1 -- 172.21.15.104:0/3289459840 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51000e070 con 0x7fd5180cd570 2023-07-14T11:54:46.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51cc60700 1 -- 172.21.15.104:0/3289459840 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd510003a40 con 0x7fd5180cd570 2023-07-14T11:54:46.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51cc60700 1 -- 172.21.15.104:0/3289459840 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508001800 con 0x7fd5180cd570 2023-07-14T11:54:46.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51fec6700 1 -- 172.21.15.104:0/3289459840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5180cd570 msgr2=0x7fd5180cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:46.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51fec6700 1 --2- 172.21.15.104:0/3289459840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5180cd570 0x7fd5180cac10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd510009d90 tx=0x7fd51000b330 comp rx=0 tx=0).stop 2023-07-14T11:54:46.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51fec6700 1 -- 172.21.15.104:0/3289459840 shutdown_connections 2023-07-14T11:54:46.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51fec6700 1 --2- 172.21.15.104:0/3289459840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5180cd570 0x7fd5180cac10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:46.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51fec6700 1 --2- 172.21.15.104:0/3289459840 >> v2:172.21.15.121:3300/0 conn(0x7fd51815bff0 0x7fd51806bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:46.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51fec6700 1 -- 172.21.15.104:0/3289459840 >> 172.21.15.104:0/3289459840 conn(0x7fd518060f70 msgr2=0x7fd51806a190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:46.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51fec6700 1 -- 172.21.15.104:0/3289459840 shutdown_connections 2023-07-14T11:54:46.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.231+0000 7fd51fec6700 1 -- 172.21.15.104:0/3289459840 wait complete. 2023-07-14T11:54:46.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.235+0000 7fd51fec6700 1 Processor -- start 2023-07-14T11:54:46.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.235+0000 7fd51fec6700 1 -- start start 2023-07-14T11:54:46.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.235+0000 7fd51fec6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd51806c550 0x7fd518157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:46.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.235+0000 7fd51fec6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5180cd570 0x7fd5180c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:46.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.235+0000 7fd51fec6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd51815bff0 0x7fd5181555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:46.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.235+0000 7fd51fec6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd518158ec0 con 0x7fd5180cd570 2023-07-14T11:54:46.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.235+0000 7fd51fec6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd518158340 con 0x7fd51815bff0 2023-07-14T11:54:46.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.235+0000 7fd51fec6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd518158620 con 0x7fd51806c550 2023-07-14T11:54:46.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.235+0000 7fd51d461700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5180cd570 0x7fd5180c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:46.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.235+0000 7fd51d461700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5180cd570 0x7fd5180c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59436/0 (socket says 172.21.15.104:59436) 2023-07-14T11:54:46.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.235+0000 7fd51d461700 1 -- 172.21.15.104:0/4116173426 learned_addr learned my addr 172.21.15.104:0/4116173426 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:46.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.235+0000 7fd51dc62700 1 --2- 172.21.15.104:0/4116173426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd51806c550 0x7fd518157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:46.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.239+0000 7fd51e463700 1 --2- 172.21.15.104:0/4116173426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd51815bff0 0x7fd5181555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:46.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.239+0000 7fd51d461700 1 -- 172.21.15.104:0/4116173426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd51806c550 msgr2=0x7fd518157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:46.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.239+0000 7fd51d461700 1 --2- 172.21.15.104:0/4116173426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd51806c550 0x7fd518157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:46.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.239+0000 7fd51d461700 1 -- 172.21.15.104:0/4116173426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd51815bff0 msgr2=0x7fd5181555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:46.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.239+0000 7fd51d461700 1 --2- 172.21.15.104:0/4116173426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd51815bff0 0x7fd5181555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:46.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.239+0000 7fd51d461700 1 -- 172.21.15.104:0/4116173426 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd510009a70 con 0x7fd5180cd570 2023-07-14T11:54:46.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.239+0000 7fd51dc62700 1 --2- 172.21.15.104:0/4116173426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd51806c550 0x7fd518157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:46.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.239+0000 7fd51d461700 1 --2- 172.21.15.104:0/4116173426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5180cd570 0x7fd5180c9d70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd50800c6d0 tx=0x7fd50800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:46.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.239+0000 7fd50e7fc700 1 -- 172.21.15.104:0/4116173426 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508004ca0 con 0x7fd5180cd570 2023-07-14T11:54:46.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.239+0000 7fd50e7fc700 1 -- 172.21.15.104:0/4116173426 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd508004e00 con 0x7fd5180cd570 2023-07-14T11:54:46.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.239+0000 7fd51fec6700 1 -- 172.21.15.104:0/4116173426 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd518158900 con 0x7fd5180cd570 2023-07-14T11:54:46.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.239+0000 7fd50e7fc700 1 -- 172.21.15.104:0/4116173426 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5080072f0 con 0x7fd5180cd570 2023-07-14T11:54:46.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.239+0000 7fd51fec6700 1 -- 172.21.15.104:0/4116173426 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd518158050 con 0x7fd5180cd570 2023-07-14T11:54:46.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.239+0000 7fd51fec6700 1 -- 172.21.15.104:0/4116173426 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd518071c00 con 0x7fd5180cd570 2023-07-14T11:54:46.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.243+0000 7fd50e7fc700 1 -- 172.21.15.104:0/4116173426 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd508007450 con 0x7fd5180cd570 2023-07-14T11:54:46.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.243+0000 7fd50e7fc700 1 --2- 172.21.15.104:0/4116173426 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd4fc0662f0 0x7fd4fc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:46.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.243+0000 7fd50e7fc700 1 -- 172.21.15.104:0/4116173426 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fd508083430 con 0x7fd5180cd570 2023-07-14T11:54:46.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.243+0000 7fd51dc62700 1 --2- 172.21.15.104:0/4116173426 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd4fc0662f0 0x7fd4fc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:46.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.243+0000 7fd51dc62700 1 --2- 172.21.15.104:0/4116173426 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd4fc0662f0 0x7fd4fc0687a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd514009ba0 tx=0x7fd514008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:46.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.247+0000 7fd50e7fc700 1 -- 172.21.15.104:0/4116173426 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fd508050520 con 0x7fd5180cd570 2023-07-14T11:54:46.447 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:46.463+0000 7fd51fec6700 1 -- 172.21.15.104:0/4116173426 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd518159130 con 0x7fd5180cd570 2023-07-14T11:54:47.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.027+0000 7fd50e7fc700 1 -- 172.21.15.104:0/4116173426 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd518159130 con 0x7fd5180cd570 2023-07-14T11:54:47.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.031+0000 7fd51fec6700 1 -- 172.21.15.104:0/4116173426 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd4fc0662f0 msgr2=0x7fd4fc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:47.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.031+0000 7fd51fec6700 1 --2- 172.21.15.104:0/4116173426 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd4fc0662f0 0x7fd4fc0687a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd514009ba0 tx=0x7fd514008040 comp rx=0 tx=0).stop 2023-07-14T11:54:47.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.031+0000 7fd51fec6700 1 -- 172.21.15.104:0/4116173426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5180cd570 msgr2=0x7fd5180c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:47.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.031+0000 7fd51fec6700 1 --2- 172.21.15.104:0/4116173426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5180cd570 0x7fd5180c9d70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd50800c6d0 tx=0x7fd50800ca90 comp rx=0 tx=0).stop 2023-07-14T11:54:47.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.031+0000 7fd51fec6700 1 -- 172.21.15.104:0/4116173426 shutdown_connections 2023-07-14T11:54:47.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.031+0000 7fd51fec6700 1 --2- 172.21.15.104:0/4116173426 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd4fc0662f0 0x7fd4fc0687a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:47.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.031+0000 7fd51fec6700 1 --2- 172.21.15.104:0/4116173426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5180cd570 0x7fd5180c9d70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:47.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.031+0000 7fd51fec6700 1 --2- 172.21.15.104:0/4116173426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd51815bff0 0x7fd5181555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:47.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.031+0000 7fd51fec6700 1 --2- 172.21.15.104:0/4116173426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd51806c550 0x7fd518157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:47.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.031+0000 7fd51fec6700 1 -- 172.21.15.104:0/4116173426 >> 172.21.15.104:0/4116173426 conn(0x7fd518060f70 msgr2=0x7fd5180cf770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:47.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.031+0000 7fd51fec6700 1 -- 172.21.15.104:0/4116173426 shutdown_connections 2023-07-14T11:54:47.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.031+0000 7fd51fec6700 1 -- 172.21.15.104:0/4116173426 wait complete. 2023-07-14T11:54:47.037 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-14T11:54:47.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.235+0000 7f402195f700 1 Processor -- start 2023-07-14T11:54:47.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.235+0000 7f402195f700 1 -- start start 2023-07-14T11:54:47.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.239+0000 7f402195f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f401c0b0ee0 0x7f401c0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:47.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.239+0000 7f402195f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f401c0d07e0 0x7f401c0af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:47.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.239+0000 7f402195f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401c0cd4e0 con 0x7f401c0cd090 2023-07-14T11:54:47.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.239+0000 7f402195f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401c0d0c60 con 0x7f401c0b0ee0 2023-07-14T11:54:47.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.239+0000 7f402195f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0d1a60 con 0x7f401c0d07e0 2023-07-14T11:54:47.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.239+0000 7f402095d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f401c0b0ee0 0x7f401c0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:47.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.239+0000 7f402095d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f401c0b0ee0 0x7f401c0ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59442/0 (socket says 172.21.15.104:59442) 2023-07-14T11:54:47.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.239+0000 7f402095d700 1 -- 172.21.15.104:0/3756120542 learned_addr learned my addr 172.21.15.104:0/3756120542 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:47.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.239+0000 7f402115e700 1 --2- 172.21.15.104:0/3756120542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f401c0d07e0 0x7f401c0af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:47.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f402115e700 1 -- 172.21.15.104:0/3756120542 >> v2:172.21.15.104:3300/0 conn(0x7f401c0b0ee0 msgr2=0x7f401c0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:47.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f402115e700 1 --2- 172.21.15.104:0/3756120542 >> v2:172.21.15.104:3300/0 conn(0x7f401c0b0ee0 0x7f401c0ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:47.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f402115e700 1 -- 172.21.15.104:0/3756120542 >> v1:172.21.15.121:6789/0 conn(0x7f401c0cd090 legacy=0x7f401c0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:47.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f402115e700 1 -- 172.21.15.104:0/3756120542 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c0d1ef0 con 0x7f401c0d07e0 2023-07-14T11:54:47.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f402095d700 1 --2- 172.21.15.104:0/3756120542 >> v2:172.21.15.104:3300/0 conn(0x7f401c0b0ee0 0x7f401c0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:47.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f402115e700 1 --2- 172.21.15.104:0/3756120542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f401c0d07e0 0x7f401c0af0c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4014009d90 tx=0x7f401400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a5a22785db5a49 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:47.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f401b7fe700 1 -- 172.21.15.104:0/3756120542 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401400e070 con 0x7f401c0d07e0 2023-07-14T11:54:47.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f401b7fe700 1 -- 172.21.15.104:0/3756120542 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401400bb20 con 0x7f401c0d07e0 2023-07-14T11:54:47.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f401b7fe700 1 -- 172.21.15.104:0/3756120542 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014012430 con 0x7f401c0d07e0 2023-07-14T11:54:47.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f402195f700 1 -- 172.21.15.104:0/3756120542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f401c0d07e0 msgr2=0x7f401c0af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:47.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f402195f700 1 --2- 172.21.15.104:0/3756120542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f401c0d07e0 0x7f401c0af0c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4014009d90 tx=0x7f401400b330 comp rx=0 tx=0).stop 2023-07-14T11:54:47.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f402195f700 1 -- 172.21.15.104:0/3756120542 shutdown_connections 2023-07-14T11:54:47.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f402195f700 1 --2- 172.21.15.104:0/3756120542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f401c0d07e0 0x7f401c0af0c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:47.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f402195f700 1 --2- 172.21.15.104:0/3756120542 >> v2:172.21.15.104:3300/0 conn(0x7f401c0b0ee0 0x7f401c0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:47.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f402195f700 1 -- 172.21.15.104:0/3756120542 >> 172.21.15.104:0/3756120542 conn(0x7f401c060c40 msgr2=0x7f401c09b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:47.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f402195f700 1 -- 172.21.15.104:0/3756120542 shutdown_connections 2023-07-14T11:54:47.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.243+0000 7f402195f700 1 -- 172.21.15.104:0/3756120542 wait complete. 2023-07-14T11:54:47.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.247+0000 7f402195f700 1 Processor -- start 2023-07-14T11:54:47.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.247+0000 7f402195f700 1 -- start start 2023-07-14T11:54:47.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.247+0000 7f402195f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f401c0b0ee0 0x7f401c157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:47.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.247+0000 7f402195f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f401c0cd090 0x7f401c0c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:47.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.247+0000 7f402195f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f401c0d07e0 0x7f401c0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:47.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.247+0000 7f402195f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0c6f60 con 0x7f401c0cd090 2023-07-14T11:54:47.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.247+0000 7f402195f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0b2030 con 0x7f401c0b0ee0 2023-07-14T11:54:47.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.247+0000 7f402195f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0cb1f0 con 0x7f401c0d07e0 2023-07-14T11:54:47.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.247+0000 7f401bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f401c0cd090 0x7f401c0c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:47.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.247+0000 7f401bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f401c0cd090 0x7f401c0c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59450/0 (socket says 172.21.15.104:59450) 2023-07-14T11:54:47.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.247+0000 7f401bfff700 1 -- 172.21.15.104:0/2784105734 learned_addr learned my addr 172.21.15.104:0/2784105734 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:47.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.247+0000 7f402115e700 1 --2- 172.21.15.104:0/2784105734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f401c0d07e0 0x7f401c0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:47.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.247+0000 7f402095d700 1 --2- 172.21.15.104:0/2784105734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f401c0b0ee0 0x7f401c157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:47.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.251+0000 7f401bfff700 1 -- 172.21.15.104:0/2784105734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f401c0d07e0 msgr2=0x7f401c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:47.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.251+0000 7f401bfff700 1 --2- 172.21.15.104:0/2784105734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f401c0d07e0 0x7f401c0b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:47.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.251+0000 7f401bfff700 1 -- 172.21.15.104:0/2784105734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f401c0b0ee0 msgr2=0x7f401c157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:47.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.251+0000 7f401bfff700 1 --2- 172.21.15.104:0/2784105734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f401c0b0ee0 0x7f401c157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:47.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.251+0000 7f401bfff700 1 -- 172.21.15.104:0/2784105734 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4010009570 con 0x7f401c0cd090 2023-07-14T11:54:47.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.251+0000 7f401bfff700 1 --2- 172.21.15.104:0/2784105734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f401c0cd090 0x7f401c0c6a20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f400800c530 tx=0x7f400800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:47.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.251+0000 7f40197fa700 1 -- 172.21.15.104:0/2784105734 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40080052d0 con 0x7f401c0cd090 2023-07-14T11:54:47.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.251+0000 7f40197fa700 1 -- 172.21.15.104:0/2784105734 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4008005430 con 0x7f401c0cd090 2023-07-14T11:54:47.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.251+0000 7f402195f700 1 -- 172.21.15.104:0/2784105734 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4014009a70 con 0x7f401c0cd090 2023-07-14T11:54:47.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.251+0000 7f40197fa700 1 -- 172.21.15.104:0/2784105734 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40080055a0 con 0x7f401c0cd090 2023-07-14T11:54:47.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.251+0000 7f402195f700 1 -- 172.21.15.104:0/2784105734 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401c0caba0 con 0x7f401c0cd090 2023-07-14T11:54:47.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.251+0000 7f402195f700 1 -- 172.21.15.104:0/2784105734 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401c04e500 con 0x7f401c0cd090 2023-07-14T11:54:47.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.251+0000 7f40197fa700 1 -- 172.21.15.104:0/2784105734 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f40080036a0 con 0x7f401c0cd090 2023-07-14T11:54:47.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.255+0000 7f40197fa700 1 --2- 172.21.15.104:0/2784105734 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f40000662f0 0x7f40000687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:47.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.255+0000 7f40197fa700 1 -- 172.21.15.104:0/2784105734 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f4008083140 con 0x7f401c0cd090 2023-07-14T11:54:47.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.255+0000 7f402095d700 1 --2- 172.21.15.104:0/2784105734 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f40000662f0 0x7f40000687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:47.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.255+0000 7f402095d700 1 --2- 172.21.15.104:0/2784105734 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f40000662f0 0x7f40000687a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4010006d50 tx=0x7f40100092e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:47.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.259+0000 7f40197fa700 1 -- 172.21.15.104:0/2784105734 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f40080501c0 con 0x7f401c0cd090 2023-07-14T11:54:47.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:47.427+0000 7f402195f700 1 -- 172.21.15.104:0/2784105734 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f401c0cd4e0 con 0x7f401c0cd090 2023-07-14T11:54:48.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.031+0000 7f40197fa700 1 -- 172.21.15.104:0/2784105734 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v32) v1 ==== 181+0+0 (secure 0 0 0) 0x7f401c0cd4e0 con 0x7f401c0cd090 2023-07-14T11:54:48.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.039+0000 7f402195f700 1 -- 172.21.15.104:0/2784105734 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f40000662f0 msgr2=0x7f40000687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:48.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.039+0000 7f402195f700 1 --2- 172.21.15.104:0/2784105734 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f40000662f0 0x7f40000687a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4010006d50 tx=0x7f40100092e0 comp rx=0 tx=0).stop 2023-07-14T11:54:48.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.039+0000 7f402195f700 1 -- 172.21.15.104:0/2784105734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f401c0cd090 msgr2=0x7f401c0c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:48.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.039+0000 7f402195f700 1 --2- 172.21.15.104:0/2784105734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f401c0cd090 0x7f401c0c6a20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f400800c530 tx=0x7f400800c840 comp rx=0 tx=0).stop 2023-07-14T11:54:48.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.039+0000 7f402195f700 1 -- 172.21.15.104:0/2784105734 shutdown_connections 2023-07-14T11:54:48.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.039+0000 7f402195f700 1 --2- 172.21.15.104:0/2784105734 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f40000662f0 0x7f40000687a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:48.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.039+0000 7f402195f700 1 --2- 172.21.15.104:0/2784105734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f401c0cd090 0x7f401c0c6a20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:48.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.039+0000 7f402195f700 1 --2- 172.21.15.104:0/2784105734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f401c0d07e0 0x7f401c0b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:48.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.039+0000 7f402195f700 1 --2- 172.21.15.104:0/2784105734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f401c0b0ee0 0x7f401c157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:48.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.039+0000 7f402195f700 1 -- 172.21.15.104:0/2784105734 >> 172.21.15.104:0/2784105734 conn(0x7f401c060c40 msgr2=0x7f401c072540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:48.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.039+0000 7f402195f700 1 -- 172.21.15.104:0/2784105734 shutdown_connections 2023-07-14T11:54:48.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.039+0000 7f402195f700 1 -- 172.21.15.104:0/2784105734 wait complete. 2023-07-14T11:54:48.028 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'cephfs_metadata' removed 2023-07-14T11:54:48.047 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-14T11:54:48.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.239+0000 7fb4957fe700 1 Processor -- start 2023-07-14T11:54:48.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.239+0000 7fb4957fe700 1 -- start start 2023-07-14T11:54:48.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.243+0000 7fb4957fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb4900cd260 0x7fb4900d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:48.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.243+0000 7fb4957fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb4900d0610 0x7fb4900af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:48.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.243+0000 7fb4957fe700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4900d1a60 con 0x7fb4900b0ee0 2023-07-14T11:54:48.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.243+0000 7fb4957fe700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4900d1ef0 con 0x7fb4900d0610 2023-07-14T11:54:48.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.243+0000 7fb4957fe700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4900d0e50 con 0x7fb4900cd260 2023-07-14T11:54:48.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.243+0000 7fb494ffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb4900d0610 0x7fb4900af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:48.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.243+0000 7fb494ffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb4900d0610 0x7fb4900af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59454/0 (socket says 172.21.15.104:59454) 2023-07-14T11:54:48.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.243+0000 7fb494ffd700 1 -- 172.21.15.104:0/816023597 learned_addr learned my addr 172.21.15.104:0/816023597 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:48.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.243+0000 7fb48f7fe700 1 --2- 172.21.15.104:0/816023597 >> v2:172.21.15.121:3300/0 conn(0x7fb4900cd260 0x7fb4900d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:48.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.243+0000 7fb494ffd700 1 -- 172.21.15.104:0/816023597 >> v2:172.21.15.121:3300/0 conn(0x7fb4900cd260 msgr2=0x7fb4900d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:48.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.243+0000 7fb494ffd700 1 --2- 172.21.15.104:0/816023597 >> v2:172.21.15.121:3300/0 conn(0x7fb4900cd260 0x7fb4900d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:48.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.243+0000 7fb494ffd700 1 -- 172.21.15.104:0/816023597 >> v1:172.21.15.121:6789/0 conn(0x7fb4900b0ee0 legacy=0x7fb4900ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:48.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.243+0000 7fb494ffd700 1 -- 172.21.15.104:0/816023597 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900d1130 con 0x7fb4900d0610 2023-07-14T11:54:48.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.247+0000 7fb494ffd700 1 --2- 172.21.15.104:0/816023597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4900d0610 0x7fb4900af0c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb4900b5090 tx=0x7fb48800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc679b4ffa70aa2a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:48.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.247+0000 7fb48effd700 1 -- 172.21.15.104:0/816023597 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48800e070 con 0x7fb4900d0610 2023-07-14T11:54:48.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.247+0000 7fb48effd700 1 -- 172.21.15.104:0/816023597 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48800b910 con 0x7fb4900d0610 2023-07-14T11:54:48.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.247+0000 7fb48effd700 1 -- 172.21.15.104:0/816023597 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48800bcc0 con 0x7fb4900d0610 2023-07-14T11:54:48.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.247+0000 7fb4957fe700 1 -- 172.21.15.104:0/816023597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4900d0610 msgr2=0x7fb4900af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:48.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.247+0000 7fb4957fe700 1 --2- 172.21.15.104:0/816023597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4900d0610 0x7fb4900af0c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb4900b5090 tx=0x7fb48800b330 comp rx=0 tx=0).stop 2023-07-14T11:54:48.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.247+0000 7fb4957fe700 1 -- 172.21.15.104:0/816023597 shutdown_connections 2023-07-14T11:54:48.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.247+0000 7fb4957fe700 1 --2- 172.21.15.104:0/816023597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4900d0610 0x7fb4900af0c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:48.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.247+0000 7fb4957fe700 1 --2- 172.21.15.104:0/816023597 >> v2:172.21.15.121:3300/0 conn(0x7fb4900cd260 0x7fb4900d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:48.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.247+0000 7fb4957fe700 1 -- 172.21.15.104:0/816023597 >> 172.21.15.104:0/816023597 conn(0x7fb490060c40 msgr2=0x7fb49009b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:48.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.247+0000 7fb4957fe700 1 -- 172.21.15.104:0/816023597 shutdown_connections 2023-07-14T11:54:48.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.247+0000 7fb4957fe700 1 -- 172.21.15.104:0/816023597 wait complete. 2023-07-14T11:54:48.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.247+0000 7fb4957fe700 1 Processor -- start 2023-07-14T11:54:48.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb4957fe700 1 -- start start 2023-07-14T11:54:48.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb4957fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4900b0ee0 0x7fb4900cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:48.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb4957fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4900cd260 0x7fb4900ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:48.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb4957fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4900d0610 0x7fb4900b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:48.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb4957fe700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900c95d0 con 0x7fb4900cd260 2023-07-14T11:54:48.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb4957fe700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900ca430 con 0x7fb4900b0ee0 2023-07-14T11:54:48.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb4957fe700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900cc420 con 0x7fb4900d0610 2023-07-14T11:54:48.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb48f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4900cd260 0x7fb4900ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:48.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb48f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4900cd260 0x7fb4900ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59460/0 (socket says 172.21.15.104:59460) 2023-07-14T11:54:48.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb48f7fe700 1 -- 172.21.15.104:0/802904901 learned_addr learned my addr 172.21.15.104:0/802904901 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:48.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb48ffff700 1 --2- 172.21.15.104:0/802904901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4900b0ee0 0x7fb4900cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:48.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb494ffd700 1 --2- 172.21.15.104:0/802904901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4900d0610 0x7fb4900b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:48.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb48f7fe700 1 -- 172.21.15.104:0/802904901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4900d0610 msgr2=0x7fb4900b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:48.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb48f7fe700 1 --2- 172.21.15.104:0/802904901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4900d0610 0x7fb4900b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:48.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb48f7fe700 1 -- 172.21.15.104:0/802904901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4900b0ee0 msgr2=0x7fb4900cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:48.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb48f7fe700 1 --2- 172.21.15.104:0/802904901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4900b0ee0 0x7fb4900cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:48.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb48f7fe700 1 -- 172.21.15.104:0/802904901 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb488009b30 con 0x7fb4900cd260 2023-07-14T11:54:48.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb48f7fe700 1 --2- 172.21.15.104:0/802904901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4900cd260 0x7fb4900ca8c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb48400e9b0 tx=0x7fb48400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:48.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb48cff9700 1 -- 172.21.15.104:0/802904901 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840096e0 con 0x7fb4900cd260 2023-07-14T11:54:48.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb48cff9700 1 -- 172.21.15.104:0/802904901 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48400c850 con 0x7fb4900cd260 2023-07-14T11:54:48.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb4957fe700 1 -- 172.21.15.104:0/802904901 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4900c9820 con 0x7fb4900cd260 2023-07-14T11:54:48.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb48cff9700 1 -- 172.21.15.104:0/802904901 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48400cbe0 con 0x7fb4900cd260 2023-07-14T11:54:48.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.251+0000 7fb4957fe700 1 -- 172.21.15.104:0/802904901 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4900c9de0 con 0x7fb4900cd260 2023-07-14T11:54:48.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.255+0000 7fb4957fe700 1 -- 172.21.15.104:0/802904901 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4900ca5c0 con 0x7fb4900cd260 2023-07-14T11:54:48.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.255+0000 7fb48cff9700 1 -- 172.21.15.104:0/802904901 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb48400cd40 con 0x7fb4900cd260 2023-07-14T11:54:48.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.259+0000 7fb48cff9700 1 --2- 172.21.15.104:0/802904901 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb47c0662f0 0x7fb47c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:48.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.259+0000 7fb48cff9700 1 -- 172.21.15.104:0/802904901 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6519+0+0 (secure 0 0 0) 0x7fb484083ff0 con 0x7fb4900cd260 2023-07-14T11:54:48.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.259+0000 7fb48ffff700 1 --2- 172.21.15.104:0/802904901 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb47c0662f0 0x7fb47c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:48.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.259+0000 7fb48ffff700 1 --2- 172.21.15.104:0/802904901 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb47c0662f0 0x7fb47c0687a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb49015bd20 tx=0x7fb480009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:48.249 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.263+0000 7fb48cff9700 1 -- 172.21.15.104:0/802904901 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb484051200 con 0x7fb4900cd260 2023-07-14T11:54:48.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:48.431+0000 7fb4957fe700 1 -- 172.21.15.104:0/802904901 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb49004e500 con 0x7fb4900cd260 2023-07-14T11:54:49.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.039+0000 7fb48cff9700 1 -- 172.21.15.104:0/802904901 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v33) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb4840170d0 con 0x7fb4900cd260 2023-07-14T11:54:49.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.043+0000 7fb4957fe700 1 -- 172.21.15.104:0/802904901 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb47c0662f0 msgr2=0x7fb47c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:49.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.043+0000 7fb4957fe700 1 --2- 172.21.15.104:0/802904901 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb47c0662f0 0x7fb47c0687a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb49015bd20 tx=0x7fb480009040 comp rx=0 tx=0).stop 2023-07-14T11:54:49.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.043+0000 7fb4957fe700 1 -- 172.21.15.104:0/802904901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4900cd260 msgr2=0x7fb4900ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:49.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.043+0000 7fb4957fe700 1 --2- 172.21.15.104:0/802904901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4900cd260 0x7fb4900ca8c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb48400e9b0 tx=0x7fb48400ecc0 comp rx=0 tx=0).stop 2023-07-14T11:54:49.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.043+0000 7fb4957fe700 1 -- 172.21.15.104:0/802904901 shutdown_connections 2023-07-14T11:54:49.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.043+0000 7fb4957fe700 1 --2- 172.21.15.104:0/802904901 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb47c0662f0 0x7fb47c0687a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:49.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.043+0000 7fb4957fe700 1 --2- 172.21.15.104:0/802904901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4900cd260 0x7fb4900ca8c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:49.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.043+0000 7fb4957fe700 1 --2- 172.21.15.104:0/802904901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb4900d0610 0x7fb4900b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:49.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.043+0000 7fb4957fe700 1 --2- 172.21.15.104:0/802904901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4900b0ee0 0x7fb4900cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:49.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.043+0000 7fb4957fe700 1 -- 172.21.15.104:0/802904901 >> 172.21.15.104:0/802904901 conn(0x7fb490060c40 msgr2=0x7fb4900af950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:49.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.047+0000 7fb4957fe700 1 -- 172.21.15.104:0/802904901 shutdown_connections 2023-07-14T11:54:49.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.047+0000 7fb4957fe700 1 -- 172.21.15.104:0/802904901 wait complete. 2023-07-14T11:54:49.038 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'cephfs_data' removed 2023-07-14T11:54:49.057 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-07-14T11:54:49.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.251+0000 7fe86ab84700 1 Processor -- start 2023-07-14T11:54:49.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.251+0000 7fe86ab84700 1 -- start start 2023-07-14T11:54:49.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe86ab84700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe8640b0ee0 0x7fe8640ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:49.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe86ab84700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8640d07e0 0x7fe8640af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:49.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe86ab84700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8640cd4e0 con 0x7fe8640cd090 2023-07-14T11:54:49.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe86ab84700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8640d0c60 con 0x7fe8640b0ee0 2023-07-14T11:54:49.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe86ab84700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8640d1a60 con 0x7fe8640d07e0 2023-07-14T11:54:49.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe869b82700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe8640b0ee0 0x7fe8640ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:49.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe869b82700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe8640b0ee0 0x7fe8640ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59472/0 (socket says 172.21.15.104:59472) 2023-07-14T11:54:49.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe869b82700 1 -- 172.21.15.104:0/3462610527 learned_addr learned my addr 172.21.15.104:0/3462610527 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:49.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe869b82700 1 -- 172.21.15.104:0/3462610527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8640d07e0 msgr2=0x7fe8640af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:49.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe86a383700 1 --2- 172.21.15.104:0/3462610527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8640d07e0 0x7fe8640af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:49.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe869b82700 1 --2- 172.21.15.104:0/3462610527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8640d07e0 0x7fe8640af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:49.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe869b82700 1 -- 172.21.15.104:0/3462610527 >> v1:172.21.15.104:6789/0 conn(0x7fe8640cd090 legacy=0x7fe8640d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:49.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe869b82700 1 -- 172.21.15.104:0/3462610527 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8640d1ef0 con 0x7fe8640b0ee0 2023-07-14T11:54:49.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe86a383700 1 --2- 172.21.15.104:0/3462610527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8640d07e0 0x7fe8640af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:49.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe869b82700 1 --2- 172.21.15.104:0/3462610527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8640b0ee0 0x7fe8640ccb50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe860009910 tx=0x7fe860009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd399284c8a18acc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:49.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe868b80700 1 -- 172.21.15.104:0/3462610527 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86000ba60 con 0x7fe8640b0ee0 2023-07-14T11:54:49.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe868b80700 1 -- 172.21.15.104:0/3462610527 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86000bbc0 con 0x7fe8640b0ee0 2023-07-14T11:54:49.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe868b80700 1 -- 172.21.15.104:0/3462610527 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860009e60 con 0x7fe8640b0ee0 2023-07-14T11:54:49.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.255+0000 7fe86ab84700 1 -- 172.21.15.104:0/3462610527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8640b0ee0 msgr2=0x7fe8640ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:49.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.259+0000 7fe86ab84700 1 --2- 172.21.15.104:0/3462610527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8640b0ee0 0x7fe8640ccb50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe860009910 tx=0x7fe860009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:49.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.259+0000 7fe86ab84700 1 -- 172.21.15.104:0/3462610527 shutdown_connections 2023-07-14T11:54:49.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.259+0000 7fe86ab84700 1 --2- 172.21.15.104:0/3462610527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8640d07e0 0x7fe8640af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:49.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.259+0000 7fe86ab84700 1 --2- 172.21.15.104:0/3462610527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8640b0ee0 0x7fe8640ccb50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:49.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.259+0000 7fe86ab84700 1 -- 172.21.15.104:0/3462610527 >> 172.21.15.104:0/3462610527 conn(0x7fe864060c40 msgr2=0x7fe86409b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:49.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.259+0000 7fe86ab84700 1 -- 172.21.15.104:0/3462610527 shutdown_connections 2023-07-14T11:54:49.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.259+0000 7fe86ab84700 1 -- 172.21.15.104:0/3462610527 wait complete. 2023-07-14T11:54:49.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.259+0000 7fe86ab84700 1 Processor -- start 2023-07-14T11:54:49.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.259+0000 7fe86ab84700 1 -- start start 2023-07-14T11:54:49.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe86ab84700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8640b0ee0 0x7fe8640b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:49.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe86ab84700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8640cd090 0x7fe8640c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:49.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe86ab84700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8640d07e0 0x7fe8640b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:49.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe86ab84700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8640c8ce0 con 0x7fe8640b0ee0 2023-07-14T11:54:49.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe86ab84700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8640c61b0 con 0x7fe8640cd090 2023-07-14T11:54:49.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe86ab84700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8640c6490 con 0x7fe8640d07e0 2023-07-14T11:54:49.249 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe869b82700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8640b0ee0 0x7fe8640b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:49.250 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe869b82700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8640b0ee0 0x7fe8640b4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59480/0 (socket says 172.21.15.104:59480) 2023-07-14T11:54:49.250 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe869b82700 1 -- 172.21.15.104:0/1289588728 learned_addr learned my addr 172.21.15.104:0/1289588728 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:49.250 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe86a383700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8640d07e0 0x7fe8640b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:49.250 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe869381700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8640cd090 0x7fe8640c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:49.250 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe869b82700 1 -- 172.21.15.104:0/1289588728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8640d07e0 msgr2=0x7fe8640b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:49.250 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe869b82700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8640d07e0 0x7fe8640b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:49.250 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe869b82700 1 -- 172.21.15.104:0/1289588728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8640cd090 msgr2=0x7fe8640c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:49.251 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe869b82700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8640cd090 0x7fe8640c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:49.251 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe869b82700 1 -- 172.21.15.104:0/1289588728 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe860009570 con 0x7fe8640b0ee0 2023-07-14T11:54:49.251 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe86a383700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8640d07e0 0x7fe8640b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:49.251 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe869381700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8640cd090 0x7fe8640c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:49.252 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe869b82700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8640b0ee0 0x7fe8640b4120 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe86408adc0 tx=0x7fe860004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:49.252 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.263+0000 7fe85a7fc700 1 -- 172.21.15.104:0/1289588728 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8600036c0 con 0x7fe8640b0ee0 2023-07-14T11:54:49.252 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.267+0000 7fe85a7fc700 1 -- 172.21.15.104:0/1289588728 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86001f070 con 0x7fe8640b0ee0 2023-07-14T11:54:49.252 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.267+0000 7fe85a7fc700 1 -- 172.21.15.104:0/1289588728 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860003a80 con 0x7fe8640b0ee0 2023-07-14T11:54:49.252 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.267+0000 7fe86ab84700 1 -- 172.21.15.104:0/1289588728 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8640c6770 con 0x7fe8640b0ee0 2023-07-14T11:54:49.253 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.267+0000 7fe86ab84700 1 -- 172.21.15.104:0/1289588728 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8640b24f0 con 0x7fe8640b0ee0 2023-07-14T11:54:49.253 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.267+0000 7fe86ab84700 1 -- 172.21.15.104:0/1289588728 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8640b2770 con 0x7fe8640b0ee0 2023-07-14T11:54:49.253 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.267+0000 7fe85a7fc700 1 -- 172.21.15.104:0/1289588728 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe860003be0 con 0x7fe8640b0ee0 2023-07-14T11:54:49.255 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.271+0000 7fe85a7fc700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8480662f0 0x7fe8480687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:49.255 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.271+0000 7fe85a7fc700 1 -- 172.21.15.104:0/1289588728 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6047+0+0 (secure 0 0 0) 0x7fe860021070 con 0x7fe8640b0ee0 2023-07-14T11:54:49.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.271+0000 7fe869381700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8480662f0 0x7fe8480687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:49.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.271+0000 7fe869381700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8480662f0 0x7fe8480687a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe8640c7d20 tx=0x7fe854009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:49.259 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.275+0000 7fe85a7fc700 1 -- 172.21.15.104:0/1289588728 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe860052ba0 con 0x7fe8640b0ee0 2023-07-14T11:54:49.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:49.447+0000 7fe86ab84700 1 -- 172.21.15.104:0/1289588728 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe86404e500 con 0x7fe8640b0ee0 2023-07-14T11:54:50.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.059+0000 7fe85a7fc700 1 -- 172.21.15.104:0/1289588728 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v34) v1 ==== 178+0+0 (secure 0 0 0) 0x7fe8600561c0 con 0x7fe8640b0ee0 2023-07-14T11:54:50.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.063+0000 7fe86ab84700 1 -- 172.21.15.104:0/1289588728 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8480662f0 msgr2=0x7fe8480687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:50.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.063+0000 7fe86ab84700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8480662f0 0x7fe8480687a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe8640c7d20 tx=0x7fe854009040 comp rx=0 tx=0).stop 2023-07-14T11:54:50.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.063+0000 7fe86ab84700 1 -- 172.21.15.104:0/1289588728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8640b0ee0 msgr2=0x7fe8640b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:50.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.063+0000 7fe86ab84700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8640b0ee0 0x7fe8640b4120 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe86408adc0 tx=0x7fe860004770 comp rx=0 tx=0).stop 2023-07-14T11:54:50.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.067+0000 7fe86ab84700 1 -- 172.21.15.104:0/1289588728 shutdown_connections 2023-07-14T11:54:50.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.067+0000 7fe86ab84700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8480662f0 0x7fe8480687a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:50.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.067+0000 7fe86ab84700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8640d07e0 0x7fe8640b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:50.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.067+0000 7fe86ab84700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8640cd090 0x7fe8640c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:50.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.067+0000 7fe86ab84700 1 --2- 172.21.15.104:0/1289588728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8640b0ee0 0x7fe8640b4120 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:50.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.067+0000 7fe86ab84700 1 -- 172.21.15.104:0/1289588728 >> 172.21.15.104:0/1289588728 conn(0x7fe864060c40 msgr2=0x7fe86406fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:50.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.067+0000 7fe86ab84700 1 -- 172.21.15.104:0/1289588728 shutdown_connections 2023-07-14T11:54:50.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.067+0000 7fe86ab84700 1 -- 172.21.15.104:0/1289588728 wait complete. 2023-07-14T11:54:50.053 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'cephfs_data_ec' removed 2023-07-14T11:54:50.080 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-14T11:54:50.080 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-14T11:54:50.081 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-14T11:54:50.081 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:54:50.081 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-14T11:54:50.082 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-14T11:54:50.082 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-14T11:54:50.083 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:54:50.083 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-14T11:54:50.083 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-14T11:54:50.084 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-14T11:54:50.084 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:54:50.084 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-14T11:54:50.085 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-14T11:54:50.085 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-14T11:54:50.086 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:54:50.099 INFO:tasks.ceph.mds.c:Stopped 2023-07-14T11:54:50.100 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-14T11:54:50.101 INFO:tasks.ceph.mds.a:Stopped 2023-07-14T11:54:50.101 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-14T11:54:50.102 INFO:tasks.ceph.mds.d:Stopped 2023-07-14T11:54:50.103 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-14T11:54:50.103 INFO:tasks.ceph.mds.b:Stopped 2023-07-14T11:54:50.103 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-14T11:54:50.148 INFO:tasks.ceph.mds.c:Started 2023-07-14T11:54:50.149 DEBUG:teuthology.parallel:result is None 2023-07-14T11:54:50.150 INFO:tasks.ceph.mds.d:Started 2023-07-14T11:54:50.150 DEBUG:teuthology.parallel:result is None 2023-07-14T11:54:50.152 INFO:tasks.ceph.mds.a:Started 2023-07-14T11:54:50.152 DEBUG:teuthology.parallel:result is None 2023-07-14T11:54:50.155 INFO:tasks.ceph.mds.b:Started 2023-07-14T11:54:50.155 DEBUG:teuthology.parallel:result is None 2023-07-14T11:54:50.156 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-14T11:54:50.252 INFO:tasks.ceph.mds.c.smithi104.stdout:starting mds.c at 2023-07-14T11:54:50.259 INFO:tasks.ceph.mds.a.smithi104.stdout:starting mds.a at 2023-07-14T11:54:50.259 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2023-07-14T11:54:50.272 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2023-07-14T11:54:50.319 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.335+0000 7ff32e311700 1 Processor -- start 2023-07-14T11:54:50.320 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.335+0000 7ff32e311700 1 -- start start 2023-07-14T11:54:50.321 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.335+0000 7ff32e311700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3280cd240 0x7ff3280ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:50.321 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.335+0000 7ff32e311700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3280af0a0 con 0x7ff3280ceff0 2023-07-14T11:54:50.321 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.335+0000 7ff32e311700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3280cf470 con 0x7ff32815bff0 2023-07-14T11:54:50.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.335+0000 7ff32e311700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3280cf720 con 0x7ff3280cd240 2023-07-14T11:54:50.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.335+0000 7ff32db10700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ff3280ceff0 0x7ff3280afdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56632/0 (socket says 172.21.15.104:56632) 2023-07-14T11:54:50.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.335+0000 7ff32db10700 1 -- 172.21.15.104:0/3225797994 learned_addr learned my addr 172.21.15.104:0/3225797994 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:50.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.335+0000 7ff32cb0e700 1 --2- 172.21.15.104:0/3225797994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3280cd240 0x7ff3280ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:50.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.335+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758472826 0 0) 0x7ff3280af0a0 con 0x7ff3280ceff0 2023-07-14T11:54:50.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.335+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3140035d0 con 0x7ff3280ceff0 2023-07-14T11:54:50.324 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762537441 0 0) 0x7ff3280cf470 con 0x7ff32815bff0 2023-07-14T11:54:50.324 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3280af0a0 con 0x7ff32815bff0 2023-07-14T11:54:50.325 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1500494837 0 0) 0x7ff3140035d0 con 0x7ff3280ceff0 2023-07-14T11:54:50.325 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3280cd240 msgr2=0x7ff3280ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:50.325 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 --2- 172.21.15.104:0/3225797994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3280cd240 0x7ff3280ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:50.325 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 >> v1:172.21.15.121:6789/0 conn(0x7ff32815bff0 legacy=0x7ff3280ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:50.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3280af2f0 con 0x7ff3280ceff0 2023-07-14T11:54:50.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7ff3200019f0 con 0x7ff3280ceff0 2023-07-14T11:54:50.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 >> v1:172.21.15.104:6789/0 conn(0x7ff3280ceff0 legacy=0x7ff3280afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:50.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 --2- 172.21.15.104:0/3225797994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3280cd240 0x7ff314003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:50.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 --2- 172.21.15.104:0/3225797994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff314004190 0x7ff314004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:50.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 --2- 172.21.15.104:0/3225797994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff314004db0 0x7ff314005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:50.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3280cf720 con 0x7ff314004190 2023-07-14T11:54:50.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31400bb40 con 0x7ff314004db0 2023-07-14T11:54:50.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31400bd00 con 0x7ff3280cd240 2023-07-14T11:54:50.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff32db10700 1 --2- 172.21.15.104:0/3225797994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff314004db0 0x7ff314005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:50.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff32d30f700 1 --2- 172.21.15.104:0/3225797994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3280cd240 0x7ff314003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:50.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff32cb0e700 1 --2- 172.21.15.104:0/3225797994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff314004190 0x7ff314004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:50.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff32cb0e700 1 -- 172.21.15.104:0/3225797994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3280cd240 msgr2=0x7ff314003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:50.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff32cb0e700 1 --2- 172.21.15.104:0/3225797994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3280cd240 0x7ff314003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:50.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff32cb0e700 1 -- 172.21.15.104:0/3225797994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff314004db0 msgr2=0x7ff314005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:50.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff32cb0e700 1 --2- 172.21.15.104:0/3225797994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff314004db0 0x7ff314005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:50.329 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff32cb0e700 1 -- 172.21.15.104:0/3225797994 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3140035d0 con 0x7ff314004190 2023-07-14T11:54:50.330 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff32cb0e700 1 --2- 172.21.15.104:0/3225797994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff314004190 0x7ff314004870 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff318009b40 tx=0x7ff3140058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b02c500f99e3bb6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:50.330 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31800e6c0 con 0x7ff314004190 2023-07-14T11:54:50.330 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31800e820 con 0x7ff314004190 2023-07-14T11:54:50.331 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff31ffff700 1 -- 172.21.15.104:0/3225797994 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31800eb90 con 0x7ff314004190 2023-07-14T11:54:50.331 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff32e311700 1 -- 172.21.15.104:0/3225797994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff314004190 msgr2=0x7ff314004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:50.331 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff32e311700 1 --2- 172.21.15.104:0/3225797994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff314004190 0x7ff314004870 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff318009b40 tx=0x7ff3140058a0 comp rx=0 tx=0).stop 2023-07-14T11:54:50.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff32db10700 1 -- 172.21.15.104:0/3225797994 reap_dead start 2023-07-14T11:54:50.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff32e311700 1 -- 172.21.15.104:0/3225797994 shutdown_connections 2023-07-14T11:54:50.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.339+0000 7ff32e311700 1 -- 172.21.15.104:0/3225797994 >> 172.21.15.104:0/3225797994 conn(0x7ff328060c40 msgr2=0x7ff32809b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:50.333 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.343+0000 7ff32e311700 1 -- 172.21.15.104:0/3225797994 shutdown_connections 2023-07-14T11:54:50.333 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.343+0000 7ff32e311700 1 -- 172.21.15.104:0/3225797994 wait complete. 2023-07-14T11:54:50.333 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.343+0000 7ff32e311700 1 Processor -- start 2023-07-14T11:54:50.333 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.343+0000 7ff32e311700 1 -- start start 2023-07-14T11:54:50.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32e311700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff314004190 0x7ff328156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:50.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32e311700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3280b7e60 0x7ff3280b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:50.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32e311700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3280b5d20 0x7ff3280b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:50.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32e311700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff328157300 con 0x7ff314004190 2023-07-14T11:54:50.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32e311700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3280b66d0 con 0x7ff3280b7e60 2023-07-14T11:54:50.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32e311700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3280b3ee0 con 0x7ff3280b5d20 2023-07-14T11:54:50.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32d30f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff314004190 0x7ff328156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:50.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32d30f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff314004190 0x7ff328156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57872/0 (socket says 172.21.15.104:57872) 2023-07-14T11:54:50.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32d30f700 1 -- 172.21.15.104:0/3697083502 learned_addr learned my addr 172.21.15.104:0/3697083502 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:50.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32db10700 1 --2- 172.21.15.104:0/3697083502 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3280b5d20 0x7ff3280b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:50.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32cb0e700 1 --2- 172.21.15.104:0/3697083502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3280b7e60 0x7ff3280b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:50.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32d30f700 1 -- 172.21.15.104:0/3697083502 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3280b5d20 msgr2=0x7ff3280b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:50.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32d30f700 1 --2- 172.21.15.104:0/3697083502 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3280b5d20 0x7ff3280b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:50.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32d30f700 1 -- 172.21.15.104:0/3697083502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3280b7e60 msgr2=0x7ff3280b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:50.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32d30f700 1 --2- 172.21.15.104:0/3697083502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3280b7e60 0x7ff3280b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:50.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32d30f700 1 -- 172.21.15.104:0/3697083502 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3280b32d0 con 0x7ff314004190 2023-07-14T11:54:50.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32db10700 1 --2- 172.21.15.104:0/3697083502 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3280b5d20 0x7ff3280b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:50.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32d30f700 1 --2- 172.21.15.104:0/3697083502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff314004190 0x7ff328156dc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff32400f940 tx=0x7ff32400fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:50.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff31dffb700 1 -- 172.21.15.104:0/3697083502 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32400fee0 con 0x7ff314004190 2023-07-14T11:54:50.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff31dffb700 1 -- 172.21.15.104:0/3697083502 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3240052e0 con 0x7ff314004190 2023-07-14T11:54:50.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32e311700 1 -- 172.21.15.104:0/3697083502 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3280b35b0 con 0x7ff314004190 2023-07-14T11:54:50.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff31dffb700 1 -- 172.21.15.104:0/3697083502 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3240036a0 con 0x7ff314004190 2023-07-14T11:54:50.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.347+0000 7ff32e311700 1 -- 172.21.15.104:0/3697083502 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3280b3b70 con 0x7ff314004190 2023-07-14T11:54:50.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.351+0000 7ff32e311700 1 -- 172.21.15.104:0/3697083502 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3280b0c70 con 0x7ff314004190 2023-07-14T11:54:50.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.351+0000 7ff31dffb700 1 -- 172.21.15.104:0/3697083502 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff324003800 con 0x7ff314004190 2023-07-14T11:54:50.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.351+0000 7ff31dffb700 1 --2- 172.21.15.104:0/3697083502 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff30c0662f0 0x7ff30c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:50.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.351+0000 7ff31dffb700 1 -- 172.21.15.104:0/3697083502 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5616+0+0 (secure 0 0 0) 0x7ff324083ae0 con 0x7ff314004190 2023-07-14T11:54:50.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.351+0000 7ff32cb0e700 1 --2- 172.21.15.104:0/3697083502 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff30c0662f0 0x7ff30c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:50.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.355+0000 7ff32cb0e700 1 --2- 172.21.15.104:0/3697083502 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff30c0662f0 0x7ff30c0687a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff3180090b0 tx=0x7ff318008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:50.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.355+0000 7ff31dffb700 1 -- 172.21.15.104:0/3697083502 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff324054c40 con 0x7ff314004190 2023-07-14T11:54:50.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:50.555+0000 7ff32e311700 1 -- 172.21.15.104:0/3697083502 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff32804e500 con 0x7ff314004190 2023-07-14T11:54:51.072 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.087+0000 7ff31dffb700 1 -- 172.21.15.104:0/3697083502 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v36) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff3240547d0 con 0x7ff314004190 2023-07-14T11:54:51.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.091+0000 7ff32e311700 1 -- 172.21.15.104:0/3697083502 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff30c0662f0 msgr2=0x7ff30c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:51.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.091+0000 7ff32e311700 1 --2- 172.21.15.104:0/3697083502 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff30c0662f0 0x7ff30c0687a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff3180090b0 tx=0x7ff318008030 comp rx=0 tx=0).stop 2023-07-14T11:54:51.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.091+0000 7ff32e311700 1 -- 172.21.15.104:0/3697083502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff314004190 msgr2=0x7ff328156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:51.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.091+0000 7ff32e311700 1 --2- 172.21.15.104:0/3697083502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff314004190 0x7ff328156dc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff32400f940 tx=0x7ff32400fd00 comp rx=0 tx=0).stop 2023-07-14T11:54:51.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.091+0000 7ff32e311700 1 -- 172.21.15.104:0/3697083502 shutdown_connections 2023-07-14T11:54:51.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.091+0000 7ff32e311700 1 --2- 172.21.15.104:0/3697083502 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff30c0662f0 0x7ff30c0687a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:51.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.091+0000 7ff32e311700 1 --2- 172.21.15.104:0/3697083502 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3280b5d20 0x7ff3280b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:51.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.091+0000 7ff32e311700 1 --2- 172.21.15.104:0/3697083502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3280b7e60 0x7ff3280b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:51.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.091+0000 7ff32e311700 1 --2- 172.21.15.104:0/3697083502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff314004190 0x7ff328156dc0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:51.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.091+0000 7ff32e311700 1 -- 172.21.15.104:0/3697083502 >> 172.21.15.104:0/3697083502 conn(0x7ff328060c40 msgr2=0x7ff3280b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:51.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.091+0000 7ff32e311700 1 -- 172.21.15.104:0/3697083502 shutdown_connections 2023-07-14T11:54:51.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.091+0000 7ff32e311700 1 -- 172.21.15.104:0/3697083502 wait complete. 2023-07-14T11:54:51.079 INFO:teuthology.orchestra.run.smithi104.stderr: removed all blocklist entries 2023-07-14T11:54:51.098 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-14T11:54:51.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.279+0000 7fddbef2d700 1 Processor -- start 2023-07-14T11:54:51.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.279+0000 7fddbef2d700 1 -- start start 2023-07-14T11:54:51.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.283+0000 7fddbef2d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fddb80b1530 0x7fddb80d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:51.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.283+0000 7fddbef2d700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb815c2f0 con 0x7fddb80d1260 2023-07-14T11:54:51.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.283+0000 7fddbef2d700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb8070830 con 0x7fddb815bf20 2023-07-14T11:54:51.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.283+0000 7fddbef2d700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb80b5b30 con 0x7fddb80b1530 2023-07-14T11:54:51.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.283+0000 7fddbd4ca700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fddb80d1260 0x7fddb80b4050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56634/0 (socket says 172.21.15.104:56634) 2023-07-14T11:54:51.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.283+0000 7fddbd4ca700 1 -- 172.21.15.104:0/3997051052 learned_addr learned my addr 172.21.15.104:0/3997051052 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:51.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.283+0000 7fddaffff700 1 --2- 172.21.15.104:0/3997051052 >> v2:172.21.15.121:3300/0 conn(0x7fddb80b1530 0x7fddb80d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:51.271 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.283+0000 7fddaf7fe700 1 -- 172.21.15.104:0/3997051052 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411371477 0 0) 0x7fddb815c2f0 con 0x7fddb80d1260 2023-07-14T11:54:51.271 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddaf7fe700 1 -- 172.21.15.104:0/3997051052 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdda00035d0 con 0x7fddb80d1260 2023-07-14T11:54:51.271 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddaf7fe700 1 -- 172.21.15.104:0/3997051052 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517171269 0 0) 0x7fddb8070830 con 0x7fddb815bf20 2023-07-14T11:54:51.271 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddaf7fe700 1 -- 172.21.15.104:0/3997051052 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddb815c2f0 con 0x7fddb815bf20 2023-07-14T11:54:51.271 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddaf7fe700 1 -- 172.21.15.104:0/3997051052 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2841687836 0 0) 0x7fdda00035d0 con 0x7fddb80d1260 2023-07-14T11:54:51.272 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddaffff700 1 -- 172.21.15.104:0/3997051052 >> v1:172.21.15.121:6789/0 conn(0x7fddb815bf20 legacy=0x7fddb80b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:51.272 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddaffff700 1 -- 172.21.15.104:0/3997051052 >> v1:172.21.15.104:6789/0 conn(0x7fddb80d1260 legacy=0x7fddb80b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:51.272 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddaffff700 1 -- 172.21.15.104:0/3997051052 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb8070b10 con 0x7fddb80b1530 2023-07-14T11:54:51.272 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddaffff700 1 --2- 172.21.15.104:0/3997051052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb80b1530 0x7fddb80d0d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009850 tx=0x7fdda8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=323a24c245a12d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:51.273 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddaf7fe700 1 -- 172.21.15.104:0/3997051052 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb00019f0 con 0x7fddb80b1530 2023-07-14T11:54:51.273 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddaf7fe700 1 -- 172.21.15.104:0/3997051052 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda8004030 con 0x7fddb80b1530 2023-07-14T11:54:51.273 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddaf7fe700 1 -- 172.21.15.104:0/3997051052 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda80043c0 con 0x7fddb80b1530 2023-07-14T11:54:51.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddbef2d700 1 -- 172.21.15.104:0/3997051052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb80b1530 msgr2=0x7fddb80d0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:51.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddbef2d700 1 --2- 172.21.15.104:0/3997051052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb80b1530 0x7fddb80d0d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009850 tx=0x7fdda8009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:51.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddbef2d700 1 -- 172.21.15.104:0/3997051052 shutdown_connections 2023-07-14T11:54:51.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddbef2d700 1 --2- 172.21.15.104:0/3997051052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb80b1530 0x7fddb80d0d20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:51.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddbef2d700 1 -- 172.21.15.104:0/3997051052 >> 172.21.15.104:0/3997051052 conn(0x7fddb806fc80 msgr2=0x7fddb8070080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:51.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.287+0000 7fddbef2d700 1 -- 172.21.15.104:0/3997051052 shutdown_connections 2023-07-14T11:54:51.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.291+0000 7fddbef2d700 1 -- 172.21.15.104:0/3997051052 wait complete. 2023-07-14T11:54:51.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.291+0000 7fddbef2d700 1 Processor -- start 2023-07-14T11:54:51.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.291+0000 7fddbef2d700 1 -- start start 2023-07-14T11:54:51.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddbef2d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb80b1530 0x7fddb8154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:51.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddbef2d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddb80d1260 0x7fddb815d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:51.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddbef2d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddb815bf20 0x7fddb80b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:51.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddbef2d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb80c7240 con 0x7fddb80d1260 2023-07-14T11:54:51.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddbef2d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb815eb60 con 0x7fddb80b1530 2023-07-14T11:54:51.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddbef2d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb815e2c0 con 0x7fddb815bf20 2023-07-14T11:54:51.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddaffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddb80d1260 0x7fddb815d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:51.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddaffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddb80d1260 0x7fddb815d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57876/0 (socket says 172.21.15.104:57876) 2023-07-14T11:54:51.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddaffff700 1 -- 172.21.15.104:0/1952476345 learned_addr learned my addr 172.21.15.104:0/1952476345 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:51.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddbccc9700 1 --2- 172.21.15.104:0/1952476345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb80b1530 0x7fddb8154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:51.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddbd4ca700 1 --2- 172.21.15.104:0/1952476345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddb815bf20 0x7fddb80b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:51.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddbd4ca700 1 -- 172.21.15.104:0/1952476345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb80b1530 msgr2=0x7fddb8154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:51.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddbd4ca700 1 --2- 172.21.15.104:0/1952476345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb80b1530 0x7fddb8154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:51.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddbd4ca700 1 -- 172.21.15.104:0/1952476345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddb80d1260 msgr2=0x7fddb815d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:51.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddbd4ca700 1 --2- 172.21.15.104:0/1952476345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddb80d1260 0x7fddb815d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:51.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddbd4ca700 1 -- 172.21.15.104:0/1952476345 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda80094b0 con 0x7fddb815bf20 2023-07-14T11:54:51.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddaffff700 1 --2- 172.21.15.104:0/1952476345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddb80d1260 0x7fddb815d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:51.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddbd4ca700 1 --2- 172.21.15.104:0/1952476345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddb815bf20 0x7fddb80b4340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fddb000cbe0 tx=0x7fddb000cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:51.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddad7fa700 1 -- 172.21.15.104:0/1952476345 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb000aaa0 con 0x7fddb815bf20 2023-07-14T11:54:51.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddad7fa700 1 -- 172.21.15.104:0/1952476345 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb000ac00 con 0x7fddb815bf20 2023-07-14T11:54:51.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddad7fa700 1 -- 172.21.15.104:0/1952476345 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb00070d0 con 0x7fddb815bf20 2023-07-14T11:54:51.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.295+0000 7fddbef2d700 1 -- 172.21.15.104:0/1952476345 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb815e5a0 con 0x7fddb815bf20 2023-07-14T11:54:51.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.299+0000 7fddbef2d700 1 -- 172.21.15.104:0/1952476345 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb80b7f70 con 0x7fddb815bf20 2023-07-14T11:54:51.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.299+0000 7fddbef2d700 1 -- 172.21.15.104:0/1952476345 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb80b8160 con 0x7fddb815bf20 2023-07-14T11:54:51.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.303+0000 7fddad7fa700 1 -- 172.21.15.104:0/1952476345 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fddb0007230 con 0x7fddb815bf20 2023-07-14T11:54:51.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.303+0000 7fddad7fa700 1 --2- 172.21.15.104:0/1952476345 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdda4066340 0x7fdda40687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:51.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.303+0000 7fddad7fa700 1 -- 172.21.15.104:0/1952476345 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fddb00838d0 con 0x7fddb815bf20 2023-07-14T11:54:51.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.303+0000 7fddbccc9700 1 --2- 172.21.15.104:0/1952476345 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdda4066340 0x7fdda40687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:51.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.303+0000 7fddbccc9700 1 --2- 172.21.15.104:0/1952476345 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdda4066340 0x7fdda40687f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fddb40011e0 tx=0x7fddb4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:51.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.311+0000 7fddad7fa700 1 -- 172.21.15.104:0/1952476345 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fddb80b8160 con 0x7fddb815bf20 2023-07-14T11:54:51.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.543+0000 7fddbef2d700 1 -- 172.21.15.104:0/1952476345 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fddb80b8160 con 0x7fddb815bf20 2023-07-14T11:54:51.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.543+0000 7fddad7fa700 1 -- 172.21.15.104:0/1952476345 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fddb80b8160 con 0x7fddb815bf20 2023-07-14T11:54:51.531 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:51.531 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:54:51.531 INFO:teuthology.orchestra.run.smithi104.stdout: "auth_dump": [ 2023-07-14T11:54:51.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.532 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "mds.a", 2023-07-14T11:54:51.532 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDqNrFkusAwKBAAd0vgoEvYzWoPG+qOPSx7xg==", 2023-07-14T11:54:51.533 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.533 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:54:51.533 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow *", 2023-07-14T11:54:51.533 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow *", 2023-07-14T11:54:51.534 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:54:51.534 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.535 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "mds.b", 2023-07-14T11:54:51.535 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDqNrFkjoJoOxAA4nNPKT2i5JIO7aF42mYIzg==", 2023-07-14T11:54:51.535 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.536 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:54:51.536 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow *", 2023-07-14T11:54:51.536 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow *", 2023-07-14T11:54:51.536 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:54:51.536 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.537 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "mds.c", 2023-07-14T11:54:51.538 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDqNrFkRtpAMxAAq3jyn4VjOGmZrpCJVy6BYw==", 2023-07-14T11:54:51.538 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.538 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:54:51.538 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow *", 2023-07-14T11:54:51.538 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow *", 2023-07-14T11:54:51.539 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:54:51.539 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.539 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "mds.d", 2023-07-14T11:54:51.539 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDrNrFkADXNChAA5ndYnLdDxZQwMJBBT4XBtQ==", 2023-07-14T11:54:51.539 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.540 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:54:51.540 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow *", 2023-07-14T11:54:51.540 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow *", 2023-07-14T11:54:51.540 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:54:51.540 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.541 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.0", 2023-07-14T11:54:51.541 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDuNrFkvPfmNhAABqDrwKAMelrRtze4ZCW4fw==", 2023-07-14T11:54:51.541 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.541 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:54:51.541 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:54:51.541 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:54:51.542 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.542 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.1", 2023-07-14T11:54:51.542 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDyNrFkNslaCRAAntw0JcsmGTwVodNy8b7LeQ==", 2023-07-14T11:54:51.542 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.542 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:54:51.542 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:54:51.543 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:54:51.543 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.543 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.2", 2023-07-14T11:54:51.543 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQD1NrFkTkx9FhAAoxh/ll+DixRrVsqpDPdKZA==", 2023-07-14T11:54:51.543 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.543 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:54:51.544 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:54:51.544 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:54:51.544 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.544 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.3", 2023-07-14T11:54:51.544 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQD4NrFkiGAHJRAA0o2NVYbQ5uGbsLaZmojHBA==", 2023-07-14T11:54:51.545 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.545 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:54:51.545 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:54:51.545 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:54:51.545 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.546 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.4", 2023-07-14T11:54:51.546 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQD+NrFkZ+13OhAAKNDNbO4EgSTYmT1sujX2TQ==", 2023-07-14T11:54:51.546 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.546 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:54:51.546 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:54:51.546 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:54:51.546 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.547 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.5", 2023-07-14T11:54:51.547 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQACN7FkY69aDBAA6qbtQcxmW84CHeizN6bCvw==", 2023-07-14T11:54:51.547 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.547 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:54:51.547 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:54:51.547 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:54:51.547 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.548 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.6", 2023-07-14T11:54:51.548 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQAFN7Fk6roFGRAA2A0Zt8ivszAYsaGf0wQH0Q==", 2023-07-14T11:54:51.548 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.548 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:54:51.548 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:54:51.549 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:54:51.549 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.549 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.7", 2023-07-14T11:54:51.549 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQAIN7FkEVH+JRAAc5EcTI6vOzyNM1iXiESbjQ==", 2023-07-14T11:54:51.549 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.549 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:54:51.550 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:54:51.550 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:54:51.550 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.550 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.0", 2023-07-14T11:54:51.550 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDrNrFkbrzjERAA+9BBzuauwUdhALUcDtd80A==", 2023-07-14T11:54:51.550 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.551 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:54:51.551 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow r", 2023-07-14T11:54:51.551 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow rw", 2023-07-14T11:54:51.551 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow rwx" 2023-07-14T11:54:51.551 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.552 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.1", 2023-07-14T11:54:51.552 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDrNrFk9gt2HRAAZ0izobs6vpgfpG6isDFRbQ==", 2023-07-14T11:54:51.552 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.552 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:54:51.552 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow r", 2023-07-14T11:54:51.552 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow rw", 2023-07-14T11:54:51.553 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow rwx" 2023-07-14T11:54:51.553 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.553 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.2", 2023-07-14T11:54:51.553 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDrNrFkNmBZFxAA96kq+aaMbx6FQIJGUdN6xQ==", 2023-07-14T11:54:51.553 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.553 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:54:51.554 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow r", 2023-07-14T11:54:51.554 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow rw", 2023-07-14T11:54:51.554 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow rwx" 2023-07-14T11:54:51.554 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.555 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.3", 2023-07-14T11:54:51.555 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDrNrFkiTBxIxAAfEZF/Xe7Kc/QZHkBmHFShg==", 2023-07-14T11:54:51.555 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.555 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:54:51.555 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow r", 2023-07-14T11:54:51.555 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow rw", 2023-07-14T11:54:51.555 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow rwx" 2023-07-14T11:54:51.555 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.556 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.admin", 2023-07-14T11:54:51.556 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDpNrFkagugORAAjrkuHLbV3wPXHZC4Y4tgag==", 2023-07-14T11:54:51.556 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.556 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow *", 2023-07-14T11:54:51.556 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow *", 2023-07-14T11:54:51.557 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow *", 2023-07-14T11:54:51.557 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:54:51.557 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.557 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.bootstrap-mds", 2023-07-14T11:54:51.557 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQAPN7FkD15VLBAA4yVxEfka1yYrQRDVWCJfNg==", 2023-07-14T11:54:51.557 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.558 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile bootstrap-mds" 2023-07-14T11:54:51.558 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.558 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.bootstrap-mgr", 2023-07-14T11:54:51.558 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQAPN7Fk/GtVLBAAg79LZGjtjMwZJS6hEa1H+g==", 2023-07-14T11:54:51.558 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.559 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile bootstrap-mgr" 2023-07-14T11:54:51.559 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.559 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.bootstrap-osd", 2023-07-14T11:54:51.559 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQAPN7Fk0ndVLBAArawli/h/EIhP+I+F7rz60g==", 2023-07-14T11:54:51.559 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.559 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile bootstrap-osd" 2023-07-14T11:54:51.560 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.560 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.bootstrap-rbd", 2023-07-14T11:54:51.560 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQAPN7FkZIVVLBAAUvwwVv8iQ5Pa5Wh1n+HPQg==", 2023-07-14T11:54:51.560 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.560 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile bootstrap-rbd" 2023-07-14T11:54:51.560 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.561 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-07-14T11:54:51.561 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQAPN7FkFJFVLBAAZfkQ3uFCG/PaAdESY/iQjw==", 2023-07-14T11:54:51.561 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.561 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-07-14T11:54:51.561 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.562 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.bootstrap-rgw", 2023-07-14T11:54:51.562 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQAPN7FkkJxVLBAA3LzQl+jl6Q/b7/xtHdkZPQ==", 2023-07-14T11:54:51.562 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.562 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile bootstrap-rgw" 2023-07-14T11:54:51.563 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.563 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "mgr.x", 2023-07-14T11:54:51.563 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDqNrFkqhD3IBAAE14Ks6MpY1PyMfpbQcUgjQ==", 2023-07-14T11:54:51.563 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.563 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow *", 2023-07-14T11:54:51.563 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile mgr", 2023-07-14T11:54:51.564 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:54:51.564 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:51.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:51.564 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "mgr.y", 2023-07-14T11:54:51.564 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDqNrFkmLyZGhAAe2/pYD9LAp8Y6JR+IxqDuA==", 2023-07-14T11:54:51.565 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:54:51.565 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow *", 2023-07-14T11:54:51.565 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile mgr", 2023-07-14T11:54:51.565 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:54:51.565 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.565 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:51.565 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2023-07-14T11:54:51.566 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:54:51.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.547+0000 7fddbef2d700 1 -- 172.21.15.104:0/1952476345 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdda4066340 msgr2=0x7fdda40687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:51.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.547+0000 7fddbef2d700 1 --2- 172.21.15.104:0/1952476345 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdda4066340 0x7fdda40687f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fddb40011e0 tx=0x7fddb4009040 comp rx=0 tx=0).stop 2023-07-14T11:54:51.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.547+0000 7fddbef2d700 1 -- 172.21.15.104:0/1952476345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddb815bf20 msgr2=0x7fddb80b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:51.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.547+0000 7fddbef2d700 1 --2- 172.21.15.104:0/1952476345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddb815bf20 0x7fddb80b4340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fddb000cbe0 tx=0x7fddb000cef0 comp rx=0 tx=0).stop 2023-07-14T11:54:51.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.547+0000 7fddbef2d700 1 -- 172.21.15.104:0/1952476345 shutdown_connections 2023-07-14T11:54:51.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.547+0000 7fddbef2d700 1 --2- 172.21.15.104:0/1952476345 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdda4066340 0x7fdda40687f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:51.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.547+0000 7fddbef2d700 1 --2- 172.21.15.104:0/1952476345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddb80d1260 0x7fddb815d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:51.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.547+0000 7fddbef2d700 1 --2- 172.21.15.104:0/1952476345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddb815bf20 0x7fddb80b4340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:51.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.547+0000 7fddbef2d700 1 --2- 172.21.15.104:0/1952476345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb80b1530 0x7fddb8154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:51.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.547+0000 7fddbef2d700 1 -- 172.21.15.104:0/1952476345 >> 172.21.15.104:0/1952476345 conn(0x7fddb806fc80 msgr2=0x7fddb80b5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:51.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.547+0000 7fddbef2d700 1 -- 172.21.15.104:0/1952476345 shutdown_connections 2023-07-14T11:54:51.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.547+0000 7fddbef2d700 1 -- 172.21.15.104:0/1952476345 wait complete. 2023-07-14T11:54:51.569 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-07-14T11:54:51.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.755+0000 7f0827fff700 1 Processor -- start 2023-07-14T11:54:51.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.755+0000 7f0827fff700 1 -- start start 2023-07-14T11:54:51.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.759+0000 7f0827fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f08280c86b0 0x7f08280c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:51.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.759+0000 7f0827fff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08280cbf80 con 0x7f08280cb100 2023-07-14T11:54:51.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.759+0000 7f0827fff700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08280cbc30 con 0x7f0828158370 2023-07-14T11:54:51.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.759+0000 7f0827fff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08280c8fc0 con 0x7f08280c86b0 2023-07-14T11:54:51.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.759+0000 7f0826ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f08280c86b0 0x7f08280c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:51.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.759+0000 7f08277fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f08280cb100 0x7f082806bc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56636/0 (socket says 172.21.15.104:56636) 2023-07-14T11:54:51.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.759+0000 7f08277fe700 1 -- 172.21.15.104:0/2846632427 learned_addr learned my addr 172.21.15.104:0/2846632427 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:51.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0825ffb700 1 -- 172.21.15.104:0/2846632427 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110962793 0 0) 0x7f08280cbc30 con 0x7f0828158370 2023-07-14T11:54:51.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0825ffb700 1 -- 172.21.15.104:0/2846632427 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08100035d0 con 0x7f0828158370 2023-07-14T11:54:51.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0825ffb700 1 -- 172.21.15.104:0/2846632427 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813071793 0 0) 0x7f08280cbf80 con 0x7f08280cb100 2023-07-14T11:54:51.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0826ffd700 1 -- 172.21.15.104:0/2846632427 >> v1:172.21.15.121:6789/0 conn(0x7f0828158370 legacy=0x7f082815bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:51.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0826ffd700 1 -- 172.21.15.104:0/2846632427 >> v1:172.21.15.104:6789/0 conn(0x7f08280cb100 legacy=0x7f082806bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:51.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0826ffd700 1 -- 172.21.15.104:0/2846632427 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082815c4d0 con 0x7f08280c86b0 2023-07-14T11:54:51.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0826ffd700 1 --2- 172.21.15.104:0/2846632427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08280c86b0 0x7f08280c8a80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0818009880 tx=0x7f0818009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af05d2b39283831b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:51.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0825ffb700 1 -- 172.21.15.104:0/2846632427 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818010800 con 0x7f08280c86b0 2023-07-14T11:54:51.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0825ffb700 1 -- 172.21.15.104:0/2846632427 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0818010960 con 0x7f08280c86b0 2023-07-14T11:54:51.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0825ffb700 1 -- 172.21.15.104:0/2846632427 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818010cf0 con 0x7f08280c86b0 2023-07-14T11:54:51.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0827fff700 1 -- 172.21.15.104:0/2846632427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08280c86b0 msgr2=0x7f08280c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:51.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0827fff700 1 --2- 172.21.15.104:0/2846632427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08280c86b0 0x7f08280c8a80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0818009880 tx=0x7f0818009b90 comp rx=0 tx=0).stop 2023-07-14T11:54:51.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0827fff700 1 -- 172.21.15.104:0/2846632427 shutdown_connections 2023-07-14T11:54:51.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0827fff700 1 --2- 172.21.15.104:0/2846632427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08280c86b0 0x7f08280c8a80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:51.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0827fff700 1 -- 172.21.15.104:0/2846632427 >> 172.21.15.104:0/2846632427 conn(0x7f082806e960 msgr2=0x7f082806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:51.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0827fff700 1 -- 172.21.15.104:0/2846632427 shutdown_connections 2023-07-14T11:54:51.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.763+0000 7f0827fff700 1 -- 172.21.15.104:0/2846632427 wait complete. 2023-07-14T11:54:51.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f0827fff700 1 Processor -- start 2023-07-14T11:54:51.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f0827fff700 1 -- start start 2023-07-14T11:54:51.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f0827fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08280c86b0 0x7f0828152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:51.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f0827fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08280cb100 0x7f08280af760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:51.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f0827fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0828158370 0x7f08280afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:51.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f0827fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08280adc00 con 0x7f0828158370 2023-07-14T11:54:51.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f0827fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08280b0700 con 0x7f08280c86b0 2023-07-14T11:54:51.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f0827fff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08280adea0 con 0x7f08280cb100 2023-07-14T11:54:51.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f08277fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0828158370 0x7f08280afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:51.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f08277fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0828158370 0x7f08280afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57888/0 (socket says 172.21.15.104:57888) 2023-07-14T11:54:51.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f08277fe700 1 -- 172.21.15.104:0/4095999583 learned_addr learned my addr 172.21.15.104:0/4095999583 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:51.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f0826ffd700 1 --2- 172.21.15.104:0/4095999583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08280c86b0 0x7f0828152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:51.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f08267fc700 1 --2- 172.21.15.104:0/4095999583 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08280cb100 0x7f08280af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:51.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f08277fe700 1 -- 172.21.15.104:0/4095999583 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08280cb100 msgr2=0x7f08280af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:51.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f08277fe700 1 --2- 172.21.15.104:0/4095999583 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08280cb100 0x7f08280af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:51.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f08277fe700 1 -- 172.21.15.104:0/4095999583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08280c86b0 msgr2=0x7f0828152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:51.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f08277fe700 1 --2- 172.21.15.104:0/4095999583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08280c86b0 0x7f0828152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:51.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.767+0000 7f08277fe700 1 -- 172.21.15.104:0/4095999583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08180094b0 con 0x7f0828158370 2023-07-14T11:54:51.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.771+0000 7f08277fe700 1 --2- 172.21.15.104:0/4095999583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0828158370 0x7f08280afca0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0820002230 tx=0x7f082000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:51.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.771+0000 7f080f7fe700 1 -- 172.21.15.104:0/4095999583 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082000aac0 con 0x7f0828158370 2023-07-14T11:54:51.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.771+0000 7f080f7fe700 1 -- 172.21.15.104:0/4095999583 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082000ac40 con 0x7f0828158370 2023-07-14T11:54:51.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.771+0000 7f0827fff700 1 -- 172.21.15.104:0/4095999583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08280ae180 con 0x7f0828158370 2023-07-14T11:54:51.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.771+0000 7f080f7fe700 1 -- 172.21.15.104:0/4095999583 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08200070d0 con 0x7f0828158370 2023-07-14T11:54:51.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.771+0000 7f0827fff700 1 -- 172.21.15.104:0/4095999583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0828151ea0 con 0x7f0828158370 2023-07-14T11:54:51.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.771+0000 7f0827fff700 1 -- 172.21.15.104:0/4095999583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08280c55c0 con 0x7f0828158370 2023-07-14T11:54:51.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.775+0000 7f080f7fe700 1 -- 172.21.15.104:0/4095999583 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f0820007250 con 0x7f0828158370 2023-07-14T11:54:51.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.775+0000 7f080f7fe700 1 --2- 172.21.15.104:0/4095999583 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f08140662f0 0x7f08140687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:51.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.775+0000 7f080f7fe700 1 -- 172.21.15.104:0/4095999583 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f0820083590 con 0x7f0828158370 2023-07-14T11:54:51.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.775+0000 7f0826ffd700 1 --2- 172.21.15.104:0/4095999583 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f08140662f0 0x7f08140687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:51.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.775+0000 7f0826ffd700 1 --2- 172.21.15.104:0/4095999583 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f08140662f0 0x7f08140687a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f08180090b0 tx=0x7f081801b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:51.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:51.783+0000 7f080f7fe700 1 -- 172.21.15.104:0/4095999583 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f08280c55c0 con 0x7f0828158370 2023-07-14T11:54:51.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.011+0000 7f0827fff700 1 -- 172.21.15.104:0/4095999583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f08280c55c0 con 0x7f0828158370 2023-07-14T11:54:52.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.019+0000 7f080f7fe700 1 -- 172.21.15.104:0/4095999583 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f08280c55c0 con 0x7f0828158370 2023-07-14T11:54:52.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.019+0000 7f0827fff700 1 -- 172.21.15.104:0/4095999583 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f08140662f0 msgr2=0x7f08140687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:52.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.019+0000 7f0827fff700 1 --2- 172.21.15.104:0/4095999583 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f08140662f0 0x7f08140687a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f08180090b0 tx=0x7f081801b040 comp rx=0 tx=0).stop 2023-07-14T11:54:52.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.019+0000 7f0827fff700 1 -- 172.21.15.104:0/4095999583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0828158370 msgr2=0x7f08280afca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:52.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.019+0000 7f0827fff700 1 --2- 172.21.15.104:0/4095999583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0828158370 0x7f08280afca0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0820002230 tx=0x7f082000cf50 comp rx=0 tx=0).stop 2023-07-14T11:54:52.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.019+0000 7f0827fff700 1 -- 172.21.15.104:0/4095999583 shutdown_connections 2023-07-14T11:54:52.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.019+0000 7f0827fff700 1 --2- 172.21.15.104:0/4095999583 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f08140662f0 0x7f08140687a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.019+0000 7f0827fff700 1 --2- 172.21.15.104:0/4095999583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0828158370 0x7f08280afca0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.019+0000 7f0827fff700 1 --2- 172.21.15.104:0/4095999583 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08280cb100 0x7f08280af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.019+0000 7f0827fff700 1 --2- 172.21.15.104:0/4095999583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08280c86b0 0x7f0828152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.019+0000 7f0827fff700 1 -- 172.21.15.104:0/4095999583 >> 172.21.15.104:0/4095999583 conn(0x7f082806e960 msgr2=0x7f082806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:52.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.019+0000 7f0827fff700 1 -- 172.21.15.104:0/4095999583 shutdown_connections 2023-07-14T11:54:52.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.019+0000 7f0827fff700 1 -- 172.21.15.104:0/4095999583 wait complete. 2023-07-14T11:54:52.026 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-07-14T11:54:52.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.211+0000 7f2991def700 1 Processor -- start 2023-07-14T11:54:52.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.211+0000 7f2991def700 1 -- start start 2023-07-14T11:54:52.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.211+0000 7f2991def700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f298c15ffa0 0x7f298c06d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:52.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.211+0000 7f2991def700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f298c0cb180 0x7f298c0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:52.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.211+0000 7f2991def700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298c0ca8a0 con 0x7f298c06dc50 2023-07-14T11:54:52.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.211+0000 7f2991def700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298c0cbcd0 con 0x7f298c0cb180 2023-07-14T11:54:52.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.211+0000 7f2991def700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298c0cbf70 con 0x7f298c15ffa0 2023-07-14T11:54:52.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f298bfff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f298c06dc50 0x7f298c0caa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56650/0 (socket says 172.21.15.104:56650) 2023-07-14T11:54:52.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f298bfff700 1 -- 172.21.15.104:0/1566017801 learned_addr learned my addr 172.21.15.104:0/1566017801 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:52.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f29915ee700 1 --2- 172.21.15.104:0/1566017801 >> v2:172.21.15.121:3300/0 conn(0x7f298c0cb180 0x7f298c0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:52.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f2990ded700 1 --2- 172.21.15.104:0/1566017801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f298c15ffa0 0x7f298c06d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:52.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f298b7fe700 1 -- 172.21.15.104:0/1566017801 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167425535 0 0) 0x7f297c001110 con 0x7f298c06dc50 2023-07-14T11:54:52.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f298b7fe700 1 -- 172.21.15.104:0/1566017801 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29740035d0 con 0x7f298c06dc50 2023-07-14T11:54:52.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f29915ee700 1 -- 172.21.15.104:0/1566017801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f298c15ffa0 msgr2=0x7f298c06d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:52.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f29915ee700 1 --2- 172.21.15.104:0/1566017801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f298c15ffa0 0x7f298c06d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f29915ee700 1 -- 172.21.15.104:0/1566017801 >> v1:172.21.15.104:6789/0 conn(0x7f298c06dc50 legacy=0x7f298c0caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:52.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f29915ee700 1 -- 172.21.15.104:0/1566017801 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c0cc250 con 0x7f298c0cb180 2023-07-14T11:54:52.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f2990ded700 1 --2- 172.21.15.104:0/1566017801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f298c15ffa0 0x7f298c06d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:52.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f29915ee700 1 --2- 172.21.15.104:0/1566017801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f298c0cb180 0x7f298c0ce340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2984009d90 tx=0x7f298400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa5a4bdb92ef12e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:52.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f298b7fe700 1 -- 172.21.15.104:0/1566017801 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298400e070 con 0x7f298c0cb180 2023-07-14T11:54:52.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f298b7fe700 1 -- 172.21.15.104:0/1566017801 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298400bb20 con 0x7f298c0cb180 2023-07-14T11:54:52.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f298b7fe700 1 -- 172.21.15.104:0/1566017801 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2984012430 con 0x7f298c0cb180 2023-07-14T11:54:52.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f2991def700 1 -- 172.21.15.104:0/1566017801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f298c0cb180 msgr2=0x7f298c0ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:52.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f2991def700 1 --2- 172.21.15.104:0/1566017801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f298c0cb180 0x7f298c0ce340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2984009d90 tx=0x7f298400b330 comp rx=0 tx=0).stop 2023-07-14T11:54:52.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f2991def700 1 -- 172.21.15.104:0/1566017801 shutdown_connections 2023-07-14T11:54:52.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f2991def700 1 --2- 172.21.15.104:0/1566017801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f298c0cb180 0x7f298c0ce340 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f2991def700 1 --2- 172.21.15.104:0/1566017801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f298c15ffa0 0x7f298c06d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f2991def700 1 -- 172.21.15.104:0/1566017801 >> 172.21.15.104:0/1566017801 conn(0x7f298c06e960 msgr2=0x7f298c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:52.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f2991def700 1 -- 172.21.15.104:0/1566017801 shutdown_connections 2023-07-14T11:54:52.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.215+0000 7f2991def700 1 -- 172.21.15.104:0/1566017801 wait complete. 2023-07-14T11:54:52.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.219+0000 7f2991def700 1 Processor -- start 2023-07-14T11:54:52.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.219+0000 7f2991def700 1 -- start start 2023-07-14T11:54:52.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f2991def700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f298c06dc50 0x7f298c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:52.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f2991def700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f298c0cb180 0x7f298c0c2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:52.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f2991def700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f298c15ffa0 0x7f298c0c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:52.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f2991def700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c15fae0 con 0x7f298c0cb180 2023-07-14T11:54:52.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f2991def700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c0c2980 con 0x7f298c06dc50 2023-07-14T11:54:52.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f2991def700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298c0c2c60 con 0x7f298c15ffa0 2023-07-14T11:54:52.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f298bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f298c0cb180 0x7f298c0c2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:52.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f298bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f298c0cb180 0x7f298c0c2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57904/0 (socket says 172.21.15.104:57904) 2023-07-14T11:54:52.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f298bfff700 1 -- 172.21.15.104:0/1714819760 learned_addr learned my addr 172.21.15.104:0/1714819760 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:52.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f2990ded700 1 --2- 172.21.15.104:0/1714819760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f298c06dc50 0x7f298c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:52.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f29915ee700 1 --2- 172.21.15.104:0/1714819760 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f298c15ffa0 0x7f298c0c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:52.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f298bfff700 1 -- 172.21.15.104:0/1714819760 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f298c15ffa0 msgr2=0x7f298c0c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:52.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f298bfff700 1 --2- 172.21.15.104:0/1714819760 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f298c15ffa0 0x7f298c0c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f298bfff700 1 -- 172.21.15.104:0/1714819760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f298c06dc50 msgr2=0x7f298c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:52.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f298bfff700 1 --2- 172.21.15.104:0/1714819760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f298c06dc50 0x7f298c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f298bfff700 1 -- 172.21.15.104:0/1714819760 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29800094b0 con 0x7f298c0cb180 2023-07-14T11:54:52.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f298bfff700 1 --2- 172.21.15.104:0/1714819760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f298c0cb180 0x7f298c0c2130 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f297c00c500 tx=0x7f297c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:52.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f29897fa700 1 -- 172.21.15.104:0/1714819760 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c005d90 con 0x7f298c0cb180 2023-07-14T11:54:52.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f29897fa700 1 -- 172.21.15.104:0/1714819760 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c0070d0 con 0x7f298c0cb180 2023-07-14T11:54:52.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f2991def700 1 -- 172.21.15.104:0/1714819760 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2984009a70 con 0x7f298c0cb180 2023-07-14T11:54:52.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f29897fa700 1 -- 172.21.15.104:0/1714819760 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c007440 con 0x7f298c0cb180 2023-07-14T11:54:52.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.223+0000 7f2991def700 1 -- 172.21.15.104:0/1714819760 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298c0c3220 con 0x7f298c0cb180 2023-07-14T11:54:52.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.227+0000 7f2991def700 1 -- 172.21.15.104:0/1714819760 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298c071620 con 0x7f298c0cb180 2023-07-14T11:54:52.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.227+0000 7f29897fa700 1 -- 172.21.15.104:0/1714819760 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f297c0075a0 con 0x7f298c0cb180 2023-07-14T11:54:52.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.231+0000 7f29897fa700 1 --2- 172.21.15.104:0/1714819760 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f29780662f0 0x7f29780687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:52.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.231+0000 7f29897fa700 1 -- 172.21.15.104:0/1714819760 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f297c082c00 con 0x7f298c0cb180 2023-07-14T11:54:52.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.231+0000 7f2990ded700 1 --2- 172.21.15.104:0/1714819760 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f29780662f0 0x7f29780687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:52.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.231+0000 7f2990ded700 1 --2- 172.21.15.104:0/1714819760 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f29780662f0 0x7f29780687a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2980009d80 tx=0x7f2980009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:52.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.235+0000 7f29897fa700 1 -- 172.21.15.104:0/1714819760 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f297c0502a0 con 0x7f298c0cb180 2023-07-14T11:54:52.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.459+0000 7f2991def700 1 -- 172.21.15.104:0/1714819760 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f298c0ca8a0 con 0x7f298c0cb180 2023-07-14T11:54:52.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.467+0000 7f29897fa700 1 -- 172.21.15.104:0/1714819760 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f298c0ca8a0 con 0x7f298c0cb180 2023-07-14T11:54:52.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.471+0000 7f2991def700 1 -- 172.21.15.104:0/1714819760 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f29780662f0 msgr2=0x7f29780687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:52.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.471+0000 7f2991def700 1 --2- 172.21.15.104:0/1714819760 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f29780662f0 0x7f29780687a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2980009d80 tx=0x7f2980009240 comp rx=0 tx=0).stop 2023-07-14T11:54:52.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.471+0000 7f2991def700 1 -- 172.21.15.104:0/1714819760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f298c0cb180 msgr2=0x7f298c0c2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:52.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.471+0000 7f2991def700 1 --2- 172.21.15.104:0/1714819760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f298c0cb180 0x7f298c0c2130 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f297c00c500 tx=0x7f297c00c810 comp rx=0 tx=0).stop 2023-07-14T11:54:52.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.471+0000 7f2991def700 1 -- 172.21.15.104:0/1714819760 shutdown_connections 2023-07-14T11:54:52.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.471+0000 7f2991def700 1 --2- 172.21.15.104:0/1714819760 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f29780662f0 0x7f29780687a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.471+0000 7f2991def700 1 --2- 172.21.15.104:0/1714819760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f298c0cb180 0x7f298c0c2130 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.471+0000 7f2991def700 1 --2- 172.21.15.104:0/1714819760 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f298c15ffa0 0x7f298c0c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.471+0000 7f2991def700 1 --2- 172.21.15.104:0/1714819760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f298c06dc50 0x7f298c15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.471+0000 7f2991def700 1 -- 172.21.15.104:0/1714819760 >> 172.21.15.104:0/1714819760 conn(0x7f298c06e960 msgr2=0x7f298c0cde90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:52.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.471+0000 7f2991def700 1 -- 172.21.15.104:0/1714819760 shutdown_connections 2023-07-14T11:54:52.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.471+0000 7f2991def700 1 -- 172.21.15.104:0/1714819760 wait complete. 2023-07-14T11:54:52.476 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-07-14T11:54:52.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.663+0000 7f15c485e700 1 Processor -- start 2023-07-14T11:54:52.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.663+0000 7f15c485e700 1 -- start start 2023-07-14T11:54:52.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.663+0000 7f15c485e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f15c00cddb0 0x7f15c0070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:52.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.663+0000 7f15c485e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c00c8900 con 0x7f15c0160640 2023-07-14T11:54:52.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.663+0000 7f15c485e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c00caae0 con 0x7f15c0070ea0 2023-07-14T11:54:52.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.663+0000 7f15c485e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c00c8fc0 con 0x7f15c00cddb0 2023-07-14T11:54:52.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15bdd9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f15c00cddb0 0x7f15c0070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:52.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15be59c700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f15c0160640 0x7f15c0164000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56656/0 (socket says 172.21.15.104:56656) 2023-07-14T11:54:52.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15be59c700 1 -- 172.21.15.104:0/833819798 learned_addr learned my addr 172.21.15.104:0/833819798 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:52.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15bdd9b700 1 -- 172.21.15.104:0/833819798 >> v1:172.21.15.121:6789/0 conn(0x7f15c0070ea0 legacy=0x7f15c00afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:52.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15bdd9b700 1 -- 172.21.15.104:0/833819798 >> v1:172.21.15.104:6789/0 conn(0x7f15c0160640 legacy=0x7f15c0164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:52.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15bdd9b700 1 -- 172.21.15.104:0/833819798 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c00ca520 con 0x7f15c00cddb0 2023-07-14T11:54:52.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15bdd9b700 1 --2- 172.21.15.104:0/833819798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c00cddb0 0x7f15c0070960 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009850 tx=0x7f15b4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1db72d53570310 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:52.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15bd59a700 1 -- 172.21.15.104:0/833819798 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4011830 con 0x7f15c00cddb0 2023-07-14T11:54:52.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15bd59a700 1 -- 172.21.15.104:0/833819798 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b4011990 con 0x7f15c00cddb0 2023-07-14T11:54:52.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15bd59a700 1 -- 172.21.15.104:0/833819798 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4011d20 con 0x7f15c00cddb0 2023-07-14T11:54:52.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15c485e700 1 -- 172.21.15.104:0/833819798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c00cddb0 msgr2=0x7f15c0070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:52.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15c485e700 1 --2- 172.21.15.104:0/833819798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c00cddb0 0x7f15c0070960 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009850 tx=0x7f15b4009b60 comp rx=0 tx=0).stop 2023-07-14T11:54:52.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15c485e700 1 -- 172.21.15.104:0/833819798 shutdown_connections 2023-07-14T11:54:52.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15c485e700 1 --2- 172.21.15.104:0/833819798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c00cddb0 0x7f15c0070960 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15c485e700 1 -- 172.21.15.104:0/833819798 >> 172.21.15.104:0/833819798 conn(0x7f15c00cad30 msgr2=0x7f15c00cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:52.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15c485e700 1 -- 172.21.15.104:0/833819798 shutdown_connections 2023-07-14T11:54:52.655 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.667+0000 7f15c485e700 1 -- 172.21.15.104:0/833819798 wait complete. 2023-07-14T11:54:52.655 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15c485e700 1 Processor -- start 2023-07-14T11:54:52.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15c485e700 1 -- start start 2023-07-14T11:54:52.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15c485e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c0070ea0 0x7f15c0156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:52.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15c485e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15c00cddb0 0x7f15c0161a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:52.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15c485e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15c0160640 0x7f15c015e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:52.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15c485e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c00712e0 con 0x7f15c0070ea0 2023-07-14T11:54:52.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15c485e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c0070540 con 0x7f15c00cddb0 2023-07-14T11:54:52.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15c485e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c015e970 con 0x7f15c0160640 2023-07-14T11:54:52.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15be59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c0070ea0 0x7f15c0156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:52.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15be59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c0070ea0 0x7f15c0156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57926/0 (socket says 172.21.15.104:57926) 2023-07-14T11:54:52.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15be59c700 1 -- 172.21.15.104:0/687065110 learned_addr learned my addr 172.21.15.104:0/687065110 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:52.662 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15bdd9b700 1 --2- 172.21.15.104:0/687065110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15c00cddb0 0x7f15c0161a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:52.662 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15bed9d700 1 --2- 172.21.15.104:0/687065110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15c0160640 0x7f15c015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:52.662 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15be59c700 1 -- 172.21.15.104:0/687065110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15c0160640 msgr2=0x7f15c015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:52.663 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15be59c700 1 --2- 172.21.15.104:0/687065110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15c0160640 0x7f15c015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.663 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15be59c700 1 -- 172.21.15.104:0/687065110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15c00cddb0 msgr2=0x7f15c0161a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:52.663 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15be59c700 1 --2- 172.21.15.104:0/687065110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15c00cddb0 0x7f15c0161a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.663 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.671+0000 7f15be59c700 1 -- 172.21.15.104:0/687065110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b40094b0 con 0x7f15c0070ea0 2023-07-14T11:54:52.664 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.675+0000 7f15be59c700 1 --2- 172.21.15.104:0/687065110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c0070ea0 0x7f15c0156630 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f15b000c4d0 tx=0x7f15b000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:52.665 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.675+0000 7f15aeffd700 1 -- 172.21.15.104:0/687065110 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0005d90 con 0x7f15c0070ea0 2023-07-14T11:54:52.666 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.675+0000 7f15aeffd700 1 -- 172.21.15.104:0/687065110 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b00070d0 con 0x7f15c0070ea0 2023-07-14T11:54:52.666 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.675+0000 7f15aeffd700 1 -- 172.21.15.104:0/687065110 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0007440 con 0x7f15c0070ea0 2023-07-14T11:54:52.666 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.675+0000 7f15c485e700 1 -- 172.21.15.104:0/687065110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c01615d0 con 0x7f15c0070ea0 2023-07-14T11:54:52.666 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.675+0000 7f15c485e700 1 -- 172.21.15.104:0/687065110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c01626b0 con 0x7f15c0070ea0 2023-07-14T11:54:52.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.675+0000 7f15c485e700 1 -- 172.21.15.104:0/687065110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c004dcb0 con 0x7f15c0070ea0 2023-07-14T11:54:52.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.679+0000 7f15aeffd700 1 -- 172.21.15.104:0/687065110 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f15b00075a0 con 0x7f15c0070ea0 2023-07-14T11:54:52.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.679+0000 7f15aeffd700 1 --2- 172.21.15.104:0/687065110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f15a0066320 0x7f15a00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:52.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.679+0000 7f15aeffd700 1 -- 172.21.15.104:0/687065110 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f15b0082e30 con 0x7f15c0070ea0 2023-07-14T11:54:52.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.679+0000 7f15bdd9b700 1 --2- 172.21.15.104:0/687065110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f15a0066320 0x7f15a00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:52.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.679+0000 7f15bdd9b700 1 --2- 172.21.15.104:0/687065110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f15a0066320 0x7f15a00687d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f15c00ce1e0 tx=0x7f15b4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:52.671 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.687+0000 7f15aeffd700 1 -- 172.21.15.104:0/687065110 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f15b00504f0 con 0x7f15c0070ea0 2023-07-14T11:54:52.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.919+0000 7f15c485e700 1 -- 172.21.15.104:0/687065110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f15c00cb9f0 con 0x7f15c0070ea0 2023-07-14T11:54:52.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.927+0000 7f15aeffd700 1 -- 172.21.15.104:0/687065110 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f15b0053b10 con 0x7f15c0070ea0 2023-07-14T11:54:52.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.931+0000 7f15c485e700 1 -- 172.21.15.104:0/687065110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f15a0066320 msgr2=0x7f15a00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:52.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.931+0000 7f15c485e700 1 --2- 172.21.15.104:0/687065110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f15a0066320 0x7f15a00687d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f15c00ce1e0 tx=0x7f15b4008030 comp rx=0 tx=0).stop 2023-07-14T11:54:52.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.931+0000 7f15c485e700 1 -- 172.21.15.104:0/687065110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c0070ea0 msgr2=0x7f15c0156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:52.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.931+0000 7f15c485e700 1 --2- 172.21.15.104:0/687065110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c0070ea0 0x7f15c0156630 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f15b000c4d0 tx=0x7f15b000c7e0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.931+0000 7f15c485e700 1 -- 172.21.15.104:0/687065110 shutdown_connections 2023-07-14T11:54:52.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.931+0000 7f15c485e700 1 --2- 172.21.15.104:0/687065110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f15a0066320 0x7f15a00687d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.931+0000 7f15c485e700 1 --2- 172.21.15.104:0/687065110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15c0160640 0x7f15c015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.931+0000 7f15c485e700 1 --2- 172.21.15.104:0/687065110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15c00cddb0 0x7f15c0161a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.931+0000 7f15c485e700 1 --2- 172.21.15.104:0/687065110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c0070ea0 0x7f15c0156630 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:52.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.931+0000 7f15c485e700 1 -- 172.21.15.104:0/687065110 >> 172.21.15.104:0/687065110 conn(0x7f15c00cad30 msgr2=0x7f15c0158a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:52.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.931+0000 7f15c485e700 1 -- 172.21.15.104:0/687065110 shutdown_connections 2023-07-14T11:54:52.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:52.931+0000 7f15c485e700 1 -- 172.21.15.104:0/687065110 wait complete. 2023-07-14T11:54:52.937 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-07-14T11:54:53.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.127+0000 7f94d9274700 1 Processor -- start 2023-07-14T11:54:53.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.127+0000 7f94d9274700 1 -- start start 2023-07-14T11:54:53.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.131+0000 7f94d9274700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94d415f670 0x7f94d40b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:53.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.131+0000 7f94d9274700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f94d40b38b0 0x7f94d40b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:53.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.131+0000 7f94d9274700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d40b3e20 con 0x7f94d40b28a0 2023-07-14T11:54:53.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.131+0000 7f94d9274700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d40b2ce0 con 0x7f94d40b38b0 2023-07-14T11:54:53.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.131+0000 7f94d9274700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40d0b20 con 0x7f94d415f670 2023-07-14T11:54:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.131+0000 7f94cb7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f94d40b38b0 0x7f94d40b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.131+0000 7f94cb7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f94d40b38b0 0x7f94d40b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57928/0 (socket says 172.21.15.104:57928) 2023-07-14T11:54:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.131+0000 7f94cb7fe700 1 -- 172.21.15.104:0/3689852426 learned_addr learned my addr 172.21.15.104:0/3689852426 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.131+0000 7f94d3fff700 1 --2- 172.21.15.104:0/3689852426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94d415f670 0x7f94d40b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.131+0000 7f94cb7fe700 1 -- 172.21.15.104:0/3689852426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94d415f670 msgr2=0x7f94d40b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:53.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.131+0000 7f94cb7fe700 1 --2- 172.21.15.104:0/3689852426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94d415f670 0x7f94d40b1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.131+0000 7f94cb7fe700 1 -- 172.21.15.104:0/3689852426 >> v1:172.21.15.121:6789/0 conn(0x7f94d40b28a0 legacy=0x7f94d40ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:53.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.131+0000 7f94cb7fe700 1 -- 172.21.15.104:0/3689852426 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d40cdb20 con 0x7f94d40b38b0 2023-07-14T11:54:53.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.131+0000 7f94cb7fe700 1 --2- 172.21.15.104:0/3689852426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94d40b38b0 0x7f94d40b2360 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009910 tx=0x7f94bc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3439edf53d842256 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:53.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.135+0000 7f94d37fe700 1 -- 172.21.15.104:0/3689852426 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc011830 con 0x7f94d40b38b0 2023-07-14T11:54:53.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.135+0000 7f94d37fe700 1 -- 172.21.15.104:0/3689852426 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94bc011990 con 0x7f94d40b38b0 2023-07-14T11:54:53.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.135+0000 7f94d37fe700 1 -- 172.21.15.104:0/3689852426 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc011d40 con 0x7f94d40b38b0 2023-07-14T11:54:53.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.135+0000 7f94d9274700 1 -- 172.21.15.104:0/3689852426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94d40b38b0 msgr2=0x7f94d40b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:53.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.135+0000 7f94d9274700 1 --2- 172.21.15.104:0/3689852426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94d40b38b0 0x7f94d40b2360 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009910 tx=0x7f94bc009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:53.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.135+0000 7f94d9274700 1 -- 172.21.15.104:0/3689852426 shutdown_connections 2023-07-14T11:54:53.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.135+0000 7f94d9274700 1 --2- 172.21.15.104:0/3689852426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94d40b38b0 0x7f94d40b2360 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.135+0000 7f94d9274700 1 --2- 172.21.15.104:0/3689852426 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94d415f670 0x7f94d40b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.135+0000 7f94d9274700 1 -- 172.21.15.104:0/3689852426 >> 172.21.15.104:0/3689852426 conn(0x7f94d4060c40 msgr2=0x7f94d409b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:53.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.135+0000 7f94d9274700 1 -- 172.21.15.104:0/3689852426 shutdown_connections 2023-07-14T11:54:53.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.135+0000 7f94d9274700 1 -- 172.21.15.104:0/3689852426 wait complete. 2023-07-14T11:54:53.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94d9274700 1 Processor -- start 2023-07-14T11:54:53.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94d9274700 1 -- start start 2023-07-14T11:54:53.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94d9274700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94d40b28a0 0x7f94d415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:53.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94d9274700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94d40b38b0 0x7f94d41f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:53.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94d9274700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94d415f670 0x7f94d40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:53.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94d9274700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d415b080 con 0x7f94d40b38b0 2023-07-14T11:54:53.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94d9274700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d415a470 con 0x7f94d40b28a0 2023-07-14T11:54:53.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94d9274700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d415a750 con 0x7f94d415f670 2023-07-14T11:54:53.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94cb7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94d40b38b0 0x7f94d41f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:53.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94cb7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94d40b38b0 0x7f94d41f5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57934/0 (socket says 172.21.15.104:57934) 2023-07-14T11:54:53.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94cb7fe700 1 -- 172.21.15.104:0/1334404450 learned_addr learned my addr 172.21.15.104:0/1334404450 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:53.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94d8a73700 1 --2- 172.21.15.104:0/1334404450 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94d415f670 0x7f94d40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:53.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94d3fff700 1 --2- 172.21.15.104:0/1334404450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94d40b28a0 0x7f94d415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:53.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94cb7fe700 1 -- 172.21.15.104:0/1334404450 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94d415f670 msgr2=0x7f94d40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:53.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94cb7fe700 1 --2- 172.21.15.104:0/1334404450 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94d415f670 0x7f94d40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94cb7fe700 1 -- 172.21.15.104:0/1334404450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94d40b28a0 msgr2=0x7f94d415b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:53.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94cb7fe700 1 --2- 172.21.15.104:0/1334404450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94d40b28a0 0x7f94d415b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94cb7fe700 1 -- 172.21.15.104:0/1334404450 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc009570 con 0x7f94d40b38b0 2023-07-14T11:54:53.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94d8a73700 1 --2- 172.21.15.104:0/1334404450 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94d415f670 0x7f94d40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:53.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.139+0000 7f94cb7fe700 1 --2- 172.21.15.104:0/1334404450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94d40b38b0 0x7f94d41f5410 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f94d40d0040 tx=0x7f94bc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:53.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.143+0000 7f94d17fa700 1 -- 172.21.15.104:0/1334404450 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc004590 con 0x7f94d40b38b0 2023-07-14T11:54:53.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.143+0000 7f94d17fa700 1 -- 172.21.15.104:0/1334404450 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94bc01e070 con 0x7f94d40b38b0 2023-07-14T11:54:53.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.143+0000 7f94d17fa700 1 -- 172.21.15.104:0/1334404450 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc003900 con 0x7f94d40b38b0 2023-07-14T11:54:53.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.143+0000 7f94d9274700 1 -- 172.21.15.104:0/1334404450 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d415aa30 con 0x7f94d40b38b0 2023-07-14T11:54:53.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.143+0000 7f94d9274700 1 -- 172.21.15.104:0/1334404450 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d415a090 con 0x7f94d40b38b0 2023-07-14T11:54:53.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.143+0000 7f94d9274700 1 -- 172.21.15.104:0/1334404450 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d415b1f0 con 0x7f94d40b38b0 2023-07-14T11:54:53.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.143+0000 7f94d17fa700 1 -- 172.21.15.104:0/1334404450 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f94bc003a60 con 0x7f94d40b38b0 2023-07-14T11:54:53.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.147+0000 7f94d17fa700 1 --2- 172.21.15.104:0/1334404450 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94b40662f0 0x7f94b40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:53.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.147+0000 7f94d17fa700 1 -- 172.21.15.104:0/1334404450 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f94bc0847d0 con 0x7f94d40b38b0 2023-07-14T11:54:53.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.147+0000 7f94d3fff700 1 --2- 172.21.15.104:0/1334404450 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94b40662f0 0x7f94b40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:53.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.147+0000 7f94d3fff700 1 --2- 172.21.15.104:0/1334404450 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94b40662f0 0x7f94b40687a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f94c40073f0 tx=0x7f94c400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:53.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.151+0000 7f94d17fa700 1 -- 172.21.15.104:0/1334404450 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f94bc051ef0 con 0x7f94d40b38b0 2023-07-14T11:54:53.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.383+0000 7f94d9274700 1 -- 172.21.15.104:0/1334404450 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f94d404e500 con 0x7f94d40b38b0 2023-07-14T11:54:53.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.395+0000 7f94d17fa700 1 -- 172.21.15.104:0/1334404450 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f94bc0200a0 con 0x7f94d40b38b0 2023-07-14T11:54:53.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.395+0000 7f94d9274700 1 -- 172.21.15.104:0/1334404450 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94b40662f0 msgr2=0x7f94b40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:53.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.395+0000 7f94d9274700 1 --2- 172.21.15.104:0/1334404450 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94b40662f0 0x7f94b40687a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f94c40073f0 tx=0x7f94c400b040 comp rx=0 tx=0).stop 2023-07-14T11:54:53.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.395+0000 7f94d9274700 1 -- 172.21.15.104:0/1334404450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94d40b38b0 msgr2=0x7f94d41f5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:53.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.395+0000 7f94d9274700 1 --2- 172.21.15.104:0/1334404450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94d40b38b0 0x7f94d41f5410 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f94d40d0040 tx=0x7f94bc004770 comp rx=0 tx=0).stop 2023-07-14T11:54:53.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.395+0000 7f94d9274700 1 -- 172.21.15.104:0/1334404450 shutdown_connections 2023-07-14T11:54:53.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.395+0000 7f94d9274700 1 --2- 172.21.15.104:0/1334404450 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94b40662f0 0x7f94b40687a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.395+0000 7f94d9274700 1 --2- 172.21.15.104:0/1334404450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94d40b38b0 0x7f94d41f5410 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.395+0000 7f94d9274700 1 --2- 172.21.15.104:0/1334404450 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94d415f670 0x7f94d40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.395+0000 7f94d9274700 1 --2- 172.21.15.104:0/1334404450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94d40b28a0 0x7f94d415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.395+0000 7f94d9274700 1 -- 172.21.15.104:0/1334404450 >> 172.21.15.104:0/1334404450 conn(0x7f94d4060c40 msgr2=0x7f94d40cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:53.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.395+0000 7f94d9274700 1 -- 172.21.15.104:0/1334404450 shutdown_connections 2023-07-14T11:54:53.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.395+0000 7f94d9274700 1 -- 172.21.15.104:0/1334404450 wait complete. 2023-07-14T11:54:53.403 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-07-14T11:54:53.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.591+0000 7ffb76240700 1 Processor -- start 2023-07-14T11:54:53.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.591+0000 7ffb76240700 1 -- start start 2023-07-14T11:54:53.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.591+0000 7ffb76240700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ffb70159f20 0x7ffb700caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:53.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.591+0000 7ffb76240700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb700b0ef0 con 0x7ffb700caf70 2023-07-14T11:54:53.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.591+0000 7ffb76240700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb700ce9c0 con 0x7ffb700c9d90 2023-07-14T11:54:53.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.591+0000 7ffb76240700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb700cee50 con 0x7ffb70159f20 2023-07-14T11:54:53.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb74a3d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ffb70159f20 0x7ffb700caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:53.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb74a3d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ffb70159f20 0x7ffb700caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57938/0 (socket says 172.21.15.104:57938) 2023-07-14T11:54:53.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb74a3d700 1 -- 172.21.15.104:0/3377493621 learned_addr learned my addr 172.21.15.104:0/3377493621 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:53.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb75a3f700 1 --1- 172.21.15.104:0/3377493621 >> v1:172.21.15.104:6789/0 conn(0x7ffb700caf70 0x7ffb7015d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56668/0 (socket says 172.21.15.104:56668) 2023-07-14T11:54:53.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb74a3d700 1 -- 172.21.15.104:0/3377493621 >> v1:172.21.15.121:6789/0 conn(0x7ffb700c9d90 legacy=0x7ffb700ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:53.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb74a3d700 1 -- 172.21.15.104:0/3377493621 >> v1:172.21.15.104:6789/0 conn(0x7ffb700caf70 legacy=0x7ffb7015d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:53.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb74a3d700 1 -- 172.21.15.104:0/3377493621 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb700cb4f0 con 0x7ffb70159f20 2023-07-14T11:54:53.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb74a3d700 1 --2- 172.21.15.104:0/3377493621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb70159f20 0x7ffb700caa30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009910 tx=0x7ffb60009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f214e78aac8a97fc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:53.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb67fff700 1 -- 172.21.15.104:0/3377493621 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000ba60 con 0x7ffb70159f20 2023-07-14T11:54:53.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb67fff700 1 -- 172.21.15.104:0/3377493621 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6000bbc0 con 0x7ffb70159f20 2023-07-14T11:54:53.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb67fff700 1 -- 172.21.15.104:0/3377493621 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60009e60 con 0x7ffb70159f20 2023-07-14T11:54:53.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb76240700 1 -- 172.21.15.104:0/3377493621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb70159f20 msgr2=0x7ffb700caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:53.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb76240700 1 --2- 172.21.15.104:0/3377493621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb70159f20 0x7ffb700caa30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009910 tx=0x7ffb60009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:53.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb76240700 1 -- 172.21.15.104:0/3377493621 shutdown_connections 2023-07-14T11:54:53.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb76240700 1 --2- 172.21.15.104:0/3377493621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb70159f20 0x7ffb700caa30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb76240700 1 -- 172.21.15.104:0/3377493621 >> 172.21.15.104:0/3377493621 conn(0x7ffb7006e960 msgr2=0x7ffb7006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:53.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb76240700 1 -- 172.21.15.104:0/3377493621 shutdown_connections 2023-07-14T11:54:53.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.595+0000 7ffb76240700 1 -- 172.21.15.104:0/3377493621 wait complete. 2023-07-14T11:54:53.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.599+0000 7ffb76240700 1 Processor -- start 2023-07-14T11:54:53.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.599+0000 7ffb76240700 1 -- start start 2023-07-14T11:54:53.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.599+0000 7ffb76240700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb700c9d90 0x7ffb700c0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:53.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.599+0000 7ffb76240700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb700caf70 0x7ffb700c1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:53.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.599+0000 7ffb76240700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb70159f20 0x7ffb700aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:53.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.599+0000 7ffb76240700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700c4560 con 0x7ffb700caf70 2023-07-14T11:54:53.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.599+0000 7ffb76240700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700c3c30 con 0x7ffb700c9d90 2023-07-14T11:54:53.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.599+0000 7ffb76240700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700c3f10 con 0x7ffb70159f20 2023-07-14T11:54:53.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.599+0000 7ffb74a3d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb700caf70 0x7ffb700c1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:53.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.599+0000 7ffb74a3d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb700caf70 0x7ffb700c1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57942/0 (socket says 172.21.15.104:57942) 2023-07-14T11:54:53.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.603+0000 7ffb74a3d700 1 -- 172.21.15.104:0/3045426425 learned_addr learned my addr 172.21.15.104:0/3045426425 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:53.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.603+0000 7ffb7523e700 1 --2- 172.21.15.104:0/3045426425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb700c9d90 0x7ffb700c0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:53.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.603+0000 7ffb75a3f700 1 --2- 172.21.15.104:0/3045426425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb70159f20 0x7ffb700aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:53.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.603+0000 7ffb74a3d700 1 -- 172.21.15.104:0/3045426425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb70159f20 msgr2=0x7ffb700aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:53.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.603+0000 7ffb74a3d700 1 --2- 172.21.15.104:0/3045426425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb70159f20 0x7ffb700aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.603+0000 7ffb74a3d700 1 -- 172.21.15.104:0/3045426425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb700c9d90 msgr2=0x7ffb700c0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:53.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.603+0000 7ffb74a3d700 1 --2- 172.21.15.104:0/3045426425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb700c9d90 0x7ffb700c0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.603+0000 7ffb74a3d700 1 -- 172.21.15.104:0/3045426425 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60009570 con 0x7ffb700caf70 2023-07-14T11:54:53.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.603+0000 7ffb74a3d700 1 --2- 172.21.15.104:0/3045426425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb700caf70 0x7ffb700c1b90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ffb60000c00 tx=0x7ffb60004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:53.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.603+0000 7ffb65ffb700 1 -- 172.21.15.104:0/3045426425 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb600036c0 con 0x7ffb700caf70 2023-07-14T11:54:53.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.603+0000 7ffb65ffb700 1 -- 172.21.15.104:0/3045426425 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6001f070 con 0x7ffb700caf70 2023-07-14T11:54:53.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.603+0000 7ffb76240700 1 -- 172.21.15.104:0/3045426425 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb700c41f0 con 0x7ffb700caf70 2023-07-14T11:54:53.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.603+0000 7ffb65ffb700 1 -- 172.21.15.104:0/3045426425 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60003a80 con 0x7ffb700caf70 2023-07-14T11:54:53.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.603+0000 7ffb76240700 1 -- 172.21.15.104:0/3045426425 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb700c20d0 con 0x7ffb700caf70 2023-07-14T11:54:53.591 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.603+0000 7ffb76240700 1 -- 172.21.15.104:0/3045426425 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb700c4380 con 0x7ffb700caf70 2023-07-14T11:54:53.591 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.607+0000 7ffb65ffb700 1 -- 172.21.15.104:0/3045426425 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ffb60003be0 con 0x7ffb700caf70 2023-07-14T11:54:53.592 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.607+0000 7ffb65ffb700 1 --2- 172.21.15.104:0/3045426425 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffb540662f0 0x7ffb540687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:53.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.607+0000 7ffb65ffb700 1 -- 172.21.15.104:0/3045426425 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7ffb60021070 con 0x7ffb700caf70 2023-07-14T11:54:53.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.607+0000 7ffb7523e700 1 --2- 172.21.15.104:0/3045426425 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffb540662f0 0x7ffb540687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:53.594 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.607+0000 7ffb7523e700 1 --2- 172.21.15.104:0/3045426425 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffb540662f0 0x7ffb540687a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffb701f49e0 tx=0x7ffb6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:53.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.611+0000 7ffb65ffb700 1 -- 172.21.15.104:0/3045426425 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ffb60052bf0 con 0x7ffb700caf70 2023-07-14T11:54:53.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.843+0000 7ffb76240700 1 -- 172.21.15.104:0/3045426425 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7ffb700c1500 con 0x7ffb700caf70 2023-07-14T11:54:53.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.855+0000 7ffb65ffb700 1 -- 172.21.15.104:0/3045426425 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffb60056210 con 0x7ffb700caf70 2023-07-14T11:54:53.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.859+0000 7ffb76240700 1 -- 172.21.15.104:0/3045426425 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffb540662f0 msgr2=0x7ffb540687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:53.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.859+0000 7ffb76240700 1 --2- 172.21.15.104:0/3045426425 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffb540662f0 0x7ffb540687a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffb701f49e0 tx=0x7ffb6c009040 comp rx=0 tx=0).stop 2023-07-14T11:54:53.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.859+0000 7ffb76240700 1 -- 172.21.15.104:0/3045426425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb700caf70 msgr2=0x7ffb700c1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:53.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.859+0000 7ffb76240700 1 --2- 172.21.15.104:0/3045426425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb700caf70 0x7ffb700c1b90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ffb60000c00 tx=0x7ffb60004770 comp rx=0 tx=0).stop 2023-07-14T11:54:53.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.859+0000 7ffb76240700 1 -- 172.21.15.104:0/3045426425 shutdown_connections 2023-07-14T11:54:53.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.859+0000 7ffb76240700 1 --2- 172.21.15.104:0/3045426425 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffb540662f0 0x7ffb540687a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.859+0000 7ffb76240700 1 --2- 172.21.15.104:0/3045426425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb700caf70 0x7ffb700c1b90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.859+0000 7ffb76240700 1 --2- 172.21.15.104:0/3045426425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb70159f20 0x7ffb700aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.859+0000 7ffb76240700 1 --2- 172.21.15.104:0/3045426425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb700c9d90 0x7ffb700c0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:53.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.859+0000 7ffb76240700 1 -- 172.21.15.104:0/3045426425 >> 172.21.15.104:0/3045426425 conn(0x7ffb7006e960 msgr2=0x7ffb700adb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:53.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.859+0000 7ffb76240700 1 -- 172.21.15.104:0/3045426425 shutdown_connections 2023-07-14T11:54:53.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:53.859+0000 7ffb76240700 1 -- 172.21.15.104:0/3045426425 wait complete. 2023-07-14T11:54:53.866 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-07-14T11:54:54.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.051+0000 7f8cb189f700 1 Processor -- start 2023-07-14T11:54:54.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.051+0000 7f8cb189f700 1 -- start start 2023-07-14T11:54:54.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb189f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8cac15ffa0 0x7f8cac06d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb189f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8cac06dc50 0x7f8cac0caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb189f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cac160370 0x7f8cac0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb189f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac0ca8a0 con 0x7f8cac15ffa0 2023-07-14T11:54:54.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb189f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac0cbcd0 con 0x7f8cac06dc50 2023-07-14T11:54:54.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb189f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0cbf70 con 0x7f8cac160370 2023-07-14T11:54:54.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb089d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8cac15ffa0 0x7f8cac06d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb089d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8cac15ffa0 0x7f8cac06d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57956/0 (socket says 172.21.15.104:57956) 2023-07-14T11:54:54.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb089d700 1 -- 172.21.15.104:0/3389982487 learned_addr learned my addr 172.21.15.104:0/3389982487 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:54.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb109e700 1 --2- 172.21.15.104:0/3389982487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cac160370 0x7f8cac0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8ca3fff700 1 --2- 172.21.15.104:0/3389982487 >> v2:172.21.15.121:3300/0 conn(0x7f8cac06dc50 0x7f8cac0caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb089d700 1 -- 172.21.15.104:0/3389982487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cac160370 msgr2=0x7f8cac0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:54.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb089d700 1 --2- 172.21.15.104:0/3389982487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cac160370 0x7f8cac0cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb089d700 1 -- 172.21.15.104:0/3389982487 >> v2:172.21.15.121:3300/0 conn(0x7f8cac06dc50 msgr2=0x7f8cac0caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:54.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb089d700 1 --2- 172.21.15.104:0/3389982487 >> v2:172.21.15.121:3300/0 conn(0x7f8cac06dc50 0x7f8cac0caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb089d700 1 -- 172.21.15.104:0/3389982487 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac0cc250 con 0x7f8cac15ffa0 2023-07-14T11:54:54.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8cb089d700 1 --2- 172.21.15.104:0/3389982487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cac15ffa0 0x7f8cac06d710 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8cac06a600 tx=0x7f8ca8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d81b9db29ced67 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:54.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8ca37fe700 1 -- 172.21.15.104:0/3389982487 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca800b8a0 con 0x7f8cac15ffa0 2023-07-14T11:54:54.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8ca37fe700 1 -- 172.21.15.104:0/3389982487 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca800ba00 con 0x7f8cac15ffa0 2023-07-14T11:54:54.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.055+0000 7f8ca37fe700 1 -- 172.21.15.104:0/3389982487 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca800bd90 con 0x7f8cac15ffa0 2023-07-14T11:54:54.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.059+0000 7f8cb189f700 1 -- 172.21.15.104:0/3389982487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cac15ffa0 msgr2=0x7f8cac06d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:54.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.059+0000 7f8cb189f700 1 --2- 172.21.15.104:0/3389982487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cac15ffa0 0x7f8cac06d710 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8cac06a600 tx=0x7f8ca8009bf0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.059+0000 7f8cb189f700 1 -- 172.21.15.104:0/3389982487 shutdown_connections 2023-07-14T11:54:54.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.059+0000 7f8cb189f700 1 --2- 172.21.15.104:0/3389982487 >> v2:172.21.15.121:3300/0 conn(0x7f8cac06dc50 0x7f8cac0caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.059+0000 7f8cb189f700 1 --2- 172.21.15.104:0/3389982487 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cac160370 0x7f8cac0cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.059+0000 7f8cb189f700 1 --2- 172.21.15.104:0/3389982487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cac15ffa0 0x7f8cac06d710 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.059+0000 7f8cb189f700 1 -- 172.21.15.104:0/3389982487 >> 172.21.15.104:0/3389982487 conn(0x7f8cac06e960 msgr2=0x7f8cac06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:54.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.059+0000 7f8cb189f700 1 -- 172.21.15.104:0/3389982487 shutdown_connections 2023-07-14T11:54:54.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.059+0000 7f8cb189f700 1 -- 172.21.15.104:0/3389982487 wait complete. 2023-07-14T11:54:54.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.059+0000 7f8cb189f700 1 Processor -- start 2023-07-14T11:54:54.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.059+0000 7f8cb189f700 1 -- start start 2023-07-14T11:54:54.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb189f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cac06dc50 0x7f8cac1629d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb189f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cac15ffa0 0x7f8cac155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb189f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cac160370 0x7f8cac0c2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb189f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac163000 con 0x7f8cac160370 2023-07-14T11:54:54.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb189f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0c20a0 con 0x7f8cac15ffa0 2023-07-14T11:54:54.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb189f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0c1ae0 con 0x7f8cac06dc50 2023-07-14T11:54:54.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb109e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cac160370 0x7f8cac0c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb109e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cac160370 0x7f8cac0c2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57960/0 (socket says 172.21.15.104:57960) 2023-07-14T11:54:54.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb109e700 1 -- 172.21.15.104:0/1037484354 learned_addr learned my addr 172.21.15.104:0/1037484354 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:54.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb089d700 1 --2- 172.21.15.104:0/1037484354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cac06dc50 0x7f8cac1629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8ca3fff700 1 --2- 172.21.15.104:0/1037484354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cac15ffa0 0x7f8cac155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb109e700 1 -- 172.21.15.104:0/1037484354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cac06dc50 msgr2=0x7f8cac1629d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:54.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb109e700 1 --2- 172.21.15.104:0/1037484354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cac06dc50 0x7f8cac1629d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb109e700 1 -- 172.21.15.104:0/1037484354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cac15ffa0 msgr2=0x7f8cac155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:54.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb109e700 1 --2- 172.21.15.104:0/1037484354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cac15ffa0 0x7f8cac155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb109e700 1 -- 172.21.15.104:0/1037484354 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca8009570 con 0x7f8cac160370 2023-07-14T11:54:54.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8ca3fff700 1 --2- 172.21.15.104:0/1037484354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cac15ffa0 0x7f8cac155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:54.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb109e700 1 --2- 172.21.15.104:0/1037484354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cac160370 0x7f8cac0c2360 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4007e40 tx=0x7f8ca400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:54.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8ca17fa700 1 -- 172.21.15.104:0/1037484354 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca400ede0 con 0x7f8cac160370 2023-07-14T11:54:54.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8ca17fa700 1 -- 172.21.15.104:0/1037484354 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca4014420 con 0x7f8cac160370 2023-07-14T11:54:54.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb189f700 1 -- 172.21.15.104:0/1037484354 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cac0c1dc0 con 0x7f8cac160370 2023-07-14T11:54:54.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8ca17fa700 1 -- 172.21.15.104:0/1037484354 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40147d0 con 0x7f8cac160370 2023-07-14T11:54:54.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.063+0000 7f8cb189f700 1 -- 172.21.15.104:0/1037484354 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac0c28a0 con 0x7f8cac160370 2023-07-14T11:54:54.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.067+0000 7f8cb189f700 1 -- 172.21.15.104:0/1037484354 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cac162000 con 0x7f8cac160370 2023-07-14T11:54:54.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.067+0000 7f8ca17fa700 1 -- 172.21.15.104:0/1037484354 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8ca4014930 con 0x7f8cac160370 2023-07-14T11:54:54.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.067+0000 7f8ca17fa700 1 --2- 172.21.15.104:0/1037484354 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8c980662f0 0x7f8c980687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.067+0000 7f8ca17fa700 1 -- 172.21.15.104:0/1037484354 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f8ca4084100 con 0x7f8cac160370 2023-07-14T11:54:54.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.071+0000 7f8cb089d700 1 --2- 172.21.15.104:0/1037484354 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8c980662f0 0x7f8c980687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.071+0000 7f8cb089d700 1 --2- 172.21.15.104:0/1037484354 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8c980662f0 0x7f8c980687a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8000c00 tx=0x7f8ca8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:54.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.075+0000 7f8ca17fa700 1 -- 172.21.15.104:0/1037484354 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f8cac162000 con 0x7f8cac160370 2023-07-14T11:54:54.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.307+0000 7f8cb189f700 1 -- 172.21.15.104:0/1037484354 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f8cac162000 con 0x7f8cac160370 2023-07-14T11:54:54.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.319+0000 7f8ca17fa700 1 -- 172.21.15.104:0/1037484354 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8cac162000 con 0x7f8cac160370 2023-07-14T11:54:54.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.319+0000 7f8cb189f700 1 -- 172.21.15.104:0/1037484354 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8c980662f0 msgr2=0x7f8c980687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:54.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.319+0000 7f8cb189f700 1 --2- 172.21.15.104:0/1037484354 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8c980662f0 0x7f8c980687a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8000c00 tx=0x7f8ca8008030 comp rx=0 tx=0).stop 2023-07-14T11:54:54.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.319+0000 7f8cb189f700 1 -- 172.21.15.104:0/1037484354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cac160370 msgr2=0x7f8cac0c2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:54.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.319+0000 7f8cb189f700 1 --2- 172.21.15.104:0/1037484354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cac160370 0x7f8cac0c2360 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4007e40 tx=0x7f8ca400e3c0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.319+0000 7f8cb189f700 1 -- 172.21.15.104:0/1037484354 shutdown_connections 2023-07-14T11:54:54.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.319+0000 7f8cb189f700 1 --2- 172.21.15.104:0/1037484354 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8c980662f0 0x7f8c980687a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.319+0000 7f8cb189f700 1 --2- 172.21.15.104:0/1037484354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cac160370 0x7f8cac0c2360 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.319+0000 7f8cb189f700 1 --2- 172.21.15.104:0/1037484354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cac15ffa0 0x7f8cac155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.319+0000 7f8cb189f700 1 --2- 172.21.15.104:0/1037484354 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cac06dc50 0x7f8cac1629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.319+0000 7f8cb189f700 1 -- 172.21.15.104:0/1037484354 >> 172.21.15.104:0/1037484354 conn(0x7f8cac06e960 msgr2=0x7f8cac15f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:54.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.319+0000 7f8cb189f700 1 -- 172.21.15.104:0/1037484354 shutdown_connections 2023-07-14T11:54:54.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.319+0000 7f8cb189f700 1 -- 172.21.15.104:0/1037484354 wait complete. 2023-07-14T11:54:54.328 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:54.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.511+0000 7f1b8ad03700 1 Processor -- start 2023-07-14T11:54:54.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.515+0000 7f1b8ad03700 1 -- start start 2023-07-14T11:54:54.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.515+0000 7f1b8ad03700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8415ffa0 0x7f1b8406d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.515+0000 7f1b8ad03700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1b8406dc50 0x7f1b840caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.515+0000 7f1b8ad03700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b840ca8a0 con 0x7f1b84160370 2023-07-14T11:54:54.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.515+0000 7f1b8ad03700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b840cbcd0 con 0x7f1b8406dc50 2023-07-14T11:54:54.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.515+0000 7f1b8ad03700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b840cbf70 con 0x7f1b8415ffa0 2023-07-14T11:54:54.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.519+0000 7f1b89d01700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8415ffa0 0x7f1b8406d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.519+0000 7f1b89d01700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8415ffa0 0x7f1b8406d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:42394/0 (socket says 172.21.15.104:42394) 2023-07-14T11:54:54.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.519+0000 7f1b89d01700 1 -- 172.21.15.104:0/379429993 learned_addr learned my addr 172.21.15.104:0/379429993 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:54.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.519+0000 7f1b89500700 1 --2- 172.21.15.104:0/379429993 >> v2:172.21.15.121:3300/0 conn(0x7f1b8406dc50 0x7f1b840caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.519+0000 7f1b89d01700 1 -- 172.21.15.104:0/379429993 >> v2:172.21.15.121:3300/0 conn(0x7f1b8406dc50 msgr2=0x7f1b840caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:54.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.519+0000 7f1b89d01700 1 --2- 172.21.15.104:0/379429993 >> v2:172.21.15.121:3300/0 conn(0x7f1b8406dc50 0x7f1b840caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.519+0000 7f1b89d01700 1 -- 172.21.15.104:0/379429993 >> v1:172.21.15.121:6789/0 conn(0x7f1b84160370 legacy=0x7f1b840ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:54.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.519+0000 7f1b89d01700 1 -- 172.21.15.104:0/379429993 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b840cc250 con 0x7f1b8415ffa0 2023-07-14T11:54:54.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.519+0000 7f1b89d01700 1 --2- 172.21.15.104:0/379429993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8415ffa0 0x7f1b8406d710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009e50 tx=0x7f1b8000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96abeaec86eb53f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:54.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.519+0000 7f1b88cff700 1 -- 172.21.15.104:0/379429993 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8000e070 con 0x7f1b8415ffa0 2023-07-14T11:54:54.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.519+0000 7f1b88cff700 1 -- 172.21.15.104:0/379429993 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b80003a40 con 0x7f1b8415ffa0 2023-07-14T11:54:54.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.519+0000 7f1b88cff700 1 -- 172.21.15.104:0/379429993 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80003df0 con 0x7f1b8415ffa0 2023-07-14T11:54:54.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.519+0000 7f1b8ad03700 1 -- 172.21.15.104:0/379429993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8415ffa0 msgr2=0x7f1b8406d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:54.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.519+0000 7f1b8ad03700 1 --2- 172.21.15.104:0/379429993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8415ffa0 0x7f1b8406d710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009e50 tx=0x7f1b8000b330 comp rx=0 tx=0).stop 2023-07-14T11:54:54.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.523+0000 7f1b8ad03700 1 -- 172.21.15.104:0/379429993 shutdown_connections 2023-07-14T11:54:54.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.523+0000 7f1b8ad03700 1 --2- 172.21.15.104:0/379429993 >> v2:172.21.15.121:3300/0 conn(0x7f1b8406dc50 0x7f1b840caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.523+0000 7f1b8ad03700 1 --2- 172.21.15.104:0/379429993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8415ffa0 0x7f1b8406d710 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.523+0000 7f1b8ad03700 1 -- 172.21.15.104:0/379429993 >> 172.21.15.104:0/379429993 conn(0x7f1b8406e960 msgr2=0x7f1b8406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:54.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.523+0000 7f1b8ad03700 1 -- 172.21.15.104:0/379429993 shutdown_connections 2023-07-14T11:54:54.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.523+0000 7f1b8ad03700 1 -- 172.21.15.104:0/379429993 wait complete. 2023-07-14T11:54:54.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.523+0000 7f1b8ad03700 1 Processor -- start 2023-07-14T11:54:54.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.523+0000 7f1b8ad03700 1 -- start start 2023-07-14T11:54:54.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8ad03700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b8406dc50 0x7f1b841f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8ad03700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8415ffa0 0x7f1b84163620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8ad03700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b84160370 0x7f1b840c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8ad03700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841f5c70 con 0x7f1b84160370 2023-07-14T11:54:54.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8ad03700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b840c2560 con 0x7f1b8406dc50 2023-07-14T11:54:54.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8ad03700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b840c2840 con 0x7f1b8415ffa0 2023-07-14T11:54:54.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8a502700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b84160370 0x7f1b840c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8a502700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b84160370 0x7f1b840c4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57964/0 (socket says 172.21.15.104:57964) 2023-07-14T11:54:54.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8a502700 1 -- 172.21.15.104:0/1240369954 learned_addr learned my addr 172.21.15.104:0/1240369954 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:54.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b89500700 1 --2- 172.21.15.104:0/1240369954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8415ffa0 0x7f1b84163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b89d01700 1 --2- 172.21.15.104:0/1240369954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b8406dc50 0x7f1b841f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8a502700 1 -- 172.21.15.104:0/1240369954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8415ffa0 msgr2=0x7f1b84163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:54.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8a502700 1 --2- 172.21.15.104:0/1240369954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8415ffa0 0x7f1b84163620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8a502700 1 -- 172.21.15.104:0/1240369954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b8406dc50 msgr2=0x7f1b841f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:54.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8a502700 1 --2- 172.21.15.104:0/1240369954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b8406dc50 0x7f1b841f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8a502700 1 -- 172.21.15.104:0/1240369954 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b80009b30 con 0x7f1b84160370 2023-07-14T11:54:54.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b89d01700 1 --2- 172.21.15.104:0/1240369954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b8406dc50 0x7f1b841f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:54.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8a502700 1 --2- 172.21.15.104:0/1240369954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b84160370 0x7f1b840c4410 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00c7c0 tx=0x7f1b7c00cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:54.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b7a7fc700 1 -- 172.21.15.104:0/1240369954 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c005480 con 0x7f1b84160370 2023-07-14T11:54:54.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b7a7fc700 1 -- 172.21.15.104:0/1240369954 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7c0036a0 con 0x7f1b84160370 2023-07-14T11:54:54.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8ad03700 1 -- 172.21.15.104:0/1240369954 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b841f5900 con 0x7f1b84160370 2023-07-14T11:54:54.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b7a7fc700 1 -- 172.21.15.104:0/1240369954 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c003a50 con 0x7f1b84160370 2023-07-14T11:54:54.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.527+0000 7f1b8ad03700 1 -- 172.21.15.104:0/1240369954 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b841f55b0 con 0x7f1b84160370 2023-07-14T11:54:54.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.531+0000 7f1b8ad03700 1 -- 172.21.15.104:0/1240369954 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b841f5de0 con 0x7f1b84160370 2023-07-14T11:54:54.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.531+0000 7f1b7a7fc700 1 -- 172.21.15.104:0/1240369954 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1b7c003bb0 con 0x7f1b84160370 2023-07-14T11:54:54.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.531+0000 7f1b7a7fc700 1 --2- 172.21.15.104:0/1240369954 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1b68066320 0x7f1b680687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.535+0000 7f1b7a7fc700 1 -- 172.21.15.104:0/1240369954 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f1b7c082a40 con 0x7f1b84160370 2023-07-14T11:54:54.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.535+0000 7f1b89d01700 1 --2- 172.21.15.104:0/1240369954 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1b68066320 0x7f1b680687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.535+0000 7f1b89d01700 1 --2- 172.21.15.104:0/1240369954 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1b68066320 0x7f1b680687d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000f80 tx=0x7f1b80009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:54.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.539+0000 7f1b7a7fc700 1 -- 172.21.15.104:0/1240369954 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f1b7c084e60 con 0x7f1b84160370 2023-07-14T11:54:54.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.707+0000 7f1b8ad03700 1 -- 172.21.15.104:0/1240369954 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b840ca8a0 con 0x7f1b84160370 2023-07-14T11:54:54.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.707+0000 7f1b7a7fc700 1 -- 172.21.15.104:0/1240369954 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+14880 (secure 0 0 0) 0x7f1b840ca8a0 con 0x7f1b84160370 2023-07-14T11:54:54.694 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:54.695 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":38,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:54:51.100906+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:54.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.711+0000 7f1b8ad03700 1 -- 172.21.15.104:0/1240369954 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1b68066320 msgr2=0x7f1b680687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:54.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.711+0000 7f1b8ad03700 1 --2- 172.21.15.104:0/1240369954 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1b68066320 0x7f1b680687d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000f80 tx=0x7f1b80009000 comp rx=0 tx=0).stop 2023-07-14T11:54:54.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.711+0000 7f1b8ad03700 1 -- 172.21.15.104:0/1240369954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b84160370 msgr2=0x7f1b840c4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:54.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.711+0000 7f1b8ad03700 1 --2- 172.21.15.104:0/1240369954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b84160370 0x7f1b840c4410 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00c7c0 tx=0x7f1b7c00cad0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.711+0000 7f1b8ad03700 1 -- 172.21.15.104:0/1240369954 shutdown_connections 2023-07-14T11:54:54.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.711+0000 7f1b8ad03700 1 --2- 172.21.15.104:0/1240369954 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1b68066320 0x7f1b680687d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.711+0000 7f1b8ad03700 1 --2- 172.21.15.104:0/1240369954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b84160370 0x7f1b840c4410 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.711+0000 7f1b8ad03700 1 --2- 172.21.15.104:0/1240369954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b8415ffa0 0x7f1b84163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.711+0000 7f1b8ad03700 1 --2- 172.21.15.104:0/1240369954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b8406dc50 0x7f1b841f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.711+0000 7f1b8ad03700 1 -- 172.21.15.104:0/1240369954 >> 172.21.15.104:0/1240369954 conn(0x7f1b8406e960 msgr2=0x7f1b840afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:54.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.711+0000 7f1b8ad03700 1 -- 172.21.15.104:0/1240369954 shutdown_connections 2023-07-14T11:54:54.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.711+0000 7f1b8ad03700 1 -- 172.21.15.104:0/1240369954 wait complete. 2023-07-14T11:54:54.718 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:54:54.719 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:54:54.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.911+0000 7fb1a66a6700 1 Processor -- start 2023-07-14T11:54:54.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.911+0000 7fb1a66a6700 1 -- start start 2023-07-14T11:54:54.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a66a6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb1a01604f0 0x7fb1a00af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a66a6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb1a00af6f0 0x7fb1a00afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a66a6700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a00d0930 con 0x7fb1a015fa10 2023-07-14T11:54:54.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a66a6700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a00b4ae0 con 0x7fb1a00af6f0 2023-07-14T11:54:54.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a66a6700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a00ce560 con 0x7fb1a01604f0 2023-07-14T11:54:54.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a5ea5700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb1a00af6f0 0x7fb1a00afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a5ea5700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb1a00af6f0 0x7fb1a00afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57976/0 (socket says 172.21.15.104:57976) 2023-07-14T11:54:54.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a5ea5700 1 -- 172.21.15.104:0/2285339213 learned_addr learned my addr 172.21.15.104:0/2285339213 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:54.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a56a4700 1 --1- 172.21.15.104:0/2285339213 >> v1:172.21.15.104:6789/0 conn(0x7fb1a015fa10 0x7fb1a015fde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56684/0 (socket says 172.21.15.104:56684) 2023-07-14T11:54:54.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a4ea3700 1 --2- 172.21.15.104:0/2285339213 >> v2:172.21.15.121:3300/0 conn(0x7fb1a01604f0 0x7fb1a00af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a5ea5700 1 -- 172.21.15.104:0/2285339213 >> v2:172.21.15.121:3300/0 conn(0x7fb1a01604f0 msgr2=0x7fb1a00af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:54.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb197fff700 1 -- 172.21.15.104:0/2285339213 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3715370809 0 0) 0x7fb1a00d0930 con 0x7fb1a015fa10 2023-07-14T11:54:54.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a5ea5700 1 --2- 172.21.15.104:0/2285339213 >> v2:172.21.15.121:3300/0 conn(0x7fb1a01604f0 0x7fb1a00af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a5ea5700 1 -- 172.21.15.104:0/2285339213 >> v1:172.21.15.104:6789/0 conn(0x7fb1a015fa10 legacy=0x7fb1a015fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:54.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a5ea5700 1 -- 172.21.15.104:0/2285339213 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a015c300 con 0x7fb1a00af6f0 2023-07-14T11:54:54.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a5ea5700 1 --2- 172.21.15.104:0/2285339213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1a00af6f0 0x7fb1a00afb70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb1a008ca30 tx=0x7fb19800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26272c4b92fe1ac2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:54.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb197fff700 1 -- 172.21.15.104:0/2285339213 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19800e070 con 0x7fb1a00af6f0 2023-07-14T11:54:54.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb197fff700 1 -- 172.21.15.104:0/2285339213 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19800b910 con 0x7fb1a00af6f0 2023-07-14T11:54:54.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb197fff700 1 -- 172.21.15.104:0/2285339213 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19800bcc0 con 0x7fb1a00af6f0 2023-07-14T11:54:54.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a66a6700 1 -- 172.21.15.104:0/2285339213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1a00af6f0 msgr2=0x7fb1a00afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:54.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a66a6700 1 --2- 172.21.15.104:0/2285339213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1a00af6f0 0x7fb1a00afb70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb1a008ca30 tx=0x7fb19800b330 comp rx=0 tx=0).stop 2023-07-14T11:54:54.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a66a6700 1 -- 172.21.15.104:0/2285339213 shutdown_connections 2023-07-14T11:54:54.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a66a6700 1 --2- 172.21.15.104:0/2285339213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1a00af6f0 0x7fb1a00afb70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a66a6700 1 --2- 172.21.15.104:0/2285339213 >> v2:172.21.15.121:3300/0 conn(0x7fb1a01604f0 0x7fb1a00af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.915+0000 7fb1a66a6700 1 -- 172.21.15.104:0/2285339213 >> 172.21.15.104:0/2285339213 conn(0x7fb1a009b850 msgr2=0x7fb1a00d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:54.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.919+0000 7fb1a66a6700 1 -- 172.21.15.104:0/2285339213 shutdown_connections 2023-07-14T11:54:54.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.919+0000 7fb1a66a6700 1 -- 172.21.15.104:0/2285339213 wait complete. 2023-07-14T11:54:54.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.919+0000 7fb1a66a6700 1 Processor -- start 2023-07-14T11:54:54.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.919+0000 7fb1a66a6700 1 -- start start 2023-07-14T11:54:54.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.919+0000 7fb1a66a6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1a00af6f0 0x7fb1a0156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.919+0000 7fb1a66a6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1a015fa10 0x7fb1a00c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.919+0000 7fb1a66a6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1a01604f0 0x7fb1a00c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.919+0000 7fb1a66a6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a00b5e90 con 0x7fb1a015fa10 2023-07-14T11:54:54.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.919+0000 7fb1a66a6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0157350 con 0x7fb1a01604f0 2023-07-14T11:54:54.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.919+0000 7fb1a66a6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0157660 con 0x7fb1a00af6f0 2023-07-14T11:54:54.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.919+0000 7fb1a4ea3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1a015fa10 0x7fb1a00c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.919+0000 7fb1a4ea3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1a015fa10 0x7fb1a00c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57980/0 (socket says 172.21.15.104:57980) 2023-07-14T11:54:54.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.919+0000 7fb1a4ea3700 1 -- 172.21.15.104:0/2355773898 learned_addr learned my addr 172.21.15.104:0/2355773898 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:54.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.919+0000 7fb1a56a4700 1 --2- 172.21.15.104:0/2355773898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1a00af6f0 0x7fb1a0156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.923+0000 7fb1a4ea3700 1 -- 172.21.15.104:0/2355773898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1a00af6f0 msgr2=0x7fb1a0156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:54.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.923+0000 7fb1a4ea3700 1 --2- 172.21.15.104:0/2355773898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1a00af6f0 0x7fb1a0156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.923+0000 7fb1a4ea3700 1 -- 172.21.15.104:0/2355773898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1a01604f0 msgr2=0x7fb1a00c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:54.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.923+0000 7fb1a5ea5700 1 --2- 172.21.15.104:0/2355773898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1a01604f0 0x7fb1a00c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.923+0000 7fb1a4ea3700 1 --2- 172.21.15.104:0/2355773898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1a01604f0 0x7fb1a00c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:54.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.923+0000 7fb1a4ea3700 1 -- 172.21.15.104:0/2355773898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb198009b30 con 0x7fb1a015fa10 2023-07-14T11:54:54.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.923+0000 7fb1a5ea5700 1 --2- 172.21.15.104:0/2355773898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1a01604f0 0x7fb1a00c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:54.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.923+0000 7fb1a4ea3700 1 --2- 172.21.15.104:0/2355773898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1a015fa10 0x7fb1a00c6020 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb19000e9d0 tx=0x7fb19000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:54.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.923+0000 7fb195ffb700 1 -- 172.21.15.104:0/2355773898 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190009720 con 0x7fb1a015fa10 2023-07-14T11:54:54.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.923+0000 7fb195ffb700 1 -- 172.21.15.104:0/2355773898 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19000c850 con 0x7fb1a015fa10 2023-07-14T11:54:54.911 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.923+0000 7fb1a66a6700 1 -- 172.21.15.104:0/2355773898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a00c8560 con 0x7fb1a015fa10 2023-07-14T11:54:54.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.923+0000 7fb195ffb700 1 -- 172.21.15.104:0/2355773898 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19000cbc0 con 0x7fb1a015fa10 2023-07-14T11:54:54.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.923+0000 7fb1a66a6700 1 -- 172.21.15.104:0/2355773898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a00c8b20 con 0x7fb1a015fa10 2023-07-14T11:54:54.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.923+0000 7fb1a66a6700 1 -- 172.21.15.104:0/2355773898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a00d3430 con 0x7fb1a015fa10 2023-07-14T11:54:54.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.927+0000 7fb195ffb700 1 -- 172.21.15.104:0/2355773898 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb19000cd20 con 0x7fb1a015fa10 2023-07-14T11:54:54.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.927+0000 7fb195ffb700 1 --2- 172.21.15.104:0/2355773898 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1840662f0 0x7fb1840687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:54.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.927+0000 7fb195ffb700 1 -- 172.21.15.104:0/2355773898 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fb190083c70 con 0x7fb1a015fa10 2023-07-14T11:54:54.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.927+0000 7fb1a56a4700 1 --2- 172.21.15.104:0/2355773898 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1840662f0 0x7fb1840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:54.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.927+0000 7fb1a56a4700 1 --2- 172.21.15.104:0/2355773898 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1840662f0 0x7fb1840687a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb19c001020 tx=0x7fb19c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:54.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:54.935+0000 7fb195ffb700 1 -- 172.21.15.104:0/2355773898 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb190054d80 con 0x7fb1a015fa10 2023-07-14T11:54:55.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.107+0000 7fb1a66a6700 1 -- 172.21.15.104:0/2355773898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1a004e500 con 0x7fb1a015fa10 2023-07-14T11:54:55.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.107+0000 7fb195ffb700 1 -- 172.21.15.104:0/2355773898 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1900170a0 con 0x7fb1a015fa10 2023-07-14T11:54:55.092 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:54:55.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.111+0000 7fb1a66a6700 1 -- 172.21.15.104:0/2355773898 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1840662f0 msgr2=0x7fb1840687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:55.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.111+0000 7fb1a66a6700 1 --2- 172.21.15.104:0/2355773898 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1840662f0 0x7fb1840687a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb19c001020 tx=0x7fb19c009040 comp rx=0 tx=0).stop 2023-07-14T11:54:55.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.111+0000 7fb1a66a6700 1 -- 172.21.15.104:0/2355773898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1a015fa10 msgr2=0x7fb1a00c6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:55.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.111+0000 7fb1a66a6700 1 --2- 172.21.15.104:0/2355773898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1a015fa10 0x7fb1a00c6020 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb19000e9d0 tx=0x7fb19000ed90 comp rx=0 tx=0).stop 2023-07-14T11:54:55.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.111+0000 7fb1a66a6700 1 -- 172.21.15.104:0/2355773898 shutdown_connections 2023-07-14T11:54:55.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.111+0000 7fb1a66a6700 1 --2- 172.21.15.104:0/2355773898 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1840662f0 0x7fb1840687a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.111+0000 7fb1a66a6700 1 --2- 172.21.15.104:0/2355773898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1a015fa10 0x7fb1a00c6020 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.111+0000 7fb1a66a6700 1 --2- 172.21.15.104:0/2355773898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1a01604f0 0x7fb1a00c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.111+0000 7fb1a66a6700 1 --2- 172.21.15.104:0/2355773898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1a00af6f0 0x7fb1a0156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.111+0000 7fb1a66a6700 1 -- 172.21.15.104:0/2355773898 >> 172.21.15.104:0/2355773898 conn(0x7fb1a009b850 msgr2=0x7fb1a00d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:55.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.111+0000 7fb1a66a6700 1 -- 172.21.15.104:0/2355773898 shutdown_connections 2023-07-14T11:54:55.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.111+0000 7fb1a66a6700 1 -- 172.21.15.104:0/2355773898 wait complete. 2023-07-14T11:54:55.118 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:54:55.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.303+0000 7fe642b30700 1 Processor -- start 2023-07-14T11:54:55.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.303+0000 7fe642b30700 1 -- start start 2023-07-14T11:54:55.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe642b30700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe63c160000 0x7fe63c06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:55.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe642b30700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe63c1603d0 0x7fe63c0c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:55.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe642b30700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe63c0cd7f0 0x7fe63c0ca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:55.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe642b30700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe63c070e40 con 0x7fe63c0cd7f0 2023-07-14T11:54:55.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe642b30700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe63c0cdfb0 con 0x7fe63c160000 2023-07-14T11:54:55.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe642b30700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c0ca8c0 con 0x7fe63c1603d0 2023-07-14T11:54:55.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe64232f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe63c0cd7f0 0x7fe63c0ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:55.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe64232f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe63c0cd7f0 0x7fe63c0ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57996/0 (socket says 172.21.15.104:57996) 2023-07-14T11:54:55.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe64232f700 1 -- 172.21.15.104:0/837046080 learned_addr learned my addr 172.21.15.104:0/837046080 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:55.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe641b2e700 1 --2- 172.21.15.104:0/837046080 >> v2:172.21.15.121:3300/0 conn(0x7fe63c160000 0x7fe63c06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:55.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe64132d700 1 --2- 172.21.15.104:0/837046080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe63c1603d0 0x7fe63c0c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:55.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe64232f700 1 -- 172.21.15.104:0/837046080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe63c1603d0 msgr2=0x7fe63c0c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:55.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe64232f700 1 --2- 172.21.15.104:0/837046080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe63c1603d0 0x7fe63c0c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe64232f700 1 -- 172.21.15.104:0/837046080 >> v2:172.21.15.121:3300/0 conn(0x7fe63c160000 msgr2=0x7fe63c06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:55.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe64232f700 1 --2- 172.21.15.104:0/837046080 >> v2:172.21.15.121:3300/0 conn(0x7fe63c160000 0x7fe63c06dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe64232f700 1 -- 172.21.15.104:0/837046080 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c0cdcd0 con 0x7fe63c0cd7f0 2023-07-14T11:54:55.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe64232f700 1 --2- 172.21.15.104:0/837046080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe63c0cd7f0 0x7fe63c0ca380 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe63c06b3b0 tx=0x7fe63400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1934d5c3d2a9722a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:55.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe640b2c700 1 -- 172.21.15.104:0/837046080 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63400e070 con 0x7fe63c0cd7f0 2023-07-14T11:54:55.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe640b2c700 1 -- 172.21.15.104:0/837046080 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63400b9c0 con 0x7fe63c0cd7f0 2023-07-14T11:54:55.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.307+0000 7fe640b2c700 1 -- 172.21.15.104:0/837046080 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63400bd70 con 0x7fe63c0cd7f0 2023-07-14T11:54:55.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.311+0000 7fe642b30700 1 -- 172.21.15.104:0/837046080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe63c0cd7f0 msgr2=0x7fe63c0ca380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:55.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.311+0000 7fe642b30700 1 --2- 172.21.15.104:0/837046080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe63c0cd7f0 0x7fe63c0ca380 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe63c06b3b0 tx=0x7fe63400b330 comp rx=0 tx=0).stop 2023-07-14T11:54:55.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.311+0000 7fe642b30700 1 -- 172.21.15.104:0/837046080 shutdown_connections 2023-07-14T11:54:55.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.311+0000 7fe642b30700 1 --2- 172.21.15.104:0/837046080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe63c0cd7f0 0x7fe63c0ca380 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.311+0000 7fe642b30700 1 --2- 172.21.15.104:0/837046080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe63c1603d0 0x7fe63c0c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.311+0000 7fe642b30700 1 --2- 172.21.15.104:0/837046080 >> v2:172.21.15.121:3300/0 conn(0x7fe63c160000 0x7fe63c06dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.311+0000 7fe642b30700 1 -- 172.21.15.104:0/837046080 >> 172.21.15.104:0/837046080 conn(0x7fe63c0afc60 msgr2=0x7fe63c0b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:55.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.311+0000 7fe642b30700 1 -- 172.21.15.104:0/837046080 shutdown_connections 2023-07-14T11:54:55.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.311+0000 7fe642b30700 1 -- 172.21.15.104:0/837046080 wait complete. 2023-07-14T11:54:55.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe642b30700 1 Processor -- start 2023-07-14T11:54:55.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe642b30700 1 -- start start 2023-07-14T11:54:55.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe642b30700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe63c0cd7f0 0x7fe63c0af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:55.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe642b30700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe63c160000 0x7fe63c15de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:55.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe642b30700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe63c1603d0 0x7fe63c15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:55.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe642b30700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c1612d0 con 0x7fe63c160000 2023-07-14T11:54:55.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe642b30700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c0c89f0 con 0x7fe63c1603d0 2023-07-14T11:54:55.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe642b30700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c161dc0 con 0x7fe63c0cd7f0 2023-07-14T11:54:55.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe64132d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe63c160000 0x7fe63c15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:55.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe64132d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe63c160000 0x7fe63c15de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57998/0 (socket says 172.21.15.104:57998) 2023-07-14T11:54:55.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe64132d700 1 -- 172.21.15.104:0/1366679479 learned_addr learned my addr 172.21.15.104:0/1366679479 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:55.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe641b2e700 1 --2- 172.21.15.104:0/1366679479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe63c0cd7f0 0x7fe63c0af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:55.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe64232f700 1 --2- 172.21.15.104:0/1366679479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe63c1603d0 0x7fe63c15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:55.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe64132d700 1 -- 172.21.15.104:0/1366679479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe63c0cd7f0 msgr2=0x7fe63c0af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:55.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe64132d700 1 --2- 172.21.15.104:0/1366679479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe63c0cd7f0 0x7fe63c0af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe64132d700 1 -- 172.21.15.104:0/1366679479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe63c1603d0 msgr2=0x7fe63c15e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:55.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe64132d700 1 --2- 172.21.15.104:0/1366679479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe63c1603d0 0x7fe63c15e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe64132d700 1 -- 172.21.15.104:0/1366679479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe634009b30 con 0x7fe63c160000 2023-07-14T11:54:55.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe641b2e700 1 --2- 172.21.15.104:0/1366679479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe63c0cd7f0 0x7fe63c0af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:55.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe64132d700 1 --2- 172.21.15.104:0/1366679479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe63c160000 0x7fe63c15de70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00e810 tx=0x7fe62c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:55.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe6327fc700 1 -- 172.21.15.104:0/1366679479 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c013610 con 0x7fe63c160000 2023-07-14T11:54:55.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe6327fc700 1 -- 172.21.15.104:0/1366679479 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62c013770 con 0x7fe63c160000 2023-07-14T11:54:55.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe642b30700 1 -- 172.21.15.104:0/1366679479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63c161520 con 0x7fe63c160000 2023-07-14T11:54:55.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe6327fc700 1 -- 172.21.15.104:0/1366679479 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c013b20 con 0x7fe63c160000 2023-07-14T11:54:55.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.315+0000 7fe642b30700 1 -- 172.21.15.104:0/1366679479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63c161ae0 con 0x7fe63c160000 2023-07-14T11:54:55.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.319+0000 7fe642b30700 1 -- 172.21.15.104:0/1366679479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63c04e500 con 0x7fe63c160000 2023-07-14T11:54:55.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.319+0000 7fe6327fc700 1 -- 172.21.15.104:0/1366679479 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe62c013c80 con 0x7fe63c160000 2023-07-14T11:54:55.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.319+0000 7fe6327fc700 1 --2- 172.21.15.104:0/1366679479 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6200662f0 0x7fe6200687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:55.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.323+0000 7fe6327fc700 1 -- 172.21.15.104:0/1366679479 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fe62c0839b0 con 0x7fe63c160000 2023-07-14T11:54:55.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.323+0000 7fe641b2e700 1 --2- 172.21.15.104:0/1366679479 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6200662f0 0x7fe6200687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:55.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.323+0000 7fe641b2e700 1 --2- 172.21.15.104:0/1366679479 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6200662f0 0x7fe6200687a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe638009a00 tx=0x7fe638008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:55.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.327+0000 7fe6327fc700 1 -- 172.21.15.104:0/1366679479 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe62c050f20 con 0x7fe63c160000 2023-07-14T11:54:55.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.499+0000 7fe642b30700 1 -- 172.21.15.104:0/1366679479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe63c0b40d0 con 0x7fe63c160000 2023-07-14T11:54:55.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.499+0000 7fe6327fc700 1 -- 172.21.15.104:0/1366679479 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v38) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe62c054540 con 0x7fe63c160000 2023-07-14T11:54:55.485 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:54:55.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.503+0000 7fe642b30700 1 -- 172.21.15.104:0/1366679479 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6200662f0 msgr2=0x7fe6200687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:55.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.503+0000 7fe642b30700 1 --2- 172.21.15.104:0/1366679479 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6200662f0 0x7fe6200687a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe638009a00 tx=0x7fe638008040 comp rx=0 tx=0).stop 2023-07-14T11:54:55.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.503+0000 7fe642b30700 1 -- 172.21.15.104:0/1366679479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe63c160000 msgr2=0x7fe63c15de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:55.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.503+0000 7fe642b30700 1 --2- 172.21.15.104:0/1366679479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe63c160000 0x7fe63c15de70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00e810 tx=0x7fe62c00ebd0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.503+0000 7fe642b30700 1 -- 172.21.15.104:0/1366679479 shutdown_connections 2023-07-14T11:54:55.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.503+0000 7fe642b30700 1 --2- 172.21.15.104:0/1366679479 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6200662f0 0x7fe6200687a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.503+0000 7fe642b30700 1 --2- 172.21.15.104:0/1366679479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe63c160000 0x7fe63c15de70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.503+0000 7fe642b30700 1 --2- 172.21.15.104:0/1366679479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe63c1603d0 0x7fe63c15e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.503+0000 7fe642b30700 1 --2- 172.21.15.104:0/1366679479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe63c0cd7f0 0x7fe63c0af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.503+0000 7fe642b30700 1 -- 172.21.15.104:0/1366679479 >> 172.21.15.104:0/1366679479 conn(0x7fe63c0afc60 msgr2=0x7fe63c0c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:55.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.503+0000 7fe642b30700 1 -- 172.21.15.104:0/1366679479 shutdown_connections 2023-07-14T11:54:55.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.503+0000 7fe642b30700 1 -- 172.21.15.104:0/1366679479 wait complete. 2023-07-14T11:54:55.511 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-14T11:54:55.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.695+0000 7fb6841a0700 1 Processor -- start 2023-07-14T11:54:55.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.695+0000 7fb6841a0700 1 -- start start 2023-07-14T11:54:55.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb6841a0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67c0afd30 0x7fb67c0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:55.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb6841a0700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c0c8900 con 0x7fb67c160640 2023-07-14T11:54:55.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb6841a0700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c0caae0 con 0x7fb67c0cddb0 2023-07-14T11:54:55.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb6841a0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0c8fc0 con 0x7fb67c0afd30 2023-07-14T11:54:55.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb681f3c700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb67c160640 0x7fb67c164000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56698/0 (socket says 172.21.15.104:56698) 2023-07-14T11:54:55.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb681f3c700 1 -- 172.21.15.104:0/1290185409 learned_addr learned my addr 172.21.15.104:0/1290185409 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:55.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb68273d700 1 --2- 172.21.15.104:0/1290185409 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67c0afd30 0x7fb67c0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:55.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb680f3a700 1 -- 172.21.15.104:0/1290185409 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207899208 0 0) 0x7fb67c0c8900 con 0x7fb67c160640 2023-07-14T11:54:55.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb680f3a700 1 -- 172.21.15.104:0/1290185409 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6680035d0 con 0x7fb67c160640 2023-07-14T11:54:55.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb680f3a700 1 -- 172.21.15.104:0/1290185409 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166824458 0 0) 0x7fb67c0caae0 con 0x7fb67c0cddb0 2023-07-14T11:54:55.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb680f3a700 1 -- 172.21.15.104:0/1290185409 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb67c0c8900 con 0x7fb67c0cddb0 2023-07-14T11:54:55.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb68273d700 1 -- 172.21.15.104:0/1290185409 >> v1:172.21.15.121:6789/0 conn(0x7fb67c0cddb0 legacy=0x7fb67c070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:55.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb68273d700 1 -- 172.21.15.104:0/1290185409 >> v1:172.21.15.104:6789/0 conn(0x7fb67c160640 legacy=0x7fb67c164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:55.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb68273d700 1 -- 172.21.15.104:0/1290185409 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c0ca520 con 0x7fb67c0afd30 2023-07-14T11:54:55.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb68273d700 1 --2- 172.21.15.104:0/1290185409 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67c0afd30 0x7fb67c0b01b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb674009dc0 tx=0x7fb67400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6516257e0d98005b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:55.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb680f3a700 1 -- 172.21.15.104:0/1290185409 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67400e070 con 0x7fb67c0afd30 2023-07-14T11:54:55.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb680f3a700 1 -- 172.21.15.104:0/1290185409 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67400bb20 con 0x7fb67c0afd30 2023-07-14T11:54:55.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.699+0000 7fb680f3a700 1 -- 172.21.15.104:0/1290185409 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67400be90 con 0x7fb67c0afd30 2023-07-14T11:54:55.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.703+0000 7fb6841a0700 1 -- 172.21.15.104:0/1290185409 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67c0afd30 msgr2=0x7fb67c0b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:55.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.703+0000 7fb6841a0700 1 --2- 172.21.15.104:0/1290185409 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67c0afd30 0x7fb67c0b01b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb674009dc0 tx=0x7fb67400b330 comp rx=0 tx=0).stop 2023-07-14T11:54:55.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.703+0000 7fb6841a0700 1 -- 172.21.15.104:0/1290185409 shutdown_connections 2023-07-14T11:54:55.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.703+0000 7fb6841a0700 1 --2- 172.21.15.104:0/1290185409 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67c0afd30 0x7fb67c0b01b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.703+0000 7fb6841a0700 1 -- 172.21.15.104:0/1290185409 >> 172.21.15.104:0/1290185409 conn(0x7fb67c0cad30 msgr2=0x7fb67c0cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:55.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.703+0000 7fb6841a0700 1 -- 172.21.15.104:0/1290185409 shutdown_connections 2023-07-14T11:54:55.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.703+0000 7fb6841a0700 1 -- 172.21.15.104:0/1290185409 wait complete. 2023-07-14T11:54:55.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb6841a0700 1 Processor -- start 2023-07-14T11:54:55.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb6841a0700 1 -- start start 2023-07-14T11:54:55.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb6841a0700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb67c0afd30 0x7fb67c155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:55.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb6841a0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb67c0cddb0 0x7fb67c162fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:55.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb6841a0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67c160640 0x7fb67c1563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:55.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb6841a0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c070540 con 0x7fb67c0cddb0 2023-07-14T11:54:55.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb6841a0700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c161f60 con 0x7fb67c0afd30 2023-07-14T11:54:55.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb6841a0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c162200 con 0x7fb67c160640 2023-07-14T11:54:55.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb68173b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb67c0cddb0 0x7fb67c162fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:55.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb681f3c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb67c0afd30 0x7fb67c155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:55.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb68173b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb67c0cddb0 0x7fb67c162fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:58000/0 (socket says 172.21.15.104:58000) 2023-07-14T11:54:55.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb68173b700 1 -- 172.21.15.104:0/2385663398 learned_addr learned my addr 172.21.15.104:0/2385663398 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:55.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb68273d700 1 --2- 172.21.15.104:0/2385663398 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67c160640 0x7fb67c1563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:55.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb68173b700 1 -- 172.21.15.104:0/2385663398 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67c160640 msgr2=0x7fb67c1563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:55.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb68173b700 1 --2- 172.21.15.104:0/2385663398 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67c160640 0x7fb67c1563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb68173b700 1 -- 172.21.15.104:0/2385663398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb67c0afd30 msgr2=0x7fb67c155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:55.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb68173b700 1 --2- 172.21.15.104:0/2385663398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb67c0afd30 0x7fb67c155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb68173b700 1 -- 172.21.15.104:0/2385663398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb674009a70 con 0x7fb67c0cddb0 2023-07-14T11:54:55.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.707+0000 7fb68173b700 1 --2- 172.21.15.104:0/2385663398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb67c0cddb0 0x7fb67c162fc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00c500 tx=0x7fb66c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:55.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.711+0000 7fb6727fc700 1 -- 172.21.15.104:0/2385663398 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c0052d0 con 0x7fb67c0cddb0 2023-07-14T11:54:55.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.711+0000 7fb6727fc700 1 -- 172.21.15.104:0/2385663398 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66c005450 con 0x7fb67c0cddb0 2023-07-14T11:54:55.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.711+0000 7fb6727fc700 1 -- 172.21.15.104:0/2385663398 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c0038e0 con 0x7fb67c0cddb0 2023-07-14T11:54:55.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.711+0000 7fb6841a0700 1 -- 172.21.15.104:0/2385663398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c1624e0 con 0x7fb67c0cddb0 2023-07-14T11:54:55.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.711+0000 7fb6841a0700 1 -- 172.21.15.104:0/2385663398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c162aa0 con 0x7fb67c0cddb0 2023-07-14T11:54:55.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.711+0000 7fb6841a0700 1 -- 172.21.15.104:0/2385663398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67c04dcb0 con 0x7fb67c0cddb0 2023-07-14T11:54:55.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.711+0000 7fb6727fc700 1 -- 172.21.15.104:0/2385663398 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb66c003a40 con 0x7fb67c0cddb0 2023-07-14T11:54:55.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.715+0000 7fb6727fc700 1 --2- 172.21.15.104:0/2385663398 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb6600662f0 0x7fb6600687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:55.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.715+0000 7fb6727fc700 1 -- 172.21.15.104:0/2385663398 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fb66c082f20 con 0x7fb67c0cddb0 2023-07-14T11:54:55.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.715+0000 7fb681f3c700 1 --2- 172.21.15.104:0/2385663398 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb6600662f0 0x7fb6600687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:55.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.715+0000 7fb681f3c700 1 --2- 172.21.15.104:0/2385663398 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb6600662f0 0x7fb6600687a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb678000f60 tx=0x7fb678007e20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:55.706 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.719+0000 7fb6727fc700 1 -- 172.21.15.104:0/2385663398 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb66c0505f0 con 0x7fb67c0cddb0 2023-07-14T11:54:55.879 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.895+0000 7fb6841a0700 1 -- 172.21.15.104:0/2385663398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb67c0cb9f0 con 0x7fb67c0cddb0 2023-07-14T11:54:55.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.895+0000 7fb6727fc700 1 -- 172.21.15.104:0/2385663398 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v38) v1 ==== 84+0+126 (secure 0 0 0) 0x7fb66c053c10 con 0x7fb67c0cddb0 2023-07-14T11:54:55.880 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:55.880 INFO:teuthology.orchestra.run.smithi104.stdout:[ 2023-07-14T11:54:55.881 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:55.881 INFO:teuthology.orchestra.run.smithi104.stdout: "poolnum": 1, 2023-07-14T11:54:55.881 INFO:teuthology.orchestra.run.smithi104.stdout: "poolname": ".mgr" 2023-07-14T11:54:55.881 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:54:55.881 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:54:55.881 INFO:teuthology.orchestra.run.smithi104.stdout: "poolnum": 2, 2023-07-14T11:54:55.881 INFO:teuthology.orchestra.run.smithi104.stdout: "poolname": "rbd" 2023-07-14T11:54:55.882 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:54:55.882 INFO:teuthology.orchestra.run.smithi104.stdout:] 2023-07-14T11:54:55.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.899+0000 7fb6841a0700 1 -- 172.21.15.104:0/2385663398 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb6600662f0 msgr2=0x7fb6600687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:55.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.899+0000 7fb6841a0700 1 --2- 172.21.15.104:0/2385663398 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb6600662f0 0x7fb6600687a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb678000f60 tx=0x7fb678007e20 comp rx=0 tx=0).stop 2023-07-14T11:54:55.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.899+0000 7fb6841a0700 1 -- 172.21.15.104:0/2385663398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb67c0cddb0 msgr2=0x7fb67c162fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:55.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.899+0000 7fb6841a0700 1 --2- 172.21.15.104:0/2385663398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb67c0cddb0 0x7fb67c162fc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00c500 tx=0x7fb66c00c810 comp rx=0 tx=0).stop 2023-07-14T11:54:55.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.899+0000 7fb6841a0700 1 -- 172.21.15.104:0/2385663398 shutdown_connections 2023-07-14T11:54:55.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.899+0000 7fb6841a0700 1 --2- 172.21.15.104:0/2385663398 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb6600662f0 0x7fb6600687a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.899+0000 7fb6841a0700 1 --2- 172.21.15.104:0/2385663398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb67c0cddb0 0x7fb67c162fc0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.899+0000 7fb6841a0700 1 --2- 172.21.15.104:0/2385663398 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67c160640 0x7fb67c1563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.899+0000 7fb6841a0700 1 --2- 172.21.15.104:0/2385663398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb67c0afd30 0x7fb67c155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:55.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.899+0000 7fb6841a0700 1 -- 172.21.15.104:0/2385663398 >> 172.21.15.104:0/2385663398 conn(0x7fb67c0cad30 msgr2=0x7fb67c0add10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:55.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.899+0000 7fb6841a0700 1 -- 172.21.15.104:0/2385663398 shutdown_connections 2023-07-14T11:54:55.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:55.899+0000 7fb6841a0700 1 -- 172.21.15.104:0/2385663398 wait complete. 2023-07-14T11:54:55.904 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-14T11:54:55.904 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-14T11:54:56.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.099+0000 7f8b8359e700 1 Processor -- start 2023-07-14T11:54:56.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.099+0000 7f8b8359e700 1 -- start start 2023-07-14T11:54:56.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.103+0000 7f8b8359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b840b0ee0 0x7f8b840ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:56.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.103+0000 7f8b8359e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b840cd4e0 con 0x7f8b840d07e0 2023-07-14T11:54:56.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.103+0000 7f8b8359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b840d1a60 con 0x7f8b840cd090 2023-07-14T11:54:56.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.103+0000 7f8b8359e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b840d0c60 con 0x7f8b840b0ee0 2023-07-14T11:54:56.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.103+0000 7f8b81d9b700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f8b840cd090 0x7f8b840d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:54822/0 (socket says 172.21.15.104:54822) 2023-07-14T11:54:56.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.103+0000 7f8b81d9b700 1 -- 172.21.15.104:0/927063094 learned_addr learned my addr 172.21.15.104:0/927063094 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:56.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.103+0000 7f8b8259c700 1 --2- 172.21.15.104:0/927063094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b840b0ee0 0x7f8b840ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:56.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.103+0000 7f8b8159a700 1 -- 172.21.15.104:0/927063094 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677277144 0 0) 0x7f8b7c001490 con 0x7f8b840d07e0 2023-07-14T11:54:56.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.103+0000 7f8b8159a700 1 -- 172.21.15.104:0/927063094 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b6c0035d0 con 0x7f8b840d07e0 2023-07-14T11:54:56.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.103+0000 7f8b8259c700 1 -- 172.21.15.104:0/927063094 >> v1:172.21.15.121:6789/0 conn(0x7f8b840cd090 legacy=0x7f8b840d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:56.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.103+0000 7f8b8259c700 1 -- 172.21.15.104:0/927063094 >> v1:172.21.15.104:6789/0 conn(0x7f8b840d07e0 legacy=0x7f8b840af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:56.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.103+0000 7f8b8259c700 1 -- 172.21.15.104:0/927063094 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b840d1ef0 con 0x7f8b840b0ee0 2023-07-14T11:54:56.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.103+0000 7f8b8259c700 1 --2- 172.21.15.104:0/927063094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b840b0ee0 0x7f8b840ccb50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b740098b0 tx=0x7f8b74009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d1244776051a293 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:56.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.107+0000 7f8b8159a700 1 -- 172.21.15.104:0/927063094 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74011830 con 0x7f8b840b0ee0 2023-07-14T11:54:56.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.107+0000 7f8b8159a700 1 -- 172.21.15.104:0/927063094 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b74011990 con 0x7f8b840b0ee0 2023-07-14T11:54:56.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.107+0000 7f8b8159a700 1 -- 172.21.15.104:0/927063094 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74011d20 con 0x7f8b840b0ee0 2023-07-14T11:54:56.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.107+0000 7f8b8359e700 1 -- 172.21.15.104:0/927063094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b840b0ee0 msgr2=0x7f8b840ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:56.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.107+0000 7f8b8359e700 1 --2- 172.21.15.104:0/927063094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b840b0ee0 0x7f8b840ccb50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b740098b0 tx=0x7f8b74009bc0 comp rx=0 tx=0).stop 2023-07-14T11:54:56.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.107+0000 7f8b8359e700 1 -- 172.21.15.104:0/927063094 shutdown_connections 2023-07-14T11:54:56.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.107+0000 7f8b8359e700 1 --2- 172.21.15.104:0/927063094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b840b0ee0 0x7f8b840ccb50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:56.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.107+0000 7f8b8359e700 1 -- 172.21.15.104:0/927063094 >> 172.21.15.104:0/927063094 conn(0x7f8b84060c40 msgr2=0x7f8b8409b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:56.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.107+0000 7f8b8359e700 1 -- 172.21.15.104:0/927063094 shutdown_connections 2023-07-14T11:54:56.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.107+0000 7f8b8359e700 1 -- 172.21.15.104:0/927063094 wait complete. 2023-07-14T11:54:56.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.111+0000 7f8b8359e700 1 Processor -- start 2023-07-14T11:54:56.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.111+0000 7f8b8359e700 1 -- start start 2023-07-14T11:54:56.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.111+0000 7f8b8359e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b840b0ee0 0x7f8b8415f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:56.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.111+0000 7f8b8359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b840cd090 0x7f8b8415fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:56.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.111+0000 7f8b8359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b840d07e0 0x7f8b840c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:56.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.111+0000 7f8b8359e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8415d3c0 con 0x7f8b840b0ee0 2023-07-14T11:54:56.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.111+0000 7f8b8359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b840b3580 con 0x7f8b840d07e0 2023-07-14T11:54:56.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.111+0000 7f8b8359e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b840b3860 con 0x7f8b840cd090 2023-07-14T11:54:56.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b8259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b840b0ee0 0x7f8b8415f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:56.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b8259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b840b0ee0 0x7f8b8415f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:58012/0 (socket says 172.21.15.104:58012) 2023-07-14T11:54:56.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b8259c700 1 -- 172.21.15.104:0/2702650839 learned_addr learned my addr 172.21.15.104:0/2702650839 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:56.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b81d9b700 1 --2- 172.21.15.104:0/2702650839 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b840cd090 0x7f8b8415fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:56.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b82d9d700 1 --2- 172.21.15.104:0/2702650839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b840d07e0 0x7f8b840c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:56.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b8259c700 1 -- 172.21.15.104:0/2702650839 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b840cd090 msgr2=0x7f8b8415fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:56.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b8259c700 1 --2- 172.21.15.104:0/2702650839 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b840cd090 0x7f8b8415fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:56.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b8259c700 1 -- 172.21.15.104:0/2702650839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b840d07e0 msgr2=0x7f8b840c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:56.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b8259c700 1 --2- 172.21.15.104:0/2702650839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b840d07e0 0x7f8b840c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:56.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b8259c700 1 -- 172.21.15.104:0/2702650839 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b740094b0 con 0x7f8b840b0ee0 2023-07-14T11:54:56.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b82d9d700 1 --2- 172.21.15.104:0/2702650839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b840d07e0 0x7f8b840c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:56.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b8259c700 1 --2- 172.21.15.104:0/2702650839 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b840b0ee0 0x7f8b8415f7e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8b740090b0 tx=0x7f8b74004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:56.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b72ffd700 1 -- 172.21.15.104:0/2702650839 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b740044c0 con 0x7f8b840b0ee0 2023-07-14T11:54:56.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b72ffd700 1 -- 172.21.15.104:0/2702650839 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7401f070 con 0x7f8b840b0ee0 2023-07-14T11:54:56.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b8359e700 1 -- 172.21.15.104:0/2702650839 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b840b3b40 con 0x7f8b840b0ee0 2023-07-14T11:54:56.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b72ffd700 1 -- 172.21.15.104:0/2702650839 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b740038e0 con 0x7f8b840b0ee0 2023-07-14T11:54:56.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b8359e700 1 -- 172.21.15.104:0/2702650839 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b840c9840 con 0x7f8b840b0ee0 2023-07-14T11:54:56.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.115+0000 7f8b8359e700 1 -- 172.21.15.104:0/2702650839 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8404e500 con 0x7f8b840b0ee0 2023-07-14T11:54:56.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.119+0000 7f8b72ffd700 1 -- 172.21.15.104:0/2702650839 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8b74021030 con 0x7f8b840b0ee0 2023-07-14T11:54:56.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.119+0000 7f8b72ffd700 1 --2- 172.21.15.104:0/2702650839 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8b64066320 0x7f8b640687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:56.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.119+0000 7f8b72ffd700 1 -- 172.21.15.104:0/2702650839 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f8b74084e70 con 0x7f8b840b0ee0 2023-07-14T11:54:56.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.119+0000 7f8b81d9b700 1 --2- 172.21.15.104:0/2702650839 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8b64066320 0x7f8b640687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:56.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.123+0000 7f8b81d9b700 1 --2- 172.21.15.104:0/2702650839 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8b64066320 0x7f8b640687d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8b840a0410 tx=0x7f8b78009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:56.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.127+0000 7f8b72ffd700 1 -- 172.21.15.104:0/2702650839 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f8b740526b0 con 0x7f8b840b0ee0 2023-07-14T11:54:56.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:56.299+0000 7f8b8359e700 1 -- 172.21.15.104:0/2702650839 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8b840cba00 con 0x7f8b840b0ee0 2023-07-14T11:54:57.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.131+0000 7f8b72ffd700 1 -- 172.21.15.104:0/2702650839 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v39) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8b74055cd0 con 0x7f8b840b0ee0 2023-07-14T11:54:57.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.139+0000 7f8b8359e700 1 -- 172.21.15.104:0/2702650839 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8b64066320 msgr2=0x7f8b640687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:57.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.139+0000 7f8b8359e700 1 --2- 172.21.15.104:0/2702650839 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8b64066320 0x7f8b640687d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8b840a0410 tx=0x7f8b78009040 comp rx=0 tx=0).stop 2023-07-14T11:54:57.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.139+0000 7f8b8359e700 1 -- 172.21.15.104:0/2702650839 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b840b0ee0 msgr2=0x7f8b8415f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:57.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.139+0000 7f8b8359e700 1 --2- 172.21.15.104:0/2702650839 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b840b0ee0 0x7f8b8415f7e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8b740090b0 tx=0x7f8b74004770 comp rx=0 tx=0).stop 2023-07-14T11:54:57.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.139+0000 7f8b8359e700 1 -- 172.21.15.104:0/2702650839 shutdown_connections 2023-07-14T11:54:57.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.139+0000 7f8b8359e700 1 --2- 172.21.15.104:0/2702650839 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8b64066320 0x7f8b640687d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:57.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.139+0000 7f8b8359e700 1 --2- 172.21.15.104:0/2702650839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b840d07e0 0x7f8b840c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:57.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.139+0000 7f8b8359e700 1 --2- 172.21.15.104:0/2702650839 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b840cd090 0x7f8b8415fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:57.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.139+0000 7f8b8359e700 1 --2- 172.21.15.104:0/2702650839 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b840b0ee0 0x7f8b8415f7e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:57.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.139+0000 7f8b8359e700 1 -- 172.21.15.104:0/2702650839 >> 172.21.15.104:0/2702650839 conn(0x7f8b84060c40 msgr2=0x7f8b8415ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:57.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.139+0000 7f8b8359e700 1 -- 172.21.15.104:0/2702650839 shutdown_connections 2023-07-14T11:54:57.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.139+0000 7f8b8359e700 1 -- 172.21.15.104:0/2702650839 wait complete. 2023-07-14T11:54:57.126 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'cephfs_metadata' created 2023-07-14T11:54:57.145 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-14T11:54:57.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.343+0000 7f053cf03700 1 Processor -- start 2023-07-14T11:54:57.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.343+0000 7f053cf03700 1 -- start start 2023-07-14T11:54:57.331 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f053cf03700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f053815bd20 0x7f053806e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:57.331 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f053cf03700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f053806f2e0 con 0x7f053815d230 2023-07-14T11:54:57.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f053cf03700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f053815c470 con 0x7f053806eea0 2023-07-14T11:54:57.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f053cf03700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05380cf070 con 0x7f053815bd20 2023-07-14T11:54:57.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f0536ffd700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f053815d230 0x7f05380ce2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56722/0 (socket says 172.21.15.104:56722) 2023-07-14T11:54:57.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f0536ffd700 1 -- 172.21.15.104:0/3003409868 learned_addr learned my addr 172.21.15.104:0/3003409868 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:57.333 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05377fe700 1 --2- 172.21.15.104:0/3003409868 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f053815bd20 0x7f053806e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:57.333 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05367fc700 1 -- 172.21.15.104:0/3003409868 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 109325762 0 0) 0x7f053806f2e0 con 0x7f053815d230 2023-07-14T11:54:57.333 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05367fc700 1 -- 172.21.15.104:0/3003409868 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05200035d0 con 0x7f053815d230 2023-07-14T11:54:57.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05367fc700 1 -- 172.21.15.104:0/3003409868 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 167812531 0 0) 0x7f05200035d0 con 0x7f053815d230 2023-07-14T11:54:57.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05367fc700 1 -- 172.21.15.104:0/3003409868 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f053815bd20 msgr2=0x7f053806e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:57.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05367fc700 1 --2- 172.21.15.104:0/3003409868 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f053815bd20 0x7f053806e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:57.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05367fc700 1 -- 172.21.15.104:0/3003409868 >> v1:172.21.15.121:6789/0 conn(0x7f053806eea0 legacy=0x7f053815f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:57.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05367fc700 1 -- 172.21.15.104:0/3003409868 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053815c750 con 0x7f053815d230 2023-07-14T11:54:57.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05367fc700 1 -- 172.21.15.104:0/3003409868 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f052c001670 con 0x7f053815d230 2023-07-14T11:54:57.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05367fc700 1 -- 172.21.15.104:0/3003409868 >> v1:172.21.15.104:6789/0 conn(0x7f053815d230 legacy=0x7f05380ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:57.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05367fc700 1 --2- 172.21.15.104:0/3003409868 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0520003a50 0x7f0520003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:57.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05367fc700 1 --2- 172.21.15.104:0/3003409868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05200043a0 0x7f0520004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:57.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05367fc700 1 --2- 172.21.15.104:0/3003409868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0520005070 0x7f052000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:57.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05367fc700 1 -- 172.21.15.104:0/3003409868 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05380cf070 con 0x7f0520005070 2023-07-14T11:54:57.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05367fc700 1 -- 172.21.15.104:0/3003409868 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052000bf70 con 0x7f05200043a0 2023-07-14T11:54:57.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05367fc700 1 -- 172.21.15.104:0/3003409868 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052000c130 con 0x7f0520003a50 2023-07-14T11:54:57.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f0537fff700 1 --2- 172.21.15.104:0/3003409868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0520005070 0x7f052000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:57.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f0536ffd700 1 --2- 172.21.15.104:0/3003409868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05200043a0 0x7f0520004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:57.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05377fe700 1 --2- 172.21.15.104:0/3003409868 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0520003a50 0x7f0520003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:57.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f0537fff700 1 -- 172.21.15.104:0/3003409868 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0520003a50 msgr2=0x7f0520003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:57.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f0537fff700 1 --2- 172.21.15.104:0/3003409868 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0520003a50 0x7f0520003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:57.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f0537fff700 1 -- 172.21.15.104:0/3003409868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05200043a0 msgr2=0x7f0520004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:57.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f0537fff700 1 --2- 172.21.15.104:0/3003409868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05200043a0 0x7f0520004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:57.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f0537fff700 1 -- 172.21.15.104:0/3003409868 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05200035d0 con 0x7f0520005070 2023-07-14T11:54:57.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f05377fe700 1 --2- 172.21.15.104:0/3003409868 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0520003a50 0x7f0520003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:57.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.347+0000 7f0537fff700 1 --2- 172.21.15.104:0/3003409868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0520005070 0x7f052000ba30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f053000c370 tx=0x7f05200058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60ba9f7f75dce7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:57.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.351+0000 7f05367fc700 1 -- 172.21.15.104:0/3003409868 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0530001f20 con 0x7f0520005070 2023-07-14T11:54:57.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.351+0000 7f05367fc700 1 -- 172.21.15.104:0/3003409868 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0530012420 con 0x7f0520005070 2023-07-14T11:54:57.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.351+0000 7f05367fc700 1 -- 172.21.15.104:0/3003409868 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05300127d0 con 0x7f0520005070 2023-07-14T11:54:57.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.351+0000 7f053cf03700 1 -- 172.21.15.104:0/3003409868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0520005070 msgr2=0x7f052000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:57.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.351+0000 7f053cf03700 1 --2- 172.21.15.104:0/3003409868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0520005070 0x7f052000ba30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f053000c370 tx=0x7f05200058a0 comp rx=0 tx=0).stop 2023-07-14T11:54:57.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.351+0000 7f0537fff700 1 -- 172.21.15.104:0/3003409868 reap_dead start 2023-07-14T11:54:57.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.351+0000 7f053cf03700 1 -- 172.21.15.104:0/3003409868 shutdown_connections 2023-07-14T11:54:57.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.351+0000 7f053cf03700 1 -- 172.21.15.104:0/3003409868 >> 172.21.15.104:0/3003409868 conn(0x7f053806fe60 msgr2=0x7f0538070260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:57.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.351+0000 7f053cf03700 1 -- 172.21.15.104:0/3003409868 shutdown_connections 2023-07-14T11:54:57.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.351+0000 7f053cf03700 1 -- 172.21.15.104:0/3003409868 wait complete. 2023-07-14T11:54:57.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f053cf03700 1 Processor -- start 2023-07-14T11:54:57.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f053cf03700 1 -- start start 2023-07-14T11:54:57.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f053cf03700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0520005070 0x7f053814fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:57.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f053cf03700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05380cc110 0x7f05380ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:57.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f053cf03700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05380c4fc0 0x7f05380c5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:57.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f053cf03700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053814ff90 con 0x7f05380cc110 2023-07-14T11:54:57.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f053cf03700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05380c30a0 con 0x7f0520005070 2023-07-14T11:54:57.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f053cf03700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05380c5970 con 0x7f05380c4fc0 2023-07-14T11:54:57.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f0536ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05380cc110 0x7f05380ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:57.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f0536ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05380cc110 0x7f05380ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:58042/0 (socket says 172.21.15.104:58042) 2023-07-14T11:54:57.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f0536ffd700 1 -- 172.21.15.104:0/3135863616 learned_addr learned my addr 172.21.15.104:0/3135863616 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:57.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f05377fe700 1 --2- 172.21.15.104:0/3135863616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0520005070 0x7f053814fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:57.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f0537fff700 1 --2- 172.21.15.104:0/3135863616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05380c4fc0 0x7f05380c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:57.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f0536ffd700 1 -- 172.21.15.104:0/3135863616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05380c4fc0 msgr2=0x7f05380c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:57.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f0536ffd700 1 --2- 172.21.15.104:0/3135863616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05380c4fc0 0x7f05380c5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:57.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f0536ffd700 1 -- 172.21.15.104:0/3135863616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0520005070 msgr2=0x7f053814fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:57.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f0536ffd700 1 --2- 172.21.15.104:0/3135863616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0520005070 0x7f053814fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:57.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f0536ffd700 1 -- 172.21.15.104:0/3135863616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05380caa40 con 0x7f05380cc110 2023-07-14T11:54:57.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f0537fff700 1 --2- 172.21.15.104:0/3135863616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05380c4fc0 0x7f05380c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:57.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f0536ffd700 1 --2- 172.21.15.104:0/3135863616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05380cc110 0x7f05380ca210 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f052c00eb50 tx=0x7f052c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:57.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f051ffff700 1 -- 172.21.15.104:0/3135863616 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c00c760 con 0x7f05380cc110 2023-07-14T11:54:57.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f051ffff700 1 -- 172.21.15.104:0/3135863616 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c00c8e0 con 0x7f05380cc110 2023-07-14T11:54:57.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f053cf03700 1 -- 172.21.15.104:0/3135863616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05380cad20 con 0x7f05380cc110 2023-07-14T11:54:57.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.355+0000 7f051ffff700 1 -- 172.21.15.104:0/3135863616 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c00cc90 con 0x7f05380cc110 2023-07-14T11:54:57.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.359+0000 7f053cf03700 1 -- 172.21.15.104:0/3135863616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05380c2d30 con 0x7f05380cc110 2023-07-14T11:54:57.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.359+0000 7f053cf03700 1 -- 172.21.15.104:0/3135863616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05380c2ef0 con 0x7f05380cc110 2023-07-14T11:54:57.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.359+0000 7f051ffff700 1 -- 172.21.15.104:0/3135863616 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f052c01d460 con 0x7f05380cc110 2023-07-14T11:54:57.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.363+0000 7f051ffff700 1 --2- 172.21.15.104:0/3135863616 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0524066320 0x7f05240687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:57.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.363+0000 7f051ffff700 1 -- 172.21.15.104:0/3135863616 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f052c083c30 con 0x7f05380cc110 2023-07-14T11:54:57.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.363+0000 7f05377fe700 1 --2- 172.21.15.104:0/3135863616 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0524066320 0x7f05240687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:57.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.363+0000 7f05377fe700 1 --2- 172.21.15.104:0/3135863616 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0524066320 0x7f05240687d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f05380cc800 tx=0x7f0528008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:57.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.371+0000 7f051ffff700 1 -- 172.21.15.104:0/3135863616 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f052c0510c0 con 0x7f05380cc110 2023-07-14T11:54:57.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:57.543+0000 7f053cf03700 1 -- 172.21.15.104:0/3135863616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f053804e500 con 0x7f05380cc110 2023-07-14T11:54:58.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.143+0000 7f051ffff700 1 -- 172.21.15.104:0/3135863616 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v40) v1 ==== 170+0+0 (secure 0 0 0) 0x7f052c0546e0 con 0x7f05380cc110 2023-07-14T11:54:58.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.147+0000 7f053cf03700 1 -- 172.21.15.104:0/3135863616 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0524066320 msgr2=0x7f05240687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:58.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.147+0000 7f053cf03700 1 --2- 172.21.15.104:0/3135863616 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0524066320 0x7f05240687d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f05380cc800 tx=0x7f0528008040 comp rx=0 tx=0).stop 2023-07-14T11:54:58.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.147+0000 7f053cf03700 1 -- 172.21.15.104:0/3135863616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05380cc110 msgr2=0x7f05380ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:58.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.147+0000 7f053cf03700 1 --2- 172.21.15.104:0/3135863616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05380cc110 0x7f05380ca210 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f052c00eb50 tx=0x7f052c00ef10 comp rx=0 tx=0).stop 2023-07-14T11:54:58.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.167+0000 7f053cf03700 1 -- 172.21.15.104:0/3135863616 shutdown_connections 2023-07-14T11:54:58.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.167+0000 7f053cf03700 1 --2- 172.21.15.104:0/3135863616 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0524066320 0x7f05240687d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:58.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.167+0000 7f053cf03700 1 --2- 172.21.15.104:0/3135863616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05380cc110 0x7f05380ca210 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:58.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.167+0000 7f053cf03700 1 --2- 172.21.15.104:0/3135863616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05380c4fc0 0x7f05380c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:58.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.167+0000 7f053cf03700 1 --2- 172.21.15.104:0/3135863616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0520005070 0x7f053814fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:58.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.167+0000 7f053cf03700 1 -- 172.21.15.104:0/3135863616 >> 172.21.15.104:0/3135863616 conn(0x7f053806fe60 msgr2=0x7f05380b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:58.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.167+0000 7f053cf03700 1 -- 172.21.15.104:0/3135863616 shutdown_connections 2023-07-14T11:54:58.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.167+0000 7f053cf03700 1 -- 172.21.15.104:0/3135863616 wait complete. 2023-07-14T11:54:58.155 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'cephfs_data' created 2023-07-14T11:54:58.173 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-14T11:54:58.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.363+0000 7f2f45b08700 1 Processor -- start 2023-07-14T11:54:58.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.363+0000 7f2f45b08700 1 -- start start 2023-07-14T11:54:58.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.363+0000 7f2f45b08700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f4015c200 0x7f2f400cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:58.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.363+0000 7f2f45b08700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2f400ad0f0 0x7f2f400cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:58.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.363+0000 7f2f45b08700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f400cbed0 con 0x7f2f400afd80 2023-07-14T11:54:58.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.363+0000 7f2f45b08700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f40071820 con 0x7f2f400ad0f0 2023-07-14T11:54:58.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.363+0000 7f2f45b08700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f400ad760 con 0x7f2f4015c200 2023-07-14T11:54:58.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.363+0000 7f2f3ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2f400ad0f0 0x7f2f400cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:58.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.363+0000 7f2f3ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2f400ad0f0 0x7f2f400cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:58052/0 (socket says 172.21.15.104:58052) 2023-07-14T11:54:58.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.363+0000 7f2f3ffff700 1 -- 172.21.15.104:0/2226722603 learned_addr learned my addr 172.21.15.104:0/2226722603 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:58.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.363+0000 7f2f3effd700 1 --2- 172.21.15.104:0/2226722603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f4015c200 0x7f2f400cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:58.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f3ffff700 1 -- 172.21.15.104:0/2226722603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f4015c200 msgr2=0x7f2f400cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:58.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f3ffff700 1 --2- 172.21.15.104:0/2226722603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f4015c200 0x7f2f400cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:58.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f3ffff700 1 -- 172.21.15.104:0/2226722603 >> v1:172.21.15.104:6789/0 conn(0x7f2f400afd80 legacy=0x7f2f400b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:58.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f3ffff700 1 -- 172.21.15.104:0/2226722603 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f400ada40 con 0x7f2f400ad0f0 2023-07-14T11:54:58.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f3effd700 1 --2- 172.21.15.104:0/2226722603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f4015c200 0x7f2f400cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:58.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f3ffff700 1 --2- 172.21.15.104:0/2226722603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f400ad0f0 0x7f2f400cdae0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009e50 tx=0x7f2f3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a7cb4b0789e9b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:58.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f3e7fc700 1 -- 172.21.15.104:0/2226722603 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3800e070 con 0x7f2f400ad0f0 2023-07-14T11:54:58.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f3e7fc700 1 -- 172.21.15.104:0/2226722603 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3800b9c0 con 0x7f2f400ad0f0 2023-07-14T11:54:58.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f3e7fc700 1 -- 172.21.15.104:0/2226722603 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3800bd70 con 0x7f2f400ad0f0 2023-07-14T11:54:58.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f45b08700 1 -- 172.21.15.104:0/2226722603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f400ad0f0 msgr2=0x7f2f400cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:58.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f45b08700 1 --2- 172.21.15.104:0/2226722603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f400ad0f0 0x7f2f400cdae0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009e50 tx=0x7f2f3800b330 comp rx=0 tx=0).stop 2023-07-14T11:54:58.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f45b08700 1 -- 172.21.15.104:0/2226722603 shutdown_connections 2023-07-14T11:54:58.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f45b08700 1 --2- 172.21.15.104:0/2226722603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f400ad0f0 0x7f2f400cdae0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:58.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f45b08700 1 --2- 172.21.15.104:0/2226722603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f4015c200 0x7f2f400cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:58.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f45b08700 1 -- 172.21.15.104:0/2226722603 >> 172.21.15.104:0/2226722603 conn(0x7f2f400cad80 msgr2=0x7f2f400cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:58.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f45b08700 1 -- 172.21.15.104:0/2226722603 shutdown_connections 2023-07-14T11:54:58.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.367+0000 7f2f45b08700 1 -- 172.21.15.104:0/2226722603 wait complete. 2023-07-14T11:54:58.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.371+0000 7f2f45b08700 1 Processor -- start 2023-07-14T11:54:58.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.371+0000 7f2f45b08700 1 -- start start 2023-07-14T11:54:58.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.371+0000 7f2f45b08700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f400ad0f0 0x7f2f40156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:58.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.371+0000 7f2f45b08700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f400afd80 0x7f2f400c4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:58.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.371+0000 7f2f45b08700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f4015c200 0x7f2f400aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:58.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.371+0000 7f2f45b08700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f401f4a80 con 0x7f2f400ad0f0 2023-07-14T11:54:58.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.371+0000 7f2f45b08700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f40156d60 con 0x7f2f4015c200 2023-07-14T11:54:58.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.371+0000 7f2f45b08700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f400b22b0 con 0x7f2f400afd80 2023-07-14T11:54:58.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.371+0000 7f2f3f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f400ad0f0 0x7f2f40156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:58.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.371+0000 7f2f3f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f400ad0f0 0x7f2f40156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:58066/0 (socket says 172.21.15.104:58066) 2023-07-14T11:54:58.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.371+0000 7f2f3f7fe700 1 -- 172.21.15.104:0/2049072580 learned_addr learned my addr 172.21.15.104:0/2049072580 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:58.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.371+0000 7f2f3ffff700 1 --2- 172.21.15.104:0/2049072580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f4015c200 0x7f2f400aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:58.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.371+0000 7f2f3effd700 1 --2- 172.21.15.104:0/2049072580 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f400afd80 0x7f2f400c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:58.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.375+0000 7f2f3f7fe700 1 -- 172.21.15.104:0/2049072580 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f400afd80 msgr2=0x7f2f400c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:58.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.375+0000 7f2f3f7fe700 1 --2- 172.21.15.104:0/2049072580 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f400afd80 0x7f2f400c4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:58.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.375+0000 7f2f3f7fe700 1 -- 172.21.15.104:0/2049072580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f4015c200 msgr2=0x7f2f400aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:58.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.375+0000 7f2f3f7fe700 1 --2- 172.21.15.104:0/2049072580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f4015c200 0x7f2f400aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:58.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.375+0000 7f2f3f7fe700 1 -- 172.21.15.104:0/2049072580 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f38009b30 con 0x7f2f400ad0f0 2023-07-14T11:54:58.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.375+0000 7f2f3f7fe700 1 --2- 172.21.15.104:0/2049072580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f400ad0f0 0x7f2f40156820 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000c500 tx=0x7f2f3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:58.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.375+0000 7f2f27fff700 1 -- 172.21.15.104:0/2049072580 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f300052d0 con 0x7f2f400ad0f0 2023-07-14T11:54:58.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.375+0000 7f2f27fff700 1 -- 172.21.15.104:0/2049072580 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f30005430 con 0x7f2f400ad0f0 2023-07-14T11:54:58.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.375+0000 7f2f45b08700 1 -- 172.21.15.104:0/2049072580 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f401f4c60 con 0x7f2f400ad0f0 2023-07-14T11:54:58.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.375+0000 7f2f27fff700 1 -- 172.21.15.104:0/2049072580 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f300055a0 con 0x7f2f400ad0f0 2023-07-14T11:54:58.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.375+0000 7f2f45b08700 1 -- 172.21.15.104:0/2049072580 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f401f5220 con 0x7f2f400ad0f0 2023-07-14T11:54:58.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.375+0000 7f2f45b08700 1 -- 172.21.15.104:0/2049072580 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4004dcb0 con 0x7f2f400ad0f0 2023-07-14T11:54:58.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.375+0000 7f2f27fff700 1 -- 172.21.15.104:0/2049072580 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2f300038e0 con 0x7f2f400ad0f0 2023-07-14T11:54:58.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.379+0000 7f2f27fff700 1 --2- 172.21.15.104:0/2049072580 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2f2c0662f0 0x7f2f2c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:58.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.379+0000 7f2f27fff700 1 -- 172.21.15.104:0/2049072580 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f2f30083250 con 0x7f2f400ad0f0 2023-07-14T11:54:58.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.379+0000 7f2f3effd700 1 --2- 172.21.15.104:0/2049072580 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2f2c0662f0 0x7f2f2c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:58.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.379+0000 7f2f3effd700 1 --2- 172.21.15.104:0/2049072580 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2f2c0662f0 0x7f2f2c0687a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2f400b4300 tx=0x7f2f34008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:58.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.387+0000 7f2f27fff700 1 -- 172.21.15.104:0/2049072580 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f2f30050600 con 0x7f2f400ad0f0 2023-07-14T11:54:58.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:58.595+0000 7f2f45b08700 1 -- 172.21.15.104:0/2049072580 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2f400b0bc0 con 0x7f2f400ad0f0 2023-07-14T11:54:59.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.147+0000 7f2f27fff700 1 -- 172.21.15.104:0/2049072580 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2f30053c20 con 0x7f2f400ad0f0 2023-07-14T11:54:59.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.151+0000 7f2f45b08700 1 -- 172.21.15.104:0/2049072580 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2f2c0662f0 msgr2=0x7f2f2c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:59.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.151+0000 7f2f45b08700 1 --2- 172.21.15.104:0/2049072580 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2f2c0662f0 0x7f2f2c0687a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2f400b4300 tx=0x7f2f34008040 comp rx=0 tx=0).stop 2023-07-14T11:54:59.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.151+0000 7f2f45b08700 1 -- 172.21.15.104:0/2049072580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f400ad0f0 msgr2=0x7f2f40156820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:59.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.151+0000 7f2f45b08700 1 --2- 172.21.15.104:0/2049072580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f400ad0f0 0x7f2f40156820 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000c500 tx=0x7f2f3000c8c0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.155+0000 7f2f45b08700 1 -- 172.21.15.104:0/2049072580 shutdown_connections 2023-07-14T11:54:59.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.155+0000 7f2f45b08700 1 --2- 172.21.15.104:0/2049072580 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2f2c0662f0 0x7f2f2c0687a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.155+0000 7f2f45b08700 1 --2- 172.21.15.104:0/2049072580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f4015c200 0x7f2f400aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.155+0000 7f2f45b08700 1 --2- 172.21.15.104:0/2049072580 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f400afd80 0x7f2f400c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.155+0000 7f2f45b08700 1 --2- 172.21.15.104:0/2049072580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f400ad0f0 0x7f2f40156820 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.155+0000 7f2f45b08700 1 -- 172.21.15.104:0/2049072580 >> 172.21.15.104:0/2049072580 conn(0x7f2f400cad80 msgr2=0x7f2f4015b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:59.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.155+0000 7f2f45b08700 1 -- 172.21.15.104:0/2049072580 shutdown_connections 2023-07-14T11:54:59.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.155+0000 7f2f45b08700 1 -- 172.21.15.104:0/2049072580 wait complete. 2023-07-14T11:54:59.143 INFO:teuthology.orchestra.run.smithi104.stderr:new fs with metadata pool 6 and data pool 7 2023-07-14T11:54:59.165 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:59.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.351+0000 7f62afa65700 1 Processor -- start 2023-07-14T11:54:59.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.351+0000 7f62afa65700 1 -- start start 2023-07-14T11:54:59.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f62afa65700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f62a8160640 0x7f62a8164000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:59.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f62afa65700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a80cddb0 0x7f62a80afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:59.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f62afa65700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a80c8900 con 0x7f62a8164540 2023-07-14T11:54:59.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f62afa65700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a80c8fc0 con 0x7f62a8160640 2023-07-14T11:54:59.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f62afa65700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a8070540 con 0x7f62a80cddb0 2023-07-14T11:54:59.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f62ae002700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a80cddb0 0x7f62a80afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:59.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f62ad801700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f62a8160640 0x7f62a8164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:59.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f62ae002700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a80cddb0 0x7f62a80afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:42512/0 (socket says 172.21.15.104:42512) 2023-07-14T11:54:59.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f62ae002700 1 -- 172.21.15.104:0/1850058707 learned_addr learned my addr 172.21.15.104:0/1850058707 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:59.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f62ad000700 1 --1- 172.21.15.104:0/1850058707 >> v1:172.21.15.121:6789/0 conn(0x7f62a8164540 0x7f62a8070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:54846/0 (socket says 172.21.15.104:54846) 2023-07-14T11:54:59.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f62ae002700 1 -- 172.21.15.104:0/1850058707 >> v2:172.21.15.121:3300/0 conn(0x7f62a8160640 msgr2=0x7f62a8164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:54:59.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f62ae002700 1 --2- 172.21.15.104:0/1850058707 >> v2:172.21.15.121:3300/0 conn(0x7f62a8160640 0x7f62a8164000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f62ae002700 1 -- 172.21.15.104:0/1850058707 >> v1:172.21.15.121:6789/0 conn(0x7f62a8164540 legacy=0x7f62a8070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:59.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f62ae002700 1 -- 172.21.15.104:0/1850058707 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a80caae0 con 0x7f62a80cddb0 2023-07-14T11:54:59.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f62ae002700 1 --2- 172.21.15.104:0/1850058707 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a80cddb0 0x7f62a80afd30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009e50 tx=0x7f62a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65c44a835ce0437f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:59.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f629ffff700 1 -- 172.21.15.104:0/1850058707 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a000e070 con 0x7f62a80cddb0 2023-07-14T11:54:59.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f629ffff700 1 -- 172.21.15.104:0/1850058707 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a000b910 con 0x7f62a80cddb0 2023-07-14T11:54:59.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.355+0000 7f629ffff700 1 -- 172.21.15.104:0/1850058707 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a000bcc0 con 0x7f62a80cddb0 2023-07-14T11:54:59.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.359+0000 7f62afa65700 1 -- 172.21.15.104:0/1850058707 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a80cddb0 msgr2=0x7f62a80afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:59.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.359+0000 7f62afa65700 1 --2- 172.21.15.104:0/1850058707 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a80cddb0 0x7f62a80afd30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009e50 tx=0x7f62a000b330 comp rx=0 tx=0).stop 2023-07-14T11:54:59.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.359+0000 7f62afa65700 1 -- 172.21.15.104:0/1850058707 shutdown_connections 2023-07-14T11:54:59.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.359+0000 7f62afa65700 1 --2- 172.21.15.104:0/1850058707 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a80cddb0 0x7f62a80afd30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.359+0000 7f62afa65700 1 --2- 172.21.15.104:0/1850058707 >> v2:172.21.15.121:3300/0 conn(0x7f62a8160640 0x7f62a8164000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.359+0000 7f62afa65700 1 -- 172.21.15.104:0/1850058707 >> 172.21.15.104:0/1850058707 conn(0x7f62a80cad30 msgr2=0x7f62a80cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:59.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.359+0000 7f62afa65700 1 -- 172.21.15.104:0/1850058707 shutdown_connections 2023-07-14T11:54:59.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.359+0000 7f62afa65700 1 -- 172.21.15.104:0/1850058707 wait complete. 2023-07-14T11:54:59.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.363+0000 7f62afa65700 1 Processor -- start 2023-07-14T11:54:59.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.363+0000 7f62afa65700 1 -- start start 2023-07-14T11:54:59.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.363+0000 7f62afa65700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a80cddb0 0x7f62a8155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:59.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.363+0000 7f62afa65700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a8160640 0x7f62a8151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:59.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.363+0000 7f62afa65700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a8164540 0x7f62a8163320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:59.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.363+0000 7f62afa65700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a81f5ba0 con 0x7f62a8160640 2023-07-14T11:54:59.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.363+0000 7f62afa65700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a8161100 con 0x7f62a80cddb0 2023-07-14T11:54:59.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.363+0000 7f62afa65700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a80c6390 con 0x7f62a8164540 2023-07-14T11:54:59.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.363+0000 7f62ad000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a8160640 0x7f62a8151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:59.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.363+0000 7f62ad801700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a80cddb0 0x7f62a8155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:59.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.363+0000 7f62ad000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a8160640 0x7f62a8151100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:58070/0 (socket says 172.21.15.104:58070) 2023-07-14T11:54:59.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.363+0000 7f62ad000700 1 -- 172.21.15.104:0/103899007 learned_addr learned my addr 172.21.15.104:0/103899007 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:59.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.363+0000 7f62ae002700 1 --2- 172.21.15.104:0/103899007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a8164540 0x7f62a8163320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:59.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.367+0000 7f62ad000700 1 -- 172.21.15.104:0/103899007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a8164540 msgr2=0x7f62a8163320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:59.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.367+0000 7f62ad000700 1 --2- 172.21.15.104:0/103899007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a8164540 0x7f62a8163320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.367+0000 7f62ad000700 1 -- 172.21.15.104:0/103899007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a80cddb0 msgr2=0x7f62a8155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:59.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.367+0000 7f62ad000700 1 --2- 172.21.15.104:0/103899007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a80cddb0 0x7f62a8155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.367+0000 7f62ad000700 1 -- 172.21.15.104:0/103899007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a0009b30 con 0x7f62a8160640 2023-07-14T11:54:59.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.367+0000 7f62ad801700 1 --2- 172.21.15.104:0/103899007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a80cddb0 0x7f62a8155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:59.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.367+0000 7f62ad000700 1 --2- 172.21.15.104:0/103899007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a8160640 0x7f62a8151100 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f629800c4d0 tx=0x7f629800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:59.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.367+0000 7f629dffb700 1 -- 172.21.15.104:0/103899007 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629800cea0 con 0x7f62a8160640 2023-07-14T11:54:59.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.367+0000 7f629dffb700 1 -- 172.21.15.104:0/103899007 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6298005cd0 con 0x7f62a8160640 2023-07-14T11:54:59.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.367+0000 7f62afa65700 1 -- 172.21.15.104:0/103899007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a8151640 con 0x7f62a8160640 2023-07-14T11:54:59.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.367+0000 7f629dffb700 1 -- 172.21.15.104:0/103899007 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62980072d0 con 0x7f62a8160640 2023-07-14T11:54:59.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.367+0000 7f62afa65700 1 -- 172.21.15.104:0/103899007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a8160ab0 con 0x7f62a8160640 2023-07-14T11:54:59.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.367+0000 7f62afa65700 1 -- 172.21.15.104:0/103899007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a8066ab0 con 0x7f62a8160640 2023-07-14T11:54:59.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.371+0000 7f629dffb700 1 -- 172.21.15.104:0/103899007 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6298007430 con 0x7f62a8160640 2023-07-14T11:54:59.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.371+0000 7f629dffb700 1 --2- 172.21.15.104:0/103899007 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f628c0662f0 0x7f628c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:59.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.371+0000 7f629dffb700 1 -- 172.21.15.104:0/103899007 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6298082fe0 con 0x7f62a8160640 2023-07-14T11:54:59.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.371+0000 7f62ad801700 1 --2- 172.21.15.104:0/103899007 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f628c0662f0 0x7f628c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:59.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.375+0000 7f62ad801700 1 --2- 172.21.15.104:0/103899007 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f628c0662f0 0x7f628c0687a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f62a40112c0 tx=0x7f62a4009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:59.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.379+0000 7f629dffb700 1 -- 172.21.15.104:0/103899007 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f6298050420 con 0x7f62a8160640 2023-07-14T11:54:59.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.555+0000 7f62afa65700 1 -- 172.21.15.104:0/103899007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62a804dcb0 con 0x7f62a8160640 2023-07-14T11:54:59.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.559+0000 7f629dffb700 1 -- 172.21.15.104:0/103899007 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18549 (secure 0 0 0) 0x7f6298017090 con 0x7f62a8160640 2023-07-14T11:54:59.546 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:59.546 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":41,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:54:58.599240+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:54:56.370313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-14T11:54:57.608947+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:59.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.563+0000 7f62afa65700 1 -- 172.21.15.104:0/103899007 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f628c0662f0 msgr2=0x7f628c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:59.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.563+0000 7f62afa65700 1 --2- 172.21.15.104:0/103899007 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f628c0662f0 0x7f628c0687a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f62a40112c0 tx=0x7f62a4009240 comp rx=0 tx=0).stop 2023-07-14T11:54:59.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.563+0000 7f62afa65700 1 -- 172.21.15.104:0/103899007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a8160640 msgr2=0x7f62a8151100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:59.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.563+0000 7f62afa65700 1 --2- 172.21.15.104:0/103899007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a8160640 0x7f62a8151100 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f629800c4d0 tx=0x7f629800c7e0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.563+0000 7f62afa65700 1 -- 172.21.15.104:0/103899007 shutdown_connections 2023-07-14T11:54:59.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.563+0000 7f62afa65700 1 --2- 172.21.15.104:0/103899007 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f628c0662f0 0x7f628c0687a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.563+0000 7f62afa65700 1 --2- 172.21.15.104:0/103899007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62a8160640 0x7f62a8151100 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.563+0000 7f62afa65700 1 --2- 172.21.15.104:0/103899007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62a8164540 0x7f62a8163320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.563+0000 7f62afa65700 1 --2- 172.21.15.104:0/103899007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a80cddb0 0x7f62a8155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.563+0000 7f62afa65700 1 -- 172.21.15.104:0/103899007 >> 172.21.15.104:0/103899007 conn(0x7f62a80cad30 msgr2=0x7f62a80b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:59.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.563+0000 7f62afa65700 1 -- 172.21.15.104:0/103899007 shutdown_connections 2023-07-14T11:54:59.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.563+0000 7f62afa65700 1 -- 172.21.15.104:0/103899007 wait complete. 2023-07-14T11:54:59.574 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:54:59.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.759+0000 7fb0e74cd700 1 Processor -- start 2023-07-14T11:54:59.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.759+0000 7fb0e74cd700 1 -- start start 2023-07-14T11:54:59.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.763+0000 7fb0e74cd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0e006d860 0x7fb0e00cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:59.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.763+0000 7fb0e74cd700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e00cc1b0 con 0x7fb0e00c1e00 2023-07-14T11:54:59.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.763+0000 7fb0e74cd700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e00cd690 con 0x7fb0e006be70 2023-07-14T11:54:59.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.763+0000 7fb0e74cd700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00cd970 con 0x7fb0e006d860 2023-07-14T11:54:59.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.763+0000 7fb0e5a6a700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb0e00c1e00 0x7fb0e00cc640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56752/0 (socket says 172.21.15.104:56752) 2023-07-14T11:54:59.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.763+0000 7fb0e5a6a700 1 -- 172.21.15.104:0/3963237003 learned_addr learned my addr 172.21.15.104:0/3963237003 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:54:59.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.763+0000 7fb0e5269700 1 --2- 172.21.15.104:0/3963237003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0e006d860 0x7fb0e00cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:59.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.763+0000 7fb0d7fff700 1 -- 172.21.15.104:0/3963237003 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156361469 0 0) 0x7fb0e00cc1b0 con 0x7fb0e00c1e00 2023-07-14T11:54:59.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.763+0000 7fb0d7fff700 1 -- 172.21.15.104:0/3963237003 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0cc0035d0 con 0x7fb0e00c1e00 2023-07-14T11:54:59.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.763+0000 7fb0d7fff700 1 -- 172.21.15.104:0/3963237003 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508506540 0 0) 0x7fb0e00cd690 con 0x7fb0e006be70 2023-07-14T11:54:59.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.763+0000 7fb0d7fff700 1 -- 172.21.15.104:0/3963237003 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0e00cc1b0 con 0x7fb0e006be70 2023-07-14T11:54:59.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.763+0000 7fb0e5269700 1 -- 172.21.15.104:0/3963237003 >> v1:172.21.15.121:6789/0 conn(0x7fb0e006be70 legacy=0x7fb0e006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:59.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.767+0000 7fb0e5269700 1 -- 172.21.15.104:0/3963237003 >> v1:172.21.15.104:6789/0 conn(0x7fb0e00c1e00 legacy=0x7fb0e00cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:59.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.767+0000 7fb0e5269700 1 -- 172.21.15.104:0/3963237003 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e00cdc50 con 0x7fb0e006d860 2023-07-14T11:54:59.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.767+0000 7fb0e5269700 1 --2- 172.21.15.104:0/3963237003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0e006d860 0x7fb0e00cb840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009df0 tx=0x7fb0dc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df77c93d649e19fd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:59.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.767+0000 7fb0d7fff700 1 -- 172.21.15.104:0/3963237003 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc00e070 con 0x7fb0e006d860 2023-07-14T11:54:59.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.767+0000 7fb0d7fff700 1 -- 172.21.15.104:0/3963237003 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0dc00bae0 con 0x7fb0e006d860 2023-07-14T11:54:59.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.767+0000 7fb0d7fff700 1 -- 172.21.15.104:0/3963237003 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc00be90 con 0x7fb0e006d860 2023-07-14T11:54:59.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.767+0000 7fb0e74cd700 1 -- 172.21.15.104:0/3963237003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0e006d860 msgr2=0x7fb0e00cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:59.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.767+0000 7fb0e74cd700 1 --2- 172.21.15.104:0/3963237003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0e006d860 0x7fb0e00cb840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009df0 tx=0x7fb0dc00b330 comp rx=0 tx=0).stop 2023-07-14T11:54:59.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.767+0000 7fb0e74cd700 1 -- 172.21.15.104:0/3963237003 shutdown_connections 2023-07-14T11:54:59.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.767+0000 7fb0e74cd700 1 --2- 172.21.15.104:0/3963237003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0e006d860 0x7fb0e00cb840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.767+0000 7fb0e74cd700 1 -- 172.21.15.104:0/3963237003 >> 172.21.15.104:0/3963237003 conn(0x7fb0e00cad30 msgr2=0x7fb0e00cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:59.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.767+0000 7fb0e74cd700 1 -- 172.21.15.104:0/3963237003 shutdown_connections 2023-07-14T11:54:59.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.767+0000 7fb0e74cd700 1 -- 172.21.15.104:0/3963237003 wait complete. 2023-07-14T11:54:59.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.771+0000 7fb0e74cd700 1 Processor -- start 2023-07-14T11:54:59.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.771+0000 7fb0e74cd700 1 -- start start 2023-07-14T11:54:59.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.771+0000 7fb0e74cd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0e006be70 0x7fb0e01f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:59.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e74cd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0e006d860 0x7fb0e014d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:59.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e74cd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0e00c1e00 0x7fb0e00c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:59.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e74cd700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e014dea0 con 0x7fb0e006d860 2023-07-14T11:54:59.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e74cd700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00c7d70 con 0x7fb0e006be70 2023-07-14T11:54:59.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e74cd700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00c8050 con 0x7fb0e00c1e00 2023-07-14T11:54:59.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e4a68700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0e006d860 0x7fb0e014d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:59.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e4a68700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0e006d860 0x7fb0e014d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:58074/0 (socket says 172.21.15.104:58074) 2023-07-14T11:54:59.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e4a68700 1 -- 172.21.15.104:0/114806499 learned_addr learned my addr 172.21.15.104:0/114806499 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:54:59.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e5a6a700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0e00c1e00 0x7fb0e00c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:59.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e5269700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0e006be70 0x7fb0e01f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:59.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e4a68700 1 -- 172.21.15.104:0/114806499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0e00c1e00 msgr2=0x7fb0e00c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:59.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e4a68700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0e00c1e00 0x7fb0e00c7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e4a68700 1 -- 172.21.15.104:0/114806499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0e006be70 msgr2=0x7fb0e01f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:59.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e4a68700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0e006be70 0x7fb0e01f57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e4a68700 1 -- 172.21.15.104:0/114806499 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc009a70 con 0x7fb0e006d860 2023-07-14T11:54:59.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e5a6a700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0e00c1e00 0x7fb0e00c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:54:59.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e5269700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0e006be70 0x7fb0e01f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:54:59.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e4a68700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0e006d860 0x7fb0e014d960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000c730 tx=0x7fb0d000caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:59.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0d5ffb700 1 -- 172.21.15.104:0/114806499 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00052d0 con 0x7fb0e006d860 2023-07-14T11:54:59.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0d5ffb700 1 -- 172.21.15.104:0/114806499 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d0005430 con 0x7fb0e006d860 2023-07-14T11:54:59.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e74cd700 1 -- 172.21.15.104:0/114806499 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e01f4ce0 con 0x7fb0e006d860 2023-07-14T11:54:59.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0d5ffb700 1 -- 172.21.15.104:0/114806499 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00055a0 con 0x7fb0e006d860 2023-07-14T11:54:59.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e74cd700 1 -- 172.21.15.104:0/114806499 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e01f52a0 con 0x7fb0e006d860 2023-07-14T11:54:59.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.775+0000 7fb0e74cd700 1 -- 172.21.15.104:0/114806499 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e004dcb0 con 0x7fb0e006d860 2023-07-14T11:54:59.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.779+0000 7fb0d5ffb700 1 -- 172.21.15.104:0/114806499 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb0d00038e0 con 0x7fb0e006d860 2023-07-14T11:54:59.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.779+0000 7fb0d5ffb700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb0c4066320 0x7fb0c40687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:59.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.779+0000 7fb0d5ffb700 1 -- 172.21.15.104:0/114806499 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb0d0083330 con 0x7fb0e006d860 2023-07-14T11:54:59.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.779+0000 7fb0e5269700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb0c4066320 0x7fb0c40687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:59.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.779+0000 7fb0e5269700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb0c4066320 0x7fb0c40687d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc000f80 tx=0x7fb0dc011000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:59.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.787+0000 7fb0d5ffb700 1 -- 172.21.15.104:0/114806499 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb0d0050700 con 0x7fb0e006d860 2023-07-14T11:54:59.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.963+0000 7fb0e74cd700 1 -- 172.21.15.104:0/114806499 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0e00b0520 con 0x7fb0e006d860 2023-07-14T11:54:59.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.967+0000 7fb0d5ffb700 1 -- 172.21.15.104:0/114806499 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18536 (secure 0 0 0) 0x7fb0d0053d20 con 0x7fb0e006d860 2023-07-14T11:54:59.953 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:54:59.953 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":42,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:54:59.619235+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:54:56.370313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-14T11:54:57.608947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:54:59.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.971+0000 7fb0e74cd700 1 -- 172.21.15.104:0/114806499 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb0c4066320 msgr2=0x7fb0c40687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:59.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.971+0000 7fb0e74cd700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb0c4066320 0x7fb0c40687d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc000f80 tx=0x7fb0dc011000 comp rx=0 tx=0).stop 2023-07-14T11:54:59.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.971+0000 7fb0e74cd700 1 -- 172.21.15.104:0/114806499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0e006d860 msgr2=0x7fb0e014d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:59.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.971+0000 7fb0e74cd700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0e006d860 0x7fb0e014d960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000c730 tx=0x7fb0d000caf0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.971+0000 7fb0e74cd700 1 -- 172.21.15.104:0/114806499 shutdown_connections 2023-07-14T11:54:59.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.971+0000 7fb0e74cd700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb0c4066320 0x7fb0c40687d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.971+0000 7fb0e74cd700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0e006d860 0x7fb0e014d960 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.971+0000 7fb0e74cd700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0e00c1e00 0x7fb0e00c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.971+0000 7fb0e74cd700 1 --2- 172.21.15.104:0/114806499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0e006be70 0x7fb0e01f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:59.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.971+0000 7fb0e74cd700 1 -- 172.21.15.104:0/114806499 >> 172.21.15.104:0/114806499 conn(0x7fb0e00cad30 msgr2=0x7fb0e0156ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:59.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.971+0000 7fb0e74cd700 1 -- 172.21.15.104:0/114806499 shutdown_connections 2023-07-14T11:54:59.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:54:59.971+0000 7fb0e74cd700 1 -- 172.21.15.104:0/114806499 wait complete. 2023-07-14T11:54:59.979 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-14T11:55:00.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.163+0000 7f192a30e700 1 Processor -- start 2023-07-14T11:55:00.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.163+0000 7f192a30e700 1 -- start start 2023-07-14T11:55:00.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.167+0000 7f192a30e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19240b0770 0x7f192406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:00.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.167+0000 7f192a30e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19240714f0 con 0x7f1924159c60 2023-07-14T11:55:00.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.167+0000 7f192a30e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f192406da00 con 0x7f19240cab00 2023-07-14T11:55:00.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.167+0000 7f192a30e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192406dce0 con 0x7f19240b0770 2023-07-14T11:55:00.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.167+0000 7f192930c700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f1924159c60 0x7f1924070de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56758/0 (socket says 172.21.15.104:56758) 2023-07-14T11:55:00.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.167+0000 7f192930c700 1 -- 172.21.15.104:0/3377795240 learned_addr learned my addr 172.21.15.104:0/3377795240 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:00.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.167+0000 7f1928b0b700 1 --2- 172.21.15.104:0/3377795240 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19240b0770 0x7f192406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:00.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.167+0000 7f191bfff700 1 -- 172.21.15.104:0/3377795240 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693473650 0 0) 0x7f192406da00 con 0x7f19240cab00 2023-07-14T11:55:00.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.167+0000 7f191bfff700 1 -- 172.21.15.104:0/3377795240 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19100035d0 con 0x7f19240cab00 2023-07-14T11:55:00.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.167+0000 7f1928b0b700 1 -- 172.21.15.104:0/3377795240 >> v1:172.21.15.121:6789/0 conn(0x7f19240cab00 legacy=0x7f19240c1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:00.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.167+0000 7f1928b0b700 1 -- 172.21.15.104:0/3377795240 >> v1:172.21.15.104:6789/0 conn(0x7f1924159c60 legacy=0x7f1924070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:00.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.167+0000 7f1928b0b700 1 -- 172.21.15.104:0/3377795240 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192406dfc0 con 0x7f19240b0770 2023-07-14T11:55:00.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.167+0000 7f1928b0b700 1 --2- 172.21.15.104:0/3377795240 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19240b0770 0x7f192406d3b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1914009880 tx=0x7f1914009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76bbb5d1604e8518 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:00.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.167+0000 7f191bfff700 1 -- 172.21.15.104:0/3377795240 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1914011830 con 0x7f19240b0770 2023-07-14T11:55:00.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.171+0000 7f191bfff700 1 -- 172.21.15.104:0/3377795240 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1914011990 con 0x7f19240b0770 2023-07-14T11:55:00.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.171+0000 7f191bfff700 1 -- 172.21.15.104:0/3377795240 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1914011d20 con 0x7f19240b0770 2023-07-14T11:55:00.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.171+0000 7f192a30e700 1 -- 172.21.15.104:0/3377795240 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19240b0770 msgr2=0x7f192406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:00.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.171+0000 7f192a30e700 1 --2- 172.21.15.104:0/3377795240 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19240b0770 0x7f192406d3b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1914009880 tx=0x7f1914009b90 comp rx=0 tx=0).stop 2023-07-14T11:55:00.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.171+0000 7f192a30e700 1 -- 172.21.15.104:0/3377795240 shutdown_connections 2023-07-14T11:55:00.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.171+0000 7f192a30e700 1 --2- 172.21.15.104:0/3377795240 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19240b0770 0x7f192406d3b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:00.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.171+0000 7f192a30e700 1 -- 172.21.15.104:0/3377795240 >> 172.21.15.104:0/3377795240 conn(0x7f19240afc60 msgr2=0x7f19240b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:00.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.171+0000 7f192a30e700 1 -- 172.21.15.104:0/3377795240 shutdown_connections 2023-07-14T11:55:00.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.171+0000 7f192a30e700 1 -- 172.21.15.104:0/3377795240 wait complete. 2023-07-14T11:55:00.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192a30e700 1 Processor -- start 2023-07-14T11:55:00.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192a30e700 1 -- start start 2023-07-14T11:55:00.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192a30e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19240b0770 0x7f19241eef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:00.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192a30e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19240cab00 0x7f19240c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:00.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192a30e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1924159c60 0x7f19240c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:00.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192a30e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240c8530 con 0x7f19240b0770 2023-07-14T11:55:00.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192a30e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240c7c90 con 0x7f1924159c60 2023-07-14T11:55:00.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192a30e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19240c7f70 con 0x7f19240cab00 2023-07-14T11:55:00.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192930c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19240b0770 0x7f19241eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:00.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192930c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19240b0770 0x7f19241eef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:58090/0 (socket says 172.21.15.104:58090) 2023-07-14T11:55:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192930c700 1 -- 172.21.15.104:0/2925374429 learned_addr learned my addr 172.21.15.104:0/2925374429 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f1928b0b700 1 --2- 172.21.15.104:0/2925374429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19240cab00 0x7f19240c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f1929b0d700 1 --2- 172.21.15.104:0/2925374429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1924159c60 0x7f19240c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192930c700 1 -- 172.21.15.104:0/2925374429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19240cab00 msgr2=0x7f19240c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192930c700 1 --2- 172.21.15.104:0/2925374429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19240cab00 0x7f19240c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192930c700 1 -- 172.21.15.104:0/2925374429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1924159c60 msgr2=0x7f19240c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:00.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192930c700 1 --2- 172.21.15.104:0/2925374429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1924159c60 0x7f19240c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192930c700 1 -- 172.21.15.104:0/2925374429 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19140094b0 con 0x7f19240b0770 2023-07-14T11:55:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f1929b0d700 1 --2- 172.21.15.104:0/2925374429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1924159c60 0x7f19240c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192930c700 1 --2- 172.21.15.104:0/2925374429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19240b0770 0x7f19241eef40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f192000c700 tx=0x7f192000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f1919ffb700 1 -- 172.21.15.104:0/2925374429 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920005d90 con 0x7f19240b0770 2023-07-14T11:55:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f1919ffb700 1 -- 172.21.15.104:0/2925374429 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19200070d0 con 0x7f19240b0770 2023-07-14T11:55:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f192a30e700 1 -- 172.21.15.104:0/2925374429 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19240c8250 con 0x7f19240b0770 2023-07-14T11:55:00.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.175+0000 7f1919ffb700 1 -- 172.21.15.104:0/2925374429 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920007440 con 0x7f19240b0770 2023-07-14T11:55:00.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.179+0000 7f192a30e700 1 -- 172.21.15.104:0/2925374429 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19241ef480 con 0x7f19240b0770 2023-07-14T11:55:00.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.179+0000 7f192a30e700 1 -- 172.21.15.104:0/2925374429 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19240c28e0 con 0x7f19240b0770 2023-07-14T11:55:00.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.179+0000 7f1919ffb700 1 -- 172.21.15.104:0/2925374429 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f19200075a0 con 0x7f19240b0770 2023-07-14T11:55:00.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.183+0000 7f1919ffb700 1 --2- 172.21.15.104:0/2925374429 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19080662f0 0x7f19080687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:00.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.183+0000 7f1919ffb700 1 -- 172.21.15.104:0/2925374429 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1920083140 con 0x7f19240b0770 2023-07-14T11:55:00.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.183+0000 7f1928b0b700 1 --2- 172.21.15.104:0/2925374429 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19080662f0 0x7f19080687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:00.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.183+0000 7f1928b0b700 1 --2- 172.21.15.104:0/2925374429 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19080662f0 0x7f19080687a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f19140090b0 tx=0x7f1914010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:00.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.187+0000 7f1919ffb700 1 -- 172.21.15.104:0/2925374429 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f19240c28e0 con 0x7f19240b0770 2023-07-14T11:55:00.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:00.399+0000 7f192a30e700 1 -- 172.21.15.104:0/2925374429 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f19240c28e0 con 0x7f19240b0770 2023-07-14T11:55:01.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.179+0000 7f1919ffb700 1 -- 172.21.15.104:0/2925374429 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f19240c28e0 con 0x7f19240b0770 2023-07-14T11:55:01.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.183+0000 7f192a30e700 1 -- 172.21.15.104:0/2925374429 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19080662f0 msgr2=0x7f19080687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:01.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.183+0000 7f192a30e700 1 --2- 172.21.15.104:0/2925374429 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19080662f0 0x7f19080687a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f19140090b0 tx=0x7f1914010000 comp rx=0 tx=0).stop 2023-07-14T11:55:01.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.183+0000 7f192a30e700 1 -- 172.21.15.104:0/2925374429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19240b0770 msgr2=0x7f19241eef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:01.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.183+0000 7f192a30e700 1 --2- 172.21.15.104:0/2925374429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19240b0770 0x7f19241eef40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f192000c700 tx=0x7f192000ca10 comp rx=0 tx=0).stop 2023-07-14T11:55:01.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.183+0000 7f192a30e700 1 -- 172.21.15.104:0/2925374429 shutdown_connections 2023-07-14T11:55:01.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.183+0000 7f192a30e700 1 --2- 172.21.15.104:0/2925374429 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19080662f0 0x7f19080687a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:01.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.183+0000 7f192a30e700 1 --2- 172.21.15.104:0/2925374429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1924159c60 0x7f19240c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:01.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.183+0000 7f192a30e700 1 --2- 172.21.15.104:0/2925374429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19240cab00 0x7f19240c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:01.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.183+0000 7f192a30e700 1 --2- 172.21.15.104:0/2925374429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19240b0770 0x7f19241eef40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:01.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.183+0000 7f192a30e700 1 -- 172.21.15.104:0/2925374429 >> 172.21.15.104:0/2925374429 conn(0x7f19240afc60 msgr2=0x7f19240adbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:01.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.183+0000 7f192a30e700 1 -- 172.21.15.104:0/2925374429 shutdown_connections 2023-07-14T11:55:01.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.183+0000 7f192a30e700 1 -- 172.21.15.104:0/2925374429 wait complete. 2023-07-14T11:55:01.188 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-14T11:55:01.188 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-14T11:55:01.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.383+0000 7ff1db59e700 1 Processor -- start 2023-07-14T11:55:01.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.383+0000 7ff1db59e700 1 -- start start 2023-07-14T11:55:01.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1db59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff1dc0b0ee0 0x7ff1dc0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:01.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1db59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff1dc0cd090 0x7ff1dc0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:01.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1db59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1dc0cd4e0 con 0x7ff1dc0d0610 2023-07-14T11:55:01.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1db59e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1dc0d1a60 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1db59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1dc0d1ef0 con 0x7ff1dc0cd090 2023-07-14T11:55:01.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1da59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff1dc0b0ee0 0x7ff1dc0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:01.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1da59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff1dc0b0ee0 0x7ff1dc0ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35626/0 (socket says 172.21.15.104:35626) 2023-07-14T11:55:01.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1da59c700 1 -- 172.21.15.104:0/609230965 learned_addr learned my addr 172.21.15.104:0/609230965 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:01.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1d9d9b700 1 --2- 172.21.15.104:0/609230965 >> v2:172.21.15.121:3300/0 conn(0x7ff1dc0cd090 0x7ff1dc0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:01.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1da59c700 1 -- 172.21.15.104:0/609230965 >> v2:172.21.15.121:3300/0 conn(0x7ff1dc0cd090 msgr2=0x7ff1dc0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:01.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1da59c700 1 --2- 172.21.15.104:0/609230965 >> v2:172.21.15.121:3300/0 conn(0x7ff1dc0cd090 0x7ff1dc0d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:01.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1da59c700 1 -- 172.21.15.104:0/609230965 >> v1:172.21.15.121:6789/0 conn(0x7ff1dc0d0610 legacy=0x7ff1dc15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:01.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1da59c700 1 -- 172.21.15.104:0/609230965 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc0d0e50 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1da59c700 1 --2- 172.21.15.104:0/609230965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1dc0b0ee0 0x7ff1dc0ccb50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0b5090 tx=0x7ff1cc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b512261420be04b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:01.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1d959a700 1 -- 172.21.15.104:0/609230965 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc011830 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1d959a700 1 -- 172.21.15.104:0/609230965 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc011990 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1d959a700 1 -- 172.21.15.104:0/609230965 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc011d20 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1db59e700 1 -- 172.21.15.104:0/609230965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1dc0b0ee0 msgr2=0x7ff1dc0ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:01.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.387+0000 7ff1db59e700 1 --2- 172.21.15.104:0/609230965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1dc0b0ee0 0x7ff1dc0ccb50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0b5090 tx=0x7ff1cc009bf0 comp rx=0 tx=0).stop 2023-07-14T11:55:01.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.391+0000 7ff1db59e700 1 -- 172.21.15.104:0/609230965 shutdown_connections 2023-07-14T11:55:01.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.391+0000 7ff1db59e700 1 --2- 172.21.15.104:0/609230965 >> v2:172.21.15.121:3300/0 conn(0x7ff1dc0cd090 0x7ff1dc0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:01.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.391+0000 7ff1db59e700 1 --2- 172.21.15.104:0/609230965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1dc0b0ee0 0x7ff1dc0ccb50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:01.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.391+0000 7ff1db59e700 1 -- 172.21.15.104:0/609230965 >> 172.21.15.104:0/609230965 conn(0x7ff1dc060c40 msgr2=0x7ff1dc09b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:01.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.391+0000 7ff1db59e700 1 -- 172.21.15.104:0/609230965 shutdown_connections 2023-07-14T11:55:01.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.391+0000 7ff1db59e700 1 -- 172.21.15.104:0/609230965 wait complete. 2023-07-14T11:55:01.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.391+0000 7ff1db59e700 1 Processor -- start 2023-07-14T11:55:01.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1db59e700 1 -- start start 2023-07-14T11:55:01.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1db59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1dc0b0ee0 0x7ff1dc0c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:01.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1db59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1dc0cd090 0x7ff1dc0c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:01.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1db59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1dc0d0610 0x7ff1dc15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:01.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1db59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc0c6800 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1db59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc0c7e70 con 0x7ff1dc0cd090 2023-07-14T11:55:01.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1db59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc0ca400 con 0x7ff1dc0d0610 2023-07-14T11:55:01.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1da59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1dc0b0ee0 0x7ff1dc0c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:01.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1da59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1dc0b0ee0 0x7ff1dc0c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35630/0 (socket says 172.21.15.104:35630) 2023-07-14T11:55:01.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1da59c700 1 -- 172.21.15.104:0/2241893028 learned_addr learned my addr 172.21.15.104:0/2241893028 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:01.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1d9d9b700 1 --2- 172.21.15.104:0/2241893028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1dc0cd090 0x7ff1dc0c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:01.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1dad9d700 1 --2- 172.21.15.104:0/2241893028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1dc0d0610 0x7ff1dc15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:01.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1da59c700 1 -- 172.21.15.104:0/2241893028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1dc0d0610 msgr2=0x7ff1dc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:01.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1da59c700 1 --2- 172.21.15.104:0/2241893028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1dc0d0610 0x7ff1dc15a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:01.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1da59c700 1 -- 172.21.15.104:0/2241893028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1dc0cd090 msgr2=0x7ff1dc0c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:01.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1da59c700 1 --2- 172.21.15.104:0/2241893028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1dc0cd090 0x7ff1dc0c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:01.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1da59c700 1 -- 172.21.15.104:0/2241893028 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc009570 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1d9d9b700 1 --2- 172.21.15.104:0/2241893028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1dc0cd090 0x7ff1dc0c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:01.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.395+0000 7ff1da59c700 1 --2- 172.21.15.104:0/2241893028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1dc0b0ee0 0x7ff1dc0c6c90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc014040 tx=0x7ff1cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:01.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.399+0000 7ff1caffd700 1 -- 172.21.15.104:0/2241893028 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc004550 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.399+0000 7ff1caffd700 1 -- 172.21.15.104:0/2241893028 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc01e070 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.399+0000 7ff1db59e700 1 -- 172.21.15.104:0/2241893028 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1dc0c61b0 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.399+0000 7ff1caffd700 1 -- 172.21.15.104:0/2241893028 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0038c0 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.399+0000 7ff1db59e700 1 -- 172.21.15.104:0/2241893028 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc15a720 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.399+0000 7ff1db59e700 1 -- 172.21.15.104:0/2241893028 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1dc0c6010 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.403+0000 7ff1caffd700 1 -- 172.21.15.104:0/2241893028 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff1cc003a20 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.403+0000 7ff1caffd700 1 --2- 172.21.15.104:0/2241893028 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff1bc066320 0x7ff1bc0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:01.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.403+0000 7ff1caffd700 1 -- 172.21.15.104:0/2241893028 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff1cc084e40 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.403+0000 7ff1d9d9b700 1 --2- 172.21.15.104:0/2241893028 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff1bc066320 0x7ff1bc0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:01.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.403+0000 7ff1d9d9b700 1 --2- 172.21.15.104:0/2241893028 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff1bc066320 0x7ff1bc0687d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc15b540 tx=0x7ff1d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:01.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.407+0000 7ff1caffd700 1 -- 172.21.15.104:0/2241893028 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff1dc0c6010 con 0x7ff1dc0b0ee0 2023-07-14T11:55:01.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:01.611+0000 7ff1db59e700 1 -- 172.21.15.104:0/2241893028 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff1dc0c6010 con 0x7ff1dc0b0ee0 2023-07-14T11:55:02.081 INFO:tasks.ceph.mds.a.smithi104.stderr:2023-07-14T11:55:02.095+0000 7fcb90b2d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-14T11:55:02.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.191+0000 7ff1caffd700 1 -- 172.21.15.104:0/2241893028 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff1dc0c6010 con 0x7ff1dc0b0ee0 2023-07-14T11:55:02.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.195+0000 7ff1db59e700 1 -- 172.21.15.104:0/2241893028 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff1bc066320 msgr2=0x7ff1bc0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:02.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.195+0000 7ff1db59e700 1 --2- 172.21.15.104:0/2241893028 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff1bc066320 0x7ff1bc0687d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc15b540 tx=0x7ff1d0008040 comp rx=0 tx=0).stop 2023-07-14T11:55:02.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.195+0000 7ff1db59e700 1 -- 172.21.15.104:0/2241893028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1dc0b0ee0 msgr2=0x7ff1dc0c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:02.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.195+0000 7ff1db59e700 1 --2- 172.21.15.104:0/2241893028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1dc0b0ee0 0x7ff1dc0c6c90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc014040 tx=0x7ff1cc004770 comp rx=0 tx=0).stop 2023-07-14T11:55:02.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.195+0000 7ff1db59e700 1 -- 172.21.15.104:0/2241893028 shutdown_connections 2023-07-14T11:55:02.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.195+0000 7ff1db59e700 1 --2- 172.21.15.104:0/2241893028 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff1bc066320 0x7ff1bc0687d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.195+0000 7ff1db59e700 1 --2- 172.21.15.104:0/2241893028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1dc0d0610 0x7ff1dc15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.199+0000 7ff1db59e700 1 --2- 172.21.15.104:0/2241893028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1dc0cd090 0x7ff1dc0c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.199+0000 7ff1db59e700 1 --2- 172.21.15.104:0/2241893028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1dc0b0ee0 0x7ff1dc0c6c90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.199+0000 7ff1db59e700 1 -- 172.21.15.104:0/2241893028 >> 172.21.15.104:0/2241893028 conn(0x7ff1dc060c40 msgr2=0x7ff1dc0ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:02.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.199+0000 7ff1db59e700 1 -- 172.21.15.104:0/2241893028 shutdown_connections 2023-07-14T11:55:02.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.199+0000 7ff1db59e700 1 -- 172.21.15.104:0/2241893028 wait complete. 2023-07-14T11:55:02.206 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-14T11:55:02.206 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:55:02.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.395+0000 7f11b0451700 1 Processor -- start 2023-07-14T11:55:02.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.399+0000 7f11b0451700 1 -- start start 2023-07-14T11:55:02.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.399+0000 7f11b0451700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f11a8158b30 0x7f11a80cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:02.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.399+0000 7f11b0451700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a815c7a0 0x7f11a806a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:02.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.399+0000 7f11b0451700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a8069b50 con 0x7f11a80b0dd0 2023-07-14T11:55:02.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.399+0000 7f11b0451700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a806aca0 con 0x7f11a8158b30 2023-07-14T11:55:02.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.399+0000 7f11b0451700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a815cd40 con 0x7f11a815c7a0 2023-07-14T11:55:02.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11ae1ed700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f11a8158b30 0x7f11a80cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:02.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11ae9ee700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f11a80b0dd0 0x7f11a80ca480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:34060/0 (socket says 172.21.15.104:34060) 2023-07-14T11:55:02.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11ae9ee700 1 -- 172.21.15.104:0/2618914306 learned_addr learned my addr 172.21.15.104:0/2618914306 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:02.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11ad9ec700 1 --2- 172.21.15.104:0/2618914306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a815c7a0 0x7f11a806a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:02.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11ad1eb700 1 -- 172.21.15.104:0/2618914306 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633205110 0 0) 0x7f11a8069b50 con 0x7f11a80b0dd0 2023-07-14T11:55:02.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11ad1eb700 1 -- 172.21.15.104:0/2618914306 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11940035d0 con 0x7f11a80b0dd0 2023-07-14T11:55:02.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11ad9ec700 1 -- 172.21.15.104:0/2618914306 >> v2:172.21.15.121:3300/0 conn(0x7f11a8158b30 msgr2=0x7f11a80cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:02.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11ad9ec700 1 --2- 172.21.15.104:0/2618914306 >> v2:172.21.15.121:3300/0 conn(0x7f11a8158b30 0x7f11a80cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11ad9ec700 1 -- 172.21.15.104:0/2618914306 >> v1:172.21.15.121:6789/0 conn(0x7f11a80b0dd0 legacy=0x7f11a80ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:02.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11ad9ec700 1 -- 172.21.15.104:0/2618914306 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a815d020 con 0x7f11a815c7a0 2023-07-14T11:55:02.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11ad9ec700 1 --2- 172.21.15.104:0/2618914306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a815c7a0 0x7f11a806a760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1198009850 tx=0x7f1198009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82493ae0a24e3794 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:02.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11ad1eb700 1 -- 172.21.15.104:0/2618914306 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198011830 con 0x7f11a815c7a0 2023-07-14T11:55:02.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11ad1eb700 1 -- 172.21.15.104:0/2618914306 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1198011990 con 0x7f11a815c7a0 2023-07-14T11:55:02.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11ad1eb700 1 -- 172.21.15.104:0/2618914306 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198011d40 con 0x7f11a815c7a0 2023-07-14T11:55:02.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11b0451700 1 -- 172.21.15.104:0/2618914306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a815c7a0 msgr2=0x7f11a806a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:02.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.403+0000 7f11b0451700 1 --2- 172.21.15.104:0/2618914306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a815c7a0 0x7f11a806a760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1198009850 tx=0x7f1198009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:02.391 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.407+0000 7f11b0451700 1 -- 172.21.15.104:0/2618914306 shutdown_connections 2023-07-14T11:55:02.391 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.407+0000 7f11b0451700 1 --2- 172.21.15.104:0/2618914306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a815c7a0 0x7f11a806a760 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.391 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.407+0000 7f11b0451700 1 --2- 172.21.15.104:0/2618914306 >> v2:172.21.15.121:3300/0 conn(0x7f11a8158b30 0x7f11a80cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.391 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.407+0000 7f11b0451700 1 -- 172.21.15.104:0/2618914306 >> 172.21.15.104:0/2618914306 conn(0x7f11a80cbdd0 msgr2=0x7f11a80cc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:02.391 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.407+0000 7f11b0451700 1 -- 172.21.15.104:0/2618914306 shutdown_connections 2023-07-14T11:55:02.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.407+0000 7f11b0451700 1 -- 172.21.15.104:0/2618914306 wait complete. 2023-07-14T11:55:02.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.407+0000 7f11b0451700 1 Processor -- start 2023-07-14T11:55:02.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11b0451700 1 -- start start 2023-07-14T11:55:02.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11b0451700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a80b0dd0 0x7f11a80aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:02.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11b0451700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11a8158b30 0x7f11a814a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:02.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11b0451700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a815c7a0 0x7f11a80c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:02.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11b0451700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a80b0550 con 0x7f11a8158b30 2023-07-14T11:55:02.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11b0451700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a80c8990 con 0x7f11a80b0dd0 2023-07-14T11:55:02.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11b0451700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a80c8c70 con 0x7f11a815c7a0 2023-07-14T11:55:02.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11ad9ec700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11a8158b30 0x7f11a814a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:02.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11ae9ee700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a815c7a0 0x7f11a80c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:02.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11ad9ec700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11a8158b30 0x7f11a814a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35632/0 (socket says 172.21.15.104:35632) 2023-07-14T11:55:02.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11ad9ec700 1 -- 172.21.15.104:0/2356772536 learned_addr learned my addr 172.21.15.104:0/2356772536 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:02.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11ae1ed700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a80b0dd0 0x7f11a80aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:02.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11ae9ee700 1 -- 172.21.15.104:0/2356772536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a80b0dd0 msgr2=0x7f11a80aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:02.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11ae9ee700 1 --2- 172.21.15.104:0/2356772536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a80b0dd0 0x7f11a80aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11ae9ee700 1 -- 172.21.15.104:0/2356772536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11a8158b30 msgr2=0x7f11a814a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:02.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11ae9ee700 1 --2- 172.21.15.104:0/2356772536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11a8158b30 0x7f11a814a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11ae9ee700 1 -- 172.21.15.104:0/2356772536 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11980094b0 con 0x7f11a815c7a0 2023-07-14T11:55:02.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11ad9ec700 1 --2- 172.21.15.104:0/2356772536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11a8158b30 0x7f11a814a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:55:02.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11ae1ed700 1 --2- 172.21.15.104:0/2356772536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a80b0dd0 0x7f11a80aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:55:02.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11ae9ee700 1 --2- 172.21.15.104:0/2356772536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a815c7a0 0x7f11a80c6420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f11a000caa0 tx=0x7f11a000cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:02.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f119effd700 1 -- 172.21.15.104:0/2356772536 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a0007480 con 0x7f11a815c7a0 2023-07-14T11:55:02.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f119effd700 1 -- 172.21.15.104:0/2356772536 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a00075e0 con 0x7f11a815c7a0 2023-07-14T11:55:02.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11b0451700 1 -- 172.21.15.104:0/2356772536 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a80c8f50 con 0x7f11a815c7a0 2023-07-14T11:55:02.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f119effd700 1 -- 172.21.15.104:0/2356772536 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a0007950 con 0x7f11a815c7a0 2023-07-14T11:55:02.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.411+0000 7f11b0451700 1 -- 172.21.15.104:0/2356772536 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a80c6960 con 0x7f11a815c7a0 2023-07-14T11:55:02.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.415+0000 7f11b0451700 1 -- 172.21.15.104:0/2356772536 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a80c9470 con 0x7f11a815c7a0 2023-07-14T11:55:02.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.419+0000 7f119effd700 1 -- 172.21.15.104:0/2356772536 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f11a001c440 con 0x7f11a815c7a0 2023-07-14T11:55:02.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.419+0000 7f119effd700 1 --2- 172.21.15.104:0/2356772536 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f118c066340 0x7f118c0687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:02.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.419+0000 7f119effd700 1 -- 172.21.15.104:0/2356772536 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f11a0083700 con 0x7f11a815c7a0 2023-07-14T11:55:02.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.419+0000 7f11ae1ed700 1 --2- 172.21.15.104:0/2356772536 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f118c066340 0x7f118c0687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:02.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.419+0000 7f11ae1ed700 1 --2- 172.21.15.104:0/2356772536 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f118c066340 0x7f118c0687f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009f40 tx=0x7f11a4009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:02.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.423+0000 7f119effd700 1 -- 172.21.15.104:0/2356772536 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f11a0054560 con 0x7f11a815c7a0 2023-07-14T11:55:02.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.631+0000 7f11b0451700 1 -- 172.21.15.104:0/2356772536 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11a80c9470 con 0x7f11a815c7a0 2023-07-14T11:55:02.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.635+0000 7f119effd700 1 -- 172.21.15.104:0/2356772536 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f11a80c9470 con 0x7f11a815c7a0 2023-07-14T11:55:02.619 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:02.619 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3696540262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3696540262},{"type":"v1","addr":"172.21.15.104:6835","nonce":3696540262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/684052037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":684052037},{"type":"v1","addr":"172.21.15.121:6835","nonce":684052037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/1267091306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":1267091306},{"type":"v1","addr":"172.21.15.121:6833","nonce":1267091306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:54:58.600122+0000","modified":"2023-07-14T11:55:02.180538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5616},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5616":{"gid":5616,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6837/398018673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":398018673},{"type":"v1","addr":"172.21.15.104:6837","nonce":398018673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-14T11:55:02.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.639+0000 7f11b0451700 1 -- 172.21.15.104:0/2356772536 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f118c066340 msgr2=0x7f118c0687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:02.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.639+0000 7f11b0451700 1 --2- 172.21.15.104:0/2356772536 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f118c066340 0x7f118c0687f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009f40 tx=0x7f11a4009240 comp rx=0 tx=0).stop 2023-07-14T11:55:02.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.639+0000 7f11b0451700 1 -- 172.21.15.104:0/2356772536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a815c7a0 msgr2=0x7f11a80c6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:02.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.639+0000 7f11b0451700 1 --2- 172.21.15.104:0/2356772536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a815c7a0 0x7f11a80c6420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f11a000caa0 tx=0x7f11a000cad0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.639+0000 7f11b0451700 1 -- 172.21.15.104:0/2356772536 shutdown_connections 2023-07-14T11:55:02.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.639+0000 7f11b0451700 1 --2- 172.21.15.104:0/2356772536 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f118c066340 0x7f118c0687f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.639+0000 7f11b0451700 1 --2- 172.21.15.104:0/2356772536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11a8158b30 0x7f11a814a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.639+0000 7f11b0451700 1 --2- 172.21.15.104:0/2356772536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a815c7a0 0x7f11a80c6420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.639+0000 7f11b0451700 1 --2- 172.21.15.104:0/2356772536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a80b0dd0 0x7f11a80aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.639+0000 7f11b0451700 1 -- 172.21.15.104:0/2356772536 >> 172.21.15.104:0/2356772536 conn(0x7f11a80cbdd0 msgr2=0x7f11a80ca2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:02.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.639+0000 7f11b0451700 1 -- 172.21.15.104:0/2356772536 shutdown_connections 2023-07-14T11:55:02.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.639+0000 7f11b0451700 1 -- 172.21.15.104:0/2356772536 wait complete. 2023-07-14T11:55:02.626 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 18 2023-07-14T11:55:02.644 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:55:02.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.827+0000 7fc1c1b33700 1 Processor -- start 2023-07-14T11:55:02.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.827+0000 7fc1c1b33700 1 -- start start 2023-07-14T11:55:02.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1c1b33700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1bc0afd30 0x7fc1bc0c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:02.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1c1b33700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc1bc0cc1b0 0x7fc1bc0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:02.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1c1b33700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1bc0c97f0 con 0x7fc1bc0b0100 2023-07-14T11:55:02.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1c1b33700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1bc0c8900 con 0x7fc1bc0cc1b0 2023-07-14T11:55:02.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1c1b33700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0c8be0 con 0x7fc1bc0afd30 2023-07-14T11:55:02.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1bb7fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1bc0afd30 0x7fc1bc0c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:02.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1bbfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc1bc0cc1b0 0x7fc1bc0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:02.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1bb7fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1bc0afd30 0x7fc1bc0c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:43610/0 (socket says 172.21.15.104:43610) 2023-07-14T11:55:02.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1bb7fe700 1 -- 172.21.15.104:0/3502089339 learned_addr learned my addr 172.21.15.104:0/3502089339 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:02.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1bb7fe700 1 -- 172.21.15.104:0/3502089339 >> v2:172.21.15.121:3300/0 conn(0x7fc1bc0cc1b0 msgr2=0x7fc1bc0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:02.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1bb7fe700 1 --2- 172.21.15.104:0/3502089339 >> v2:172.21.15.121:3300/0 conn(0x7fc1bc0cc1b0 0x7fc1bc0cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1bb7fe700 1 -- 172.21.15.104:0/3502089339 >> v1:172.21.15.121:6789/0 conn(0x7fc1bc0b0100 legacy=0x7fc1bc06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:02.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1bb7fe700 1 -- 172.21.15.104:0/3502089339 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc0c8ec0 con 0x7fc1bc0afd30 2023-07-14T11:55:02.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1bb7fe700 1 --2- 172.21.15.104:0/3502089339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1bc0afd30 0x7fc1bc0c9c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac009910 tx=0x7fc1ac009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48fdea5e88556c49 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:02.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1baffd700 1 -- 172.21.15.104:0/3502089339 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac011830 con 0x7fc1bc0afd30 2023-07-14T11:55:02.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1baffd700 1 -- 172.21.15.104:0/3502089339 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac011990 con 0x7fc1bc0afd30 2023-07-14T11:55:02.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.831+0000 7fc1baffd700 1 -- 172.21.15.104:0/3502089339 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac011d40 con 0x7fc1bc0afd30 2023-07-14T11:55:02.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.835+0000 7fc1c1b33700 1 -- 172.21.15.104:0/3502089339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1bc0afd30 msgr2=0x7fc1bc0c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:02.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.835+0000 7fc1c1b33700 1 --2- 172.21.15.104:0/3502089339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1bc0afd30 0x7fc1bc0c9c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac009910 tx=0x7fc1ac009c20 comp rx=0 tx=0).stop 2023-07-14T11:55:02.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.835+0000 7fc1c1b33700 1 -- 172.21.15.104:0/3502089339 shutdown_connections 2023-07-14T11:55:02.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.835+0000 7fc1c1b33700 1 --2- 172.21.15.104:0/3502089339 >> v2:172.21.15.121:3300/0 conn(0x7fc1bc0cc1b0 0x7fc1bc0cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.835+0000 7fc1c1b33700 1 --2- 172.21.15.104:0/3502089339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1bc0afd30 0x7fc1bc0c9c80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.835+0000 7fc1c1b33700 1 -- 172.21.15.104:0/3502089339 >> 172.21.15.104:0/3502089339 conn(0x7fc1bc0cad30 msgr2=0x7fc1bc0cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:02.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.835+0000 7fc1c1b33700 1 -- 172.21.15.104:0/3502089339 shutdown_connections 2023-07-14T11:55:02.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.835+0000 7fc1c1b33700 1 -- 172.21.15.104:0/3502089339 wait complete. 2023-07-14T11:55:02.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.835+0000 7fc1c1b33700 1 Processor -- start 2023-07-14T11:55:02.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.835+0000 7fc1c1b33700 1 -- start start 2023-07-14T11:55:02.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1c1b33700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1bc0afd30 0x7fc1bc149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:02.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1c1b33700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1bc0b0100 0x7fc1bc0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:02.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1c1b33700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1bc0cc1b0 0x7fc1bc06cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:02.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1c1b33700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc1e7580 con 0x7fc1bc0b0100 2023-07-14T11:55:02.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1c1b33700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0c7ed0 con 0x7fc1bc0afd30 2023-07-14T11:55:02.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1c1b33700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0c7550 con 0x7fc1bc0cc1b0 2023-07-14T11:55:02.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1b3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1bc0b0100 0x7fc1bc0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:02.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1b3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1bc0b0100 0x7fc1bc0c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35640/0 (socket says 172.21.15.104:35640) 2023-07-14T11:55:02.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1b3fff700 1 -- 172.21.15.104:0/84240209 learned_addr learned my addr 172.21.15.104:0/84240209 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:02.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1bb7fe700 1 --2- 172.21.15.104:0/84240209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1bc0afd30 0x7fc1bc149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:02.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1bbfff700 1 --2- 172.21.15.104:0/84240209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1bc0cc1b0 0x7fc1bc06cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:02.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1b3fff700 1 -- 172.21.15.104:0/84240209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1bc0cc1b0 msgr2=0x7fc1bc06cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:02.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1b3fff700 1 --2- 172.21.15.104:0/84240209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1bc0cc1b0 0x7fc1bc06cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1b3fff700 1 -- 172.21.15.104:0/84240209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1bc0afd30 msgr2=0x7fc1bc149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:02.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1b3fff700 1 --2- 172.21.15.104:0/84240209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1bc0afd30 0x7fc1bc149990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:02.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1b3fff700 1 -- 172.21.15.104:0/84240209 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac009570 con 0x7fc1bc0b0100 2023-07-14T11:55:02.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1b3fff700 1 --2- 172.21.15.104:0/84240209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1bc0b0100 0x7fc1bc0c7990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400c500 tx=0x7fc1a400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:02.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1b8ff9700 1 -- 172.21.15.104:0/84240209 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a40052d0 con 0x7fc1bc0b0100 2023-07-14T11:55:02.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1b8ff9700 1 -- 172.21.15.104:0/84240209 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a4005430 con 0x7fc1bc0b0100 2023-07-14T11:55:02.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1b8ff9700 1 -- 172.21.15.104:0/84240209 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a40055a0 con 0x7fc1bc0b0100 2023-07-14T11:55:02.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1c1b33700 1 -- 172.21.15.104:0/84240209 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc1f23d0 con 0x7fc1bc0b0100 2023-07-14T11:55:02.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.839+0000 7fc1c1b33700 1 -- 172.21.15.104:0/84240209 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc1f2990 con 0x7fc1bc0b0100 2023-07-14T11:55:02.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.843+0000 7fc1c1b33700 1 -- 172.21.15.104:0/84240209 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1bc0ce0b0 con 0x7fc1bc0b0100 2023-07-14T11:55:02.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.847+0000 7fc1b8ff9700 1 -- 172.21.15.104:0/84240209 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc1a40038e0 con 0x7fc1bc0b0100 2023-07-14T11:55:02.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.847+0000 7fc1b8ff9700 1 --2- 172.21.15.104:0/84240209 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc19c0662f0 0x7fc19c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:02.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.847+0000 7fc1b8ff9700 1 -- 172.21.15.104:0/84240209 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc1a40831c0 con 0x7fc1bc0b0100 2023-07-14T11:55:02.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.847+0000 7fc1bb7fe700 1 --2- 172.21.15.104:0/84240209 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc19c0662f0 0x7fc19c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:02.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.847+0000 7fc1bb7fe700 1 --2- 172.21.15.104:0/84240209 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc19c0662f0 0x7fc19c0687a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac014040 tx=0x7fc1ac008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:02.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:02.851+0000 7fc1b8ff9700 1 -- 172.21.15.104:0/84240209 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fc1a40504e0 con 0x7fc1bc0b0100 2023-07-14T11:55:03.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.019+0000 7fc1c1b33700 1 -- 172.21.15.104:0/84240209 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1bc04dcb0 con 0x7fc1bc0b0100 2023-07-14T11:55:03.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.023+0000 7fc1b8ff9700 1 -- 172.21.15.104:0/84240209 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18536 (secure 0 0 0) 0x7fc1a4053b00 con 0x7fc1bc0b0100 2023-07-14T11:55:03.009 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:03.009 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":42,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:54:59.619235+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:54:56.370313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-14T11:54:57.608947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:55:03.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.027+0000 7fc1c1b33700 1 -- 172.21.15.104:0/84240209 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc19c0662f0 msgr2=0x7fc19c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:03.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.027+0000 7fc1c1b33700 1 --2- 172.21.15.104:0/84240209 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc19c0662f0 0x7fc19c0687a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac014040 tx=0x7fc1ac008030 comp rx=0 tx=0).stop 2023-07-14T11:55:03.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.027+0000 7fc1c1b33700 1 -- 172.21.15.104:0/84240209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1bc0b0100 msgr2=0x7fc1bc0c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:03.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.027+0000 7fc1c1b33700 1 --2- 172.21.15.104:0/84240209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1bc0b0100 0x7fc1bc0c7990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400c500 tx=0x7fc1a400c810 comp rx=0 tx=0).stop 2023-07-14T11:55:03.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.027+0000 7fc1c1b33700 1 -- 172.21.15.104:0/84240209 shutdown_connections 2023-07-14T11:55:03.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.027+0000 7fc1c1b33700 1 --2- 172.21.15.104:0/84240209 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc19c0662f0 0x7fc19c0687a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.027+0000 7fc1c1b33700 1 --2- 172.21.15.104:0/84240209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1bc0b0100 0x7fc1bc0c7990 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.027+0000 7fc1c1b33700 1 --2- 172.21.15.104:0/84240209 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1bc0cc1b0 0x7fc1bc06cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.027+0000 7fc1c1b33700 1 --2- 172.21.15.104:0/84240209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1bc0afd30 0x7fc1bc149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.027+0000 7fc1c1b33700 1 -- 172.21.15.104:0/84240209 >> 172.21.15.104:0/84240209 conn(0x7fc1bc0cad30 msgr2=0x7fc1bc0af530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:03.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.027+0000 7fc1c1b33700 1 -- 172.21.15.104:0/84240209 shutdown_connections 2023-07-14T11:55:03.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.027+0000 7fc1c1b33700 1 -- 172.21.15.104:0/84240209 wait complete. 2023-07-14T11:55:03.038 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-14T11:55:03.039 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:55:03.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.227+0000 7f4681493700 1 Processor -- start 2023-07-14T11:55:03.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.227+0000 7f4681493700 1 -- start start 2023-07-14T11:55:03.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f4681493700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f467c0c9d90 0x7f467c0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:03.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f4681493700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f467c159f20 0x7f467c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:03.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f4681493700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c0b0ef0 con 0x7f467c0caf70 2023-07-14T11:55:03.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f4681493700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467c0ce9c0 con 0x7f467c0c9d90 2023-07-14T11:55:03.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f4681493700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c0cee50 con 0x7f467c159f20 2023-07-14T11:55:03.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f467bfff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f467c0c9d90 0x7f467c0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:03.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f467bfff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f467c0c9d90 0x7f467c0ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35642/0 (socket says 172.21.15.104:35642) 2023-07-14T11:55:03.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f467bfff700 1 -- 172.21.15.104:0/1042652482 learned_addr learned my addr 172.21.15.104:0/1042652482 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:03.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f46737fe700 1 --2- 172.21.15.104:0/1042652482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f467c159f20 0x7f467c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:03.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f467bfff700 1 -- 172.21.15.104:0/1042652482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f467c159f20 msgr2=0x7f467c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:03.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f467bfff700 1 --2- 172.21.15.104:0/1042652482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f467c159f20 0x7f467c0caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f467bfff700 1 -- 172.21.15.104:0/1042652482 >> v1:172.21.15.121:6789/0 conn(0x7f467c0caf70 legacy=0x7f467c0c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:03.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f467bfff700 1 -- 172.21.15.104:0/1042652482 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c0ca6a0 con 0x7f467c0c9d90 2023-07-14T11:55:03.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f467bfff700 1 --2- 172.21.15.104:0/1042652482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467c0c9d90 0x7f467c0ca160 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f466c009910 tx=0x7f466c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7317e63a7fe2bf3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:03.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f467b7fe700 1 -- 172.21.15.104:0/1042652482 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c00ba60 con 0x7f467c0c9d90 2023-07-14T11:55:03.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f467b7fe700 1 -- 172.21.15.104:0/1042652482 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466c00bbc0 con 0x7f467c0c9d90 2023-07-14T11:55:03.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.231+0000 7f467b7fe700 1 -- 172.21.15.104:0/1042652482 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c009e60 con 0x7f467c0c9d90 2023-07-14T11:55:03.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.235+0000 7f4681493700 1 -- 172.21.15.104:0/1042652482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467c0c9d90 msgr2=0x7f467c0ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:03.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.235+0000 7f4681493700 1 --2- 172.21.15.104:0/1042652482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467c0c9d90 0x7f467c0ca160 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f466c009910 tx=0x7f466c009c20 comp rx=0 tx=0).stop 2023-07-14T11:55:03.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.235+0000 7f4681493700 1 -- 172.21.15.104:0/1042652482 shutdown_connections 2023-07-14T11:55:03.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.235+0000 7f4681493700 1 --2- 172.21.15.104:0/1042652482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f467c159f20 0x7f467c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.235+0000 7f4681493700 1 --2- 172.21.15.104:0/1042652482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467c0c9d90 0x7f467c0ca160 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.235+0000 7f4681493700 1 -- 172.21.15.104:0/1042652482 >> 172.21.15.104:0/1042652482 conn(0x7f467c06e960 msgr2=0x7f467c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:03.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.235+0000 7f4681493700 1 -- 172.21.15.104:0/1042652482 shutdown_connections 2023-07-14T11:55:03.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.235+0000 7f4681493700 1 -- 172.21.15.104:0/1042652482 wait complete. 2023-07-14T11:55:03.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.235+0000 7f4681493700 1 Processor -- start 2023-07-14T11:55:03.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.235+0000 7f4681493700 1 -- start start 2023-07-14T11:55:03.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4681493700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f467c0c9d90 0x7f467c14d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:03.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4681493700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f467c0caf70 0x7f467c1f1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:03.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4681493700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467c159f20 0x7f467c158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:03.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4681493700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c0c31f0 con 0x7f467c159f20 2023-07-14T11:55:03.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4681493700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c1f1850 con 0x7f467c0caf70 2023-07-14T11:55:03.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4681493700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c1f1b30 con 0x7f467c0c9d90 2023-07-14T11:55:03.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4680c92700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467c159f20 0x7f467c158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:03.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4680c92700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467c159f20 0x7f467c158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35656/0 (socket says 172.21.15.104:35656) 2023-07-14T11:55:03.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4680c92700 1 -- 172.21.15.104:0/3958791110 learned_addr learned my addr 172.21.15.104:0/3958791110 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:03.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f467bfff700 1 --2- 172.21.15.104:0/3958791110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f467c0c9d90 0x7f467c14d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:03.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f46737fe700 1 --2- 172.21.15.104:0/3958791110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f467c0caf70 0x7f467c1f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:03.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4680c92700 1 -- 172.21.15.104:0/3958791110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f467c0c9d90 msgr2=0x7f467c14d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:03.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4680c92700 1 --2- 172.21.15.104:0/3958791110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f467c0c9d90 0x7f467c14d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4680c92700 1 -- 172.21.15.104:0/3958791110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f467c0caf70 msgr2=0x7f467c1f1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:03.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4680c92700 1 --2- 172.21.15.104:0/3958791110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f467c0caf70 0x7f467c1f1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4680c92700 1 -- 172.21.15.104:0/3958791110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c009570 con 0x7f467c159f20 2023-07-14T11:55:03.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4680c92700 1 --2- 172.21.15.104:0/3958791110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467c159f20 0x7f467c158ec0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f467400cc50 tx=0x7f467400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:03.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f46797fa700 1 -- 172.21.15.104:0/3958791110 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4674003c20 con 0x7f467c159f20 2023-07-14T11:55:03.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f46797fa700 1 -- 172.21.15.104:0/3958791110 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4674003d80 con 0x7f467c159f20 2023-07-14T11:55:03.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4681493700 1 -- 172.21.15.104:0/3958791110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c1f1e10 con 0x7f467c159f20 2023-07-14T11:55:03.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f46797fa700 1 -- 172.21.15.104:0/3958791110 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467400aab0 con 0x7f467c159f20 2023-07-14T11:55:03.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.239+0000 7f4681493700 1 -- 172.21.15.104:0/3958791110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c156520 con 0x7f467c159f20 2023-07-14T11:55:03.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.243+0000 7f4681493700 1 -- 172.21.15.104:0/3958791110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c0c3330 con 0x7f467c159f20 2023-07-14T11:55:03.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.243+0000 7f46797fa700 1 -- 172.21.15.104:0/3958791110 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f467400ac10 con 0x7f467c159f20 2023-07-14T11:55:03.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.247+0000 7f46797fa700 1 --2- 172.21.15.104:0/3958791110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f465c0662f0 0x7f465c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:03.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.247+0000 7f46797fa700 1 -- 172.21.15.104:0/3958791110 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4674083db0 con 0x7f467c159f20 2023-07-14T11:55:03.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.247+0000 7f467bfff700 1 --2- 172.21.15.104:0/3958791110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f465c0662f0 0x7f465c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:03.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.247+0000 7f467bfff700 1 --2- 172.21.15.104:0/3958791110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f465c0662f0 0x7f465c0687a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f466c000c00 tx=0x7f466c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:03.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.251+0000 7f46797fa700 1 -- 172.21.15.104:0/3958791110 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f467c0c3330 con 0x7f467c159f20 2023-07-14T11:55:03.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.415+0000 7f4681493700 1 -- 172.21.15.104:0/3958791110 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f467c0ce090 con 0x7f465c0662f0 2023-07-14T11:55:03.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.427+0000 7f46797fa700 1 -- 172.21.15.104:0/3958791110 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309209 (secure 0 0 0) 0x7f466c00b8e0 con 0x7f465c0662f0 2023-07-14T11:55:03.418 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:03.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.431+0000 7f4681493700 1 -- 172.21.15.104:0/3958791110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f465c0662f0 msgr2=0x7f465c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:03.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.431+0000 7f4681493700 1 --2- 172.21.15.104:0/3958791110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f465c0662f0 0x7f465c0687a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f466c000c00 tx=0x7f466c008030 comp rx=0 tx=0).stop 2023-07-14T11:55:03.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.431+0000 7f4681493700 1 -- 172.21.15.104:0/3958791110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467c159f20 msgr2=0x7f467c158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:03.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.431+0000 7f4681493700 1 --2- 172.21.15.104:0/3958791110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467c159f20 0x7f467c158ec0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f467400cc50 tx=0x7f467400cf60 comp rx=0 tx=0).stop 2023-07-14T11:55:03.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.435+0000 7f4681493700 1 -- 172.21.15.104:0/3958791110 shutdown_connections 2023-07-14T11:55:03.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.435+0000 7f4681493700 1 --2- 172.21.15.104:0/3958791110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f465c0662f0 0x7f465c0687a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.435+0000 7f4681493700 1 --2- 172.21.15.104:0/3958791110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467c159f20 0x7f467c158ec0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.435+0000 7f4681493700 1 --2- 172.21.15.104:0/3958791110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f467c0caf70 0x7f467c1f1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.435+0000 7f4681493700 1 --2- 172.21.15.104:0/3958791110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f467c0c9d90 0x7f467c14d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.435+0000 7f4681493700 1 -- 172.21.15.104:0/3958791110 >> 172.21.15.104:0/3958791110 conn(0x7f467c06e960 msgr2=0x7f467c0c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:03.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.435+0000 7f4681493700 1 -- 172.21.15.104:0/3958791110 shutdown_connections 2023-07-14T11:55:03.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.435+0000 7f4681493700 1 -- 172.21.15.104:0/3958791110 wait complete. 2023-07-14T11:55:03.423 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:55:03.441 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2023-07-14T11:55:01.921674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220624,"kb_used_data":5008,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754096,"statfs":{"total":773094113280,"available":772868194304,"internally_reserved":0,"allocated":5128192,"data_stored":2911122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.792492"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901539+0000","last_change":"2023-07-14T11:54:58.641456+0000","last_active":"2023-07-14T11:54:59.901539+0000","last_peered":"2023-07-14T11:54:59.901539+0000","last_clean":"2023-07-14T11:54:59.901539+0000","last_became_active":"2023-07-14T11:54:58.641202+0000","last_became_peered":"2023-07-14T11:54:58.641202+0000","last_unstale":"2023-07-14T11:54:59.901539+0000","last_undegraded":"2023-07-14T11:54:59.901539+0000","last_fullsized":"2023-07-14T11:54:59.901539+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:53:46.284892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.657733+0000","last_change":"2023-07-14T11:54:58.655411+0000","last_active":"2023-07-14T11:54:58.657733+0000","last_peered":"2023-07-14T11:54:58.657733+0000","last_clean":"2023-07-14T11:54:58.657733+0000","last_became_active":"2023-07-14T11:54:58.655127+0000","last_became_peered":"2023-07-14T11:54:58.655127+0000","last_unstale":"2023-07-14T11:54:58.657733+0000","last_undegraded":"2023-07-14T11:54:58.657733+0000","last_fullsized":"2023-07-14T11:54:58.657733+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:09:45.008219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296692+0000","last_change":"2023-07-14T11:54:58.646737+0000","last_active":"2023-07-14T11:55:00.296692+0000","last_peered":"2023-07-14T11:55:00.296692+0000","last_clean":"2023-07-14T11:55:00.296692+0000","last_became_active":"2023-07-14T11:54:58.645742+0000","last_became_peered":"2023-07-14T11:54:58.645742+0000","last_unstale":"2023-07-14T11:55:00.296692+0000","last_undegraded":"2023-07-14T11:55:00.296692+0000","last_fullsized":"2023-07-14T11:55:00.296692+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:06.938057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782481+0000","last_change":"2023-07-14T11:54:58.654871+0000","last_active":"2023-07-14T11:55:00.782481+0000","last_peered":"2023-07-14T11:55:00.782481+0000","last_clean":"2023-07-14T11:55:00.782481+0000","last_became_active":"2023-07-14T11:54:58.654375+0000","last_became_peered":"2023-07-14T11:54:58.654375+0000","last_unstale":"2023-07-14T11:55:00.782481+0000","last_undegraded":"2023-07-14T11:55:00.782481+0000","last_fullsized":"2023-07-14T11:55:00.782481+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:09.069253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.651353+0000","last_change":"2023-07-14T11:54:58.649723+0000","last_active":"2023-07-14T11:54:58.651353+0000","last_peered":"2023-07-14T11:54:58.651353+0000","last_clean":"2023-07-14T11:54:58.651353+0000","last_became_active":"2023-07-14T11:54:58.649186+0000","last_became_peered":"2023-07-14T11:54:58.649186+0000","last_unstale":"2023-07-14T11:54:58.651353+0000","last_undegraded":"2023-07-14T11:54:58.651353+0000","last_fullsized":"2023-07-14T11:54:58.651353+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:59:33.942540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.290952+0000","last_change":"2023-07-14T11:54:58.639034+0000","last_active":"2023-07-14T11:55:00.290952+0000","last_peered":"2023-07-14T11:55:00.290952+0000","last_clean":"2023-07-14T11:55:00.290952+0000","last_became_active":"2023-07-14T11:54:58.638742+0000","last_became_peered":"2023-07-14T11:54:58.638742+0000","last_unstale":"2023-07-14T11:55:00.290952+0000","last_undegraded":"2023-07-14T11:55:00.290952+0000","last_fullsized":"2023-07-14T11:55:00.290952+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:03:33.742656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901095+0000","last_change":"2023-07-14T11:54:58.643140+0000","last_active":"2023-07-14T11:54:59.901095+0000","last_peered":"2023-07-14T11:54:59.901095+0000","last_clean":"2023-07-14T11:54:59.901095+0000","last_became_active":"2023-07-14T11:54:58.642669+0000","last_became_peered":"2023-07-14T11:54:58.642669+0000","last_unstale":"2023-07-14T11:54:59.901095+0000","last_undegraded":"2023-07-14T11:54:59.901095+0000","last_fullsized":"2023-07-14T11:54:59.901095+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:38:19.284483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.781924+0000","last_change":"2023-07-14T11:54:58.649185+0000","last_active":"2023-07-14T11:55:00.781924+0000","last_peered":"2023-07-14T11:55:00.781924+0000","last_clean":"2023-07-14T11:55:00.781924+0000","last_became_active":"2023-07-14T11:54:58.648890+0000","last_became_peered":"2023-07-14T11:54:58.648890+0000","last_unstale":"2023-07-14T11:55:00.781924+0000","last_undegraded":"2023-07-14T11:55:00.781924+0000","last_fullsized":"2023-07-14T11:55:00.781924+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:14.527613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782539+0000","last_change":"2023-07-14T11:54:58.654781+0000","last_active":"2023-07-14T11:55:00.782539+0000","last_peered":"2023-07-14T11:55:00.782539+0000","last_clean":"2023-07-14T11:55:00.782539+0000","last_became_active":"2023-07-14T11:54:58.654369+0000","last_became_peered":"2023-07-14T11:54:58.654369+0000","last_unstale":"2023-07-14T11:55:00.782539+0000","last_undegraded":"2023-07-14T11:55:00.782539+0000","last_fullsized":"2023-07-14T11:55:00.782539+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:55:09.545959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782961+0000","last_change":"2023-07-14T11:54:58.648314+0000","last_active":"2023-07-14T11:55:00.782961+0000","last_peered":"2023-07-14T11:55:00.782961+0000","last_clean":"2023-07-14T11:55:00.782961+0000","last_became_active":"2023-07-14T11:54:58.648008+0000","last_became_peered":"2023-07-14T11:54:58.648008+0000","last_unstale":"2023-07-14T11:55:00.782961+0000","last_undegraded":"2023-07-14T11:55:00.782961+0000","last_fullsized":"2023-07-14T11:55:00.782961+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:58:36.210098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782692+0000","last_change":"2023-07-14T11:54:58.642338+0000","last_active":"2023-07-14T11:55:00.782692+0000","last_peered":"2023-07-14T11:55:00.782692+0000","last_clean":"2023-07-14T11:55:00.782692+0000","last_became_active":"2023-07-14T11:54:58.642102+0000","last_became_peered":"2023-07-14T11:54:58.642102+0000","last_unstale":"2023-07-14T11:55:00.782692+0000","last_undegraded":"2023-07-14T11:55:00.782692+0000","last_fullsized":"2023-07-14T11:55:00.782692+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:19:33.144908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.650796+0000","last_change":"2023-07-14T11:54:58.648047+0000","last_active":"2023-07-14T11:54:58.650796+0000","last_peered":"2023-07-14T11:54:58.650796+0000","last_clean":"2023-07-14T11:54:58.650796+0000","last_became_active":"2023-07-14T11:54:58.647793+0000","last_became_peered":"2023-07-14T11:54:58.647793+0000","last_unstale":"2023-07-14T11:54:58.650796+0000","last_undegraded":"2023-07-14T11:54:58.650796+0000","last_fullsized":"2023-07-14T11:54:58.650796+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:58:55.875713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.628255+0000","last_change":"2023-07-14T11:54:58.633898+0000","last_active":"2023-07-14T11:54:59.628255+0000","last_peered":"2023-07-14T11:54:59.628255+0000","last_clean":"2023-07-14T11:54:59.628255+0000","last_became_active":"2023-07-14T11:54:58.633658+0000","last_became_peered":"2023-07-14T11:54:58.633658+0000","last_unstale":"2023-07-14T11:54:59.628255+0000","last_undegraded":"2023-07-14T11:54:59.628255+0000","last_fullsized":"2023-07-14T11:54:59.628255+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:01:34.531440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.781909+0000","last_change":"2023-07-14T11:54:58.649364+0000","last_active":"2023-07-14T11:55:00.781909+0000","last_peered":"2023-07-14T11:55:00.781909+0000","last_clean":"2023-07-14T11:55:00.781909+0000","last_became_active":"2023-07-14T11:54:58.648940+0000","last_became_peered":"2023-07-14T11:54:58.648940+0000","last_unstale":"2023-07-14T11:55:00.781909+0000","last_undegraded":"2023-07-14T11:55:00.781909+0000","last_fullsized":"2023-07-14T11:55:00.781909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:31.432081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.290673+0000","last_change":"2023-07-14T11:54:58.642704+0000","last_active":"2023-07-14T11:55:00.290673+0000","last_peered":"2023-07-14T11:55:00.290673+0000","last_clean":"2023-07-14T11:55:00.290673+0000","last_became_active":"2023-07-14T11:54:58.642462+0000","last_became_peered":"2023-07-14T11:54:58.642462+0000","last_unstale":"2023-07-14T11:55:00.290673+0000","last_undegraded":"2023-07-14T11:55:00.290673+0000","last_fullsized":"2023-07-14T11:55:00.290673+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:40.247598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.630270+0000","last_change":"2023-07-14T11:54:58.652323+0000","last_active":"2023-07-14T11:54:59.630270+0000","last_peered":"2023-07-14T11:54:59.630270+0000","last_clean":"2023-07-14T11:54:59.630270+0000","last_became_active":"2023-07-14T11:54:58.652071+0000","last_became_peered":"2023-07-14T11:54:58.652071+0000","last_unstale":"2023-07-14T11:54:59.630270+0000","last_undegraded":"2023-07-14T11:54:59.630270+0000","last_fullsized":"2023-07-14T11:54:59.630270+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:43:37.194857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296863+0000","last_change":"2023-07-14T11:54:58.644090+0000","last_active":"2023-07-14T11:55:00.296863+0000","last_peered":"2023-07-14T11:55:00.296863+0000","last_clean":"2023-07-14T11:55:00.296863+0000","last_became_active":"2023-07-14T11:54:58.643814+0000","last_became_peered":"2023-07-14T11:54:58.643814+0000","last_unstale":"2023-07-14T11:55:00.296863+0000","last_undegraded":"2023-07-14T11:55:00.296863+0000","last_fullsized":"2023-07-14T11:55:00.296863+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:31:07.632677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296994+0000","last_change":"2023-07-14T11:54:58.655934+0000","last_active":"2023-07-14T11:55:00.296994+0000","last_peered":"2023-07-14T11:55:00.296994+0000","last_clean":"2023-07-14T11:55:00.296994+0000","last_became_active":"2023-07-14T11:54:58.655682+0000","last_became_peered":"2023-07-14T11:54:58.655682+0000","last_unstale":"2023-07-14T11:55:00.296994+0000","last_undegraded":"2023-07-14T11:55:00.296994+0000","last_fullsized":"2023-07-14T11:55:00.296994+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:26:42.767185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.631081+0000","last_change":"2023-07-14T11:54:58.651344+0000","last_active":"2023-07-14T11:54:59.631081+0000","last_peered":"2023-07-14T11:54:59.631081+0000","last_clean":"2023-07-14T11:54:59.631081+0000","last_became_active":"2023-07-14T11:54:58.650799+0000","last_became_peered":"2023-07-14T11:54:58.650799+0000","last_unstale":"2023-07-14T11:54:59.631081+0000","last_undegraded":"2023-07-14T11:54:59.631081+0000","last_fullsized":"2023-07-14T11:54:59.631081+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:00:21.272936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.628494+0000","last_change":"2023-07-14T11:54:58.655596+0000","last_active":"2023-07-14T11:54:59.628494+0000","last_peered":"2023-07-14T11:54:59.628494+0000","last_clean":"2023-07-14T11:54:59.628494+0000","last_became_active":"2023-07-14T11:54:58.655335+0000","last_became_peered":"2023-07-14T11:54:58.655335+0000","last_unstale":"2023-07-14T11:54:59.628494+0000","last_undegraded":"2023-07-14T11:54:59.628494+0000","last_fullsized":"2023-07-14T11:54:59.628494+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:21:40.500904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291171+0000","last_change":"2023-07-14T11:54:58.634785+0000","last_active":"2023-07-14T11:55:00.291171+0000","last_peered":"2023-07-14T11:55:00.291171+0000","last_clean":"2023-07-14T11:55:00.291171+0000","last_became_active":"2023-07-14T11:54:58.634281+0000","last_became_peered":"2023-07-14T11:54:58.634281+0000","last_unstale":"2023-07-14T11:55:00.291171+0000","last_undegraded":"2023-07-14T11:55:00.291171+0000","last_fullsized":"2023-07-14T11:55:00.291171+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:22:01.665793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782615+0000","last_change":"2023-07-14T11:54:58.649577+0000","last_active":"2023-07-14T11:55:00.782615+0000","last_peered":"2023-07-14T11:55:00.782615+0000","last_clean":"2023-07-14T11:55:00.782615+0000","last_became_active":"2023-07-14T11:54:58.649048+0000","last_became_peered":"2023-07-14T11:54:58.649048+0000","last_unstale":"2023-07-14T11:55:00.782615+0000","last_undegraded":"2023-07-14T11:55:00.782615+0000","last_fullsized":"2023-07-14T11:55:00.782615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:16:06.190084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.630233+0000","last_change":"2023-07-14T11:54:58.640369+0000","last_active":"2023-07-14T11:54:59.630233+0000","last_peered":"2023-07-14T11:54:59.630233+0000","last_clean":"2023-07-14T11:54:59.630233+0000","last_became_active":"2023-07-14T11:54:58.640049+0000","last_became_peered":"2023-07-14T11:54:58.640049+0000","last_unstale":"2023-07-14T11:54:59.630233+0000","last_undegraded":"2023-07-14T11:54:59.630233+0000","last_fullsized":"2023-07-14T11:54:59.630233+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:19:22.661093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.630076+0000","last_change":"2023-07-14T11:54:58.637243+0000","last_active":"2023-07-14T11:54:59.630076+0000","last_peered":"2023-07-14T11:54:59.630076+0000","last_clean":"2023-07-14T11:54:59.630076+0000","last_became_active":"2023-07-14T11:54:58.636990+0000","last_became_peered":"2023-07-14T11:54:58.636990+0000","last_unstale":"2023-07-14T11:54:59.630076+0000","last_undegraded":"2023-07-14T11:54:59.630076+0000","last_fullsized":"2023-07-14T11:54:59.630076+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:57:20.686316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.631008+0000","last_change":"2023-07-14T11:54:58.643343+0000","last_active":"2023-07-14T11:54:59.631008+0000","last_peered":"2023-07-14T11:54:59.631008+0000","last_clean":"2023-07-14T11:54:59.631008+0000","last_became_active":"2023-07-14T11:54:58.642999+0000","last_became_peered":"2023-07-14T11:54:58.642999+0000","last_unstale":"2023-07-14T11:54:59.631008+0000","last_undegraded":"2023-07-14T11:54:59.631008+0000","last_fullsized":"2023-07-14T11:54:59.631008+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:01:34.531440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.627938+0000","last_change":"2023-07-14T11:54:58.639362+0000","last_active":"2023-07-14T11:54:59.627938+0000","last_peered":"2023-07-14T11:54:59.627938+0000","last_clean":"2023-07-14T11:54:59.627938+0000","last_became_active":"2023-07-14T11:54:58.639045+0000","last_became_peered":"2023-07-14T11:54:58.639045+0000","last_unstale":"2023-07-14T11:54:59.627938+0000","last_undegraded":"2023-07-14T11:54:59.627938+0000","last_fullsized":"2023-07-14T11:54:59.627938+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:57:48.273317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296556+0000","last_change":"2023-07-14T11:54:58.643049+0000","last_active":"2023-07-14T11:55:00.296556+0000","last_peered":"2023-07-14T11:55:00.296556+0000","last_clean":"2023-07-14T11:55:00.296556+0000","last_became_active":"2023-07-14T11:54:58.642784+0000","last_became_peered":"2023-07-14T11:54:58.642784+0000","last_unstale":"2023-07-14T11:55:00.296556+0000","last_undegraded":"2023-07-14T11:55:00.296556+0000","last_fullsized":"2023-07-14T11:55:00.296556+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:05:07.290823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.650779+0000","last_change":"2023-07-14T11:54:58.646381+0000","last_active":"2023-07-14T11:54:58.650779+0000","last_peered":"2023-07-14T11:54:58.650779+0000","last_clean":"2023-07-14T11:54:58.650779+0000","last_became_active":"2023-07-14T11:54:58.646121+0000","last_became_peered":"2023-07-14T11:54:58.646121+0000","last_unstale":"2023-07-14T11:54:58.650779+0000","last_undegraded":"2023-07-14T11:54:58.650779+0000","last_fullsized":"2023-07-14T11:54:58.650779+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:19:33.144908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.630424+0000","last_change":"2023-07-14T11:54:58.643178+0000","last_active":"2023-07-14T11:54:59.630424+0000","last_peered":"2023-07-14T11:54:59.630424+0000","last_clean":"2023-07-14T11:54:59.630424+0000","last_became_active":"2023-07-14T11:54:58.642846+0000","last_became_peered":"2023-07-14T11:54:58.642846+0000","last_unstale":"2023-07-14T11:54:59.630424+0000","last_undegraded":"2023-07-14T11:54:59.630424+0000","last_fullsized":"2023-07-14T11:54:59.630424+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:49:42.397101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296394+0000","last_change":"2023-07-14T11:54:58.650834+0000","last_active":"2023-07-14T11:55:00.296394+0000","last_peered":"2023-07-14T11:55:00.296394+0000","last_clean":"2023-07-14T11:55:00.296394+0000","last_became_active":"2023-07-14T11:54:58.650464+0000","last_became_peered":"2023-07-14T11:54:58.650464+0000","last_unstale":"2023-07-14T11:55:00.296394+0000","last_undegraded":"2023-07-14T11:55:00.296394+0000","last_fullsized":"2023-07-14T11:55:00.296394+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:42.897417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.290850+0000","last_change":"2023-07-14T11:54:58.642158+0000","last_active":"2023-07-14T11:55:00.290850+0000","last_peered":"2023-07-14T11:55:00.290850+0000","last_clean":"2023-07-14T11:55:00.290850+0000","last_became_active":"2023-07-14T11:54:58.641871+0000","last_became_peered":"2023-07-14T11:54:58.641871+0000","last_unstale":"2023-07-14T11:55:00.290850+0000","last_undegraded":"2023-07-14T11:55:00.290850+0000","last_fullsized":"2023-07-14T11:55:00.290850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:06:31.160677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.627708+0000","last_change":"2023-07-14T11:54:58.642071+0000","last_active":"2023-07-14T11:54:59.627708+0000","last_peered":"2023-07-14T11:54:59.627708+0000","last_clean":"2023-07-14T11:54:59.627708+0000","last_became_active":"2023-07-14T11:54:58.641800+0000","last_became_peered":"2023-07-14T11:54:58.641800+0000","last_unstale":"2023-07-14T11:54:59.627708+0000","last_undegraded":"2023-07-14T11:54:59.627708+0000","last_fullsized":"2023-07-14T11:54:59.627708+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:49:42.397101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629692+0000","last_change":"2023-07-14T11:54:58.641609+0000","last_active":"2023-07-14T11:54:59.629692+0000","last_peered":"2023-07-14T11:54:59.629692+0000","last_clean":"2023-07-14T11:54:59.629692+0000","last_became_active":"2023-07-14T11:54:58.641357+0000","last_became_peered":"2023-07-14T11:54:58.641357+0000","last_unstale":"2023-07-14T11:54:59.629692+0000","last_undegraded":"2023-07-14T11:54:59.629692+0000","last_fullsized":"2023-07-14T11:54:59.629692+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:53:46.284892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"41'4","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291783+0000","last_change":"2023-07-14T11:54:58.159508+0000","last_active":"2023-07-14T11:55:00.291783+0000","last_peered":"2023-07-14T11:55:00.291783+0000","last_clean":"2023-07-14T11:55:00.291783+0000","last_became_active":"2023-07-14T11:54:58.158645+0000","last_became_peered":"2023-07-14T11:54:58.158645+0000","last_unstale":"2023-07-14T11:55:00.291783+0000","last_undegraded":"2023-07-14T11:55:00.291783+0000","last_fullsized":"2023-07-14T11:55:00.291783+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:52.474136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.631025+0000","last_change":"2023-07-14T11:54:58.646592+0000","last_active":"2023-07-14T11:54:59.631025+0000","last_peered":"2023-07-14T11:54:59.631025+0000","last_clean":"2023-07-14T11:54:59.631025+0000","last_became_active":"2023-07-14T11:54:58.646286+0000","last_became_peered":"2023-07-14T11:54:58.646286+0000","last_unstale":"2023-07-14T11:54:59.631025+0000","last_undegraded":"2023-07-14T11:54:59.631025+0000","last_fullsized":"2023-07-14T11:54:59.631025+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:21.925534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296799+0000","last_change":"2023-07-14T11:54:58.159113+0000","last_active":"2023-07-14T11:55:00.296799+0000","last_peered":"2023-07-14T11:55:00.296799+0000","last_clean":"2023-07-14T11:55:00.296799+0000","last_became_active":"2023-07-14T11:54:58.158798+0000","last_became_peered":"2023-07-14T11:54:58.158798+0000","last_unstale":"2023-07-14T11:55:00.296799+0000","last_undegraded":"2023-07-14T11:55:00.296799+0000","last_fullsized":"2023-07-14T11:55:00.296799+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:50:49.327824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.648970+0000","last_change":"2023-07-14T11:54:58.641877+0000","last_active":"2023-07-14T11:54:58.648970+0000","last_peered":"2023-07-14T11:54:58.648970+0000","last_clean":"2023-07-14T11:54:58.648970+0000","last_became_active":"2023-07-14T11:54:58.641599+0000","last_became_peered":"2023-07-14T11:54:58.641599+0000","last_unstale":"2023-07-14T11:54:58.648970+0000","last_undegraded":"2023-07-14T11:54:58.648970+0000","last_fullsized":"2023-07-14T11:54:58.648970+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:55:09.545959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629755+0000","last_change":"2023-07-14T11:54:58.167470+0000","last_active":"2023-07-14T11:54:59.629755+0000","last_peered":"2023-07-14T11:54:59.629755+0000","last_clean":"2023-07-14T11:54:59.629755+0000","last_became_active":"2023-07-14T11:54:58.167193+0000","last_became_peered":"2023-07-14T11:54:58.167193+0000","last_unstale":"2023-07-14T11:54:59.629755+0000","last_undegraded":"2023-07-14T11:54:59.629755+0000","last_fullsized":"2023-07-14T11:54:59.629755+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:29:49.187978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629691+0000","last_change":"2023-07-14T11:54:58.642675+0000","last_active":"2023-07-14T11:54:59.629691+0000","last_peered":"2023-07-14T11:54:59.629691+0000","last_clean":"2023-07-14T11:54:59.629691+0000","last_became_active":"2023-07-14T11:54:58.642175+0000","last_became_peered":"2023-07-14T11:54:58.642175+0000","last_unstale":"2023-07-14T11:54:59.629691+0000","last_undegraded":"2023-07-14T11:54:59.629691+0000","last_fullsized":"2023-07-14T11:54:59.629691+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:40:21.203057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901470+0000","last_change":"2023-07-14T11:54:58.170311+0000","last_active":"2023-07-14T11:54:59.901470+0000","last_peered":"2023-07-14T11:54:59.901470+0000","last_clean":"2023-07-14T11:54:59.901470+0000","last_became_active":"2023-07-14T11:54:58.170035+0000","last_became_peered":"2023-07-14T11:54:58.170035+0000","last_unstale":"2023-07-14T11:54:59.901470+0000","last_undegraded":"2023-07-14T11:54:59.901470+0000","last_fullsized":"2023-07-14T11:54:59.901470+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:40:20.187965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296407+0000","last_change":"2023-07-14T11:54:58.643745+0000","last_active":"2023-07-14T11:55:00.296407+0000","last_peered":"2023-07-14T11:55:00.296407+0000","last_clean":"2023-07-14T11:55:00.296407+0000","last_became_active":"2023-07-14T11:54:58.643473+0000","last_became_peered":"2023-07-14T11:54:58.643473+0000","last_unstale":"2023-07-14T11:55:00.296407+0000","last_undegraded":"2023-07-14T11:55:00.296407+0000","last_fullsized":"2023-07-14T11:55:00.296407+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:38:34.474674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291473+0000","last_change":"2023-07-14T11:54:58.165926+0000","last_active":"2023-07-14T11:55:00.291473+0000","last_peered":"2023-07-14T11:55:00.291473+0000","last_clean":"2023-07-14T11:55:00.291473+0000","last_became_active":"2023-07-14T11:54:58.165521+0000","last_became_peered":"2023-07-14T11:54:58.165521+0000","last_unstale":"2023-07-14T11:55:00.291473+0000","last_undegraded":"2023-07-14T11:55:00.291473+0000","last_fullsized":"2023-07-14T11:55:00.291473+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:38:29.398816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782085+0000","last_change":"2023-07-14T11:54:58.655258+0000","last_active":"2023-07-14T11:55:00.782085+0000","last_peered":"2023-07-14T11:55:00.782085+0000","last_clean":"2023-07-14T11:55:00.782085+0000","last_became_active":"2023-07-14T11:54:58.654324+0000","last_became_peered":"2023-07-14T11:54:58.654324+0000","last_unstale":"2023-07-14T11:55:00.782085+0000","last_undegraded":"2023-07-14T11:55:00.782085+0000","last_fullsized":"2023-07-14T11:55:00.782085+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:14:01.595650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296697+0000","last_change":"2023-07-14T11:54:58.163946+0000","last_active":"2023-07-14T11:55:00.296697+0000","last_peered":"2023-07-14T11:55:00.296697+0000","last_clean":"2023-07-14T11:55:00.296697+0000","last_became_active":"2023-07-14T11:54:58.163671+0000","last_became_peered":"2023-07-14T11:54:58.163671+0000","last_unstale":"2023-07-14T11:55:00.296697+0000","last_undegraded":"2023-07-14T11:55:00.296697+0000","last_fullsized":"2023-07-14T11:55:00.296697+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:18:00.737557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782269+0000","last_change":"2023-07-14T11:54:58.648406+0000","last_active":"2023-07-14T11:55:00.782269+0000","last_peered":"2023-07-14T11:55:00.782269+0000","last_clean":"2023-07-14T11:55:00.782269+0000","last_became_active":"2023-07-14T11:54:58.648063+0000","last_became_peered":"2023-07-14T11:54:58.648063+0000","last_unstale":"2023-07-14T11:55:00.782269+0000","last_undegraded":"2023-07-14T11:55:00.782269+0000","last_fullsized":"2023-07-14T11:55:00.782269+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:49:16.280244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.635497+0000","last_change":"2023-07-14T11:54:58.162038+0000","last_active":"2023-07-14T11:54:58.635497+0000","last_peered":"2023-07-14T11:54:58.635497+0000","last_clean":"2023-07-14T11:54:58.635497+0000","last_became_active":"2023-07-14T11:54:58.161713+0000","last_became_peered":"2023-07-14T11:54:58.161713+0000","last_unstale":"2023-07-14T11:54:58.635497+0000","last_undegraded":"2023-07-14T11:54:58.635497+0000","last_fullsized":"2023-07-14T11:54:58.635497+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:58:35.195007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629969+0000","last_change":"2023-07-14T11:54:58.641004+0000","last_active":"2023-07-14T11:54:59.629969+0000","last_peered":"2023-07-14T11:54:59.629969+0000","last_clean":"2023-07-14T11:54:59.629969+0000","last_became_active":"2023-07-14T11:54:58.640715+0000","last_became_peered":"2023-07-14T11:54:58.640715+0000","last_unstale":"2023-07-14T11:54:59.629969+0000","last_undegraded":"2023-07-14T11:54:59.629969+0000","last_fullsized":"2023-07-14T11:54:59.629969+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:50:32.746766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782559+0000","last_change":"2023-07-14T11:54:58.163012+0000","last_active":"2023-07-14T11:55:00.782559+0000","last_peered":"2023-07-14T11:55:00.782559+0000","last_clean":"2023-07-14T11:55:00.782559+0000","last_became_active":"2023-07-14T11:54:58.162424+0000","last_became_peered":"2023-07-14T11:54:58.162424+0000","last_unstale":"2023-07-14T11:55:00.782559+0000","last_undegraded":"2023-07-14T11:55:00.782559+0000","last_fullsized":"2023-07-14T11:55:00.782559+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:59:32.927448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296649+0000","last_change":"2023-07-14T11:54:58.652546+0000","last_active":"2023-07-14T11:55:00.296649+0000","last_peered":"2023-07-14T11:55:00.296649+0000","last_clean":"2023-07-14T11:55:00.296649+0000","last_became_active":"2023-07-14T11:54:58.652196+0000","last_became_peered":"2023-07-14T11:54:58.652196+0000","last_unstale":"2023-07-14T11:55:00.296649+0000","last_undegraded":"2023-07-14T11:55:00.296649+0000","last_fullsized":"2023-07-14T11:55:00.296649+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:42:18.814080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.297104+0000","last_change":"2023-07-14T11:54:58.165179+0000","last_active":"2023-07-14T11:55:00.297104+0000","last_peered":"2023-07-14T11:55:00.297104+0000","last_clean":"2023-07-14T11:55:00.297104+0000","last_became_active":"2023-07-14T11:54:58.164894+0000","last_became_peered":"2023-07-14T11:54:58.164894+0000","last_unstale":"2023-07-14T11:55:00.297104+0000","last_undegraded":"2023-07-14T11:55:00.297104+0000","last_fullsized":"2023-07-14T11:55:00.297104+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:08:47.514744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901962+0000","last_change":"2023-07-14T11:54:58.643098+0000","last_active":"2023-07-14T11:54:59.901962+0000","last_peered":"2023-07-14T11:54:59.901962+0000","last_clean":"2023-07-14T11:54:59.901962+0000","last_became_active":"2023-07-14T11:54:58.642779+0000","last_became_peered":"2023-07-14T11:54:58.642779+0000","last_unstale":"2023-07-14T11:54:59.901962+0000","last_undegraded":"2023-07-14T11:54:59.901962+0000","last_fullsized":"2023-07-14T11:54:59.901962+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:14:01.595650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.783116+0000","last_change":"2023-07-14T11:54:58.167572+0000","last_active":"2023-07-14T11:55:00.783116+0000","last_peered":"2023-07-14T11:55:00.783116+0000","last_clean":"2023-07-14T11:55:00.783116+0000","last_became_active":"2023-07-14T11:54:58.167255+0000","last_became_peered":"2023-07-14T11:54:58.167255+0000","last_unstale":"2023-07-14T11:55:00.783116+0000","last_undegraded":"2023-07-14T11:55:00.783116+0000","last_fullsized":"2023-07-14T11:55:00.783116+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:15:07.280920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291456+0000","last_change":"2023-07-14T11:54:58.645141+0000","last_active":"2023-07-14T11:55:00.291456+0000","last_peered":"2023-07-14T11:55:00.291456+0000","last_clean":"2023-07-14T11:55:00.291456+0000","last_became_active":"2023-07-14T11:54:58.644876+0000","last_became_peered":"2023-07-14T11:54:58.644876+0000","last_unstale":"2023-07-14T11:55:00.291456+0000","last_undegraded":"2023-07-14T11:55:00.291456+0000","last_fullsized":"2023-07-14T11:55:00.291456+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:39:38.147706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"41'3","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901891+0000","last_change":"2023-07-14T11:54:58.167766+0000","last_active":"2023-07-14T11:54:59.901891+0000","last_peered":"2023-07-14T11:54:59.901891+0000","last_clean":"2023-07-14T11:54:59.901891+0000","last_became_active":"2023-07-14T11:54:58.167390+0000","last_became_peered":"2023-07-14T11:54:58.167390+0000","last_unstale":"2023-07-14T11:54:59.901891+0000","last_undegraded":"2023-07-14T11:54:59.901891+0000","last_fullsized":"2023-07-14T11:54:59.901891+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:13.512521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901574+0000","last_change":"2023-07-14T11:54:58.642997+0000","last_active":"2023-07-14T11:54:59.901574+0000","last_peered":"2023-07-14T11:54:59.901574+0000","last_clean":"2023-07-14T11:54:59.901574+0000","last_became_active":"2023-07-14T11:54:58.642645+0000","last_became_peered":"2023-07-14T11:54:58.642645+0000","last_unstale":"2023-07-14T11:54:59.901574+0000","last_undegraded":"2023-07-14T11:54:59.901574+0000","last_fullsized":"2023-07-14T11:54:59.901574+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:26:10.479891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"41'6","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.628490+0000","last_change":"2023-07-14T11:54:58.166850+0000","last_active":"2023-07-14T11:54:59.628490+0000","last_peered":"2023-07-14T11:54:59.628490+0000","last_clean":"2023-07-14T11:54:59.628490+0000","last_became_active":"2023-07-14T11:54:58.166569+0000","last_became_peered":"2023-07-14T11:54:58.166569+0000","last_unstale":"2023-07-14T11:54:59.628490+0000","last_undegraded":"2023-07-14T11:54:59.628490+0000","last_fullsized":"2023-07-14T11:54:59.628490+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:00:20.257845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.630463+0000","last_change":"2023-07-14T11:54:58.651466+0000","last_active":"2023-07-14T11:54:59.630463+0000","last_peered":"2023-07-14T11:54:59.630463+0000","last_clean":"2023-07-14T11:54:59.630463+0000","last_became_active":"2023-07-14T11:54:58.651173+0000","last_became_peered":"2023-07-14T11:54:58.651173+0000","last_unstale":"2023-07-14T11:54:59.630463+0000","last_undegraded":"2023-07-14T11:54:59.630463+0000","last_fullsized":"2023-07-14T11:54:59.630463+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:52:02.273145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291371+0000","last_change":"2023-07-14T11:54:58.169271+0000","last_active":"2023-07-14T11:55:00.291371+0000","last_peered":"2023-07-14T11:55:00.291371+0000","last_clean":"2023-07-14T11:55:00.291371+0000","last_became_active":"2023-07-14T11:54:58.168963+0000","last_became_peered":"2023-07-14T11:54:58.168963+0000","last_unstale":"2023-07-14T11:55:00.291371+0000","last_undegraded":"2023-07-14T11:55:00.291371+0000","last_fullsized":"2023-07-14T11:55:00.291371+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:42:03.611134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.781922+0000","last_change":"2023-07-14T11:54:58.652579+0000","last_active":"2023-07-14T11:55:00.781922+0000","last_peered":"2023-07-14T11:55:00.781922+0000","last_clean":"2023-07-14T11:55:00.781922+0000","last_became_active":"2023-07-14T11:54:58.652293+0000","last_became_peered":"2023-07-14T11:54:58.652293+0000","last_unstale":"2023-07-14T11:55:00.781922+0000","last_undegraded":"2023-07-14T11:55:00.781922+0000","last_fullsized":"2023-07-14T11:55:00.781922+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:26:10.479891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.900956+0000","last_change":"2023-07-14T11:54:58.171084+0000","last_active":"2023-07-14T11:54:59.900956+0000","last_peered":"2023-07-14T11:54:59.900956+0000","last_clean":"2023-07-14T11:54:59.900956+0000","last_became_active":"2023-07-14T11:54:58.170771+0000","last_became_peered":"2023-07-14T11:54:58.170771+0000","last_unstale":"2023-07-14T11:54:59.900956+0000","last_undegraded":"2023-07-14T11:54:59.900956+0000","last_fullsized":"2023-07-14T11:54:59.900956+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:56:22.253649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.290651+0000","last_change":"2023-07-14T11:54:58.645227+0000","last_active":"2023-07-14T11:55:00.290651+0000","last_peered":"2023-07-14T11:55:00.290651+0000","last_clean":"2023-07-14T11:55:00.290651+0000","last_became_active":"2023-07-14T11:54:58.644841+0000","last_became_peered":"2023-07-14T11:54:58.644841+0000","last_unstale":"2023-07-14T11:55:00.290651+0000","last_undegraded":"2023-07-14T11:55:00.290651+0000","last_fullsized":"2023-07-14T11:55:00.290651+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:09:45.008219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291143+0000","last_change":"2023-07-14T11:54:58.168874+0000","last_active":"2023-07-14T11:55:00.291143+0000","last_peered":"2023-07-14T11:55:00.291143+0000","last_clean":"2023-07-14T11:55:00.291143+0000","last_became_active":"2023-07-14T11:54:58.168582+0000","last_became_peered":"2023-07-14T11:54:58.168582+0000","last_unstale":"2023-07-14T11:55:00.291143+0000","last_undegraded":"2023-07-14T11:55:00.291143+0000","last_fullsized":"2023-07-14T11:55:00.291143+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:05:25.206219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901553+0000","last_change":"2023-07-14T11:54:58.646464+0000","last_active":"2023-07-14T11:54:59.901553+0000","last_peered":"2023-07-14T11:54:59.901553+0000","last_clean":"2023-07-14T11:54:59.901553+0000","last_became_active":"2023-07-14T11:54:58.646167+0000","last_became_peered":"2023-07-14T11:54:58.646167+0000","last_unstale":"2023-07-14T11:54:59.901553+0000","last_undegraded":"2023-07-14T11:54:59.901553+0000","last_fullsized":"2023-07-14T11:54:59.901553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:29:50.203069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"41'8","reported_seq":21,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782843+0000","last_change":"2023-07-14T11:54:58.164070+0000","last_active":"2023-07-14T11:55:00.782843+0000","last_peered":"2023-07-14T11:55:00.782843+0000","last_clean":"2023-07-14T11:55:00.782843+0000","last_became_active":"2023-07-14T11:54:58.163791+0000","last_became_peered":"2023-07-14T11:54:58.163791+0000","last_unstale":"2023-07-14T11:55:00.782843+0000","last_undegraded":"2023-07-14T11:55:00.782843+0000","last_fullsized":"2023-07-14T11:55:00.782843+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:54:13.946861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.628253+0000","last_change":"2023-07-14T11:54:58.643475+0000","last_active":"2023-07-14T11:54:59.628253+0000","last_peered":"2023-07-14T11:54:59.628253+0000","last_clean":"2023-07-14T11:54:59.628253+0000","last_became_active":"2023-07-14T11:54:58.643228+0000","last_became_peered":"2023-07-14T11:54:58.643228+0000","last_unstale":"2023-07-14T11:54:59.628253+0000","last_undegraded":"2023-07-14T11:54:59.628253+0000","last_fullsized":"2023-07-14T11:54:59.628253+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:21.925534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.628200+0000","last_change":"2023-07-14T11:54:58.171865+0000","last_active":"2023-07-14T11:54:59.628200+0000","last_peered":"2023-07-14T11:54:59.628200+0000","last_clean":"2023-07-14T11:54:59.628200+0000","last_became_active":"2023-07-14T11:54:58.171553+0000","last_became_peered":"2023-07-14T11:54:58.171553+0000","last_unstale":"2023-07-14T11:54:59.628200+0000","last_undegraded":"2023-07-14T11:54:59.628200+0000","last_fullsized":"2023-07-14T11:54:59.628200+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:58:28.547775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782865+0000","last_change":"2023-07-14T11:54:58.656460+0000","last_active":"2023-07-14T11:55:00.782865+0000","last_peered":"2023-07-14T11:55:00.782865+0000","last_clean":"2023-07-14T11:55:00.782865+0000","last_became_active":"2023-07-14T11:54:58.655089+0000","last_became_peered":"2023-07-14T11:54:58.655089+0000","last_unstale":"2023-07-14T11:55:00.782865+0000","last_undegraded":"2023-07-14T11:55:00.782865+0000","last_fullsized":"2023-07-14T11:55:00.782865+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:09:45.008219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.297009+0000","last_change":"2023-07-14T11:54:58.165160+0000","last_active":"2023-07-14T11:55:00.297009+0000","last_peered":"2023-07-14T11:55:00.297009+0000","last_clean":"2023-07-14T11:55:00.297009+0000","last_became_active":"2023-07-14T11:54:58.164863+0000","last_became_peered":"2023-07-14T11:54:58.164863+0000","last_unstale":"2023-07-14T11:55:00.297009+0000","last_undegraded":"2023-07-14T11:55:00.297009+0000","last_fullsized":"2023-07-14T11:55:00.297009+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:14:40.794990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.630575+0000","last_change":"2023-07-14T11:54:58.645608+0000","last_active":"2023-07-14T11:54:59.630575+0000","last_peered":"2023-07-14T11:54:59.630575+0000","last_clean":"2023-07-14T11:54:59.630575+0000","last_became_active":"2023-07-14T11:54:58.645310+0000","last_became_peered":"2023-07-14T11:54:58.645310+0000","last_unstale":"2023-07-14T11:54:59.630575+0000","last_undegraded":"2023-07-14T11:54:59.630575+0000","last_fullsized":"2023-07-14T11:54:59.630575+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:57:48.273317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"41'1","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.169305+0000","last_change":"2023-07-14T11:54:58.162911+0000","last_active":"2023-07-14T11:54:59.169305+0000","last_peered":"2023-07-14T11:54:59.169305+0000","last_clean":"2023-07-14T11:54:59.169305+0000","last_became_active":"2023-07-14T11:54:58.162640+0000","last_became_peered":"2023-07-14T11:54:58.162640+0000","last_unstale":"2023-07-14T11:54:59.169305+0000","last_undegraded":"2023-07-14T11:54:59.169305+0000","last_fullsized":"2023-07-14T11:54:59.169305+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:54:13.946861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.630797+0000","last_change":"2023-07-14T11:54:58.644821+0000","last_active":"2023-07-14T11:54:59.630797+0000","last_peered":"2023-07-14T11:54:59.630797+0000","last_clean":"2023-07-14T11:54:59.630797+0000","last_became_active":"2023-07-14T11:54:58.644337+0000","last_became_peered":"2023-07-14T11:54:58.644337+0000","last_unstale":"2023-07-14T11:54:59.630797+0000","last_undegraded":"2023-07-14T11:54:59.630797+0000","last_fullsized":"2023-07-14T11:54:59.630797+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:58:29.562867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291709+0000","last_change":"2023-07-14T11:54:58.159296+0000","last_active":"2023-07-14T11:55:00.291709+0000","last_peered":"2023-07-14T11:55:00.291709+0000","last_clean":"2023-07-14T11:55:00.291709+0000","last_became_active":"2023-07-14T11:54:58.159030+0000","last_became_peered":"2023-07-14T11:54:58.159030+0000","last_unstale":"2023-07-14T11:55:00.291709+0000","last_undegraded":"2023-07-14T11:55:00.291709+0000","last_fullsized":"2023-07-14T11:55:00.291709+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:14:24.554179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.290711+0000","last_change":"2023-07-14T11:54:58.632702+0000","last_active":"2023-07-14T11:55:00.290711+0000","last_peered":"2023-07-14T11:55:00.290711+0000","last_clean":"2023-07-14T11:55:00.290711+0000","last_became_active":"2023-07-14T11:54:58.632450+0000","last_became_peered":"2023-07-14T11:54:58.632450+0000","last_unstale":"2023-07-14T11:55:00.290711+0000","last_undegraded":"2023-07-14T11:55:00.290711+0000","last_fullsized":"2023-07-14T11:55:00.290711+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:58:36.210098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291610+0000","last_change":"2023-07-14T11:54:58.165758+0000","last_active":"2023-07-14T11:55:00.291610+0000","last_peered":"2023-07-14T11:55:00.291610+0000","last_clean":"2023-07-14T11:55:00.291610+0000","last_became_active":"2023-07-14T11:54:58.165198+0000","last_became_peered":"2023-07-14T11:54:58.165198+0000","last_unstale":"2023-07-14T11:55:00.291610+0000","last_undegraded":"2023-07-14T11:55:00.291610+0000","last_fullsized":"2023-07-14T11:55:00.291610+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:47.597308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629461+0000","last_change":"2023-07-14T11:54:58.644367+0000","last_active":"2023-07-14T11:54:59.629461+0000","last_peered":"2023-07-14T11:54:59.629461+0000","last_clean":"2023-07-14T11:54:59.629461+0000","last_became_active":"2023-07-14T11:54:58.644094+0000","last_became_peered":"2023-07-14T11:54:58.644094+0000","last_unstale":"2023-07-14T11:54:59.629461+0000","last_undegraded":"2023-07-14T11:54:59.629461+0000","last_fullsized":"2023-07-14T11:54:59.629461+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:31:42.113951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.633458+0000","last_change":"2023-07-14T11:54:58.166448+0000","last_active":"2023-07-14T11:54:58.633458+0000","last_peered":"2023-07-14T11:54:58.633458+0000","last_clean":"2023-07-14T11:54:58.633458+0000","last_became_active":"2023-07-14T11:54:58.166048+0000","last_became_peered":"2023-07-14T11:54:58.166048+0000","last_unstale":"2023-07-14T11:54:58.633458+0000","last_undegraded":"2023-07-14T11:54:58.633458+0000","last_fullsized":"2023-07-14T11:54:58.633458+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:14:00.580558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.627781+0000","last_change":"2023-07-14T11:54:58.641466+0000","last_active":"2023-07-14T11:54:59.627781+0000","last_peered":"2023-07-14T11:54:59.627781+0000","last_clean":"2023-07-14T11:54:59.627781+0000","last_became_active":"2023-07-14T11:54:58.641183+0000","last_became_peered":"2023-07-14T11:54:58.641183+0000","last_unstale":"2023-07-14T11:54:59.627781+0000","last_undegraded":"2023-07-14T11:54:59.627781+0000","last_fullsized":"2023-07-14T11:54:59.627781+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:52:02.273145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782010+0000","last_change":"2023-07-14T11:54:58.167804+0000","last_active":"2023-07-14T11:55:00.782010+0000","last_peered":"2023-07-14T11:55:00.782010+0000","last_clean":"2023-07-14T11:55:00.782010+0000","last_became_active":"2023-07-14T11:54:58.167503+0000","last_became_peered":"2023-07-14T11:54:58.167503+0000","last_unstale":"2023-07-14T11:55:00.782010+0000","last_undegraded":"2023-07-14T11:55:00.782010+0000","last_fullsized":"2023-07-14T11:55:00.782010+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:48:00.910272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.781906+0000","last_change":"2023-07-14T11:54:58.654651+0000","last_active":"2023-07-14T11:55:00.781906+0000","last_peered":"2023-07-14T11:55:00.781906+0000","last_clean":"2023-07-14T11:55:00.781906+0000","last_became_active":"2023-07-14T11:54:58.654342+0000","last_became_peered":"2023-07-14T11:54:58.654342+0000","last_unstale":"2023-07-14T11:55:00.781906+0000","last_undegraded":"2023-07-14T11:55:00.781906+0000","last_fullsized":"2023-07-14T11:55:00.781906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:29:50.203069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.900909+0000","last_change":"2023-07-14T11:54:58.168907+0000","last_active":"2023-07-14T11:54:59.900909+0000","last_peered":"2023-07-14T11:54:59.900909+0000","last_clean":"2023-07-14T11:54:59.900909+0000","last_became_active":"2023-07-14T11:54:58.168601+0000","last_became_peered":"2023-07-14T11:54:58.168601+0000","last_unstale":"2023-07-14T11:54:59.900909+0000","last_undegraded":"2023-07-14T11:54:59.900909+0000","last_fullsized":"2023-07-14T11:54:59.900909+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:15:00.073097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.290852+0000","last_change":"2023-07-14T11:54:58.641931+0000","last_active":"2023-07-14T11:55:00.290852+0000","last_peered":"2023-07-14T11:55:00.290852+0000","last_clean":"2023-07-14T11:55:00.290852+0000","last_became_active":"2023-07-14T11:54:58.641574+0000","last_became_peered":"2023-07-14T11:54:58.641574+0000","last_unstale":"2023-07-14T11:55:00.290852+0000","last_undegraded":"2023-07-14T11:55:00.290852+0000","last_fullsized":"2023-07-14T11:55:00.290852+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:15:08.296012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901436+0000","last_change":"2023-07-14T11:54:58.167698+0000","last_active":"2023-07-14T11:54:59.901436+0000","last_peered":"2023-07-14T11:54:59.901436+0000","last_clean":"2023-07-14T11:54:59.901436+0000","last_became_active":"2023-07-14T11:54:58.167388+0000","last_became_peered":"2023-07-14T11:54:58.167388+0000","last_unstale":"2023-07-14T11:54:59.901436+0000","last_undegraded":"2023-07-14T11:54:59.901436+0000","last_fullsized":"2023-07-14T11:54:59.901436+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:29:44.534250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.631145+0000","last_change":"2023-07-14T11:53:09.145201+0000","last_active":"2023-07-14T11:54:59.631145+0000","last_peered":"2023-07-14T11:54:59.631145+0000","last_clean":"2023-07-14T11:54:59.631145+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:54:59.631145+0000","last_undegraded":"2023-07-14T11:54:59.631145+0000","last_fullsized":"2023-07-14T11:54:59.631145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:19:50.489223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291167+0000","last_change":"2023-07-14T11:54:58.645165+0000","last_active":"2023-07-14T11:55:00.291167+0000","last_peered":"2023-07-14T11:55:00.291167+0000","last_clean":"2023-07-14T11:55:00.291167+0000","last_became_active":"2023-07-14T11:54:58.644831+0000","last_became_peered":"2023-07-14T11:54:58.644831+0000","last_unstale":"2023-07-14T11:55:00.291167+0000","last_undegraded":"2023-07-14T11:55:00.291167+0000","last_fullsized":"2023-07-14T11:55:00.291167+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:38:13.006723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296772+0000","last_change":"2023-07-14T11:54:58.169443+0000","last_active":"2023-07-14T11:55:00.296772+0000","last_peered":"2023-07-14T11:55:00.296772+0000","last_clean":"2023-07-14T11:55:00.296772+0000","last_became_active":"2023-07-14T11:54:58.169079+0000","last_became_peered":"2023-07-14T11:54:58.169079+0000","last_unstale":"2023-07-14T11:55:00.296772+0000","last_undegraded":"2023-07-14T11:55:00.296772+0000","last_fullsized":"2023-07-14T11:55:00.296772+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:02:11.868139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629758+0000","last_change":"2023-07-14T11:53:09.146150+0000","last_active":"2023-07-14T11:54:59.629758+0000","last_peered":"2023-07-14T11:54:59.629758+0000","last_clean":"2023-07-14T11:54:59.629758+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:54:59.629758+0000","last_undegraded":"2023-07-14T11:54:59.629758+0000","last_fullsized":"2023-07-14T11:54:59.629758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:40:41.420401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.648985+0000","last_change":"2023-07-14T11:54:58.642131+0000","last_active":"2023-07-14T11:54:58.648985+0000","last_peered":"2023-07-14T11:54:58.648985+0000","last_clean":"2023-07-14T11:54:58.648985+0000","last_became_active":"2023-07-14T11:54:58.641858+0000","last_became_peered":"2023-07-14T11:54:58.641858+0000","last_unstale":"2023-07-14T11:54:58.648985+0000","last_undegraded":"2023-07-14T11:54:58.648985+0000","last_fullsized":"2023-07-14T11:54:58.648985+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:09.069253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629413+0000","last_change":"2023-07-14T11:54:58.175137+0000","last_active":"2023-07-14T11:54:59.629413+0000","last_peered":"2023-07-14T11:54:59.629413+0000","last_clean":"2023-07-14T11:54:59.629413+0000","last_became_active":"2023-07-14T11:54:58.174914+0000","last_became_peered":"2023-07-14T11:54:58.174914+0000","last_unstale":"2023-07-14T11:54:59.629413+0000","last_undegraded":"2023-07-14T11:54:59.629413+0000","last_fullsized":"2023-07-14T11:54:59.629413+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:56:22.253649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.902055+0000","last_change":"2023-07-14T11:53:09.435876+0000","last_active":"2023-07-14T11:54:59.902055+0000","last_peered":"2023-07-14T11:54:59.902055+0000","last_clean":"2023-07-14T11:54:59.902055+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:54:59.902055+0000","last_undegraded":"2023-07-14T11:54:59.902055+0000","last_fullsized":"2023-07-14T11:54:59.902055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:26.268564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296305+0000","last_change":"2023-07-14T11:54:58.641928+0000","last_active":"2023-07-14T11:55:00.296305+0000","last_peered":"2023-07-14T11:55:00.296305+0000","last_clean":"2023-07-14T11:55:00.296305+0000","last_became_active":"2023-07-14T11:54:58.641629+0000","last_became_peered":"2023-07-14T11:54:58.641629+0000","last_unstale":"2023-07-14T11:55:00.296305+0000","last_undegraded":"2023-07-14T11:55:00.296305+0000","last_fullsized":"2023-07-14T11:55:00.296305+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:11:16.296910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782900+0000","last_change":"2023-07-14T11:54:58.169976+0000","last_active":"2023-07-14T11:55:00.782900+0000","last_peered":"2023-07-14T11:55:00.782900+0000","last_clean":"2023-07-14T11:55:00.782900+0000","last_became_active":"2023-07-14T11:54:58.169591+0000","last_became_peered":"2023-07-14T11:54:58.169591+0000","last_unstale":"2023-07-14T11:55:00.782900+0000","last_undegraded":"2023-07-14T11:55:00.782900+0000","last_fullsized":"2023-07-14T11:55:00.782900+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:39.232507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629465+0000","last_change":"2023-07-14T11:53:09.434255+0000","last_active":"2023-07-14T11:54:59.629465+0000","last_peered":"2023-07-14T11:54:59.629465+0000","last_clean":"2023-07-14T11:54:59.629465+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:54:59.629465+0000","last_undegraded":"2023-07-14T11:54:59.629465+0000","last_fullsized":"2023-07-14T11:54:59.629465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:27:55.537660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.631206+0000","last_change":"2023-07-14T11:54:58.655331+0000","last_active":"2023-07-14T11:54:59.631206+0000","last_peered":"2023-07-14T11:54:59.631206+0000","last_clean":"2023-07-14T11:54:59.631206+0000","last_became_active":"2023-07-14T11:54:58.655051+0000","last_became_peered":"2023-07-14T11:54:58.655051+0000","last_unstale":"2023-07-14T11:54:59.631206+0000","last_undegraded":"2023-07-14T11:54:59.631206+0000","last_fullsized":"2023-07-14T11:54:59.631206+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:55:06.438207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"42'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.295282+0000","last_change":"2023-07-14T11:54:58.161597+0000","last_active":"2023-07-14T11:55:00.295282+0000","last_peered":"2023-07-14T11:55:00.295282+0000","last_clean":"2023-07-14T11:55:00.295282+0000","last_became_active":"2023-07-14T11:54:58.161341+0000","last_became_peered":"2023-07-14T11:54:58.161341+0000","last_unstale":"2023-07-14T11:55:00.295282+0000","last_undegraded":"2023-07-14T11:55:00.295282+0000","last_fullsized":"2023-07-14T11:55:00.295282+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:55:08.242109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.633551+0000","last_change":"2023-07-14T11:53:09.147040+0000","last_active":"2023-07-14T11:54:58.633551+0000","last_peered":"2023-07-14T11:54:58.633551+0000","last_clean":"2023-07-14T11:54:58.633551+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:54:58.633551+0000","last_undegraded":"2023-07-14T11:54:58.633551+0000","last_fullsized":"2023-07-14T11:54:58.633551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:26.268564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782843+0000","last_change":"2023-07-14T11:54:58.655331+0000","last_active":"2023-07-14T11:55:00.782843+0000","last_peered":"2023-07-14T11:55:00.782843+0000","last_clean":"2023-07-14T11:55:00.782843+0000","last_became_active":"2023-07-14T11:54:58.655033+0000","last_became_peered":"2023-07-14T11:54:58.655033+0000","last_unstale":"2023-07-14T11:55:00.782843+0000","last_undegraded":"2023-07-14T11:55:00.782843+0000","last_fullsized":"2023-07-14T11:55:00.782843+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:58:55.875713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.902000+0000","last_change":"2023-07-14T11:54:58.172009+0000","last_active":"2023-07-14T11:54:59.902000+0000","last_peered":"2023-07-14T11:54:59.902000+0000","last_clean":"2023-07-14T11:54:59.902000+0000","last_became_active":"2023-07-14T11:54:58.171721+0000","last_became_peered":"2023-07-14T11:54:58.171721+0000","last_unstale":"2023-07-14T11:54:59.902000+0000","last_undegraded":"2023-07-14T11:54:59.902000+0000","last_fullsized":"2023-07-14T11:54:59.902000+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:48:00.910272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.627626+0000","last_change":"2023-07-14T11:53:09.619294+0000","last_active":"2023-07-14T11:54:59.627626+0000","last_peered":"2023-07-14T11:54:59.627626+0000","last_clean":"2023-07-14T11:54:59.627626+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:54:59.627626+0000","last_undegraded":"2023-07-14T11:54:59.627626+0000","last_fullsized":"2023-07-14T11:54:59.627626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:41:47.183177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901574+0000","last_change":"2023-07-14T11:54:58.646315+0000","last_active":"2023-07-14T11:54:59.901574+0000","last_peered":"2023-07-14T11:54:59.901574+0000","last_clean":"2023-07-14T11:54:59.901574+0000","last_became_active":"2023-07-14T11:54:58.645996+0000","last_became_peered":"2023-07-14T11:54:58.645996+0000","last_unstale":"2023-07-14T11:54:59.901574+0000","last_undegraded":"2023-07-14T11:54:59.901574+0000","last_fullsized":"2023-07-14T11:54:59.901574+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:31.432081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629411+0000","last_change":"2023-07-14T11:54:58.166098+0000","last_active":"2023-07-14T11:54:59.629411+0000","last_peered":"2023-07-14T11:54:59.629411+0000","last_clean":"2023-07-14T11:54:59.629411+0000","last_became_active":"2023-07-14T11:54:58.165847+0000","last_became_peered":"2023-07-14T11:54:58.165847+0000","last_unstale":"2023-07-14T11:54:59.629411+0000","last_undegraded":"2023-07-14T11:54:59.629411+0000","last_fullsized":"2023-07-14T11:54:59.629411+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:38:18.269392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.900724+0000","last_change":"2023-07-14T11:53:09.146156+0000","last_active":"2023-07-14T11:54:59.900724+0000","last_peered":"2023-07-14T11:54:59.900724+0000","last_clean":"2023-07-14T11:54:59.900724+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:54:59.900724+0000","last_undegraded":"2023-07-14T11:54:59.900724+0000","last_fullsized":"2023-07-14T11:54:59.900724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:42:51.151986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.783095+0000","last_change":"2023-07-14T11:54:58.652451+0000","last_active":"2023-07-14T11:55:00.783095+0000","last_peered":"2023-07-14T11:55:00.783095+0000","last_clean":"2023-07-14T11:55:00.783095+0000","last_became_active":"2023-07-14T11:54:58.652143+0000","last_became_peered":"2023-07-14T11:54:58.652143+0000","last_unstale":"2023-07-14T11:55:00.783095+0000","last_undegraded":"2023-07-14T11:55:00.783095+0000","last_fullsized":"2023-07-14T11:55:00.783095+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:06:31.160677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"41'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.168234+0000","last_change":"2023-07-14T11:54:58.169336+0000","last_active":"2023-07-14T11:54:59.168234+0000","last_peered":"2023-07-14T11:54:59.168234+0000","last_clean":"2023-07-14T11:54:59.168234+0000","last_became_active":"2023-07-14T11:54:58.168910+0000","last_became_peered":"2023-07-14T11:54:58.168910+0000","last_unstale":"2023-07-14T11:54:59.168234+0000","last_undegraded":"2023-07-14T11:54:59.168234+0000","last_fullsized":"2023-07-14T11:54:59.168234+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:39.232507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.634301+0000","last_change":"2023-07-14T11:53:09.146556+0000","last_active":"2023-07-14T11:54:58.634301+0000","last_peered":"2023-07-14T11:54:58.634301+0000","last_clean":"2023-07-14T11:54:58.634301+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:54:58.634301+0000","last_undegraded":"2023-07-14T11:54:58.634301+0000","last_fullsized":"2023-07-14T11:54:58.634301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:14:16.178403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901146+0000","last_change":"2023-07-14T11:53:03.159818+0000","last_active":"2023-07-14T11:54:59.901146+0000","last_peered":"2023-07-14T11:54:59.901146+0000","last_clean":"2023-07-14T11:54:59.901146+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:54:59.901146+0000","last_undegraded":"2023-07-14T11:54:59.901146+0000","last_fullsized":"2023-07-14T11:54:59.901146+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_clean_scrub_stamp":"2023-07-14T11:53:02.054348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:29:46.040542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1085440,"data_stored":807909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.302}]},{"osd":1,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.73},{"interface":"front","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":1.1040000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":1.6419999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.355}]},{"osd":4,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":4.0720000000000001,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.169}]},{"osd":5,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.579,"5min":1.579,"15min":1.579},"min":{"1min":1.028,"5min":1.028,"15min":1.028},"max":{"1min":4.1630000000000003,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":1.587}]},{"osd":6,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"min":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":4.2619999999999996,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"min":{"1min":1.091,"5min":1.091,"15min":1.091},"max":{"1min":2.7570000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.4810000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":495616,"data_stored":218123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.3939999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.038}]},{"osd":2,"last update":"Fri Jul 14 11:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.3},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":1.008,"5min":1.008,"15min":1.008},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.45}]},{"osd":3,"last update":"Fri Jul 14 11:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.26},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.5329999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.496}]},{"osd":5,"last update":"Fri Jul 14 11:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.429},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":1.034,"5min":1.034,"15min":1.034},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":1.5640000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.995,"5min":0.995,"15min":0.995},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.617},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":1.649}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":479232,"data_stored":210548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.0800000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.02}]},{"osd":2,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.1359999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.2669999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.397}]},{"osd":6,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":1.036,"5min":1.036,"15min":1.036},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.4390000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.482},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"min":{"1min":1.157,"5min":1.157,"15min":1.157},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.595}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":499712,"data_stored":219015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.3169999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.482},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.381}]},{"osd":2,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":3.2759999999999998,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":1.286}]},{"osd":3,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.26},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.448}]},{"osd":5,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":3.1280000000000001,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.177},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.1219999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.409}]},{"osd":6,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":1.083,"5min":1.083,"15min":1.083},"max":{"1min":3.5289999999999999,"5min":3.5289999999999999,"15min":3.5289999999999999},"last":0.97999999999999998}]},{"osd":7,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"max":{"1min":3.5640000000000001,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":1.339}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":499712,"data_stored":218089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.502}]},{"osd":1,"last update":"Fri Jul 14 11:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.92200000000000004}]},{"osd":2,"last update":"Fri Jul 14 11:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.90100000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.335,"5min":2.335,"15min":2.335},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0349999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":2.7959999999999998,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.286}]},{"osd":6,"last update":"Fri Jul 14 11:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.335},{"interface":"front","average":{"1min":1.524,"5min":1.524,"15min":1.524},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.266}]},{"osd":7,"last update":"Fri Jul 14 11:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":1.097,"5min":1.097,"15min":1.097},"max":{"1min":3.395,"5min":3.395,"15min":3.395},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.238}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":503808,"data_stored":219037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.91000000000000003}]},{"osd":1,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.6070000000000002,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":0.73999999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.0229999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.123}]},{"osd":5,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.1819999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.4489999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.2310000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.468,"15min":1.468},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":1.321}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":491520,"data_stored":217521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.77000000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.88900000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.68000000000000005}]},{"osd":4,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.875},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.96999999999999997}]},{"osd":5,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":1.006,"5min":1.006,"15min":1.006},"max":{"1min":3.2949999999999999,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.014}]},{"osd":6,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.403,"15min":1.403},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.2270000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":3.3799999999999999,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.163},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.375,"5min":2.375,"15min":2.375},"last":0.92300000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":1073152,"data_stored":800880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":1.4490000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.1040000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.3260000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":1.05},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.93200000000000005}]},{"osd":4,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":2.2120000000000002,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.4830000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.153}]},{"osd":6,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.607},{"interface":"front","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.2529999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.536},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.667}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:55:03.450 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:55:03.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.671+0000 7febb323b700 1 Processor -- start 2023-07-14T11:55:03.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.671+0000 7febb323b700 1 -- start start 2023-07-14T11:55:03.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.675+0000 7febb323b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7febac06d860 0x7febac0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:03.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.675+0000 7febb323b700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac0cc1b0 con 0x7febac06be70 2023-07-14T11:55:03.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.675+0000 7febb323b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac0cd690 con 0x7febac0c1e00 2023-07-14T11:55:03.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.675+0000 7febb323b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac0cd970 con 0x7febac06d860 2023-07-14T11:55:03.662 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.675+0000 7feba3fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7febac06be70 0x7febac06dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:47206/0 (socket says 172.21.15.104:47206) 2023-07-14T11:55:03.662 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.675+0000 7feba3fff700 1 -- 172.21.15.104:0/4023155546 learned_addr learned my addr 172.21.15.104:0/4023155546 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:03.663 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.675+0000 7febb0fd7700 1 --2- 172.21.15.104:0/4023155546 >> v2:172.21.15.121:3300/0 conn(0x7febac06d860 0x7febac0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:03.663 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.675+0000 7feba37fe700 1 -- 172.21.15.104:0/4023155546 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268865562 0 0) 0x7febac0cc1b0 con 0x7febac06be70 2023-07-14T11:55:03.663 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.675+0000 7feba37fe700 1 -- 172.21.15.104:0/4023155546 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb940035d0 con 0x7febac06be70 2023-07-14T11:55:03.663 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7feba37fe700 1 -- 172.21.15.104:0/4023155546 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382359724 0 0) 0x7febac0cd690 con 0x7febac0c1e00 2023-07-14T11:55:03.663 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7feba37fe700 1 -- 172.21.15.104:0/4023155546 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febac0cc1b0 con 0x7febac0c1e00 2023-07-14T11:55:03.663 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7feba37fe700 1 -- 172.21.15.104:0/4023155546 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2726159836 0 0) 0x7feb940035d0 con 0x7febac06be70 2023-07-14T11:55:03.664 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7febb0fd7700 1 -- 172.21.15.104:0/4023155546 >> v1:172.21.15.121:6789/0 conn(0x7febac0c1e00 legacy=0x7febac0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:03.664 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7febb0fd7700 1 -- 172.21.15.104:0/4023155546 >> v1:172.21.15.104:6789/0 conn(0x7febac06be70 legacy=0x7febac06dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:03.664 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7febb0fd7700 1 -- 172.21.15.104:0/4023155546 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac0cdc50 con 0x7febac06d860 2023-07-14T11:55:03.664 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7febb0fd7700 1 --2- 172.21.15.104:0/4023155546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febac06d860 0x7febac0cb840 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7feba8009850 tx=0x7feba8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4f1b4e7fd74d073 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:03.665 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7feba37fe700 1 -- 172.21.15.104:0/4023155546 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c001670 con 0x7febac06d860 2023-07-14T11:55:03.665 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7feba37fe700 1 -- 172.21.15.104:0/4023155546 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba8004030 con 0x7febac06d860 2023-07-14T11:55:03.665 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7feba37fe700 1 -- 172.21.15.104:0/4023155546 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba80043c0 con 0x7febac06d860 2023-07-14T11:55:03.665 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7febb323b700 1 -- 172.21.15.104:0/4023155546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febac06d860 msgr2=0x7febac0cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:03.666 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7febb323b700 1 --2- 172.21.15.104:0/4023155546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febac06d860 0x7febac0cb840 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7feba8009850 tx=0x7feba8009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:03.666 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7febb323b700 1 -- 172.21.15.104:0/4023155546 shutdown_connections 2023-07-14T11:55:03.666 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7febb323b700 1 --2- 172.21.15.104:0/4023155546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febac06d860 0x7febac0cb840 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.666 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7febb323b700 1 -- 172.21.15.104:0/4023155546 >> 172.21.15.104:0/4023155546 conn(0x7febac0cad30 msgr2=0x7febac0cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:03.666 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7febb323b700 1 -- 172.21.15.104:0/4023155546 shutdown_connections 2023-07-14T11:55:03.666 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.679+0000 7febb323b700 1 -- 172.21.15.104:0/4023155546 wait complete. 2023-07-14T11:55:03.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb323b700 1 Processor -- start 2023-07-14T11:55:03.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb323b700 1 -- start start 2023-07-14T11:55:03.669 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb323b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febac06be70 0x7febac1f4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:03.669 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb323b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febac06d860 0x7febac156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:03.669 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb323b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febac0c1e00 0x7febac1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:03.669 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb323b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0ab8f0 con 0x7febac06be70 2023-07-14T11:55:03.669 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb323b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0abbd0 con 0x7febac06d860 2023-07-14T11:55:03.670 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb323b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac0abeb0 con 0x7febac0c1e00 2023-07-14T11:55:03.670 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb0fd7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febac06be70 0x7febac1f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:03.671 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb0fd7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febac06be70 0x7febac1f4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35666/0 (socket says 172.21.15.104:35666) 2023-07-14T11:55:03.671 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb0fd7700 1 -- 172.21.15.104:0/664106864 learned_addr learned my addr 172.21.15.104:0/664106864 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:03.671 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb17d8700 1 --2- 172.21.15.104:0/664106864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febac0c1e00 0x7febac1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:03.671 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7feba3fff700 1 --2- 172.21.15.104:0/664106864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febac06d860 0x7febac156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:03.672 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb0fd7700 1 -- 172.21.15.104:0/664106864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febac0c1e00 msgr2=0x7febac1557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:03.672 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb0fd7700 1 --2- 172.21.15.104:0/664106864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febac0c1e00 0x7febac1557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.672 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb0fd7700 1 -- 172.21.15.104:0/664106864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febac06d860 msgr2=0x7febac156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:03.672 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb0fd7700 1 --2- 172.21.15.104:0/664106864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febac06d860 0x7febac156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.672 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7febb0fd7700 1 -- 172.21.15.104:0/664106864 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba80094b0 con 0x7febac06be70 2023-07-14T11:55:03.672 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.683+0000 7feba3fff700 1 --2- 172.21.15.104:0/664106864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febac06d860 0x7febac156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:03.672 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.687+0000 7febb0fd7700 1 --2- 172.21.15.104:0/664106864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febac06be70 0x7febac1f4020 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7feba80090b0 tx=0x7feba8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:03.673 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.687+0000 7feba17fa700 1 -- 172.21.15.104:0/664106864 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8011630 con 0x7febac06be70 2023-07-14T11:55:03.673 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.687+0000 7feba17fa700 1 -- 172.21.15.104:0/664106864 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba801f070 con 0x7febac06be70 2023-07-14T11:55:03.673 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.687+0000 7febb323b700 1 -- 172.21.15.104:0/664106864 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac0ac190 con 0x7febac06be70 2023-07-14T11:55:03.673 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.687+0000 7feba17fa700 1 -- 172.21.15.104:0/664106864 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba80119f0 con 0x7febac06be70 2023-07-14T11:55:03.673 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.687+0000 7febb323b700 1 -- 172.21.15.104:0/664106864 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac1573a0 con 0x7febac06be70 2023-07-14T11:55:03.673 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.687+0000 7febb323b700 1 -- 172.21.15.104:0/664106864 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac157620 con 0x7febac06be70 2023-07-14T11:55:03.674 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.687+0000 7feba17fa700 1 -- 172.21.15.104:0/664106864 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7feba8011b50 con 0x7febac06be70 2023-07-14T11:55:03.676 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.691+0000 7feba17fa700 1 --2- 172.21.15.104:0/664106864 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feb980662f0 0x7feb980687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:03.677 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.691+0000 7feba17fa700 1 -- 172.21.15.104:0/664106864 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7feba8021070 con 0x7febac06be70 2023-07-14T11:55:03.677 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.691+0000 7feba3fff700 1 --2- 172.21.15.104:0/664106864 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feb980662f0 0x7feb980687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:03.677 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.691+0000 7feba3fff700 1 --2- 172.21.15.104:0/664106864 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feb980662f0 0x7feb980687a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feb9c001eb0 tx=0x7feb9c001ca0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:03.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.695+0000 7feba17fa700 1 -- 172.21.15.104:0/664106864 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7feba8052a40 con 0x7febac06be70 2023-07-14T11:55:03.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.863+0000 7febb323b700 1 -- 172.21.15.104:0/664106864 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7febac0cdf30 con 0x7feb980662f0 2023-07-14T11:55:03.863 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.879+0000 7feba17fa700 1 -- 172.21.15.104:0/664106864 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309209 (secure 0 0 0) 0x7feb9c00e620 con 0x7feb980662f0 2023-07-14T11:55:03.867 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:03.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.883+0000 7febb323b700 1 -- 172.21.15.104:0/664106864 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feb980662f0 msgr2=0x7feb980687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:03.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.883+0000 7febb323b700 1 --2- 172.21.15.104:0/664106864 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feb980662f0 0x7feb980687a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feb9c001eb0 tx=0x7feb9c001ca0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.883+0000 7febb323b700 1 -- 172.21.15.104:0/664106864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febac06be70 msgr2=0x7febac1f4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:03.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.883+0000 7febb323b700 1 --2- 172.21.15.104:0/664106864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febac06be70 0x7febac1f4020 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7feba80090b0 tx=0x7feba8004770 comp rx=0 tx=0).stop 2023-07-14T11:55:03.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.887+0000 7febb323b700 1 -- 172.21.15.104:0/664106864 shutdown_connections 2023-07-14T11:55:03.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.887+0000 7febb323b700 1 --2- 172.21.15.104:0/664106864 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feb980662f0 0x7feb980687a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.887+0000 7febb323b700 1 --2- 172.21.15.104:0/664106864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febac0c1e00 0x7febac1557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.887+0000 7febb323b700 1 --2- 172.21.15.104:0/664106864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febac06d860 0x7febac156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.887+0000 7febb323b700 1 --2- 172.21.15.104:0/664106864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febac06be70 0x7febac1f4020 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:03.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.887+0000 7febb323b700 1 -- 172.21.15.104:0/664106864 >> 172.21.15.104:0/664106864 conn(0x7febac0cad30 msgr2=0x7febac0c2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:03.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.887+0000 7febb323b700 1 -- 172.21.15.104:0/664106864 shutdown_connections 2023-07-14T11:55:03.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:03.887+0000 7febb323b700 1 -- 172.21.15.104:0/664106864 wait complete. 2023-07-14T11:55:03.873 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:55:03.893 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2023-07-14T11:55:01.921674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220624,"kb_used_data":5008,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754096,"statfs":{"total":773094113280,"available":772868194304,"internally_reserved":0,"allocated":5128192,"data_stored":2911122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.792492"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901539+0000","last_change":"2023-07-14T11:54:58.641456+0000","last_active":"2023-07-14T11:54:59.901539+0000","last_peered":"2023-07-14T11:54:59.901539+0000","last_clean":"2023-07-14T11:54:59.901539+0000","last_became_active":"2023-07-14T11:54:58.641202+0000","last_became_peered":"2023-07-14T11:54:58.641202+0000","last_unstale":"2023-07-14T11:54:59.901539+0000","last_undegraded":"2023-07-14T11:54:59.901539+0000","last_fullsized":"2023-07-14T11:54:59.901539+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:53:46.284892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.657733+0000","last_change":"2023-07-14T11:54:58.655411+0000","last_active":"2023-07-14T11:54:58.657733+0000","last_peered":"2023-07-14T11:54:58.657733+0000","last_clean":"2023-07-14T11:54:58.657733+0000","last_became_active":"2023-07-14T11:54:58.655127+0000","last_became_peered":"2023-07-14T11:54:58.655127+0000","last_unstale":"2023-07-14T11:54:58.657733+0000","last_undegraded":"2023-07-14T11:54:58.657733+0000","last_fullsized":"2023-07-14T11:54:58.657733+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:09:45.008219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296692+0000","last_change":"2023-07-14T11:54:58.646737+0000","last_active":"2023-07-14T11:55:00.296692+0000","last_peered":"2023-07-14T11:55:00.296692+0000","last_clean":"2023-07-14T11:55:00.296692+0000","last_became_active":"2023-07-14T11:54:58.645742+0000","last_became_peered":"2023-07-14T11:54:58.645742+0000","last_unstale":"2023-07-14T11:55:00.296692+0000","last_undegraded":"2023-07-14T11:55:00.296692+0000","last_fullsized":"2023-07-14T11:55:00.296692+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:06.938057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782481+0000","last_change":"2023-07-14T11:54:58.654871+0000","last_active":"2023-07-14T11:55:00.782481+0000","last_peered":"2023-07-14T11:55:00.782481+0000","last_clean":"2023-07-14T11:55:00.782481+0000","last_became_active":"2023-07-14T11:54:58.654375+0000","last_became_peered":"2023-07-14T11:54:58.654375+0000","last_unstale":"2023-07-14T11:55:00.782481+0000","last_undegraded":"2023-07-14T11:55:00.782481+0000","last_fullsized":"2023-07-14T11:55:00.782481+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:09.069253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.651353+0000","last_change":"2023-07-14T11:54:58.649723+0000","last_active":"2023-07-14T11:54:58.651353+0000","last_peered":"2023-07-14T11:54:58.651353+0000","last_clean":"2023-07-14T11:54:58.651353+0000","last_became_active":"2023-07-14T11:54:58.649186+0000","last_became_peered":"2023-07-14T11:54:58.649186+0000","last_unstale":"2023-07-14T11:54:58.651353+0000","last_undegraded":"2023-07-14T11:54:58.651353+0000","last_fullsized":"2023-07-14T11:54:58.651353+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:59:33.942540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.290952+0000","last_change":"2023-07-14T11:54:58.639034+0000","last_active":"2023-07-14T11:55:00.290952+0000","last_peered":"2023-07-14T11:55:00.290952+0000","last_clean":"2023-07-14T11:55:00.290952+0000","last_became_active":"2023-07-14T11:54:58.638742+0000","last_became_peered":"2023-07-14T11:54:58.638742+0000","last_unstale":"2023-07-14T11:55:00.290952+0000","last_undegraded":"2023-07-14T11:55:00.290952+0000","last_fullsized":"2023-07-14T11:55:00.290952+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:03:33.742656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901095+0000","last_change":"2023-07-14T11:54:58.643140+0000","last_active":"2023-07-14T11:54:59.901095+0000","last_peered":"2023-07-14T11:54:59.901095+0000","last_clean":"2023-07-14T11:54:59.901095+0000","last_became_active":"2023-07-14T11:54:58.642669+0000","last_became_peered":"2023-07-14T11:54:58.642669+0000","last_unstale":"2023-07-14T11:54:59.901095+0000","last_undegraded":"2023-07-14T11:54:59.901095+0000","last_fullsized":"2023-07-14T11:54:59.901095+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:38:19.284483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.781924+0000","last_change":"2023-07-14T11:54:58.649185+0000","last_active":"2023-07-14T11:55:00.781924+0000","last_peered":"2023-07-14T11:55:00.781924+0000","last_clean":"2023-07-14T11:55:00.781924+0000","last_became_active":"2023-07-14T11:54:58.648890+0000","last_became_peered":"2023-07-14T11:54:58.648890+0000","last_unstale":"2023-07-14T11:55:00.781924+0000","last_undegraded":"2023-07-14T11:55:00.781924+0000","last_fullsized":"2023-07-14T11:55:00.781924+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:14.527613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782539+0000","last_change":"2023-07-14T11:54:58.654781+0000","last_active":"2023-07-14T11:55:00.782539+0000","last_peered":"2023-07-14T11:55:00.782539+0000","last_clean":"2023-07-14T11:55:00.782539+0000","last_became_active":"2023-07-14T11:54:58.654369+0000","last_became_peered":"2023-07-14T11:54:58.654369+0000","last_unstale":"2023-07-14T11:55:00.782539+0000","last_undegraded":"2023-07-14T11:55:00.782539+0000","last_fullsized":"2023-07-14T11:55:00.782539+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:55:09.545959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782961+0000","last_change":"2023-07-14T11:54:58.648314+0000","last_active":"2023-07-14T11:55:00.782961+0000","last_peered":"2023-07-14T11:55:00.782961+0000","last_clean":"2023-07-14T11:55:00.782961+0000","last_became_active":"2023-07-14T11:54:58.648008+0000","last_became_peered":"2023-07-14T11:54:58.648008+0000","last_unstale":"2023-07-14T11:55:00.782961+0000","last_undegraded":"2023-07-14T11:55:00.782961+0000","last_fullsized":"2023-07-14T11:55:00.782961+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:58:36.210098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782692+0000","last_change":"2023-07-14T11:54:58.642338+0000","last_active":"2023-07-14T11:55:00.782692+0000","last_peered":"2023-07-14T11:55:00.782692+0000","last_clean":"2023-07-14T11:55:00.782692+0000","last_became_active":"2023-07-14T11:54:58.642102+0000","last_became_peered":"2023-07-14T11:54:58.642102+0000","last_unstale":"2023-07-14T11:55:00.782692+0000","last_undegraded":"2023-07-14T11:55:00.782692+0000","last_fullsized":"2023-07-14T11:55:00.782692+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:19:33.144908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.650796+0000","last_change":"2023-07-14T11:54:58.648047+0000","last_active":"2023-07-14T11:54:58.650796+0000","last_peered":"2023-07-14T11:54:58.650796+0000","last_clean":"2023-07-14T11:54:58.650796+0000","last_became_active":"2023-07-14T11:54:58.647793+0000","last_became_peered":"2023-07-14T11:54:58.647793+0000","last_unstale":"2023-07-14T11:54:58.650796+0000","last_undegraded":"2023-07-14T11:54:58.650796+0000","last_fullsized":"2023-07-14T11:54:58.650796+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:58:55.875713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.628255+0000","last_change":"2023-07-14T11:54:58.633898+0000","last_active":"2023-07-14T11:54:59.628255+0000","last_peered":"2023-07-14T11:54:59.628255+0000","last_clean":"2023-07-14T11:54:59.628255+0000","last_became_active":"2023-07-14T11:54:58.633658+0000","last_became_peered":"2023-07-14T11:54:58.633658+0000","last_unstale":"2023-07-14T11:54:59.628255+0000","last_undegraded":"2023-07-14T11:54:59.628255+0000","last_fullsized":"2023-07-14T11:54:59.628255+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:01:34.531440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.781909+0000","last_change":"2023-07-14T11:54:58.649364+0000","last_active":"2023-07-14T11:55:00.781909+0000","last_peered":"2023-07-14T11:55:00.781909+0000","last_clean":"2023-07-14T11:55:00.781909+0000","last_became_active":"2023-07-14T11:54:58.648940+0000","last_became_peered":"2023-07-14T11:54:58.648940+0000","last_unstale":"2023-07-14T11:55:00.781909+0000","last_undegraded":"2023-07-14T11:55:00.781909+0000","last_fullsized":"2023-07-14T11:55:00.781909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:31.432081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.290673+0000","last_change":"2023-07-14T11:54:58.642704+0000","last_active":"2023-07-14T11:55:00.290673+0000","last_peered":"2023-07-14T11:55:00.290673+0000","last_clean":"2023-07-14T11:55:00.290673+0000","last_became_active":"2023-07-14T11:54:58.642462+0000","last_became_peered":"2023-07-14T11:54:58.642462+0000","last_unstale":"2023-07-14T11:55:00.290673+0000","last_undegraded":"2023-07-14T11:55:00.290673+0000","last_fullsized":"2023-07-14T11:55:00.290673+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:40.247598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.630270+0000","last_change":"2023-07-14T11:54:58.652323+0000","last_active":"2023-07-14T11:54:59.630270+0000","last_peered":"2023-07-14T11:54:59.630270+0000","last_clean":"2023-07-14T11:54:59.630270+0000","last_became_active":"2023-07-14T11:54:58.652071+0000","last_became_peered":"2023-07-14T11:54:58.652071+0000","last_unstale":"2023-07-14T11:54:59.630270+0000","last_undegraded":"2023-07-14T11:54:59.630270+0000","last_fullsized":"2023-07-14T11:54:59.630270+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:43:37.194857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296863+0000","last_change":"2023-07-14T11:54:58.644090+0000","last_active":"2023-07-14T11:55:00.296863+0000","last_peered":"2023-07-14T11:55:00.296863+0000","last_clean":"2023-07-14T11:55:00.296863+0000","last_became_active":"2023-07-14T11:54:58.643814+0000","last_became_peered":"2023-07-14T11:54:58.643814+0000","last_unstale":"2023-07-14T11:55:00.296863+0000","last_undegraded":"2023-07-14T11:55:00.296863+0000","last_fullsized":"2023-07-14T11:55:00.296863+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:31:07.632677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296994+0000","last_change":"2023-07-14T11:54:58.655934+0000","last_active":"2023-07-14T11:55:00.296994+0000","last_peered":"2023-07-14T11:55:00.296994+0000","last_clean":"2023-07-14T11:55:00.296994+0000","last_became_active":"2023-07-14T11:54:58.655682+0000","last_became_peered":"2023-07-14T11:54:58.655682+0000","last_unstale":"2023-07-14T11:55:00.296994+0000","last_undegraded":"2023-07-14T11:55:00.296994+0000","last_fullsized":"2023-07-14T11:55:00.296994+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:26:42.767185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.631081+0000","last_change":"2023-07-14T11:54:58.651344+0000","last_active":"2023-07-14T11:54:59.631081+0000","last_peered":"2023-07-14T11:54:59.631081+0000","last_clean":"2023-07-14T11:54:59.631081+0000","last_became_active":"2023-07-14T11:54:58.650799+0000","last_became_peered":"2023-07-14T11:54:58.650799+0000","last_unstale":"2023-07-14T11:54:59.631081+0000","last_undegraded":"2023-07-14T11:54:59.631081+0000","last_fullsized":"2023-07-14T11:54:59.631081+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:00:21.272936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.628494+0000","last_change":"2023-07-14T11:54:58.655596+0000","last_active":"2023-07-14T11:54:59.628494+0000","last_peered":"2023-07-14T11:54:59.628494+0000","last_clean":"2023-07-14T11:54:59.628494+0000","last_became_active":"2023-07-14T11:54:58.655335+0000","last_became_peered":"2023-07-14T11:54:58.655335+0000","last_unstale":"2023-07-14T11:54:59.628494+0000","last_undegraded":"2023-07-14T11:54:59.628494+0000","last_fullsized":"2023-07-14T11:54:59.628494+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:21:40.500904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291171+0000","last_change":"2023-07-14T11:54:58.634785+0000","last_active":"2023-07-14T11:55:00.291171+0000","last_peered":"2023-07-14T11:55:00.291171+0000","last_clean":"2023-07-14T11:55:00.291171+0000","last_became_active":"2023-07-14T11:54:58.634281+0000","last_became_peered":"2023-07-14T11:54:58.634281+0000","last_unstale":"2023-07-14T11:55:00.291171+0000","last_undegraded":"2023-07-14T11:55:00.291171+0000","last_fullsized":"2023-07-14T11:55:00.291171+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:22:01.665793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782615+0000","last_change":"2023-07-14T11:54:58.649577+0000","last_active":"2023-07-14T11:55:00.782615+0000","last_peered":"2023-07-14T11:55:00.782615+0000","last_clean":"2023-07-14T11:55:00.782615+0000","last_became_active":"2023-07-14T11:54:58.649048+0000","last_became_peered":"2023-07-14T11:54:58.649048+0000","last_unstale":"2023-07-14T11:55:00.782615+0000","last_undegraded":"2023-07-14T11:55:00.782615+0000","last_fullsized":"2023-07-14T11:55:00.782615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:16:06.190084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.630233+0000","last_change":"2023-07-14T11:54:58.640369+0000","last_active":"2023-07-14T11:54:59.630233+0000","last_peered":"2023-07-14T11:54:59.630233+0000","last_clean":"2023-07-14T11:54:59.630233+0000","last_became_active":"2023-07-14T11:54:58.640049+0000","last_became_peered":"2023-07-14T11:54:58.640049+0000","last_unstale":"2023-07-14T11:54:59.630233+0000","last_undegraded":"2023-07-14T11:54:59.630233+0000","last_fullsized":"2023-07-14T11:54:59.630233+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:19:22.661093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.630076+0000","last_change":"2023-07-14T11:54:58.637243+0000","last_active":"2023-07-14T11:54:59.630076+0000","last_peered":"2023-07-14T11:54:59.630076+0000","last_clean":"2023-07-14T11:54:59.630076+0000","last_became_active":"2023-07-14T11:54:58.636990+0000","last_became_peered":"2023-07-14T11:54:58.636990+0000","last_unstale":"2023-07-14T11:54:59.630076+0000","last_undegraded":"2023-07-14T11:54:59.630076+0000","last_fullsized":"2023-07-14T11:54:59.630076+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:57:20.686316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.631008+0000","last_change":"2023-07-14T11:54:58.643343+0000","last_active":"2023-07-14T11:54:59.631008+0000","last_peered":"2023-07-14T11:54:59.631008+0000","last_clean":"2023-07-14T11:54:59.631008+0000","last_became_active":"2023-07-14T11:54:58.642999+0000","last_became_peered":"2023-07-14T11:54:58.642999+0000","last_unstale":"2023-07-14T11:54:59.631008+0000","last_undegraded":"2023-07-14T11:54:59.631008+0000","last_fullsized":"2023-07-14T11:54:59.631008+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:01:34.531440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.627938+0000","last_change":"2023-07-14T11:54:58.639362+0000","last_active":"2023-07-14T11:54:59.627938+0000","last_peered":"2023-07-14T11:54:59.627938+0000","last_clean":"2023-07-14T11:54:59.627938+0000","last_became_active":"2023-07-14T11:54:58.639045+0000","last_became_peered":"2023-07-14T11:54:58.639045+0000","last_unstale":"2023-07-14T11:54:59.627938+0000","last_undegraded":"2023-07-14T11:54:59.627938+0000","last_fullsized":"2023-07-14T11:54:59.627938+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:57:48.273317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296556+0000","last_change":"2023-07-14T11:54:58.643049+0000","last_active":"2023-07-14T11:55:00.296556+0000","last_peered":"2023-07-14T11:55:00.296556+0000","last_clean":"2023-07-14T11:55:00.296556+0000","last_became_active":"2023-07-14T11:54:58.642784+0000","last_became_peered":"2023-07-14T11:54:58.642784+0000","last_unstale":"2023-07-14T11:55:00.296556+0000","last_undegraded":"2023-07-14T11:55:00.296556+0000","last_fullsized":"2023-07-14T11:55:00.296556+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:05:07.290823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.650779+0000","last_change":"2023-07-14T11:54:58.646381+0000","last_active":"2023-07-14T11:54:58.650779+0000","last_peered":"2023-07-14T11:54:58.650779+0000","last_clean":"2023-07-14T11:54:58.650779+0000","last_became_active":"2023-07-14T11:54:58.646121+0000","last_became_peered":"2023-07-14T11:54:58.646121+0000","last_unstale":"2023-07-14T11:54:58.650779+0000","last_undegraded":"2023-07-14T11:54:58.650779+0000","last_fullsized":"2023-07-14T11:54:58.650779+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:19:33.144908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.630424+0000","last_change":"2023-07-14T11:54:58.643178+0000","last_active":"2023-07-14T11:54:59.630424+0000","last_peered":"2023-07-14T11:54:59.630424+0000","last_clean":"2023-07-14T11:54:59.630424+0000","last_became_active":"2023-07-14T11:54:58.642846+0000","last_became_peered":"2023-07-14T11:54:58.642846+0000","last_unstale":"2023-07-14T11:54:59.630424+0000","last_undegraded":"2023-07-14T11:54:59.630424+0000","last_fullsized":"2023-07-14T11:54:59.630424+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:49:42.397101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296394+0000","last_change":"2023-07-14T11:54:58.650834+0000","last_active":"2023-07-14T11:55:00.296394+0000","last_peered":"2023-07-14T11:55:00.296394+0000","last_clean":"2023-07-14T11:55:00.296394+0000","last_became_active":"2023-07-14T11:54:58.650464+0000","last_became_peered":"2023-07-14T11:54:58.650464+0000","last_unstale":"2023-07-14T11:55:00.296394+0000","last_undegraded":"2023-07-14T11:55:00.296394+0000","last_fullsized":"2023-07-14T11:55:00.296394+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:42.897417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.290850+0000","last_change":"2023-07-14T11:54:58.642158+0000","last_active":"2023-07-14T11:55:00.290850+0000","last_peered":"2023-07-14T11:55:00.290850+0000","last_clean":"2023-07-14T11:55:00.290850+0000","last_became_active":"2023-07-14T11:54:58.641871+0000","last_became_peered":"2023-07-14T11:54:58.641871+0000","last_unstale":"2023-07-14T11:55:00.290850+0000","last_undegraded":"2023-07-14T11:55:00.290850+0000","last_fullsized":"2023-07-14T11:55:00.290850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:06:31.160677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.627708+0000","last_change":"2023-07-14T11:54:58.642071+0000","last_active":"2023-07-14T11:54:59.627708+0000","last_peered":"2023-07-14T11:54:59.627708+0000","last_clean":"2023-07-14T11:54:59.627708+0000","last_became_active":"2023-07-14T11:54:58.641800+0000","last_became_peered":"2023-07-14T11:54:58.641800+0000","last_unstale":"2023-07-14T11:54:59.627708+0000","last_undegraded":"2023-07-14T11:54:59.627708+0000","last_fullsized":"2023-07-14T11:54:59.627708+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:49:42.397101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629692+0000","last_change":"2023-07-14T11:54:58.641609+0000","last_active":"2023-07-14T11:54:59.629692+0000","last_peered":"2023-07-14T11:54:59.629692+0000","last_clean":"2023-07-14T11:54:59.629692+0000","last_became_active":"2023-07-14T11:54:58.641357+0000","last_became_peered":"2023-07-14T11:54:58.641357+0000","last_unstale":"2023-07-14T11:54:59.629692+0000","last_undegraded":"2023-07-14T11:54:59.629692+0000","last_fullsized":"2023-07-14T11:54:59.629692+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:53:46.284892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"41'4","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291783+0000","last_change":"2023-07-14T11:54:58.159508+0000","last_active":"2023-07-14T11:55:00.291783+0000","last_peered":"2023-07-14T11:55:00.291783+0000","last_clean":"2023-07-14T11:55:00.291783+0000","last_became_active":"2023-07-14T11:54:58.158645+0000","last_became_peered":"2023-07-14T11:54:58.158645+0000","last_unstale":"2023-07-14T11:55:00.291783+0000","last_undegraded":"2023-07-14T11:55:00.291783+0000","last_fullsized":"2023-07-14T11:55:00.291783+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:52.474136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.631025+0000","last_change":"2023-07-14T11:54:58.646592+0000","last_active":"2023-07-14T11:54:59.631025+0000","last_peered":"2023-07-14T11:54:59.631025+0000","last_clean":"2023-07-14T11:54:59.631025+0000","last_became_active":"2023-07-14T11:54:58.646286+0000","last_became_peered":"2023-07-14T11:54:58.646286+0000","last_unstale":"2023-07-14T11:54:59.631025+0000","last_undegraded":"2023-07-14T11:54:59.631025+0000","last_fullsized":"2023-07-14T11:54:59.631025+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:21.925534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296799+0000","last_change":"2023-07-14T11:54:58.159113+0000","last_active":"2023-07-14T11:55:00.296799+0000","last_peered":"2023-07-14T11:55:00.296799+0000","last_clean":"2023-07-14T11:55:00.296799+0000","last_became_active":"2023-07-14T11:54:58.158798+0000","last_became_peered":"2023-07-14T11:54:58.158798+0000","last_unstale":"2023-07-14T11:55:00.296799+0000","last_undegraded":"2023-07-14T11:55:00.296799+0000","last_fullsized":"2023-07-14T11:55:00.296799+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:50:49.327824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.648970+0000","last_change":"2023-07-14T11:54:58.641877+0000","last_active":"2023-07-14T11:54:58.648970+0000","last_peered":"2023-07-14T11:54:58.648970+0000","last_clean":"2023-07-14T11:54:58.648970+0000","last_became_active":"2023-07-14T11:54:58.641599+0000","last_became_peered":"2023-07-14T11:54:58.641599+0000","last_unstale":"2023-07-14T11:54:58.648970+0000","last_undegraded":"2023-07-14T11:54:58.648970+0000","last_fullsized":"2023-07-14T11:54:58.648970+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:55:09.545959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629755+0000","last_change":"2023-07-14T11:54:58.167470+0000","last_active":"2023-07-14T11:54:59.629755+0000","last_peered":"2023-07-14T11:54:59.629755+0000","last_clean":"2023-07-14T11:54:59.629755+0000","last_became_active":"2023-07-14T11:54:58.167193+0000","last_became_peered":"2023-07-14T11:54:58.167193+0000","last_unstale":"2023-07-14T11:54:59.629755+0000","last_undegraded":"2023-07-14T11:54:59.629755+0000","last_fullsized":"2023-07-14T11:54:59.629755+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:29:49.187978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629691+0000","last_change":"2023-07-14T11:54:58.642675+0000","last_active":"2023-07-14T11:54:59.629691+0000","last_peered":"2023-07-14T11:54:59.629691+0000","last_clean":"2023-07-14T11:54:59.629691+0000","last_became_active":"2023-07-14T11:54:58.642175+0000","last_became_peered":"2023-07-14T11:54:58.642175+0000","last_unstale":"2023-07-14T11:54:59.629691+0000","last_undegraded":"2023-07-14T11:54:59.629691+0000","last_fullsized":"2023-07-14T11:54:59.629691+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:40:21.203057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901470+0000","last_change":"2023-07-14T11:54:58.170311+0000","last_active":"2023-07-14T11:54:59.901470+0000","last_peered":"2023-07-14T11:54:59.901470+0000","last_clean":"2023-07-14T11:54:59.901470+0000","last_became_active":"2023-07-14T11:54:58.170035+0000","last_became_peered":"2023-07-14T11:54:58.170035+0000","last_unstale":"2023-07-14T11:54:59.901470+0000","last_undegraded":"2023-07-14T11:54:59.901470+0000","last_fullsized":"2023-07-14T11:54:59.901470+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:40:20.187965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296407+0000","last_change":"2023-07-14T11:54:58.643745+0000","last_active":"2023-07-14T11:55:00.296407+0000","last_peered":"2023-07-14T11:55:00.296407+0000","last_clean":"2023-07-14T11:55:00.296407+0000","last_became_active":"2023-07-14T11:54:58.643473+0000","last_became_peered":"2023-07-14T11:54:58.643473+0000","last_unstale":"2023-07-14T11:55:00.296407+0000","last_undegraded":"2023-07-14T11:55:00.296407+0000","last_fullsized":"2023-07-14T11:55:00.296407+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:38:34.474674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291473+0000","last_change":"2023-07-14T11:54:58.165926+0000","last_active":"2023-07-14T11:55:00.291473+0000","last_peered":"2023-07-14T11:55:00.291473+0000","last_clean":"2023-07-14T11:55:00.291473+0000","last_became_active":"2023-07-14T11:54:58.165521+0000","last_became_peered":"2023-07-14T11:54:58.165521+0000","last_unstale":"2023-07-14T11:55:00.291473+0000","last_undegraded":"2023-07-14T11:55:00.291473+0000","last_fullsized":"2023-07-14T11:55:00.291473+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:38:29.398816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782085+0000","last_change":"2023-07-14T11:54:58.655258+0000","last_active":"2023-07-14T11:55:00.782085+0000","last_peered":"2023-07-14T11:55:00.782085+0000","last_clean":"2023-07-14T11:55:00.782085+0000","last_became_active":"2023-07-14T11:54:58.654324+0000","last_became_peered":"2023-07-14T11:54:58.654324+0000","last_unstale":"2023-07-14T11:55:00.782085+0000","last_undegraded":"2023-07-14T11:55:00.782085+0000","last_fullsized":"2023-07-14T11:55:00.782085+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:14:01.595650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296697+0000","last_change":"2023-07-14T11:54:58.163946+0000","last_active":"2023-07-14T11:55:00.296697+0000","last_peered":"2023-07-14T11:55:00.296697+0000","last_clean":"2023-07-14T11:55:00.296697+0000","last_became_active":"2023-07-14T11:54:58.163671+0000","last_became_peered":"2023-07-14T11:54:58.163671+0000","last_unstale":"2023-07-14T11:55:00.296697+0000","last_undegraded":"2023-07-14T11:55:00.296697+0000","last_fullsized":"2023-07-14T11:55:00.296697+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:18:00.737557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782269+0000","last_change":"2023-07-14T11:54:58.648406+0000","last_active":"2023-07-14T11:55:00.782269+0000","last_peered":"2023-07-14T11:55:00.782269+0000","last_clean":"2023-07-14T11:55:00.782269+0000","last_became_active":"2023-07-14T11:54:58.648063+0000","last_became_peered":"2023-07-14T11:54:58.648063+0000","last_unstale":"2023-07-14T11:55:00.782269+0000","last_undegraded":"2023-07-14T11:55:00.782269+0000","last_fullsized":"2023-07-14T11:55:00.782269+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:49:16.280244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.635497+0000","last_change":"2023-07-14T11:54:58.162038+0000","last_active":"2023-07-14T11:54:58.635497+0000","last_peered":"2023-07-14T11:54:58.635497+0000","last_clean":"2023-07-14T11:54:58.635497+0000","last_became_active":"2023-07-14T11:54:58.161713+0000","last_became_peered":"2023-07-14T11:54:58.161713+0000","last_unstale":"2023-07-14T11:54:58.635497+0000","last_undegraded":"2023-07-14T11:54:58.635497+0000","last_fullsized":"2023-07-14T11:54:58.635497+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:58:35.195007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629969+0000","last_change":"2023-07-14T11:54:58.641004+0000","last_active":"2023-07-14T11:54:59.629969+0000","last_peered":"2023-07-14T11:54:59.629969+0000","last_clean":"2023-07-14T11:54:59.629969+0000","last_became_active":"2023-07-14T11:54:58.640715+0000","last_became_peered":"2023-07-14T11:54:58.640715+0000","last_unstale":"2023-07-14T11:54:59.629969+0000","last_undegraded":"2023-07-14T11:54:59.629969+0000","last_fullsized":"2023-07-14T11:54:59.629969+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:50:32.746766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782559+0000","last_change":"2023-07-14T11:54:58.163012+0000","last_active":"2023-07-14T11:55:00.782559+0000","last_peered":"2023-07-14T11:55:00.782559+0000","last_clean":"2023-07-14T11:55:00.782559+0000","last_became_active":"2023-07-14T11:54:58.162424+0000","last_became_peered":"2023-07-14T11:54:58.162424+0000","last_unstale":"2023-07-14T11:55:00.782559+0000","last_undegraded":"2023-07-14T11:55:00.782559+0000","last_fullsized":"2023-07-14T11:55:00.782559+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:59:32.927448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296649+0000","last_change":"2023-07-14T11:54:58.652546+0000","last_active":"2023-07-14T11:55:00.296649+0000","last_peered":"2023-07-14T11:55:00.296649+0000","last_clean":"2023-07-14T11:55:00.296649+0000","last_became_active":"2023-07-14T11:54:58.652196+0000","last_became_peered":"2023-07-14T11:54:58.652196+0000","last_unstale":"2023-07-14T11:55:00.296649+0000","last_undegraded":"2023-07-14T11:55:00.296649+0000","last_fullsized":"2023-07-14T11:55:00.296649+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:42:18.814080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.297104+0000","last_change":"2023-07-14T11:54:58.165179+0000","last_active":"2023-07-14T11:55:00.297104+0000","last_peered":"2023-07-14T11:55:00.297104+0000","last_clean":"2023-07-14T11:55:00.297104+0000","last_became_active":"2023-07-14T11:54:58.164894+0000","last_became_peered":"2023-07-14T11:54:58.164894+0000","last_unstale":"2023-07-14T11:55:00.297104+0000","last_undegraded":"2023-07-14T11:55:00.297104+0000","last_fullsized":"2023-07-14T11:55:00.297104+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:08:47.514744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901962+0000","last_change":"2023-07-14T11:54:58.643098+0000","last_active":"2023-07-14T11:54:59.901962+0000","last_peered":"2023-07-14T11:54:59.901962+0000","last_clean":"2023-07-14T11:54:59.901962+0000","last_became_active":"2023-07-14T11:54:58.642779+0000","last_became_peered":"2023-07-14T11:54:58.642779+0000","last_unstale":"2023-07-14T11:54:59.901962+0000","last_undegraded":"2023-07-14T11:54:59.901962+0000","last_fullsized":"2023-07-14T11:54:59.901962+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:14:01.595650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.783116+0000","last_change":"2023-07-14T11:54:58.167572+0000","last_active":"2023-07-14T11:55:00.783116+0000","last_peered":"2023-07-14T11:55:00.783116+0000","last_clean":"2023-07-14T11:55:00.783116+0000","last_became_active":"2023-07-14T11:54:58.167255+0000","last_became_peered":"2023-07-14T11:54:58.167255+0000","last_unstale":"2023-07-14T11:55:00.783116+0000","last_undegraded":"2023-07-14T11:55:00.783116+0000","last_fullsized":"2023-07-14T11:55:00.783116+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:15:07.280920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291456+0000","last_change":"2023-07-14T11:54:58.645141+0000","last_active":"2023-07-14T11:55:00.291456+0000","last_peered":"2023-07-14T11:55:00.291456+0000","last_clean":"2023-07-14T11:55:00.291456+0000","last_became_active":"2023-07-14T11:54:58.644876+0000","last_became_peered":"2023-07-14T11:54:58.644876+0000","last_unstale":"2023-07-14T11:55:00.291456+0000","last_undegraded":"2023-07-14T11:55:00.291456+0000","last_fullsized":"2023-07-14T11:55:00.291456+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:39:38.147706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"41'3","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901891+0000","last_change":"2023-07-14T11:54:58.167766+0000","last_active":"2023-07-14T11:54:59.901891+0000","last_peered":"2023-07-14T11:54:59.901891+0000","last_clean":"2023-07-14T11:54:59.901891+0000","last_became_active":"2023-07-14T11:54:58.167390+0000","last_became_peered":"2023-07-14T11:54:58.167390+0000","last_unstale":"2023-07-14T11:54:59.901891+0000","last_undegraded":"2023-07-14T11:54:59.901891+0000","last_fullsized":"2023-07-14T11:54:59.901891+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:13.512521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901574+0000","last_change":"2023-07-14T11:54:58.642997+0000","last_active":"2023-07-14T11:54:59.901574+0000","last_peered":"2023-07-14T11:54:59.901574+0000","last_clean":"2023-07-14T11:54:59.901574+0000","last_became_active":"2023-07-14T11:54:58.642645+0000","last_became_peered":"2023-07-14T11:54:58.642645+0000","last_unstale":"2023-07-14T11:54:59.901574+0000","last_undegraded":"2023-07-14T11:54:59.901574+0000","last_fullsized":"2023-07-14T11:54:59.901574+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:26:10.479891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"41'6","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.628490+0000","last_change":"2023-07-14T11:54:58.166850+0000","last_active":"2023-07-14T11:54:59.628490+0000","last_peered":"2023-07-14T11:54:59.628490+0000","last_clean":"2023-07-14T11:54:59.628490+0000","last_became_active":"2023-07-14T11:54:58.166569+0000","last_became_peered":"2023-07-14T11:54:58.166569+0000","last_unstale":"2023-07-14T11:54:59.628490+0000","last_undegraded":"2023-07-14T11:54:59.628490+0000","last_fullsized":"2023-07-14T11:54:59.628490+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:00:20.257845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.630463+0000","last_change":"2023-07-14T11:54:58.651466+0000","last_active":"2023-07-14T11:54:59.630463+0000","last_peered":"2023-07-14T11:54:59.630463+0000","last_clean":"2023-07-14T11:54:59.630463+0000","last_became_active":"2023-07-14T11:54:58.651173+0000","last_became_peered":"2023-07-14T11:54:58.651173+0000","last_unstale":"2023-07-14T11:54:59.630463+0000","last_undegraded":"2023-07-14T11:54:59.630463+0000","last_fullsized":"2023-07-14T11:54:59.630463+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:52:02.273145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291371+0000","last_change":"2023-07-14T11:54:58.169271+0000","last_active":"2023-07-14T11:55:00.291371+0000","last_peered":"2023-07-14T11:55:00.291371+0000","last_clean":"2023-07-14T11:55:00.291371+0000","last_became_active":"2023-07-14T11:54:58.168963+0000","last_became_peered":"2023-07-14T11:54:58.168963+0000","last_unstale":"2023-07-14T11:55:00.291371+0000","last_undegraded":"2023-07-14T11:55:00.291371+0000","last_fullsized":"2023-07-14T11:55:00.291371+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:42:03.611134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.781922+0000","last_change":"2023-07-14T11:54:58.652579+0000","last_active":"2023-07-14T11:55:00.781922+0000","last_peered":"2023-07-14T11:55:00.781922+0000","last_clean":"2023-07-14T11:55:00.781922+0000","last_became_active":"2023-07-14T11:54:58.652293+0000","last_became_peered":"2023-07-14T11:54:58.652293+0000","last_unstale":"2023-07-14T11:55:00.781922+0000","last_undegraded":"2023-07-14T11:55:00.781922+0000","last_fullsized":"2023-07-14T11:55:00.781922+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:26:10.479891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.900956+0000","last_change":"2023-07-14T11:54:58.171084+0000","last_active":"2023-07-14T11:54:59.900956+0000","last_peered":"2023-07-14T11:54:59.900956+0000","last_clean":"2023-07-14T11:54:59.900956+0000","last_became_active":"2023-07-14T11:54:58.170771+0000","last_became_peered":"2023-07-14T11:54:58.170771+0000","last_unstale":"2023-07-14T11:54:59.900956+0000","last_undegraded":"2023-07-14T11:54:59.900956+0000","last_fullsized":"2023-07-14T11:54:59.900956+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:56:22.253649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.290651+0000","last_change":"2023-07-14T11:54:58.645227+0000","last_active":"2023-07-14T11:55:00.290651+0000","last_peered":"2023-07-14T11:55:00.290651+0000","last_clean":"2023-07-14T11:55:00.290651+0000","last_became_active":"2023-07-14T11:54:58.644841+0000","last_became_peered":"2023-07-14T11:54:58.644841+0000","last_unstale":"2023-07-14T11:55:00.290651+0000","last_undegraded":"2023-07-14T11:55:00.290651+0000","last_fullsized":"2023-07-14T11:55:00.290651+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:09:45.008219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291143+0000","last_change":"2023-07-14T11:54:58.168874+0000","last_active":"2023-07-14T11:55:00.291143+0000","last_peered":"2023-07-14T11:55:00.291143+0000","last_clean":"2023-07-14T11:55:00.291143+0000","last_became_active":"2023-07-14T11:54:58.168582+0000","last_became_peered":"2023-07-14T11:54:58.168582+0000","last_unstale":"2023-07-14T11:55:00.291143+0000","last_undegraded":"2023-07-14T11:55:00.291143+0000","last_fullsized":"2023-07-14T11:55:00.291143+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:05:25.206219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901553+0000","last_change":"2023-07-14T11:54:58.646464+0000","last_active":"2023-07-14T11:54:59.901553+0000","last_peered":"2023-07-14T11:54:59.901553+0000","last_clean":"2023-07-14T11:54:59.901553+0000","last_became_active":"2023-07-14T11:54:58.646167+0000","last_became_peered":"2023-07-14T11:54:58.646167+0000","last_unstale":"2023-07-14T11:54:59.901553+0000","last_undegraded":"2023-07-14T11:54:59.901553+0000","last_fullsized":"2023-07-14T11:54:59.901553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:29:50.203069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"41'8","reported_seq":21,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782843+0000","last_change":"2023-07-14T11:54:58.164070+0000","last_active":"2023-07-14T11:55:00.782843+0000","last_peered":"2023-07-14T11:55:00.782843+0000","last_clean":"2023-07-14T11:55:00.782843+0000","last_became_active":"2023-07-14T11:54:58.163791+0000","last_became_peered":"2023-07-14T11:54:58.163791+0000","last_unstale":"2023-07-14T11:55:00.782843+0000","last_undegraded":"2023-07-14T11:55:00.782843+0000","last_fullsized":"2023-07-14T11:55:00.782843+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:54:13.946861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.628253+0000","last_change":"2023-07-14T11:54:58.643475+0000","last_active":"2023-07-14T11:54:59.628253+0000","last_peered":"2023-07-14T11:54:59.628253+0000","last_clean":"2023-07-14T11:54:59.628253+0000","last_became_active":"2023-07-14T11:54:58.643228+0000","last_became_peered":"2023-07-14T11:54:58.643228+0000","last_unstale":"2023-07-14T11:54:59.628253+0000","last_undegraded":"2023-07-14T11:54:59.628253+0000","last_fullsized":"2023-07-14T11:54:59.628253+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:03:21.925534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.628200+0000","last_change":"2023-07-14T11:54:58.171865+0000","last_active":"2023-07-14T11:54:59.628200+0000","last_peered":"2023-07-14T11:54:59.628200+0000","last_clean":"2023-07-14T11:54:59.628200+0000","last_became_active":"2023-07-14T11:54:58.171553+0000","last_became_peered":"2023-07-14T11:54:58.171553+0000","last_unstale":"2023-07-14T11:54:59.628200+0000","last_undegraded":"2023-07-14T11:54:59.628200+0000","last_fullsized":"2023-07-14T11:54:59.628200+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:58:28.547775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782865+0000","last_change":"2023-07-14T11:54:58.656460+0000","last_active":"2023-07-14T11:55:00.782865+0000","last_peered":"2023-07-14T11:55:00.782865+0000","last_clean":"2023-07-14T11:55:00.782865+0000","last_became_active":"2023-07-14T11:54:58.655089+0000","last_became_peered":"2023-07-14T11:54:58.655089+0000","last_unstale":"2023-07-14T11:55:00.782865+0000","last_undegraded":"2023-07-14T11:55:00.782865+0000","last_fullsized":"2023-07-14T11:55:00.782865+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:09:45.008219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.297009+0000","last_change":"2023-07-14T11:54:58.165160+0000","last_active":"2023-07-14T11:55:00.297009+0000","last_peered":"2023-07-14T11:55:00.297009+0000","last_clean":"2023-07-14T11:55:00.297009+0000","last_became_active":"2023-07-14T11:54:58.164863+0000","last_became_peered":"2023-07-14T11:54:58.164863+0000","last_unstale":"2023-07-14T11:55:00.297009+0000","last_undegraded":"2023-07-14T11:55:00.297009+0000","last_fullsized":"2023-07-14T11:55:00.297009+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:14:40.794990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.630575+0000","last_change":"2023-07-14T11:54:58.645608+0000","last_active":"2023-07-14T11:54:59.630575+0000","last_peered":"2023-07-14T11:54:59.630575+0000","last_clean":"2023-07-14T11:54:59.630575+0000","last_became_active":"2023-07-14T11:54:58.645310+0000","last_became_peered":"2023-07-14T11:54:58.645310+0000","last_unstale":"2023-07-14T11:54:59.630575+0000","last_undegraded":"2023-07-14T11:54:59.630575+0000","last_fullsized":"2023-07-14T11:54:59.630575+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:57:48.273317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"41'1","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.169305+0000","last_change":"2023-07-14T11:54:58.162911+0000","last_active":"2023-07-14T11:54:59.169305+0000","last_peered":"2023-07-14T11:54:59.169305+0000","last_clean":"2023-07-14T11:54:59.169305+0000","last_became_active":"2023-07-14T11:54:58.162640+0000","last_became_peered":"2023-07-14T11:54:58.162640+0000","last_unstale":"2023-07-14T11:54:59.169305+0000","last_undegraded":"2023-07-14T11:54:59.169305+0000","last_fullsized":"2023-07-14T11:54:59.169305+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:54:13.946861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.630797+0000","last_change":"2023-07-14T11:54:58.644821+0000","last_active":"2023-07-14T11:54:59.630797+0000","last_peered":"2023-07-14T11:54:59.630797+0000","last_clean":"2023-07-14T11:54:59.630797+0000","last_became_active":"2023-07-14T11:54:58.644337+0000","last_became_peered":"2023-07-14T11:54:58.644337+0000","last_unstale":"2023-07-14T11:54:59.630797+0000","last_undegraded":"2023-07-14T11:54:59.630797+0000","last_fullsized":"2023-07-14T11:54:59.630797+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:58:29.562867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291709+0000","last_change":"2023-07-14T11:54:58.159296+0000","last_active":"2023-07-14T11:55:00.291709+0000","last_peered":"2023-07-14T11:55:00.291709+0000","last_clean":"2023-07-14T11:55:00.291709+0000","last_became_active":"2023-07-14T11:54:58.159030+0000","last_became_peered":"2023-07-14T11:54:58.159030+0000","last_unstale":"2023-07-14T11:55:00.291709+0000","last_undegraded":"2023-07-14T11:55:00.291709+0000","last_fullsized":"2023-07-14T11:55:00.291709+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:14:24.554179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.290711+0000","last_change":"2023-07-14T11:54:58.632702+0000","last_active":"2023-07-14T11:55:00.290711+0000","last_peered":"2023-07-14T11:55:00.290711+0000","last_clean":"2023-07-14T11:55:00.290711+0000","last_became_active":"2023-07-14T11:54:58.632450+0000","last_became_peered":"2023-07-14T11:54:58.632450+0000","last_unstale":"2023-07-14T11:55:00.290711+0000","last_undegraded":"2023-07-14T11:55:00.290711+0000","last_fullsized":"2023-07-14T11:55:00.290711+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:58:36.210098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291610+0000","last_change":"2023-07-14T11:54:58.165758+0000","last_active":"2023-07-14T11:55:00.291610+0000","last_peered":"2023-07-14T11:55:00.291610+0000","last_clean":"2023-07-14T11:55:00.291610+0000","last_became_active":"2023-07-14T11:54:58.165198+0000","last_became_peered":"2023-07-14T11:54:58.165198+0000","last_unstale":"2023-07-14T11:55:00.291610+0000","last_undegraded":"2023-07-14T11:55:00.291610+0000","last_fullsized":"2023-07-14T11:55:00.291610+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:47.597308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629461+0000","last_change":"2023-07-14T11:54:58.644367+0000","last_active":"2023-07-14T11:54:59.629461+0000","last_peered":"2023-07-14T11:54:59.629461+0000","last_clean":"2023-07-14T11:54:59.629461+0000","last_became_active":"2023-07-14T11:54:58.644094+0000","last_became_peered":"2023-07-14T11:54:58.644094+0000","last_unstale":"2023-07-14T11:54:59.629461+0000","last_undegraded":"2023-07-14T11:54:59.629461+0000","last_fullsized":"2023-07-14T11:54:59.629461+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:31:42.113951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.633458+0000","last_change":"2023-07-14T11:54:58.166448+0000","last_active":"2023-07-14T11:54:58.633458+0000","last_peered":"2023-07-14T11:54:58.633458+0000","last_clean":"2023-07-14T11:54:58.633458+0000","last_became_active":"2023-07-14T11:54:58.166048+0000","last_became_peered":"2023-07-14T11:54:58.166048+0000","last_unstale":"2023-07-14T11:54:58.633458+0000","last_undegraded":"2023-07-14T11:54:58.633458+0000","last_fullsized":"2023-07-14T11:54:58.633458+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:14:00.580558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.627781+0000","last_change":"2023-07-14T11:54:58.641466+0000","last_active":"2023-07-14T11:54:59.627781+0000","last_peered":"2023-07-14T11:54:59.627781+0000","last_clean":"2023-07-14T11:54:59.627781+0000","last_became_active":"2023-07-14T11:54:58.641183+0000","last_became_peered":"2023-07-14T11:54:58.641183+0000","last_unstale":"2023-07-14T11:54:59.627781+0000","last_undegraded":"2023-07-14T11:54:59.627781+0000","last_fullsized":"2023-07-14T11:54:59.627781+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:52:02.273145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782010+0000","last_change":"2023-07-14T11:54:58.167804+0000","last_active":"2023-07-14T11:55:00.782010+0000","last_peered":"2023-07-14T11:55:00.782010+0000","last_clean":"2023-07-14T11:55:00.782010+0000","last_became_active":"2023-07-14T11:54:58.167503+0000","last_became_peered":"2023-07-14T11:54:58.167503+0000","last_unstale":"2023-07-14T11:55:00.782010+0000","last_undegraded":"2023-07-14T11:55:00.782010+0000","last_fullsized":"2023-07-14T11:55:00.782010+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:48:00.910272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.781906+0000","last_change":"2023-07-14T11:54:58.654651+0000","last_active":"2023-07-14T11:55:00.781906+0000","last_peered":"2023-07-14T11:55:00.781906+0000","last_clean":"2023-07-14T11:55:00.781906+0000","last_became_active":"2023-07-14T11:54:58.654342+0000","last_became_peered":"2023-07-14T11:54:58.654342+0000","last_unstale":"2023-07-14T11:55:00.781906+0000","last_undegraded":"2023-07-14T11:55:00.781906+0000","last_fullsized":"2023-07-14T11:55:00.781906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:29:50.203069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.900909+0000","last_change":"2023-07-14T11:54:58.168907+0000","last_active":"2023-07-14T11:54:59.900909+0000","last_peered":"2023-07-14T11:54:59.900909+0000","last_clean":"2023-07-14T11:54:59.900909+0000","last_became_active":"2023-07-14T11:54:58.168601+0000","last_became_peered":"2023-07-14T11:54:58.168601+0000","last_unstale":"2023-07-14T11:54:59.900909+0000","last_undegraded":"2023-07-14T11:54:59.900909+0000","last_fullsized":"2023-07-14T11:54:59.900909+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:15:00.073097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.290852+0000","last_change":"2023-07-14T11:54:58.641931+0000","last_active":"2023-07-14T11:55:00.290852+0000","last_peered":"2023-07-14T11:55:00.290852+0000","last_clean":"2023-07-14T11:55:00.290852+0000","last_became_active":"2023-07-14T11:54:58.641574+0000","last_became_peered":"2023-07-14T11:54:58.641574+0000","last_unstale":"2023-07-14T11:55:00.290852+0000","last_undegraded":"2023-07-14T11:55:00.290852+0000","last_fullsized":"2023-07-14T11:55:00.290852+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:15:08.296012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901436+0000","last_change":"2023-07-14T11:54:58.167698+0000","last_active":"2023-07-14T11:54:59.901436+0000","last_peered":"2023-07-14T11:54:59.901436+0000","last_clean":"2023-07-14T11:54:59.901436+0000","last_became_active":"2023-07-14T11:54:58.167388+0000","last_became_peered":"2023-07-14T11:54:58.167388+0000","last_unstale":"2023-07-14T11:54:59.901436+0000","last_undegraded":"2023-07-14T11:54:59.901436+0000","last_fullsized":"2023-07-14T11:54:59.901436+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:29:44.534250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.631145+0000","last_change":"2023-07-14T11:53:09.145201+0000","last_active":"2023-07-14T11:54:59.631145+0000","last_peered":"2023-07-14T11:54:59.631145+0000","last_clean":"2023-07-14T11:54:59.631145+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:54:59.631145+0000","last_undegraded":"2023-07-14T11:54:59.631145+0000","last_fullsized":"2023-07-14T11:54:59.631145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:19:50.489223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.291167+0000","last_change":"2023-07-14T11:54:58.645165+0000","last_active":"2023-07-14T11:55:00.291167+0000","last_peered":"2023-07-14T11:55:00.291167+0000","last_clean":"2023-07-14T11:55:00.291167+0000","last_became_active":"2023-07-14T11:54:58.644831+0000","last_became_peered":"2023-07-14T11:54:58.644831+0000","last_unstale":"2023-07-14T11:55:00.291167+0000","last_undegraded":"2023-07-14T11:55:00.291167+0000","last_fullsized":"2023-07-14T11:55:00.291167+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:38:13.006723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296772+0000","last_change":"2023-07-14T11:54:58.169443+0000","last_active":"2023-07-14T11:55:00.296772+0000","last_peered":"2023-07-14T11:55:00.296772+0000","last_clean":"2023-07-14T11:55:00.296772+0000","last_became_active":"2023-07-14T11:54:58.169079+0000","last_became_peered":"2023-07-14T11:54:58.169079+0000","last_unstale":"2023-07-14T11:55:00.296772+0000","last_undegraded":"2023-07-14T11:55:00.296772+0000","last_fullsized":"2023-07-14T11:55:00.296772+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:02:11.868139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629758+0000","last_change":"2023-07-14T11:53:09.146150+0000","last_active":"2023-07-14T11:54:59.629758+0000","last_peered":"2023-07-14T11:54:59.629758+0000","last_clean":"2023-07-14T11:54:59.629758+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:54:59.629758+0000","last_undegraded":"2023-07-14T11:54:59.629758+0000","last_fullsized":"2023-07-14T11:54:59.629758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:40:41.420401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.648985+0000","last_change":"2023-07-14T11:54:58.642131+0000","last_active":"2023-07-14T11:54:58.648985+0000","last_peered":"2023-07-14T11:54:58.648985+0000","last_clean":"2023-07-14T11:54:58.648985+0000","last_became_active":"2023-07-14T11:54:58.641858+0000","last_became_peered":"2023-07-14T11:54:58.641858+0000","last_unstale":"2023-07-14T11:54:58.648985+0000","last_undegraded":"2023-07-14T11:54:58.648985+0000","last_fullsized":"2023-07-14T11:54:58.648985+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:09.069253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629413+0000","last_change":"2023-07-14T11:54:58.175137+0000","last_active":"2023-07-14T11:54:59.629413+0000","last_peered":"2023-07-14T11:54:59.629413+0000","last_clean":"2023-07-14T11:54:59.629413+0000","last_became_active":"2023-07-14T11:54:58.174914+0000","last_became_peered":"2023-07-14T11:54:58.174914+0000","last_unstale":"2023-07-14T11:54:59.629413+0000","last_undegraded":"2023-07-14T11:54:59.629413+0000","last_fullsized":"2023-07-14T11:54:59.629413+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:56:22.253649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.902055+0000","last_change":"2023-07-14T11:53:09.435876+0000","last_active":"2023-07-14T11:54:59.902055+0000","last_peered":"2023-07-14T11:54:59.902055+0000","last_clean":"2023-07-14T11:54:59.902055+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:54:59.902055+0000","last_undegraded":"2023-07-14T11:54:59.902055+0000","last_fullsized":"2023-07-14T11:54:59.902055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:26.268564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.296305+0000","last_change":"2023-07-14T11:54:58.641928+0000","last_active":"2023-07-14T11:55:00.296305+0000","last_peered":"2023-07-14T11:55:00.296305+0000","last_clean":"2023-07-14T11:55:00.296305+0000","last_became_active":"2023-07-14T11:54:58.641629+0000","last_became_peered":"2023-07-14T11:54:58.641629+0000","last_unstale":"2023-07-14T11:55:00.296305+0000","last_undegraded":"2023-07-14T11:55:00.296305+0000","last_fullsized":"2023-07-14T11:55:00.296305+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:11:16.296910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782900+0000","last_change":"2023-07-14T11:54:58.169976+0000","last_active":"2023-07-14T11:55:00.782900+0000","last_peered":"2023-07-14T11:55:00.782900+0000","last_clean":"2023-07-14T11:55:00.782900+0000","last_became_active":"2023-07-14T11:54:58.169591+0000","last_became_peered":"2023-07-14T11:54:58.169591+0000","last_unstale":"2023-07-14T11:55:00.782900+0000","last_undegraded":"2023-07-14T11:55:00.782900+0000","last_fullsized":"2023-07-14T11:55:00.782900+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:39.232507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629465+0000","last_change":"2023-07-14T11:53:09.434255+0000","last_active":"2023-07-14T11:54:59.629465+0000","last_peered":"2023-07-14T11:54:59.629465+0000","last_clean":"2023-07-14T11:54:59.629465+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:54:59.629465+0000","last_undegraded":"2023-07-14T11:54:59.629465+0000","last_fullsized":"2023-07-14T11:54:59.629465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:27:55.537660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.631206+0000","last_change":"2023-07-14T11:54:58.655331+0000","last_active":"2023-07-14T11:54:59.631206+0000","last_peered":"2023-07-14T11:54:59.631206+0000","last_clean":"2023-07-14T11:54:59.631206+0000","last_became_active":"2023-07-14T11:54:58.655051+0000","last_became_peered":"2023-07-14T11:54:58.655051+0000","last_unstale":"2023-07-14T11:54:59.631206+0000","last_undegraded":"2023-07-14T11:54:59.631206+0000","last_fullsized":"2023-07-14T11:54:59.631206+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:55:06.438207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"42'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.295282+0000","last_change":"2023-07-14T11:54:58.161597+0000","last_active":"2023-07-14T11:55:00.295282+0000","last_peered":"2023-07-14T11:55:00.295282+0000","last_clean":"2023-07-14T11:55:00.295282+0000","last_became_active":"2023-07-14T11:54:58.161341+0000","last_became_peered":"2023-07-14T11:54:58.161341+0000","last_unstale":"2023-07-14T11:55:00.295282+0000","last_undegraded":"2023-07-14T11:55:00.295282+0000","last_fullsized":"2023-07-14T11:55:00.295282+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:55:08.242109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.633551+0000","last_change":"2023-07-14T11:53:09.147040+0000","last_active":"2023-07-14T11:54:58.633551+0000","last_peered":"2023-07-14T11:54:58.633551+0000","last_clean":"2023-07-14T11:54:58.633551+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:54:58.633551+0000","last_undegraded":"2023-07-14T11:54:58.633551+0000","last_fullsized":"2023-07-14T11:54:58.633551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:26.268564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.782843+0000","last_change":"2023-07-14T11:54:58.655331+0000","last_active":"2023-07-14T11:55:00.782843+0000","last_peered":"2023-07-14T11:55:00.782843+0000","last_clean":"2023-07-14T11:55:00.782843+0000","last_became_active":"2023-07-14T11:54:58.655033+0000","last_became_peered":"2023-07-14T11:54:58.655033+0000","last_unstale":"2023-07-14T11:55:00.782843+0000","last_undegraded":"2023-07-14T11:55:00.782843+0000","last_fullsized":"2023-07-14T11:55:00.782843+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T11:58:55.875713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.902000+0000","last_change":"2023-07-14T11:54:58.172009+0000","last_active":"2023-07-14T11:54:59.902000+0000","last_peered":"2023-07-14T11:54:59.902000+0000","last_clean":"2023-07-14T11:54:59.902000+0000","last_became_active":"2023-07-14T11:54:58.171721+0000","last_became_peered":"2023-07-14T11:54:58.171721+0000","last_unstale":"2023-07-14T11:54:59.902000+0000","last_undegraded":"2023-07-14T11:54:59.902000+0000","last_fullsized":"2023-07-14T11:54:59.902000+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:48:00.910272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.627626+0000","last_change":"2023-07-14T11:53:09.619294+0000","last_active":"2023-07-14T11:54:59.627626+0000","last_peered":"2023-07-14T11:54:59.627626+0000","last_clean":"2023-07-14T11:54:59.627626+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:54:59.627626+0000","last_undegraded":"2023-07-14T11:54:59.627626+0000","last_fullsized":"2023-07-14T11:54:59.627626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:41:47.183177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901574+0000","last_change":"2023-07-14T11:54:58.646315+0000","last_active":"2023-07-14T11:54:59.901574+0000","last_peered":"2023-07-14T11:54:59.901574+0000","last_clean":"2023-07-14T11:54:59.901574+0000","last_became_active":"2023-07-14T11:54:58.645996+0000","last_became_peered":"2023-07-14T11:54:58.645996+0000","last_unstale":"2023-07-14T11:54:59.901574+0000","last_undegraded":"2023-07-14T11:54:59.901574+0000","last_fullsized":"2023-07-14T11:54:59.901574+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:31.432081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.629411+0000","last_change":"2023-07-14T11:54:58.166098+0000","last_active":"2023-07-14T11:54:59.629411+0000","last_peered":"2023-07-14T11:54:59.629411+0000","last_clean":"2023-07-14T11:54:59.629411+0000","last_became_active":"2023-07-14T11:54:58.165847+0000","last_became_peered":"2023-07-14T11:54:58.165847+0000","last_unstale":"2023-07-14T11:54:59.629411+0000","last_undegraded":"2023-07-14T11:54:59.629411+0000","last_fullsized":"2023-07-14T11:54:59.629411+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:38:18.269392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.900724+0000","last_change":"2023-07-14T11:53:09.146156+0000","last_active":"2023-07-14T11:54:59.900724+0000","last_peered":"2023-07-14T11:54:59.900724+0000","last_clean":"2023-07-14T11:54:59.900724+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:54:59.900724+0000","last_undegraded":"2023-07-14T11:54:59.900724+0000","last_fullsized":"2023-07-14T11:54:59.900724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:42:51.151986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:55:00.783095+0000","last_change":"2023-07-14T11:54:58.652451+0000","last_active":"2023-07-14T11:55:00.783095+0000","last_peered":"2023-07-14T11:55:00.783095+0000","last_clean":"2023-07-14T11:55:00.783095+0000","last_became_active":"2023-07-14T11:54:58.652143+0000","last_became_peered":"2023-07-14T11:54:58.652143+0000","last_unstale":"2023-07-14T11:55:00.783095+0000","last_undegraded":"2023-07-14T11:55:00.783095+0000","last_fullsized":"2023-07-14T11:55:00.783095+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:58.127757+0000","last_clean_scrub_stamp":"2023-07-14T11:54:58.127757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:06:31.160677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"41'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.168234+0000","last_change":"2023-07-14T11:54:58.169336+0000","last_active":"2023-07-14T11:54:59.168234+0000","last_peered":"2023-07-14T11:54:59.168234+0000","last_clean":"2023-07-14T11:54:59.168234+0000","last_became_active":"2023-07-14T11:54:58.168910+0000","last_became_peered":"2023-07-14T11:54:58.168910+0000","last_unstale":"2023-07-14T11:54:59.168234+0000","last_undegraded":"2023-07-14T11:54:59.168234+0000","last_fullsized":"2023-07-14T11:54:59.168234+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:54:57.112665+0000","last_clean_scrub_stamp":"2023-07-14T11:54:57.112665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:39.232507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-14T11:54:58.634301+0000","last_change":"2023-07-14T11:53:09.146556+0000","last_active":"2023-07-14T11:54:58.634301+0000","last_peered":"2023-07-14T11:54:58.634301+0000","last_clean":"2023-07-14T11:54:58.634301+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:54:58.634301+0000","last_undegraded":"2023-07-14T11:54:58.634301+0000","last_fullsized":"2023-07-14T11:54:58.634301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:14:16.178403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-14T11:54:59.901146+0000","last_change":"2023-07-14T11:53:03.159818+0000","last_active":"2023-07-14T11:54:59.901146+0000","last_peered":"2023-07-14T11:54:59.901146+0000","last_clean":"2023-07-14T11:54:59.901146+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:54:59.901146+0000","last_undegraded":"2023-07-14T11:54:59.901146+0000","last_fullsized":"2023-07-14T11:54:59.901146+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_clean_scrub_stamp":"2023-07-14T11:53:02.054348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:29:46.040542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1085440,"data_stored":807909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.302}]},{"osd":1,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.73},{"interface":"front","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":1.1040000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":1.6419999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.355}]},{"osd":4,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":4.0720000000000001,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.169}]},{"osd":5,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.579,"5min":1.579,"15min":1.579},"min":{"1min":1.028,"5min":1.028,"15min":1.028},"max":{"1min":4.1630000000000003,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":1.587}]},{"osd":6,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"min":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":4.2619999999999996,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"min":{"1min":1.091,"5min":1.091,"15min":1.091},"max":{"1min":2.7570000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.4810000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":495616,"data_stored":218123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.3939999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.038}]},{"osd":2,"last update":"Fri Jul 14 11:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.3},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":1.008,"5min":1.008,"15min":1.008},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.45}]},{"osd":3,"last update":"Fri Jul 14 11:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.26},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.5329999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.496}]},{"osd":5,"last update":"Fri Jul 14 11:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.429},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":1.034,"5min":1.034,"15min":1.034},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":1.5640000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.995,"5min":0.995,"15min":0.995},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.617},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":1.649}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":479232,"data_stored":210548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.0800000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.02}]},{"osd":2,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.1359999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.2669999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.397}]},{"osd":6,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":1.036,"5min":1.036,"15min":1.036},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.4390000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.482},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"min":{"1min":1.157,"5min":1.157,"15min":1.157},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.595}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":499712,"data_stored":219015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.3169999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.482},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.381}]},{"osd":2,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":3.2759999999999998,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":1.286}]},{"osd":3,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.26},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.448}]},{"osd":5,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":3.1280000000000001,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.177},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.1219999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.409}]},{"osd":6,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":1.083,"5min":1.083,"15min":1.083},"max":{"1min":3.5289999999999999,"5min":3.5289999999999999,"15min":3.5289999999999999},"last":0.97999999999999998}]},{"osd":7,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"max":{"1min":3.5640000000000001,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":1.339}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":499712,"data_stored":218089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.502}]},{"osd":1,"last update":"Fri Jul 14 11:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.92200000000000004}]},{"osd":2,"last update":"Fri Jul 14 11:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.90100000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.335,"5min":2.335,"15min":2.335},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0349999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":2.7959999999999998,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.286}]},{"osd":6,"last update":"Fri Jul 14 11:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.335},{"interface":"front","average":{"1min":1.524,"5min":1.524,"15min":1.524},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.266}]},{"osd":7,"last update":"Fri Jul 14 11:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":1.097,"5min":1.097,"15min":1.097},"max":{"1min":3.395,"5min":3.395,"15min":3.395},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.238}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":503808,"data_stored":219037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.91000000000000003}]},{"osd":1,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.6070000000000002,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":0.73999999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.0229999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.123}]},{"osd":5,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.1819999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.4489999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.2310000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.468,"15min":1.468},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":1.321}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":491520,"data_stored":217521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.77000000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.88900000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.68000000000000005}]},{"osd":4,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.875},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.96999999999999997}]},{"osd":5,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":1.006,"5min":1.006,"15min":1.006},"max":{"1min":3.2949999999999999,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.014}]},{"osd":6,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.403,"15min":1.403},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.2270000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":3.3799999999999999,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.163},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.375,"5min":2.375,"15min":2.375},"last":0.92300000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":1073152,"data_stored":800880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":1.4490000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.1040000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.3260000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":1.05},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.93200000000000005}]},{"osd":4,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":2.2120000000000002,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.4830000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.153}]},{"osd":6,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.607},{"interface":"front","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.2529999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.536},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.667}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:55:03.902 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-14T11:55:03.902 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-14T11:55:04.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.135+0000 7f6d81581700 1 Processor -- start 2023-07-14T11:55:04.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.135+0000 7f6d81581700 1 -- start start 2023-07-14T11:55:04.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.135+0000 7f6d81581700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6d7c0ca370 0x7f6d7c0cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:04.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.135+0000 7f6d81581700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7c1544d0 con 0x7f6d7c15aa20 2023-07-14T11:55:04.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.135+0000 7f6d81581700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7c06bcf0 con 0x7f6d7c154100 2023-07-14T11:55:04.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d81581700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d7c06c590 con 0x7f6d7c0ca370 2023-07-14T11:55:04.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d7a7fc700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f6d7c15aa20 0x7f6d7c0c9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:47214/0 (socket says 172.21.15.104:47214) 2023-07-14T11:55:04.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d7a7fc700 1 -- 172.21.15.104:0/383773680 learned_addr learned my addr 172.21.15.104:0/383773680 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:04.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d7b7fe700 1 --2- 172.21.15.104:0/383773680 >> v2:172.21.15.104:3300/0 conn(0x7f6d7c0ca370 0x7f6d7c0cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:04.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d79ffb700 1 -- 172.21.15.104:0/383773680 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2188440516 0 0) 0x7f6d7c1544d0 con 0x7f6d7c15aa20 2023-07-14T11:55:04.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d79ffb700 1 -- 172.21.15.104:0/383773680 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d640035d0 con 0x7f6d7c15aa20 2023-07-14T11:55:04.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d7b7fe700 1 -- 172.21.15.104:0/383773680 >> v1:172.21.15.121:6789/0 conn(0x7f6d7c154100 legacy=0x7f6d7c06ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:04.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d79ffb700 1 -- 172.21.15.104:0/383773680 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227040664 0 0) 0x7f6d7c06bcf0 con 0x7f6d7c154100 2023-07-14T11:55:04.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d7b7fe700 1 -- 172.21.15.104:0/383773680 >> v1:172.21.15.104:6789/0 conn(0x7f6d7c15aa20 legacy=0x7f6d7c0c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:04.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d7b7fe700 1 -- 172.21.15.104:0/383773680 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c15ae90 con 0x7f6d7c0ca370 2023-07-14T11:55:04.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d7b7fe700 1 --2- 172.21.15.104:0/383773680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d7c0ca370 0x7f6d7c0cdfc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009d90 tx=0x7f6d7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c870a2b813d68f47 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:04.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d79ffb700 1 -- 172.21.15.104:0/383773680 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7400e070 con 0x7f6d7c0ca370 2023-07-14T11:55:04.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d79ffb700 1 -- 172.21.15.104:0/383773680 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d74003a40 con 0x7f6d7c0ca370 2023-07-14T11:55:04.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d79ffb700 1 -- 172.21.15.104:0/383773680 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d70001670 con 0x7f6d7c0ca370 2023-07-14T11:55:04.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d81581700 1 -- 172.21.15.104:0/383773680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d7c0ca370 msgr2=0x7f6d7c0cdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:04.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d81581700 1 --2- 172.21.15.104:0/383773680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d7c0ca370 0x7f6d7c0cdfc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009d90 tx=0x7f6d7400b330 comp rx=0 tx=0).stop 2023-07-14T11:55:04.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d81581700 1 -- 172.21.15.104:0/383773680 shutdown_connections 2023-07-14T11:55:04.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d81581700 1 --2- 172.21.15.104:0/383773680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d7c0ca370 0x7f6d7c0cdfc0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d81581700 1 -- 172.21.15.104:0/383773680 >> 172.21.15.104:0/383773680 conn(0x7f6d7c06e960 msgr2=0x7f6d7c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:04.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d81581700 1 -- 172.21.15.104:0/383773680 shutdown_connections 2023-07-14T11:55:04.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.139+0000 7f6d81581700 1 -- 172.21.15.104:0/383773680 wait complete. 2023-07-14T11:55:04.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.143+0000 7f6d81581700 1 Processor -- start 2023-07-14T11:55:04.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.143+0000 7f6d81581700 1 -- start start 2023-07-14T11:55:04.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d81581700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d7c0ca370 0x7f6d7c14ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:04.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d81581700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d7c154100 0x7f6d7c0b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:04.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d81581700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c15aa20 0x7f6d7c158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:04.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d81581700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c0b20e0 con 0x7f6d7c154100 2023-07-14T11:55:04.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d81581700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c14b170 con 0x7f6d7c15aa20 2023-07-14T11:55:04.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d81581700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c0b14d0 con 0x7f6d7c0ca370 2023-07-14T11:55:04.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d7a7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d7c154100 0x7f6d7c0b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:04.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d7a7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d7c154100 0x7f6d7c0b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35686/0 (socket says 172.21.15.104:35686) 2023-07-14T11:55:04.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d7a7fc700 1 -- 172.21.15.104:0/2847403725 learned_addr learned my addr 172.21.15.104:0/2847403725 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:04.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d7b7fe700 1 --2- 172.21.15.104:0/2847403725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c15aa20 0x7f6d7c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:04.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d7affd700 1 --2- 172.21.15.104:0/2847403725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d7c0ca370 0x7f6d7c14ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:04.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d7a7fc700 1 -- 172.21.15.104:0/2847403725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d7c0ca370 msgr2=0x7f6d7c14ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:04.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d7a7fc700 1 --2- 172.21.15.104:0/2847403725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d7c0ca370 0x7f6d7c14ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d7a7fc700 1 -- 172.21.15.104:0/2847403725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c15aa20 msgr2=0x7f6d7c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:04.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d7a7fc700 1 --2- 172.21.15.104:0/2847403725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c15aa20 0x7f6d7c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d7a7fc700 1 -- 172.21.15.104:0/2847403725 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d74009a70 con 0x7f6d7c154100 2023-07-14T11:55:04.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d7a7fc700 1 --2- 172.21.15.104:0/2847403725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d7c154100 0x7f6d7c0b2570 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6d700011e0 tx=0x7f6d7000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:04.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d637fe700 1 -- 172.21.15.104:0/2847403725 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d70004ca0 con 0x7f6d7c154100 2023-07-14T11:55:04.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d637fe700 1 -- 172.21.15.104:0/2847403725 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d70004e00 con 0x7f6d7c154100 2023-07-14T11:55:04.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d81581700 1 -- 172.21.15.104:0/2847403725 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7c0b17b0 con 0x7f6d7c154100 2023-07-14T11:55:04.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d637fe700 1 -- 172.21.15.104:0/2847403725 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d70007310 con 0x7f6d7c154100 2023-07-14T11:55:04.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.147+0000 7f6d81581700 1 -- 172.21.15.104:0/2847403725 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7c0b1d70 con 0x7f6d7c154100 2023-07-14T11:55:04.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.151+0000 7f6d81581700 1 -- 172.21.15.104:0/2847403725 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d7c04e500 con 0x7f6d7c154100 2023-07-14T11:55:04.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.151+0000 7f6d637fe700 1 -- 172.21.15.104:0/2847403725 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6d70007470 con 0x7f6d7c154100 2023-07-14T11:55:04.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.155+0000 7f6d637fe700 1 --2- 172.21.15.104:0/2847403725 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6d680662f0 0x7f6d680687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:04.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.155+0000 7f6d637fe700 1 -- 172.21.15.104:0/2847403725 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6d70083c30 con 0x7f6d7c154100 2023-07-14T11:55:04.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.155+0000 7f6d7affd700 1 --2- 172.21.15.104:0/2847403725 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6d680662f0 0x7f6d680687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:04.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.155+0000 7f6d7affd700 1 --2- 172.21.15.104:0/2847403725 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6d680662f0 0x7f6d680687a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c007fd0 tx=0x7f6d6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:04.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.159+0000 7f6d637fe700 1 -- 172.21.15.104:0/2847403725 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f6d70050ed0 con 0x7f6d7c154100 2023-07-14T11:55:04.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.391+0000 7f6d81581700 1 -- 172.21.15.104:0/2847403725 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6d7c06d030 con 0x7f6d7c154100 2023-07-14T11:55:04.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.403+0000 7f6d637fe700 1 -- 172.21.15.104:0/2847403725 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6d70007720 con 0x7f6d7c154100 2023-07-14T11:55:04.389 INFO:teuthology.orchestra.run.smithi104.stdout:[client.0] 2023-07-14T11:55:04.389 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDrNrFkbrzjERAA+9BBzuauwUdhALUcDtd80A== 2023-07-14T11:55:04.389 INFO:teuthology.orchestra.run.smithi104.stdout: caps mds = "allow" 2023-07-14T11:55:04.389 INFO:teuthology.orchestra.run.smithi104.stdout: caps mon = "allow r" 2023-07-14T11:55:04.389 INFO:teuthology.orchestra.run.smithi104.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-14T11:55:04.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.407+0000 7f6d81581700 1 -- 172.21.15.104:0/2847403725 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6d680662f0 msgr2=0x7f6d680687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:04.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.407+0000 7f6d81581700 1 --2- 172.21.15.104:0/2847403725 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6d680662f0 0x7f6d680687a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c007fd0 tx=0x7f6d6c009040 comp rx=0 tx=0).stop 2023-07-14T11:55:04.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.407+0000 7f6d81581700 1 -- 172.21.15.104:0/2847403725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d7c154100 msgr2=0x7f6d7c0b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:04.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.407+0000 7f6d81581700 1 --2- 172.21.15.104:0/2847403725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d7c154100 0x7f6d7c0b2570 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6d700011e0 tx=0x7f6d7000c9b0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.407+0000 7f6d81581700 1 -- 172.21.15.104:0/2847403725 shutdown_connections 2023-07-14T11:55:04.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.407+0000 7f6d81581700 1 --2- 172.21.15.104:0/2847403725 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6d680662f0 0x7f6d680687a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.407+0000 7f6d81581700 1 --2- 172.21.15.104:0/2847403725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d7c154100 0x7f6d7c0b2570 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.407+0000 7f6d81581700 1 --2- 172.21.15.104:0/2847403725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c15aa20 0x7f6d7c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.407+0000 7f6d81581700 1 --2- 172.21.15.104:0/2847403725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d7c0ca370 0x7f6d7c14ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.407+0000 7f6d81581700 1 -- 172.21.15.104:0/2847403725 >> 172.21.15.104:0/2847403725 conn(0x7f6d7c06e960 msgr2=0x7f6d7c15b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:04.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.407+0000 7f6d81581700 1 -- 172.21.15.104:0/2847403725 shutdown_connections 2023-07-14T11:55:04.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.407+0000 7f6d81581700 1 -- 172.21.15.104:0/2847403725 wait complete. 2023-07-14T11:55:04.396 INFO:teuthology.orchestra.run.smithi104.stderr:updated caps for client.0 2023-07-14T11:55:04.415 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:55:04.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.602+0000 7fac9759e700 1 Processor -- start 2023-07-14T11:55:04.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.602+0000 7fac9759e700 1 -- start start 2023-07-14T11:55:04.591 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac9759e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fac980cb180 0x7fac980ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:04.591 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac9759e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac980ca8a0 con 0x7fac9815ffa0 2023-07-14T11:55:04.592 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac9759e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac980cbcd0 con 0x7fac98160370 2023-07-14T11:55:04.592 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac9759e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac980cbf70 con 0x7fac980cb180 2023-07-14T11:55:04.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac96d9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fac980cb180 0x7fac980ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:04.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac96d9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fac980cb180 0x7fac980ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35696/0 (socket says 172.21.15.104:35696) 2023-07-14T11:55:04.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac96d9d700 1 -- 172.21.15.104:0/2630067139 learned_addr learned my addr 172.21.15.104:0/2630067139 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:04.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac9659c700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fac9815ffa0 0x7fac9806d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:47226/0 (socket says 172.21.15.104:47226) 2023-07-14T11:55:04.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac96d9d700 1 -- 172.21.15.104:0/2630067139 >> v1:172.21.15.121:6789/0 conn(0x7fac98160370 legacy=0x7fac980caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:04.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac96d9d700 1 -- 172.21.15.104:0/2630067139 >> v1:172.21.15.104:6789/0 conn(0x7fac9815ffa0 legacy=0x7fac9806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:04.594 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac96d9d700 1 -- 172.21.15.104:0/2630067139 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980cc250 con 0x7fac980cb180 2023-07-14T11:55:04.594 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac96d9d700 1 --2- 172.21.15.104:0/2630067139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac980cb180 0x7fac980ce340 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fac90009e50 tx=0x7fac9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4269697d2424ab server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:04.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac95d9b700 1 -- 172.21.15.104:0/2630067139 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9000e070 con 0x7fac980cb180 2023-07-14T11:55:04.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac95d9b700 1 -- 172.21.15.104:0/2630067139 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac90003a40 con 0x7fac980cb180 2023-07-14T11:55:04.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac95d9b700 1 -- 172.21.15.104:0/2630067139 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac90003df0 con 0x7fac980cb180 2023-07-14T11:55:04.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac9759e700 1 -- 172.21.15.104:0/2630067139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac980cb180 msgr2=0x7fac980ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:04.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.606+0000 7fac9759e700 1 --2- 172.21.15.104:0/2630067139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac980cb180 0x7fac980ce340 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fac90009e50 tx=0x7fac9000b330 comp rx=0 tx=0).stop 2023-07-14T11:55:04.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.610+0000 7fac9759e700 1 -- 172.21.15.104:0/2630067139 shutdown_connections 2023-07-14T11:55:04.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.610+0000 7fac9759e700 1 --2- 172.21.15.104:0/2630067139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac980cb180 0x7fac980ce340 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.610+0000 7fac9759e700 1 -- 172.21.15.104:0/2630067139 >> 172.21.15.104:0/2630067139 conn(0x7fac9806e960 msgr2=0x7fac9806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:04.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.610+0000 7fac9759e700 1 -- 172.21.15.104:0/2630067139 shutdown_connections 2023-07-14T11:55:04.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.610+0000 7fac9759e700 1 -- 172.21.15.104:0/2630067139 wait complete. 2023-07-14T11:55:04.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.610+0000 7fac9759e700 1 Processor -- start 2023-07-14T11:55:04.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.610+0000 7fac9759e700 1 -- start start 2023-07-14T11:55:04.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9759e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac980cb180 0x7fac980c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:04.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac9815ffa0 0x7fac9815e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:04.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac98160370 0x7fac980c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:04.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9759e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac980c6a50 con 0x7fac980cb180 2023-07-14T11:55:04.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac980c1c20 con 0x7fac9815ffa0 2023-07-14T11:55:04.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac980c61b0 con 0x7fac98160370 2023-07-14T11:55:04.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac980cb180 0x7fac980c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:04.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac980cb180 0x7fac980c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35710/0 (socket says 172.21.15.104:35710) 2023-07-14T11:55:04.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9659c700 1 -- 172.21.15.104:0/3060233586 learned_addr learned my addr 172.21.15.104:0/3060233586 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:04.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac8ffff700 1 --2- 172.21.15.104:0/3060233586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac9815ffa0 0x7fac9815e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:04.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac96d9d700 1 --2- 172.21.15.104:0/3060233586 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac98160370 0x7fac980c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:04.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9659c700 1 -- 172.21.15.104:0/3060233586 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac98160370 msgr2=0x7fac980c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:04.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9659c700 1 --2- 172.21.15.104:0/3060233586 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac98160370 0x7fac980c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9659c700 1 -- 172.21.15.104:0/3060233586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac9815ffa0 msgr2=0x7fac9815e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:04.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9659c700 1 --2- 172.21.15.104:0/3060233586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac9815ffa0 0x7fac9815e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9659c700 1 -- 172.21.15.104:0/3060233586 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac90009b30 con 0x7fac980cb180 2023-07-14T11:55:04.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9659c700 1 --2- 172.21.15.104:0/3060233586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac980cb180 0x7fac980c5b10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fac8800c420 tx=0x7fac8800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:04.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac8effd700 1 -- 172.21.15.104:0/3060233586 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88005cd0 con 0x7fac980cb180 2023-07-14T11:55:04.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac8effd700 1 -- 172.21.15.104:0/3060233586 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac88005e30 con 0x7fac980cb180 2023-07-14T11:55:04.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac8effd700 1 -- 172.21.15.104:0/3060233586 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88007310 con 0x7fac980cb180 2023-07-14T11:55:04.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9759e700 1 -- 172.21.15.104:0/3060233586 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac980c6490 con 0x7fac980cb180 2023-07-14T11:55:04.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.614+0000 7fac9759e700 1 -- 172.21.15.104:0/3060233586 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac980be0a0 con 0x7fac980cb180 2023-07-14T11:55:04.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.618+0000 7fac9759e700 1 -- 172.21.15.104:0/3060233586 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9804e500 con 0x7fac980cb180 2023-07-14T11:55:04.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.618+0000 7fac8effd700 1 -- 172.21.15.104:0/3060233586 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fac88007470 con 0x7fac980cb180 2023-07-14T11:55:04.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.622+0000 7fac8effd700 1 --2- 172.21.15.104:0/3060233586 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fac780662f0 0x7fac780687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:04.607 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.622+0000 7fac8effd700 1 -- 172.21.15.104:0/3060233586 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fac88083060 con 0x7fac980cb180 2023-07-14T11:55:04.608 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.622+0000 7fac8ffff700 1 --2- 172.21.15.104:0/3060233586 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fac780662f0 0x7fac780687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:04.608 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.622+0000 7fac8ffff700 1 --2- 172.21.15.104:0/3060233586 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fac780662f0 0x7fac780687a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fac980cd700 tx=0x7fac80009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:04.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.626+0000 7fac8effd700 1 -- 172.21.15.104:0/3060233586 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fac88050430 con 0x7fac980cb180 2023-07-14T11:55:04.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.830+0000 7fac9759e700 1 -- 172.21.15.104:0/3060233586 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac980ca8a0 con 0x7fac980cb180 2023-07-14T11:55:04.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.830+0000 7fac8effd700 1 -- 172.21.15.104:0/3060233586 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4715 (secure 0 0 0) 0x7fac980ca8a0 con 0x7fac980cb180 2023-07-14T11:55:04.816 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:04.816 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3696540262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3696540262},{"type":"v1","addr":"172.21.15.104:6835","nonce":3696540262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/684052037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":684052037},{"type":"v1","addr":"172.21.15.121:6835","nonce":684052037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/1267091306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":1267091306},{"type":"v1","addr":"172.21.15.121:6833","nonce":1267091306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:54:58.600122+0000","modified":"2023-07-14T11:55:02.180538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5616},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5616":{"gid":5616,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6837/398018673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":398018673},{"type":"v1","addr":"172.21.15.104:6837","nonce":398018673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-14T11:55:04.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.834+0000 7fac9759e700 1 -- 172.21.15.104:0/3060233586 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fac780662f0 msgr2=0x7fac780687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:04.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.834+0000 7fac9759e700 1 --2- 172.21.15.104:0/3060233586 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fac780662f0 0x7fac780687a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fac980cd700 tx=0x7fac80009040 comp rx=0 tx=0).stop 2023-07-14T11:55:04.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.834+0000 7fac9759e700 1 -- 172.21.15.104:0/3060233586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac980cb180 msgr2=0x7fac980c5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:04.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.834+0000 7fac9759e700 1 --2- 172.21.15.104:0/3060233586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac980cb180 0x7fac980c5b10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fac8800c420 tx=0x7fac8800c7e0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.834+0000 7fac9759e700 1 -- 172.21.15.104:0/3060233586 shutdown_connections 2023-07-14T11:55:04.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.834+0000 7fac9759e700 1 --2- 172.21.15.104:0/3060233586 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fac780662f0 0x7fac780687a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.834+0000 7fac9759e700 1 --2- 172.21.15.104:0/3060233586 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac98160370 0x7fac980c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.834+0000 7fac9759e700 1 --2- 172.21.15.104:0/3060233586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac9815ffa0 0x7fac9815e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.834+0000 7fac9759e700 1 --2- 172.21.15.104:0/3060233586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac980cb180 0x7fac980c5b10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:04.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.834+0000 7fac9759e700 1 -- 172.21.15.104:0/3060233586 >> 172.21.15.104:0/3060233586 conn(0x7fac9806e960 msgr2=0x7fac980caa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:04.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.834+0000 7fac9759e700 1 -- 172.21.15.104:0/3060233586 shutdown_connections 2023-07-14T11:55:04.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:04.834+0000 7fac9759e700 1 -- 172.21.15.104:0/3060233586 wait complete. 2023-07-14T11:55:04.823 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 18 2023-07-14T11:55:05.243 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-14T11:54:58.600122+0000', 'modified': '2023-07-14T11:55:02.180538+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5616}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5616': {'gid': 5616, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.104:6837/398018673', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 398018673}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 398018673}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-14T11:55:05.243 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-14T11:55:05.244 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-14T11:55:05.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.438+0000 7fa9f87ac700 1 Processor -- start 2023-07-14T11:55:05.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.438+0000 7fa9f87ac700 1 -- start start 2023-07-14T11:55:05.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.438+0000 7fa9f87ac700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa9f00afd30 0x7fa9f00b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.438+0000 7fa9f87ac700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa9f00703a0 0x7fa9f0070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.438+0000 7fa9f87ac700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f00cd530 con 0x7fa9f015c070 2023-07-14T11:55:05.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.438+0000 7fa9f87ac700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f00b06e0 con 0x7fa9f00afd30 2023-07-14T11:55:05.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f87ac700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f015c440 con 0x7fa9f00703a0 2023-07-14T11:55:05.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f5d47700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa9f00afd30 0x7fa9f00b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f5d47700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa9f00afd30 0x7fa9f00b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35724/0 (socket says 172.21.15.104:35724) 2023-07-14T11:55:05.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f5d47700 1 -- 172.21.15.104:0/1420107183 learned_addr learned my addr 172.21.15.104:0/1420107183 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:05.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f6d49700 1 --2- 172.21.15.104:0/1420107183 >> v2:172.21.15.121:3300/0 conn(0x7fa9f00703a0 0x7fa9f0070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f5d47700 1 -- 172.21.15.104:0/1420107183 >> v2:172.21.15.121:3300/0 conn(0x7fa9f00703a0 msgr2=0x7fa9f0070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:05.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f5d47700 1 --2- 172.21.15.104:0/1420107183 >> v2:172.21.15.121:3300/0 conn(0x7fa9f00703a0 0x7fa9f0070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f5d47700 1 -- 172.21.15.104:0/1420107183 >> v1:172.21.15.121:6789/0 conn(0x7fa9f015c070 legacy=0x7fa9f00ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f5d47700 1 -- 172.21.15.104:0/1420107183 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f00cc1b0 con 0x7fa9f00afd30 2023-07-14T11:55:05.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f5d47700 1 --2- 172.21.15.104:0/1420107183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00afd30 0x7fa9f00b01a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0071900 tx=0x7fa9e0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c7ff9bf1a08036 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:05.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f5546700 1 -- 172.21.15.104:0/1420107183 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0011830 con 0x7fa9f00afd30 2023-07-14T11:55:05.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f5546700 1 -- 172.21.15.104:0/1420107183 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e0011990 con 0x7fa9f00afd30 2023-07-14T11:55:05.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f5546700 1 -- 172.21.15.104:0/1420107183 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0011d20 con 0x7fa9f00afd30 2023-07-14T11:55:05.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f87ac700 1 -- 172.21.15.104:0/1420107183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00afd30 msgr2=0x7fa9f00b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f87ac700 1 --2- 172.21.15.104:0/1420107183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00afd30 0x7fa9f00b01a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0071900 tx=0x7fa9e0009bf0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f87ac700 1 -- 172.21.15.104:0/1420107183 shutdown_connections 2023-07-14T11:55:05.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f87ac700 1 --2- 172.21.15.104:0/1420107183 >> v2:172.21.15.121:3300/0 conn(0x7fa9f00703a0 0x7fa9f0070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f87ac700 1 --2- 172.21.15.104:0/1420107183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00afd30 0x7fa9f00b01a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f87ac700 1 -- 172.21.15.104:0/1420107183 >> 172.21.15.104:0/1420107183 conn(0x7fa9f00cad30 msgr2=0x7fa9f00cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:05.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f87ac700 1 -- 172.21.15.104:0/1420107183 shutdown_connections 2023-07-14T11:55:05.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.442+0000 7fa9f87ac700 1 -- 172.21.15.104:0/1420107183 wait complete. 2023-07-14T11:55:05.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.446+0000 7fa9f87ac700 1 Processor -- start 2023-07-14T11:55:05.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.446+0000 7fa9f87ac700 1 -- start start 2023-07-14T11:55:05.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.446+0000 7fa9f87ac700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00703a0 0x7fa9f00c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.446+0000 7fa9f87ac700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9f00afd30 0x7fa9f015a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.446+0000 7fa9f87ac700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9f015c070 0x7fa9f0153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.446+0000 7fa9f87ac700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00c59b0 con 0x7fa9f00703a0 2023-07-14T11:55:05.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.446+0000 7fa9f87ac700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00c4c80 con 0x7fa9f015c070 2023-07-14T11:55:05.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.446+0000 7fa9f87ac700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0154100 con 0x7fa9f00afd30 2023-07-14T11:55:05.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9f6548700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00703a0 0x7fa9f00c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9f6548700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00703a0 0x7fa9f00c2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35736/0 (socket says 172.21.15.104:35736) 2023-07-14T11:55:05.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9f6548700 1 -- 172.21.15.104:0/538025122 learned_addr learned my addr 172.21.15.104:0/538025122 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:05.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9f5d47700 1 --2- 172.21.15.104:0/538025122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9f00afd30 0x7fa9f015a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9f6d49700 1 --2- 172.21.15.104:0/538025122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9f015c070 0x7fa9f0153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9f6548700 1 -- 172.21.15.104:0/538025122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9f00afd30 msgr2=0x7fa9f015a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9f6548700 1 --2- 172.21.15.104:0/538025122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9f00afd30 0x7fa9f015a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9f6548700 1 -- 172.21.15.104:0/538025122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9f015c070 msgr2=0x7fa9f0153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9f6548700 1 --2- 172.21.15.104:0/538025122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9f015c070 0x7fa9f0153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9f6548700 1 -- 172.21.15.104:0/538025122 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e0009570 con 0x7fa9f00703a0 2023-07-14T11:55:05.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9f6d49700 1 --2- 172.21.15.104:0/538025122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9f015c070 0x7fa9f0153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:55:05.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9f6548700 1 --2- 172.21.15.104:0/538025122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00703a0 0x7fa9f00c2fa0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00c410 tx=0x7fa9ec00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:05.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9e6ffd700 1 -- 172.21.15.104:0/538025122 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0052d0 con 0x7fa9f00703a0 2023-07-14T11:55:05.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9e6ffd700 1 -- 172.21.15.104:0/538025122 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec005430 con 0x7fa9f00703a0 2023-07-14T11:55:05.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9f87ac700 1 -- 172.21.15.104:0/538025122 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f00c5380 con 0x7fa9f00703a0 2023-07-14T11:55:05.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9e6ffd700 1 -- 172.21.15.104:0/538025122 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0055a0 con 0x7fa9f00703a0 2023-07-14T11:55:05.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.450+0000 7fa9f87ac700 1 -- 172.21.15.104:0/538025122 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f015b0a0 con 0x7fa9f00703a0 2023-07-14T11:55:05.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.454+0000 7fa9e6ffd700 1 -- 172.21.15.104:0/538025122 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa9ec0036a0 con 0x7fa9f00703a0 2023-07-14T11:55:05.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9e6ffd700 1 --2- 172.21.15.104:0/538025122 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa9d40662f0 0x7fa9d40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9e6ffd700 1 -- 172.21.15.104:0/538025122 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa9ec083180 con 0x7fa9f00703a0 2023-07-14T11:55:05.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9f5d47700 1 --2- 172.21.15.104:0/538025122 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa9d40662f0 0x7fa9d40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9fba28740 1 Processor -- start 2023-07-14T11:55:05.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9fba28740 1 -- start start 2023-07-14T11:55:05.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9f5d47700 1 --2- 172.21.15.104:0/538025122 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa9d40662f0 0x7fa9d40687a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0000c00 tx=0x7fa9e0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:05.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9fba28740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eeb0 0x13278b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9fba28740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106eae0 0x1327df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9fba28740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106e710 0x12dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9fba28740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x11fb270 con 0x106eae0 2023-07-14T11:55:05.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9fba28740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x124e3a0 con 0x106e710 2023-07-14T11:55:05.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9fba28740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x1247600 con 0x106eeb0 2023-07-14T11:55:05.447 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9f5d47700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106eae0 0x1327df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.447 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9f5d47700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106eae0 0x1327df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35746/0 (socket says 172.21.15.104:35746) 2023-07-14T11:55:05.447 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9f5d47700 1 -- 172.21.15.104:0/618920388 learned_addr learned my addr 172.21.15.104:0/618920388 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:05.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9f6548700 1 --2- 172.21.15.104:0/618920388 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eeb0 0x13278b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.458+0000 7fa9f6d49700 1 --2- 172.21.15.104:0/618920388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106e710 0x12dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9f5d47700 1 -- 172.21.15.104:0/618920388 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eeb0 msgr2=0x13278b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:05.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9f5d47700 1 --2- 172.21.15.104:0/618920388 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eeb0 0x13278b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9f5d47700 1 -- 172.21.15.104:0/618920388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106e710 msgr2=0x12dcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:05.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9f5d47700 1 --2- 172.21.15.104:0/618920388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106e710 0x12dcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9f5d47700 1 -- 172.21.15.104:0/618920388 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x10ba9e0 con 0x106eae0 2023-07-14T11:55:05.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9f5d47700 1 --2- 172.21.15.104:0/618920388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106eae0 0x1327df0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009150 tx=0x7fa9e0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4870e2023449ab7b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:05.449 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9d3fff700 1 -- 172.21.15.104:0/618920388 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e00038c0 con 0x106eae0 2023-07-14T11:55:05.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9d3fff700 1 -- 172.21.15.104:0/618920388 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e0003a40 con 0x106eae0 2023-07-14T11:55:05.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9d3fff700 1 -- 172.21.15.104:0/618920388 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0003df0 con 0x106eae0 2023-07-14T11:55:05.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9fba28740 1 -- 172.21.15.104:0/618920388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106eae0 msgr2=0x1327df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9fba28740 1 --2- 172.21.15.104:0/618920388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106eae0 0x1327df0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009150 tx=0x7fa9e0010000 comp rx=0 tx=0).stop 2023-07-14T11:55:05.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9fba28740 1 -- 172.21.15.104:0/618920388 shutdown_connections 2023-07-14T11:55:05.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9fba28740 1 --2- 172.21.15.104:0/618920388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106eae0 0x1327df0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9fba28740 1 --2- 172.21.15.104:0/618920388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106e710 0x12dcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9fba28740 1 --2- 172.21.15.104:0/618920388 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eeb0 0x13278b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9fba28740 1 -- 172.21.15.104:0/618920388 >> 172.21.15.104:0/618920388 conn(0x12d8b60 msgr2=0x12e3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:05.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9fba28740 1 -- 172.21.15.104:0/618920388 shutdown_connections 2023-07-14T11:55:05.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9fba28740 1 -- 172.21.15.104:0/618920388 wait complete. 2023-07-14T11:55:05.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.462+0000 7fa9fba28740 1 -- start start 2023-07-14T11:55:05.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9fba28740 1 Processor -- start 2023-07-14T11:55:05.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9fba28740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106e710 0x130ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9fba28740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 0x1310a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9fba28740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106eeb0 0x130ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9fba28740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x1209b00 con 0x106eeb0 2023-07-14T11:55:05.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9fba28740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1208470 con 0x106e710 2023-07-14T11:55:05.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9fba28740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x10f3b20 con 0x106eae0 2023-07-14T11:55:05.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9fba28740 10 client.0 _ll_register_callbacks cb 0x126d530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:55:05.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9f6d49700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106eeb0 0x130ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9f6d49700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106eeb0 0x130ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35760/0 (socket says 172.21.15.104:35760) 2023-07-14T11:55:05.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9f6d49700 1 -- 172.21.15.104:0/3600480390 learned_addr learned my addr 172.21.15.104:0/3600480390 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:05.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9f6548700 1 --2- 172.21.15.104:0/3600480390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106e710 0x130ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9f5d47700 1 --2- 172.21.15.104:0/3600480390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 0x1310a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9f6d49700 1 -- 172.21.15.104:0/3600480390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 msgr2=0x1310a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:05.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9f6d49700 1 --2- 172.21.15.104:0/3600480390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 0x1310a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9f6d49700 1 -- 172.21.15.104:0/3600480390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106e710 msgr2=0x130ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:05.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9f6d49700 1 --2- 172.21.15.104:0/3600480390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106e710 0x130ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9f6d49700 1 -- 172.21.15.104:0/3600480390 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e0022070 con 0x106eeb0 2023-07-14T11:55:05.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9f5d47700 1 --2- 172.21.15.104:0/3600480390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 0x1310a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:05.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9f6d49700 1 --2- 172.21.15.104:0/3600480390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106eeb0 0x130ca60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8007dd0 tx=0x7fa9e8012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a620c17cc8823a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:05.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-14T11:55:05.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9d17fa700 1 -- 172.21.15.104:0/3600480390 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8012ec0 con 0x106eeb0 2023-07-14T11:55:05.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9fba28740 1 -- 172.21.15.104:0/3600480390 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x12405e0 con 0x106eeb0 2023-07-14T11:55:05.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9d17fa700 1 -- 172.21.15.104:0/3600480390 <== mon.0 v2:172.21.15.104:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa9e8018470 con 0x106eeb0 2023-07-14T11:55:05.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9d17fa700 1 -- 172.21.15.104:0/3600480390 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9e8007eb0 con 0x106eeb0 2023-07-14T11:55:05.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9fba28740 10 client.5790 fetch_fsmap learned FSMap version 18 2023-07-14T11:55:05.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.466+0000 7fa9fba28740 1 -- 172.21.15.104:0/3600480390 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x120d250 con 0x106eeb0 2023-07-14T11:55:05.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9d17fa700 1 -- 172.21.15.104:0/3600480390 <== mon.0 v2:172.21.15.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x12405e0 con 0x106eeb0 2023-07-14T11:55:05.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9d17fa700 1 -- 172.21.15.104:0/3600480390 <== mon.0 v2:172.21.15.104:3300/0 5 ==== fsmap(e 18) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa9e8014040 con 0x106eeb0 2023-07-14T11:55:05.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 10 client.5790 fetch_fsmap finished waiting for FSMap version 18 2023-07-14T11:55:05.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 10 client.5790 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-14T11:55:05.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 20 client.5790 populate_metadata read hostname 'smithi104' 2023-07-14T11:55:05.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 1 --2- 172.21.15.104:0/3600480390 >> [v2:172.21.15.104:6836/398018673,v1:172.21.15.104:6837/398018673] conn(0x10b9790 0x132e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 4 client.5790 mds_command: new command op to 5616 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:55:05.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 1 -- 172.21.15.104:0/3600480390 --> [v2:172.21.15.104:6836/398018673,v1:172.21.15.104:6837/398018673] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1240300 con 0x10b9790 2023-07-14T11:55:05.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9f6548700 1 --2- 172.21.15.104:0/3600480390 >> [v2:172.21.15.104:6836/398018673,v1:172.21.15.104:6837/398018673] conn(0x10b9790 0x132e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9d17fa700 10 client.5790 ms_handle_connect on v2:172.21.15.104:6836/398018673 2023-07-14T11:55:05.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9f6548700 1 --2- 172.21.15.104:0/3600480390 >> [v2:172.21.15.104:6836/398018673,v1:172.21.15.104:6837/398018673] conn(0x10b9790 0x132e4e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f14808bb9ed77e server_cookie=7edd611674ae5e61 in_seq=0 out_seq=0 2023-07-14T11:55:05.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9d17fa700 1 -- 172.21.15.104:0/3600480390 <== mds.0 v2:172.21.15.104:6836/398018673 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x1240300 con 0x10b9790 2023-07-14T11:55:05.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9d17fa700 10 client.5790 handle_command_reply: tid=0 2023-07-14T11:55:05.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 1 client.5790 shutdown 2023-07-14T11:55:05.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 1 -- 172.21.15.104:0/3600480390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106eeb0 msgr2=0x130ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 1 --2- 172.21.15.104:0/3600480390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106eeb0 0x130ca60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8007dd0 tx=0x7fa9e8012520 comp rx=0 tx=0).stop 2023-07-14T11:55:05.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 1 -- 172.21.15.104:0/3600480390 shutdown_connections 2023-07-14T11:55:05.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 1 --2- 172.21.15.104:0/3600480390 >> [v2:172.21.15.104:6836/398018673,v1:172.21.15.104:6837/398018673] conn(0x10b9790 0x132e4e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 1 --2- 172.21.15.104:0/3600480390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106eeb0 0x130ca60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 1 --2- 172.21.15.104:0/3600480390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 0x1310a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9d17fa700 0 client.5790 ms_handle_reset on v2:172.21.15.104:6836/398018673 2023-07-14T11:55:05.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 1 --2- 172.21.15.104:0/3600480390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106e710 0x130ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 1 -- 172.21.15.104:0/3600480390 >> 172.21.15.104:0/3600480390 conn(0x12d8b60 msgr2=0x12dcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:05.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 1 -- 172.21.15.104:0/3600480390 shutdown_connections 2023-07-14T11:55:05.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.470+0000 7fa9fba28740 1 -- 172.21.15.104:0/3600480390 wait complete. 2023-07-14T11:55:05.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.474+0000 7fa9fba28740 20 client.5790 trim_cache size 0 max 16384 2023-07-14T11:55:05.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9fba28740 1 Processor -- start 2023-07-14T11:55:05.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9fba28740 1 -- start start 2023-07-14T11:55:05.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9fba28740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106e710 0x13256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9fba28740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 0x1325c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9fba28740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106eeb0 0x12dd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9fba28740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x124aee0 con 0x106e710 2023-07-14T11:55:05.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9fba28740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x132fa30 con 0x106eeb0 2023-07-14T11:55:05.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9fba28740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x11e5f30 con 0x106eae0 2023-07-14T11:55:05.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9f6548700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106e710 0x13256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9f6548700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106e710 0x13256c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35776/0 (socket says 172.21.15.104:35776) 2023-07-14T11:55:05.532 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9f6548700 1 -- 172.21.15.104:0/1301204750 learned_addr learned my addr 172.21.15.104:0/1301204750 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:05.532 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9f6d49700 1 --2- 172.21.15.104:0/1301204750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106eeb0 0x12dd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.532 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9f5d47700 1 --2- 172.21.15.104:0/1301204750 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 0x1325c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.532 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9f6548700 1 -- 172.21.15.104:0/1301204750 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 msgr2=0x1325c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:05.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9f6548700 1 --2- 172.21.15.104:0/1301204750 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 0x1325c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9f6548700 1 -- 172.21.15.104:0/1301204750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106eeb0 msgr2=0x12dd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:05.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9f6548700 1 --2- 172.21.15.104:0/1301204750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106eeb0 0x12dd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.494+0000 7fa9f6548700 1 -- 172.21.15.104:0/1301204750 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f00c9870 con 0x106e710 2023-07-14T11:55:05.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.498+0000 7fa9f6548700 1 --2- 172.21.15.104:0/1301204750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106e710 0x13256c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec054080 tx=0x7fa9ec083040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729858419534c539 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:05.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.498+0000 7fa9e4ff9700 1 -- 172.21.15.104:0/1301204750 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec081480 con 0x106e710 2023-07-14T11:55:05.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.498+0000 7fa9e4ff9700 1 -- 172.21.15.104:0/1301204750 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec052490 con 0x106e710 2023-07-14T11:55:05.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.498+0000 7fa9e4ff9700 1 -- 172.21.15.104:0/1301204750 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec05c1d0 con 0x106e710 2023-07-14T11:55:05.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.498+0000 7fa9fba28740 1 -- 172.21.15.104:0/1301204750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106e710 msgr2=0x13256c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.498+0000 7fa9fba28740 1 --2- 172.21.15.104:0/1301204750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106e710 0x13256c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec054080 tx=0x7fa9ec083040 comp rx=0 tx=0).stop 2023-07-14T11:55:05.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.498+0000 7fa9fba28740 1 -- 172.21.15.104:0/1301204750 shutdown_connections 2023-07-14T11:55:05.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.498+0000 7fa9fba28740 1 --2- 172.21.15.104:0/1301204750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106eeb0 0x12dd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.498+0000 7fa9fba28740 1 --2- 172.21.15.104:0/1301204750 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 0x1325c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.498+0000 7fa9fba28740 1 --2- 172.21.15.104:0/1301204750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106e710 0x13256c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.498+0000 7fa9fba28740 1 -- 172.21.15.104:0/1301204750 >> 172.21.15.104:0/1301204750 conn(0x12d8b60 msgr2=0x1310430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:05.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.498+0000 7fa9fba28740 1 -- 172.21.15.104:0/1301204750 shutdown_connections 2023-07-14T11:55:05.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.498+0000 7fa9fba28740 1 -- 172.21.15.104:0/1301204750 wait complete. 2023-07-14T11:55:05.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.498+0000 7fa9fba28740 1 -- start start 2023-07-14T11:55:05.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.498+0000 7fa9fba28740 1 Processor -- start 2023-07-14T11:55:05.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106e710 0x138d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 0x138e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106eeb0 0x138ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x1216d50 con 0x106e710 2023-07-14T11:55:05.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1201720 con 0x106eeb0 2023-07-14T11:55:05.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x1244a00 con 0x106eae0 2023-07-14T11:55:05.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 10 client.0 _ll_register_callbacks cb 0x11ff800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:55:05.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9f6548700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106e710 0x138d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9f6548700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106e710 0x138d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35782/0 (socket says 172.21.15.104:35782) 2023-07-14T11:55:05.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9f6548700 1 -- 172.21.15.104:0/3933815811 learned_addr learned my addr 172.21.15.104:0/3933815811 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:05.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9f5d47700 1 --2- 172.21.15.104:0/3933815811 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 0x138e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9f6548700 1 -- 172.21.15.104:0/3933815811 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 msgr2=0x138e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:05.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9f6d49700 1 --2- 172.21.15.104:0/3933815811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106eeb0 0x138ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9f6548700 1 --2- 172.21.15.104:0/3933815811 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 0x138e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9f6548700 1 -- 172.21.15.104:0/3933815811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106eeb0 msgr2=0x138ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:05.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9f6548700 1 --2- 172.21.15.104:0/3933815811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106eeb0 0x138ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9f6548700 1 -- 172.21.15.104:0/3933815811 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f015b0a0 con 0x106e710 2023-07-14T11:55:05.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9f5d47700 1 --2- 172.21.15.104:0/3933815811 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 0x138e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:55:05.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9f6548700 1 --2- 172.21.15.104:0/3933815811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106e710 0x138d690 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec069010 tx=0x7fa9ec0812d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe0158959ce53bd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:05.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-14T11:55:05.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9d17fa700 1 -- 172.21.15.104:0/3933815811 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec04fee0 con 0x106e710 2023-07-14T11:55:05.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9d17fa700 1 -- 172.21.15.104:0/3933815811 <== mon.0 v2:172.21.15.104:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa9ec01cd40 con 0x106e710 2023-07-14T11:55:05.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 1 -- 172.21.15.104:0/3933815811 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9e0003a40 con 0x106e710 2023-07-14T11:55:05.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9d17fa700 1 -- 172.21.15.104:0/3933815811 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9ec01cf00 con 0x106e710 2023-07-14T11:55:05.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 10 client.5796 fetch_fsmap learned FSMap version 18 2023-07-14T11:55:05.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 1 -- 172.21.15.104:0/3933815811 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x11f35c0 con 0x106e710 2023-07-14T11:55:05.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9d17fa700 1 -- 172.21.15.104:0/3933815811 <== mon.0 v2:172.21.15.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e0003a40 con 0x106e710 2023-07-14T11:55:05.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9d17fa700 1 -- 172.21.15.104:0/3933815811 <== mon.0 v2:172.21.15.104:3300/0 5 ==== fsmap(e 18) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa9ec050040 con 0x106e710 2023-07-14T11:55:05.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 10 client.5796 fetch_fsmap finished waiting for FSMap version 18 2023-07-14T11:55:05.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 10 client.5796 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-14T11:55:05.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 20 client.5796 populate_metadata read hostname 'smithi104' 2023-07-14T11:55:05.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 1 --2- 172.21.15.104:0/3933815811 >> [v2:172.21.15.104:6836/398018673,v1:172.21.15.104:6837/398018673] conn(0x132b630 0x13296d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 4 client.5796 mds_command: new command op to 5616 tid=0[{"prefix": "status"}] 2023-07-14T11:55:05.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9fba28740 1 -- 172.21.15.104:0/3933815811 --> [v2:172.21.15.104:6836/398018673,v1:172.21.15.104:6837/398018673] -- command(tid 0: {"prefix": "status"}) v1 -- 0x10c64e0 con 0x132b630 2023-07-14T11:55:05.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.502+0000 7fa9f5d47700 1 --2- 172.21.15.104:0/3933815811 >> [v2:172.21.15.104:6836/398018673,v1:172.21.15.104:6837/398018673] conn(0x132b630 0x13296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9f5d47700 1 --2- 172.21.15.104:0/3933815811 >> [v2:172.21.15.104:6836/398018673,v1:172.21.15.104:6837/398018673] conn(0x132b630 0x13296d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb3db6e49474a2d server_cookie=af9f2ed0b8fafb1a in_seq=0 out_seq=0 2023-07-14T11:55:05.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9d17fa700 10 client.5796 ms_handle_connect on v2:172.21.15.104:6836/398018673 2023-07-14T11:55:05.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9d17fa700 1 -- 172.21.15.104:0/3933815811 <== mds.0 v2:172.21.15.104:6836/398018673 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x10c64e0 con 0x132b630 2023-07-14T11:55:05.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9d17fa700 10 client.5796 handle_command_reply: tid=0 2023-07-14T11:55:05.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9fba28740 1 client.5796 shutdown 2023-07-14T11:55:05.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9fba28740 1 -- 172.21.15.104:0/3933815811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106e710 msgr2=0x138d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9fba28740 1 --2- 172.21.15.104:0/3933815811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106e710 0x138d690 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec069010 tx=0x7fa9ec0812d0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9fba28740 1 -- 172.21.15.104:0/3933815811 shutdown_connections 2023-07-14T11:55:05.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9fba28740 1 --2- 172.21.15.104:0/3933815811 >> [v2:172.21.15.104:6836/398018673,v1:172.21.15.104:6837/398018673] conn(0x132b630 0x13296d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9fba28740 1 --2- 172.21.15.104:0/3933815811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x106eeb0 0x138ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9fba28740 1 --2- 172.21.15.104:0/3933815811 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x106eae0 0x138e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9fba28740 1 --2- 172.21.15.104:0/3933815811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x106e710 0x138d690 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9d17fa700 0 client.5796 ms_handle_reset on v2:172.21.15.104:6836/398018673 2023-07-14T11:55:05.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9fba28740 1 -- 172.21.15.104:0/3933815811 >> 172.21.15.104:0/3933815811 conn(0x12d8b60 msgr2=0x13258d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:05.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9fba28740 1 -- 172.21.15.104:0/3933815811 shutdown_connections 2023-07-14T11:55:05.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9fba28740 1 -- 172.21.15.104:0/3933815811 wait complete. 2023-07-14T11:55:05.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9fba28740 20 client.5796 trim_cache size 0 max 16384 2023-07-14T11:55:05.547 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:55:05.548 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_fsid": "c5576993-99e1-43b1-a253-659a3cee5212", 2023-07-14T11:55:05.548 INFO:teuthology.orchestra.run.smithi104.stdout: "whoami": 0, 2023-07-14T11:55:05.548 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 5616, 2023-07-14T11:55:05.548 INFO:teuthology.orchestra.run.smithi104.stdout: "want_state": "up:active", 2023-07-14T11:55:05.548 INFO:teuthology.orchestra.run.smithi104.stdout: "state": "up:active", 2023-07-14T11:55:05.548 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs", 2023-07-14T11:55:05.549 INFO:teuthology.orchestra.run.smithi104.stdout: "rank_uptime": 6.3616237900000003, 2023-07-14T11:55:05.549 INFO:teuthology.orchestra.run.smithi104.stdout: "mdsmap_epoch": 18, 2023-07-14T11:55:05.549 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_epoch": 42, 2023-07-14T11:55:05.549 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_epoch_barrier": 42, 2023-07-14T11:55:05.549 INFO:teuthology.orchestra.run.smithi104.stdout: "uptime": 15.226442295 2023-07-14T11:55:05.549 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:55:05.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9f87ac700 1 -- 172.21.15.104:0/538025122 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa9d40662f0 msgr2=0x7fa9d40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9f87ac700 1 --2- 172.21.15.104:0/538025122 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa9d40662f0 0x7fa9d40687a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0000c00 tx=0x7fa9e0008030 comp rx=0 tx=0).stop 2023-07-14T11:55:05.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9f87ac700 1 -- 172.21.15.104:0/538025122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00703a0 msgr2=0x7fa9f00c2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9f87ac700 1 --2- 172.21.15.104:0/538025122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00703a0 0x7fa9f00c2fa0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00c410 tx=0x7fa9ec00c7d0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9f87ac700 1 -- 172.21.15.104:0/538025122 shutdown_connections 2023-07-14T11:55:05.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9f87ac700 1 --2- 172.21.15.104:0/538025122 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa9d40662f0 0x7fa9d40687a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9f87ac700 1 --2- 172.21.15.104:0/538025122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9f015c070 0x7fa9f0153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9f87ac700 1 --2- 172.21.15.104:0/538025122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9f00afd30 0x7fa9f015a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9f87ac700 1 --2- 172.21.15.104:0/538025122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00703a0 0x7fa9f00c2fa0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9f87ac700 1 -- 172.21.15.104:0/538025122 >> 172.21.15.104:0/538025122 conn(0x7fa9f00cad30 msgr2=0x7fa9f006bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:05.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9f87ac700 1 -- 172.21.15.104:0/538025122 shutdown_connections 2023-07-14T11:55:05.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.506+0000 7fa9f87ac700 1 -- 172.21.15.104:0/538025122 wait complete. 2023-07-14T11:55:05.554 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-14T11:55:05.730 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.742+0000 7f1bb59c1700 1 Processor -- start 2023-07-14T11:55:05.731 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.746+0000 7f1bb59c1700 1 -- start start 2023-07-14T11:55:05.733 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.746+0000 7f1bb59c1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1bb00c9d90 0x7f1bb00ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.733 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.746+0000 7f1bb59c1700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1bb0159f20 0x7f1bb00caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.733 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.746+0000 7f1bb59c1700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb00caf70 0x7f1bb00c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.733 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.746+0000 7f1bb59c1700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb00b0ef0 con 0x7f1bb0159f20 2023-07-14T11:55:05.733 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.746+0000 7f1bb59c1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb00ce9c0 con 0x7f1bb00c9d90 2023-07-14T11:55:05.734 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.746+0000 7f1bb59c1700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb00cee50 con 0x7f1bb00caf70 2023-07-14T11:55:05.734 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.746+0000 7f1ba7fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1bb0159f20 0x7f1bb00caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.746+0000 7f1baeffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1bb00c9d90 0x7f1bb00ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.746+0000 7f1baf7fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb00caf70 0x7f1bb00c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.746+0000 7f1baeffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1bb00c9d90 0x7f1bb00ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:34268/0 (socket says 172.21.15.104:34268) 2023-07-14T11:55:05.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.746+0000 7f1baeffd700 1 -- 172.21.15.104:0/2799625126 learned_addr learned my addr 172.21.15.104:0/2799625126 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:05.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.746+0000 7f1baf7fe700 1 -- 172.21.15.104:0/2799625126 >> v2:172.21.15.121:3300/0 conn(0x7f1bb00c9d90 msgr2=0x7f1bb00ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:05.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.746+0000 7f1baf7fe700 1 --2- 172.21.15.104:0/2799625126 >> v2:172.21.15.121:3300/0 conn(0x7f1bb00c9d90 0x7f1bb00ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.746+0000 7f1baf7fe700 1 -- 172.21.15.104:0/2799625126 >> v2:172.21.15.104:3300/0 conn(0x7f1bb0159f20 msgr2=0x7f1bb00caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:05.736 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.750+0000 7f1baf7fe700 1 --2- 172.21.15.104:0/2799625126 >> v2:172.21.15.104:3300/0 conn(0x7f1bb0159f20 0x7f1bb00caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.736 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.750+0000 7f1baf7fe700 1 -- 172.21.15.104:0/2799625126 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb00ca6a0 con 0x7f1bb00caf70 2023-07-14T11:55:05.736 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.750+0000 7f1baf7fe700 1 --2- 172.21.15.104:0/2799625126 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb00caf70 0x7f1bb00c86b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00cdf30 tx=0x7f1ba800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56b8d8c0285a88b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:05.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.750+0000 7f1bae7fc700 1 -- 172.21.15.104:0/2799625126 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800e070 con 0x7f1bb00caf70 2023-07-14T11:55:05.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.750+0000 7f1bae7fc700 1 -- 172.21.15.104:0/2799625126 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba800bb20 con 0x7f1bb00caf70 2023-07-14T11:55:05.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.750+0000 7f1bae7fc700 1 -- 172.21.15.104:0/2799625126 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8012430 con 0x7f1bb00caf70 2023-07-14T11:55:05.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.750+0000 7f1bb59c1700 1 -- 172.21.15.104:0/2799625126 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb00caf70 msgr2=0x7f1bb00c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.750+0000 7f1bb59c1700 1 --2- 172.21.15.104:0/2799625126 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb00caf70 0x7f1bb00c86b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00cdf30 tx=0x7f1ba800b330 comp rx=0 tx=0).stop 2023-07-14T11:55:05.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.750+0000 7f1bb59c1700 1 -- 172.21.15.104:0/2799625126 shutdown_connections 2023-07-14T11:55:05.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.750+0000 7f1bb59c1700 1 --2- 172.21.15.104:0/2799625126 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb00caf70 0x7f1bb00c86b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.750+0000 7f1bb59c1700 1 --2- 172.21.15.104:0/2799625126 >> v2:172.21.15.104:3300/0 conn(0x7f1bb0159f20 0x7f1bb00caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.750+0000 7f1bb59c1700 1 --2- 172.21.15.104:0/2799625126 >> v2:172.21.15.121:3300/0 conn(0x7f1bb00c9d90 0x7f1bb00ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.750+0000 7f1bb59c1700 1 -- 172.21.15.104:0/2799625126 >> 172.21.15.104:0/2799625126 conn(0x7f1bb006e960 msgr2=0x7f1bb006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:05.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.750+0000 7f1bb59c1700 1 -- 172.21.15.104:0/2799625126 shutdown_connections 2023-07-14T11:55:05.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.750+0000 7f1bb59c1700 1 -- 172.21.15.104:0/2799625126 wait complete. 2023-07-14T11:55:05.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1bb59c1700 1 Processor -- start 2023-07-14T11:55:05.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1bb59c1700 1 -- start start 2023-07-14T11:55:05.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1bb59c1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb00c9d90 0x7f1bb0156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1bb59c1700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb00caf70 0x7f1bb0151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1bb59c1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bb0159f20 0x7f1bb00c1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1bb59c1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb0157c70 con 0x7f1bb0159f20 2023-07-14T11:55:05.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1bb59c1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb0155e40 con 0x7f1bb00c9d90 2023-07-14T11:55:05.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1bb59c1700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb0156400 con 0x7f1bb00caf70 2023-07-14T11:55:05.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1baf7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bb0159f20 0x7f1bb00c1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1baf7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bb0159f20 0x7f1bb00c1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35802/0 (socket says 172.21.15.104:35802) 2023-07-14T11:55:05.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1baf7fe700 1 -- 172.21.15.104:0/1719033339 learned_addr learned my addr 172.21.15.104:0/1719033339 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:05.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1baeffd700 1 --2- 172.21.15.104:0/1719033339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb00c9d90 0x7f1bb0156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1ba7fff700 1 --2- 172.21.15.104:0/1719033339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb00caf70 0x7f1bb0151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1baf7fe700 1 -- 172.21.15.104:0/1719033339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb00caf70 msgr2=0x7f1bb0151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1baf7fe700 1 --2- 172.21.15.104:0/1719033339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb00caf70 0x7f1bb0151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1baf7fe700 1 -- 172.21.15.104:0/1719033339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb00c9d90 msgr2=0x7f1bb0156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1baf7fe700 1 --2- 172.21.15.104:0/1719033339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb00c9d90 0x7f1bb0156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1baf7fe700 1 -- 172.21.15.104:0/1719033339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba8009a70 con 0x7f1bb0159f20 2023-07-14T11:55:05.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1ba7fff700 1 --2- 172.21.15.104:0/1719033339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb00caf70 0x7f1bb0151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:55:05.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1baf7fe700 1 --2- 172.21.15.104:0/1719033339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bb0159f20 0x7f1bb00c1b90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00cef90 tx=0x7f1ba8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:05.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1bacff9700 1 -- 172.21.15.104:0/1719033339 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8004960 con 0x7f1bb0159f20 2023-07-14T11:55:05.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1bacff9700 1 -- 172.21.15.104:0/1719033339 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba8020070 con 0x7f1bb0159f20 2023-07-14T11:55:05.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1bacff9700 1 -- 172.21.15.104:0/1719033339 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba80276e0 con 0x7f1bb0159f20 2023-07-14T11:55:05.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.754+0000 7f1bb59c1700 1 -- 172.21.15.104:0/1719033339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb00c2650 con 0x7f1bb0159f20 2023-07-14T11:55:05.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.758+0000 7f1bb59c1700 1 -- 172.21.15.104:0/1719033339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb0156090 con 0x7f1bb0159f20 2023-07-14T11:55:05.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.758+0000 7f1bb59c1700 1 -- 172.21.15.104:0/1719033339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb004e500 con 0x7f1bb0159f20 2023-07-14T11:55:05.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.758+0000 7f1bacff9700 1 -- 172.21.15.104:0/1719033339 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1ba8012430 con 0x7f1bb0159f20 2023-07-14T11:55:05.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.762+0000 7f1bacff9700 1 --2- 172.21.15.104:0/1719033339 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1b900662f0 0x7f1b900687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.762+0000 7f1bacff9700 1 -- 172.21.15.104:0/1719033339 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1ba8022070 con 0x7f1bb0159f20 2023-07-14T11:55:05.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.762+0000 7f1baeffd700 1 --2- 172.21.15.104:0/1719033339 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1b900662f0 0x7f1b900687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.762+0000 7f1baeffd700 1 --2- 172.21.15.104:0/1719033339 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1b900662f0 0x7f1b900687a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009a60 tx=0x7f1ba0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:05.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.766+0000 7f1bacff9700 1 -- 172.21.15.104:0/1719033339 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f1ba8052dc0 con 0x7f1bb0159f20 2023-07-14T11:55:05.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.978+0000 7f1bb59c1700 1 -- 172.21.15.104:0/1719033339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1bb0156540 con 0x7f1bb0159f20 2023-07-14T11:55:05.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.978+0000 7f1bacff9700 1 -- 172.21.15.104:0/1719033339 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1ba802c060 con 0x7f1bb0159f20 2023-07-14T11:55:05.964 INFO:teuthology.orchestra.run.smithi104.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-14T11:55:05.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.978+0000 7f1bb59c1700 1 -- 172.21.15.104:0/1719033339 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1b900662f0 msgr2=0x7f1b900687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.978+0000 7f1bb59c1700 1 --2- 172.21.15.104:0/1719033339 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1b900662f0 0x7f1b900687a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009a60 tx=0x7f1ba0008040 comp rx=0 tx=0).stop 2023-07-14T11:55:05.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.978+0000 7f1bb59c1700 1 -- 172.21.15.104:0/1719033339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bb0159f20 msgr2=0x7f1bb00c1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.978+0000 7f1bb59c1700 1 --2- 172.21.15.104:0/1719033339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bb0159f20 0x7f1bb00c1b90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00cef90 tx=0x7f1ba8004af0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.978+0000 7f1bb59c1700 1 -- 172.21.15.104:0/1719033339 shutdown_connections 2023-07-14T11:55:05.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.978+0000 7f1bb59c1700 1 --2- 172.21.15.104:0/1719033339 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1b900662f0 0x7f1b900687a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.978+0000 7f1bb59c1700 1 --2- 172.21.15.104:0/1719033339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bb0159f20 0x7f1bb00c1b90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.978+0000 7f1bb59c1700 1 --2- 172.21.15.104:0/1719033339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bb00caf70 0x7f1bb0151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.978+0000 7f1bb59c1700 1 --2- 172.21.15.104:0/1719033339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb00c9d90 0x7f1bb0156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.978+0000 7f1bb59c1700 1 -- 172.21.15.104:0/1719033339 >> 172.21.15.104:0/1719033339 conn(0x7f1bb006e960 msgr2=0x7f1bb00afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:05.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.978+0000 7f1bb59c1700 1 -- 172.21.15.104:0/1719033339 shutdown_connections 2023-07-14T11:55:05.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:05.978+0000 7f1bb59c1700 1 -- 172.21.15.104:0/1719033339 wait complete. 2023-07-14T11:55:05.983 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-14T11:55:05.983 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T11:55:05.984 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2023-07-14T11:55:05.984 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-14T11:55:05.984 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:05.985 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-14T11:55:05.985 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T11:55:05.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T11:55:05.985 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:55:05.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:05.986 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2023-07-14T11:55:06.003 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:55:06.003 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:55:06.003 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:55:06.003 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:06.004 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2023-07-14T11:55:06.004 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:06.004 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:55:06.004 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:06.004 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2023-07-14T11:55:06.004 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:55:06.004 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:06.005 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2023-07-14T11:55:06.005 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:55:06.005 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:06.005 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T11:55:06.005 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:06.005 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-07-14T11:55:06.005 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42290sec preferred_lft 42290sec 2023-07-14T11:55:06.005 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2023-07-14T11:55:06.006 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:06.006 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-07-14T11:55:06.006 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:a2:a1:53:16 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:06.006 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-07-14T11:55:06.006 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:06.006 INFO:teuthology.orchestra.run.smithi104.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-14T11:55:06.006 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:b8:27:80:0d:ab brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:06.007 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-14T11:55:06.007 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:06.007 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5020:39ff:fe23:761d/64 scope link 2023-07-14T11:55:06.007 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:06.007 INFO:teuthology.orchestra.run.smithi104.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-14T11:55:06.007 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:b8:27:80:0d:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-14T11:55:06.007 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::e8b8:27ff:fe80:dab/64 scope link 2023-07-14T11:55:06.007 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:06.008 INFO:teuthology.orchestra.run.smithi104.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-14T11:55:06.008 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:e6:23:3b:72:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-14T11:55:06.008 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::e8e6:23ff:fe3b:7247/64 scope link 2023-07-14T11:55:06.008 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:06.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:06.009 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T11:55:06.091 INFO:teuthology.orchestra.run.smithi104.stdout:1 2023-07-14T11:55:06.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:06.094 DEBUG:teuthology.orchestra.run.smithi104:> route 2023-07-14T11:55:06.153 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2023-07-14T11:55:06.153 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T11:55:06.153 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-07-14T11:55:06.153 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T11:55:06.153 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-07-14T11:55:06.154 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-07-14T11:55:06.154 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-07-14T11:55:06.154 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T11:55:06.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:06.155 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:55:06.155 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:55:06.155 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T11:55:06.156 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T11:55:06.156 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T11:55:06.156 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:55:06.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:06.283 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2023-07-14T11:55:06.300 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-14T11:55:06.301 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-14T11:55:06.301 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:55:06.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:06.302 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2023-07-14T11:55:06.378 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-14T11:55:06.379 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:55:06.379 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:06.443 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:55:06.443 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:55:06.443 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:06.504 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:55:06.587 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:55:06.588 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:55:06.662 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:55:06.664 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:55:06.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:06.667 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:55:06.728 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-14T11:55:06.729 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-14T11:55:06.776 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:55:06.808 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:55:06.809 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:55:06.885 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:55:06.887 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:55:06.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:06.889 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:55:06.926 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:2023-07-14T11:55:06.938+0000 7f1c86a86640 -1 init, newargv = 0x560fcc289c10 newargc=15 2023-07-14T11:55:06.926 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[47028]: starting ceph client 2023-07-14T11:55:06.937 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[47028]: starting fuse 2023-07-14T11:55:06.940 INFO:teuthology.orchestra.run.smithi104.stdout:49 2023-07-14T11:55:06.941 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-07-14T11:55:06.942 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T11:55:06.942 DEBUG:teuthology.orchestra.run.smithi104:> import glob 2023-07-14T11:55:06.943 DEBUG:teuthology.orchestra.run.smithi104:> import re 2023-07-14T11:55:06.943 DEBUG:teuthology.orchestra.run.smithi104:> import os 2023-07-14T11:55:06.944 DEBUG:teuthology.orchestra.run.smithi104:> import subprocess 2023-07-14T11:55:06.944 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:55:06.944 DEBUG:teuthology.orchestra.run.smithi104:> def _find_admin_socket(client_name): 2023-07-14T11:55:06.945 DEBUG:teuthology.orchestra.run.smithi104:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-14T11:55:06.945 DEBUG:teuthology.orchestra.run.smithi104:> files = glob.glob(asok_path) 2023-07-14T11:55:06.945 DEBUG:teuthology.orchestra.run.smithi104:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-14T11:55:06.946 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:55:06.946 DEBUG:teuthology.orchestra.run.smithi104:> # Given a non-glob path, it better be there 2023-07-14T11:55:06.946 DEBUG:teuthology.orchestra.run.smithi104:> if "*" not in asok_path: 2023-07-14T11:55:06.947 DEBUG:teuthology.orchestra.run.smithi104:> assert(len(files) == 1) 2023-07-14T11:55:06.947 DEBUG:teuthology.orchestra.run.smithi104:> return files[0] 2023-07-14T11:55:06.948 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:55:06.948 DEBUG:teuthology.orchestra.run.smithi104:> for f in files: 2023-07-14T11:55:06.949 DEBUG:teuthology.orchestra.run.smithi104:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-14T11:55:06.949 DEBUG:teuthology.orchestra.run.smithi104:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-14T11:55:06.949 DEBUG:teuthology.orchestra.run.smithi104:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-14T11:55:06.950 DEBUG:teuthology.orchestra.run.smithi104:> contents = proc_f.read() 2023-07-14T11:55:06.950 DEBUG:teuthology.orchestra.run.smithi104:> if mountpoint in contents: 2023-07-14T11:55:06.951 DEBUG:teuthology.orchestra.run.smithi104:> return f 2023-07-14T11:55:06.951 DEBUG:teuthology.orchestra.run.smithi104:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-14T11:55:06.951 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:55:06.951 DEBUG:teuthology.orchestra.run.smithi104:> print(_find_admin_socket("client.0")) 2023-07-14T11:55:06.952 DEBUG:teuthology.orchestra.run.smithi104:> ' 2023-07-14T11:55:07.102 INFO:teuthology.orchestra.run.smithi104.stdout:/var/run/ceph/ceph-client.0.47028.asok 2023-07-14T11:55:07.287 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47028.asok 2023-07-14T11:55:07.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:07.288 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47028.asok status 2023-07-14T11:55:07.486 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:55:07.486 INFO:teuthology.orchestra.run.smithi104.stdout: "metadata": { 2023-07-14T11:55:07.486 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-14T11:55:07.487 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:55:07.487 INFO:teuthology.orchestra.run.smithi104.stdout: "entity_id": "0", 2023-07-14T11:55:07.487 INFO:teuthology.orchestra.run.smithi104.stdout: "hostname": "smithi104", 2023-07-14T11:55:07.487 INFO:teuthology.orchestra.run.smithi104.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-14T11:55:07.487 INFO:teuthology.orchestra.run.smithi104.stdout: "pid": "47028", 2023-07-14T11:55:07.487 INFO:teuthology.orchestra.run.smithi104.stdout: "root": "/" 2023-07-14T11:55:07.487 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:55:07.488 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_count": 0, 2023-07-14T11:55:07.488 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_pinned_count": 0, 2023-07-14T11:55:07.488 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 5808, 2023-07-14T11:55:07.488 INFO:teuthology.orchestra.run.smithi104.stdout: "inst": { 2023-07-14T11:55:07.488 INFO:teuthology.orchestra.run.smithi104.stdout: "name": { 2023-07-14T11:55:07.488 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "client", 2023-07-14T11:55:07.488 INFO:teuthology.orchestra.run.smithi104.stdout: "num": 5808 2023-07-14T11:55:07.489 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:55:07.489 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:55:07.489 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:55:07.489 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:55:07.489 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 3449775486 2023-07-14T11:55:07.489 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:55:07.489 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:55:07.489 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:55:07.490 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:55:07.490 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:55:07.490 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 3449775486 2023-07-14T11:55:07.490 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:55:07.490 INFO:teuthology.orchestra.run.smithi104.stdout: "inst_str": "client.5808 192.168.0.1:0/3449775486", 2023-07-14T11:55:07.490 INFO:teuthology.orchestra.run.smithi104.stdout: "addr_str": "192.168.0.1:0/3449775486", 2023-07-14T11:55:07.490 INFO:teuthology.orchestra.run.smithi104.stdout: "inode_count": 1, 2023-07-14T11:55:07.490 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_epoch": 18, 2023-07-14T11:55:07.491 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch": 42, 2023-07-14T11:55:07.491 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch_barrier": 0, 2023-07-14T11:55:07.491 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklisted": false, 2023-07-14T11:55:07.491 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs" 2023-07-14T11:55:07.491 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:55:07.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:07.501 DEBUG:teuthology.orchestra.run.smithi104:> stat --file-system '--printf=%T 2023-07-14T11:55:07.501 DEBUG:teuthology.orchestra.run.smithi104:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:07.519 INFO:teuthology.orchestra.run.smithi104.stdout:fuseblk 2023-07-14T11:55:07.520 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:07.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:07.522 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:07.596 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2023-07-14T11:55:07.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.830+0000 7f8d159a7700 1 Processor -- start 2023-07-14T11:55:07.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.830+0000 7f8d159a7700 1 -- start start 2023-07-14T11:55:07.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.830+0000 7f8d159a7700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8d100ce300 0x7f8d1006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:07.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.830+0000 7f8d159a7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d100b40f0 0x7f8d10070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:07.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.830+0000 7f8d159a7700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d10071310 con 0x7f8d100d02d0 2023-07-14T11:55:07.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.830+0000 7f8d159a7700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d10071bb0 con 0x7f8d100ce300 2023-07-14T11:55:07.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.830+0000 7f8d159a7700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d100715f0 con 0x7f8d100b40f0 2023-07-14T11:55:07.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.830+0000 7f8d0e7fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8d100ce300 0x7f8d1006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.830+0000 7f8d0f7fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d100b40f0 0x7f8d10070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.830+0000 7f8d0e7fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8d100ce300 0x7f8d1006fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35818/0 (socket says 172.21.15.104:35818) 2023-07-14T11:55:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.830+0000 7f8d0e7fc700 1 -- 172.21.15.104:0/3697915263 learned_addr learned my addr 172.21.15.104:0/3697915263 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d0e7fc700 1 -- 172.21.15.104:0/3697915263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d100b40f0 msgr2=0x7f8d10070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d0e7fc700 1 --2- 172.21.15.104:0/3697915263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d100b40f0 0x7f8d10070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d0e7fc700 1 -- 172.21.15.104:0/3697915263 >> v1:172.21.15.121:6789/0 conn(0x7f8d100d02d0 legacy=0x7f8d100d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d0e7fc700 1 -- 172.21.15.104:0/3697915263 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d100718d0 con 0x7f8d100ce300 2023-07-14T11:55:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d0e7fc700 1 --2- 172.21.15.104:0/3697915263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d100ce300 0x7f8d1006fd20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009910 tx=0x7f8d04009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492e5a13df7f9772 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d0dffb700 1 -- 172.21.15.104:0/3697915263 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04011830 con 0x7f8d100ce300 2023-07-14T11:55:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d0dffb700 1 -- 172.21.15.104:0/3697915263 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d04011990 con 0x7f8d100ce300 2023-07-14T11:55:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d0dffb700 1 -- 172.21.15.104:0/3697915263 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04011d40 con 0x7f8d100ce300 2023-07-14T11:55:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d159a7700 1 -- 172.21.15.104:0/3697915263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d100ce300 msgr2=0x7f8d1006fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d159a7700 1 --2- 172.21.15.104:0/3697915263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d100ce300 0x7f8d1006fd20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009910 tx=0x7f8d04009c20 comp rx=0 tx=0).stop 2023-07-14T11:55:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d159a7700 1 -- 172.21.15.104:0/3697915263 shutdown_connections 2023-07-14T11:55:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d159a7700 1 --2- 172.21.15.104:0/3697915263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d100b40f0 0x7f8d10070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d159a7700 1 --2- 172.21.15.104:0/3697915263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d100ce300 0x7f8d1006fd20 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d159a7700 1 -- 172.21.15.104:0/3697915263 >> 172.21.15.104:0/3697915263 conn(0x7f8d1009b9e0 msgr2=0x7f8d100d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d159a7700 1 -- 172.21.15.104:0/3697915263 shutdown_connections 2023-07-14T11:55:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.834+0000 7f8d159a7700 1 -- 172.21.15.104:0/3697915263 wait complete. 2023-07-14T11:55:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.838+0000 7f8d159a7700 1 Processor -- start 2023-07-14T11:55:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.838+0000 7f8d159a7700 1 -- start start 2023-07-14T11:55:07.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.838+0000 7f8d159a7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d100b40f0 0x7f8d1009ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.838+0000 7f8d159a7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d100ce300 0x7f8d1015cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.838+0000 7f8d159a7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d100d02d0 0x7f8d100ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.838+0000 7f8d159a7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101f1ec0 con 0x7f8d100ce300 2023-07-14T11:55:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.838+0000 7f8d159a7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101f1900 con 0x7f8d100b40f0 2023-07-14T11:55:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.838+0000 7f8d159a7700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101f1be0 con 0x7f8d100d02d0 2023-07-14T11:55:07.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.838+0000 7f8d0e7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d100ce300 0x7f8d1015cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.838+0000 7f8d0e7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d100ce300 0x7f8d1015cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35828/0 (socket says 172.21.15.104:35828) 2023-07-14T11:55:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.838+0000 7f8d0e7fc700 1 -- 172.21.15.104:0/987431845 learned_addr learned my addr 172.21.15.104:0/987431845 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.838+0000 7f8d0f7fe700 1 --2- 172.21.15.104:0/987431845 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d100d02d0 0x7f8d100ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.838+0000 7f8d0effd700 1 --2- 172.21.15.104:0/987431845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d100b40f0 0x7f8d1009ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.842+0000 7f8d0e7fc700 1 -- 172.21.15.104:0/987431845 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d100d02d0 msgr2=0x7f8d100ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.842+0000 7f8d0e7fc700 1 --2- 172.21.15.104:0/987431845 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d100d02d0 0x7f8d100ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.842+0000 7f8d0e7fc700 1 -- 172.21.15.104:0/987431845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d100b40f0 msgr2=0x7f8d1009ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.842+0000 7f8d0e7fc700 1 --2- 172.21.15.104:0/987431845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d100b40f0 0x7f8d1009ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.842+0000 7f8d0e7fc700 1 -- 172.21.15.104:0/987431845 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d04009570 con 0x7f8d100ce300 2023-07-14T11:55:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.842+0000 7f8d0e7fc700 1 --2- 172.21.15.104:0/987431845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d100ce300 0x7f8d1015cdd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009860 tx=0x7f8d04004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.842+0000 7f8cf7fff700 1 -- 172.21.15.104:0/987431845 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04004590 con 0x7f8d100ce300 2023-07-14T11:55:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.842+0000 7f8cf7fff700 1 -- 172.21.15.104:0/987431845 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0401f070 con 0x7f8d100ce300 2023-07-14T11:55:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.842+0000 7f8cf7fff700 1 -- 172.21.15.104:0/987431845 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04003900 con 0x7f8d100ce300 2023-07-14T11:55:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.842+0000 7f8d159a7700 1 -- 172.21.15.104:0/987431845 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d100c9830 con 0x7f8d100ce300 2023-07-14T11:55:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.842+0000 7f8d159a7700 1 -- 172.21.15.104:0/987431845 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d101f2040 con 0x7f8d100ce300 2023-07-14T11:55:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.842+0000 7f8d159a7700 1 -- 172.21.15.104:0/987431845 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1004dcb0 con 0x7f8d100ce300 2023-07-14T11:55:07.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.846+0000 7f8cf7fff700 1 -- 172.21.15.104:0/987431845 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8d04003a60 con 0x7f8d100ce300 2023-07-14T11:55:07.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.846+0000 7f8cf7fff700 1 --2- 172.21.15.104:0/987431845 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cfc0662f0 0x7f8cfc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:07.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.846+0000 7f8cf7fff700 1 -- 172.21.15.104:0/987431845 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8d04021070 con 0x7f8d100ce300 2023-07-14T11:55:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.846+0000 7f8d0effd700 1 --2- 172.21.15.104:0/987431845 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cfc0662f0 0x7f8cfc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.846+0000 7f8d0effd700 1 --2- 172.21.15.104:0/987431845 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cfc0662f0 0x7f8cfc0687a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8d00000f60 tx=0x7f8d00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:07.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:07.854+0000 7f8cf7fff700 1 -- 172.21.15.104:0/987431845 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f8d040524a0 con 0x7f8d100ce300 2023-07-14T11:55:07.882 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:55:07.889 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:55:08.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.030+0000 7f8d159a7700 1 -- 172.21.15.104:0/987431845 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) v1 -- 0x7f8d101f2230 con 0x7f8d100ce300 2023-07-14T11:55:08.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.630+0000 7f8cf7fff700 1 -- 172.21.15.104:0/987431845 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v43) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8d04055ac0 con 0x7f8d100ce300 2023-07-14T11:55:08.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.634+0000 7f8d159a7700 1 -- 172.21.15.104:0/987431845 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cfc0662f0 msgr2=0x7f8cfc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:08.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.634+0000 7f8d159a7700 1 --2- 172.21.15.104:0/987431845 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cfc0662f0 0x7f8cfc0687a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8d00000f60 tx=0x7f8d00009040 comp rx=0 tx=0).stop 2023-07-14T11:55:08.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.634+0000 7f8d159a7700 1 -- 172.21.15.104:0/987431845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d100ce300 msgr2=0x7f8d1015cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:08.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.634+0000 7f8d159a7700 1 --2- 172.21.15.104:0/987431845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d100ce300 0x7f8d1015cdd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009860 tx=0x7f8d04004770 comp rx=0 tx=0).stop 2023-07-14T11:55:08.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.634+0000 7f8d159a7700 1 -- 172.21.15.104:0/987431845 shutdown_connections 2023-07-14T11:55:08.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.634+0000 7f8d159a7700 1 --2- 172.21.15.104:0/987431845 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cfc0662f0 0x7f8cfc0687a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:08.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.634+0000 7f8d159a7700 1 --2- 172.21.15.104:0/987431845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d100ce300 0x7f8d1015cdd0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.634+0000 7f8d159a7700 1 --2- 172.21.15.104:0/987431845 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d100d02d0 0x7f8d100ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.634+0000 7f8d159a7700 1 --2- 172.21.15.104:0/987431845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d100b40f0 0x7f8d1009ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.634+0000 7f8d159a7700 1 -- 172.21.15.104:0/987431845 >> 172.21.15.104:0/987431845 conn(0x7f8d1009b9e0 msgr2=0x7f8d100cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:08.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.634+0000 7f8d159a7700 1 -- 172.21.15.104:0/987431845 shutdown_connections 2023-07-14T11:55:08.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.638+0000 7f8d159a7700 1 -- 172.21.15.104:0/987431845 wait complete. 2023-07-14T11:55:08.626 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'data_new' created 2023-07-14T11:55:08.647 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs data_new 2023-07-14T11:55:08.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.834+0000 7f82e239e700 1 Processor -- start 2023-07-14T11:55:08.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.834+0000 7f82e239e700 1 -- start start 2023-07-14T11:55:08.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.838+0000 7f82e239e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f82dc0b0ee0 0x7f82dc0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:08.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.838+0000 7f82e239e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82dc0cd090 0x7f82dc0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:08.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.838+0000 7f82e239e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc0cd4e0 con 0x7f82dc0d0610 2023-07-14T11:55:08.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.838+0000 7f82e239e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82dc0d1a60 con 0x7f82dc0b0ee0 2023-07-14T11:55:08.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.838+0000 7f82e239e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0d1ef0 con 0x7f82dc0cd090 2023-07-14T11:55:08.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.838+0000 7f82e139c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f82dc0b0ee0 0x7f82dc0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:08.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.838+0000 7f82e1b9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f82dc0d0610 0x7f82dc15f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:47236/0 (socket says 172.21.15.104:47236) 2023-07-14T11:55:08.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.838+0000 7f82e1b9d700 1 -- 172.21.15.104:0/3385455272 learned_addr learned my addr 172.21.15.104:0/3385455272 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:08.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.838+0000 7f82e0b9b700 1 --2- 172.21.15.104:0/3385455272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82dc0cd090 0x7f82dc0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:08.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.838+0000 7f82e139c700 1 -- 172.21.15.104:0/3385455272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82dc0cd090 msgr2=0x7f82dc0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:08.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.838+0000 7f82e139c700 1 --2- 172.21.15.104:0/3385455272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82dc0cd090 0x7f82dc0d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:08.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.838+0000 7f82e139c700 1 -- 172.21.15.104:0/3385455272 >> v1:172.21.15.104:6789/0 conn(0x7f82dc0d0610 legacy=0x7f82dc15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:08.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.838+0000 7f82e139c700 1 -- 172.21.15.104:0/3385455272 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc0d0e50 con 0x7f82dc0b0ee0 2023-07-14T11:55:08.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.838+0000 7f82e0b9b700 1 --2- 172.21.15.104:0/3385455272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82dc0cd090 0x7f82dc0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:08.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.838+0000 7f82d3fff700 1 -- 172.21.15.104:0/3385455272 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063277778 0 0) 0x7f82d4001490 con 0x7f82dc0d0610 2023-07-14T11:55:08.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.842+0000 7f82e139c700 1 --2- 172.21.15.104:0/3385455272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82dc0b0ee0 0x7f82dc0ccb50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009880 tx=0x7f82d8009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9f5342abacd03d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:08.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.842+0000 7f82d3fff700 1 -- 172.21.15.104:0/3385455272 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d8011830 con 0x7f82dc0b0ee0 2023-07-14T11:55:08.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.842+0000 7f82d3fff700 1 -- 172.21.15.104:0/3385455272 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d8011990 con 0x7f82dc0b0ee0 2023-07-14T11:55:08.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.842+0000 7f82d3fff700 1 -- 172.21.15.104:0/3385455272 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d8011d20 con 0x7f82dc0b0ee0 2023-07-14T11:55:08.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.842+0000 7f82e239e700 1 -- 172.21.15.104:0/3385455272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82dc0b0ee0 msgr2=0x7f82dc0ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:08.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.842+0000 7f82e239e700 1 --2- 172.21.15.104:0/3385455272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82dc0b0ee0 0x7f82dc0ccb50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009880 tx=0x7f82d8009b90 comp rx=0 tx=0).stop 2023-07-14T11:55:08.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.842+0000 7f82e239e700 1 -- 172.21.15.104:0/3385455272 shutdown_connections 2023-07-14T11:55:08.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.842+0000 7f82e239e700 1 --2- 172.21.15.104:0/3385455272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82dc0cd090 0x7f82dc0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:08.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.842+0000 7f82e239e700 1 --2- 172.21.15.104:0/3385455272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82dc0b0ee0 0x7f82dc0ccb50 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:08.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.842+0000 7f82e239e700 1 -- 172.21.15.104:0/3385455272 >> 172.21.15.104:0/3385455272 conn(0x7f82dc060c40 msgr2=0x7f82dc09b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:08.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.842+0000 7f82e239e700 1 -- 172.21.15.104:0/3385455272 shutdown_connections 2023-07-14T11:55:08.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.842+0000 7f82e239e700 1 -- 172.21.15.104:0/3385455272 wait complete. 2023-07-14T11:55:08.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.846+0000 7f82e239e700 1 Processor -- start 2023-07-14T11:55:08.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.846+0000 7f82e239e700 1 -- start start 2023-07-14T11:55:08.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.846+0000 7f82e239e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82dc0b0ee0 0x7f82dc15abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:08.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.846+0000 7f82e239e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82dc0cd090 0x7f82dc0b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:08.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.846+0000 7f82e239e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82dc0d0610 0x7f82dc0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:08.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.846+0000 7f82e239e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc15c380 con 0x7f82dc0d0610 2023-07-14T11:55:08.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.846+0000 7f82e239e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc15c660 con 0x7f82dc0cd090 2023-07-14T11:55:08.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.846+0000 7f82e239e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc15c940 con 0x7f82dc0b0ee0 2023-07-14T11:55:08.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.846+0000 7f82e1b9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82dc0d0610 0x7f82dc0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:08.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.846+0000 7f82e139c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82dc0b0ee0 0x7f82dc15abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:08.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.846+0000 7f82e139c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82dc0b0ee0 0x7f82dc15abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:43742/0 (socket says 172.21.15.104:43742) 2023-07-14T11:55:08.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.846+0000 7f82e139c700 1 -- 172.21.15.104:0/3550298201 learned_addr learned my addr 172.21.15.104:0/3550298201 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:08.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.846+0000 7f82e1b9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82dc0d0610 0x7f82dc0cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35850/0 (socket says 172.21.15.104:35850) 2023-07-14T11:55:08.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.846+0000 7f82e0b9b700 1 --2- 172.21.15.104:0/3550298201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82dc0cd090 0x7f82dc0b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:08.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.850+0000 7f82e1b9d700 1 -- 172.21.15.104:0/3550298201 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82dc0b0ee0 msgr2=0x7f82dc15abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:08.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.850+0000 7f82e1b9d700 1 --2- 172.21.15.104:0/3550298201 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82dc0b0ee0 0x7f82dc15abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:08.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.850+0000 7f82e1b9d700 1 -- 172.21.15.104:0/3550298201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82dc0cd090 msgr2=0x7f82dc0b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:08.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.850+0000 7f82e1b9d700 1 --2- 172.21.15.104:0/3550298201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82dc0cd090 0x7f82dc0b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:08.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.850+0000 7f82e1b9d700 1 -- 172.21.15.104:0/3550298201 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d80094b0 con 0x7f82dc0d0610 2023-07-14T11:55:08.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.850+0000 7f82e139c700 1 --2- 172.21.15.104:0/3550298201 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82dc0b0ee0 0x7f82dc15abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:55:08.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.850+0000 7f82e1b9d700 1 --2- 172.21.15.104:0/3550298201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82dc0d0610 0x7f82dc0cb7a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f82d400cc10 tx=0x7f82d400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:08.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.850+0000 7f82d1ffb700 1 -- 172.21.15.104:0/3550298201 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d400aaa0 con 0x7f82dc0d0610 2023-07-14T11:55:08.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.850+0000 7f82d1ffb700 1 -- 172.21.15.104:0/3550298201 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d400ac20 con 0x7f82dc0d0610 2023-07-14T11:55:08.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.850+0000 7f82d1ffb700 1 -- 172.21.15.104:0/3550298201 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d40070d0 con 0x7f82dc0d0610 2023-07-14T11:55:08.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.850+0000 7f82e239e700 1 -- 172.21.15.104:0/3550298201 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc15cc20 con 0x7f82dc0d0610 2023-07-14T11:55:08.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.850+0000 7f82e239e700 1 -- 172.21.15.104:0/3550298201 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc15b0f0 con 0x7f82dc0d0610 2023-07-14T11:55:08.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.850+0000 7f82e239e700 1 -- 172.21.15.104:0/3550298201 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc15d140 con 0x7f82dc0d0610 2023-07-14T11:55:08.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.854+0000 7f82d1ffb700 1 -- 172.21.15.104:0/3550298201 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f82d4007230 con 0x7f82dc0d0610 2023-07-14T11:55:08.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.854+0000 7f82d1ffb700 1 --2- 172.21.15.104:0/3550298201 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f82c00662f0 0x7f82c00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:08.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.854+0000 7f82d1ffb700 1 -- 172.21.15.104:0/3550298201 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f82d4083710 con 0x7f82dc0d0610 2023-07-14T11:55:08.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.858+0000 7f82e139c700 1 --2- 172.21.15.104:0/3550298201 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f82c00662f0 0x7f82c00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:08.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.858+0000 7f82e139c700 1 --2- 172.21.15.104:0/3550298201 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f82c00662f0 0x7f82c00687a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f82d8000c00 tx=0x7f82d8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:08.847 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:08.862+0000 7f82d1ffb700 1 -- 172.21.15.104:0/3550298201 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f82dc15d140 con 0x7f82dc0d0610 2023-07-14T11:55:09.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.070+0000 7f82e239e700 1 -- 172.21.15.104:0/3550298201 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"} v 0) v1 -- 0x7f82dc0cd4e0 con 0x7f82dc0d0610 2023-07-14T11:55:09.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.662+0000 7f82d1ffb700 1 -- 172.21.15.104:0/3550298201 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"}]=0 added data pool 8 to fsmap v19) v1 ==== 131+0+0 (secure 0 0 0) 0x7f82dc0cd4e0 con 0x7f82dc0d0610 2023-07-14T11:55:09.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.666+0000 7f82e239e700 1 -- 172.21.15.104:0/3550298201 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f82c00662f0 msgr2=0x7f82c00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:09.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.666+0000 7f82e239e700 1 --2- 172.21.15.104:0/3550298201 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f82c00662f0 0x7f82c00687a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f82d8000c00 tx=0x7f82d8008030 comp rx=0 tx=0).stop 2023-07-14T11:55:09.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.666+0000 7f82e239e700 1 -- 172.21.15.104:0/3550298201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82dc0d0610 msgr2=0x7f82dc0cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:09.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.666+0000 7f82e239e700 1 --2- 172.21.15.104:0/3550298201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82dc0d0610 0x7f82dc0cb7a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f82d400cc10 tx=0x7f82d400cf20 comp rx=0 tx=0).stop 2023-07-14T11:55:09.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.666+0000 7f82e239e700 1 -- 172.21.15.104:0/3550298201 shutdown_connections 2023-07-14T11:55:09.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.666+0000 7f82e239e700 1 --2- 172.21.15.104:0/3550298201 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f82c00662f0 0x7f82c00687a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:09.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.666+0000 7f82e239e700 1 --2- 172.21.15.104:0/3550298201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82dc0d0610 0x7f82dc0cb7a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:09.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.666+0000 7f82e239e700 1 --2- 172.21.15.104:0/3550298201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82dc0cd090 0x7f82dc0b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:09.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.666+0000 7f82e239e700 1 --2- 172.21.15.104:0/3550298201 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82dc0b0ee0 0x7f82dc15abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:09.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.666+0000 7f82e239e700 1 -- 172.21.15.104:0/3550298201 >> 172.21.15.104:0/3550298201 conn(0x7f82dc060c40 msgr2=0x7f82dc15b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:09.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.666+0000 7f82e239e700 1 -- 172.21.15.104:0/3550298201 shutdown_connections 2023-07-14T11:55:09.655 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.666+0000 7f82e239e700 1 -- 172.21.15.104:0/3550298201 wait complete. 2023-07-14T11:55:09.655 INFO:teuthology.orchestra.run.smithi104.stderr:added data pool 8 to fsmap 2023-07-14T11:55:09.672 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:55:09.847 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.862+0000 7f02c759e700 1 Processor -- start 2023-07-14T11:55:09.847 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.862+0000 7f02c759e700 1 -- start start 2023-07-14T11:55:09.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c759e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f02c80b0de0 0x7f02c81669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:09.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c759e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c80af0a0 con 0x7f02c80cd240 2023-07-14T11:55:09.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c759e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c80cf720 con 0x7f02c815bff0 2023-07-14T11:55:09.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c759e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c80af2f0 con 0x7f02c80b0de0 2023-07-14T11:55:09.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c6d9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f02c80b0de0 0x7f02c81669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:09.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c5d9b700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f02c80cd240 0x7f02c80ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:47250/0 (socket says 172.21.15.104:47250) 2023-07-14T11:55:09.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c5d9b700 1 -- 172.21.15.104:0/4237819386 learned_addr learned my addr 172.21.15.104:0/4237819386 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:09.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c6d9d700 1 -- 172.21.15.104:0/4237819386 >> v1:172.21.15.121:6789/0 conn(0x7f02c815bff0 legacy=0x7f02c80ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:09.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c6d9d700 1 -- 172.21.15.104:0/4237819386 >> v1:172.21.15.104:6789/0 conn(0x7f02c80cd240 legacy=0x7f02c80ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:09.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c6d9d700 1 -- 172.21.15.104:0/4237819386 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c80af5d0 con 0x7f02c80b0de0 2023-07-14T11:55:09.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c559a700 1 -- 172.21.15.104:0/4237819386 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625546906 0 0) 0x7f02c80af0a0 con 0x7f02c80cd240 2023-07-14T11:55:09.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c6d9d700 1 --2- 172.21.15.104:0/4237819386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02c80b0de0 0x7f02c81669d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009d90 tx=0x7f02c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d016e234dc61e842 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:09.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c559a700 1 -- 172.21.15.104:0/4237819386 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c000e070 con 0x7f02c80b0de0 2023-07-14T11:55:09.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c559a700 1 -- 172.21.15.104:0/4237819386 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c000bbd0 con 0x7f02c80b0de0 2023-07-14T11:55:09.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c559a700 1 -- 172.21.15.104:0/4237819386 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0012430 con 0x7f02c80b0de0 2023-07-14T11:55:09.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c759e700 1 -- 172.21.15.104:0/4237819386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02c80b0de0 msgr2=0x7f02c81669d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:09.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c759e700 1 --2- 172.21.15.104:0/4237819386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02c80b0de0 0x7f02c81669d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009d90 tx=0x7f02c000b330 comp rx=0 tx=0).stop 2023-07-14T11:55:09.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c759e700 1 -- 172.21.15.104:0/4237819386 shutdown_connections 2023-07-14T11:55:09.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c759e700 1 --2- 172.21.15.104:0/4237819386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02c80b0de0 0x7f02c81669d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:09.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c759e700 1 -- 172.21.15.104:0/4237819386 >> 172.21.15.104:0/4237819386 conn(0x7f02c8060c40 msgr2=0x7f02c809b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:09.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.866+0000 7f02c759e700 1 -- 172.21.15.104:0/4237819386 shutdown_connections 2023-07-14T11:55:09.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.870+0000 7f02c759e700 1 -- 172.21.15.104:0/4237819386 wait complete. 2023-07-14T11:55:09.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.870+0000 7f02c759e700 1 Processor -- start 2023-07-14T11:55:09.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.870+0000 7f02c759e700 1 -- start start 2023-07-14T11:55:09.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02c80b0de0 0x7f02c815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:09.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02c80cd240 0x7f02c81529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:09.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c759e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02c815bff0 0x7f02c815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:09.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c759e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c80c6500 con 0x7f02c815bff0 2023-07-14T11:55:09.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c80b5310 con 0x7f02c80cd240 2023-07-14T11:55:09.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c80b55f0 con 0x7f02c80b0de0 2023-07-14T11:55:09.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c6d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02c815bff0 0x7f02c815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:09.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c6d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02c815bff0 0x7f02c815c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35866/0 (socket says 172.21.15.104:35866) 2023-07-14T11:55:09.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c6d9d700 1 -- 172.21.15.104:0/2861851572 learned_addr learned my addr 172.21.15.104:0/2861851572 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:09.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c5d9b700 1 --2- 172.21.15.104:0/2861851572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02c80cd240 0x7f02c81529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:09.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c659c700 1 --2- 172.21.15.104:0/2861851572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02c80b0de0 0x7f02c815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:09.863 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c6d9d700 1 -- 172.21.15.104:0/2861851572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02c80b0de0 msgr2=0x7f02c815ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:09.863 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c6d9d700 1 --2- 172.21.15.104:0/2861851572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02c80b0de0 0x7f02c815ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:09.863 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c6d9d700 1 -- 172.21.15.104:0/2861851572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02c80cd240 msgr2=0x7f02c81529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:09.863 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c6d9d700 1 --2- 172.21.15.104:0/2861851572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02c80cd240 0x7f02c81529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:09.863 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c6d9d700 1 -- 172.21.15.104:0/2861851572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c0009a70 con 0x7f02c815bff0 2023-07-14T11:55:09.863 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.874+0000 7f02c659c700 1 --2- 172.21.15.104:0/2861851572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02c80b0de0 0x7f02c815ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:09.865 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.878+0000 7f02c6d9d700 1 --2- 172.21.15.104:0/2861851572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02c815bff0 0x7f02c815c3c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f02c0014040 tx=0x7f02c0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:09.865 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.878+0000 7f02b6ffd700 1 -- 172.21.15.104:0/2861851572 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0012430 con 0x7f02c815bff0 2023-07-14T11:55:09.865 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.878+0000 7f02b6ffd700 1 -- 172.21.15.104:0/2861851572 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c0020070 con 0x7f02c815bff0 2023-07-14T11:55:09.865 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.878+0000 7f02b6ffd700 1 -- 172.21.15.104:0/2861851572 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0004280 con 0x7f02c815bff0 2023-07-14T11:55:09.865 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.878+0000 7f02c759e700 1 -- 172.21.15.104:0/2861851572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c80b58d0 con 0x7f02c815bff0 2023-07-14T11:55:09.865 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.878+0000 7f02c759e700 1 -- 172.21.15.104:0/2861851572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c80b5e90 con 0x7f02c815bff0 2023-07-14T11:55:09.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.878+0000 7f02c759e700 1 -- 172.21.15.104:0/2861851572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c804e500 con 0x7f02c815bff0 2023-07-14T11:55:09.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.882+0000 7f02b6ffd700 1 -- 172.21.15.104:0/2861851572 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f02c00043e0 con 0x7f02c815bff0 2023-07-14T11:55:09.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.882+0000 7f02b6ffd700 1 --2- 172.21.15.104:0/2861851572 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f02a8066320 0x7f02a80687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:09.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.882+0000 7f02b6ffd700 1 -- 172.21.15.104:0/2861851572 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f02c0022070 con 0x7f02c815bff0 2023-07-14T11:55:09.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.882+0000 7f02c659c700 1 --2- 172.21.15.104:0/2861851572 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f02a8066320 0x7f02a80687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:09.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.882+0000 7f02c659c700 1 --2- 172.21.15.104:0/2861851572 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f02a8066320 0x7f02a80687d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f02b8000f60 tx=0x7f02b8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:09.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:09.886+0000 7f02b6ffd700 1 -- 172.21.15.104:0/2861851572 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f02c0052fe0 con 0x7f02c815bff0 2023-07-14T11:55:10.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.098+0000 7f02c759e700 1 -- 172.21.15.104:0/2861851572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02c815f250 con 0x7f02c815bff0 2023-07-14T11:55:10.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.102+0000 7f02b6ffd700 1 -- 172.21.15.104:0/2861851572 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f02c0056600 con 0x7f02c815bff0 2023-07-14T11:55:10.088 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:10.088 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3696540262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3696540262},{"type":"v1","addr":"172.21.15.104:6835","nonce":3696540262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/684052037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":684052037},{"type":"v1","addr":"172.21.15.121:6835","nonce":684052037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/1267091306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":1267091306},{"type":"v1","addr":"172.21.15.121:6833","nonce":1267091306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:54:58.600122+0000","modified":"2023-07-14T11:55:09.633871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5616},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5616":{"gid":5616,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6837/398018673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":398018673},{"type":"v1","addr":"172.21.15.104:6837","nonce":398018673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-14T11:55:10.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.102+0000 7f02c759e700 1 -- 172.21.15.104:0/2861851572 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f02a8066320 msgr2=0x7f02a80687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:10.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.102+0000 7f02c759e700 1 --2- 172.21.15.104:0/2861851572 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f02a8066320 0x7f02a80687d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f02b8000f60 tx=0x7f02b8009040 comp rx=0 tx=0).stop 2023-07-14T11:55:10.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.102+0000 7f02c759e700 1 -- 172.21.15.104:0/2861851572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02c815bff0 msgr2=0x7f02c815c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:10.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.106+0000 7f02c759e700 1 --2- 172.21.15.104:0/2861851572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02c815bff0 0x7f02c815c3c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f02c0014040 tx=0x7f02c0004af0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.106+0000 7f02c759e700 1 -- 172.21.15.104:0/2861851572 shutdown_connections 2023-07-14T11:55:10.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.106+0000 7f02c759e700 1 --2- 172.21.15.104:0/2861851572 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f02a8066320 0x7f02a80687d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.106+0000 7f02c759e700 1 --2- 172.21.15.104:0/2861851572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02c815bff0 0x7f02c815c3c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.106+0000 7f02c759e700 1 --2- 172.21.15.104:0/2861851572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02c80cd240 0x7f02c81529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.106+0000 7f02c759e700 1 --2- 172.21.15.104:0/2861851572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02c80b0de0 0x7f02c815ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.106+0000 7f02c759e700 1 -- 172.21.15.104:0/2861851572 >> 172.21.15.104:0/2861851572 conn(0x7f02c8060c40 msgr2=0x7f02c80d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:10.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.106+0000 7f02c759e700 1 -- 172.21.15.104:0/2861851572 shutdown_connections 2023-07-14T11:55:10.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.106+0000 7f02c759e700 1 -- 172.21.15.104:0/2861851572 wait complete. 2023-07-14T11:55:10.093 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 19 2023-07-14T11:55:10.111 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:55:10.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.294+0000 7f5e2e78b700 1 Processor -- start 2023-07-14T11:55:10.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.294+0000 7f5e2e78b700 1 -- start start 2023-07-14T11:55:10.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.294+0000 7f5e2e78b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f5e2815ffa0 0x7f5e2806d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:10.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.294+0000 7f5e2e78b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e2806dc50 0x7f5e280caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:10.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.294+0000 7f5e2e78b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e280ca8a0 con 0x7f5e28160370 2023-07-14T11:55:10.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.294+0000 7f5e2e78b700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e280cbcd0 con 0x7f5e2815ffa0 2023-07-14T11:55:10.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.294+0000 7f5e2e78b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e280cbf70 con 0x7f5e2806dc50 2023-07-14T11:55:10.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.294+0000 7f5e2d789700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f5e2815ffa0 0x7f5e2806d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:10.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2d789700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f5e2815ffa0 0x7f5e2806d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35876/0 (socket says 172.21.15.104:35876) 2023-07-14T11:55:10.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2d789700 1 -- 172.21.15.104:0/1207037431 learned_addr learned my addr 172.21.15.104:0/1207037431 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:10.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2cf88700 1 --2- 172.21.15.104:0/1207037431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e2806dc50 0x7f5e280caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:10.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2d789700 1 -- 172.21.15.104:0/1207037431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e2806dc50 msgr2=0x7f5e280caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:10.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2d789700 1 --2- 172.21.15.104:0/1207037431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e2806dc50 0x7f5e280caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2d789700 1 -- 172.21.15.104:0/1207037431 >> v1:172.21.15.121:6789/0 conn(0x7f5e28160370 legacy=0x7f5e280ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:10.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2d789700 1 -- 172.21.15.104:0/1207037431 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e280cc250 con 0x7f5e2815ffa0 2023-07-14T11:55:10.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2d789700 1 --2- 172.21.15.104:0/1207037431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e2815ffa0 0x7f5e2806d710 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009910 tx=0x7f5e24009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c201169da2e67984 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:10.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e1ffff700 1 -- 172.21.15.104:0/1207037431 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400ba60 con 0x7f5e2815ffa0 2023-07-14T11:55:10.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e1ffff700 1 -- 172.21.15.104:0/1207037431 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2400bbc0 con 0x7f5e2815ffa0 2023-07-14T11:55:10.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e1ffff700 1 -- 172.21.15.104:0/1207037431 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24009e60 con 0x7f5e2815ffa0 2023-07-14T11:55:10.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2e78b700 1 -- 172.21.15.104:0/1207037431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e2815ffa0 msgr2=0x7f5e2806d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:10.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2e78b700 1 --2- 172.21.15.104:0/1207037431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e2815ffa0 0x7f5e2806d710 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009910 tx=0x7f5e24009c20 comp rx=0 tx=0).stop 2023-07-14T11:55:10.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2e78b700 1 -- 172.21.15.104:0/1207037431 shutdown_connections 2023-07-14T11:55:10.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2e78b700 1 --2- 172.21.15.104:0/1207037431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e2806dc50 0x7f5e280caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2e78b700 1 --2- 172.21.15.104:0/1207037431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e2815ffa0 0x7f5e2806d710 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2e78b700 1 -- 172.21.15.104:0/1207037431 >> 172.21.15.104:0/1207037431 conn(0x7f5e2806e960 msgr2=0x7f5e2806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:10.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2e78b700 1 -- 172.21.15.104:0/1207037431 shutdown_connections 2023-07-14T11:55:10.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.298+0000 7f5e2e78b700 1 -- 172.21.15.104:0/1207037431 wait complete. 2023-07-14T11:55:10.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2e78b700 1 Processor -- start 2023-07-14T11:55:10.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2e78b700 1 -- start start 2023-07-14T11:55:10.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2e78b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e2806dc50 0x7f5e280c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:10.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2e78b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e2815ffa0 0x7f5e28159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:10.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2e78b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e28160370 0x7f5e28158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:10.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2e78b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e280c6120 con 0x7f5e28160370 2023-07-14T11:55:10.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2e78b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e280c5130 con 0x7f5e2806dc50 2023-07-14T11:55:10.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2e78b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e28158fa0 con 0x7f5e2815ffa0 2023-07-14T11:55:10.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2df8a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e28160370 0x7f5e28158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:10.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2df8a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e28160370 0x7f5e28158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35882/0 (socket says 172.21.15.104:35882) 2023-07-14T11:55:10.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2df8a700 1 -- 172.21.15.104:0/3046395090 learned_addr learned my addr 172.21.15.104:0/3046395090 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:10.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2cf88700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e2815ffa0 0x7f5e28159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:10.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2d789700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e2806dc50 0x7f5e280c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:10.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2df8a700 1 -- 172.21.15.104:0/3046395090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e2815ffa0 msgr2=0x7f5e28159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:10.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2df8a700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e2815ffa0 0x7f5e28159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2df8a700 1 -- 172.21.15.104:0/3046395090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e2806dc50 msgr2=0x7f5e280c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:10.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2df8a700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e2806dc50 0x7f5e280c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2df8a700 1 -- 172.21.15.104:0/3046395090 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e24009570 con 0x7f5e28160370 2023-07-14T11:55:10.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2cf88700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e2815ffa0 0x7f5e28159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:55:10.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.302+0000 7f5e2d789700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e2806dc50 0x7f5e280c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:55:10.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.306+0000 7f5e2df8a700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e28160370 0x7f5e28158a60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000cc50 tx=0x7f5e2000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:10.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.306+0000 7f5e1dffb700 1 -- 172.21.15.104:0/3046395090 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2000ad80 con 0x7f5e28160370 2023-07-14T11:55:10.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.306+0000 7f5e1dffb700 1 -- 172.21.15.104:0/3046395090 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2000ce50 con 0x7f5e28160370 2023-07-14T11:55:10.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.306+0000 7f5e1dffb700 1 -- 172.21.15.104:0/3046395090 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20003c60 con 0x7f5e28160370 2023-07-14T11:55:10.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.306+0000 7f5e2e78b700 1 -- 172.21.15.104:0/3046395090 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e280c5340 con 0x7f5e28160370 2023-07-14T11:55:10.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.306+0000 7f5e2e78b700 1 -- 172.21.15.104:0/3046395090 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e280ae820 con 0x7f5e28160370 2023-07-14T11:55:10.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.306+0000 7f5e2e78b700 1 -- 172.21.15.104:0/3046395090 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e280c5860 con 0x7f5e28160370 2023-07-14T11:55:10.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.310+0000 7f5e1dffb700 1 -- 172.21.15.104:0/3046395090 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5e200050d0 con 0x7f5e28160370 2023-07-14T11:55:10.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.310+0000 7f5e1dffb700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5e0c066320 0x7f5e0c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:10.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.310+0000 7f5e1dffb700 1 -- 172.21.15.104:0/3046395090 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f5e20083ef0 con 0x7f5e28160370 2023-07-14T11:55:10.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.310+0000 7f5e2d789700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5e0c066320 0x7f5e0c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:10.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.310+0000 7f5e2d789700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5e0c066320 0x7f5e0c0687d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5e280ae5d0 tx=0x7f5e24008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:10.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.318+0000 7f5e1dffb700 1 -- 172.21.15.104:0/3046395090 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f5e280c5860 con 0x7f5e28160370 2023-07-14T11:55:10.475 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.490+0000 7f5e2e78b700 1 -- 172.21.15.104:0/3046395090 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e280ae680 con 0x7f5e28160370 2023-07-14T11:55:10.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.494+0000 7f5e1dffb700 1 -- 172.21.15.104:0/3046395090 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+20324 (secure 0 0 0) 0x7f5e280ae680 con 0x7f5e28160370 2023-07-14T11:55:10.479 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:10.480 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":44,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:55:09.075845+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:54:56.370313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-14T11:54:57.608947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"data_new","create_time":"2023-07-14T11:55:08.099546+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:55:10.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.494+0000 7f5e2e78b700 1 -- 172.21.15.104:0/3046395090 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5e0c066320 msgr2=0x7f5e0c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:10.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.494+0000 7f5e2e78b700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5e0c066320 0x7f5e0c0687d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5e280ae5d0 tx=0x7f5e24008030 comp rx=0 tx=0).stop 2023-07-14T11:55:10.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.494+0000 7f5e2e78b700 1 -- 172.21.15.104:0/3046395090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e28160370 msgr2=0x7f5e28158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:10.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.494+0000 7f5e2e78b700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e28160370 0x7f5e28158a60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000cc50 tx=0x7f5e2000a340 comp rx=0 tx=0).stop 2023-07-14T11:55:10.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.494+0000 7f5e2e78b700 1 -- 172.21.15.104:0/3046395090 shutdown_connections 2023-07-14T11:55:10.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.494+0000 7f5e2e78b700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5e0c066320 0x7f5e0c0687d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.494+0000 7f5e2e78b700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e28160370 0x7f5e28158a60 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.494+0000 7f5e2e78b700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e2815ffa0 0x7f5e28159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.494+0000 7f5e2e78b700 1 --2- 172.21.15.104:0/3046395090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e2806dc50 0x7f5e280c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.494+0000 7f5e2e78b700 1 -- 172.21.15.104:0/3046395090 >> 172.21.15.104:0/3046395090 conn(0x7f5e2806e960 msgr2=0x7f5e280b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:10.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.494+0000 7f5e2e78b700 1 -- 172.21.15.104:0/3046395090 shutdown_connections 2023-07-14T11:55:10.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.494+0000 7f5e2e78b700 1 -- 172.21.15.104:0/3046395090 wait complete. 2023-07-14T11:55:10.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:10.506 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch layoutfile) 2023-07-14T11:55:10.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:10.533 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir layoutdir) 2023-07-14T11:55:10.599 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rw' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2023-07-14T11:55:10.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.838+0000 7fabcffe6700 1 Processor -- start 2023-07-14T11:55:10.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.838+0000 7fabcffe6700 1 -- start start 2023-07-14T11:55:10.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabcffe6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabc80cf710 0x7fabc80d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:10.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabcffe6700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc8071d10 con 0x7fabc80d0210 2023-07-14T11:55:10.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabcffe6700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc815c420 con 0x7fabc80b1c50 2023-07-14T11:55:10.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabcffe6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc8070840 con 0x7fabc80cf710 2023-07-14T11:55:10.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabcd581700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fabc80d0210 0x7fabc80cf000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35184/0 (socket says 172.21.15.104:35184) 2023-07-14T11:55:10.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabcd581700 1 -- 172.21.15.104:0/973935988 learned_addr learned my addr 172.21.15.104:0/973935988 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:10.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabce583700 1 --2- 172.21.15.104:0/973935988 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabc80cf710 0x7fabc80d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:10.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579127426 0 0) 0x7fabb8001110 con 0x7fabc80d0210 2023-07-14T11:55:10.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabb40035d0 con 0x7fabc80d0210 2023-07-14T11:55:10.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354614336 0 0) 0x7fabc815c420 con 0x7fabc80b1c50 2023-07-14T11:55:10.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabb8001110 con 0x7fabc80b1c50 2023-07-14T11:55:10.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2349916104 0 0) 0x7fabb40035d0 con 0x7fabc80d0210 2023-07-14T11:55:10.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabc80cf710 msgr2=0x7fabc80d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:10.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 --2- 172.21.15.104:0/973935988 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabc80cf710 0x7fabc80d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 >> v1:172.21.15.121:6789/0 conn(0x7fabc80b1c50 legacy=0x7fabc8071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:10.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc8070b20 con 0x7fabc80d0210 2023-07-14T11:55:10.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7fabb8001820 con 0x7fabc80d0210 2023-07-14T11:55:10.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 >> v1:172.21.15.104:6789/0 conn(0x7fabc80d0210 legacy=0x7fabc80cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:10.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 --2- 172.21.15.104:0/973935988 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabc80cf710 0x7fabb4003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:10.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 --2- 172.21.15.104:0/973935988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabb4004140 0x7fabb4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:10.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 --2- 172.21.15.104:0/973935988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabb4004db0 0x7fabb4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:10.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc8070840 con 0x7fabb4004db0 2023-07-14T11:55:10.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb400bbc0 con 0x7fabb4004140 2023-07-14T11:55:10.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb400bd50 con 0x7fabc80cf710 2023-07-14T11:55:10.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.842+0000 7fabce583700 1 --2- 172.21.15.104:0/973935988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabb4004db0 0x7fabb4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:10.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabcd581700 1 --2- 172.21.15.104:0/973935988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabb4004140 0x7fabb4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:10.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabcdd82700 1 --2- 172.21.15.104:0/973935988 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabc80cf710 0x7fabb4003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:10.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabce583700 1 -- 172.21.15.104:0/973935988 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabc80cf710 msgr2=0x7fabb4003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:10.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabce583700 1 --2- 172.21.15.104:0/973935988 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabc80cf710 0x7fabb4003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabce583700 1 -- 172.21.15.104:0/973935988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabb4004140 msgr2=0x7fabb4004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:10.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabce583700 1 --2- 172.21.15.104:0/973935988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabb4004140 0x7fabb4004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabce583700 1 -- 172.21.15.104:0/973935988 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb40035d0 con 0x7fabb4004db0 2023-07-14T11:55:10.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabcd581700 1 --2- 172.21.15.104:0/973935988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabb4004140 0x7fabb4004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:10.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabce583700 1 --2- 172.21.15.104:0/973935988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabb4004db0 0x7fabb4005220 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009b40 tx=0x7fabb40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5569e442606b60ef server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:10.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc000e6c0 con 0x7fabb4004db0 2023-07-14T11:55:10.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc000e820 con 0x7fabb4004db0 2023-07-14T11:55:10.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabccd80700 1 -- 172.21.15.104:0/973935988 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc000ebb0 con 0x7fabb4004db0 2023-07-14T11:55:10.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabcffe6700 1 -- 172.21.15.104:0/973935988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabb4004db0 msgr2=0x7fabb4005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:10.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabcffe6700 1 --2- 172.21.15.104:0/973935988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabb4004db0 0x7fabb4005220 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009b40 tx=0x7fabb40058a0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabce583700 1 -- 172.21.15.104:0/973935988 reap_dead start 2023-07-14T11:55:10.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabcffe6700 1 -- 172.21.15.104:0/973935988 shutdown_connections 2023-07-14T11:55:10.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabcffe6700 1 -- 172.21.15.104:0/973935988 >> 172.21.15.104:0/973935988 conn(0x7fabc809b9c0 msgr2=0x7fabc80d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:10.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabcffe6700 1 -- 172.21.15.104:0/973935988 shutdown_connections 2023-07-14T11:55:10.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.846+0000 7fabcffe6700 1 -- 172.21.15.104:0/973935988 wait complete. 2023-07-14T11:55:10.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcffe6700 1 Processor -- start 2023-07-14T11:55:10.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcffe6700 1 -- start start 2023-07-14T11:55:10.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcffe6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabb4004db0 0x7fabc814d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:10.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcffe6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabc8159b80 0x7fabc815ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:10.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcffe6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc80b5710 0x7fabc80b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:10.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcffe6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80b60d0 con 0x7fabc8159b80 2023-07-14T11:55:10.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcffe6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80b44a0 con 0x7fabc80b5710 2023-07-14T11:55:10.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcffe6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80b46f0 con 0x7fabb4004db0 2023-07-14T11:55:10.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcd581700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabc8159b80 0x7fabc815ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:10.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcd581700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabc8159b80 0x7fabc815ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53754/0 (socket says 172.21.15.104:53754) 2023-07-14T11:55:10.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcd581700 1 -- 172.21.15.104:0/3599178695 learned_addr learned my addr 172.21.15.104:0/3599178695 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:10.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcdd82700 1 --2- 172.21.15.104:0/3599178695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabb4004db0 0x7fabc814d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:10.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabce583700 1 --2- 172.21.15.104:0/3599178695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc80b5710 0x7fabc80b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:10.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcd581700 1 -- 172.21.15.104:0/3599178695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabb4004db0 msgr2=0x7fabc814d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:10.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcd581700 1 --2- 172.21.15.104:0/3599178695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabb4004db0 0x7fabc814d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcd581700 1 -- 172.21.15.104:0/3599178695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc80b5710 msgr2=0x7fabc80b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:10.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcd581700 1 --2- 172.21.15.104:0/3599178695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc80b5710 0x7fabc80b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:10.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcd581700 1 -- 172.21.15.104:0/3599178695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc80b49d0 con 0x7fabc8159b80 2023-07-14T11:55:10.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.850+0000 7fabcd581700 1 --2- 172.21.15.104:0/3599178695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabc8159b80 0x7fabc815ec10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fabb8007330 tx=0x7fabb8007640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:10.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.854+0000 7fabbe7fc700 1 -- 172.21.15.104:0/3599178695 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8012590 con 0x7fabc8159b80 2023-07-14T11:55:10.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.854+0000 7fabbe7fc700 1 -- 172.21.15.104:0/3599178695 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb80126f0 con 0x7fabc8159b80 2023-07-14T11:55:10.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.854+0000 7fabbe7fc700 1 -- 172.21.15.104:0/3599178695 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8012aa0 con 0x7fabc8159b80 2023-07-14T11:55:10.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.854+0000 7fabcffe6700 1 -- 172.21.15.104:0/3599178695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc80b4cb0 con 0x7fabc8159b80 2023-07-14T11:55:10.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.854+0000 7fabcffe6700 1 -- 172.21.15.104:0/3599178695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc814dd30 con 0x7fabc8159b80 2023-07-14T11:55:10.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.854+0000 7fabcffe6700 1 -- 172.21.15.104:0/3599178695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc804dcb0 con 0x7fabc8159b80 2023-07-14T11:55:10.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.854+0000 7fabbe7fc700 1 -- 172.21.15.104:0/3599178695 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fabb8012c00 con 0x7fabc8159b80 2023-07-14T11:55:10.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.858+0000 7fabbe7fc700 1 --2- 172.21.15.104:0/3599178695 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fabac0662f0 0x7fabac0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:10.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.858+0000 7fabbe7fc700 1 -- 172.21.15.104:0/3599178695 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fabb8083e10 con 0x7fabc8159b80 2023-07-14T11:55:10.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.858+0000 7fabcdd82700 1 --2- 172.21.15.104:0/3599178695 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fabac0662f0 0x7fabac0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:10.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.858+0000 7fabcdd82700 1 --2- 172.21.15.104:0/3599178695 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fabac0662f0 0x7fabac0687a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fabc4001a50 tx=0x7fabc4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:10.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:10.862+0000 7fabbe7fc700 1 -- 172.21.15.104:0/3599178695 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fabb8087030 con 0x7fabc8159b80 2023-07-14T11:55:11.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.098+0000 7fabcffe6700 1 -- 172.21.15.104:0/3599178695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7fabc8071d10 con 0x7fabc8159b80 2023-07-14T11:55:11.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.110+0000 7fabbe7fc700 1 -- 172.21.15.104:0/3599178695 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v14) v1 ==== 210+0+167 (secure 0 0 0) 0x7fabc8071d10 con 0x7fabc8159b80 2023-07-14T11:55:11.097 INFO:teuthology.orchestra.run.smithi104.stdout:[client.0] 2023-07-14T11:55:11.097 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDrNrFkbrzjERAA+9BBzuauwUdhALUcDtd80A== 2023-07-14T11:55:11.097 INFO:teuthology.orchestra.run.smithi104.stdout: caps mds = "allow rw" 2023-07-14T11:55:11.097 INFO:teuthology.orchestra.run.smithi104.stdout: caps mon = "allow r" 2023-07-14T11:55:11.098 INFO:teuthology.orchestra.run.smithi104.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2023-07-14T11:55:11.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.114+0000 7fabcffe6700 1 -- 172.21.15.104:0/3599178695 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fabac0662f0 msgr2=0x7fabac0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:11.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.114+0000 7fabcffe6700 1 --2- 172.21.15.104:0/3599178695 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fabac0662f0 0x7fabac0687a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fabc4001a50 tx=0x7fabc4009040 comp rx=0 tx=0).stop 2023-07-14T11:55:11.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.114+0000 7fabcffe6700 1 -- 172.21.15.104:0/3599178695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabc8159b80 msgr2=0x7fabc815ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:11.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.114+0000 7fabcffe6700 1 --2- 172.21.15.104:0/3599178695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabc8159b80 0x7fabc815ec10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fabb8007330 tx=0x7fabb8007640 comp rx=0 tx=0).stop 2023-07-14T11:55:11.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.114+0000 7fabcffe6700 1 -- 172.21.15.104:0/3599178695 shutdown_connections 2023-07-14T11:55:11.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.114+0000 7fabcffe6700 1 --2- 172.21.15.104:0/3599178695 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fabac0662f0 0x7fabac0687a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.114+0000 7fabcffe6700 1 --2- 172.21.15.104:0/3599178695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabc8159b80 0x7fabc815ec10 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.114+0000 7fabcffe6700 1 --2- 172.21.15.104:0/3599178695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc80b5710 0x7fabc80b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.114+0000 7fabcffe6700 1 --2- 172.21.15.104:0/3599178695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabb4004db0 0x7fabc814d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.114+0000 7fabcffe6700 1 -- 172.21.15.104:0/3599178695 >> 172.21.15.104:0/3599178695 conn(0x7fabc809b9c0 msgr2=0x7fabc815b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:11.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.114+0000 7fabcffe6700 1 -- 172.21.15.104:0/3599178695 shutdown_connections 2023-07-14T11:55:11.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.114+0000 7fabcffe6700 1 -- 172.21.15.104:0/3599178695 wait complete. 2023-07-14T11:55:11.102 INFO:teuthology.orchestra.run.smithi104.stderr:updated caps for client.0 2023-07-14T11:55:11.123 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:55:11.124 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:55:11.144 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:55:11.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.378+0000 7fce0c665700 1 Processor -- start 2023-07-14T11:55:11.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.378+0000 7fce0c665700 1 -- start start 2023-07-14T11:55:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.378+0000 7fce0c665700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fce04160640 0x7fce04164000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.378+0000 7fce0c665700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce04164540 0x7fce04070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.378+0000 7fce0c665700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce040c8900 con 0x7fce040cddb0 2023-07-14T11:55:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.378+0000 7fce0c665700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce040c8fc0 con 0x7fce04160640 2023-07-14T11:55:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.378+0000 7fce0c665700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce04070540 con 0x7fce04164540 2023-07-14T11:55:11.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce09c00700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce04164540 0x7fce04070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:11.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce09c00700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce04164540 0x7fce04070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:46396/0 (socket says 172.21.15.104:46396) 2023-07-14T11:55:11.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce09c00700 1 -- 172.21.15.104:0/1627821678 learned_addr learned my addr 172.21.15.104:0/1627821678 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:11.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce0a401700 1 --2- 172.21.15.104:0/1627821678 >> v2:172.21.15.121:3300/0 conn(0x7fce04160640 0x7fce04164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:11.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce09c00700 1 -- 172.21.15.104:0/1627821678 >> v2:172.21.15.121:3300/0 conn(0x7fce04160640 msgr2=0x7fce04164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:11.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce09c00700 1 --2- 172.21.15.104:0/1627821678 >> v2:172.21.15.121:3300/0 conn(0x7fce04160640 0x7fce04164000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce09c00700 1 -- 172.21.15.104:0/1627821678 >> v1:172.21.15.121:6789/0 conn(0x7fce040cddb0 legacy=0x7fce040afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:11.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce09c00700 1 -- 172.21.15.104:0/1627821678 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce040caae0 con 0x7fce04164540 2023-07-14T11:55:11.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce09c00700 1 --2- 172.21.15.104:0/1627821678 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce04164540 0x7fce04070960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009910 tx=0x7fcdf4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df904049709ba78e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:11.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce093ff700 1 -- 172.21.15.104:0/1627821678 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4011830 con 0x7fce04164540 2023-07-14T11:55:11.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce093ff700 1 -- 172.21.15.104:0/1627821678 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf4011990 con 0x7fce04164540 2023-07-14T11:55:11.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce093ff700 1 -- 172.21.15.104:0/1627821678 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4011d40 con 0x7fce04164540 2023-07-14T11:55:11.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce0c665700 1 -- 172.21.15.104:0/1627821678 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce04164540 msgr2=0x7fce04070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:11.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce0c665700 1 --2- 172.21.15.104:0/1627821678 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce04164540 0x7fce04070960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009910 tx=0x7fcdf4009c20 comp rx=0 tx=0).stop 2023-07-14T11:55:11.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce0c665700 1 -- 172.21.15.104:0/1627821678 shutdown_connections 2023-07-14T11:55:11.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce0c665700 1 --2- 172.21.15.104:0/1627821678 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce04164540 0x7fce04070960 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce0c665700 1 --2- 172.21.15.104:0/1627821678 >> v2:172.21.15.121:3300/0 conn(0x7fce04160640 0x7fce04164000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce0c665700 1 -- 172.21.15.104:0/1627821678 >> 172.21.15.104:0/1627821678 conn(0x7fce040cad30 msgr2=0x7fce040cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:11.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce0c665700 1 -- 172.21.15.104:0/1627821678 shutdown_connections 2023-07-14T11:55:11.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.382+0000 7fce0c665700 1 -- 172.21.15.104:0/1627821678 wait complete. 2023-07-14T11:55:11.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.386+0000 7fce0c665700 1 Processor -- start 2023-07-14T11:55:11.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.386+0000 7fce0c665700 1 -- start start 2023-07-14T11:55:11.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.386+0000 7fce0c665700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce040cddb0 0x7fce04151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:11.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.386+0000 7fce0c665700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce04160640 0x7fce041f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:11.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.386+0000 7fce0c665700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce04164540 0x7fce040c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:11.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.386+0000 7fce0c665700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce040c5380 con 0x7fce04160640 2023-07-14T11:55:11.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.386+0000 7fce0c665700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce040c5660 con 0x7fce040cddb0 2023-07-14T11:55:11.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.386+0000 7fce0c665700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce040c5940 con 0x7fce04164540 2023-07-14T11:55:11.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.386+0000 7fce09c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce04160640 0x7fce041f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:11.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.386+0000 7fce09c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce04160640 0x7fce041f70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53758/0 (socket says 172.21.15.104:53758) 2023-07-14T11:55:11.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.386+0000 7fce0a401700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce040cddb0 0x7fce04151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:11.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.386+0000 7fce09c00700 1 -- 172.21.15.104:0/4182477165 learned_addr learned my addr 172.21.15.104:0/4182477165 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:11.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.386+0000 7fce0ac02700 1 --2- 172.21.15.104:0/4182477165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce04164540 0x7fce040c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:11.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.390+0000 7fce0a401700 1 -- 172.21.15.104:0/4182477165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce04164540 msgr2=0x7fce040c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:11.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.390+0000 7fce0a401700 1 --2- 172.21.15.104:0/4182477165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce04164540 0x7fce040c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.390+0000 7fce0a401700 1 -- 172.21.15.104:0/4182477165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce04160640 msgr2=0x7fce041f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:11.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.390+0000 7fce0a401700 1 --2- 172.21.15.104:0/4182477165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce04160640 0x7fce041f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.390+0000 7fce0a401700 1 -- 172.21.15.104:0/4182477165 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf4009570 con 0x7fce040cddb0 2023-07-14T11:55:11.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.390+0000 7fce0a401700 1 --2- 172.21.15.104:0/4182477165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce040cddb0 0x7fce04151210 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fce0000e900 tx=0x7fce0000ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:11.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.390+0000 7fcdfaffd700 1 -- 172.21.15.104:0/4182477165 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce00013680 con 0x7fce040cddb0 2023-07-14T11:55:11.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.390+0000 7fcdfaffd700 1 -- 172.21.15.104:0/4182477165 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce000137e0 con 0x7fce040cddb0 2023-07-14T11:55:11.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.390+0000 7fce0c665700 1 -- 172.21.15.104:0/4182477165 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce040c5c20 con 0x7fce040cddb0 2023-07-14T11:55:11.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.390+0000 7fcdfaffd700 1 -- 172.21.15.104:0/4182477165 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce00013b50 con 0x7fce040cddb0 2023-07-14T11:55:11.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.390+0000 7fce0c665700 1 -- 172.21.15.104:0/4182477165 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce041f7630 con 0x7fce040cddb0 2023-07-14T11:55:11.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.390+0000 7fce0c665700 1 -- 172.21.15.104:0/4182477165 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0404dcb0 con 0x7fce040cddb0 2023-07-14T11:55:11.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.394+0000 7fcdfaffd700 1 -- 172.21.15.104:0/4182477165 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fce00013cb0 con 0x7fce040cddb0 2023-07-14T11:55:11.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.394+0000 7fcdfaffd700 1 --2- 172.21.15.104:0/4182477165 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcde8066370 0x7fcde8068820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:11.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.394+0000 7fcdfaffd700 1 -- 172.21.15.104:0/4182477165 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fce00084230 con 0x7fce040cddb0 2023-07-14T11:55:11.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.394+0000 7fce09c00700 1 --2- 172.21.15.104:0/4182477165 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcde8066370 0x7fcde8068820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:11.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.394+0000 7fce09c00700 1 --2- 172.21.15.104:0/4182477165 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcde8066370 0x7fcde8068820 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fce0406d410 tx=0x7fcdf4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:11.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.402+0000 7fcdfaffd700 1 -- 172.21.15.104:0/4182477165 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fce00054e50 con 0x7fce040cddb0 2023-07-14T11:55:11.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.574+0000 7fce0c665700 1 -- 172.21.15.104:0/4182477165 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce040c57a0 con 0x7fce040cddb0 2023-07-14T11:55:11.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.578+0000 7fcdfaffd700 1 -- 172.21.15.104:0/4182477165 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+20311 (secure 0 0 0) 0x7fce040c57a0 con 0x7fce040cddb0 2023-07-14T11:55:11.565 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:11.566 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":45,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:55:10.654581+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:54:56.370313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-14T11:54:57.608947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"data_new","create_time":"2023-07-14T11:55:08.099546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:55:11.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.578+0000 7fce0c665700 1 -- 172.21.15.104:0/4182477165 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcde8066370 msgr2=0x7fcde8068820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:11.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.578+0000 7fce0c665700 1 --2- 172.21.15.104:0/4182477165 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcde8066370 0x7fcde8068820 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fce0406d410 tx=0x7fcdf4008030 comp rx=0 tx=0).stop 2023-07-14T11:55:11.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.578+0000 7fce0c665700 1 -- 172.21.15.104:0/4182477165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce040cddb0 msgr2=0x7fce04151210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:11.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.578+0000 7fce0c665700 1 --2- 172.21.15.104:0/4182477165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce040cddb0 0x7fce04151210 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fce0000e900 tx=0x7fce0000ec10 comp rx=0 tx=0).stop 2023-07-14T11:55:11.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.578+0000 7fce0c665700 1 -- 172.21.15.104:0/4182477165 shutdown_connections 2023-07-14T11:55:11.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.578+0000 7fce0c665700 1 --2- 172.21.15.104:0/4182477165 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcde8066370 0x7fcde8068820 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.578+0000 7fce0c665700 1 --2- 172.21.15.104:0/4182477165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce04160640 0x7fce041f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.578+0000 7fce0c665700 1 --2- 172.21.15.104:0/4182477165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce04164540 0x7fce040c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.578+0000 7fce0c665700 1 --2- 172.21.15.104:0/4182477165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce040cddb0 0x7fce04151210 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.578+0000 7fce0c665700 1 -- 172.21.15.104:0/4182477165 >> 172.21.15.104:0/4182477165 conn(0x7fce040cad30 msgr2=0x7fce04072300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:11.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.582+0000 7fce0c665700 1 -- 172.21.15.104:0/4182477165 shutdown_connections 2023-07-14T11:55:11.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.582+0000 7fce0c665700 1 -- 172.21.15.104:0/4182477165 wait complete. 2023-07-14T11:55:11.589 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:54:56.370313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:54:57.608947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'data_new', 'create_time': '2023-07-14T11:55:08.099546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:55:11.591 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:55:11.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.782+0000 7f7122b84700 1 Processor -- start 2023-07-14T11:55:11.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.782+0000 7f7122b84700 1 -- start start 2023-07-14T11:55:11.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.786+0000 7f7122b84700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f711c0b0ee0 0x7f711c0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:11.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.786+0000 7f7122b84700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711c0cd4e0 con 0x7f711c0d07e0 2023-07-14T11:55:11.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.786+0000 7f7122b84700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711c0d1a60 con 0x7f711c0cd090 2023-07-14T11:55:11.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.786+0000 7f7122b84700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711c0d0c60 con 0x7f711c0b0ee0 2023-07-14T11:55:11.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.786+0000 7f7121b82700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f711c0b0ee0 0x7f711c0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:11.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.786+0000 7f7122383700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f711c0d07e0 0x7f711c0af0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35200/0 (socket says 172.21.15.104:35200) 2023-07-14T11:55:11.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.786+0000 7f7122383700 1 -- 172.21.15.104:0/2716141305 learned_addr learned my addr 172.21.15.104:0/2716141305 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:11.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.786+0000 7f7120b80700 1 -- 172.21.15.104:0/2716141305 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4012259092 0 0) 0x7f7114001490 con 0x7f711c0d07e0 2023-07-14T11:55:11.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.786+0000 7f7120b80700 1 -- 172.21.15.104:0/2716141305 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71080035d0 con 0x7f711c0d07e0 2023-07-14T11:55:11.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.786+0000 7f7121b82700 1 -- 172.21.15.104:0/2716141305 >> v1:172.21.15.121:6789/0 conn(0x7f711c0cd090 legacy=0x7f711c0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:11.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.786+0000 7f7121b82700 1 -- 172.21.15.104:0/2716141305 >> v1:172.21.15.104:6789/0 conn(0x7f711c0d07e0 legacy=0x7f711c0af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:11.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.786+0000 7f7121b82700 1 -- 172.21.15.104:0/2716141305 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c0d1ef0 con 0x7f711c0b0ee0 2023-07-14T11:55:11.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.786+0000 7f7121b82700 1 --2- 172.21.15.104:0/2716141305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f711c0b0ee0 0x7f711c0ccb50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7118009850 tx=0x7f7118009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5b4003a02297c85 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:11.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.790+0000 7f7120b80700 1 -- 172.21.15.104:0/2716141305 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118011830 con 0x7f711c0b0ee0 2023-07-14T11:55:11.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.790+0000 7f7120b80700 1 -- 172.21.15.104:0/2716141305 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7118011990 con 0x7f711c0b0ee0 2023-07-14T11:55:11.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.790+0000 7f7120b80700 1 -- 172.21.15.104:0/2716141305 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118011d40 con 0x7f711c0b0ee0 2023-07-14T11:55:11.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.790+0000 7f7122b84700 1 -- 172.21.15.104:0/2716141305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f711c0b0ee0 msgr2=0x7f711c0ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:11.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.790+0000 7f7122b84700 1 --2- 172.21.15.104:0/2716141305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f711c0b0ee0 0x7f711c0ccb50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7118009850 tx=0x7f7118009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:11.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.790+0000 7f7122b84700 1 -- 172.21.15.104:0/2716141305 shutdown_connections 2023-07-14T11:55:11.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.790+0000 7f7122b84700 1 --2- 172.21.15.104:0/2716141305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f711c0b0ee0 0x7f711c0ccb50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.790+0000 7f7122b84700 1 -- 172.21.15.104:0/2716141305 >> 172.21.15.104:0/2716141305 conn(0x7f711c060c40 msgr2=0x7f711c09b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:11.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.790+0000 7f7122b84700 1 -- 172.21.15.104:0/2716141305 shutdown_connections 2023-07-14T11:55:11.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.790+0000 7f7122b84700 1 -- 172.21.15.104:0/2716141305 wait complete. 2023-07-14T11:55:11.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.794+0000 7f7122b84700 1 Processor -- start 2023-07-14T11:55:11.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.794+0000 7f7122b84700 1 -- start start 2023-07-14T11:55:11.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.794+0000 7f7122b84700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f711c0b0ee0 0x7f711c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:11.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.794+0000 7f7122b84700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f711c0cd090 0x7f711c0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:11.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.794+0000 7f7122b84700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c0d07e0 0x7f711c0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:11.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.794+0000 7f7122b84700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c0b2f60 con 0x7f711c0b0ee0 2023-07-14T11:55:11.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.794+0000 7f7122b84700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c0b23f0 con 0x7f711c0cd090 2023-07-14T11:55:11.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.794+0000 7f7122b84700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711c0b26d0 con 0x7f711c0d07e0 2023-07-14T11:55:11.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.794+0000 7f7121b82700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f711c0b0ee0 0x7f711c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:11.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.794+0000 7f7121b82700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f711c0b0ee0 0x7f711c15afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53770/0 (socket says 172.21.15.104:53770) 2023-07-14T11:55:11.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.794+0000 7f7121b82700 1 -- 172.21.15.104:0/720931303 learned_addr learned my addr 172.21.15.104:0/720931303 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:11.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.794+0000 7f7121381700 1 --2- 172.21.15.104:0/720931303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f711c0cd090 0x7f711c0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:11.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.794+0000 7f7122383700 1 --2- 172.21.15.104:0/720931303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c0d07e0 0x7f711c0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:11.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.798+0000 7f7121b82700 1 -- 172.21.15.104:0/720931303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c0d07e0 msgr2=0x7f711c0cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:11.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.798+0000 7f7121b82700 1 --2- 172.21.15.104:0/720931303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c0d07e0 0x7f711c0cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.798+0000 7f7121b82700 1 -- 172.21.15.104:0/720931303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f711c0cd090 msgr2=0x7f711c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:11.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.798+0000 7f7121b82700 1 --2- 172.21.15.104:0/720931303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f711c0cd090 0x7f711c0b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.798+0000 7f7121b82700 1 -- 172.21.15.104:0/720931303 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71180094b0 con 0x7f711c0b0ee0 2023-07-14T11:55:11.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.798+0000 7f7122383700 1 --2- 172.21.15.104:0/720931303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c0d07e0 0x7f711c0cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:11.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.798+0000 7f7121b82700 1 --2- 172.21.15.104:0/720931303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f711c0b0ee0 0x7f711c15afe0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7118014040 tx=0x7f7118004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:11.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.798+0000 7f71127fc700 1 -- 172.21.15.104:0/720931303 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118019a00 con 0x7f711c0b0ee0 2023-07-14T11:55:11.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.798+0000 7f71127fc700 1 -- 172.21.15.104:0/720931303 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711801e070 con 0x7f711c0b0ee0 2023-07-14T11:55:11.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.798+0000 7f71127fc700 1 -- 172.21.15.104:0/720931303 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118019da0 con 0x7f711c0b0ee0 2023-07-14T11:55:11.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.798+0000 7f7122b84700 1 -- 172.21.15.104:0/720931303 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711c0b29b0 con 0x7f711c0b0ee0 2023-07-14T11:55:11.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.798+0000 7f7122b84700 1 -- 172.21.15.104:0/720931303 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711c15b520 con 0x7f711c0b0ee0 2023-07-14T11:55:11.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.798+0000 7f7122b84700 1 -- 172.21.15.104:0/720931303 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711c0b33f0 con 0x7f711c0b0ee0 2023-07-14T11:55:11.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.802+0000 7f71127fc700 1 -- 172.21.15.104:0/720931303 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f711802d430 con 0x7f711c0b0ee0 2023-07-14T11:55:11.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.802+0000 7f71127fc700 1 --2- 172.21.15.104:0/720931303 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7100066320 0x7f71000687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:11.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.806+0000 7f71127fc700 1 -- 172.21.15.104:0/720931303 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f7118084fe0 con 0x7f711c0b0ee0 2023-07-14T11:55:11.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.806+0000 7f7121381700 1 --2- 172.21.15.104:0/720931303 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7100066320 0x7f71000687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:11.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.806+0000 7f7121381700 1 --2- 172.21.15.104:0/720931303 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7100066320 0x7f71000687d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f711c15bc20 tx=0x7f710c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:11.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.810+0000 7f71127fc700 1 -- 172.21.15.104:0/720931303 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f711c0b33f0 con 0x7f711c0b0ee0 2023-07-14T11:55:11.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.982+0000 7f7122b84700 1 -- 172.21.15.104:0/720931303 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f711c0b33f0 con 0x7f711c0b0ee0 2023-07-14T11:55:11.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.982+0000 7f71127fc700 1 -- 172.21.15.104:0/720931303 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f711c0b33f0 con 0x7f711c0b0ee0 2023-07-14T11:55:11.971 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:55:11.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.986+0000 7f7122b84700 1 -- 172.21.15.104:0/720931303 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7100066320 msgr2=0x7f71000687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:11.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.986+0000 7f7122b84700 1 --2- 172.21.15.104:0/720931303 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7100066320 0x7f71000687d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f711c15bc20 tx=0x7f710c009040 comp rx=0 tx=0).stop 2023-07-14T11:55:11.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.986+0000 7f7122b84700 1 -- 172.21.15.104:0/720931303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f711c0b0ee0 msgr2=0x7f711c15afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:11.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.986+0000 7f7122b84700 1 --2- 172.21.15.104:0/720931303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f711c0b0ee0 0x7f711c15afe0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7118014040 tx=0x7f7118004770 comp rx=0 tx=0).stop 2023-07-14T11:55:11.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.986+0000 7f7122b84700 1 -- 172.21.15.104:0/720931303 shutdown_connections 2023-07-14T11:55:11.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.986+0000 7f7122b84700 1 --2- 172.21.15.104:0/720931303 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7100066320 0x7f71000687d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.986+0000 7f7122b84700 1 --2- 172.21.15.104:0/720931303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f711c0d07e0 0x7f711c0cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.986+0000 7f7122b84700 1 --2- 172.21.15.104:0/720931303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f711c0cd090 0x7f711c0b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.986+0000 7f7122b84700 1 --2- 172.21.15.104:0/720931303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f711c0b0ee0 0x7f711c15afe0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:11.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.986+0000 7f7122b84700 1 -- 172.21.15.104:0/720931303 >> 172.21.15.104:0/720931303 conn(0x7f711c060c40 msgr2=0x7f711c09b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:11.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.986+0000 7f7122b84700 1 -- 172.21.15.104:0/720931303 shutdown_connections 2023-07-14T11:55:11.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:11.986+0000 7f7122b84700 1 -- 172.21.15.104:0/720931303 wait complete. 2023-07-14T11:55:11.996 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:55:12.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.182+0000 7f26de737700 1 Processor -- start 2023-07-14T11:55:12.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.182+0000 7f26de737700 1 -- start start 2023-07-14T11:55:12.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26de737700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f26d80b0ee0 0x7f26d80ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26de737700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26d80cd090 0x7f26d80d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26de737700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f26d80d0610 0x7f26d815f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26de737700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d80cd4e0 con 0x7f26d80d0610 2023-07-14T11:55:12.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26de737700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d80d1a60 con 0x7f26d80b0ee0 2023-07-14T11:55:12.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26de737700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d80d1ef0 con 0x7f26d80cd090 2023-07-14T11:55:12.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26ddf36700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f26d80d0610 0x7f26d815f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26ddf36700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f26d80d0610 0x7f26d815f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53774/0 (socket says 172.21.15.104:53774) 2023-07-14T11:55:12.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26ddf36700 1 -- 172.21.15.104:0/2279724689 learned_addr learned my addr 172.21.15.104:0/2279724689 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:12.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26dcf34700 1 --2- 172.21.15.104:0/2279724689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26d80cd090 0x7f26d80d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26dd735700 1 --2- 172.21.15.104:0/2279724689 >> v2:172.21.15.121:3300/0 conn(0x7f26d80b0ee0 0x7f26d80ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26dcf34700 1 -- 172.21.15.104:0/2279724689 >> v2:172.21.15.121:3300/0 conn(0x7f26d80b0ee0 msgr2=0x7f26d80ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:12.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26dcf34700 1 --2- 172.21.15.104:0/2279724689 >> v2:172.21.15.121:3300/0 conn(0x7f26d80b0ee0 0x7f26d80ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26dcf34700 1 -- 172.21.15.104:0/2279724689 >> v2:172.21.15.104:3300/0 conn(0x7f26d80d0610 msgr2=0x7f26d815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:12.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26dcf34700 1 --2- 172.21.15.104:0/2279724689 >> v2:172.21.15.104:3300/0 conn(0x7f26d80d0610 0x7f26d815f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26dcf34700 1 -- 172.21.15.104:0/2279724689 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d80d0e50 con 0x7f26d80cd090 2023-07-14T11:55:12.176 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.186+0000 7f26dcf34700 1 --2- 172.21.15.104:0/2279724689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26d80cd090 0x7f26d80d00d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009850 tx=0x7f26c8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b5aeec3507927da server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:12.176 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.190+0000 7f26cffff700 1 -- 172.21.15.104:0/2279724689 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8011830 con 0x7f26d80cd090 2023-07-14T11:55:12.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.190+0000 7f26cffff700 1 -- 172.21.15.104:0/2279724689 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c8011990 con 0x7f26d80cd090 2023-07-14T11:55:12.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.190+0000 7f26cffff700 1 -- 172.21.15.104:0/2279724689 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8011d20 con 0x7f26d80cd090 2023-07-14T11:55:12.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.190+0000 7f26de737700 1 -- 172.21.15.104:0/2279724689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26d80cd090 msgr2=0x7f26d80d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.190+0000 7f26de737700 1 --2- 172.21.15.104:0/2279724689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26d80cd090 0x7f26d80d00d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009850 tx=0x7f26c8009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:12.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.190+0000 7f26de737700 1 -- 172.21.15.104:0/2279724689 shutdown_connections 2023-07-14T11:55:12.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.190+0000 7f26de737700 1 --2- 172.21.15.104:0/2279724689 >> v2:172.21.15.104:3300/0 conn(0x7f26d80d0610 0x7f26d815f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.190+0000 7f26de737700 1 --2- 172.21.15.104:0/2279724689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26d80cd090 0x7f26d80d00d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.190+0000 7f26de737700 1 --2- 172.21.15.104:0/2279724689 >> v2:172.21.15.121:3300/0 conn(0x7f26d80b0ee0 0x7f26d80ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.190+0000 7f26de737700 1 -- 172.21.15.104:0/2279724689 >> 172.21.15.104:0/2279724689 conn(0x7f26d8060c40 msgr2=0x7f26d809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:12.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.190+0000 7f26de737700 1 -- 172.21.15.104:0/2279724689 shutdown_connections 2023-07-14T11:55:12.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.190+0000 7f26de737700 1 -- 172.21.15.104:0/2279724689 wait complete. 2023-07-14T11:55:12.180 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.194+0000 7f26de737700 1 Processor -- start 2023-07-14T11:55:12.180 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.194+0000 7f26de737700 1 -- start start 2023-07-14T11:55:12.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.194+0000 7f26de737700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26d80b0ee0 0x7f26d814bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.194+0000 7f26de737700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26d80cd090 0x7f26d80b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.194+0000 7f26de737700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26d80d0610 0x7f26d809c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.194+0000 7f26de737700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d80b4b80 con 0x7f26d80d0610 2023-07-14T11:55:12.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.194+0000 7f26de737700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d809ea90 con 0x7f26d80cd090 2023-07-14T11:55:12.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.194+0000 7f26de737700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d809e410 con 0x7f26d80b0ee0 2023-07-14T11:55:12.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.194+0000 7f26ddf36700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26d80d0610 0x7f26d809c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.194+0000 7f26ddf36700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26d80d0610 0x7f26d809c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53782/0 (socket says 172.21.15.104:53782) 2023-07-14T11:55:12.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.194+0000 7f26dd735700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26d80b0ee0 0x7f26d814bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.194+0000 7f26ddf36700 1 -- 172.21.15.104:0/1318952228 learned_addr learned my addr 172.21.15.104:0/1318952228 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:12.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.194+0000 7f26dcf34700 1 --2- 172.21.15.104:0/1318952228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26d80cd090 0x7f26d80b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.198+0000 7f26dd735700 1 -- 172.21.15.104:0/1318952228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26d80cd090 msgr2=0x7f26d80b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.198+0000 7f26dd735700 1 --2- 172.21.15.104:0/1318952228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26d80cd090 0x7f26d80b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.198+0000 7f26dd735700 1 -- 172.21.15.104:0/1318952228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26d80d0610 msgr2=0x7f26d809c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.198+0000 7f26dd735700 1 --2- 172.21.15.104:0/1318952228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26d80d0610 0x7f26d809c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.198+0000 7f26dd735700 1 -- 172.21.15.104:0/1318952228 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c80094b0 con 0x7f26d80b0ee0 2023-07-14T11:55:12.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.198+0000 7f26dd735700 1 --2- 172.21.15.104:0/1318952228 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26d80b0ee0 0x7f26d814bb50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f26d400e870 tx=0x7f26d400eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:12.186 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.198+0000 7f26cdffb700 1 -- 172.21.15.104:0/1318952228 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4013680 con 0x7f26d80b0ee0 2023-07-14T11:55:12.186 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.198+0000 7f26cdffb700 1 -- 172.21.15.104:0/1318952228 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d40137e0 con 0x7f26d80b0ee0 2023-07-14T11:55:12.186 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.198+0000 7f26cdffb700 1 -- 172.21.15.104:0/1318952228 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4013b50 con 0x7f26d80b0ee0 2023-07-14T11:55:12.187 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.198+0000 7f26de737700 1 -- 172.21.15.104:0/1318952228 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d80b45c0 con 0x7f26d80b0ee0 2023-07-14T11:55:12.187 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.198+0000 7f26de737700 1 -- 172.21.15.104:0/1318952228 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d809e6b0 con 0x7f26d80b0ee0 2023-07-14T11:55:12.187 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.198+0000 7f26de737700 1 -- 172.21.15.104:0/1318952228 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d804e500 con 0x7f26d80b0ee0 2023-07-14T11:55:12.187 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.202+0000 7f26cdffb700 1 -- 172.21.15.104:0/1318952228 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f26d4013cb0 con 0x7f26d80b0ee0 2023-07-14T11:55:12.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.202+0000 7f26cdffb700 1 --2- 172.21.15.104:0/1318952228 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f26bc0662f0 0x7f26bc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.202+0000 7f26cdffb700 1 -- 172.21.15.104:0/1318952228 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f26d4083d80 con 0x7f26d80b0ee0 2023-07-14T11:55:12.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.202+0000 7f26dcf34700 1 --2- 172.21.15.104:0/1318952228 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f26bc0662f0 0x7f26bc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.206+0000 7f26dcf34700 1 --2- 172.21.15.104:0/1318952228 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f26bc0662f0 0x7f26bc0687a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f26c8000c00 tx=0x7f26c8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:12.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.210+0000 7f26cdffb700 1 -- 172.21.15.104:0/1318952228 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f26d4054920 con 0x7f26d80b0ee0 2023-07-14T11:55:12.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.386+0000 7f26de737700 1 -- 172.21.15.104:0/1318952228 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f26d80ce260 con 0x7f26d80b0ee0 2023-07-14T11:55:12.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.386+0000 7f26cdffb700 1 -- 172.21.15.104:0/1318952228 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f26d40170a0 con 0x7f26d80b0ee0 2023-07-14T11:55:12.373 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:55:12.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.390+0000 7f26de737700 1 -- 172.21.15.104:0/1318952228 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f26bc0662f0 msgr2=0x7f26bc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.390+0000 7f26de737700 1 --2- 172.21.15.104:0/1318952228 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f26bc0662f0 0x7f26bc0687a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f26c8000c00 tx=0x7f26c8008030 comp rx=0 tx=0).stop 2023-07-14T11:55:12.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.390+0000 7f26de737700 1 -- 172.21.15.104:0/1318952228 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26d80b0ee0 msgr2=0x7f26d814bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.390+0000 7f26de737700 1 --2- 172.21.15.104:0/1318952228 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26d80b0ee0 0x7f26d814bb50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f26d400e870 tx=0x7f26d400eb80 comp rx=0 tx=0).stop 2023-07-14T11:55:12.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.390+0000 7f26de737700 1 -- 172.21.15.104:0/1318952228 shutdown_connections 2023-07-14T11:55:12.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.390+0000 7f26de737700 1 --2- 172.21.15.104:0/1318952228 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f26bc0662f0 0x7f26bc0687a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.390+0000 7f26de737700 1 --2- 172.21.15.104:0/1318952228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26d80d0610 0x7f26d809c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.390+0000 7f26de737700 1 --2- 172.21.15.104:0/1318952228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26d80cd090 0x7f26d80b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.390+0000 7f26de737700 1 --2- 172.21.15.104:0/1318952228 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26d80b0ee0 0x7f26d814bb50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.390+0000 7f26de737700 1 -- 172.21.15.104:0/1318952228 >> 172.21.15.104:0/1318952228 conn(0x7f26d8060c40 msgr2=0x7f26d80b7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:12.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.390+0000 7f26de737700 1 -- 172.21.15.104:0/1318952228 shutdown_connections 2023-07-14T11:55:12.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.390+0000 7f26de737700 1 -- 172.21.15.104:0/1318952228 wait complete. 2023-07-14T11:55:12.399 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:55:12.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.586+0000 7f8757fff700 1 Processor -- start 2023-07-14T11:55:12.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.586+0000 7f8757fff700 1 -- start start 2023-07-14T11:55:12.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f8757fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87580b0ee0 0x7f87580ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f8757fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f87580cd090 0x7f87580d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f8757fff700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87580cd4e0 con 0x7f87580d0610 2023-07-14T11:55:12.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f8757fff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87580d1a60 con 0x7f87580cd090 2023-07-14T11:55:12.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f8757fff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87580d1ef0 con 0x7f87580b0ee0 2023-07-14T11:55:12.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f87567fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f87580cd090 0x7f87580d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f87567fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f87580cd090 0x7f87580d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53788/0 (socket says 172.21.15.104:53788) 2023-07-14T11:55:12.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f87567fc700 1 -- 172.21.15.104:0/1723753960 learned_addr learned my addr 172.21.15.104:0/1723753960 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:12.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f8756ffd700 1 --2- 172.21.15.104:0/1723753960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87580b0ee0 0x7f87580ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f87567fc700 1 -- 172.21.15.104:0/1723753960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87580b0ee0 msgr2=0x7f87580ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:12.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f87567fc700 1 --2- 172.21.15.104:0/1723753960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87580b0ee0 0x7f87580ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f87567fc700 1 -- 172.21.15.104:0/1723753960 >> v1:172.21.15.121:6789/0 conn(0x7f87580d0610 legacy=0x7f875815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f87567fc700 1 -- 172.21.15.104:0/1723753960 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87580d0e50 con 0x7f87580cd090 2023-07-14T11:55:12.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f8756ffd700 1 --2- 172.21.15.104:0/1723753960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87580b0ee0 0x7f87580ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:12.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f8755ffb700 1 -- 172.21.15.104:0/1723753960 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573244907 0 0) 0x7f8750001490 con 0x7f87580d0610 2023-07-14T11:55:12.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f87567fc700 1 --2- 172.21.15.104:0/1723753960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87580cd090 0x7f87580d00d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f874c009940 tx=0x7f874c009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03070ad17abca83 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:12.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f8755ffb700 1 -- 172.21.15.104:0/1723753960 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c00ba20 con 0x7f87580cd090 2023-07-14T11:55:12.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f8755ffb700 1 -- 172.21.15.104:0/1723753960 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874c009e90 con 0x7f87580cd090 2023-07-14T11:55:12.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.590+0000 7f8755ffb700 1 -- 172.21.15.104:0/1723753960 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c00bda0 con 0x7f87580cd090 2023-07-14T11:55:12.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.594+0000 7f8757fff700 1 -- 172.21.15.104:0/1723753960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87580cd090 msgr2=0x7f87580d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.594+0000 7f8757fff700 1 --2- 172.21.15.104:0/1723753960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87580cd090 0x7f87580d00d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f874c009940 tx=0x7f874c009c50 comp rx=0 tx=0).stop 2023-07-14T11:55:12.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.594+0000 7f8757fff700 1 -- 172.21.15.104:0/1723753960 shutdown_connections 2023-07-14T11:55:12.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.594+0000 7f8757fff700 1 --2- 172.21.15.104:0/1723753960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87580cd090 0x7f87580d00d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.594+0000 7f8757fff700 1 --2- 172.21.15.104:0/1723753960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87580b0ee0 0x7f87580ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.594+0000 7f8757fff700 1 -- 172.21.15.104:0/1723753960 >> 172.21.15.104:0/1723753960 conn(0x7f8758060c40 msgr2=0x7f875809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:12.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.594+0000 7f8757fff700 1 -- 172.21.15.104:0/1723753960 shutdown_connections 2023-07-14T11:55:12.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.594+0000 7f8757fff700 1 -- 172.21.15.104:0/1723753960 wait complete. 2023-07-14T11:55:12.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f8757fff700 1 Processor -- start 2023-07-14T11:55:12.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f8757fff700 1 -- start start 2023-07-14T11:55:12.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f8757fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87580b0ee0 0x7f875815e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f8757fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87580cd090 0x7f875815a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f8757fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87580d0610 0x7f87580b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f8757fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875815ddf0 con 0x7f87580cd090 2023-07-14T11:55:12.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f8757fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875815a6c0 con 0x7f87580d0610 2023-07-14T11:55:12.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f8757fff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87580c9050 con 0x7f87580b0ee0 2023-07-14T11:55:12.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f87567fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87580cd090 0x7f875815a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f8756ffd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87580b0ee0 0x7f875815e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f87567fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87580cd090 0x7f875815a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53794/0 (socket says 172.21.15.104:53794) 2023-07-14T11:55:12.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f87567fc700 1 -- 172.21.15.104:0/4287847498 learned_addr learned my addr 172.21.15.104:0/4287847498 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:12.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f87577fe700 1 --2- 172.21.15.104:0/4287847498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87580d0610 0x7f87580b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f87567fc700 1 -- 172.21.15.104:0/4287847498 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87580b0ee0 msgr2=0x7f875815e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f87567fc700 1 --2- 172.21.15.104:0/4287847498 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87580b0ee0 0x7f875815e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f87567fc700 1 -- 172.21.15.104:0/4287847498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87580d0610 msgr2=0x7f87580b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f87567fc700 1 --2- 172.21.15.104:0/4287847498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87580d0610 0x7f87580b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.598+0000 7f87567fc700 1 -- 172.21.15.104:0/4287847498 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c009570 con 0x7f87580cd090 2023-07-14T11:55:12.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.602+0000 7f87567fc700 1 --2- 172.21.15.104:0/4287847498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87580cd090 0x7f875815a180 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f874c000c00 tx=0x7f874c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:12.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.602+0000 7f873ffff700 1 -- 172.21.15.104:0/4287847498 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c011830 con 0x7f87580cd090 2023-07-14T11:55:12.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.602+0000 7f873ffff700 1 -- 172.21.15.104:0/4287847498 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874c01e070 con 0x7f87580cd090 2023-07-14T11:55:12.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.602+0000 7f873ffff700 1 -- 172.21.15.104:0/4287847498 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c011b90 con 0x7f87580cd090 2023-07-14T11:55:12.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.602+0000 7f8757fff700 1 -- 172.21.15.104:0/4287847498 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875815e090 con 0x7f87580cd090 2023-07-14T11:55:12.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.602+0000 7f8757fff700 1 -- 172.21.15.104:0/4287847498 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87580c9330 con 0x7f87580cd090 2023-07-14T11:55:12.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.602+0000 7f8757fff700 1 -- 172.21.15.104:0/4287847498 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f875815e5b0 con 0x7f87580cd090 2023-07-14T11:55:12.591 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.606+0000 7f873ffff700 1 -- 172.21.15.104:0/4287847498 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f874c011cf0 con 0x7f87580cd090 2023-07-14T11:55:12.592 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.606+0000 7f873ffff700 1 --2- 172.21.15.104:0/4287847498 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87440662f0 0x7f87440687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.592 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.606+0000 7f873ffff700 1 -- 172.21.15.104:0/4287847498 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f874c020070 con 0x7f87580cd090 2023-07-14T11:55:12.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.606+0000 7f8756ffd700 1 --2- 172.21.15.104:0/4287847498 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87440662f0 0x7f87440687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.594 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.606+0000 7f8756ffd700 1 --2- 172.21.15.104:0/4287847498 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87440662f0 0x7f87440687a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f874800c960 tx=0x7f8748008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:12.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.610+0000 7f873ffff700 1 -- 172.21.15.104:0/4287847498 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f875815e5b0 con 0x7f87580cd090 2023-07-14T11:55:12.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.790+0000 7f8757fff700 1 -- 172.21.15.104:0/4287847498 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f875815e5b0 con 0x7f87580cd090 2023-07-14T11:55:12.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.790+0000 7f873ffff700 1 -- 172.21.15.104:0/4287847498 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7f875815e5b0 con 0x7f87580cd090 2023-07-14T11:55:12.777 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:55:12.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.794+0000 7f8757fff700 1 -- 172.21.15.104:0/4287847498 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87440662f0 msgr2=0x7f87440687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.794+0000 7f8757fff700 1 --2- 172.21.15.104:0/4287847498 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87440662f0 0x7f87440687a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f874800c960 tx=0x7f8748008040 comp rx=0 tx=0).stop 2023-07-14T11:55:12.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.794+0000 7f8757fff700 1 -- 172.21.15.104:0/4287847498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87580cd090 msgr2=0x7f875815a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.794+0000 7f8757fff700 1 --2- 172.21.15.104:0/4287847498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87580cd090 0x7f875815a180 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f874c000c00 tx=0x7f874c004770 comp rx=0 tx=0).stop 2023-07-14T11:55:12.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.794+0000 7f8757fff700 1 -- 172.21.15.104:0/4287847498 shutdown_connections 2023-07-14T11:55:12.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.794+0000 7f8757fff700 1 --2- 172.21.15.104:0/4287847498 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f87440662f0 0x7f87440687a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.794+0000 7f8757fff700 1 --2- 172.21.15.104:0/4287847498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87580cd090 0x7f875815a180 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.794+0000 7f8757fff700 1 --2- 172.21.15.104:0/4287847498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87580d0610 0x7f87580b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.794+0000 7f8757fff700 1 --2- 172.21.15.104:0/4287847498 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87580b0ee0 0x7f875815e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.794+0000 7f8757fff700 1 -- 172.21.15.104:0/4287847498 >> 172.21.15.104:0/4287847498 conn(0x7f8758060c40 msgr2=0x7f875815b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:12.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.794+0000 7f8757fff700 1 -- 172.21.15.104:0/4287847498 shutdown_connections 2023-07-14T11:55:12.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.794+0000 7f8757fff700 1 -- 172.21.15.104:0/4287847498 wait complete. 2023-07-14T11:55:12.798 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:55:12.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.982+0000 7f3a6a1e3700 1 Processor -- start 2023-07-14T11:55:12.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.982+0000 7f3a6a1e3700 1 -- start start 2023-07-14T11:55:12.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a6a1e3700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a64159c60 0x7f3a6406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a6a1e3700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a640714f0 con 0x7f3a640b0770 2023-07-14T11:55:12.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a6a1e3700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a6406da00 con 0x7f3a640cab00 2023-07-14T11:55:12.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a6a1e3700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6406dce0 con 0x7f3a64159c60 2023-07-14T11:55:12.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a689e0700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f3a640b0770 0x7f3a64070de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35204/0 (socket says 172.21.15.104:35204) 2023-07-14T11:55:12.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a689e0700 1 -- 172.21.15.104:0/2371057830 learned_addr learned my addr 172.21.15.104:0/2371057830 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:12.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a691e1700 1 --2- 172.21.15.104:0/2371057830 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a64159c60 0x7f3a6406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a5bfff700 1 -- 172.21.15.104:0/2371057830 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346203297 0 0) 0x7f3a640714f0 con 0x7f3a640b0770 2023-07-14T11:55:12.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a5bfff700 1 -- 172.21.15.104:0/2371057830 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a500035d0 con 0x7f3a640b0770 2023-07-14T11:55:12.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a5bfff700 1 -- 172.21.15.104:0/2371057830 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737046258 0 0) 0x7f3a6406da00 con 0x7f3a640cab00 2023-07-14T11:55:12.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a5bfff700 1 -- 172.21.15.104:0/2371057830 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a640714f0 con 0x7f3a640cab00 2023-07-14T11:55:12.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a691e1700 1 -- 172.21.15.104:0/2371057830 >> v1:172.21.15.121:6789/0 conn(0x7f3a640cab00 legacy=0x7f3a640c1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a691e1700 1 -- 172.21.15.104:0/2371057830 >> v1:172.21.15.104:6789/0 conn(0x7f3a640b0770 legacy=0x7f3a64070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a5bfff700 1 -- 172.21.15.104:0/2371057830 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 459506921 0 0) 0x7f3a500035d0 con 0x7f3a640b0770 2023-07-14T11:55:12.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a691e1700 1 -- 172.21.15.104:0/2371057830 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6406dfc0 con 0x7f3a64159c60 2023-07-14T11:55:12.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a691e1700 1 --2- 172.21.15.104:0/2371057830 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a64159c60 0x7f3a6406d3b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009880 tx=0x7f3a60009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5da8c50a6f9ebb34 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:12.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a5bfff700 1 -- 172.21.15.104:0/2371057830 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54001670 con 0x7f3a64159c60 2023-07-14T11:55:12.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a5bfff700 1 -- 172.21.15.104:0/2371057830 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a60004030 con 0x7f3a64159c60 2023-07-14T11:55:12.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.986+0000 7f3a5bfff700 1 -- 172.21.15.104:0/2371057830 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a600043c0 con 0x7f3a64159c60 2023-07-14T11:55:12.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.990+0000 7f3a6a1e3700 1 -- 172.21.15.104:0/2371057830 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a64159c60 msgr2=0x7f3a6406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.990+0000 7f3a6a1e3700 1 --2- 172.21.15.104:0/2371057830 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a64159c60 0x7f3a6406d3b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009880 tx=0x7f3a60009b90 comp rx=0 tx=0).stop 2023-07-14T11:55:12.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.990+0000 7f3a6a1e3700 1 -- 172.21.15.104:0/2371057830 shutdown_connections 2023-07-14T11:55:12.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.990+0000 7f3a6a1e3700 1 --2- 172.21.15.104:0/2371057830 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a64159c60 0x7f3a6406d3b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.990+0000 7f3a6a1e3700 1 -- 172.21.15.104:0/2371057830 >> 172.21.15.104:0/2371057830 conn(0x7f3a640afc60 msgr2=0x7f3a640b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:12.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.990+0000 7f3a6a1e3700 1 -- 172.21.15.104:0/2371057830 shutdown_connections 2023-07-14T11:55:12.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.990+0000 7f3a6a1e3700 1 -- 172.21.15.104:0/2371057830 wait complete. 2023-07-14T11:55:12.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.990+0000 7f3a6a1e3700 1 Processor -- start 2023-07-14T11:55:12.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.990+0000 7f3a6a1e3700 1 -- start start 2023-07-14T11:55:12.979 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a6a1e3700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a640b0770 0x7f3a640b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.979 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a6a1e3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a640cab00 0x7f3a640b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.979 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a6a1e3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64159c60 0x7f3a64157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.979 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a6a1e3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a640b39c0 con 0x7f3a640cab00 2023-07-14T11:55:12.979 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a6a1e3700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a640b1ea0 con 0x7f3a64159c60 2023-07-14T11:55:12.980 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a6a1e3700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a640b2db0 con 0x7f3a640b0770 2023-07-14T11:55:12.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a689e0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a640cab00 0x7f3a640b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a689e0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a640cab00 0x7f3a640b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53806/0 (socket says 172.21.15.104:53806) 2023-07-14T11:55:12.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a689e0700 1 -- 172.21.15.104:0/2772334960 learned_addr learned my addr 172.21.15.104:0/2772334960 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:12.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a699e2700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64159c60 0x7f3a64157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.982 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a691e1700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a640b0770 0x7f3a640b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.982 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a689e0700 1 -- 172.21.15.104:0/2772334960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a640b0770 msgr2=0x7f3a640b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.982 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a689e0700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a640b0770 0x7f3a640b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.982 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a689e0700 1 -- 172.21.15.104:0/2772334960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64159c60 msgr2=0x7f3a64157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:12.982 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a689e0700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64159c60 0x7f3a64157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:12.982 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a689e0700 1 -- 172.21.15.104:0/2772334960 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a600094b0 con 0x7f3a640cab00 2023-07-14T11:55:12.982 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a699e2700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64159c60 0x7f3a64157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:12.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a691e1700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a640b0770 0x7f3a640b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:12.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a689e0700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a640cab00 0x7f3a640b1960 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400c700 tx=0x7f3a5400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:12.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a59ffb700 1 -- 172.21.15.104:0/2772334960 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54004ca0 con 0x7f3a640cab00 2023-07-14T11:55:12.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a59ffb700 1 -- 172.21.15.104:0/2772334960 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a54004e20 con 0x7f3a640cab00 2023-07-14T11:55:12.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a6a1e3700 1 -- 172.21.15.104:0/2772334960 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a640b3090 con 0x7f3a640cab00 2023-07-14T11:55:12.984 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a59ffb700 1 -- 172.21.15.104:0/2772334960 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54007310 con 0x7f3a640cab00 2023-07-14T11:55:12.984 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.994+0000 7f3a6a1e3700 1 -- 172.21.15.104:0/2772334960 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a640b3650 con 0x7f3a640cab00 2023-07-14T11:55:12.984 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.998+0000 7f3a6a1e3700 1 -- 172.21.15.104:0/2772334960 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a640b3b70 con 0x7f3a640cab00 2023-07-14T11:55:12.984 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.998+0000 7f3a59ffb700 1 -- 172.21.15.104:0/2772334960 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3a54007470 con 0x7f3a640cab00 2023-07-14T11:55:12.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.998+0000 7f3a59ffb700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3a480662f0 0x7f3a480687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:12.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.998+0000 7f3a59ffb700 1 -- 172.21.15.104:0/2772334960 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f3a54083e10 con 0x7f3a640cab00 2023-07-14T11:55:12.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.998+0000 7f3a691e1700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3a480662f0 0x7f3a480687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:12.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:12.998+0000 7f3a691e1700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3a480662f0 0x7f3a480687a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a600090b0 tx=0x7f3a60010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:12.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.006+0000 7f3a59ffb700 1 -- 172.21.15.104:0/2772334960 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f3a640b3b70 con 0x7f3a640cab00 2023-07-14T11:55:13.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.174+0000 7f3a6a1e3700 1 -- 172.21.15.104:0/2772334960 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3a6404e500 con 0x7f3a640cab00 2023-07-14T11:55:13.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.174+0000 7f3a59ffb700 1 -- 172.21.15.104:0/2772334960 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3a54087030 con 0x7f3a640cab00 2023-07-14T11:55:13.163 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:55:13.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.178+0000 7f3a6a1e3700 1 -- 172.21.15.104:0/2772334960 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3a480662f0 msgr2=0x7f3a480687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:13.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.178+0000 7f3a6a1e3700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3a480662f0 0x7f3a480687a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a600090b0 tx=0x7f3a60010000 comp rx=0 tx=0).stop 2023-07-14T11:55:13.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.178+0000 7f3a6a1e3700 1 -- 172.21.15.104:0/2772334960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a640cab00 msgr2=0x7f3a640b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:13.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.178+0000 7f3a6a1e3700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a640cab00 0x7f3a640b1960 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400c700 tx=0x7f3a5400cac0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.178+0000 7f3a6a1e3700 1 -- 172.21.15.104:0/2772334960 shutdown_connections 2023-07-14T11:55:13.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.178+0000 7f3a6a1e3700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3a480662f0 0x7f3a480687a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.178+0000 7f3a6a1e3700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a640cab00 0x7f3a640b1960 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.178+0000 7f3a6a1e3700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64159c60 0x7f3a64157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.178+0000 7f3a6a1e3700 1 --2- 172.21.15.104:0/2772334960 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a640b0770 0x7f3a640b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.178+0000 7f3a6a1e3700 1 -- 172.21.15.104:0/2772334960 >> 172.21.15.104:0/2772334960 conn(0x7f3a640afc60 msgr2=0x7f3a640c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:13.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.178+0000 7f3a6a1e3700 1 -- 172.21.15.104:0/2772334960 shutdown_connections 2023-07-14T11:55:13.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.178+0000 7f3a6a1e3700 1 -- 172.21.15.104:0/2772334960 wait complete. 2023-07-14T11:55:13.184 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-07-14T11:55:13.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.382+0000 7f69a44df700 1 Processor -- start 2023-07-14T11:55:13.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.382+0000 7f69a44df700 1 -- start start 2023-07-14T11:55:13.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.382+0000 7f69a44df700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f699c0b1130 0x7f699c0d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:13.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.382+0000 7f69a44df700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699c0b0ca0 con 0x7f699c15c1a0 2023-07-14T11:55:13.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.382+0000 7f69a44df700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699c0b03b0 con 0x7f699c0b4130 2023-07-14T11:55:13.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.382+0000 7f69a44df700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699c0d2530 con 0x7f699c0b1130 2023-07-14T11:55:13.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.386+0000 7f69a1a7a700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f699c15c1a0 0x7f699c06fdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35216/0 (socket says 172.21.15.104:35216) 2023-07-14T11:55:13.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.386+0000 7f69a1a7a700 1 -- 172.21.15.104:0/1485440549 learned_addr learned my addr 172.21.15.104:0/1485440549 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:13.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.386+0000 7f69a2a7c700 1 --2- 172.21.15.104:0/1485440549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f699c0b1130 0x7f699c0d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:13.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.386+0000 7f69a1279700 1 -- 172.21.15.104:0/1485440549 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284562346 0 0) 0x7f699c0b0ca0 con 0x7f699c15c1a0 2023-07-14T11:55:13.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.386+0000 7f69a1279700 1 -- 172.21.15.104:0/1485440549 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69880035d0 con 0x7f699c15c1a0 2023-07-14T11:55:13.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.386+0000 7f69a2a7c700 1 -- 172.21.15.104:0/1485440549 >> v1:172.21.15.121:6789/0 conn(0x7f699c0b4130 legacy=0x7f699c0b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:13.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.386+0000 7f69a1279700 1 -- 172.21.15.104:0/1485440549 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009088819 0 0) 0x7f699c0b03b0 con 0x7f699c0b4130 2023-07-14T11:55:13.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.386+0000 7f69a2a7c700 1 -- 172.21.15.104:0/1485440549 >> v1:172.21.15.104:6789/0 conn(0x7f699c15c1a0 legacy=0x7f699c06fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:13.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.386+0000 7f69a2a7c700 1 -- 172.21.15.104:0/1485440549 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c0b0650 con 0x7f699c0b1130 2023-07-14T11:55:13.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.386+0000 7f69a2a7c700 1 --2- 172.21.15.104:0/1485440549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f699c0b1130 0x7f699c0d1ff0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6994009dc0 tx=0x7f699400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7a875098a99e3f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:13.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.386+0000 7f69a1279700 1 -- 172.21.15.104:0/1485440549 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699400e070 con 0x7f699c0b1130 2023-07-14T11:55:13.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.386+0000 7f69a1279700 1 -- 172.21.15.104:0/1485440549 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699400bb20 con 0x7f699c0b1130 2023-07-14T11:55:13.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.386+0000 7f69a1279700 1 -- 172.21.15.104:0/1485440549 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699400bed0 con 0x7f699c0b1130 2023-07-14T11:55:13.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.386+0000 7f69a44df700 1 -- 172.21.15.104:0/1485440549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f699c0b1130 msgr2=0x7f699c0d1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:13.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.386+0000 7f69a44df700 1 --2- 172.21.15.104:0/1485440549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f699c0b1130 0x7f699c0d1ff0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6994009dc0 tx=0x7f699400b330 comp rx=0 tx=0).stop 2023-07-14T11:55:13.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.390+0000 7f69a44df700 1 -- 172.21.15.104:0/1485440549 shutdown_connections 2023-07-14T11:55:13.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.390+0000 7f69a44df700 1 --2- 172.21.15.104:0/1485440549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f699c0b1130 0x7f699c0d1ff0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.390+0000 7f69a44df700 1 -- 172.21.15.104:0/1485440549 >> 172.21.15.104:0/1485440549 conn(0x7f699c09ba10 msgr2=0x7f699c0d26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:13.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.390+0000 7f69a44df700 1 -- 172.21.15.104:0/1485440549 shutdown_connections 2023-07-14T11:55:13.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.390+0000 7f69a44df700 1 -- 172.21.15.104:0/1485440549 wait complete. 2023-07-14T11:55:13.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.390+0000 7f69a44df700 1 Processor -- start 2023-07-14T11:55:13.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.390+0000 7f69a44df700 1 -- start start 2023-07-14T11:55:13.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a44df700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f699c0b1130 0x7f699c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:13.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a44df700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f699c0b4130 0x7f699c0b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:13.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a44df700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f699c15c1a0 0x7f699c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:13.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a44df700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c0ca3a0 con 0x7f699c0b1130 2023-07-14T11:55:13.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a44df700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c0c9790 con 0x7f699c0b4130 2023-07-14T11:55:13.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a44df700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699c0c9a00 con 0x7f699c15c1a0 2023-07-14T11:55:13.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a227b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f699c0b1130 0x7f699c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:13.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a227b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f699c0b1130 0x7f699c0b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53822/0 (socket says 172.21.15.104:53822) 2023-07-14T11:55:13.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a227b700 1 -- 172.21.15.104:0/107884278 learned_addr learned my addr 172.21.15.104:0/107884278 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:13.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a2a7c700 1 --2- 172.21.15.104:0/107884278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f699c15c1a0 0x7f699c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:13.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a1a7a700 1 --2- 172.21.15.104:0/107884278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f699c0b4130 0x7f699c0b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:13.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a227b700 1 -- 172.21.15.104:0/107884278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f699c15c1a0 msgr2=0x7f699c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:13.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a227b700 1 --2- 172.21.15.104:0/107884278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f699c15c1a0 0x7f699c159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a227b700 1 -- 172.21.15.104:0/107884278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f699c0b4130 msgr2=0x7f699c0b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:13.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a227b700 1 --2- 172.21.15.104:0/107884278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f699c0b4130 0x7f699c0b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a227b700 1 -- 172.21.15.104:0/107884278 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6994009a70 con 0x7f699c0b1130 2023-07-14T11:55:13.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a1a7a700 1 --2- 172.21.15.104:0/107884278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f699c0b4130 0x7f699c0b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:13.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a227b700 1 --2- 172.21.15.104:0/107884278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f699c0b1130 0x7f699c0b2b90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f699800c780 tx=0x7f699800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:13.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f6992ffd700 1 -- 172.21.15.104:0/107884278 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69980054e0 con 0x7f699c0b1130 2023-07-14T11:55:13.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f6992ffd700 1 -- 172.21.15.104:0/107884278 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69980036f0 con 0x7f699c0b1130 2023-07-14T11:55:13.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f6992ffd700 1 -- 172.21.15.104:0/107884278 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998003aa0 con 0x7f699c0b1130 2023-07-14T11:55:13.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a44df700 1 -- 172.21.15.104:0/107884278 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699c0c9ce0 con 0x7f699c0b1130 2023-07-14T11:55:13.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.394+0000 7f69a44df700 1 -- 172.21.15.104:0/107884278 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699c15a1f0 con 0x7f699c0b1130 2023-07-14T11:55:13.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.398+0000 7f69a44df700 1 -- 172.21.15.104:0/107884278 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f699c04dcb0 con 0x7f699c0b1130 2023-07-14T11:55:13.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.398+0000 7f6992ffd700 1 -- 172.21.15.104:0/107884278 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6998003c00 con 0x7f699c0b1130 2023-07-14T11:55:13.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.398+0000 7f6992ffd700 1 --2- 172.21.15.104:0/107884278 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f69800662f0 0x7f69800687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:13.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.398+0000 7f6992ffd700 1 -- 172.21.15.104:0/107884278 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f6998083580 con 0x7f699c0b1130 2023-07-14T11:55:13.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.398+0000 7f69a1a7a700 1 --2- 172.21.15.104:0/107884278 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f69800662f0 0x7f69800687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:13.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.402+0000 7f69a1a7a700 1 --2- 172.21.15.104:0/107884278 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f69800662f0 0x7f69800687a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f699c0b5590 tx=0x7f698c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:13.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.406+0000 7f6992ffd700 1 -- 172.21.15.104:0/107884278 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f6998050690 con 0x7f699c0b1130 2023-07-14T11:55:13.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.574+0000 7f69a44df700 1 -- 172.21.15.104:0/107884278 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f699c0b47c0 con 0x7f699c0b1130 2023-07-14T11:55:13.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.574+0000 7f6992ffd700 1 -- 172.21.15.104:0/107884278 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v45) v1 ==== 97+0+11 (secure 0 0 0) 0x7f6998053cb0 con 0x7f699c0b1130 2023-07-14T11:55:13.562 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:55:13.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.578+0000 7f69a44df700 1 -- 172.21.15.104:0/107884278 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f69800662f0 msgr2=0x7f69800687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:13.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.578+0000 7f69a44df700 1 --2- 172.21.15.104:0/107884278 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f69800662f0 0x7f69800687a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f699c0b5590 tx=0x7f698c009040 comp rx=0 tx=0).stop 2023-07-14T11:55:13.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.578+0000 7f69a44df700 1 -- 172.21.15.104:0/107884278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f699c0b1130 msgr2=0x7f699c0b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:13.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.578+0000 7f69a44df700 1 --2- 172.21.15.104:0/107884278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f699c0b1130 0x7f699c0b2b90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f699800c780 tx=0x7f699800ca90 comp rx=0 tx=0).stop 2023-07-14T11:55:13.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.578+0000 7f69a44df700 1 -- 172.21.15.104:0/107884278 shutdown_connections 2023-07-14T11:55:13.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.578+0000 7f69a44df700 1 --2- 172.21.15.104:0/107884278 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f69800662f0 0x7f69800687a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.578+0000 7f69a44df700 1 --2- 172.21.15.104:0/107884278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f699c15c1a0 0x7f699c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.578+0000 7f69a44df700 1 --2- 172.21.15.104:0/107884278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f699c0b4130 0x7f699c0b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.578+0000 7f69a44df700 1 --2- 172.21.15.104:0/107884278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f699c0b1130 0x7f699c0b2b90 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.578+0000 7f69a44df700 1 -- 172.21.15.104:0/107884278 >> 172.21.15.104:0/107884278 conn(0x7f699c09ba10 msgr2=0x7f699c070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:13.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.578+0000 7f69a44df700 1 -- 172.21.15.104:0/107884278 shutdown_connections 2023-07-14T11:55:13.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.578+0000 7f69a44df700 1 -- 172.21.15.104:0/107884278 wait complete. 2023-07-14T11:55:13.587 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:55:13.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.766+0000 7f81f2f37700 1 Processor -- start 2023-07-14T11:55:13.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f2f37700 1 -- start start 2023-07-14T11:55:13.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f2f37700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81ec0c9d90 0x7f81ec0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:13.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f2f37700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f81ec159f20 0x7f81ec0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:13.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f2f37700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec0b0ef0 con 0x7f81ec0caf70 2023-07-14T11:55:13.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f2f37700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ec0ce9c0 con 0x7f81ec159f20 2023-07-14T11:55:13.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f2f37700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec0cee50 con 0x7f81ec0c9d90 2023-07-14T11:55:13.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f2736700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f81ec0caf70 0x7f81ec0c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35222/0 (socket says 172.21.15.104:35222) 2023-07-14T11:55:13.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f2736700 1 -- 172.21.15.104:0/3383711716 learned_addr learned my addr 172.21.15.104:0/3383711716 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:13.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f1734700 1 --2- 172.21.15.104:0/3383711716 >> v2:172.21.15.121:3300/0 conn(0x7f81ec159f20 0x7f81ec0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:13.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f1f35700 1 --2- 172.21.15.104:0/3383711716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81ec0c9d90 0x7f81ec0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:13.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f0f33700 1 -- 172.21.15.104:0/3383711716 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593924222 0 0) 0x7f81ec0b0ef0 con 0x7f81ec0caf70 2023-07-14T11:55:13.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f0f33700 1 -- 172.21.15.104:0/3383711716 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81d80035d0 con 0x7f81ec0caf70 2023-07-14T11:55:13.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f1734700 1 -- 172.21.15.104:0/3383711716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81ec0c9d90 msgr2=0x7f81ec0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:13.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f1734700 1 --2- 172.21.15.104:0/3383711716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81ec0c9d90 0x7f81ec0ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f0f33700 1 -- 172.21.15.104:0/3383711716 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 733954436 0 0) 0x7f81d80035d0 con 0x7f81ec0caf70 2023-07-14T11:55:13.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f1734700 1 -- 172.21.15.104:0/3383711716 >> v1:172.21.15.104:6789/0 conn(0x7f81ec0caf70 legacy=0x7f81ec0c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:13.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.770+0000 7f81f1734700 1 -- 172.21.15.104:0/3383711716 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec0ca6a0 con 0x7f81ec159f20 2023-07-14T11:55:13.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.774+0000 7f81f1734700 1 --2- 172.21.15.104:0/3383711716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81ec159f20 0x7f81ec0caa30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009850 tx=0x7f81dc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dfd944e0e32819d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:13.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.774+0000 7f81f0f33700 1 -- 172.21.15.104:0/3383711716 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e40019f0 con 0x7f81ec159f20 2023-07-14T11:55:13.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.774+0000 7f81f0f33700 1 -- 172.21.15.104:0/3383711716 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc004030 con 0x7f81ec159f20 2023-07-14T11:55:13.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.774+0000 7f81f0f33700 1 -- 172.21.15.104:0/3383711716 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc0043e0 con 0x7f81ec159f20 2023-07-14T11:55:13.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.774+0000 7f81f2f37700 1 -- 172.21.15.104:0/3383711716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81ec159f20 msgr2=0x7f81ec0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:13.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.774+0000 7f81f2f37700 1 --2- 172.21.15.104:0/3383711716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81ec159f20 0x7f81ec0caa30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009850 tx=0x7f81dc009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:13.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.774+0000 7f81f2f37700 1 -- 172.21.15.104:0/3383711716 shutdown_connections 2023-07-14T11:55:13.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.774+0000 7f81f2f37700 1 --2- 172.21.15.104:0/3383711716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81ec159f20 0x7f81ec0caa30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.774+0000 7f81f2f37700 1 --2- 172.21.15.104:0/3383711716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81ec0c9d90 0x7f81ec0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.774+0000 7f81f2f37700 1 -- 172.21.15.104:0/3383711716 >> 172.21.15.104:0/3383711716 conn(0x7f81ec06e960 msgr2=0x7f81ec06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:13.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.774+0000 7f81f2f37700 1 -- 172.21.15.104:0/3383711716 shutdown_connections 2023-07-14T11:55:13.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.774+0000 7f81f2f37700 1 -- 172.21.15.104:0/3383711716 wait complete. 2023-07-14T11:55:13.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.774+0000 7f81f2f37700 1 Processor -- start 2023-07-14T11:55:13.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.774+0000 7f81f2f37700 1 -- start start 2023-07-14T11:55:13.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f2f37700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ec0c9d90 0x7f81ec153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:13.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f2f37700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81ec0caf70 0x7f81ec155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:13.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f2f37700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81ec159f20 0x7f81ec0c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:13.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f2f37700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec153fc0 con 0x7f81ec0c9d90 2023-07-14T11:55:13.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f2f37700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec1570a0 con 0x7f81ec159f20 2023-07-14T11:55:13.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f2f37700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec156800 con 0x7f81ec0caf70 2023-07-14T11:55:13.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f1f35700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ec0c9d90 0x7f81ec153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:13.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f1f35700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ec0c9d90 0x7f81ec153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53838/0 (socket says 172.21.15.104:53838) 2023-07-14T11:55:13.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f1f35700 1 -- 172.21.15.104:0/774957019 learned_addr learned my addr 172.21.15.104:0/774957019 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:13.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f1734700 1 --2- 172.21.15.104:0/774957019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81ec0caf70 0x7f81ec155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:13.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f2736700 1 --2- 172.21.15.104:0/774957019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81ec159f20 0x7f81ec0c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:13.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f1f35700 1 -- 172.21.15.104:0/774957019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81ec0caf70 msgr2=0x7f81ec155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:13.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f1f35700 1 --2- 172.21.15.104:0/774957019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81ec0caf70 0x7f81ec155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f1f35700 1 -- 172.21.15.104:0/774957019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81ec159f20 msgr2=0x7f81ec0c3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:13.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f1f35700 1 --2- 172.21.15.104:0/774957019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81ec159f20 0x7f81ec0c3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f1f35700 1 -- 172.21.15.104:0/774957019 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc0094b0 con 0x7f81ec0c9d90 2023-07-14T11:55:13.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f1f35700 1 --2- 172.21.15.104:0/774957019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ec0c9d90 0x7f81ec153a80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f81e800e840 tx=0x7f81e800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:13.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81e27fc700 1 -- 172.21.15.104:0/774957019 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e80095a0 con 0x7f81ec0c9d90 2023-07-14T11:55:13.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81e27fc700 1 -- 172.21.15.104:0/774957019 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e8013680 con 0x7f81ec0c9d90 2023-07-14T11:55:13.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f2f37700 1 -- 172.21.15.104:0/774957019 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ec156ae0 con 0x7f81ec0c9d90 2023-07-14T11:55:13.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81e27fc700 1 -- 172.21.15.104:0/774957019 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e80139f0 con 0x7f81ec0c9d90 2023-07-14T11:55:13.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.778+0000 7f81f2f37700 1 -- 172.21.15.104:0/774957019 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ec0c1cc0 con 0x7f81ec0c9d90 2023-07-14T11:55:13.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.782+0000 7f81f2f37700 1 -- 172.21.15.104:0/774957019 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ec04e500 con 0x7f81ec0c9d90 2023-07-14T11:55:13.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.782+0000 7f81e27fc700 1 -- 172.21.15.104:0/774957019 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f81e8013b50 con 0x7f81ec0c9d90 2023-07-14T11:55:13.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.786+0000 7f81e27fc700 1 --2- 172.21.15.104:0/774957019 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f81d0066320 0x7f81d00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:13.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.786+0000 7f81e27fc700 1 -- 172.21.15.104:0/774957019 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f81e8083ef0 con 0x7f81ec0c9d90 2023-07-14T11:55:13.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.786+0000 7f81f1734700 1 --2- 172.21.15.104:0/774957019 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f81d0066320 0x7f81d00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:13.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.786+0000 7f81f1734700 1 --2- 172.21.15.104:0/774957019 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f81d0066320 0x7f81d00687d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0090b0 tx=0x7f81dc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:13.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.790+0000 7f81e27fc700 1 -- 172.21.15.104:0/774957019 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f81e8054b20 con 0x7f81ec0c9d90 2023-07-14T11:55:13.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.978+0000 7f81f2f37700 1 -- 172.21.15.104:0/774957019 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f81ec0ce340 con 0x7f81ec0c9d90 2023-07-14T11:55:13.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.978+0000 7f81e27fc700 1 -- 172.21.15.104:0/774957019 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) v1 ==== 80+0+0 (secure 0 0 0) 0x7f81e80546b0 con 0x7f81ec0c9d90 2023-07-14T11:55:13.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.982+0000 7f81f2f37700 1 -- 172.21.15.104:0/774957019 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f81d0066320 msgr2=0x7f81d00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:13.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.982+0000 7f81f2f37700 1 --2- 172.21.15.104:0/774957019 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f81d0066320 0x7f81d00687d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0090b0 tx=0x7f81dc008030 comp rx=0 tx=0).stop 2023-07-14T11:55:13.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.982+0000 7f81f2f37700 1 -- 172.21.15.104:0/774957019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ec0c9d90 msgr2=0x7f81ec153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:13.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.982+0000 7f81f2f37700 1 --2- 172.21.15.104:0/774957019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ec0c9d90 0x7f81ec153a80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f81e800e840 tx=0x7f81e800eb50 comp rx=0 tx=0).stop 2023-07-14T11:55:13.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.982+0000 7f81f2f37700 1 -- 172.21.15.104:0/774957019 shutdown_connections 2023-07-14T11:55:13.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.982+0000 7f81f2f37700 1 --2- 172.21.15.104:0/774957019 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f81d0066320 0x7f81d00687d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.982+0000 7f81f2f37700 1 --2- 172.21.15.104:0/774957019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81ec159f20 0x7f81ec0c3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.982+0000 7f81f2f37700 1 --2- 172.21.15.104:0/774957019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81ec0caf70 0x7f81ec155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.982+0000 7f81f2f37700 1 --2- 172.21.15.104:0/774957019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ec0c9d90 0x7f81ec153a80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:13.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.982+0000 7f81f2f37700 1 -- 172.21.15.104:0/774957019 >> 172.21.15.104:0/774957019 conn(0x7f81ec06e960 msgr2=0x7f81ec15b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:13.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.982+0000 7f81f2f37700 1 -- 172.21.15.104:0/774957019 shutdown_connections 2023-07-14T11:55:13.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:13.982+0000 7f81f2f37700 1 -- 172.21.15.104:0/774957019 wait complete. 2023-07-14T11:55:13.971 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:55:13.992 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:55:13.992 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:55:14.008 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:55:14.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.238+0000 7fe910d8b700 1 Processor -- start 2023-07-14T11:55:14.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.238+0000 7fe910d8b700 1 -- start start 2023-07-14T11:55:14.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.238+0000 7fe910d8b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe90c0ce5e0 0x7fe90c158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:14.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.238+0000 7fe910d8b700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90c159d60 con 0x7fe90c0c9820 2023-07-14T11:55:14.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.238+0000 7fe910d8b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90c0c8a60 con 0x7fe90c06dc20 2023-07-14T11:55:14.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.238+0000 7fe910d8b700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90c0cd250 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe90bfff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fe90c0c9820 0x7fe90c159520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35232/0 (socket says 172.21.15.104:35232) 2023-07-14T11:55:14.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe90bfff700 1 -- 172.21.15.104:0/3012882279 learned_addr learned my addr 172.21.15.104:0/3012882279 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:14.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe902ffd700 1 --2- 172.21.15.104:0/3012882279 >> v2:172.21.15.104:3300/0 conn(0x7fe90c0ce5e0 0x7fe90c158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:14.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe902ffd700 1 -- 172.21.15.104:0/3012882279 >> v1:172.21.15.121:6789/0 conn(0x7fe90c06dc20 legacy=0x7fe90c0cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:14.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe902ffd700 1 -- 172.21.15.104:0/3012882279 >> v1:172.21.15.104:6789/0 conn(0x7fe90c0c9820 legacy=0x7fe90c159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:14.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe90affd700 1 -- 172.21.15.104:0/3012882279 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155709116 0 0) 0x7fe90c0c8a60 con 0x7fe90c06dc20 2023-07-14T11:55:14.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe902ffd700 1 -- 172.21.15.104:0/3012882279 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c0c8d40 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe902ffd700 1 --2- 172.21.15.104:0/3012882279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe90c0ce5e0 0x7fe90c158fe0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009850 tx=0x7fe8f4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436632d4c78ebaa8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:14.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe90affd700 1 -- 172.21.15.104:0/3012882279 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4004230 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe90affd700 1 -- 172.21.15.104:0/3012882279 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f4004390 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe90affd700 1 -- 172.21.15.104:0/3012882279 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40036c0 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe910d8b700 1 -- 172.21.15.104:0/3012882279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe90c0ce5e0 msgr2=0x7fe90c158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:14.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe910d8b700 1 --2- 172.21.15.104:0/3012882279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe90c0ce5e0 0x7fe90c158fe0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009850 tx=0x7fe8f4009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:14.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe910d8b700 1 -- 172.21.15.104:0/3012882279 shutdown_connections 2023-07-14T11:55:14.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe910d8b700 1 --2- 172.21.15.104:0/3012882279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe90c0ce5e0 0x7fe90c158fe0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe910d8b700 1 -- 172.21.15.104:0/3012882279 >> 172.21.15.104:0/3012882279 conn(0x7fe90c06e960 msgr2=0x7fe90c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:14.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe910d8b700 1 -- 172.21.15.104:0/3012882279 shutdown_connections 2023-07-14T11:55:14.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.242+0000 7fe910d8b700 1 -- 172.21.15.104:0/3012882279 wait complete. 2023-07-14T11:55:14.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.246+0000 7fe910d8b700 1 Processor -- start 2023-07-14T11:55:14.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.246+0000 7fe910d8b700 1 -- start start 2023-07-14T11:55:14.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe910d8b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe90c06dc20 0x7fe90c0b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:14.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe910d8b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe90c0c9820 0x7fe90c06c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:14.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe910d8b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe90c0ce5e0 0x7fe90c1f0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:14.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe910d8b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c142820 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe910d8b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c142b00 con 0x7fe90c06dc20 2023-07-14T11:55:14.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe910d8b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c142de0 con 0x7fe90c0c9820 2023-07-14T11:55:14.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe90bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe90c0ce5e0 0x7fe90c1f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:14.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe90bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe90c0ce5e0 0x7fe90c1f0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53866/0 (socket says 172.21.15.104:53866) 2023-07-14T11:55:14.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe90bfff700 1 -- 172.21.15.104:0/1917026509 learned_addr learned my addr 172.21.15.104:0/1917026509 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:14.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe90b7fe700 1 --2- 172.21.15.104:0/1917026509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe90c06dc20 0x7fe90c0b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:14.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe902ffd700 1 --2- 172.21.15.104:0/1917026509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe90c0c9820 0x7fe90c06c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:14.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe90bfff700 1 -- 172.21.15.104:0/1917026509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe90c0c9820 msgr2=0x7fe90c06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:14.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe90bfff700 1 --2- 172.21.15.104:0/1917026509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe90c0c9820 0x7fe90c06c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe90bfff700 1 -- 172.21.15.104:0/1917026509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe90c06dc20 msgr2=0x7fe90c0b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:14.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe90bfff700 1 --2- 172.21.15.104:0/1917026509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe90c06dc20 0x7fe90c0b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe90bfff700 1 -- 172.21.15.104:0/1917026509 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f40094b0 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe902ffd700 1 --2- 172.21.15.104:0/1917026509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe90c0c9820 0x7fe90c06c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:14.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe90bfff700 1 --2- 172.21.15.104:0/1917026509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe90c0ce5e0 0x7fe90c1f0600 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe904001fa0 tx=0x7fe90400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:14.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe908ff9700 1 -- 172.21.15.104:0/1917026509 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90400ad40 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe908ff9700 1 -- 172.21.15.104:0/1917026509 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9040020f0 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe910d8b700 1 -- 172.21.15.104:0/1917026509 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90c1430c0 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe908ff9700 1 -- 172.21.15.104:0/1917026509 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90400aea0 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.250+0000 7fe910d8b700 1 -- 172.21.15.104:0/1917026509 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90c06cf50 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.254+0000 7fe910d8b700 1 -- 172.21.15.104:0/1917026509 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90c1435e0 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.254+0000 7fe908ff9700 1 -- 172.21.15.104:0/1917026509 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe904007310 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.254+0000 7fe908ff9700 1 --2- 172.21.15.104:0/1917026509 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8ec0662f0 0x7fe8ec0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:14.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.254+0000 7fe908ff9700 1 -- 172.21.15.104:0/1917026509 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fe904083a00 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.258+0000 7fe90b7fe700 1 --2- 172.21.15.104:0/1917026509 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8ec0662f0 0x7fe8ec0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:14.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.258+0000 7fe90b7fe700 1 --2- 172.21.15.104:0/1917026509 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8ec0662f0 0x7fe8ec0687a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc007fd0 tx=0x7fe8fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:14.249 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.262+0000 7fe908ff9700 1 -- 172.21.15.104:0/1917026509 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe90c1435e0 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.434+0000 7fe910d8b700 1 -- 172.21.15.104:0/1917026509 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe90c1435e0 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.438+0000 7fe908ff9700 1 -- 172.21.15.104:0/1917026509 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+20311 (secure 0 0 0) 0x7fe90c1435e0 con 0x7fe90c0ce5e0 2023-07-14T11:55:14.423 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:14.423 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":45,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:55:10.654581+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:54:56.370313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-14T11:54:57.608947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"data_new","create_time":"2023-07-14T11:55:08.099546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:55:14.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.442+0000 7fe910d8b700 1 -- 172.21.15.104:0/1917026509 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8ec0662f0 msgr2=0x7fe8ec0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:14.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.442+0000 7fe910d8b700 1 --2- 172.21.15.104:0/1917026509 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8ec0662f0 0x7fe8ec0687a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc007fd0 tx=0x7fe8fc009040 comp rx=0 tx=0).stop 2023-07-14T11:55:14.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.442+0000 7fe910d8b700 1 -- 172.21.15.104:0/1917026509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe90c0ce5e0 msgr2=0x7fe90c1f0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:14.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.442+0000 7fe910d8b700 1 --2- 172.21.15.104:0/1917026509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe90c0ce5e0 0x7fe90c1f0600 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe904001fa0 tx=0x7fe90400a340 comp rx=0 tx=0).stop 2023-07-14T11:55:14.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.442+0000 7fe910d8b700 1 -- 172.21.15.104:0/1917026509 shutdown_connections 2023-07-14T11:55:14.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.442+0000 7fe910d8b700 1 --2- 172.21.15.104:0/1917026509 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8ec0662f0 0x7fe8ec0687a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.442+0000 7fe910d8b700 1 --2- 172.21.15.104:0/1917026509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe90c0ce5e0 0x7fe90c1f0600 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.442+0000 7fe910d8b700 1 --2- 172.21.15.104:0/1917026509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe90c0c9820 0x7fe90c06c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.442+0000 7fe910d8b700 1 --2- 172.21.15.104:0/1917026509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe90c06dc20 0x7fe90c0b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.442+0000 7fe910d8b700 1 -- 172.21.15.104:0/1917026509 >> 172.21.15.104:0/1917026509 conn(0x7fe90c06e960 msgr2=0x7fe90c0c9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:14.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.442+0000 7fe910d8b700 1 -- 172.21.15.104:0/1917026509 shutdown_connections 2023-07-14T11:55:14.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.442+0000 7fe910d8b700 1 -- 172.21.15.104:0/1917026509 wait complete. 2023-07-14T11:55:14.451 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:54:56.370313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:54:57.608947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'data_new', 'create_time': '2023-07-14T11:55:08.099546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:55:14.452 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:55:14.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.642+0000 7fb2fd35f700 1 Processor -- start 2023-07-14T11:55:14.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.642+0000 7fb2fd35f700 1 -- start start 2023-07-14T11:55:14.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2fd35f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb2f8159c60 0x7fb2f806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:14.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2fd35f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2f80b0770 0x7fb2f8070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:14.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2fd35f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb2f80cab00 0x7fb2f80caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:14.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2fd35f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f80cb4c0 con 0x7fb2f8159c60 2023-07-14T11:55:14.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2fd35f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f806da00 con 0x7fb2f80cab00 2023-07-14T11:55:14.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2fd35f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f806dce0 con 0x7fb2f80b0770 2023-07-14T11:55:14.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2f7fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb2f8159c60 0x7fb2f806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:14.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2f7fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb2f8159c60 0x7fb2f806d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53878/0 (socket says 172.21.15.104:53878) 2023-07-14T11:55:14.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2f7fff700 1 -- 172.21.15.104:0/1321591100 learned_addr learned my addr 172.21.15.104:0/1321591100 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:14.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2f77fe700 1 --2- 172.21.15.104:0/1321591100 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2f80b0770 0x7fb2f8070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:14.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2fcb5e700 1 --2- 172.21.15.104:0/1321591100 >> v2:172.21.15.121:3300/0 conn(0x7fb2f80cab00 0x7fb2f80caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:14.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2f7fff700 1 -- 172.21.15.104:0/1321591100 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2f80b0770 msgr2=0x7fb2f8070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:14.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2f7fff700 1 --2- 172.21.15.104:0/1321591100 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2f80b0770 0x7fb2f8070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2f7fff700 1 -- 172.21.15.104:0/1321591100 >> v2:172.21.15.121:3300/0 conn(0x7fb2f80cab00 msgr2=0x7fb2f80caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:14.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2f7fff700 1 --2- 172.21.15.104:0/1321591100 >> v2:172.21.15.121:3300/0 conn(0x7fb2f80cab00 0x7fb2f80caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2f7fff700 1 -- 172.21.15.104:0/1321591100 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f806dfc0 con 0x7fb2f8159c60 2023-07-14T11:55:14.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2f7fff700 1 --2- 172.21.15.104:0/1321591100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2f8159c60 0x7fb2f806d3b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80c8730 tx=0x7fb2e8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdf53dc3e7ae53e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:14.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2f6ffd700 1 -- 172.21.15.104:0/1321591100 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8011830 con 0x7fb2f8159c60 2023-07-14T11:55:14.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.646+0000 7fb2f6ffd700 1 -- 172.21.15.104:0/1321591100 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e8011990 con 0x7fb2f8159c60 2023-07-14T11:55:14.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.650+0000 7fb2f6ffd700 1 -- 172.21.15.104:0/1321591100 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8011d20 con 0x7fb2f8159c60 2023-07-14T11:55:14.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.650+0000 7fb2fd35f700 1 -- 172.21.15.104:0/1321591100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2f8159c60 msgr2=0x7fb2f806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:14.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.650+0000 7fb2fd35f700 1 --2- 172.21.15.104:0/1321591100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2f8159c60 0x7fb2f806d3b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80c8730 tx=0x7fb2e8009bf0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.650+0000 7fb2fd35f700 1 -- 172.21.15.104:0/1321591100 shutdown_connections 2023-07-14T11:55:14.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.650+0000 7fb2fd35f700 1 --2- 172.21.15.104:0/1321591100 >> v2:172.21.15.121:3300/0 conn(0x7fb2f80cab00 0x7fb2f80caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.650+0000 7fb2fd35f700 1 --2- 172.21.15.104:0/1321591100 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2f80b0770 0x7fb2f8070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.650+0000 7fb2fd35f700 1 --2- 172.21.15.104:0/1321591100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2f8159c60 0x7fb2f806d3b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.650+0000 7fb2fd35f700 1 -- 172.21.15.104:0/1321591100 >> 172.21.15.104:0/1321591100 conn(0x7fb2f80afc60 msgr2=0x7fb2f80b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:14.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.650+0000 7fb2fd35f700 1 -- 172.21.15.104:0/1321591100 shutdown_connections 2023-07-14T11:55:14.641 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.650+0000 7fb2fd35f700 1 -- 172.21.15.104:0/1321591100 wait complete. 2023-07-14T11:55:14.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.650+0000 7fb2fd35f700 1 Processor -- start 2023-07-14T11:55:14.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2fd35f700 1 -- start start 2023-07-14T11:55:14.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2fd35f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2f80b0770 0x7fb2f80c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:14.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2fd35f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2f80cab00 0x7fb2f8149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:14.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2fd35f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2f8159c60 0x7fb2f814a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:14.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2fd35f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80b1360 con 0x7fb2f80b0770 2023-07-14T11:55:14.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2fd35f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80cdd00 con 0x7fb2f80cab00 2023-07-14T11:55:14.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2fd35f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f814aa00 con 0x7fb2f8159c60 2023-07-14T11:55:14.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2f7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2f80b0770 0x7fb2f80c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:14.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2f7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2f80b0770 0x7fb2f80c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53892/0 (socket says 172.21.15.104:53892) 2023-07-14T11:55:14.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2f7fff700 1 -- 172.21.15.104:0/738737196 learned_addr learned my addr 172.21.15.104:0/738737196 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:14.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2f77fe700 1 --2- 172.21.15.104:0/738737196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2f80cab00 0x7fb2f8149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:14.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2fcb5e700 1 --2- 172.21.15.104:0/738737196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2f8159c60 0x7fb2f814a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:14.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2f7fff700 1 -- 172.21.15.104:0/738737196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2f8159c60 msgr2=0x7fb2f814a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:14.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2f7fff700 1 --2- 172.21.15.104:0/738737196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2f8159c60 0x7fb2f814a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2f7fff700 1 -- 172.21.15.104:0/738737196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2f80cab00 msgr2=0x7fb2f8149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:14.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2f7fff700 1 --2- 172.21.15.104:0/738737196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2f80cab00 0x7fb2f8149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2f7fff700 1 -- 172.21.15.104:0/738737196 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e8009570 con 0x7fb2f80b0770 2023-07-14T11:55:14.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2fcb5e700 1 --2- 172.21.15.104:0/738737196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2f8159c60 0x7fb2f814a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:14.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2f7fff700 1 --2- 172.21.15.104:0/738737196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2f80b0770 0x7fb2f80c9480 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8000c00 tx=0x7fb2e8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:14.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2f4ff9700 1 -- 172.21.15.104:0/738737196 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8004450 con 0x7fb2f80b0770 2023-07-14T11:55:14.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2f4ff9700 1 -- 172.21.15.104:0/738737196 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e80045b0 con 0x7fb2f80b0770 2023-07-14T11:55:14.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2f4ff9700 1 -- 172.21.15.104:0/738737196 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e80038e0 con 0x7fb2f80b0770 2023-07-14T11:55:14.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2fd35f700 1 -- 172.21.15.104:0/738737196 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f80cd4c0 con 0x7fb2f80b0770 2023-07-14T11:55:14.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.654+0000 7fb2fd35f700 1 -- 172.21.15.104:0/738737196 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f80b1590 con 0x7fb2f80b0770 2023-07-14T11:55:14.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.658+0000 7fb2fd35f700 1 -- 172.21.15.104:0/738737196 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f804e500 con 0x7fb2f80b0770 2023-07-14T11:55:14.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.658+0000 7fb2f4ff9700 1 -- 172.21.15.104:0/738737196 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb2e8021070 con 0x7fb2f80b0770 2023-07-14T11:55:14.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.658+0000 7fb2f4ff9700 1 --2- 172.21.15.104:0/738737196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2e40662f0 0x7fb2e40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:14.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.658+0000 7fb2f4ff9700 1 -- 172.21.15.104:0/738737196 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fb2e8085580 con 0x7fb2f80b0770 2023-07-14T11:55:14.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.658+0000 7fb2f77fe700 1 --2- 172.21.15.104:0/738737196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2e40662f0 0x7fb2e40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:14.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.662+0000 7fb2f77fe700 1 --2- 172.21.15.104:0/738737196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2e40662f0 0x7fb2e40687a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80c8700 tx=0x7fb2ec009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:14.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.666+0000 7fb2f4ff9700 1 -- 172.21.15.104:0/738737196 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb2e8052720 con 0x7fb2f80b0770 2023-07-14T11:55:14.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.838+0000 7fb2fd35f700 1 -- 172.21.15.104:0/738737196 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb2f806c380 con 0x7fb2f80b0770 2023-07-14T11:55:14.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.838+0000 7fb2f4ff9700 1 -- 172.21.15.104:0/738737196 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb2e8055d40 con 0x7fb2f80b0770 2023-07-14T11:55:14.824 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:55:14.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.842+0000 7fb2fd35f700 1 -- 172.21.15.104:0/738737196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2e40662f0 msgr2=0x7fb2e40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:14.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.842+0000 7fb2fd35f700 1 --2- 172.21.15.104:0/738737196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2e40662f0 0x7fb2e40687a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80c8700 tx=0x7fb2ec009240 comp rx=0 tx=0).stop 2023-07-14T11:55:14.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.842+0000 7fb2fd35f700 1 -- 172.21.15.104:0/738737196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2f80b0770 msgr2=0x7fb2f80c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:14.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.842+0000 7fb2fd35f700 1 --2- 172.21.15.104:0/738737196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2f80b0770 0x7fb2f80c9480 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8000c00 tx=0x7fb2e8004770 comp rx=0 tx=0).stop 2023-07-14T11:55:14.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.842+0000 7fb2fd35f700 1 -- 172.21.15.104:0/738737196 shutdown_connections 2023-07-14T11:55:14.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.842+0000 7fb2fd35f700 1 --2- 172.21.15.104:0/738737196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2e40662f0 0x7fb2e40687a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.842+0000 7fb2fd35f700 1 --2- 172.21.15.104:0/738737196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2f8159c60 0x7fb2f814a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.842+0000 7fb2fd35f700 1 --2- 172.21.15.104:0/738737196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2f80cab00 0x7fb2f8149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.842+0000 7fb2fd35f700 1 --2- 172.21.15.104:0/738737196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2f80b0770 0x7fb2f80c9480 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:14.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.842+0000 7fb2fd35f700 1 -- 172.21.15.104:0/738737196 >> 172.21.15.104:0/738737196 conn(0x7fb2f80afc60 msgr2=0x7fb2f80adbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:14.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.842+0000 7fb2fd35f700 1 -- 172.21.15.104:0/738737196 shutdown_connections 2023-07-14T11:55:14.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:14.842+0000 7fb2fd35f700 1 -- 172.21.15.104:0/738737196 wait complete. 2023-07-14T11:55:14.852 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:55:15.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.046+0000 7fcd961dc700 1 Processor -- start 2023-07-14T11:55:15.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.046+0000 7fcd961dc700 1 -- start start 2023-07-14T11:55:15.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd961dc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcd900b0ee0 0x7fcd900ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd961dc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fcd900d07e0 0x7fcd900af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd961dc700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd900cd4e0 con 0x7fcd900cd090 2023-07-14T11:55:15.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd961dc700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd900d0c60 con 0x7fcd900d07e0 2023-07-14T11:55:15.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd961dc700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd900d1a60 con 0x7fcd900b0ee0 2023-07-14T11:55:15.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd949d9700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fcd900cd090 0x7fcd900d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35244/0 (socket says 172.21.15.104:35244) 2023-07-14T11:55:15.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd949d9700 1 -- 172.21.15.104:0/1416117237 learned_addr learned my addr 172.21.15.104:0/1416117237 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:15.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd959db700 1 --2- 172.21.15.104:0/1416117237 >> v2:172.21.15.104:3300/0 conn(0x7fcd900d07e0 0x7fcd900af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd951da700 1 --2- 172.21.15.104:0/1416117237 >> v2:172.21.15.121:3300/0 conn(0x7fcd900b0ee0 0x7fcd900ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd959db700 1 -- 172.21.15.104:0/1416117237 >> v2:172.21.15.121:3300/0 conn(0x7fcd900b0ee0 msgr2=0x7fcd900ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:15.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd959db700 1 --2- 172.21.15.104:0/1416117237 >> v2:172.21.15.121:3300/0 conn(0x7fcd900b0ee0 0x7fcd900ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd959db700 1 -- 172.21.15.104:0/1416117237 >> v1:172.21.15.104:6789/0 conn(0x7fcd900cd090 legacy=0x7fcd900d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd959db700 1 -- 172.21.15.104:0/1416117237 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd900d1ef0 con 0x7fcd900d07e0 2023-07-14T11:55:15.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd959db700 1 --2- 172.21.15.104:0/1416117237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd900d07e0 0x7fcd900af0c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcd9008b380 tx=0x7fcd8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66da607e7a8251d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:15.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd87fff700 1 -- 172.21.15.104:0/1416117237 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8800e070 con 0x7fcd900d07e0 2023-07-14T11:55:15.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd87fff700 1 -- 172.21.15.104:0/1416117237 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8800bb20 con 0x7fcd900d07e0 2023-07-14T11:55:15.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.050+0000 7fcd87fff700 1 -- 172.21.15.104:0/1416117237 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88012430 con 0x7fcd900d07e0 2023-07-14T11:55:15.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.054+0000 7fcd961dc700 1 -- 172.21.15.104:0/1416117237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd900d07e0 msgr2=0x7fcd900af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.054+0000 7fcd961dc700 1 --2- 172.21.15.104:0/1416117237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd900d07e0 0x7fcd900af0c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcd9008b380 tx=0x7fcd8800b330 comp rx=0 tx=0).stop 2023-07-14T11:55:15.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.054+0000 7fcd961dc700 1 -- 172.21.15.104:0/1416117237 shutdown_connections 2023-07-14T11:55:15.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.054+0000 7fcd961dc700 1 --2- 172.21.15.104:0/1416117237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd900d07e0 0x7fcd900af0c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.054+0000 7fcd961dc700 1 --2- 172.21.15.104:0/1416117237 >> v2:172.21.15.121:3300/0 conn(0x7fcd900b0ee0 0x7fcd900ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.054+0000 7fcd961dc700 1 -- 172.21.15.104:0/1416117237 >> 172.21.15.104:0/1416117237 conn(0x7fcd90060c40 msgr2=0x7fcd9009b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:15.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.054+0000 7fcd961dc700 1 -- 172.21.15.104:0/1416117237 shutdown_connections 2023-07-14T11:55:15.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.054+0000 7fcd961dc700 1 -- 172.21.15.104:0/1416117237 wait complete. 2023-07-14T11:55:15.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd961dc700 1 Processor -- start 2023-07-14T11:55:15.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd961dc700 1 -- start start 2023-07-14T11:55:15.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd961dc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd900b0ee0 0x7fcd900b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd961dc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd900cd090 0x7fcd901586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd961dc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd900d07e0 0x7fcd900cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd961dc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd900b49a0 con 0x7fcd900d07e0 2023-07-14T11:55:15.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd961dc700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd900c8c60 con 0x7fcd900b0ee0 2023-07-14T11:55:15.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd961dc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd900c7ea0 con 0x7fcd900cd090 2023-07-14T11:55:15.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd959db700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd900d07e0 0x7fcd900cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd959db700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd900d07e0 0x7fcd900cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53910/0 (socket says 172.21.15.104:53910) 2023-07-14T11:55:15.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd959db700 1 -- 172.21.15.104:0/1182942911 learned_addr learned my addr 172.21.15.104:0/1182942911 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:15.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd951da700 1 --2- 172.21.15.104:0/1182942911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd900b0ee0 0x7fcd900b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd949d9700 1 --2- 172.21.15.104:0/1182942911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd900cd090 0x7fcd901586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd959db700 1 -- 172.21.15.104:0/1182942911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd900cd090 msgr2=0x7fcd901586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd959db700 1 --2- 172.21.15.104:0/1182942911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd900cd090 0x7fcd901586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd959db700 1 -- 172.21.15.104:0/1182942911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd900b0ee0 msgr2=0x7fcd900b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.051 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd959db700 1 --2- 172.21.15.104:0/1182942911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd900b0ee0 0x7fcd900b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.058+0000 7fcd959db700 1 -- 172.21.15.104:0/1182942911 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd88009a70 con 0x7fcd900d07e0 2023-07-14T11:55:15.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.062+0000 7fcd959db700 1 --2- 172.21.15.104:0/1182942911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd900d07e0 0x7fcd900cb5a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fcd88014040 tx=0x7fcd88004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:15.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.062+0000 7fcd85ffb700 1 -- 172.21.15.104:0/1182942911 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88012430 con 0x7fcd900d07e0 2023-07-14T11:55:15.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.062+0000 7fcd85ffb700 1 -- 172.21.15.104:0/1182942911 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd88020070 con 0x7fcd900d07e0 2023-07-14T11:55:15.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.062+0000 7fcd961dc700 1 -- 172.21.15.104:0/1182942911 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd900c8180 con 0x7fcd900d07e0 2023-07-14T11:55:15.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.062+0000 7fcd85ffb700 1 -- 172.21.15.104:0/1182942911 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd880098f0 con 0x7fcd900d07e0 2023-07-14T11:55:15.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.062+0000 7fcd961dc700 1 -- 172.21.15.104:0/1182942911 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd900c8740 con 0x7fcd900d07e0 2023-07-14T11:55:15.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.062+0000 7fcd961dc700 1 -- 172.21.15.104:0/1182942911 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9004e500 con 0x7fcd900d07e0 2023-07-14T11:55:15.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.066+0000 7fcd85ffb700 1 -- 172.21.15.104:0/1182942911 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fcd88004040 con 0x7fcd900d07e0 2023-07-14T11:55:15.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.066+0000 7fcd85ffb700 1 --2- 172.21.15.104:0/1182942911 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcd740662f0 0x7fcd740687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.066+0000 7fcd85ffb700 1 -- 172.21.15.104:0/1182942911 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fcd88022070 con 0x7fcd900d07e0 2023-07-14T11:55:15.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.066+0000 7fcd951da700 1 --2- 172.21.15.104:0/1182942911 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcd740662f0 0x7fcd740687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.066+0000 7fcd951da700 1 --2- 172.21.15.104:0/1182942911 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcd740662f0 0x7fcd740687a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009b50 tx=0x7fcd8c009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:15.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.070+0000 7fcd85ffb700 1 -- 172.21.15.104:0/1182942911 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fcd88052dd0 con 0x7fcd900d07e0 2023-07-14T11:55:15.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.242+0000 7fcd961dc700 1 -- 172.21.15.104:0/1182942911 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcd900c8900 con 0x7fcd900d07e0 2023-07-14T11:55:15.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.242+0000 7fcd85ffb700 1 -- 172.21.15.104:0/1182942911 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcd8802c0a0 con 0x7fcd900d07e0 2023-07-14T11:55:15.228 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:55:15.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.246+0000 7fcd961dc700 1 -- 172.21.15.104:0/1182942911 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcd740662f0 msgr2=0x7fcd740687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.246+0000 7fcd961dc700 1 --2- 172.21.15.104:0/1182942911 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcd740662f0 0x7fcd740687a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009b50 tx=0x7fcd8c009240 comp rx=0 tx=0).stop 2023-07-14T11:55:15.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.246+0000 7fcd961dc700 1 -- 172.21.15.104:0/1182942911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd900d07e0 msgr2=0x7fcd900cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.246+0000 7fcd961dc700 1 --2- 172.21.15.104:0/1182942911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd900d07e0 0x7fcd900cb5a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fcd88014040 tx=0x7fcd88004af0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.246+0000 7fcd961dc700 1 -- 172.21.15.104:0/1182942911 shutdown_connections 2023-07-14T11:55:15.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.246+0000 7fcd961dc700 1 --2- 172.21.15.104:0/1182942911 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcd740662f0 0x7fcd740687a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.246+0000 7fcd961dc700 1 --2- 172.21.15.104:0/1182942911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd900d07e0 0x7fcd900cb5a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.246+0000 7fcd961dc700 1 --2- 172.21.15.104:0/1182942911 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd900cd090 0x7fcd901586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.246+0000 7fcd961dc700 1 --2- 172.21.15.104:0/1182942911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd900b0ee0 0x7fcd900b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.246+0000 7fcd961dc700 1 -- 172.21.15.104:0/1182942911 >> 172.21.15.104:0/1182942911 conn(0x7fcd90060c40 msgr2=0x7fcd900b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:15.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.246+0000 7fcd961dc700 1 -- 172.21.15.104:0/1182942911 shutdown_connections 2023-07-14T11:55:15.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.246+0000 7fcd961dc700 1 -- 172.21.15.104:0/1182942911 wait complete. 2023-07-14T11:55:15.250 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:55:15.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.438+0000 7fab65ba0700 1 Processor -- start 2023-07-14T11:55:15.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.438+0000 7fab65ba0700 1 -- start start 2023-07-14T11:55:15.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.438+0000 7fab65ba0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab600b1b80 0x7fab600b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.438+0000 7fab65ba0700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fab600ce050 0x7fab600ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab65ba0700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab600ce980 con 0x7fab600caf20 2023-07-14T11:55:15.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab65ba0700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab600d1410 con 0x7fab600ce050 2023-07-14T11:55:15.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab65ba0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab600b5710 con 0x7fab600b1b80 2023-07-14T11:55:15.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab5f7fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab600b1b80 0x7fab600b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab5f7fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab600b1b80 0x7fab600b51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:46556/0 (socket says 172.21.15.104:46556) 2023-07-14T11:55:15.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab5f7fe700 1 -- 172.21.15.104:0/2656301868 learned_addr learned my addr 172.21.15.104:0/2656301868 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:15.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab5effd700 1 --2- 172.21.15.104:0/2656301868 >> v2:172.21.15.121:3300/0 conn(0x7fab600ce050 0x7fab600ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab5effd700 1 -- 172.21.15.104:0/2656301868 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab600b1b80 msgr2=0x7fab600b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:15.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab5effd700 1 --2- 172.21.15.104:0/2656301868 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab600b1b80 0x7fab600b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab5effd700 1 -- 172.21.15.104:0/2656301868 >> v1:172.21.15.121:6789/0 conn(0x7fab600caf20 legacy=0x7fab600cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab5effd700 1 -- 172.21.15.104:0/2656301868 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab600b5910 con 0x7fab600ce050 2023-07-14T11:55:15.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab5f7fe700 1 --2- 172.21.15.104:0/2656301868 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab600b1b80 0x7fab600b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:55:15.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab5effd700 1 --2- 172.21.15.104:0/2656301868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600ce050 0x7fab600ce440 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fab54009850 tx=0x7fab54009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9040a772c86364c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:15.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab5e7fc700 1 -- 172.21.15.104:0/2656301868 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54011830 con 0x7fab600ce050 2023-07-14T11:55:15.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab5e7fc700 1 -- 172.21.15.104:0/2656301868 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab54011990 con 0x7fab600ce050 2023-07-14T11:55:15.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.442+0000 7fab5e7fc700 1 -- 172.21.15.104:0/2656301868 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54011d40 con 0x7fab600ce050 2023-07-14T11:55:15.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 -- 172.21.15.104:0/2656301868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600ce050 msgr2=0x7fab600ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 --2- 172.21.15.104:0/2656301868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600ce050 0x7fab600ce440 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fab54009850 tx=0x7fab54009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:15.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 -- 172.21.15.104:0/2656301868 shutdown_connections 2023-07-14T11:55:15.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 --2- 172.21.15.104:0/2656301868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600ce050 0x7fab600ce440 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 --2- 172.21.15.104:0/2656301868 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab600b1b80 0x7fab600b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 -- 172.21.15.104:0/2656301868 >> 172.21.15.104:0/2656301868 conn(0x7fab6008a000 msgr2=0x7fab600d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:15.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 -- 172.21.15.104:0/2656301868 shutdown_connections 2023-07-14T11:55:15.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 -- 172.21.15.104:0/2656301868 wait complete. 2023-07-14T11:55:15.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 Processor -- start 2023-07-14T11:55:15.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 -- start start 2023-07-14T11:55:15.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600b1b80 0x7fab60159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab600caf20 0x7fab6015ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab600ce050 0x7fab600cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6015a2f0 con 0x7fab600ce050 2023-07-14T11:55:15.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab600c5390 con 0x7fab600b1b80 2023-07-14T11:55:15.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.446+0000 7fab65ba0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab600c5670 con 0x7fab600caf20 2023-07-14T11:55:15.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab5f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600b1b80 0x7fab60159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab5ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab600ce050 0x7fab600cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab5f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600b1b80 0x7fab60159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:39584/0 (socket says 172.21.15.104:39584) 2023-07-14T11:55:15.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab5f7fe700 1 -- 172.21.15.104:0/1048838023 learned_addr learned my addr 172.21.15.104:0/1048838023 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:15.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab5effd700 1 --2- 172.21.15.104:0/1048838023 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab600caf20 0x7fab6015ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab5f7fe700 1 -- 172.21.15.104:0/1048838023 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab600caf20 msgr2=0x7fab6015ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab5f7fe700 1 --2- 172.21.15.104:0/1048838023 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab600caf20 0x7fab6015ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab5f7fe700 1 -- 172.21.15.104:0/1048838023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab600ce050 msgr2=0x7fab600cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab5f7fe700 1 --2- 172.21.15.104:0/1048838023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab600ce050 0x7fab600cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab5f7fe700 1 -- 172.21.15.104:0/1048838023 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab50009520 con 0x7fab600b1b80 2023-07-14T11:55:15.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab5f7fe700 1 --2- 172.21.15.104:0/1048838023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600b1b80 0x7fab60159990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fab5000e9d0 tx=0x7fab5000ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:15.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab47fff700 1 -- 172.21.15.104:0/1048838023 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000ee70 con 0x7fab600b1b80 2023-07-14T11:55:15.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab47fff700 1 -- 172.21.15.104:0/1048838023 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab50005a40 con 0x7fab600b1b80 2023-07-14T11:55:15.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab65ba0700 1 -- 172.21.15.104:0/1048838023 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab540094b0 con 0x7fab600b1b80 2023-07-14T11:55:15.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab47fff700 1 -- 172.21.15.104:0/1048838023 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50005db0 con 0x7fab600b1b80 2023-07-14T11:55:15.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab65ba0700 1 -- 172.21.15.104:0/1048838023 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6015a820 con 0x7fab600b1b80 2023-07-14T11:55:15.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.450+0000 7fab65ba0700 1 -- 172.21.15.104:0/1048838023 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6015a9b0 con 0x7fab600b1b80 2023-07-14T11:55:15.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.454+0000 7fab47fff700 1 -- 172.21.15.104:0/1048838023 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fab5000c850 con 0x7fab600b1b80 2023-07-14T11:55:15.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.454+0000 7fab47fff700 1 --2- 172.21.15.104:0/1048838023 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fab4c066320 0x7fab4c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.454+0000 7fab47fff700 1 -- 172.21.15.104:0/1048838023 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fab50083e80 con 0x7fab600b1b80 2023-07-14T11:55:15.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.454+0000 7fab5effd700 1 --2- 172.21.15.104:0/1048838023 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fab4c066320 0x7fab4c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.454+0000 7fab5effd700 1 --2- 172.21.15.104:0/1048838023 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fab4c066320 0x7fab4c0687d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fab540090b0 tx=0x7fab54008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:15.449 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.462+0000 7fab47fff700 1 -- 172.21.15.104:0/1048838023 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fab50050f90 con 0x7fab600b1b80 2023-07-14T11:55:15.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.634+0000 7fab65ba0700 1 -- 172.21.15.104:0/1048838023 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab6004dcb0 con 0x7fab600b1b80 2023-07-14T11:55:15.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.634+0000 7fab47fff700 1 -- 172.21.15.104:0/1048838023 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab500545b0 con 0x7fab600b1b80 2023-07-14T11:55:15.622 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:55:15.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.638+0000 7fab65ba0700 1 -- 172.21.15.104:0/1048838023 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fab4c066320 msgr2=0x7fab4c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.638+0000 7fab65ba0700 1 --2- 172.21.15.104:0/1048838023 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fab4c066320 0x7fab4c0687d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fab540090b0 tx=0x7fab54008030 comp rx=0 tx=0).stop 2023-07-14T11:55:15.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.638+0000 7fab65ba0700 1 -- 172.21.15.104:0/1048838023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600b1b80 msgr2=0x7fab60159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.638+0000 7fab65ba0700 1 --2- 172.21.15.104:0/1048838023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600b1b80 0x7fab60159990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fab5000e9d0 tx=0x7fab5000ece0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.638+0000 7fab65ba0700 1 -- 172.21.15.104:0/1048838023 shutdown_connections 2023-07-14T11:55:15.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.638+0000 7fab65ba0700 1 --2- 172.21.15.104:0/1048838023 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fab4c066320 0x7fab4c0687d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.638+0000 7fab65ba0700 1 --2- 172.21.15.104:0/1048838023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab600ce050 0x7fab600cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.638+0000 7fab65ba0700 1 --2- 172.21.15.104:0/1048838023 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab600caf20 0x7fab6015ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.638+0000 7fab65ba0700 1 --2- 172.21.15.104:0/1048838023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600b1b80 0x7fab60159990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.638+0000 7fab65ba0700 1 -- 172.21.15.104:0/1048838023 >> 172.21.15.104:0/1048838023 conn(0x7fab6008a000 msgr2=0x7fab6006fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:15.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.638+0000 7fab65ba0700 1 -- 172.21.15.104:0/1048838023 shutdown_connections 2023-07-14T11:55:15.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.638+0000 7fab65ba0700 1 -- 172.21.15.104:0/1048838023 wait complete. 2023-07-14T11:55:15.645 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:55:15.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.838+0000 7fb905a25700 1 Processor -- start 2023-07-14T11:55:15.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.838+0000 7fb905a25700 1 -- start start 2023-07-14T11:55:15.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb905a25700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb9000cb460 0x7fb90006bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb905a25700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9000cbdc0 con 0x7fb9000705e0 2023-07-14T11:55:15.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb905a25700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9000cc0a0 con 0x7fb900158360 2023-07-14T11:55:15.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb905a25700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9000cc380 con 0x7fb9000cb460 2023-07-14T11:55:15.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb8feffd700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb9000705e0 0x7fb9000709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35250/0 (socket says 172.21.15.104:35250) 2023-07-14T11:55:15.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb8feffd700 1 -- 172.21.15.104:0/3771406842 learned_addr learned my addr 172.21.15.104:0/3771406842 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:15.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb8ff7fe700 1 --2- 172.21.15.104:0/3771406842 >> v2:172.21.15.121:3300/0 conn(0x7fb9000cb460 0x7fb90006bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb8fdffb700 1 -- 172.21.15.104:0/3771406842 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911073590 0 0) 0x7fb9000cbdc0 con 0x7fb9000705e0 2023-07-14T11:55:15.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb8fdffb700 1 -- 172.21.15.104:0/3771406842 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8e80035d0 con 0x7fb9000705e0 2023-07-14T11:55:15.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb8fdffb700 1 -- 172.21.15.104:0/3771406842 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4221453238 0 0) 0x7fb8e80035d0 con 0x7fb9000705e0 2023-07-14T11:55:15.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb8ff7fe700 1 -- 172.21.15.104:0/3771406842 >> v1:172.21.15.121:6789/0 conn(0x7fb900158360 legacy=0x7fb900071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb8ff7fe700 1 -- 172.21.15.104:0/3771406842 >> v1:172.21.15.104:6789/0 conn(0x7fb9000705e0 legacy=0x7fb9000709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb8ff7fe700 1 -- 172.21.15.104:0/3771406842 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9000cc660 con 0x7fb9000cb460 2023-07-14T11:55:15.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb8ff7fe700 1 --2- 172.21.15.104:0/3771406842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9000cb460 0x7fb90006bdd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800aea0 tx=0x7fb8f800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30bae49338fe00c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:15.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb8fdffb700 1 -- 172.21.15.104:0/3771406842 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800f070 con 0x7fb9000cb460 2023-07-14T11:55:15.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb8fdffb700 1 -- 172.21.15.104:0/3771406842 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f8003a40 con 0x7fb9000cb460 2023-07-14T11:55:15.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.842+0000 7fb8fdffb700 1 -- 172.21.15.104:0/3771406842 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0001670 con 0x7fb9000cb460 2023-07-14T11:55:15.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.846+0000 7fb905a25700 1 -- 172.21.15.104:0/3771406842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9000cb460 msgr2=0x7fb90006bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.846+0000 7fb905a25700 1 --2- 172.21.15.104:0/3771406842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9000cb460 0x7fb90006bdd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800aea0 tx=0x7fb8f800c330 comp rx=0 tx=0).stop 2023-07-14T11:55:15.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.846+0000 7fb905a25700 1 -- 172.21.15.104:0/3771406842 shutdown_connections 2023-07-14T11:55:15.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.846+0000 7fb905a25700 1 --2- 172.21.15.104:0/3771406842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9000cb460 0x7fb90006bdd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.846+0000 7fb905a25700 1 -- 172.21.15.104:0/3771406842 >> 172.21.15.104:0/3771406842 conn(0x7fb9000afc60 msgr2=0x7fb9000b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:15.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.846+0000 7fb905a25700 1 -- 172.21.15.104:0/3771406842 shutdown_connections 2023-07-14T11:55:15.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.846+0000 7fb905a25700 1 -- 172.21.15.104:0/3771406842 wait complete. 2023-07-14T11:55:15.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.846+0000 7fb905a25700 1 Processor -- start 2023-07-14T11:55:15.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb905a25700 1 -- start start 2023-07-14T11:55:15.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb905a25700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9000705e0 0x7fb900149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb905a25700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9000cb460 0x7fb9000af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb905a25700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb900158360 0x7fb9000adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb905a25700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9000c6750 con 0x7fb900158360 2023-07-14T11:55:15.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb905a25700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb900149b50 con 0x7fb9000cb460 2023-07-14T11:55:15.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb905a25700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9000c56b0 con 0x7fb9000705e0 2023-07-14T11:55:15.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb8ff7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb900158360 0x7fb9000adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb8ff7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb900158360 0x7fb9000adbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53936/0 (socket says 172.21.15.104:53936) 2023-07-14T11:55:15.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb8ff7fe700 1 -- 172.21.15.104:0/512442059 learned_addr learned my addr 172.21.15.104:0/512442059 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:15.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb8feffd700 1 --2- 172.21.15.104:0/512442059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9000705e0 0x7fb900149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb8fe7fc700 1 --2- 172.21.15.104:0/512442059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9000cb460 0x7fb9000af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb8ff7fe700 1 -- 172.21.15.104:0/512442059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9000705e0 msgr2=0x7fb900149610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb8ff7fe700 1 --2- 172.21.15.104:0/512442059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9000705e0 0x7fb900149610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb8ff7fe700 1 -- 172.21.15.104:0/512442059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9000cb460 msgr2=0x7fb9000af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:15.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb8ff7fe700 1 --2- 172.21.15.104:0/512442059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9000cb460 0x7fb9000af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:15.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb8ff7fe700 1 -- 172.21.15.104:0/512442059 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f800ab80 con 0x7fb900158360 2023-07-14T11:55:15.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.850+0000 7fb8ff7fe700 1 --2- 172.21.15.104:0/512442059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb900158360 0x7fb9000adbd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800c370 tx=0x7fb8f8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:15.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.854+0000 7fb8e7fff700 1 -- 172.21.15.104:0/512442059 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8004240 con 0x7fb900158360 2023-07-14T11:55:15.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.854+0000 7fb8e7fff700 1 -- 172.21.15.104:0/512442059 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f80043a0 con 0x7fb900158360 2023-07-14T11:55:15.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.854+0000 7fb905a25700 1 -- 172.21.15.104:0/512442059 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9000c5990 con 0x7fb900158360 2023-07-14T11:55:15.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.854+0000 7fb8e7fff700 1 -- 172.21.15.104:0/512442059 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8004730 con 0x7fb900158360 2023-07-14T11:55:15.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.854+0000 7fb905a25700 1 -- 172.21.15.104:0/512442059 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9000c5f50 con 0x7fb900158360 2023-07-14T11:55:15.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.854+0000 7fb905a25700 1 -- 172.21.15.104:0/512442059 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb90004e500 con 0x7fb900158360 2023-07-14T11:55:15.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.858+0000 7fb8e7fff700 1 -- 172.21.15.104:0/512442059 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb8f801d450 con 0x7fb900158360 2023-07-14T11:55:15.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.858+0000 7fb8e7fff700 1 --2- 172.21.15.104:0/512442059 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb8ec0662f0 0x7fb8ec0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:15.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.858+0000 7fb8e7fff700 1 -- 172.21.15.104:0/512442059 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fb8f80859a0 con 0x7fb900158360 2023-07-14T11:55:15.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.858+0000 7fb8feffd700 1 --2- 172.21.15.104:0/512442059 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb8ec0662f0 0x7fb8ec0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:15.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.858+0000 7fb8feffd700 1 --2- 172.21.15.104:0/512442059 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb8ec0662f0 0x7fb8ec0687a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0000f60 tx=0x7fb8f0001e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:15.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:15.866+0000 7fb8e7fff700 1 -- 172.21.15.104:0/512442059 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb8f8052b40 con 0x7fb900158360 2023-07-14T11:55:16.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.042+0000 7fb905a25700 1 -- 172.21.15.104:0/512442059 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb9000cbae0 con 0x7fb900158360 2023-07-14T11:55:16.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.046+0000 7fb8e7fff700 1 -- 172.21.15.104:0/512442059 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb8f8056160 con 0x7fb900158360 2023-07-14T11:55:16.031 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:55:16.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.046+0000 7fb905a25700 1 -- 172.21.15.104:0/512442059 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb8ec0662f0 msgr2=0x7fb8ec0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:16.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.046+0000 7fb905a25700 1 --2- 172.21.15.104:0/512442059 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb8ec0662f0 0x7fb8ec0687a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0000f60 tx=0x7fb8f0001e40 comp rx=0 tx=0).stop 2023-07-14T11:55:16.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.046+0000 7fb905a25700 1 -- 172.21.15.104:0/512442059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb900158360 msgr2=0x7fb9000adbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:16.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.046+0000 7fb905a25700 1 --2- 172.21.15.104:0/512442059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb900158360 0x7fb9000adbd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800c370 tx=0x7fb8f8004af0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.046+0000 7fb905a25700 1 -- 172.21.15.104:0/512442059 shutdown_connections 2023-07-14T11:55:16.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.046+0000 7fb905a25700 1 --2- 172.21.15.104:0/512442059 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb8ec0662f0 0x7fb8ec0687a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.046+0000 7fb905a25700 1 --2- 172.21.15.104:0/512442059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb900158360 0x7fb9000adbd0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.046+0000 7fb905a25700 1 --2- 172.21.15.104:0/512442059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9000cb460 0x7fb9000af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.046+0000 7fb905a25700 1 --2- 172.21.15.104:0/512442059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb9000705e0 0x7fb900149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.046+0000 7fb905a25700 1 -- 172.21.15.104:0/512442059 >> 172.21.15.104:0/512442059 conn(0x7fb9000afc60 msgr2=0x7fb900157730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:16.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.046+0000 7fb905a25700 1 -- 172.21.15.104:0/512442059 shutdown_connections 2023-07-14T11:55:16.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.046+0000 7fb905a25700 1 -- 172.21.15.104:0/512442059 wait complete. 2023-07-14T11:55:16.054 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-07-14T11:55:16.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.246+0000 7f84d5c67700 1 Processor -- start 2023-07-14T11:55:16.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.246+0000 7f84d5c67700 1 -- start start 2023-07-14T11:55:16.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.250+0000 7f84d5c67700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d015c410 0x7f84d00c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:16.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.250+0000 7f84d5c67700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f84d00c8d90 0x7f84d00afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:16.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.250+0000 7f84d5c67700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d006d860 con 0x7f84d006c700 2023-07-14T11:55:16.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.250+0000 7f84d5c67700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d00ca710 con 0x7f84d00c8d90 2023-07-14T11:55:16.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.250+0000 7f84d5c67700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d006dcf0 con 0x7f84d015c410 2023-07-14T11:55:16.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.250+0000 7f84ceffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f84d00c8d90 0x7f84d00afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:16.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.250+0000 7f84cf7fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d015c410 0x7f84d00c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:16.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.250+0000 7f84ceffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f84d00c8d90 0x7f84d00afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:39618/0 (socket says 172.21.15.104:39618) 2023-07-14T11:55:16.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.250+0000 7f84ceffd700 1 -- 172.21.15.104:0/1332570011 learned_addr learned my addr 172.21.15.104:0/1332570011 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:16.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.250+0000 7f84ceffd700 1 -- 172.21.15.104:0/1332570011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d015c410 msgr2=0x7f84d00c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:16.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.250+0000 7f84ceffd700 1 --2- 172.21.15.104:0/1332570011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d015c410 0x7f84d00c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.250+0000 7f84ceffd700 1 -- 172.21.15.104:0/1332570011 >> v1:172.21.15.121:6789/0 conn(0x7f84d006c700 legacy=0x7f84d015c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:16.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.250+0000 7f84ceffd700 1 -- 172.21.15.104:0/1332570011 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d006cc50 con 0x7f84d00c8d90 2023-07-14T11:55:16.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.250+0000 7f84ceffd700 1 --2- 172.21.15.104:0/1332570011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d00c8d90 0x7f84d00afd80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009910 tx=0x7f84c4009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46916eaf51aeda91 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:16.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.254+0000 7f84ce7fc700 1 -- 172.21.15.104:0/1332570011 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4011830 con 0x7f84d00c8d90 2023-07-14T11:55:16.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.254+0000 7f84ce7fc700 1 -- 172.21.15.104:0/1332570011 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c4011990 con 0x7f84d00c8d90 2023-07-14T11:55:16.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.254+0000 7f84ce7fc700 1 -- 172.21.15.104:0/1332570011 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4011d40 con 0x7f84d00c8d90 2023-07-14T11:55:16.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.254+0000 7f84d5c67700 1 -- 172.21.15.104:0/1332570011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d00c8d90 msgr2=0x7f84d00afd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:16.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.254+0000 7f84d5c67700 1 --2- 172.21.15.104:0/1332570011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d00c8d90 0x7f84d00afd80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009910 tx=0x7f84c4009c20 comp rx=0 tx=0).stop 2023-07-14T11:55:16.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.254+0000 7f84d5c67700 1 -- 172.21.15.104:0/1332570011 shutdown_connections 2023-07-14T11:55:16.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.254+0000 7f84d5c67700 1 --2- 172.21.15.104:0/1332570011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d00c8d90 0x7f84d00afd80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.254+0000 7f84d5c67700 1 --2- 172.21.15.104:0/1332570011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d015c410 0x7f84d00c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.254+0000 7f84d5c67700 1 -- 172.21.15.104:0/1332570011 >> 172.21.15.104:0/1332570011 conn(0x7f84d00cad80 msgr2=0x7f84d00cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:16.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.254+0000 7f84d5c67700 1 -- 172.21.15.104:0/1332570011 shutdown_connections 2023-07-14T11:55:16.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.254+0000 7f84d5c67700 1 -- 172.21.15.104:0/1332570011 wait complete. 2023-07-14T11:55:16.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84d5c67700 1 Processor -- start 2023-07-14T11:55:16.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84d5c67700 1 -- start start 2023-07-14T11:55:16.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84d5c67700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d006c700 0x7f84d0153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:16.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84d5c67700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d00c8d90 0x7f84d00c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:16.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84d5c67700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d015c410 0x7f84d00ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:16.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84d5c67700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00abe60 con 0x7f84d015c410 2023-07-14T11:55:16.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84d5c67700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00ac140 con 0x7f84d006c700 2023-07-14T11:55:16.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84d5c67700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00ac420 con 0x7f84d00c8d90 2023-07-14T11:55:16.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84cffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d015c410 0x7f84d00ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:16.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84cf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d006c700 0x7f84d0153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:16.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84cffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d015c410 0x7f84d00ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53940/0 (socket says 172.21.15.104:53940) 2023-07-14T11:55:16.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84cffff700 1 -- 172.21.15.104:0/2046639432 learned_addr learned my addr 172.21.15.104:0/2046639432 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:16.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84ceffd700 1 --2- 172.21.15.104:0/2046639432 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d00c8d90 0x7f84d00c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:16.247 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84cf7fe700 1 -- 172.21.15.104:0/2046639432 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d00c8d90 msgr2=0x7f84d00c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:16.247 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84cf7fe700 1 --2- 172.21.15.104:0/2046639432 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d00c8d90 0x7f84d00c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.247 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84cf7fe700 1 -- 172.21.15.104:0/2046639432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d015c410 msgr2=0x7f84d00ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:16.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84cf7fe700 1 --2- 172.21.15.104:0/2046639432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d015c410 0x7f84d00ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84cf7fe700 1 -- 172.21.15.104:0/2046639432 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c4009570 con 0x7f84d006c700 2023-07-14T11:55:16.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.258+0000 7f84cf7fe700 1 --2- 172.21.15.104:0/2046639432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d006c700 0x7f84d0153b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f84c000e870 tx=0x7f84c000eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:16.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.262+0000 7f84b7fff700 1 -- 172.21.15.104:0/2046639432 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0013680 con 0x7f84d006c700 2023-07-14T11:55:16.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.262+0000 7f84b7fff700 1 -- 172.21.15.104:0/2046639432 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c00137e0 con 0x7f84d006c700 2023-07-14T11:55:16.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.262+0000 7f84d5c67700 1 -- 172.21.15.104:0/2046639432 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d00ac700 con 0x7f84d006c700 2023-07-14T11:55:16.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.262+0000 7f84b7fff700 1 -- 172.21.15.104:0/2046639432 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0013b70 con 0x7f84d006c700 2023-07-14T11:55:16.249 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.262+0000 7f84d5c67700 1 -- 172.21.15.104:0/2046639432 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d00ab8f0 con 0x7f84d006c700 2023-07-14T11:55:16.249 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.262+0000 7f84d5c67700 1 -- 172.21.15.104:0/2046639432 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d004dcb0 con 0x7f84d006c700 2023-07-14T11:55:16.250 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.262+0000 7f84b7fff700 1 -- 172.21.15.104:0/2046639432 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f84c003b080 con 0x7f84d006c700 2023-07-14T11:55:16.252 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.266+0000 7f84b7fff700 1 --2- 172.21.15.104:0/2046639432 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f84bc0662f0 0x7f84bc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:16.252 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.266+0000 7f84b7fff700 1 -- 172.21.15.104:0/2046639432 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f84c00840e0 con 0x7f84d006c700 2023-07-14T11:55:16.253 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.266+0000 7f84ceffd700 1 --2- 172.21.15.104:0/2046639432 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f84bc0662f0 0x7f84bc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:16.253 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.266+0000 7f84ceffd700 1 --2- 172.21.15.104:0/2046639432 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f84bc0662f0 0x7f84bc0687a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f84c4000c00 tx=0x7f84c4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:16.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.270+0000 7f84b7fff700 1 -- 172.21.15.104:0/2046639432 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f84c0054d00 con 0x7f84d006c700 2023-07-14T11:55:16.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.442+0000 7f84d5c67700 1 -- 172.21.15.104:0/2046639432 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f84d00ac280 con 0x7f84d006c700 2023-07-14T11:55:16.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.442+0000 7f84b7fff700 1 -- 172.21.15.104:0/2046639432 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v45) v1 ==== 97+0+11 (secure 0 0 0) 0x7f84d00ac280 con 0x7f84d006c700 2023-07-14T11:55:16.429 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:55:16.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.446+0000 7f84d5c67700 1 -- 172.21.15.104:0/2046639432 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f84bc0662f0 msgr2=0x7f84bc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:16.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.446+0000 7f84d5c67700 1 --2- 172.21.15.104:0/2046639432 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f84bc0662f0 0x7f84bc0687a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f84c4000c00 tx=0x7f84c4008030 comp rx=0 tx=0).stop 2023-07-14T11:55:16.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.446+0000 7f84d5c67700 1 -- 172.21.15.104:0/2046639432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d006c700 msgr2=0x7f84d0153b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:16.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.446+0000 7f84d5c67700 1 --2- 172.21.15.104:0/2046639432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d006c700 0x7f84d0153b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f84c000e870 tx=0x7f84c000eb80 comp rx=0 tx=0).stop 2023-07-14T11:55:16.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.446+0000 7f84d5c67700 1 -- 172.21.15.104:0/2046639432 shutdown_connections 2023-07-14T11:55:16.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.446+0000 7f84d5c67700 1 --2- 172.21.15.104:0/2046639432 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f84bc0662f0 0x7f84bc0687a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.446+0000 7f84d5c67700 1 --2- 172.21.15.104:0/2046639432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d015c410 0x7f84d00ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.446+0000 7f84d5c67700 1 --2- 172.21.15.104:0/2046639432 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84d00c8d90 0x7f84d00c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.446+0000 7f84d5c67700 1 --2- 172.21.15.104:0/2046639432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84d006c700 0x7f84d0153b80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.446+0000 7f84d5c67700 1 -- 172.21.15.104:0/2046639432 >> 172.21.15.104:0/2046639432 conn(0x7f84d00cad80 msgr2=0x7f84d00ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:16.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.446+0000 7f84d5c67700 1 -- 172.21.15.104:0/2046639432 shutdown_connections 2023-07-14T11:55:16.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.446+0000 7f84d5c67700 1 -- 172.21.15.104:0/2046639432 wait complete. 2023-07-14T11:55:16.455 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:55:16.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.646+0000 7fa3d2cdd700 1 Processor -- start 2023-07-14T11:55:16.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.646+0000 7fa3d2cdd700 1 -- start start 2023-07-14T11:55:16.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.646+0000 7fa3d2cdd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa3cc06d860 0x7fa3cc0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:16.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.646+0000 7fa3d2cdd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc06be70 0x7fa3cc06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:16.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.646+0000 7fa3d2cdd700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3cc0cc1b0 con 0x7fa3cc0c1e00 2023-07-14T11:55:16.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.646+0000 7fa3d2cdd700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3cc06e170 con 0x7fa3cc06d860 2023-07-14T11:55:16.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.646+0000 7fa3d2cdd700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc0cd690 con 0x7fa3cc06be70 2023-07-14T11:55:16.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3d0a79700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa3cc06d860 0x7fa3cc0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:16.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3d127a700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fa3cc0c1e00 0x7fa3cc0cc640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35254/0 (socket says 172.21.15.104:35254) 2023-07-14T11:55:16.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3d127a700 1 -- 172.21.15.104:0/2179460186 learned_addr learned my addr 172.21.15.104:0/2179460186 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:16.636 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3cbfff700 1 --2- 172.21.15.104:0/2179460186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc06be70 0x7fa3cc06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:16.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3cb7fe700 1 -- 172.21.15.104:0/2179460186 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130321965 0 0) 0x7fa3cc0cc1b0 con 0x7fa3cc0c1e00 2023-07-14T11:55:16.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3cbfff700 1 -- 172.21.15.104:0/2179460186 >> v2:172.21.15.104:3300/0 conn(0x7fa3cc06d860 msgr2=0x7fa3cc0cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:16.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3cbfff700 1 --2- 172.21.15.104:0/2179460186 >> v2:172.21.15.104:3300/0 conn(0x7fa3cc06d860 0x7fa3cc0cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3cbfff700 1 -- 172.21.15.104:0/2179460186 >> v1:172.21.15.104:6789/0 conn(0x7fa3cc0c1e00 legacy=0x7fa3cc0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:16.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3cbfff700 1 -- 172.21.15.104:0/2179460186 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc0cd970 con 0x7fa3cc06be70 2023-07-14T11:55:16.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3cbfff700 1 --2- 172.21.15.104:0/2179460186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc06be70 0x7fa3cc06dc30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009880 tx=0x7fa3b8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd50789a66fdc222 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:16.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3cb7fe700 1 -- 172.21.15.104:0/2179460186 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8011830 con 0x7fa3cc06be70 2023-07-14T11:55:16.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3cb7fe700 1 -- 172.21.15.104:0/2179460186 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b8011990 con 0x7fa3cc06be70 2023-07-14T11:55:16.638 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3cb7fe700 1 -- 172.21.15.104:0/2179460186 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8011d20 con 0x7fa3cc06be70 2023-07-14T11:55:16.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3d2cdd700 1 -- 172.21.15.104:0/2179460186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc06be70 msgr2=0x7fa3cc06dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:16.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3d2cdd700 1 --2- 172.21.15.104:0/2179460186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc06be70 0x7fa3cc06dc30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009880 tx=0x7fa3b8009b90 comp rx=0 tx=0).stop 2023-07-14T11:55:16.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3d2cdd700 1 -- 172.21.15.104:0/2179460186 shutdown_connections 2023-07-14T11:55:16.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3d2cdd700 1 --2- 172.21.15.104:0/2179460186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc06be70 0x7fa3cc06dc30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3d2cdd700 1 --2- 172.21.15.104:0/2179460186 >> v2:172.21.15.104:3300/0 conn(0x7fa3cc06d860 0x7fa3cc0cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3d2cdd700 1 -- 172.21.15.104:0/2179460186 >> 172.21.15.104:0/2179460186 conn(0x7fa3cc0cad30 msgr2=0x7fa3cc0cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:16.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3d2cdd700 1 -- 172.21.15.104:0/2179460186 shutdown_connections 2023-07-14T11:55:16.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.650+0000 7fa3d2cdd700 1 -- 172.21.15.104:0/2179460186 wait complete. 2023-07-14T11:55:16.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.654+0000 7fa3d2cdd700 1 Processor -- start 2023-07-14T11:55:16.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.654+0000 7fa3d2cdd700 1 -- start start 2023-07-14T11:55:16.641 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.654+0000 7fa3d2cdd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc06be70 0x7fa3cc1f01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:16.642 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.654+0000 7fa3d2cdd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3cc06d860 0x7fa3cc158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:16.642 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.654+0000 7fa3d2cdd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc0c1e00 0x7fa3cc1591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:16.642 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.654+0000 7fa3d2cdd700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc15a4e0 con 0x7fa3cc06d860 2023-07-14T11:55:16.642 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.654+0000 7fa3d2cdd700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc15a7c0 con 0x7fa3cc06be70 2023-07-14T11:55:16.642 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.654+0000 7fa3d2cdd700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc15aaa0 con 0x7fa3cc0c1e00 2023-07-14T11:55:16.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.654+0000 7fa3cbfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3cc06d860 0x7fa3cc158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:16.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.654+0000 7fa3cbfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3cc06d860 0x7fa3cc158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53970/0 (socket says 172.21.15.104:53970) 2023-07-14T11:55:16.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.654+0000 7fa3cbfff700 1 -- 172.21.15.104:0/3837469702 learned_addr learned my addr 172.21.15.104:0/3837469702 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:16.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.654+0000 7fa3d0a79700 1 --2- 172.21.15.104:0/3837469702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc06be70 0x7fa3cc1f01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:16.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.654+0000 7fa3d127a700 1 --2- 172.21.15.104:0/3837469702 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc0c1e00 0x7fa3cc1591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:16.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.658+0000 7fa3cbfff700 1 -- 172.21.15.104:0/3837469702 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc0c1e00 msgr2=0x7fa3cc1591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:16.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.658+0000 7fa3cbfff700 1 --2- 172.21.15.104:0/3837469702 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc0c1e00 0x7fa3cc1591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.658+0000 7fa3cbfff700 1 -- 172.21.15.104:0/3837469702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc06be70 msgr2=0x7fa3cc1f01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:16.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.658+0000 7fa3cbfff700 1 --2- 172.21.15.104:0/3837469702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc06be70 0x7fa3cc1f01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.658+0000 7fa3cbfff700 1 -- 172.21.15.104:0/3837469702 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b80094b0 con 0x7fa3cc06d860 2023-07-14T11:55:16.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.658+0000 7fa3d127a700 1 --2- 172.21.15.104:0/3837469702 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc0c1e00 0x7fa3cc1591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:16.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.658+0000 7fa3cbfff700 1 --2- 172.21.15.104:0/3837469702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3cc06d860 0x7fa3cc158c60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80090b0 tx=0x7fa3b8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:16.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.658+0000 7fa3c97fa700 1 -- 172.21.15.104:0/3837469702 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8004380 con 0x7fa3cc06d860 2023-07-14T11:55:16.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.658+0000 7fa3c97fa700 1 -- 172.21.15.104:0/3837469702 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b801f070 con 0x7fa3cc06d860 2023-07-14T11:55:16.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.658+0000 7fa3d2cdd700 1 -- 172.21.15.104:0/3837469702 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc15ad80 con 0x7fa3cc06d860 2023-07-14T11:55:16.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.658+0000 7fa3c97fa700 1 -- 172.21.15.104:0/3837469702 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80036c0 con 0x7fa3cc06d860 2023-07-14T11:55:16.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.658+0000 7fa3d2cdd700 1 -- 172.21.15.104:0/3837469702 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc15b580 con 0x7fa3cc06d860 2023-07-14T11:55:16.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.658+0000 7fa3d2cdd700 1 -- 172.21.15.104:0/3837469702 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3cc15b2a0 con 0x7fa3cc06d860 2023-07-14T11:55:16.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.662+0000 7fa3c97fa700 1 -- 172.21.15.104:0/3837469702 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa3b8021030 con 0x7fa3cc06d860 2023-07-14T11:55:16.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.662+0000 7fa3c97fa700 1 --2- 172.21.15.104:0/3837469702 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3b00662f0 0x7fa3b00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:16.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.662+0000 7fa3c97fa700 1 -- 172.21.15.104:0/3837469702 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fa3b8085280 con 0x7fa3cc06d860 2023-07-14T11:55:16.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.662+0000 7fa3d0a79700 1 --2- 172.21.15.104:0/3837469702 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3b00662f0 0x7fa3b00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:16.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.662+0000 7fa3d0a79700 1 --2- 172.21.15.104:0/3837469702 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3b00662f0 0x7fa3b00687a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009bf0 tx=0x7fa3c0009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:16.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.666+0000 7fa3c97fa700 1 -- 172.21.15.104:0/3837469702 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fa3cc15b2a0 con 0x7fa3cc06d860 2023-07-14T11:55:16.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.846+0000 7fa3d2cdd700 1 -- 172.21.15.104:0/3837469702 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa3cc15b2a0 con 0x7fa3cc06d860 2023-07-14T11:55:16.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.850+0000 7fa3c97fa700 1 -- 172.21.15.104:0/3837469702 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa3cc15b2a0 con 0x7fa3cc06d860 2023-07-14T11:55:16.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.850+0000 7fa3d2cdd700 1 -- 172.21.15.104:0/3837469702 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3b00662f0 msgr2=0x7fa3b00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:16.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.850+0000 7fa3d2cdd700 1 --2- 172.21.15.104:0/3837469702 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3b00662f0 0x7fa3b00687a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009bf0 tx=0x7fa3c0009240 comp rx=0 tx=0).stop 2023-07-14T11:55:16.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.850+0000 7fa3d2cdd700 1 -- 172.21.15.104:0/3837469702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3cc06d860 msgr2=0x7fa3cc158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:16.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.850+0000 7fa3d2cdd700 1 --2- 172.21.15.104:0/3837469702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3cc06d860 0x7fa3cc158c60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80090b0 tx=0x7fa3b8004770 comp rx=0 tx=0).stop 2023-07-14T11:55:16.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.850+0000 7fa3d2cdd700 1 -- 172.21.15.104:0/3837469702 shutdown_connections 2023-07-14T11:55:16.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.850+0000 7fa3d2cdd700 1 --2- 172.21.15.104:0/3837469702 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3b00662f0 0x7fa3b00687a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.850+0000 7fa3d2cdd700 1 --2- 172.21.15.104:0/3837469702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3cc06d860 0x7fa3cc158c60 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.850+0000 7fa3d2cdd700 1 --2- 172.21.15.104:0/3837469702 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc0c1e00 0x7fa3cc1591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.850+0000 7fa3d2cdd700 1 --2- 172.21.15.104:0/3837469702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc06be70 0x7fa3cc1f01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:16.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.850+0000 7fa3d2cdd700 1 -- 172.21.15.104:0/3837469702 >> 172.21.15.104:0/3837469702 conn(0x7fa3cc0cad30 msgr2=0x7fa3cc157410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:16.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.850+0000 7fa3d2cdd700 1 -- 172.21.15.104:0/3837469702 shutdown_connections 2023-07-14T11:55:16.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:16.850+0000 7fa3d2cdd700 1 -- 172.21.15.104:0/3837469702 wait complete. 2023-07-14T11:55:16.841 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:55:16.859 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi104.front.sepia.ceph.com... 2023-07-14T11:55:16.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:16.860 DEBUG:teuthology.orchestra.run.smithi104:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:16.887 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[47028]: fuse finished with error 0 and tester_r 0 2023-07-14T11:55:16.891 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:55:22.993 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:55:22.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:22.994 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-14T11:55:23.016 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-14T11:55:23.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.254+0000 7fcdee6f2700 1 Processor -- start 2023-07-14T11:55:23.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.254+0000 7fcdee6f2700 1 -- start start 2023-07-14T11:55:23.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcdee6f2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcde80cb340 0x7fcde80cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:23.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcdee6f2700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde80cfc90 con 0x7fcde814f920 2023-07-14T11:55:23.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcdee6f2700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde80b3310 con 0x7fcde815c4f0 2023-07-14T11:55:23.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcdee6f2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80b29e0 con 0x7fcde80cb340 2023-07-14T11:55:23.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcded6f0700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fcde814f920 0x7fcde80cf1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:39358/0 (socket says 172.21.15.104:39358) 2023-07-14T11:55:23.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcded6f0700 1 -- 172.21.15.104:0/1807508685 learned_addr learned my addr 172.21.15.104:0/1807508685 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:23.244 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcdeceef700 1 --2- 172.21.15.104:0/1807508685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcde80cb340 0x7fcde80cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:23.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcddffff700 1 -- 172.21.15.104:0/1807508685 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384816059 0 0) 0x7fcde80cfc90 con 0x7fcde814f920 2023-07-14T11:55:23.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcddffff700 1 -- 172.21.15.104:0/1807508685 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdd40057e0 con 0x7fcde814f920 2023-07-14T11:55:23.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcddffff700 1 -- 172.21.15.104:0/1807508685 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1113920016 0 0) 0x7fcde80b3310 con 0x7fcde815c4f0 2023-07-14T11:55:23.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcddffff700 1 -- 172.21.15.104:0/1807508685 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde80cfc90 con 0x7fcde815c4f0 2023-07-14T11:55:23.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcdeceef700 1 -- 172.21.15.104:0/1807508685 >> v1:172.21.15.121:6789/0 conn(0x7fcde815c4f0 legacy=0x7fcde815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:23.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcdeceef700 1 -- 172.21.15.104:0/1807508685 >> v1:172.21.15.104:6789/0 conn(0x7fcde814f920 legacy=0x7fcde80cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:23.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcdeceef700 1 -- 172.21.15.104:0/1807508685 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde80b2cc0 con 0x7fcde80cb340 2023-07-14T11:55:23.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcdeceef700 1 --2- 172.21.15.104:0/1807508685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcde80cb340 0x7fcde80cb790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009880 tx=0x7fcdd8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99b104f002b4a691 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:23.247 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcddffff700 1 -- 172.21.15.104:0/1807508685 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8011830 con 0x7fcde80cb340 2023-07-14T11:55:23.247 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcddffff700 1 -- 172.21.15.104:0/1807508685 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd8011990 con 0x7fcde80cb340 2023-07-14T11:55:23.247 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.258+0000 7fcddffff700 1 -- 172.21.15.104:0/1807508685 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8011d20 con 0x7fcde80cb340 2023-07-14T11:55:23.247 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.262+0000 7fcdee6f2700 1 -- 172.21.15.104:0/1807508685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcde80cb340 msgr2=0x7fcde80cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:23.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.262+0000 7fcdee6f2700 1 --2- 172.21.15.104:0/1807508685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcde80cb340 0x7fcde80cb790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009880 tx=0x7fcdd8009b90 comp rx=0 tx=0).stop 2023-07-14T11:55:23.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.262+0000 7fcdee6f2700 1 -- 172.21.15.104:0/1807508685 shutdown_connections 2023-07-14T11:55:23.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.262+0000 7fcdee6f2700 1 --2- 172.21.15.104:0/1807508685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcde80cb340 0x7fcde80cb790 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:23.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.262+0000 7fcdee6f2700 1 -- 172.21.15.104:0/1807508685 >> 172.21.15.104:0/1807508685 conn(0x7fcde806fa70 msgr2=0x7fcde806fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:23.249 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.262+0000 7fcdee6f2700 1 -- 172.21.15.104:0/1807508685 shutdown_connections 2023-07-14T11:55:23.249 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.262+0000 7fcdee6f2700 1 -- 172.21.15.104:0/1807508685 wait complete. 2023-07-14T11:55:23.252 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.266+0000 7fcdee6f2700 1 Processor -- start 2023-07-14T11:55:23.252 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.266+0000 7fcdee6f2700 1 -- start start 2023-07-14T11:55:23.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.266+0000 7fcdee6f2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcde80cb340 0x7fcde814ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:23.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.266+0000 7fcdee6f2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcde814f920 0x7fcde80cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:23.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.266+0000 7fcdee6f2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcde815c4f0 0x7fcde80cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:23.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.266+0000 7fcdee6f2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde814b0c0 con 0x7fcde80cb340 2023-07-14T11:55:23.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.266+0000 7fcdee6f2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde8150ab0 con 0x7fcde814f920 2023-07-14T11:55:23.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.266+0000 7fcdee6f2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde8150d70 con 0x7fcde815c4f0 2023-07-14T11:55:23.255 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.266+0000 7fcded6f0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcde80cb340 0x7fcde814ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:23.255 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.266+0000 7fcded6f0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcde80cb340 0x7fcde814ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41264/0 (socket says 172.21.15.104:41264) 2023-07-14T11:55:23.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.266+0000 7fcded6f0700 1 -- 172.21.15.104:0/1597067712 learned_addr learned my addr 172.21.15.104:0/1597067712 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:23.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.270+0000 7fcdeceef700 1 --2- 172.21.15.104:0/1597067712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcde814f920 0x7fcde80cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:23.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.270+0000 7fcdedef1700 1 --2- 172.21.15.104:0/1597067712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcde815c4f0 0x7fcde80cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:23.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.270+0000 7fcded6f0700 1 -- 172.21.15.104:0/1597067712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcde815c4f0 msgr2=0x7fcde80cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:23.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.270+0000 7fcded6f0700 1 --2- 172.21.15.104:0/1597067712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcde815c4f0 0x7fcde80cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:23.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.270+0000 7fcded6f0700 1 -- 172.21.15.104:0/1597067712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcde814f920 msgr2=0x7fcde80cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:23.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.270+0000 7fcded6f0700 1 --2- 172.21.15.104:0/1597067712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcde814f920 0x7fcde80cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:23.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.270+0000 7fcded6f0700 1 -- 172.21.15.104:0/1597067712 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd80094b0 con 0x7fcde80cb340 2023-07-14T11:55:23.257 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.270+0000 7fcded6f0700 1 --2- 172.21.15.104:0/1597067712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcde80cb340 0x7fcde814ab80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcde400cbc0 tx=0x7fcde400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:23.257 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.270+0000 7fcdddffb700 1 -- 172.21.15.104:0/1597067712 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde400a770 con 0x7fcde80cb340 2023-07-14T11:55:23.257 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.270+0000 7fcdddffb700 1 -- 172.21.15.104:0/1597067712 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde400a8d0 con 0x7fcde80cb340 2023-07-14T11:55:23.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.270+0000 7fcdddffb700 1 -- 172.21.15.104:0/1597067712 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde400ac40 con 0x7fcde80cb340 2023-07-14T11:55:23.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.270+0000 7fcdee6f2700 1 -- 172.21.15.104:0/1597067712 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde8151050 con 0x7fcde80cb340 2023-07-14T11:55:23.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.270+0000 7fcdee6f2700 1 -- 172.21.15.104:0/1597067712 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde80cd770 con 0x7fcde80cb340 2023-07-14T11:55:23.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.270+0000 7fcdee6f2700 1 -- 172.21.15.104:0/1597067712 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde804e500 con 0x7fcde80cb340 2023-07-14T11:55:23.260 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.274+0000 7fcdddffb700 1 -- 172.21.15.104:0/1597067712 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fcde401b3b0 con 0x7fcde80cb340 2023-07-14T11:55:23.262 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.274+0000 7fcdddffb700 1 --2- 172.21.15.104:0/1597067712 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcdcc066340 0x7fcdcc0687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:23.262 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.274+0000 7fcdddffb700 1 -- 172.21.15.104:0/1597067712 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fcde40839d0 con 0x7fcde80cb340 2023-07-14T11:55:23.263 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.278+0000 7fcdeceef700 1 --2- 172.21.15.104:0/1597067712 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcdcc066340 0x7fcdcc0687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:23.264 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.278+0000 7fcdeceef700 1 --2- 172.21.15.104:0/1597067712 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcdcc066340 0x7fcdcc0687f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80090b0 tx=0x7fcdd801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:23.268 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.282+0000 7fcdddffb700 1 -- 172.21.15.104:0/1597067712 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fcde4050bf0 con 0x7fcde80cb340 2023-07-14T11:55:23.475 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.490+0000 7fcdee6f2700 1 -- 172.21.15.104:0/1597067712 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcde8069cc0 con 0x7fcde80cb340 2023-07-14T11:55:23.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.490+0000 7fcdddffb700 1 -- 172.21.15.104:0/1597067712 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+82 (secure 0 0 0) 0x7fcde4054210 con 0x7fcde80cb340 2023-07-14T11:55:23.477 INFO:teuthology.orchestra.run.smithi104.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2023-07-14T11:55:23.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.494+0000 7fcdee6f2700 1 -- 172.21.15.104:0/1597067712 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcdcc066340 msgr2=0x7fcdcc0687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:23.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.494+0000 7fcdee6f2700 1 --2- 172.21.15.104:0/1597067712 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcdcc066340 0x7fcdcc0687f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80090b0 tx=0x7fcdd801c040 comp rx=0 tx=0).stop 2023-07-14T11:55:23.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.494+0000 7fcdee6f2700 1 -- 172.21.15.104:0/1597067712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcde80cb340 msgr2=0x7fcde814ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:23.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.494+0000 7fcdee6f2700 1 --2- 172.21.15.104:0/1597067712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcde80cb340 0x7fcde814ab80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcde400cbc0 tx=0x7fcde400cbf0 comp rx=0 tx=0).stop 2023-07-14T11:55:23.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.494+0000 7fcdee6f2700 1 -- 172.21.15.104:0/1597067712 shutdown_connections 2023-07-14T11:55:23.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.494+0000 7fcdee6f2700 1 --2- 172.21.15.104:0/1597067712 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcdcc066340 0x7fcdcc0687f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:23.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.494+0000 7fcdee6f2700 1 --2- 172.21.15.104:0/1597067712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcde815c4f0 0x7fcde80cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:23.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.494+0000 7fcdee6f2700 1 --2- 172.21.15.104:0/1597067712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcde814f920 0x7fcde80cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:23.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.494+0000 7fcdee6f2700 1 --2- 172.21.15.104:0/1597067712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcde80cb340 0x7fcde814ab80 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:23.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.494+0000 7fcdee6f2700 1 -- 172.21.15.104:0/1597067712 >> 172.21.15.104:0/1597067712 conn(0x7fcde806fa70 msgr2=0x7fcde815c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:23.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.494+0000 7fcdee6f2700 1 -- 172.21.15.104:0/1597067712 shutdown_connections 2023-07-14T11:55:23.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:23.494+0000 7fcdee6f2700 1 -- 172.21.15.104:0/1597067712 wait complete. 2023-07-14T11:55:23.500 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-14T11:55:23.501 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T11:55:23.501 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2023-07-14T11:55:23.501 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-14T11:55:23.501 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:23.502 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-14T11:55:23.502 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T11:55:23.502 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T11:55:23.502 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:55:23.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:23.503 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2023-07-14T11:55:23.521 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:55:23.521 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:55:23.522 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:55:23.522 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:23.522 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2023-07-14T11:55:23.522 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:23.522 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:55:23.522 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:23.522 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2023-07-14T11:55:23.523 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:55:23.523 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:23.523 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2023-07-14T11:55:23.523 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:55:23.523 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:23.523 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T11:55:23.523 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:23.523 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-07-14T11:55:23.524 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42272sec preferred_lft 42272sec 2023-07-14T11:55:23.524 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2023-07-14T11:55:23.524 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:23.524 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-07-14T11:55:23.524 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:a2:a1:53:16 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:23.524 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-07-14T11:55:23.524 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:23.525 INFO:teuthology.orchestra.run.smithi104.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-14T11:55:23.525 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:b8:27:80:0d:ab brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:23.525 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-14T11:55:23.525 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:23.525 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5020:39ff:fe23:761d/64 scope link 2023-07-14T11:55:23.525 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:23.525 INFO:teuthology.orchestra.run.smithi104.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-14T11:55:23.525 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:b8:27:80:0d:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-14T11:55:23.526 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::e8b8:27ff:fe80:dab/64 scope link 2023-07-14T11:55:23.526 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:23.526 INFO:teuthology.orchestra.run.smithi104.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-14T11:55:23.526 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:e6:23:3b:72:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-14T11:55:23.526 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::e8e6:23ff:fe3b:7247/64 scope link 2023-07-14T11:55:23.526 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:23.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:23.527 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T11:55:23.609 INFO:teuthology.orchestra.run.smithi104.stdout:1 2023-07-14T11:55:23.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:23.613 DEBUG:teuthology.orchestra.run.smithi104:> route 2023-07-14T11:55:23.673 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2023-07-14T11:55:23.673 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T11:55:23.674 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-07-14T11:55:23.674 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T11:55:23.674 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-07-14T11:55:23.674 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-07-14T11:55:23.675 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-07-14T11:55:23.675 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T11:55:23.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:23.676 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:55:23.676 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:55:23.676 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T11:55:23.677 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T11:55:23.677 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T11:55:23.677 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:55:23.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:23.812 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2023-07-14T11:55:23.829 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-14T11:55:23.830 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-14T11:55:23.830 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:55:23.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:23.831 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2023-07-14T11:55:23.909 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-14T11:55:23.909 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:55:23.910 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:23.969 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:55:23.970 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:55:23.970 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:24.032 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:55:24.117 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:55:24.118 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:55:24.192 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:55:24.195 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:55:24.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:24.196 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:55:24.257 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-14T11:55:24.257 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-14T11:55:24.302 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:55:24.336 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:55:24.337 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:55:24.413 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:55:24.416 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:55:24.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:24.419 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:55:24.447 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:2023-07-14T11:55:24.458+0000 7f6b66bcd640 -1 init, newargv = 0x564826ab6e40 newargc=15 2023-07-14T11:55:24.447 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[47662]: starting ceph client 2023-07-14T11:55:24.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[47662]: starting fuse 2023-07-14T11:55:24.462 INFO:teuthology.orchestra.run.smithi104.stdout:49 2023-07-14T11:55:24.463 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-07-14T11:55:24.463 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T11:55:24.464 DEBUG:teuthology.orchestra.run.smithi104:> import glob 2023-07-14T11:55:24.465 DEBUG:teuthology.orchestra.run.smithi104:> import re 2023-07-14T11:55:24.466 DEBUG:teuthology.orchestra.run.smithi104:> import os 2023-07-14T11:55:24.466 DEBUG:teuthology.orchestra.run.smithi104:> import subprocess 2023-07-14T11:55:24.466 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:55:24.467 DEBUG:teuthology.orchestra.run.smithi104:> def _find_admin_socket(client_name): 2023-07-14T11:55:24.467 DEBUG:teuthology.orchestra.run.smithi104:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-14T11:55:24.467 DEBUG:teuthology.orchestra.run.smithi104:> files = glob.glob(asok_path) 2023-07-14T11:55:24.468 DEBUG:teuthology.orchestra.run.smithi104:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-14T11:55:24.468 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:55:24.468 DEBUG:teuthology.orchestra.run.smithi104:> # Given a non-glob path, it better be there 2023-07-14T11:55:24.469 DEBUG:teuthology.orchestra.run.smithi104:> if "*" not in asok_path: 2023-07-14T11:55:24.469 DEBUG:teuthology.orchestra.run.smithi104:> assert(len(files) == 1) 2023-07-14T11:55:24.469 DEBUG:teuthology.orchestra.run.smithi104:> return files[0] 2023-07-14T11:55:24.470 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:55:24.470 DEBUG:teuthology.orchestra.run.smithi104:> for f in files: 2023-07-14T11:55:24.470 DEBUG:teuthology.orchestra.run.smithi104:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-14T11:55:24.471 DEBUG:teuthology.orchestra.run.smithi104:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-14T11:55:24.471 DEBUG:teuthology.orchestra.run.smithi104:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-14T11:55:24.471 DEBUG:teuthology.orchestra.run.smithi104:> contents = proc_f.read() 2023-07-14T11:55:24.471 DEBUG:teuthology.orchestra.run.smithi104:> if mountpoint in contents: 2023-07-14T11:55:24.472 DEBUG:teuthology.orchestra.run.smithi104:> return f 2023-07-14T11:55:24.473 DEBUG:teuthology.orchestra.run.smithi104:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-14T11:55:24.473 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:55:24.473 DEBUG:teuthology.orchestra.run.smithi104:> print(_find_admin_socket("client.0")) 2023-07-14T11:55:24.474 DEBUG:teuthology.orchestra.run.smithi104:> ' 2023-07-14T11:55:24.619 INFO:teuthology.orchestra.run.smithi104.stdout:/var/run/ceph/ceph-client.0.47662.asok 2023-07-14T11:55:24.805 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47662.asok 2023-07-14T11:55:24.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:24.806 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47662.asok status 2023-07-14T11:55:24.999 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:55:25.000 INFO:teuthology.orchestra.run.smithi104.stdout: "metadata": { 2023-07-14T11:55:25.000 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-14T11:55:25.000 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:55:25.000 INFO:teuthology.orchestra.run.smithi104.stdout: "entity_id": "0", 2023-07-14T11:55:25.000 INFO:teuthology.orchestra.run.smithi104.stdout: "hostname": "smithi104", 2023-07-14T11:55:25.000 INFO:teuthology.orchestra.run.smithi104.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-14T11:55:25.000 INFO:teuthology.orchestra.run.smithi104.stdout: "pid": "47662", 2023-07-14T11:55:25.001 INFO:teuthology.orchestra.run.smithi104.stdout: "root": "/" 2023-07-14T11:55:25.001 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:55:25.001 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_count": 0, 2023-07-14T11:55:25.001 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_pinned_count": 0, 2023-07-14T11:55:25.001 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 5940, 2023-07-14T11:55:25.001 INFO:teuthology.orchestra.run.smithi104.stdout: "inst": { 2023-07-14T11:55:25.001 INFO:teuthology.orchestra.run.smithi104.stdout: "name": { 2023-07-14T11:55:25.001 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "client", 2023-07-14T11:55:25.002 INFO:teuthology.orchestra.run.smithi104.stdout: "num": 5940 2023-07-14T11:55:25.002 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:55:25.002 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:55:25.002 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:55:25.002 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:55:25.002 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 4060938759 2023-07-14T11:55:25.002 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:55:25.003 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:55:25.003 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:55:25.003 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:55:25.003 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:55:25.003 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 4060938759 2023-07-14T11:55:25.003 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:55:25.003 INFO:teuthology.orchestra.run.smithi104.stdout: "inst_str": "client.5940 192.168.0.1:0/4060938759", 2023-07-14T11:55:25.003 INFO:teuthology.orchestra.run.smithi104.stdout: "addr_str": "192.168.0.1:0/4060938759", 2023-07-14T11:55:25.004 INFO:teuthology.orchestra.run.smithi104.stdout: "inode_count": 1, 2023-07-14T11:55:25.004 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_epoch": 19, 2023-07-14T11:55:25.004 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch": 45, 2023-07-14T11:55:25.004 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch_barrier": 0, 2023-07-14T11:55:25.004 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklisted": false, 2023-07-14T11:55:25.005 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs" 2023-07-14T11:55:25.005 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:55:25.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:25.015 DEBUG:teuthology.orchestra.run.smithi104:> stat --file-system '--printf=%T 2023-07-14T11:55:25.016 DEBUG:teuthology.orchestra.run.smithi104:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:25.037 INFO:teuthology.orchestra.run.smithi104.stdout:fuseblk 2023-07-14T11:55:25.038 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:25.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:25.039 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:25.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:25.111 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2023-07-14T11:55:25.179 INFO:teuthology.orchestra.run.smithi104.stderr:setfattr: layoutfile: Permission denied 2023-07-14T11:55:25.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:55:25.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:25.182 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2023-07-14T11:55:25.246 INFO:teuthology.orchestra.run.smithi104.stderr:setfattr: layoutdir: Permission denied 2023-07-14T11:55:25.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:55:25.248 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:55:25.248 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:55:25.307 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:55:25.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.546+0000 7fe8a28bc700 1 Processor -- start 2023-07-14T11:55:25.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.546+0000 7fe8a28bc700 1 -- start start 2023-07-14T11:55:25.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.550+0000 7fe8a28bc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe89c0cddb0 0x7fe89c070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:25.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.550+0000 7fe8a28bc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe89c070ea0 0x7fe89c0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:25.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.550+0000 7fe8a28bc700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89c0c8900 con 0x7fe89c160640 2023-07-14T11:55:25.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.550+0000 7fe8a28bc700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89c0c8fc0 con 0x7fe89c0cddb0 2023-07-14T11:55:25.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.550+0000 7fe8a28bc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c070540 con 0x7fe89c070ea0 2023-07-14T11:55:25.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.550+0000 7fe8937fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe89c0cddb0 0x7fe89c070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:25.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.550+0000 7fe8a0e59700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe89c070ea0 0x7fe89c0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:25.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.550+0000 7fe8937fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe89c0cddb0 0x7fe89c070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:49556/0 (socket says 172.21.15.104:49556) 2023-07-14T11:55:25.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.550+0000 7fe8937fe700 1 -- 172.21.15.104:0/3534887312 learned_addr learned my addr 172.21.15.104:0/3534887312 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:25.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.550+0000 7fe8a0e59700 1 -- 172.21.15.104:0/3534887312 >> v2:172.21.15.121:3300/0 conn(0x7fe89c0cddb0 msgr2=0x7fe89c070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:25.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.550+0000 7fe8a0e59700 1 --2- 172.21.15.104:0/3534887312 >> v2:172.21.15.121:3300/0 conn(0x7fe89c0cddb0 0x7fe89c070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.550+0000 7fe8a0e59700 1 -- 172.21.15.104:0/3534887312 >> v1:172.21.15.121:6789/0 conn(0x7fe89c160640 legacy=0x7fe89c164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:25.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.550+0000 7fe8a0e59700 1 -- 172.21.15.104:0/3534887312 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c0caae0 con 0x7fe89c070ea0 2023-07-14T11:55:25.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.550+0000 7fe8937fe700 1 --2- 172.21.15.104:0/3534887312 >> v2:172.21.15.121:3300/0 conn(0x7fe89c0cddb0 0x7fe89c070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:55:25.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.554+0000 7fe8a0e59700 1 --2- 172.21.15.104:0/3534887312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe89c070ea0 0x7fe89c0afd30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe894009850 tx=0x7fe894009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b85cf7d0ddba113 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:25.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.554+0000 7fe89b7fe700 1 -- 172.21.15.104:0/3534887312 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894011830 con 0x7fe89c070ea0 2023-07-14T11:55:25.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.554+0000 7fe89b7fe700 1 -- 172.21.15.104:0/3534887312 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe894011990 con 0x7fe89c070ea0 2023-07-14T11:55:25.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.554+0000 7fe89b7fe700 1 -- 172.21.15.104:0/3534887312 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894011d40 con 0x7fe89c070ea0 2023-07-14T11:55:25.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.554+0000 7fe8a28bc700 1 -- 172.21.15.104:0/3534887312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe89c070ea0 msgr2=0x7fe89c0afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:25.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.554+0000 7fe8a28bc700 1 --2- 172.21.15.104:0/3534887312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe89c070ea0 0x7fe89c0afd30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe894009850 tx=0x7fe894009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:25.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.554+0000 7fe8a28bc700 1 -- 172.21.15.104:0/3534887312 shutdown_connections 2023-07-14T11:55:25.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.554+0000 7fe8a28bc700 1 --2- 172.21.15.104:0/3534887312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe89c070ea0 0x7fe89c0afd30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.554+0000 7fe8a28bc700 1 --2- 172.21.15.104:0/3534887312 >> v2:172.21.15.121:3300/0 conn(0x7fe89c0cddb0 0x7fe89c070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.554+0000 7fe8a28bc700 1 -- 172.21.15.104:0/3534887312 >> 172.21.15.104:0/3534887312 conn(0x7fe89c0cad30 msgr2=0x7fe89c0cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:25.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.554+0000 7fe8a28bc700 1 -- 172.21.15.104:0/3534887312 shutdown_connections 2023-07-14T11:55:25.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.554+0000 7fe8a28bc700 1 -- 172.21.15.104:0/3534887312 wait complete. 2023-07-14T11:55:25.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.554+0000 7fe8a28bc700 1 Processor -- start 2023-07-14T11:55:25.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.554+0000 7fe8a28bc700 1 -- start start 2023-07-14T11:55:25.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe8a28bc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe89c070ea0 0x7fe89c156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:25.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe8a28bc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe89c0cddb0 0x7fe89c157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:25.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe8a28bc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe89c160640 0x7fe89c160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:25.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe8a28bc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c158520 con 0x7fe89c070ea0 2023-07-14T11:55:25.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe8a28bc700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c1579a0 con 0x7fe89c160640 2023-07-14T11:55:25.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe8a28bc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c157c80 con 0x7fe89c0cddb0 2023-07-14T11:55:25.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe89bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe89c070ea0 0x7fe89c156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:25.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe89bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe89c070ea0 0x7fe89c156c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41280/0 (socket says 172.21.15.104:41280) 2023-07-14T11:55:25.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe89bfff700 1 -- 172.21.15.104:0/2089104553 learned_addr learned my addr 172.21.15.104:0/2089104553 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:25.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe8a0e59700 1 --2- 172.21.15.104:0/2089104553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe89c160640 0x7fe89c160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:25.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe8937fe700 1 --2- 172.21.15.104:0/2089104553 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe89c0cddb0 0x7fe89c157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:25.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe89bfff700 1 -- 172.21.15.104:0/2089104553 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe89c0cddb0 msgr2=0x7fe89c157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:25.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe89bfff700 1 --2- 172.21.15.104:0/2089104553 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe89c0cddb0 0x7fe89c157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe89bfff700 1 -- 172.21.15.104:0/2089104553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe89c160640 msgr2=0x7fe89c160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:25.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe89bfff700 1 --2- 172.21.15.104:0/2089104553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe89c160640 0x7fe89c160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe89bfff700 1 -- 172.21.15.104:0/2089104553 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe884009b30 con 0x7fe89c070ea0 2023-07-14T11:55:25.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe89bfff700 1 --2- 172.21.15.104:0/2089104553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe89c070ea0 0x7fe89c156c40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00c500 tx=0x7fe88c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:25.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe8997fa700 1 -- 172.21.15.104:0/2089104553 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c0052d0 con 0x7fe89c070ea0 2023-07-14T11:55:25.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe8997fa700 1 -- 172.21.15.104:0/2089104553 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c005430 con 0x7fe89c070ea0 2023-07-14T11:55:25.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe8a28bc700 1 -- 172.21.15.104:0/2089104553 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8940094b0 con 0x7fe89c070ea0 2023-07-14T11:55:25.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe8997fa700 1 -- 172.21.15.104:0/2089104553 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c0055a0 con 0x7fe89c070ea0 2023-07-14T11:55:25.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.558+0000 7fe8a28bc700 1 -- 172.21.15.104:0/2089104553 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89c158240 con 0x7fe89c070ea0 2023-07-14T11:55:25.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.562+0000 7fe8a28bc700 1 -- 172.21.15.104:0/2089104553 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89c0c5410 con 0x7fe89c070ea0 2023-07-14T11:55:25.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.562+0000 7fe8997fa700 1 -- 172.21.15.104:0/2089104553 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe88c0038e0 con 0x7fe89c070ea0 2023-07-14T11:55:25.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.562+0000 7fe8997fa700 1 --2- 172.21.15.104:0/2089104553 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe87c0662f0 0x7fe87c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:25.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.566+0000 7fe8997fa700 1 -- 172.21.15.104:0/2089104553 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fe88c083460 con 0x7fe89c070ea0 2023-07-14T11:55:25.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.566+0000 7fe8937fe700 1 --2- 172.21.15.104:0/2089104553 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe87c0662f0 0x7fe87c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:25.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.566+0000 7fe8937fe700 1 --2- 172.21.15.104:0/2089104553 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe87c0662f0 0x7fe87c0687a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0b2fc0 tx=0x7fe88400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:25.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.570+0000 7fe8997fa700 1 -- 172.21.15.104:0/2089104553 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe89c0c5410 con 0x7fe89c070ea0 2023-07-14T11:55:25.732 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.746+0000 7fe8a28bc700 1 -- 172.21.15.104:0/2089104553 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe89c0c5410 con 0x7fe89c070ea0 2023-07-14T11:55:25.734 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.750+0000 7fe8997fa700 1 -- 172.21.15.104:0/2089104553 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+20311 (secure 0 0 0) 0x7fe89c0c5410 con 0x7fe89c070ea0 2023-07-14T11:55:25.735 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:25.735 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":45,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:55:10.654581+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:54:56.370313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-14T11:54:57.608947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"data_new","create_time":"2023-07-14T11:55:08.099546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:55:25.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.750+0000 7fe8a28bc700 1 -- 172.21.15.104:0/2089104553 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe87c0662f0 msgr2=0x7fe87c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:25.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.750+0000 7fe8a28bc700 1 --2- 172.21.15.104:0/2089104553 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe87c0662f0 0x7fe87c0687a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0b2fc0 tx=0x7fe88400b040 comp rx=0 tx=0).stop 2023-07-14T11:55:25.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.750+0000 7fe8a28bc700 1 -- 172.21.15.104:0/2089104553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe89c070ea0 msgr2=0x7fe89c156c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:25.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.750+0000 7fe8a28bc700 1 --2- 172.21.15.104:0/2089104553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe89c070ea0 0x7fe89c156c40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00c500 tx=0x7fe88c00c8c0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.750+0000 7fe8a28bc700 1 -- 172.21.15.104:0/2089104553 shutdown_connections 2023-07-14T11:55:25.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.750+0000 7fe8a28bc700 1 --2- 172.21.15.104:0/2089104553 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe87c0662f0 0x7fe87c0687a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.750+0000 7fe8a28bc700 1 --2- 172.21.15.104:0/2089104553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe89c160640 0x7fe89c160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.750+0000 7fe8a28bc700 1 --2- 172.21.15.104:0/2089104553 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe89c0cddb0 0x7fe89c157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.750+0000 7fe8a28bc700 1 --2- 172.21.15.104:0/2089104553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe89c070ea0 0x7fe89c156c40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.750+0000 7fe8a28bc700 1 -- 172.21.15.104:0/2089104553 >> 172.21.15.104:0/2089104553 conn(0x7fe89c0cad30 msgr2=0x7fe89c0b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:25.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.750+0000 7fe8a28bc700 1 -- 172.21.15.104:0/2089104553 shutdown_connections 2023-07-14T11:55:25.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.750+0000 7fe8a28bc700 1 -- 172.21.15.104:0/2089104553 wait complete. 2023-07-14T11:55:25.760 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:54:56.370313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:54:57.608947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'data_new', 'create_time': '2023-07-14T11:55:08.099546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:55:25.761 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:55:25.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.954+0000 7f7c7e719700 1 Processor -- start 2023-07-14T11:55:25.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.954+0000 7f7c7e719700 1 -- start start 2023-07-14T11:55:25.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.958+0000 7f7c7e719700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c780d21c0 0x7f7c780b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:25.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.958+0000 7f7c7e719700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c780af2e0 con 0x7f7c780cf7a0 2023-07-14T11:55:25.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.958+0000 7f7c7e719700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c780af5c0 con 0x7f7c780d0920 2023-07-14T11:55:25.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.958+0000 7f7c7e719700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c780d0030 con 0x7f7c780d21c0 2023-07-14T11:55:25.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.958+0000 7f7c77fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f7c780cf7a0 0x7f7c780d0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:39368/0 (socket says 172.21.15.104:39368) 2023-07-14T11:55:25.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.958+0000 7f7c77fff700 1 -- 172.21.15.104:0/1884908247 learned_addr learned my addr 172.21.15.104:0/1884908247 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:25.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.958+0000 7f7c7ccb6700 1 --2- 172.21.15.104:0/1884908247 >> v2:172.21.15.121:3300/0 conn(0x7f7c780d21c0 0x7f7c780b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:25.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.958+0000 7f7c777fe700 1 -- 172.21.15.104:0/1884908247 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1120898502 0 0) 0x7f7c780af2e0 con 0x7f7c780cf7a0 2023-07-14T11:55:25.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.958+0000 7f7c777fe700 1 -- 172.21.15.104:0/1884908247 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c640035d0 con 0x7f7c780cf7a0 2023-07-14T11:55:25.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.958+0000 7f7c777fe700 1 -- 172.21.15.104:0/1884908247 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3810553556 0 0) 0x7f7c640035d0 con 0x7f7c780cf7a0 2023-07-14T11:55:25.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.958+0000 7f7c777fe700 1 -- 172.21.15.104:0/1884908247 >> v2:172.21.15.121:3300/0 conn(0x7f7c780d21c0 msgr2=0x7f7c780b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:25.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.958+0000 7f7c777fe700 1 --2- 172.21.15.104:0/1884908247 >> v2:172.21.15.121:3300/0 conn(0x7f7c780d21c0 0x7f7c780b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.958+0000 7f7c777fe700 1 -- 172.21.15.104:0/1884908247 >> v1:172.21.15.121:6789/0 conn(0x7f7c780d0920 legacy=0x7f7c780b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:25.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c777fe700 1 -- 172.21.15.104:0/1884908247 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c780af8a0 con 0x7f7c780cf7a0 2023-07-14T11:55:25.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c777fe700 1 -- 172.21.15.104:0/1884908247 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f7c68001670 con 0x7f7c780cf7a0 2023-07-14T11:55:25.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c777fe700 1 -- 172.21.15.104:0/1884908247 >> v1:172.21.15.104:6789/0 conn(0x7f7c780cf7a0 legacy=0x7f7c780d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:25.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c777fe700 1 --2- 172.21.15.104:0/1884908247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c64003a20 0x7f7c64003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:25.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c777fe700 1 --2- 172.21.15.104:0/1884908247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c64004870 0x7f7c64004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:25.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c777fe700 1 --2- 172.21.15.104:0/1884908247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c64005220 0x7f7c6400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:25.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c777fe700 1 -- 172.21.15.104:0/1884908247 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c780d0030 con 0x7f7c64003a20 2023-07-14T11:55:25.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c777fe700 1 -- 172.21.15.104:0/1884908247 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6400c130 con 0x7f7c64004870 2023-07-14T11:55:25.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c777fe700 1 -- 172.21.15.104:0/1884908247 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6400c2f0 con 0x7f7c64005220 2023-07-14T11:55:25.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c77fff700 1 --2- 172.21.15.104:0/1884908247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c64003a20 0x7f7c64003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:25.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c6f7fe700 1 --2- 172.21.15.104:0/1884908247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c64004870 0x7f7c64004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:25.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c7ccb6700 1 --2- 172.21.15.104:0/1884908247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c64005220 0x7f7c6400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:25.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c6f7fe700 1 -- 172.21.15.104:0/1884908247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c64005220 msgr2=0x7f7c6400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:25.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c6f7fe700 1 --2- 172.21.15.104:0/1884908247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c64005220 0x7f7c6400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c6f7fe700 1 -- 172.21.15.104:0/1884908247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c64003a20 msgr2=0x7f7c64003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:25.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c6f7fe700 1 --2- 172.21.15.104:0/1884908247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c64003a20 0x7f7c64003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c6f7fe700 1 -- 172.21.15.104:0/1884908247 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c640035d0 con 0x7f7c64004870 2023-07-14T11:55:25.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c7ccb6700 1 -- 172.21.15.104:0/1884908247 reap_dead start 2023-07-14T11:55:25.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c6f7fe700 1 --2- 172.21.15.104:0/1884908247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c64004870 0x7f7c64004ce0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7c780d0d10 tx=0x7f7c640058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ad85b04191e0a67 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:25.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c777fe700 1 -- 172.21.15.104:0/1884908247 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c600052d0 con 0x7f7c64004870 2023-07-14T11:55:25.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c777fe700 1 -- 172.21.15.104:0/1884908247 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c60005450 con 0x7f7c64004870 2023-07-14T11:55:25.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c777fe700 1 -- 172.21.15.104:0/1884908247 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c600038e0 con 0x7f7c64004870 2023-07-14T11:55:25.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c7e719700 1 -- 172.21.15.104:0/1884908247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c64004870 msgr2=0x7f7c64004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:25.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c7e719700 1 --2- 172.21.15.104:0/1884908247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c64004870 0x7f7c64004ce0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7c780d0d10 tx=0x7f7c640058a0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c7e719700 1 -- 172.21.15.104:0/1884908247 shutdown_connections 2023-07-14T11:55:25.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c7e719700 1 --2- 172.21.15.104:0/1884908247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c64004870 0x7f7c64004ce0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.962+0000 7f7c7e719700 1 -- 172.21.15.104:0/1884908247 >> 172.21.15.104:0/1884908247 conn(0x7f7c7809b9c0 msgr2=0x7f7c780d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:25.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.966+0000 7f7c7e719700 1 -- 172.21.15.104:0/1884908247 shutdown_connections 2023-07-14T11:55:25.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.966+0000 7f7c7e719700 1 -- 172.21.15.104:0/1884908247 wait complete. 2023-07-14T11:55:25.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.966+0000 7f7c7e719700 1 Processor -- start 2023-07-14T11:55:25.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.966+0000 7f7c7e719700 1 -- start start 2023-07-14T11:55:25.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c7e719700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c64004870 0x7f7c781594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:25.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c7e719700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c780b5fa0 0x7f7c780b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:25.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c7e719700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c7814e670 0x7f7c7814eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:25.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c7e719700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7814f020 con 0x7f7c780b5fa0 2023-07-14T11:55:25.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c7e719700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c78159aa0 con 0x7f7c7814e670 2023-07-14T11:55:25.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c7e719700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c78159d80 con 0x7f7c64004870 2023-07-14T11:55:25.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c6f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c780b5fa0 0x7f7c780b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:25.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c6f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c780b5fa0 0x7f7c780b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41294/0 (socket says 172.21.15.104:41294) 2023-07-14T11:55:25.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c6f7fe700 1 -- 172.21.15.104:0/2133037572 learned_addr learned my addr 172.21.15.104:0/2133037572 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:25.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c77fff700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c64004870 0x7f7c781594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:25.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c7ccb6700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c7814e670 0x7f7c7814eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:25.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c6f7fe700 1 -- 172.21.15.104:0/2133037572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c64004870 msgr2=0x7f7c781594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:25.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c6f7fe700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c64004870 0x7f7c781594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c6f7fe700 1 -- 172.21.15.104:0/2133037572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c7814e670 msgr2=0x7f7c7814eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:25.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c6f7fe700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c7814e670 0x7f7c7814eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:25.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c6f7fe700 1 -- 172.21.15.104:0/2133037572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7815a060 con 0x7f7c780b5fa0 2023-07-14T11:55:25.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c77fff700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c64004870 0x7f7c781594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:55:25.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c7ccb6700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c7814e670 0x7f7c7814eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:55:25.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c6f7fe700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c780b5fa0 0x7f7c780b2df0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7c60005040 tx=0x7f7c60001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:25.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c757fa700 1 -- 172.21.15.104:0/2133037572 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c60001290 con 0x7f7c780b5fa0 2023-07-14T11:55:25.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c757fa700 1 -- 172.21.15.104:0/2133037572 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c600036a0 con 0x7f7c780b5fa0 2023-07-14T11:55:25.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.970+0000 7f7c7e719700 1 -- 172.21.15.104:0/2133037572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7815a340 con 0x7f7c780b5fa0 2023-07-14T11:55:25.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.974+0000 7f7c757fa700 1 -- 172.21.15.104:0/2133037572 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c60003a50 con 0x7f7c780b5fa0 2023-07-14T11:55:25.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.974+0000 7f7c7e719700 1 -- 172.21.15.104:0/2133037572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c780b6410 con 0x7f7c780b5fa0 2023-07-14T11:55:25.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.974+0000 7f7c7e719700 1 -- 172.21.15.104:0/2133037572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c7804dcb0 con 0x7f7c780b5fa0 2023-07-14T11:55:25.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.974+0000 7f7c757fa700 1 -- 172.21.15.104:0/2133037572 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7c60003bb0 con 0x7f7c780b5fa0 2023-07-14T11:55:25.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.978+0000 7f7c757fa700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c580662f0 0x7f7c580687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:25.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.978+0000 7f7c757fa700 1 -- 172.21.15.104:0/2133037572 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f7c60084680 con 0x7f7c780b5fa0 2023-07-14T11:55:25.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.978+0000 7f7c77fff700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c580662f0 0x7f7c580687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:25.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.978+0000 7f7c77fff700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c580662f0 0x7f7c580687a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7c68001050 tx=0x7f7c68000fe0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:25.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:25.982+0000 7f7c757fa700 1 -- 172.21.15.104:0/2133037572 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f7c600518d0 con 0x7f7c780b5fa0 2023-07-14T11:55:26.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.150+0000 7f7c7e719700 1 -- 172.21.15.104:0/2133037572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c780d0440 con 0x7f7c780b5fa0 2023-07-14T11:55:26.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.150+0000 7f7c757fa700 1 -- 172.21.15.104:0/2133037572 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c60003e60 con 0x7f7c780b5fa0 2023-07-14T11:55:26.139 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:55:26.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.154+0000 7f7c7e719700 1 -- 172.21.15.104:0/2133037572 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c580662f0 msgr2=0x7f7c580687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:26.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.154+0000 7f7c7e719700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c580662f0 0x7f7c580687a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7c68001050 tx=0x7f7c68000fe0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.154+0000 7f7c7e719700 1 -- 172.21.15.104:0/2133037572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c780b5fa0 msgr2=0x7f7c780b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:26.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.154+0000 7f7c7e719700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c780b5fa0 0x7f7c780b2df0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7c60005040 tx=0x7f7c60001120 comp rx=0 tx=0).stop 2023-07-14T11:55:26.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.154+0000 7f7c7e719700 1 -- 172.21.15.104:0/2133037572 shutdown_connections 2023-07-14T11:55:26.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.154+0000 7f7c7e719700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c580662f0 0x7f7c580687a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.154+0000 7f7c7e719700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c780b5fa0 0x7f7c780b2df0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.154+0000 7f7c7e719700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c7814e670 0x7f7c7814eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.154+0000 7f7c7e719700 1 --2- 172.21.15.104:0/2133037572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c64004870 0x7f7c781594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.154+0000 7f7c7e719700 1 -- 172.21.15.104:0/2133037572 >> 172.21.15.104:0/2133037572 conn(0x7f7c7809b9c0 msgr2=0x7f7c7815ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:26.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.154+0000 7f7c7e719700 1 -- 172.21.15.104:0/2133037572 shutdown_connections 2023-07-14T11:55:26.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.154+0000 7f7c7e719700 1 -- 172.21.15.104:0/2133037572 wait complete. 2023-07-14T11:55:26.162 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:55:26.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.350+0000 7f90ef145700 1 Processor -- start 2023-07-14T11:55:26.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.350+0000 7f90ef145700 1 -- start start 2023-07-14T11:55:26.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.354+0000 7f90ef145700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f90e80cd260 0x7f90e80d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:26.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.354+0000 7f90ef145700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e80d1a60 con 0x7f90e80b0ee0 2023-07-14T11:55:26.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.354+0000 7f90ef145700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e80d1ef0 con 0x7f90e80d0610 2023-07-14T11:55:26.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.354+0000 7f90ef145700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e80d0e50 con 0x7f90e80cd260 2023-07-14T11:55:26.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.354+0000 7f90ee143700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f90e80b0ee0 0x7f90e80ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:39378/0 (socket says 172.21.15.104:39378) 2023-07-14T11:55:26.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.354+0000 7f90ed942700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f90e80cd260 0x7f90e80d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:26.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.354+0000 7f90ee143700 1 -- 172.21.15.104:0/2633174251 learned_addr learned my addr 172.21.15.104:0/2633174251 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:26.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.354+0000 7f90ed942700 1 -- 172.21.15.104:0/2633174251 >> v1:172.21.15.121:6789/0 conn(0x7f90e80d0610 legacy=0x7f90e80af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:26.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.354+0000 7f90ed942700 1 -- 172.21.15.104:0/2633174251 >> v1:172.21.15.104:6789/0 conn(0x7f90e80b0ee0 legacy=0x7f90e80ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:26.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.354+0000 7f90ed141700 1 -- 172.21.15.104:0/2633174251 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252477591 0 0) 0x7f90e80d1a60 con 0x7f90e80b0ee0 2023-07-14T11:55:26.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.354+0000 7f90ed942700 1 -- 172.21.15.104:0/2633174251 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e80d1130 con 0x7f90e80cd260 2023-07-14T11:55:26.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.354+0000 7f90ed942700 1 --2- 172.21.15.104:0/2633174251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90e80cd260 0x7f90e80d00d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009850 tx=0x7f90d8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4880008675464b58 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:26.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.358+0000 7f90ed141700 1 -- 172.21.15.104:0/2633174251 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8011830 con 0x7f90e80cd260 2023-07-14T11:55:26.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.358+0000 7f90ed141700 1 -- 172.21.15.104:0/2633174251 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d8011990 con 0x7f90e80cd260 2023-07-14T11:55:26.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.358+0000 7f90ed141700 1 -- 172.21.15.104:0/2633174251 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8011d20 con 0x7f90e80cd260 2023-07-14T11:55:26.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.358+0000 7f90ef145700 1 -- 172.21.15.104:0/2633174251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90e80cd260 msgr2=0x7f90e80d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:26.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.358+0000 7f90ef145700 1 --2- 172.21.15.104:0/2633174251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90e80cd260 0x7f90e80d00d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009850 tx=0x7f90d8009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:26.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.358+0000 7f90ef145700 1 -- 172.21.15.104:0/2633174251 shutdown_connections 2023-07-14T11:55:26.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.358+0000 7f90ef145700 1 --2- 172.21.15.104:0/2633174251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90e80cd260 0x7f90e80d00d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.358+0000 7f90ef145700 1 -- 172.21.15.104:0/2633174251 >> 172.21.15.104:0/2633174251 conn(0x7f90e8060c40 msgr2=0x7f90e809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:26.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.358+0000 7f90ef145700 1 -- 172.21.15.104:0/2633174251 shutdown_connections 2023-07-14T11:55:26.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.358+0000 7f90ef145700 1 -- 172.21.15.104:0/2633174251 wait complete. 2023-07-14T11:55:26.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.362+0000 7f90ef145700 1 Processor -- start 2023-07-14T11:55:26.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.362+0000 7f90ef145700 1 -- start start 2023-07-14T11:55:26.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.362+0000 7f90ef145700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90e80b0ee0 0x7f90e815a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:26.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.362+0000 7f90ef145700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90e80cd260 0x7f90e80cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:26.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.362+0000 7f90ef145700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90e80d0610 0x7f90e80c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:26.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.362+0000 7f90ef145700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e80cac90 con 0x7f90e80b0ee0 2023-07-14T11:55:26.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.362+0000 7f90ef145700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e80cb120 con 0x7f90e80cd260 2023-07-14T11:55:26.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.362+0000 7f90ef145700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e815ada0 con 0x7f90e80d0610 2023-07-14T11:55:26.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.362+0000 7f90ee143700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90e80b0ee0 0x7f90e815a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:26.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.362+0000 7f90ee143700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90e80b0ee0 0x7f90e815a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41314/0 (socket says 172.21.15.104:41314) 2023-07-14T11:55:26.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.362+0000 7f90ee143700 1 -- 172.21.15.104:0/1742274218 learned_addr learned my addr 172.21.15.104:0/1742274218 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:26.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.366+0000 7f90ed942700 1 --2- 172.21.15.104:0/1742274218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90e80cd260 0x7f90e80cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:26.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.366+0000 7f90ee944700 1 --2- 172.21.15.104:0/1742274218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90e80d0610 0x7f90e80c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:26.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.366+0000 7f90ee143700 1 -- 172.21.15.104:0/1742274218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90e80d0610 msgr2=0x7f90e80c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:26.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.366+0000 7f90ee143700 1 --2- 172.21.15.104:0/1742274218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90e80d0610 0x7f90e80c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.366+0000 7f90ee143700 1 -- 172.21.15.104:0/1742274218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90e80cd260 msgr2=0x7f90e80cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:26.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.366+0000 7f90ee143700 1 --2- 172.21.15.104:0/1742274218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90e80cd260 0x7f90e80cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.366+0000 7f90ee143700 1 -- 172.21.15.104:0/1742274218 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d80094b0 con 0x7f90e80b0ee0 2023-07-14T11:55:26.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.366+0000 7f90ed942700 1 --2- 172.21.15.104:0/1742274218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90e80cd260 0x7f90e80cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:55:26.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.366+0000 7f90ee143700 1 --2- 172.21.15.104:0/1742274218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90e80b0ee0 0x7f90e815a860 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f90e400c4a0 tx=0x7f90e400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:26.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.366+0000 7f90deffd700 1 -- 172.21.15.104:0/1742274218 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4004ca0 con 0x7f90e80b0ee0 2023-07-14T11:55:26.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.366+0000 7f90deffd700 1 -- 172.21.15.104:0/1742274218 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e4004e00 con 0x7f90e80b0ee0 2023-07-14T11:55:26.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.366+0000 7f90deffd700 1 -- 172.21.15.104:0/1742274218 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4007310 con 0x7f90e80b0ee0 2023-07-14T11:55:26.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.366+0000 7f90ef145700 1 -- 172.21.15.104:0/1742274218 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e80ca080 con 0x7f90e80b0ee0 2023-07-14T11:55:26.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.366+0000 7f90ef145700 1 -- 172.21.15.104:0/1742274218 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e80ca640 con 0x7f90e80b0ee0 2023-07-14T11:55:26.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.366+0000 7f90ef145700 1 -- 172.21.15.104:0/1742274218 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e804e500 con 0x7f90e80b0ee0 2023-07-14T11:55:26.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.370+0000 7f90deffd700 1 -- 172.21.15.104:0/1742274218 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f90e4007470 con 0x7f90e80b0ee0 2023-07-14T11:55:26.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.370+0000 7f90deffd700 1 --2- 172.21.15.104:0/1742274218 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f90cc0662f0 0x7f90cc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:26.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.370+0000 7f90deffd700 1 -- 172.21.15.104:0/1742274218 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f90e4083da0 con 0x7f90e80b0ee0 2023-07-14T11:55:26.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.370+0000 7f90ed942700 1 --2- 172.21.15.104:0/1742274218 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f90cc0662f0 0x7f90cc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:26.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.374+0000 7f90ed942700 1 --2- 172.21.15.104:0/1742274218 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f90cc0662f0 0x7f90cc0687a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f90e80cd650 tx=0x7f90d8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:26.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.378+0000 7f90deffd700 1 -- 172.21.15.104:0/1742274218 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f90e4050e20 con 0x7f90e80b0ee0 2023-07-14T11:55:26.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.550+0000 7f90ef145700 1 -- 172.21.15.104:0/1742274218 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f90e80cf4b0 con 0x7f90e80b0ee0 2023-07-14T11:55:26.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.554+0000 7f90deffd700 1 -- 172.21.15.104:0/1742274218 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f90e4007720 con 0x7f90e80b0ee0 2023-07-14T11:55:26.539 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:55:26.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.558+0000 7f90ef145700 1 -- 172.21.15.104:0/1742274218 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f90cc0662f0 msgr2=0x7f90cc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:26.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.558+0000 7f90ef145700 1 --2- 172.21.15.104:0/1742274218 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f90cc0662f0 0x7f90cc0687a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f90e80cd650 tx=0x7f90d8008030 comp rx=0 tx=0).stop 2023-07-14T11:55:26.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.558+0000 7f90ef145700 1 -- 172.21.15.104:0/1742274218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90e80b0ee0 msgr2=0x7f90e815a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:26.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.558+0000 7f90ef145700 1 --2- 172.21.15.104:0/1742274218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90e80b0ee0 0x7f90e815a860 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f90e400c4a0 tx=0x7f90e400c860 comp rx=0 tx=0).stop 2023-07-14T11:55:26.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.558+0000 7f90ef145700 1 -- 172.21.15.104:0/1742274218 shutdown_connections 2023-07-14T11:55:26.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.558+0000 7f90ef145700 1 --2- 172.21.15.104:0/1742274218 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f90cc0662f0 0x7f90cc0687a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.558+0000 7f90ef145700 1 --2- 172.21.15.104:0/1742274218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90e80d0610 0x7f90e80c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.558+0000 7f90ef145700 1 --2- 172.21.15.104:0/1742274218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90e80cd260 0x7f90e80cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.558+0000 7f90ef145700 1 --2- 172.21.15.104:0/1742274218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90e80b0ee0 0x7f90e815a860 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.558+0000 7f90ef145700 1 -- 172.21.15.104:0/1742274218 >> 172.21.15.104:0/1742274218 conn(0x7f90e8060c40 msgr2=0x7f90e80b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:26.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.558+0000 7f90ef145700 1 -- 172.21.15.104:0/1742274218 shutdown_connections 2023-07-14T11:55:26.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.558+0000 7f90ef145700 1 -- 172.21.15.104:0/1742274218 wait complete. 2023-07-14T11:55:26.564 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:55:26.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.750+0000 7efcd59ae700 1 Processor -- start 2023-07-14T11:55:26.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.750+0000 7efcd59ae700 1 -- start start 2023-07-14T11:55:26.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efcd59ae700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efcd0159c60 0x7efcd006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:26.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efcd59ae700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7efcd00b0770 0x7efcd0070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:26.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efcd59ae700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd00cab00 0x7efcd00caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:26.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efcd59ae700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd00cb4c0 con 0x7efcd00b0770 2023-07-14T11:55:26.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efcd59ae700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd006da00 con 0x7efcd0159c60 2023-07-14T11:55:26.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efcd59ae700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd006dce0 con 0x7efcd00cab00 2023-07-14T11:55:26.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efcce7fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7efcd00b0770 0x7efcd0070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:26.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efccf7fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd00cab00 0x7efcd00caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:26.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efcceffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efcd0159c60 0x7efcd006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:26.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efccf7fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd00cab00 0x7efcd00caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:44466/0 (socket says 172.21.15.104:44466) 2023-07-14T11:55:26.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efccf7fe700 1 -- 172.21.15.104:0/4097288900 learned_addr learned my addr 172.21.15.104:0/4097288900 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:26.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efccf7fe700 1 -- 172.21.15.104:0/4097288900 >> v2:172.21.15.121:3300/0 conn(0x7efcd0159c60 msgr2=0x7efcd006d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:26.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efccf7fe700 1 --2- 172.21.15.104:0/4097288900 >> v2:172.21.15.121:3300/0 conn(0x7efcd0159c60 0x7efcd006d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efccf7fe700 1 -- 172.21.15.104:0/4097288900 >> v2:172.21.15.104:3300/0 conn(0x7efcd00b0770 msgr2=0x7efcd0070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:26.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efccf7fe700 1 --2- 172.21.15.104:0/4097288900 >> v2:172.21.15.104:3300/0 conn(0x7efcd00b0770 0x7efcd0070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efccf7fe700 1 -- 172.21.15.104:0/4097288900 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd006dfc0 con 0x7efcd00cab00 2023-07-14T11:55:26.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efccf7fe700 1 --2- 172.21.15.104:0/4097288900 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd00cab00 0x7efcd00caf80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efcd006bbd0 tx=0x7efcc800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4a21d970c0388b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:26.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efccdffb700 1 -- 172.21.15.104:0/4097288900 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc800e070 con 0x7efcd00cab00 2023-07-14T11:55:26.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efccdffb700 1 -- 172.21.15.104:0/4097288900 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc800b910 con 0x7efcd00cab00 2023-07-14T11:55:26.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.754+0000 7efccdffb700 1 -- 172.21.15.104:0/4097288900 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc800bcc0 con 0x7efcd00cab00 2023-07-14T11:55:26.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.758+0000 7efcd59ae700 1 -- 172.21.15.104:0/4097288900 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd00cab00 msgr2=0x7efcd00caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:26.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.758+0000 7efcd59ae700 1 --2- 172.21.15.104:0/4097288900 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd00cab00 0x7efcd00caf80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efcd006bbd0 tx=0x7efcc800b330 comp rx=0 tx=0).stop 2023-07-14T11:55:26.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.758+0000 7efcd59ae700 1 -- 172.21.15.104:0/4097288900 shutdown_connections 2023-07-14T11:55:26.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.758+0000 7efcd59ae700 1 --2- 172.21.15.104:0/4097288900 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd00cab00 0x7efcd00caf80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.758+0000 7efcd59ae700 1 --2- 172.21.15.104:0/4097288900 >> v2:172.21.15.104:3300/0 conn(0x7efcd00b0770 0x7efcd0070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.758+0000 7efcd59ae700 1 --2- 172.21.15.104:0/4097288900 >> v2:172.21.15.121:3300/0 conn(0x7efcd0159c60 0x7efcd006d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.758+0000 7efcd59ae700 1 -- 172.21.15.104:0/4097288900 >> 172.21.15.104:0/4097288900 conn(0x7efcd00afc60 msgr2=0x7efcd00b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:26.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.758+0000 7efcd59ae700 1 -- 172.21.15.104:0/4097288900 shutdown_connections 2023-07-14T11:55:26.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.758+0000 7efcd59ae700 1 -- 172.21.15.104:0/4097288900 wait complete. 2023-07-14T11:55:26.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efcd59ae700 1 Processor -- start 2023-07-14T11:55:26.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efcd59ae700 1 -- start start 2023-07-14T11:55:26.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efcd59ae700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd00b0770 0x7efcd01537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:26.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efcd59ae700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd00cab00 0x7efcd00c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:26.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efcd59ae700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcd0159c60 0x7efcd00c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:26.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efcd59ae700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd00cc330 con 0x7efcd0159c60 2023-07-14T11:55:26.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efcd59ae700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd00c2290 con 0x7efcd00cab00 2023-07-14T11:55:26.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efcd59ae700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd00c2570 con 0x7efcd00b0770 2023-07-14T11:55:26.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efccf7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcd0159c60 0x7efcd00c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:26.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efccf7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcd0159c60 0x7efcd00c94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41336/0 (socket says 172.21.15.104:41336) 2023-07-14T11:55:26.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efccf7fe700 1 -- 172.21.15.104:0/2360030278 learned_addr learned my addr 172.21.15.104:0/2360030278 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:26.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efcceffd700 1 --2- 172.21.15.104:0/2360030278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd00b0770 0x7efcd01537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:26.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efcce7fc700 1 --2- 172.21.15.104:0/2360030278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd00cab00 0x7efcd00c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:26.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efccf7fe700 1 -- 172.21.15.104:0/2360030278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd00b0770 msgr2=0x7efcd01537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:26.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efccf7fe700 1 --2- 172.21.15.104:0/2360030278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd00b0770 0x7efcd01537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efccf7fe700 1 -- 172.21.15.104:0/2360030278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd00cab00 msgr2=0x7efcd00c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:26.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efccf7fe700 1 --2- 172.21.15.104:0/2360030278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd00cab00 0x7efcd00c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efccf7fe700 1 -- 172.21.15.104:0/2360030278 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc8009b30 con 0x7efcd0159c60 2023-07-14T11:55:26.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.762+0000 7efcce7fc700 1 --2- 172.21.15.104:0/2360030278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd00cab00 0x7efcd00c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:26.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.766+0000 7efccf7fe700 1 --2- 172.21.15.104:0/2360030278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcd0159c60 0x7efcd00c94d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7efcd006de20 tx=0x7efcc8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:26.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.766+0000 7efcb7fff700 1 -- 172.21.15.104:0/2360030278 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc8004930 con 0x7efcd0159c60 2023-07-14T11:55:26.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.766+0000 7efcb7fff700 1 -- 172.21.15.104:0/2360030278 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc80099b0 con 0x7efcd0159c60 2023-07-14T11:55:26.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.766+0000 7efcb7fff700 1 -- 172.21.15.104:0/2360030278 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc801b670 con 0x7efcd0159c60 2023-07-14T11:55:26.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.766+0000 7efcd59ae700 1 -- 172.21.15.104:0/2360030278 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd00cbf50 con 0x7efcd0159c60 2023-07-14T11:55:26.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.766+0000 7efcd59ae700 1 -- 172.21.15.104:0/2360030278 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd00c1a50 con 0x7efcd0159c60 2023-07-14T11:55:26.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.766+0000 7efcd59ae700 1 -- 172.21.15.104:0/2360030278 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd00cc470 con 0x7efcd0159c60 2023-07-14T11:55:26.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.770+0000 7efcb7fff700 1 -- 172.21.15.104:0/2360030278 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7efcc801b7d0 con 0x7efcd0159c60 2023-07-14T11:55:26.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.770+0000 7efcb7fff700 1 --2- 172.21.15.104:0/2360030278 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efcbc0662f0 0x7efcbc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:26.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.770+0000 7efcb7fff700 1 -- 172.21.15.104:0/2360030278 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7efcc8085840 con 0x7efcd0159c60 2023-07-14T11:55:26.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.770+0000 7efcceffd700 1 --2- 172.21.15.104:0/2360030278 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efcbc0662f0 0x7efcbc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:26.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.770+0000 7efcceffd700 1 --2- 172.21.15.104:0/2360030278 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efcbc0662f0 0x7efcbc0687a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009bf0 tx=0x7efcc0009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:26.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.778+0000 7efcb7fff700 1 -- 172.21.15.104:0/2360030278 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7efcd00cc470 con 0x7efcd0159c60 2023-07-14T11:55:26.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.950+0000 7efcd59ae700 1 -- 172.21.15.104:0/2360030278 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efcd00cc470 con 0x7efcd0159c60 2023-07-14T11:55:26.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.950+0000 7efcb7fff700 1 -- 172.21.15.104:0/2360030278 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7efcd00cc470 con 0x7efcd0159c60 2023-07-14T11:55:26.938 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:55:26.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.954+0000 7efcd59ae700 1 -- 172.21.15.104:0/2360030278 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efcbc0662f0 msgr2=0x7efcbc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:26.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.954+0000 7efcd59ae700 1 --2- 172.21.15.104:0/2360030278 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efcbc0662f0 0x7efcbc0687a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009bf0 tx=0x7efcc0009240 comp rx=0 tx=0).stop 2023-07-14T11:55:26.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.954+0000 7efcd59ae700 1 -- 172.21.15.104:0/2360030278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcd0159c60 msgr2=0x7efcd00c94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:26.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.954+0000 7efcd59ae700 1 --2- 172.21.15.104:0/2360030278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcd0159c60 0x7efcd00c94d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7efcd006de20 tx=0x7efcc8004af0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.954+0000 7efcd59ae700 1 -- 172.21.15.104:0/2360030278 shutdown_connections 2023-07-14T11:55:26.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.954+0000 7efcd59ae700 1 --2- 172.21.15.104:0/2360030278 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efcbc0662f0 0x7efcbc0687a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.954+0000 7efcd59ae700 1 --2- 172.21.15.104:0/2360030278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcd0159c60 0x7efcd00c94d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.954+0000 7efcd59ae700 1 --2- 172.21.15.104:0/2360030278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd00cab00 0x7efcd00c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.954+0000 7efcd59ae700 1 --2- 172.21.15.104:0/2360030278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd00b0770 0x7efcd01537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:26.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.954+0000 7efcd59ae700 1 -- 172.21.15.104:0/2360030278 >> 172.21.15.104:0/2360030278 conn(0x7efcd00afc60 msgr2=0x7efcd00c8880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:26.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.954+0000 7efcd59ae700 1 -- 172.21.15.104:0/2360030278 shutdown_connections 2023-07-14T11:55:26.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:26.954+0000 7efcd59ae700 1 -- 172.21.15.104:0/2360030278 wait complete. 2023-07-14T11:55:26.964 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:55:27.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.154+0000 7f7704a53700 1 Processor -- start 2023-07-14T11:55:27.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.154+0000 7f7704a53700 1 -- start start 2023-07-14T11:55:27.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.158+0000 7f7704a53700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7700159c60 0x7f770006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.158+0000 7f7704a53700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77000cab00 0x7f77000caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.158+0000 7f7704a53700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77000cb4c0 con 0x7f77000b0770 2023-07-14T11:55:27.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.158+0000 7f7704a53700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77000714f0 con 0x7f7700159c60 2023-07-14T11:55:27.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.158+0000 7f7704a53700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770006da00 con 0x7f77000cab00 2023-07-14T11:55:27.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.158+0000 7f76ff7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7700159c60 0x7f770006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.158+0000 7f76ff7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7700159c60 0x7f770006d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:49612/0 (socket says 172.21.15.104:49612) 2023-07-14T11:55:27.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.158+0000 7f76ff7fe700 1 -- 172.21.15.104:0/3593723455 learned_addr learned my addr 172.21.15.104:0/3593723455 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:27.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.158+0000 7f76fffff700 1 --2- 172.21.15.104:0/3593723455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77000cab00 0x7f77000caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.158+0000 7f76ff7fe700 1 -- 172.21.15.104:0/3593723455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77000cab00 msgr2=0x7f77000caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:27.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.158+0000 7f76ff7fe700 1 --2- 172.21.15.104:0/3593723455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77000cab00 0x7f77000caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.158+0000 7f76ff7fe700 1 -- 172.21.15.104:0/3593723455 >> v1:172.21.15.121:6789/0 conn(0x7f77000b0770 legacy=0x7f7700070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.158+0000 7f76ff7fe700 1 -- 172.21.15.104:0/3593723455 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770006dce0 con 0x7f7700159c60 2023-07-14T11:55:27.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.158+0000 7f76ff7fe700 1 --2- 172.21.15.104:0/3593723455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7700159c60 0x7f770006d3b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009910 tx=0x7f76f0009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8531b6b370defe8e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:27.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.162+0000 7f76fe7fc700 1 -- 172.21.15.104:0/3593723455 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0011830 con 0x7f7700159c60 2023-07-14T11:55:27.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.162+0000 7f76fe7fc700 1 -- 172.21.15.104:0/3593723455 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f0011990 con 0x7f7700159c60 2023-07-14T11:55:27.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.162+0000 7f76fe7fc700 1 -- 172.21.15.104:0/3593723455 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0011d40 con 0x7f7700159c60 2023-07-14T11:55:27.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.162+0000 7f7704a53700 1 -- 172.21.15.104:0/3593723455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7700159c60 msgr2=0x7f770006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.162+0000 7f7704a53700 1 --2- 172.21.15.104:0/3593723455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7700159c60 0x7f770006d3b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009910 tx=0x7f76f0009c20 comp rx=0 tx=0).stop 2023-07-14T11:55:27.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.162+0000 7f7704a53700 1 -- 172.21.15.104:0/3593723455 shutdown_connections 2023-07-14T11:55:27.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.162+0000 7f7704a53700 1 --2- 172.21.15.104:0/3593723455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77000cab00 0x7f77000caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.162+0000 7f7704a53700 1 --2- 172.21.15.104:0/3593723455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7700159c60 0x7f770006d3b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.162+0000 7f7704a53700 1 -- 172.21.15.104:0/3593723455 >> 172.21.15.104:0/3593723455 conn(0x7f77000afc60 msgr2=0x7f77000b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:27.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.162+0000 7f7704a53700 1 -- 172.21.15.104:0/3593723455 shutdown_connections 2023-07-14T11:55:27.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.162+0000 7f7704a53700 1 -- 172.21.15.104:0/3593723455 wait complete. 2023-07-14T11:55:27.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.166+0000 7f7704a53700 1 Processor -- start 2023-07-14T11:55:27.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.166+0000 7f7704a53700 1 -- start start 2023-07-14T11:55:27.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.166+0000 7f7704a53700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77000b0770 0x7f77000c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.166+0000 7f7704a53700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77000cab00 0x7f770014d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.166+0000 7f7704a53700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7700159c60 0x7f77000c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.166+0000 7f7704a53700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770015b080 con 0x7f77000b0770 2023-07-14T11:55:27.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.166+0000 7f7704a53700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77000c6070 con 0x7f7700159c60 2023-07-14T11:55:27.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.166+0000 7f7704a53700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77000c2c20 con 0x7f77000cab00 2023-07-14T11:55:27.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.166+0000 7f76ff7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77000b0770 0x7f77000c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.166+0000 7f76ff7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77000b0770 0x7f77000c5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41350/0 (socket says 172.21.15.104:41350) 2023-07-14T11:55:27.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.166+0000 7f76ff7fe700 1 -- 172.21.15.104:0/1109494177 learned_addr learned my addr 172.21.15.104:0/1109494177 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:27.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.166+0000 7f76feffd700 1 --2- 172.21.15.104:0/1109494177 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77000cab00 0x7f770014d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.166+0000 7f76fffff700 1 --2- 172.21.15.104:0/1109494177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7700159c60 0x7f77000c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.170+0000 7f76ff7fe700 1 -- 172.21.15.104:0/1109494177 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77000cab00 msgr2=0x7f770014d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.170+0000 7f76ff7fe700 1 --2- 172.21.15.104:0/1109494177 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77000cab00 0x7f770014d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.170+0000 7f76ff7fe700 1 -- 172.21.15.104:0/1109494177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7700159c60 msgr2=0x7f77000c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.170+0000 7f76ff7fe700 1 --2- 172.21.15.104:0/1109494177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7700159c60 0x7f77000c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.170+0000 7f76ff7fe700 1 -- 172.21.15.104:0/1109494177 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f0009570 con 0x7f77000b0770 2023-07-14T11:55:27.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.170+0000 7f76fffff700 1 --2- 172.21.15.104:0/1109494177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7700159c60 0x7f77000c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:27.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.170+0000 7f76ff7fe700 1 --2- 172.21.15.104:0/1109494177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77000b0770 0x7f77000c5b30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f76f0000c00 tx=0x7f76f0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:27.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.170+0000 7f76e7fff700 1 -- 172.21.15.104:0/1109494177 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0004590 con 0x7f77000b0770 2023-07-14T11:55:27.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.170+0000 7f76e7fff700 1 -- 172.21.15.104:0/1109494177 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f00036c0 con 0x7f77000b0770 2023-07-14T11:55:27.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.170+0000 7f7704a53700 1 -- 172.21.15.104:0/1109494177 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770015b290 con 0x7f77000b0770 2023-07-14T11:55:27.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.170+0000 7f76e7fff700 1 -- 172.21.15.104:0/1109494177 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0003a70 con 0x7f77000b0770 2023-07-14T11:55:27.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.170+0000 7f7704a53700 1 -- 172.21.15.104:0/1109494177 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770015b850 con 0x7f77000b0770 2023-07-14T11:55:27.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.170+0000 7f7704a53700 1 -- 172.21.15.104:0/1109494177 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770004e500 con 0x7f77000b0770 2023-07-14T11:55:27.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.170+0000 7f76e7fff700 1 -- 172.21.15.104:0/1109494177 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f76f0003bd0 con 0x7f77000b0770 2023-07-14T11:55:27.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.174+0000 7f76e7fff700 1 --2- 172.21.15.104:0/1109494177 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f76ec0662f0 0x7f76ec0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.174+0000 7f76e7fff700 1 -- 172.21.15.104:0/1109494177 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f76f0021030 con 0x7f77000b0770 2023-07-14T11:55:27.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.174+0000 7f76feffd700 1 --2- 172.21.15.104:0/1109494177 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f76ec0662f0 0x7f76ec0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.174+0000 7f76feffd700 1 --2- 172.21.15.104:0/1109494177 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f76ec0662f0 0x7f76ec0687a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f77000cdf00 tx=0x7f76f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:27.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.178+0000 7f76e7fff700 1 -- 172.21.15.104:0/1109494177 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f76f0052850 con 0x7f77000b0770 2023-07-14T11:55:27.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.346+0000 7f7704a53700 1 -- 172.21.15.104:0/1109494177 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f770006c6f0 con 0x7f77000b0770 2023-07-14T11:55:27.333 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.346+0000 7f76e7fff700 1 -- 172.21.15.104:0/1109494177 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7f76f0055e70 con 0x7f77000b0770 2023-07-14T11:55:27.333 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:55:27.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.350+0000 7f7704a53700 1 -- 172.21.15.104:0/1109494177 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f76ec0662f0 msgr2=0x7f76ec0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.350+0000 7f7704a53700 1 --2- 172.21.15.104:0/1109494177 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f76ec0662f0 0x7f76ec0687a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f77000cdf00 tx=0x7f76f4009040 comp rx=0 tx=0).stop 2023-07-14T11:55:27.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.350+0000 7f7704a53700 1 -- 172.21.15.104:0/1109494177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77000b0770 msgr2=0x7f77000c5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.350+0000 7f7704a53700 1 --2- 172.21.15.104:0/1109494177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77000b0770 0x7f77000c5b30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f76f0000c00 tx=0x7f76f0004770 comp rx=0 tx=0).stop 2023-07-14T11:55:27.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.350+0000 7f7704a53700 1 -- 172.21.15.104:0/1109494177 shutdown_connections 2023-07-14T11:55:27.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.350+0000 7f7704a53700 1 --2- 172.21.15.104:0/1109494177 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f76ec0662f0 0x7f76ec0687a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.350+0000 7f7704a53700 1 --2- 172.21.15.104:0/1109494177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7700159c60 0x7f77000c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.350+0000 7f7704a53700 1 --2- 172.21.15.104:0/1109494177 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77000cab00 0x7f770014d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.350+0000 7f7704a53700 1 --2- 172.21.15.104:0/1109494177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77000b0770 0x7f77000c5b30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.350+0000 7f7704a53700 1 -- 172.21.15.104:0/1109494177 >> 172.21.15.104:0/1109494177 conn(0x7f77000afc60 msgr2=0x7f77000b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:27.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.350+0000 7f7704a53700 1 -- 172.21.15.104:0/1109494177 shutdown_connections 2023-07-14T11:55:27.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.350+0000 7f7704a53700 1 -- 172.21.15.104:0/1109494177 wait complete. 2023-07-14T11:55:27.355 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-07-14T11:55:27.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.546+0000 7f6f5759e700 1 Processor -- start 2023-07-14T11:55:27.532 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.546+0000 7f6f5759e700 1 -- start start 2023-07-14T11:55:27.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.546+0000 7f6f5759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6f58159c60 0x7f6f5806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.546+0000 7f6f5759e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6f580b0770 0x7f6f58070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.546+0000 7f6f5759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f580cab00 0x7f6f580caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.546+0000 7f6f5759e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f580cb4c0 con 0x7f6f580b0770 2023-07-14T11:55:27.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.546+0000 7f6f5759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5806da00 con 0x7f6f58159c60 2023-07-14T11:55:27.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.546+0000 7f6f5759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5806dce0 con 0x7f6f580cab00 2023-07-14T11:55:27.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.546+0000 7f6f4dd9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6f580b0770 0x7f6f58070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f4dd9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6f580b0770 0x7f6f58070de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41360/0 (socket says 172.21.15.104:41360) 2023-07-14T11:55:27.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f4dd9b700 1 -- 172.21.15.104:0/674559802 learned_addr learned my addr 172.21.15.104:0/674559802 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:27.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f5659c700 1 --2- 172.21.15.104:0/674559802 >> v2:172.21.15.121:3300/0 conn(0x7f6f58159c60 0x7f6f5806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f56d9d700 1 --2- 172.21.15.104:0/674559802 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f580cab00 0x7f6f580caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f56d9d700 1 -- 172.21.15.104:0/674559802 >> v2:172.21.15.121:3300/0 conn(0x7f6f58159c60 msgr2=0x7f6f5806d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:27.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f56d9d700 1 --2- 172.21.15.104:0/674559802 >> v2:172.21.15.121:3300/0 conn(0x7f6f58159c60 0x7f6f5806d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f56d9d700 1 -- 172.21.15.104:0/674559802 >> v2:172.21.15.104:3300/0 conn(0x7f6f580b0770 msgr2=0x7f6f58070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:27.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f56d9d700 1 --2- 172.21.15.104:0/674559802 >> v2:172.21.15.104:3300/0 conn(0x7f6f580b0770 0x7f6f58070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f56d9d700 1 -- 172.21.15.104:0/674559802 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5806dfc0 con 0x7f6f580cab00 2023-07-14T11:55:27.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f56d9d700 1 --2- 172.21.15.104:0/674559802 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f580cab00 0x7f6f580caf80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6f5806bbd0 tx=0x7f6f5000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c845ff5db5ff44 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:27.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f55d9b700 1 -- 172.21.15.104:0/674559802 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5000e070 con 0x7f6f580cab00 2023-07-14T11:55:27.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f55d9b700 1 -- 172.21.15.104:0/674559802 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5000bb20 con 0x7f6f580cab00 2023-07-14T11:55:27.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f55d9b700 1 -- 172.21.15.104:0/674559802 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f50012430 con 0x7f6f580cab00 2023-07-14T11:55:27.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f5759e700 1 -- 172.21.15.104:0/674559802 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f580cab00 msgr2=0x7f6f580caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f5759e700 1 --2- 172.21.15.104:0/674559802 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f580cab00 0x7f6f580caf80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6f5806bbd0 tx=0x7f6f5000b330 comp rx=0 tx=0).stop 2023-07-14T11:55:27.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f5759e700 1 -- 172.21.15.104:0/674559802 shutdown_connections 2023-07-14T11:55:27.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f5759e700 1 --2- 172.21.15.104:0/674559802 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f580cab00 0x7f6f580caf80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f5759e700 1 --2- 172.21.15.104:0/674559802 >> v2:172.21.15.104:3300/0 conn(0x7f6f580b0770 0x7f6f58070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f5759e700 1 --2- 172.21.15.104:0/674559802 >> v2:172.21.15.121:3300/0 conn(0x7f6f58159c60 0x7f6f5806d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f5759e700 1 -- 172.21.15.104:0/674559802 >> 172.21.15.104:0/674559802 conn(0x7f6f580afc60 msgr2=0x7f6f580b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:27.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.550+0000 7f6f5759e700 1 -- 172.21.15.104:0/674559802 shutdown_connections 2023-07-14T11:55:27.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.554+0000 7f6f5759e700 1 -- 172.21.15.104:0/674559802 wait complete. 2023-07-14T11:55:27.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.554+0000 7f6f5759e700 1 Processor -- start 2023-07-14T11:55:27.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5759e700 1 -- start start 2023-07-14T11:55:27.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5759e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f580b0770 0x7f6f580b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f580cab00 0x7f6f580c4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f58159c60 0x7f6f580b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5759e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f580c1890 con 0x7f6f580b0770 2023-07-14T11:55:27.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f580c23d0 con 0x7f6f58159c60 2023-07-14T11:55:27.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f580b14c0 con 0x7f6f580cab00 2023-07-14T11:55:27.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f580b0770 0x7f6f580b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f580b0770 0x7f6f580b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41372/0 (socket says 172.21.15.104:41372) 2023-07-14T11:55:27.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5659c700 1 -- 172.21.15.104:0/2091594531 learned_addr learned my addr 172.21.15.104:0/2091594531 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:27.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f4dd9b700 1 --2- 172.21.15.104:0/2091594531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f580cab00 0x7f6f580c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f56d9d700 1 --2- 172.21.15.104:0/2091594531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f58159c60 0x7f6f580b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5659c700 1 -- 172.21.15.104:0/2091594531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f580cab00 msgr2=0x7f6f580c4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5659c700 1 --2- 172.21.15.104:0/2091594531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f580cab00 0x7f6f580c4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5659c700 1 -- 172.21.15.104:0/2091594531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f58159c60 msgr2=0x7f6f580b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5659c700 1 --2- 172.21.15.104:0/2091594531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f58159c60 0x7f6f580b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5659c700 1 -- 172.21.15.104:0/2091594531 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f50009a70 con 0x7f6f580b0770 2023-07-14T11:55:27.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.558+0000 7f6f5659c700 1 --2- 172.21.15.104:0/2091594531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f580b0770 0x7f6f580b0f80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800ea00 tx=0x7f6f4800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:27.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.562+0000 7f6f4f7fe700 1 -- 172.21.15.104:0/2091594531 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48009720 con 0x7f6f580b0770 2023-07-14T11:55:27.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.562+0000 7f6f4f7fe700 1 -- 172.21.15.104:0/2091594531 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4800c7f0 con 0x7f6f580b0770 2023-07-14T11:55:27.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.562+0000 7f6f5759e700 1 -- 172.21.15.104:0/2091594531 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f580c4540 con 0x7f6f580b0770 2023-07-14T11:55:27.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.562+0000 7f6f4f7fe700 1 -- 172.21.15.104:0/2091594531 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4800cb60 con 0x7f6f580b0770 2023-07-14T11:55:27.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.562+0000 7f6f5759e700 1 -- 172.21.15.104:0/2091594531 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f580c1d80 con 0x7f6f580b0770 2023-07-14T11:55:27.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.562+0000 7f6f5759e700 1 -- 172.21.15.104:0/2091594531 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5804e500 con 0x7f6f580b0770 2023-07-14T11:55:27.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.562+0000 7f6f4f7fe700 1 -- 172.21.15.104:0/2091594531 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6f4800ccc0 con 0x7f6f580b0770 2023-07-14T11:55:27.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.566+0000 7f6f4f7fe700 1 --2- 172.21.15.104:0/2091594531 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6f380662f0 0x7f6f380687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.566+0000 7f6f4f7fe700 1 -- 172.21.15.104:0/2091594531 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f6f48084380 con 0x7f6f580b0770 2023-07-14T11:55:27.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.566+0000 7f6f4dd9b700 1 --2- 172.21.15.104:0/2091594531 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6f380662f0 0x7f6f380687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.566+0000 7f6f4dd9b700 1 --2- 172.21.15.104:0/2091594531 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6f380662f0 0x7f6f380687a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6f58154cf0 tx=0x7f6f400092e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:27.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.574+0000 7f6f4f7fe700 1 -- 172.21.15.104:0/2091594531 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f6f48051510 con 0x7f6f580b0770 2023-07-14T11:55:27.731 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.746+0000 7f6f5759e700 1 -- 172.21.15.104:0/2091594531 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f6f58155cf0 con 0x7f6f580b0770 2023-07-14T11:55:27.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.750+0000 7f6f4f7fe700 1 -- 172.21.15.104:0/2091594531 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v45) v1 ==== 97+0+11 (secure 0 0 0) 0x7f6f48054b30 con 0x7f6f580b0770 2023-07-14T11:55:27.735 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:55:27.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.754+0000 7f6f5759e700 1 -- 172.21.15.104:0/2091594531 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6f380662f0 msgr2=0x7f6f380687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.754+0000 7f6f5759e700 1 --2- 172.21.15.104:0/2091594531 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6f380662f0 0x7f6f380687a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6f58154cf0 tx=0x7f6f400092e0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.754+0000 7f6f5759e700 1 -- 172.21.15.104:0/2091594531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f580b0770 msgr2=0x7f6f580b0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.754+0000 7f6f5759e700 1 --2- 172.21.15.104:0/2091594531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f580b0770 0x7f6f580b0f80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800ea00 tx=0x7f6f4800edc0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.754+0000 7f6f5759e700 1 -- 172.21.15.104:0/2091594531 shutdown_connections 2023-07-14T11:55:27.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.754+0000 7f6f5759e700 1 --2- 172.21.15.104:0/2091594531 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6f380662f0 0x7f6f380687a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.754+0000 7f6f5759e700 1 --2- 172.21.15.104:0/2091594531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f58159c60 0x7f6f580b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.754+0000 7f6f5759e700 1 --2- 172.21.15.104:0/2091594531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f580cab00 0x7f6f580c4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.754+0000 7f6f5759e700 1 --2- 172.21.15.104:0/2091594531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f580b0770 0x7f6f580b0f80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.754+0000 7f6f5759e700 1 -- 172.21.15.104:0/2091594531 >> 172.21.15.104:0/2091594531 conn(0x7f6f580afc60 msgr2=0x7f6f580b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:27.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.754+0000 7f6f5759e700 1 -- 172.21.15.104:0/2091594531 shutdown_connections 2023-07-14T11:55:27.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.754+0000 7f6f5759e700 1 -- 172.21.15.104:0/2091594531 wait complete. 2023-07-14T11:55:27.761 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:55:27.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.946+0000 7fab64890700 1 Processor -- start 2023-07-14T11:55:27.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.946+0000 7fab64890700 1 -- start start 2023-07-14T11:55:27.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.946+0000 7fab64890700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fab6015ffa0 0x7fab6006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.946+0000 7fab64890700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab600ca8a0 con 0x7fab600cb180 2023-07-14T11:55:27.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.946+0000 7fab64890700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab600cbcd0 con 0x7fab6006dc50 2023-07-14T11:55:27.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.946+0000 7fab64890700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab600cbf70 con 0x7fab6015ffa0 2023-07-14T11:55:27.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.946+0000 7fab5f59e700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fab600cb180 0x7fab600ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:39388/0 (socket says 172.21.15.104:39388) 2023-07-14T11:55:27.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.946+0000 7fab5f59e700 1 -- 172.21.15.104:0/3333472126 learned_addr learned my addr 172.21.15.104:0/3333472126 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:27.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.946+0000 7fab5ed9d700 1 --2- 172.21.15.104:0/3333472126 >> v2:172.21.15.121:3300/0 conn(0x7fab6015ffa0 0x7fab6006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.950+0000 7fab5ed9d700 1 -- 172.21.15.104:0/3333472126 >> v1:172.21.15.121:6789/0 conn(0x7fab6006dc50 legacy=0x7fab600caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.950+0000 7fab5ed9d700 1 -- 172.21.15.104:0/3333472126 >> v1:172.21.15.104:6789/0 conn(0x7fab600cb180 legacy=0x7fab600ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.950+0000 7fab5dd9b700 1 -- 172.21.15.104:0/3333472126 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528071143 0 0) 0x7fab58001490 con 0x7fab600cb180 2023-07-14T11:55:27.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.950+0000 7fab5ed9d700 1 -- 172.21.15.104:0/3333472126 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab600cc250 con 0x7fab6015ffa0 2023-07-14T11:55:27.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.950+0000 7fab5ed9d700 1 --2- 172.21.15.104:0/3333472126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6015ffa0 0x7fab6006d710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fab50009850 tx=0x7fab50009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd0932d199ad99ec server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:27.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.950+0000 7fab5dd9b700 1 -- 172.21.15.104:0/3333472126 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50011830 con 0x7fab6015ffa0 2023-07-14T11:55:27.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.950+0000 7fab5dd9b700 1 -- 172.21.15.104:0/3333472126 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab50011990 con 0x7fab6015ffa0 2023-07-14T11:55:27.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.950+0000 7fab5dd9b700 1 -- 172.21.15.104:0/3333472126 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50011d20 con 0x7fab6015ffa0 2023-07-14T11:55:27.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.950+0000 7fab64890700 1 -- 172.21.15.104:0/3333472126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6015ffa0 msgr2=0x7fab6006d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.950+0000 7fab64890700 1 --2- 172.21.15.104:0/3333472126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6015ffa0 0x7fab6006d710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fab50009850 tx=0x7fab50009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:27.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.950+0000 7fab64890700 1 -- 172.21.15.104:0/3333472126 shutdown_connections 2023-07-14T11:55:27.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.950+0000 7fab64890700 1 --2- 172.21.15.104:0/3333472126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6015ffa0 0x7fab6006d710 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.950+0000 7fab64890700 1 -- 172.21.15.104:0/3333472126 >> 172.21.15.104:0/3333472126 conn(0x7fab6006e960 msgr2=0x7fab6006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:27.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.950+0000 7fab64890700 1 -- 172.21.15.104:0/3333472126 shutdown_connections 2023-07-14T11:55:27.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.950+0000 7fab64890700 1 -- 172.21.15.104:0/3333472126 wait complete. 2023-07-14T11:55:27.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.954+0000 7fab64890700 1 Processor -- start 2023-07-14T11:55:27.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab64890700 1 -- start start 2023-07-14T11:55:27.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab64890700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab6006dc50 0x7fab601f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab64890700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600cb180 0x7fab600c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab64890700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab6015ffa0 0x7fab600c19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab64890700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab600c8470 con 0x7fab6015ffa0 2023-07-14T11:55:27.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab64890700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab601612f0 con 0x7fab600cb180 2023-07-14T11:55:27.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab64890700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab60160650 con 0x7fab6006dc50 2023-07-14T11:55:27.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab5f59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab6015ffa0 0x7fab600c19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab5e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600cb180 0x7fab600c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab5ed9d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab6006dc50 0x7fab601f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab5ed9d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab6006dc50 0x7fab601f5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:44526/0 (socket says 172.21.15.104:44526) 2023-07-14T11:55:27.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab5ed9d700 1 -- 172.21.15.104:0/2659313141 learned_addr learned my addr 172.21.15.104:0/2659313141 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:27.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab5ed9d700 1 -- 172.21.15.104:0/2659313141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600cb180 msgr2=0x7fab600c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab5ed9d700 1 --2- 172.21.15.104:0/2659313141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600cb180 0x7fab600c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab5ed9d700 1 -- 172.21.15.104:0/2659313141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab6015ffa0 msgr2=0x7fab600c19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab5ed9d700 1 --2- 172.21.15.104:0/2659313141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab6015ffa0 0x7fab600c19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab5ed9d700 1 -- 172.21.15.104:0/2659313141 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab500094b0 con 0x7fab6006dc50 2023-07-14T11:55:27.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab5ed9d700 1 --2- 172.21.15.104:0/2659313141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab6006dc50 0x7fab601f5340 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab50009fd0 tx=0x7fab50004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:27.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab577fe700 1 -- 172.21.15.104:0/2659313141 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000bab0 con 0x7fab6006dc50 2023-07-14T11:55:27.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab577fe700 1 -- 172.21.15.104:0/2659313141 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5001e070 con 0x7fab6006dc50 2023-07-14T11:55:27.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab64890700 1 -- 172.21.15.104:0/2659313141 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab60161010 con 0x7fab6006dc50 2023-07-14T11:55:27.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.958+0000 7fab577fe700 1 -- 172.21.15.104:0/2659313141 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000be50 con 0x7fab6006dc50 2023-07-14T11:55:27.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.962+0000 7fab64890700 1 -- 172.21.15.104:0/2659313141 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab600c65c0 con 0x7fab6006dc50 2023-07-14T11:55:27.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.962+0000 7fab64890700 1 -- 172.21.15.104:0/2659313141 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab60071620 con 0x7fab6006dc50 2023-07-14T11:55:27.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.962+0000 7fab577fe700 1 -- 172.21.15.104:0/2659313141 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fab5002d440 con 0x7fab6006dc50 2023-07-14T11:55:27.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.966+0000 7fab577fe700 1 --2- 172.21.15.104:0/2659313141 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fab400662f0 0x7fab400687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.966+0000 7fab577fe700 1 -- 172.21.15.104:0/2659313141 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fab50085310 con 0x7fab6006dc50 2023-07-14T11:55:27.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.966+0000 7fab5e59c700 1 --2- 172.21.15.104:0/2659313141 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fab400662f0 0x7fab400687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.966+0000 7fab5e59c700 1 --2- 172.21.15.104:0/2659313141 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fab400662f0 0x7fab400687a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fab6015fce0 tx=0x7fab48009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:27.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:27.974+0000 7fab577fe700 1 -- 172.21.15.104:0/2659313141 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fab500524b0 con 0x7fab6006dc50 2023-07-14T11:55:28.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.158+0000 7fab64890700 1 -- 172.21.15.104:0/2659313141 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fab6004e500 con 0x7fab6006dc50 2023-07-14T11:55:28.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.158+0000 7fab577fe700 1 -- 172.21.15.104:0/2659313141 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) v1 ==== 80+0+0 (secure 0 0 0) 0x7fab50055ad0 con 0x7fab6006dc50 2023-07-14T11:55:28.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.162+0000 7fab64890700 1 -- 172.21.15.104:0/2659313141 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fab400662f0 msgr2=0x7fab400687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:28.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.162+0000 7fab64890700 1 --2- 172.21.15.104:0/2659313141 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fab400662f0 0x7fab400687a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fab6015fce0 tx=0x7fab48009040 comp rx=0 tx=0).stop 2023-07-14T11:55:28.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.162+0000 7fab64890700 1 -- 172.21.15.104:0/2659313141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab6006dc50 msgr2=0x7fab601f5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:28.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.162+0000 7fab64890700 1 --2- 172.21.15.104:0/2659313141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab6006dc50 0x7fab601f5340 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab50009fd0 tx=0x7fab50004770 comp rx=0 tx=0).stop 2023-07-14T11:55:28.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.162+0000 7fab64890700 1 -- 172.21.15.104:0/2659313141 shutdown_connections 2023-07-14T11:55:28.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.162+0000 7fab64890700 1 --2- 172.21.15.104:0/2659313141 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fab400662f0 0x7fab400687a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.162+0000 7fab64890700 1 --2- 172.21.15.104:0/2659313141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab6015ffa0 0x7fab600c19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.162+0000 7fab64890700 1 --2- 172.21.15.104:0/2659313141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab600cb180 0x7fab600c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.162+0000 7fab64890700 1 --2- 172.21.15.104:0/2659313141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fab6006dc50 0x7fab601f5340 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.162+0000 7fab64890700 1 -- 172.21.15.104:0/2659313141 >> 172.21.15.104:0/2659313141 conn(0x7fab6006e960 msgr2=0x7fab6006bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:28.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.162+0000 7fab64890700 1 -- 172.21.15.104:0/2659313141 shutdown_connections 2023-07-14T11:55:28.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.162+0000 7fab64890700 1 -- 172.21.15.104:0/2659313141 wait complete. 2023-07-14T11:55:28.151 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:55:28.172 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:55:28.173 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:55:28.193 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:55:28.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.422+0000 7f12bdb4c700 1 Processor -- start 2023-07-14T11:55:28.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.422+0000 7f12bdb4c700 1 -- start start 2023-07-14T11:55:28.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12bdb4c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f12b80c9d90 0x7f12b80ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:28.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12bdb4c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f12b8159f20 0x7f12b80caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:28.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12bdb4c700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b80b0ef0 con 0x7f12b80caf70 2023-07-14T11:55:28.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12bdb4c700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b80ce9c0 con 0x7f12b8159f20 2023-07-14T11:55:28.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12bdb4c700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b80cee50 con 0x7f12b80c9d90 2023-07-14T11:55:28.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12bd34b700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f12b80caf70 0x7f12b80c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:39394/0 (socket says 172.21.15.104:39394) 2023-07-14T11:55:28.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12bd34b700 1 -- 172.21.15.104:0/4275277248 learned_addr learned my addr 172.21.15.104:0/4275277248 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:28.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12affff700 1 --2- 172.21.15.104:0/4275277248 >> v2:172.21.15.104:3300/0 conn(0x7f12b8159f20 0x7f12b80caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:28.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12bcb4a700 1 --2- 172.21.15.104:0/4275277248 >> v2:172.21.15.121:3300/0 conn(0x7f12b80c9d90 0x7f12b80ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:28.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12af7fe700 1 -- 172.21.15.104:0/4275277248 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244349564 0 0) 0x7f12b80b0ef0 con 0x7f12b80caf70 2023-07-14T11:55:28.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12af7fe700 1 -- 172.21.15.104:0/4275277248 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12a00035d0 con 0x7f12b80caf70 2023-07-14T11:55:28.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12affff700 1 -- 172.21.15.104:0/4275277248 >> v2:172.21.15.121:3300/0 conn(0x7f12b80c9d90 msgr2=0x7f12b80ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:28.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12affff700 1 --2- 172.21.15.104:0/4275277248 >> v2:172.21.15.121:3300/0 conn(0x7f12b80c9d90 0x7f12b80ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12affff700 1 -- 172.21.15.104:0/4275277248 >> v1:172.21.15.104:6789/0 conn(0x7f12b80caf70 legacy=0x7f12b80c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:28.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12affff700 1 -- 172.21.15.104:0/4275277248 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b80ca6a0 con 0x7f12b8159f20 2023-07-14T11:55:28.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12affff700 1 --2- 172.21.15.104:0/4275277248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b8159f20 0x7f12b80caa30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f12b80cdf30 tx=0x7f12a8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c8de2537bfc973 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:28.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12af7fe700 1 -- 172.21.15.104:0/4275277248 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8011830 con 0x7f12b8159f20 2023-07-14T11:55:28.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12af7fe700 1 -- 172.21.15.104:0/4275277248 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a8011990 con 0x7f12b8159f20 2023-07-14T11:55:28.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12af7fe700 1 -- 172.21.15.104:0/4275277248 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8011d20 con 0x7f12b8159f20 2023-07-14T11:55:28.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12bdb4c700 1 -- 172.21.15.104:0/4275277248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b8159f20 msgr2=0x7f12b80caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:28.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.426+0000 7f12bdb4c700 1 --2- 172.21.15.104:0/4275277248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b8159f20 0x7f12b80caa30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f12b80cdf30 tx=0x7f12a8009b30 comp rx=0 tx=0).stop 2023-07-14T11:55:28.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.430+0000 7f12bdb4c700 1 -- 172.21.15.104:0/4275277248 shutdown_connections 2023-07-14T11:55:28.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.430+0000 7f12bdb4c700 1 --2- 172.21.15.104:0/4275277248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b8159f20 0x7f12b80caa30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.430+0000 7f12bdb4c700 1 --2- 172.21.15.104:0/4275277248 >> v2:172.21.15.121:3300/0 conn(0x7f12b80c9d90 0x7f12b80ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.430+0000 7f12bdb4c700 1 -- 172.21.15.104:0/4275277248 >> 172.21.15.104:0/4275277248 conn(0x7f12b806e960 msgr2=0x7f12b806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:28.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.430+0000 7f12bdb4c700 1 -- 172.21.15.104:0/4275277248 shutdown_connections 2023-07-14T11:55:28.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.430+0000 7f12bdb4c700 1 -- 172.21.15.104:0/4275277248 wait complete. 2023-07-14T11:55:28.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12bdb4c700 1 Processor -- start 2023-07-14T11:55:28.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12bdb4c700 1 -- start start 2023-07-14T11:55:28.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12bdb4c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12b80c9d90 0x7f12b80b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:28.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12bdb4c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b80caf70 0x7f12b81525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:28.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12bdb4c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b8159f20 0x7f12b8156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:28.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12bdb4c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b81ef280 con 0x7f12b80caf70 2023-07-14T11:55:28.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12bdb4c700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80c2d60 con 0x7f12b80c9d90 2023-07-14T11:55:28.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12bdb4c700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b8152af0 con 0x7f12b8159f20 2023-07-14T11:55:28.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12affff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b80caf70 0x7f12b81525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:28.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12affff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b80caf70 0x7f12b81525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41392/0 (socket says 172.21.15.104:41392) 2023-07-14T11:55:28.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12affff700 1 -- 172.21.15.104:0/914213181 learned_addr learned my addr 172.21.15.104:0/914213181 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:28.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12bd34b700 1 --2- 172.21.15.104:0/914213181 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b8159f20 0x7f12b8156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:28.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12bcb4a700 1 --2- 172.21.15.104:0/914213181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12b80c9d90 0x7f12b80b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:28.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12affff700 1 -- 172.21.15.104:0/914213181 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b8159f20 msgr2=0x7f12b8156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:28.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12affff700 1 --2- 172.21.15.104:0/914213181 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b8159f20 0x7f12b8156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12affff700 1 -- 172.21.15.104:0/914213181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12b80c9d90 msgr2=0x7f12b80b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:28.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12affff700 1 --2- 172.21.15.104:0/914213181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12b80c9d90 0x7f12b80b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12affff700 1 -- 172.21.15.104:0/914213181 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a80094b0 con 0x7f12b80caf70 2023-07-14T11:55:28.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12affff700 1 --2- 172.21.15.104:0/914213181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b80caf70 0x7f12b81525b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f12a80090b0 tx=0x7f12a8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:28.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12ad7fa700 1 -- 172.21.15.104:0/914213181 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a80045d0 con 0x7f12b80caf70 2023-07-14T11:55:28.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12ad7fa700 1 -- 172.21.15.104:0/914213181 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a801f070 con 0x7f12b80caf70 2023-07-14T11:55:28.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12ad7fa700 1 -- 172.21.15.104:0/914213181 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a80038c0 con 0x7f12b80caf70 2023-07-14T11:55:28.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.434+0000 7f12bdb4c700 1 -- 172.21.15.104:0/914213181 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b8156e70 con 0x7f12b80caf70 2023-07-14T11:55:28.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.438+0000 7f12bdb4c700 1 -- 172.21.15.104:0/914213181 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b80b2900 con 0x7f12b80caf70 2023-07-14T11:55:28.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.438+0000 7f12bdb4c700 1 -- 172.21.15.104:0/914213181 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b804e500 con 0x7f12b80caf70 2023-07-14T11:55:28.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.438+0000 7f12ad7fa700 1 -- 172.21.15.104:0/914213181 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f12a8003a20 con 0x7f12b80caf70 2023-07-14T11:55:28.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.442+0000 7f12ad7fa700 1 --2- 172.21.15.104:0/914213181 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12a406a700 0x7f12a406cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:28.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.442+0000 7f12ad7fa700 1 -- 172.21.15.104:0/914213181 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f12a8021070 con 0x7f12b80caf70 2023-07-14T11:55:28.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.442+0000 7f12bcb4a700 1 --2- 172.21.15.104:0/914213181 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12a406a700 0x7f12a406cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:28.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.442+0000 7f12bcb4a700 1 --2- 172.21.15.104:0/914213181 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12a406a700 0x7f12a406cbb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009ae0 tx=0x7f12b4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:28.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.446+0000 7f12ad7fa700 1 -- 172.21.15.104:0/914213181 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f12a8052ae0 con 0x7f12b80caf70 2023-07-14T11:55:28.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.618+0000 7f12bdb4c700 1 -- 172.21.15.104:0/914213181 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12b806edd0 con 0x7f12b80caf70 2023-07-14T11:55:28.608 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.622+0000 7f12ad7fa700 1 -- 172.21.15.104:0/914213181 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+20311 (secure 0 0 0) 0x7f12a8056100 con 0x7f12b80caf70 2023-07-14T11:55:28.609 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:28.609 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":45,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:55:10.654581+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:54:56.370313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-14T11:54:57.608947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"data_new","create_time":"2023-07-14T11:55:08.099546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:55:28.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.626+0000 7f12bdb4c700 1 -- 172.21.15.104:0/914213181 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12a406a700 msgr2=0x7f12a406cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:28.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.626+0000 7f12bdb4c700 1 --2- 172.21.15.104:0/914213181 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12a406a700 0x7f12a406cbb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009ae0 tx=0x7f12b4008040 comp rx=0 tx=0).stop 2023-07-14T11:55:28.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.626+0000 7f12bdb4c700 1 -- 172.21.15.104:0/914213181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b80caf70 msgr2=0x7f12b81525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:28.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.626+0000 7f12bdb4c700 1 --2- 172.21.15.104:0/914213181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b80caf70 0x7f12b81525b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f12a80090b0 tx=0x7f12a8004770 comp rx=0 tx=0).stop 2023-07-14T11:55:28.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.626+0000 7f12bdb4c700 1 -- 172.21.15.104:0/914213181 shutdown_connections 2023-07-14T11:55:28.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.626+0000 7f12bdb4c700 1 --2- 172.21.15.104:0/914213181 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12a406a700 0x7f12a406cbb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.626+0000 7f12bdb4c700 1 --2- 172.21.15.104:0/914213181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b80caf70 0x7f12b81525b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.626+0000 7f12bdb4c700 1 --2- 172.21.15.104:0/914213181 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b8159f20 0x7f12b8156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.626+0000 7f12bdb4c700 1 --2- 172.21.15.104:0/914213181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12b80c9d90 0x7f12b80b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.626+0000 7f12bdb4c700 1 -- 172.21.15.104:0/914213181 >> 172.21.15.104:0/914213181 conn(0x7f12b806e960 msgr2=0x7f12b815bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:28.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.626+0000 7f12bdb4c700 1 -- 172.21.15.104:0/914213181 shutdown_connections 2023-07-14T11:55:28.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.626+0000 7f12bdb4c700 1 -- 172.21.15.104:0/914213181 wait complete. 2023-07-14T11:55:28.635 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:54:56.370313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:54:57.608947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'data_new', 'create_time': '2023-07-14T11:55:08.099546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:55:28.636 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:55:28.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.830+0000 7fe88e9ae700 1 Processor -- start 2023-07-14T11:55:28.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.830+0000 7fe88e9ae700 1 -- start start 2023-07-14T11:55:28.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.830+0000 7fe88e9ae700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe8880d0210 0x7fe8880d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:28.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.830+0000 7fe88e9ae700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8880cf000 0x7fe8880cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:28.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.830+0000 7fe88e9ae700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe888071d10 con 0x7fe8880b1c50 2023-07-14T11:55:28.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.830+0000 7fe88e9ae700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8880cf9c0 con 0x7fe8880d0210 2023-07-14T11:55:28.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.830+0000 7fe88e9ae700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88815c420 con 0x7fe8880cf000 2023-07-14T11:55:28.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.830+0000 7fe8877fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe8880d0210 0x7fe8880d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:28.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.830+0000 7fe8877fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe8880d0210 0x7fe8880d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41400/0 (socket says 172.21.15.104:41400) 2023-07-14T11:55:28.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.830+0000 7fe8877fe700 1 -- 172.21.15.104:0/2793477673 learned_addr learned my addr 172.21.15.104:0/2793477673 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:28.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.830+0000 7fe88cf4b700 1 --2- 172.21.15.104:0/2793477673 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8880cf000 0x7fe8880cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:28.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe8877fe700 1 -- 172.21.15.104:0/2793477673 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8880cf000 msgr2=0x7fe8880cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:28.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe8877fe700 1 --2- 172.21.15.104:0/2793477673 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8880cf000 0x7fe8880cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe8877fe700 1 -- 172.21.15.104:0/2793477673 >> v1:172.21.15.121:6789/0 conn(0x7fe8880b1c50 legacy=0x7fe888071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:28.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe8877fe700 1 -- 172.21.15.104:0/2793477673 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe888070840 con 0x7fe8880d0210 2023-07-14T11:55:28.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe88cf4b700 1 --2- 172.21.15.104:0/2793477673 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8880cf000 0x7fe8880cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:28.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe8877fe700 1 --2- 172.21.15.104:0/2793477673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8880d0210 0x7fe8880d0640 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009910 tx=0x7fe87c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e1d93fe577f861 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:28.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe886ffd700 1 -- 172.21.15.104:0/2793477673 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c00ba60 con 0x7fe8880d0210 2023-07-14T11:55:28.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe886ffd700 1 -- 172.21.15.104:0/2793477673 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87c00bbc0 con 0x7fe8880d0210 2023-07-14T11:55:28.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe886ffd700 1 -- 172.21.15.104:0/2793477673 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c009e60 con 0x7fe8880d0210 2023-07-14T11:55:28.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe88e9ae700 1 -- 172.21.15.104:0/2793477673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8880d0210 msgr2=0x7fe8880d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:28.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe88e9ae700 1 --2- 172.21.15.104:0/2793477673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8880d0210 0x7fe8880d0640 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009910 tx=0x7fe87c009c20 comp rx=0 tx=0).stop 2023-07-14T11:55:28.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe88e9ae700 1 -- 172.21.15.104:0/2793477673 shutdown_connections 2023-07-14T11:55:28.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe88e9ae700 1 --2- 172.21.15.104:0/2793477673 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8880cf000 0x7fe8880cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe88e9ae700 1 --2- 172.21.15.104:0/2793477673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8880d0210 0x7fe8880d0640 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe88e9ae700 1 -- 172.21.15.104:0/2793477673 >> 172.21.15.104:0/2793477673 conn(0x7fe88809b9c0 msgr2=0x7fe8880d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:28.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe88e9ae700 1 -- 172.21.15.104:0/2793477673 shutdown_connections 2023-07-14T11:55:28.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.834+0000 7fe88e9ae700 1 -- 172.21.15.104:0/2793477673 wait complete. 2023-07-14T11:55:28.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe88e9ae700 1 Processor -- start 2023-07-14T11:55:28.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe88e9ae700 1 -- start start 2023-07-14T11:55:28.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe88e9ae700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8880b1c50 0x7fe88814fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:28.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe88e9ae700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8880cf000 0x7fe8880b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:28.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe88e9ae700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8880d0210 0x7fe888156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:28.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe88e9ae700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8881e4080 con 0x7fe8880b1c50 2023-07-14T11:55:28.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe88e9ae700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8880b24b0 con 0x7fe8880cf000 2023-07-14T11:55:28.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe88e9ae700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8880b2790 con 0x7fe8880d0210 2023-07-14T11:55:28.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe887fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8880b1c50 0x7fe88814fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:28.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe887fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8880b1c50 0x7fe88814fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41414/0 (socket says 172.21.15.104:41414) 2023-07-14T11:55:28.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe887fff700 1 -- 172.21.15.104:0/2844023258 learned_addr learned my addr 172.21.15.104:0/2844023258 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:28.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe88cf4b700 1 --2- 172.21.15.104:0/2844023258 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8880d0210 0x7fe888156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:28.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe8877fe700 1 --2- 172.21.15.104:0/2844023258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8880cf000 0x7fe8880b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:28.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe887fff700 1 -- 172.21.15.104:0/2844023258 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8880d0210 msgr2=0x7fe888156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:28.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe887fff700 1 --2- 172.21.15.104:0/2844023258 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8880d0210 0x7fe888156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe887fff700 1 -- 172.21.15.104:0/2844023258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8880cf000 msgr2=0x7fe8880b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:28.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe887fff700 1 --2- 172.21.15.104:0/2844023258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8880cf000 0x7fe8880b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:28.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe887fff700 1 -- 172.21.15.104:0/2844023258 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87c009570 con 0x7fe8880b1c50 2023-07-14T11:55:28.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe887fff700 1 --2- 172.21.15.104:0/2844023258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8880b1c50 0x7fe88814fc40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe87800c500 tx=0x7fe87800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:28.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe884ff9700 1 -- 172.21.15.104:0/2844023258 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8780052d0 con 0x7fe8880b1c50 2023-07-14T11:55:28.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe88e9ae700 1 -- 172.21.15.104:0/2844023258 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8880b2a70 con 0x7fe8880b1c50 2023-07-14T11:55:28.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe884ff9700 1 -- 172.21.15.104:0/2844023258 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe878005430 con 0x7fe8880b1c50 2023-07-14T11:55:28.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe884ff9700 1 -- 172.21.15.104:0/2844023258 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8780055a0 con 0x7fe8880b1c50 2023-07-14T11:55:28.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.838+0000 7fe88e9ae700 1 -- 172.21.15.104:0/2844023258 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8880b3030 con 0x7fe8880b1c50 2023-07-14T11:55:28.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.842+0000 7fe88e9ae700 1 -- 172.21.15.104:0/2844023258 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88804dcb0 con 0x7fe8880b1c50 2023-07-14T11:55:28.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.842+0000 7fe884ff9700 1 -- 172.21.15.104:0/2844023258 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe8780038e0 con 0x7fe8880b1c50 2023-07-14T11:55:28.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.842+0000 7fe884ff9700 1 --2- 172.21.15.104:0/2844023258 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8740662f0 0x7fe8740687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:28.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.846+0000 7fe884ff9700 1 -- 172.21.15.104:0/2844023258 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fe8780833c0 con 0x7fe8880b1c50 2023-07-14T11:55:28.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.846+0000 7fe8877fe700 1 --2- 172.21.15.104:0/2844023258 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8740662f0 0x7fe8740687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:28.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.846+0000 7fe8877fe700 1 --2- 172.21.15.104:0/2844023258 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8740662f0 0x7fe8740687a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe87c000c00 tx=0x7fe87c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:28.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:28.850+0000 7fe884ff9700 1 -- 172.21.15.104:0/2844023258 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe8780504e0 con 0x7fe8880b1c50 2023-07-14T11:55:29.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.022+0000 7fe88e9ae700 1 -- 172.21.15.104:0/2844023258 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe888071d10 con 0x7fe8880b1c50 2023-07-14T11:55:29.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.026+0000 7fe884ff9700 1 -- 172.21.15.104:0/2844023258 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe888071d10 con 0x7fe8880b1c50 2023-07-14T11:55:29.011 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:55:29.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.026+0000 7fe88e9ae700 1 -- 172.21.15.104:0/2844023258 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8740662f0 msgr2=0x7fe8740687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:29.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.026+0000 7fe88e9ae700 1 --2- 172.21.15.104:0/2844023258 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8740662f0 0x7fe8740687a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe87c000c00 tx=0x7fe87c008030 comp rx=0 tx=0).stop 2023-07-14T11:55:29.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.026+0000 7fe88e9ae700 1 -- 172.21.15.104:0/2844023258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8880b1c50 msgr2=0x7fe88814fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:29.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.026+0000 7fe88e9ae700 1 --2- 172.21.15.104:0/2844023258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8880b1c50 0x7fe88814fc40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe87800c500 tx=0x7fe87800c8c0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.026+0000 7fe88e9ae700 1 -- 172.21.15.104:0/2844023258 shutdown_connections 2023-07-14T11:55:29.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.026+0000 7fe88e9ae700 1 --2- 172.21.15.104:0/2844023258 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe8740662f0 0x7fe8740687a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.026+0000 7fe88e9ae700 1 --2- 172.21.15.104:0/2844023258 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe8880d0210 0x7fe888156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.026+0000 7fe88e9ae700 1 --2- 172.21.15.104:0/2844023258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8880cf000 0x7fe8880b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.026+0000 7fe88e9ae700 1 --2- 172.21.15.104:0/2844023258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8880b1c50 0x7fe88814fc40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.026+0000 7fe88e9ae700 1 -- 172.21.15.104:0/2844023258 >> 172.21.15.104:0/2844023258 conn(0x7fe88809b9c0 msgr2=0x7fe8880b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:29.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.030+0000 7fe88e9ae700 1 -- 172.21.15.104:0/2844023258 shutdown_connections 2023-07-14T11:55:29.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.030+0000 7fe88e9ae700 1 -- 172.21.15.104:0/2844023258 wait complete. 2023-07-14T11:55:29.035 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:55:29.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.222+0000 7f8689889700 1 Processor -- start 2023-07-14T11:55:29.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.222+0000 7f8689889700 1 -- start start 2023-07-14T11:55:29.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.226+0000 7f8689889700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f86840af8c0 0x7f86840afd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:29.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.226+0000 7f8689889700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86840d0930 con 0x7f868415fa10 2023-07-14T11:55:29.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.226+0000 7f8689889700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86840b4ae0 con 0x7f86841604f0 2023-07-14T11:55:29.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.226+0000 7f8689889700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86840ce560 con 0x7f86840af8c0 2023-07-14T11:55:29.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.226+0000 7f8682ffd700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f868415fa10 0x7f868415fde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:39404/0 (socket says 172.21.15.104:39404) 2023-07-14T11:55:29.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.226+0000 7f8682ffd700 1 -- 172.21.15.104:0/2376886720 learned_addr learned my addr 172.21.15.104:0/2376886720 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:29.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.226+0000 7f86837fe700 1 --2- 172.21.15.104:0/2376886720 >> v2:172.21.15.121:3300/0 conn(0x7f86840af8c0 0x7f86840afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:29.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.226+0000 7f8681ffb700 1 -- 172.21.15.104:0/2376886720 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206121520 0 0) 0x7f86840d0930 con 0x7f868415fa10 2023-07-14T11:55:29.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.226+0000 7f8681ffb700 1 -- 172.21.15.104:0/2376886720 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f866c0035d0 con 0x7f868415fa10 2023-07-14T11:55:29.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.226+0000 7f8681ffb700 1 -- 172.21.15.104:0/2376886720 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1793385071 0 0) 0x7f8678001110 con 0x7f86841604f0 2023-07-14T11:55:29.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.226+0000 7f8681ffb700 1 -- 172.21.15.104:0/2376886720 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86840d0930 con 0x7f86841604f0 2023-07-14T11:55:29.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.226+0000 7f86837fe700 1 -- 172.21.15.104:0/2376886720 >> v1:172.21.15.121:6789/0 conn(0x7f86841604f0 legacy=0x7f86840af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:29.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.226+0000 7f86837fe700 1 -- 172.21.15.104:0/2376886720 >> v1:172.21.15.104:6789/0 conn(0x7f868415fa10 legacy=0x7f868415fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:29.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.226+0000 7f86837fe700 1 -- 172.21.15.104:0/2376886720 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868415c300 con 0x7f86840af8c0 2023-07-14T11:55:29.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.226+0000 7f86837fe700 1 --2- 172.21.15.104:0/2376886720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86840af8c0 0x7f86840afd40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f867c009d90 tx=0x7f867c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ac61d6b3e53f7b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:29.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.230+0000 7f8681ffb700 1 -- 172.21.15.104:0/2376886720 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00e070 con 0x7f86840af8c0 2023-07-14T11:55:29.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.230+0000 7f8681ffb700 1 -- 172.21.15.104:0/2376886720 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c00bb20 con 0x7f86840af8c0 2023-07-14T11:55:29.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.230+0000 7f8681ffb700 1 -- 172.21.15.104:0/2376886720 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c012430 con 0x7f86840af8c0 2023-07-14T11:55:29.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.230+0000 7f8689889700 1 -- 172.21.15.104:0/2376886720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86840af8c0 msgr2=0x7f86840afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:29.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.230+0000 7f8689889700 1 --2- 172.21.15.104:0/2376886720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86840af8c0 0x7f86840afd40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f867c009d90 tx=0x7f867c00b330 comp rx=0 tx=0).stop 2023-07-14T11:55:29.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.230+0000 7f8689889700 1 -- 172.21.15.104:0/2376886720 shutdown_connections 2023-07-14T11:55:29.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.230+0000 7f8689889700 1 --2- 172.21.15.104:0/2376886720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86840af8c0 0x7f86840afd40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.230+0000 7f8689889700 1 -- 172.21.15.104:0/2376886720 >> 172.21.15.104:0/2376886720 conn(0x7f868409b850 msgr2=0x7f86840d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:29.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.230+0000 7f8689889700 1 -- 172.21.15.104:0/2376886720 shutdown_connections 2023-07-14T11:55:29.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.230+0000 7f8689889700 1 -- 172.21.15.104:0/2376886720 wait complete. 2023-07-14T11:55:29.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.230+0000 7f8689889700 1 Processor -- start 2023-07-14T11:55:29.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.230+0000 7f8689889700 1 -- start start 2023-07-14T11:55:29.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8689889700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86840af8c0 0x7f86841f2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:29.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8689889700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868415fa10 0x7f8684153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:29.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8689889700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86841604f0 0x7f868415e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:29.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8689889700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86841f1bf0 con 0x7f86840af8c0 2023-07-14T11:55:29.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8689889700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8684153fa0 con 0x7f868415fa10 2023-07-14T11:55:29.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8689889700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86840b4270 con 0x7f86841604f0 2023-07-14T11:55:29.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8682ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86840af8c0 0x7f86841f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:29.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8682ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86840af8c0 0x7f86841f2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41416/0 (socket says 172.21.15.104:41416) 2023-07-14T11:55:29.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8682ffd700 1 -- 172.21.15.104:0/3478665308 learned_addr learned my addr 172.21.15.104:0/3478665308 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:29.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f86827fc700 1 --2- 172.21.15.104:0/3478665308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868415fa10 0x7f8684153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:29.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f86837fe700 1 --2- 172.21.15.104:0/3478665308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86841604f0 0x7f868415e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:29.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8682ffd700 1 -- 172.21.15.104:0/3478665308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86841604f0 msgr2=0x7f868415e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:29.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8682ffd700 1 --2- 172.21.15.104:0/3478665308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86841604f0 0x7f868415e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8682ffd700 1 -- 172.21.15.104:0/3478665308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868415fa10 msgr2=0x7f8684153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:29.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8682ffd700 1 --2- 172.21.15.104:0/3478665308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868415fa10 0x7f8684153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8682ffd700 1 -- 172.21.15.104:0/3478665308 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c009a70 con 0x7f86840af8c0 2023-07-14T11:55:29.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8682ffd700 1 --2- 172.21.15.104:0/3478665308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86840af8c0 0x7f86841f2080 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8674001eb0 tx=0x7f867400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:29.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f866bfff700 1 -- 172.21.15.104:0/3478665308 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86740074e0 con 0x7f86840af8c0 2023-07-14T11:55:29.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f866bfff700 1 -- 172.21.15.104:0/3478665308 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8674007640 con 0x7f86840af8c0 2023-07-14T11:55:29.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8689889700 1 -- 172.21.15.104:0/3478665308 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86840b4510 con 0x7f86840af8c0 2023-07-14T11:55:29.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f866bfff700 1 -- 172.21.15.104:0/3478665308 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674005a50 con 0x7f86840af8c0 2023-07-14T11:55:29.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.234+0000 7f8689889700 1 -- 172.21.15.104:0/3478665308 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86841f15a0 con 0x7f86840af8c0 2023-07-14T11:55:29.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.238+0000 7f8689889700 1 -- 172.21.15.104:0/3478665308 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86840d3430 con 0x7f86840af8c0 2023-07-14T11:55:29.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.238+0000 7f866bfff700 1 -- 172.21.15.104:0/3478665308 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8674005bb0 con 0x7f86840af8c0 2023-07-14T11:55:29.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.238+0000 7f866bfff700 1 --2- 172.21.15.104:0/3478665308 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86700662f0 0x7f86700687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:29.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.238+0000 7f866bfff700 1 -- 172.21.15.104:0/3478665308 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f8674083330 con 0x7f86840af8c0 2023-07-14T11:55:29.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.238+0000 7f86827fc700 1 --2- 172.21.15.104:0/3478665308 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86700662f0 0x7f86700687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:29.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.242+0000 7f86827fc700 1 --2- 172.21.15.104:0/3478665308 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86700662f0 0x7f86700687a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f868406e160 tx=0x7f8678009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:29.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.250+0000 7f866bfff700 1 -- 172.21.15.104:0/3478665308 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f86740504f0 con 0x7f86840af8c0 2023-07-14T11:55:29.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.418+0000 7f8689889700 1 -- 172.21.15.104:0/3478665308 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f868404e500 con 0x7f86840af8c0 2023-07-14T11:55:29.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.418+0000 7f866bfff700 1 -- 172.21.15.104:0/3478665308 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8674017020 con 0x7f86840af8c0 2023-07-14T11:55:29.405 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:55:29.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.422+0000 7f8689889700 1 -- 172.21.15.104:0/3478665308 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86700662f0 msgr2=0x7f86700687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:29.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.422+0000 7f8689889700 1 --2- 172.21.15.104:0/3478665308 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86700662f0 0x7f86700687a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f868406e160 tx=0x7f8678009040 comp rx=0 tx=0).stop 2023-07-14T11:55:29.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.422+0000 7f8689889700 1 -- 172.21.15.104:0/3478665308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86840af8c0 msgr2=0x7f86841f2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:29.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.422+0000 7f8689889700 1 --2- 172.21.15.104:0/3478665308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86840af8c0 0x7f86841f2080 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8674001eb0 tx=0x7f867400c7b0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.422+0000 7f8689889700 1 -- 172.21.15.104:0/3478665308 shutdown_connections 2023-07-14T11:55:29.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.422+0000 7f8689889700 1 --2- 172.21.15.104:0/3478665308 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86700662f0 0x7f86700687a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.422+0000 7f8689889700 1 --2- 172.21.15.104:0/3478665308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86841604f0 0x7f868415e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.422+0000 7f8689889700 1 --2- 172.21.15.104:0/3478665308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868415fa10 0x7f8684153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.422+0000 7f8689889700 1 --2- 172.21.15.104:0/3478665308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86840af8c0 0x7f86841f2080 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.422+0000 7f8689889700 1 -- 172.21.15.104:0/3478665308 >> 172.21.15.104:0/3478665308 conn(0x7f868409b850 msgr2=0x7f868415fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:29.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.422+0000 7f8689889700 1 -- 172.21.15.104:0/3478665308 shutdown_connections 2023-07-14T11:55:29.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.422+0000 7f8689889700 1 -- 172.21.15.104:0/3478665308 wait complete. 2023-07-14T11:55:29.431 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:55:29.609 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.622+0000 7f55877bf700 1 Processor -- start 2023-07-14T11:55:29.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.622+0000 7f55877bf700 1 -- start start 2023-07-14T11:55:29.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.626+0000 7f55877bf700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015df10 0x7f558015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:29.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.626+0000 7f55877bf700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f55800c8950 0x7f55800c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:29.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.626+0000 7f55877bf700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5580070a30 con 0x7f55800afd80 2023-07-14T11:55:29.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.626+0000 7f55877bf700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55800c92f0 con 0x7f55800c8950 2023-07-14T11:55:29.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.626+0000 7f55877bf700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558015e830 con 0x7f558015df10 2023-07-14T11:55:29.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.626+0000 7f5584d5a700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f55800afd80 0x7f55800711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:39418/0 (socket says 172.21.15.104:39418) 2023-07-14T11:55:29.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.626+0000 7f5584d5a700 1 -- 172.21.15.104:0/457355639 learned_addr learned my addr 172.21.15.104:0/457355639 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:29.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.626+0000 7f5585d5c700 1 --2- 172.21.15.104:0/457355639 >> v2:172.21.15.104:3300/0 conn(0x7f55800c8950 0x7f55800c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:29.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.626+0000 7f558555b700 1 --2- 172.21.15.104:0/457355639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015df10 0x7f558015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:29.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.626+0000 7f5577fff700 1 -- 172.21.15.104:0/457355639 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441752465 0 0) 0x7f5580070a30 con 0x7f55800afd80 2023-07-14T11:55:29.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.626+0000 7f5577fff700 1 -- 172.21.15.104:0/457355639 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f556c0035d0 con 0x7f55800afd80 2023-07-14T11:55:29.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.626+0000 7f5585d5c700 1 -- 172.21.15.104:0/457355639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015df10 msgr2=0x7f558015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:29.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.626+0000 7f5585d5c700 1 --2- 172.21.15.104:0/457355639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015df10 0x7f558015e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.626+0000 7f5585d5c700 1 -- 172.21.15.104:0/457355639 >> v1:172.21.15.104:6789/0 conn(0x7f55800afd80 legacy=0x7f55800711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:29.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.626+0000 7f5585d5c700 1 -- 172.21.15.104:0/457355639 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55800b0210 con 0x7f55800c8950 2023-07-14T11:55:29.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.630+0000 7f5585d5c700 1 --2- 172.21.15.104:0/457355639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55800c8950 0x7f55800c8db0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5578009d90 tx=0x7f557800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f7bcea9749416d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:29.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.630+0000 7f5577fff700 1 -- 172.21.15.104:0/457355639 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557800e070 con 0x7f55800c8950 2023-07-14T11:55:29.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.630+0000 7f5577fff700 1 -- 172.21.15.104:0/457355639 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557800bb20 con 0x7f55800c8950 2023-07-14T11:55:29.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.630+0000 7f5577fff700 1 -- 172.21.15.104:0/457355639 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5578012430 con 0x7f55800c8950 2023-07-14T11:55:29.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.630+0000 7f55877bf700 1 -- 172.21.15.104:0/457355639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55800c8950 msgr2=0x7f55800c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:29.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.630+0000 7f55877bf700 1 --2- 172.21.15.104:0/457355639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55800c8950 0x7f55800c8db0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5578009d90 tx=0x7f557800b330 comp rx=0 tx=0).stop 2023-07-14T11:55:29.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.630+0000 7f55877bf700 1 -- 172.21.15.104:0/457355639 shutdown_connections 2023-07-14T11:55:29.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.630+0000 7f55877bf700 1 --2- 172.21.15.104:0/457355639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55800c8950 0x7f55800c8db0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.630+0000 7f55877bf700 1 --2- 172.21.15.104:0/457355639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015df10 0x7f558015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.630+0000 7f55877bf700 1 -- 172.21.15.104:0/457355639 >> 172.21.15.104:0/457355639 conn(0x7f55800cad80 msgr2=0x7f55800cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:29.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.630+0000 7f55877bf700 1 -- 172.21.15.104:0/457355639 shutdown_connections 2023-07-14T11:55:29.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.630+0000 7f55877bf700 1 -- 172.21.15.104:0/457355639 wait complete. 2023-07-14T11:55:29.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.630+0000 7f55877bf700 1 Processor -- start 2023-07-14T11:55:29.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.630+0000 7f55877bf700 1 -- start start 2023-07-14T11:55:29.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f55877bf700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55800afd80 0x7f558015c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:29.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f55877bf700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55800c8950 0x7f5580147300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:29.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f55877bf700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5580158100 0x7f5580147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:29.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f55877bf700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5580157ce0 con 0x7f55800afd80 2023-07-14T11:55:29.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f55877bf700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5580157340 con 0x7f5580158100 2023-07-14T11:55:29.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f55877bf700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5580157620 con 0x7f55800c8950 2023-07-14T11:55:29.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f558555b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55800afd80 0x7f558015c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:29.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f558555b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55800afd80 0x7f558015c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41438/0 (socket says 172.21.15.104:41438) 2023-07-14T11:55:29.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f558555b700 1 -- 172.21.15.104:0/1401558995 learned_addr learned my addr 172.21.15.104:0/1401558995 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:29.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f5584d5a700 1 --2- 172.21.15.104:0/1401558995 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55800c8950 0x7f5580147300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:29.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f5585d5c700 1 --2- 172.21.15.104:0/1401558995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5580158100 0x7f5580147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:29.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f558555b700 1 -- 172.21.15.104:0/1401558995 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55800c8950 msgr2=0x7f5580147300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:29.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f558555b700 1 --2- 172.21.15.104:0/1401558995 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55800c8950 0x7f5580147300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f558555b700 1 -- 172.21.15.104:0/1401558995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5580158100 msgr2=0x7f5580147840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:29.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f558555b700 1 --2- 172.21.15.104:0/1401558995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5580158100 0x7f5580147840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f558555b700 1 -- 172.21.15.104:0/1401558995 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5578009a70 con 0x7f55800afd80 2023-07-14T11:55:29.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f5585d5c700 1 --2- 172.21.15.104:0/1401558995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5580158100 0x7f5580147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:29.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f558555b700 1 --2- 172.21.15.104:0/1401558995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55800afd80 0x7f558015c130 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f557c00e840 tx=0x7f557c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:29.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f5575ffb700 1 -- 172.21.15.104:0/1401558995 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c0095a0 con 0x7f55800afd80 2023-07-14T11:55:29.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f5575ffb700 1 -- 172.21.15.104:0/1401558995 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557c013680 con 0x7f55800afd80 2023-07-14T11:55:29.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f55877bf700 1 -- 172.21.15.104:0/1401558995 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5580157900 con 0x7f55800afd80 2023-07-14T11:55:29.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f5575ffb700 1 -- 172.21.15.104:0/1401558995 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c0139f0 con 0x7f55800afd80 2023-07-14T11:55:29.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.634+0000 7f55877bf700 1 -- 172.21.15.104:0/1401558995 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558015c850 con 0x7f55800afd80 2023-07-14T11:55:29.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.638+0000 7f55877bf700 1 -- 172.21.15.104:0/1401558995 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f558004dcb0 con 0x7f55800afd80 2023-07-14T11:55:29.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.638+0000 7f5575ffb700 1 -- 172.21.15.104:0/1401558995 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f557c013b50 con 0x7f55800afd80 2023-07-14T11:55:29.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.642+0000 7f5575ffb700 1 --2- 172.21.15.104:0/1401558995 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5564066320 0x7f55640687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:29.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.642+0000 7f5575ffb700 1 -- 172.21.15.104:0/1401558995 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f557c083d10 con 0x7f55800afd80 2023-07-14T11:55:29.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.642+0000 7f5584d5a700 1 --2- 172.21.15.104:0/1401558995 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5564066320 0x7f55640687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:29.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.642+0000 7f5584d5a700 1 --2- 172.21.15.104:0/1401558995 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5564066320 0x7f55640687d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f55800b3200 tx=0x7f5570007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:29.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.646+0000 7f5575ffb700 1 -- 172.21.15.104:0/1401558995 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f557c089020 con 0x7f55800afd80 2023-07-14T11:55:29.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.830+0000 7f55877bf700 1 -- 172.21.15.104:0/1401558995 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f558015c850 con 0x7f55800afd80 2023-07-14T11:55:29.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.830+0000 7f5575ffb700 1 -- 172.21.15.104:0/1401558995 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7f558015c850 con 0x7f55800afd80 2023-07-14T11:55:29.818 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:55:29.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.834+0000 7f55877bf700 1 -- 172.21.15.104:0/1401558995 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5564066320 msgr2=0x7f55640687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:29.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.834+0000 7f55877bf700 1 --2- 172.21.15.104:0/1401558995 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5564066320 0x7f55640687d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f55800b3200 tx=0x7f5570007de0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.834+0000 7f55877bf700 1 -- 172.21.15.104:0/1401558995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55800afd80 msgr2=0x7f558015c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:29.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.834+0000 7f55877bf700 1 --2- 172.21.15.104:0/1401558995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55800afd80 0x7f558015c130 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f557c00e840 tx=0x7f557c00eb50 comp rx=0 tx=0).stop 2023-07-14T11:55:29.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.834+0000 7f55877bf700 1 -- 172.21.15.104:0/1401558995 shutdown_connections 2023-07-14T11:55:29.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.834+0000 7f55877bf700 1 --2- 172.21.15.104:0/1401558995 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5564066320 0x7f55640687d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.834+0000 7f55877bf700 1 --2- 172.21.15.104:0/1401558995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5580158100 0x7f5580147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.834+0000 7f55877bf700 1 --2- 172.21.15.104:0/1401558995 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55800c8950 0x7f5580147300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.834+0000 7f55877bf700 1 --2- 172.21.15.104:0/1401558995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55800afd80 0x7f558015c130 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:29.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.834+0000 7f55877bf700 1 -- 172.21.15.104:0/1401558995 >> 172.21.15.104:0/1401558995 conn(0x7f55800cad80 msgr2=0x7f55800cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:29.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.834+0000 7f55877bf700 1 -- 172.21.15.104:0/1401558995 shutdown_connections 2023-07-14T11:55:29.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:29.834+0000 7f55877bf700 1 -- 172.21.15.104:0/1401558995 wait complete. 2023-07-14T11:55:29.844 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:55:30.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.042+0000 7f4efe94d700 1 Processor -- start 2023-07-14T11:55:30.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.042+0000 7f4efe94d700 1 -- start start 2023-07-14T11:55:30.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4efe94d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef80cdcb0 0x7f4ef8071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4efe94d700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef8158970 con 0x7f4ef80c9270 2023-07-14T11:55:30.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4efe94d700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef80ce080 con 0x7f4ef80b0680 2023-07-14T11:55:30.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4efe94d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef8158bf0 con 0x7f4ef80cdcb0 2023-07-14T11:55:30.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4efceea700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f4ef80c9270 0x7f4ef80cc080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:39430/0 (socket says 172.21.15.104:39430) 2023-07-14T11:55:30.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4efceea700 1 -- 172.21.15.104:0/320687760 learned_addr learned my addr 172.21.15.104:0/320687760 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:30.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef77fe700 1 -- 172.21.15.104:0/320687760 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1740011041 0 0) 0x7f4ef8158970 con 0x7f4ef80c9270 2023-07-14T11:55:30.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef77fe700 1 -- 172.21.15.104:0/320687760 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ee40035d0 con 0x7f4ef80c9270 2023-07-14T11:55:30.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef7fff700 1 --2- 172.21.15.104:0/320687760 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef80cdcb0 0x7f4ef8071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef77fe700 1 -- 172.21.15.104:0/320687760 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2723016455 0 0) 0x7f4ee40035d0 con 0x7f4ef80c9270 2023-07-14T11:55:30.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef77fe700 1 -- 172.21.15.104:0/320687760 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef80cdcb0 msgr2=0x7f4ef8071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:30.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef77fe700 1 --2- 172.21.15.104:0/320687760 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef80cdcb0 0x7f4ef8071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef77fe700 1 -- 172.21.15.104:0/320687760 >> v1:172.21.15.121:6789/0 conn(0x7f4ef80b0680 legacy=0x7f4ef80c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:30.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef77fe700 1 -- 172.21.15.104:0/320687760 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef8158ed0 con 0x7f4ef80c9270 2023-07-14T11:55:30.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef77fe700 1 -- 172.21.15.104:0/320687760 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f4ef00019f0 con 0x7f4ef80c9270 2023-07-14T11:55:30.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef77fe700 1 -- 172.21.15.104:0/320687760 >> v1:172.21.15.104:6789/0 conn(0x7f4ef80c9270 legacy=0x7f4ef80cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef77fe700 1 --2- 172.21.15.104:0/320687760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee4003a50 0x7f4ee4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef77fe700 1 --2- 172.21.15.104:0/320687760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee4004870 0x7f4ee4004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef77fe700 1 --2- 172.21.15.104:0/320687760 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef80cdcb0 0x7f4ee4005a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef77fe700 1 -- 172.21.15.104:0/320687760 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef8158bf0 con 0x7f4ee4003a50 2023-07-14T11:55:30.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef77fe700 1 -- 172.21.15.104:0/320687760 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40055a0 con 0x7f4ee4004870 2023-07-14T11:55:30.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef77fe700 1 -- 172.21.15.104:0/320687760 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee4005f40 con 0x7f4ef80cdcb0 2023-07-14T11:55:30.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef7fff700 1 --2- 172.21.15.104:0/320687760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee4003a50 0x7f4ee4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4eeffff700 1 --2- 172.21.15.104:0/320687760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee4004870 0x7f4ee4004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4efceea700 1 --2- 172.21.15.104:0/320687760 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef80cdcb0 0x7f4ee4005a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef7fff700 1 -- 172.21.15.104:0/320687760 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef80cdcb0 msgr2=0x7f4ee4005a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:30.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef7fff700 1 --2- 172.21.15.104:0/320687760 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef80cdcb0 0x7f4ee4005a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef7fff700 1 -- 172.21.15.104:0/320687760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee4004870 msgr2=0x7f4ee4004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:30.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef7fff700 1 --2- 172.21.15.104:0/320687760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee4004870 0x7f4ee4004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.046+0000 7f4ef7fff700 1 -- 172.21.15.104:0/320687760 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee40035d0 con 0x7f4ee4003a50 2023-07-14T11:55:30.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.050+0000 7f4ef7fff700 1 --2- 172.21.15.104:0/320687760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee4003a50 0x7f4ee4003e60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80098b0 tx=0x7f4ee40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70528ddcb6d67c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:30.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.050+0000 7f4ef77fe700 1 -- 172.21.15.104:0/320687760 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee800c070 con 0x7f4ee4003a50 2023-07-14T11:55:30.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.050+0000 7f4ef77fe700 1 -- 172.21.15.104:0/320687760 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee8013610 con 0x7f4ee4003a50 2023-07-14T11:55:30.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.050+0000 7f4ef77fe700 1 -- 172.21.15.104:0/320687760 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee80139c0 con 0x7f4ee4003a50 2023-07-14T11:55:30.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.050+0000 7f4efe94d700 1 -- 172.21.15.104:0/320687760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee4003a50 msgr2=0x7f4ee4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.050+0000 7f4efe94d700 1 --2- 172.21.15.104:0/320687760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee4003a50 0x7f4ee4003e60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80098b0 tx=0x7f4ee40058a0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.050+0000 7f4efceea700 1 -- 172.21.15.104:0/320687760 reap_dead start 2023-07-14T11:55:30.043 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.050+0000 7f4efe94d700 1 -- 172.21.15.104:0/320687760 shutdown_connections 2023-07-14T11:55:30.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.050+0000 7f4efe94d700 1 -- 172.21.15.104:0/320687760 >> 172.21.15.104:0/320687760 conn(0x7f4ef80cad80 msgr2=0x7f4ef80cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:30.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.050+0000 7f4efe94d700 1 -- 172.21.15.104:0/320687760 shutdown_connections 2023-07-14T11:55:30.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.050+0000 7f4efe94d700 1 -- 172.21.15.104:0/320687760 wait complete. 2023-07-14T11:55:30.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4efe94d700 1 Processor -- start 2023-07-14T11:55:30.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4efe94d700 1 -- start start 2023-07-14T11:55:30.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4efe94d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee4003a50 0x7f4ef80b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.044 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4efe94d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef80b21b0 0x7f4ef814ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4efe94d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ef80c5d70 0x7f4ef80c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4efe94d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef814c250 con 0x7f4ee4003a50 2023-07-14T11:55:30.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4efe94d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef80c6710 con 0x7f4ef80c5d70 2023-07-14T11:55:30.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4efe94d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef80b3100 con 0x7f4ef80b21b0 2023-07-14T11:55:30.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4ef7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee4003a50 0x7f4ef80b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4ef7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee4003a50 0x7f4ef80b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41462/0 (socket says 172.21.15.104:41462) 2023-07-14T11:55:30.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4ef7fff700 1 -- 172.21.15.104:0/1423245370 learned_addr learned my addr 172.21.15.104:0/1423245370 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:30.045 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4eeffff700 1 --2- 172.21.15.104:0/1423245370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef80b21b0 0x7f4ef814ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4efceea700 1 --2- 172.21.15.104:0/1423245370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ef80c5d70 0x7f4ef80c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4ef7fff700 1 -- 172.21.15.104:0/1423245370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef80b21b0 msgr2=0x7f4ef814ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4ef7fff700 1 --2- 172.21.15.104:0/1423245370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef80b21b0 0x7f4ef814ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4ef7fff700 1 -- 172.21.15.104:0/1423245370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ef80c5d70 msgr2=0x7f4ef80c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4ef7fff700 1 --2- 172.21.15.104:0/1423245370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ef80c5d70 0x7f4ef80c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4ef7fff700 1 -- 172.21.15.104:0/1423245370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef80b33e0 con 0x7f4ee4003a50 2023-07-14T11:55:30.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.054+0000 7f4efceea700 1 --2- 172.21.15.104:0/1423245370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ef80c5d70 0x7f4ef80c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:55:30.046 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.058+0000 7f4ef7fff700 1 --2- 172.21.15.104:0/1423245370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee4003a50 0x7f4ef80b2a90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8000c00 tx=0x7f4ee80047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:30.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.058+0000 7f4ef57fa700 1 -- 172.21.15.104:0/1423245370 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee801c630 con 0x7f4ee4003a50 2023-07-14T11:55:30.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.058+0000 7f4ef57fa700 1 -- 172.21.15.104:0/1423245370 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee801c7b0 con 0x7f4ee4003a50 2023-07-14T11:55:30.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.058+0000 7f4ef57fa700 1 -- 172.21.15.104:0/1423245370 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee801cb60 con 0x7f4ee4003a50 2023-07-14T11:55:30.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.058+0000 7f4efe94d700 1 -- 172.21.15.104:0/1423245370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef80b36c0 con 0x7f4ee4003a50 2023-07-14T11:55:30.047 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.058+0000 7f4efe94d700 1 -- 172.21.15.104:0/1423245370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef80b3c80 con 0x7f4ee4003a50 2023-07-14T11:55:30.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.058+0000 7f4efe94d700 1 -- 172.21.15.104:0/1423245370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef8061060 con 0x7f4ee4003a50 2023-07-14T11:55:30.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.058+0000 7f4ef57fa700 1 -- 172.21.15.104:0/1423245370 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f4ee801ccc0 con 0x7f4ee4003a50 2023-07-14T11:55:30.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.062+0000 7f4ef57fa700 1 --2- 172.21.15.104:0/1423245370 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4ed8066320 0x7f4ed80687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.062+0000 7f4ef57fa700 1 -- 172.21.15.104:0/1423245370 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f4ee8085070 con 0x7f4ee4003a50 2023-07-14T11:55:30.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.062+0000 7f4eeffff700 1 --2- 172.21.15.104:0/1423245370 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4ed8066320 0x7f4ed80687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.062+0000 7f4eeffff700 1 --2- 172.21.15.104:0/1423245370 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4ed8066320 0x7f4ed80687d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0000c00 tx=0x7f4ee0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:30.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.070+0000 7f4ef57fa700 1 -- 172.21.15.104:0/1423245370 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f4ee80522b0 con 0x7f4ee4003a50 2023-07-14T11:55:30.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.246+0000 7f4efe94d700 1 -- 172.21.15.104:0/1423245370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4ef80b3e10 con 0x7f4ee4003a50 2023-07-14T11:55:30.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.246+0000 7f4ef57fa700 1 -- 172.21.15.104:0/1423245370 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4ee80170a0 con 0x7f4ee4003a50 2023-07-14T11:55:30.233 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:55:30.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.250+0000 7f4efe94d700 1 -- 172.21.15.104:0/1423245370 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4ed8066320 msgr2=0x7f4ed80687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.250+0000 7f4efe94d700 1 --2- 172.21.15.104:0/1423245370 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4ed8066320 0x7f4ed80687d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0000c00 tx=0x7f4ee0009040 comp rx=0 tx=0).stop 2023-07-14T11:55:30.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.250+0000 7f4efe94d700 1 -- 172.21.15.104:0/1423245370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee4003a50 msgr2=0x7f4ef80b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.250+0000 7f4efe94d700 1 --2- 172.21.15.104:0/1423245370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee4003a50 0x7f4ef80b2a90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8000c00 tx=0x7f4ee80047e0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.250+0000 7f4efe94d700 1 -- 172.21.15.104:0/1423245370 shutdown_connections 2023-07-14T11:55:30.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.250+0000 7f4efe94d700 1 --2- 172.21.15.104:0/1423245370 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4ed8066320 0x7f4ed80687d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.250+0000 7f4efe94d700 1 --2- 172.21.15.104:0/1423245370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ef80c5d70 0x7f4ef80c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.250+0000 7f4efe94d700 1 --2- 172.21.15.104:0/1423245370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ef80b21b0 0x7f4ef814ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.250+0000 7f4efe94d700 1 --2- 172.21.15.104:0/1423245370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee4003a50 0x7f4ef80b2a90 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.250+0000 7f4efe94d700 1 -- 172.21.15.104:0/1423245370 >> 172.21.15.104:0/1423245370 conn(0x7f4ef80cad80 msgr2=0x7f4ef806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:30.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.250+0000 7f4efe94d700 1 -- 172.21.15.104:0/1423245370 shutdown_connections 2023-07-14T11:55:30.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.250+0000 7f4efe94d700 1 -- 172.21.15.104:0/1423245370 wait complete. 2023-07-14T11:55:30.254 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-07-14T11:55:30.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.446+0000 7f54b3865700 1 Processor -- start 2023-07-14T11:55:30.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.446+0000 7f54b3865700 1 -- start start 2023-07-14T11:55:30.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54b3865700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f54ac0afd80 0x7f54ac15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54b3865700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f54ac0c8950 0x7f54ac0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54b3865700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ac0c9290 con 0x7f54ac15df10 2023-07-14T11:55:30.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54b3865700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ac15e830 con 0x7f54ac0c8950 2023-07-14T11:55:30.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54b3865700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ac070a30 con 0x7f54ac0afd80 2023-07-14T11:55:30.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54b1601700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f54ac15df10 0x7f54ac0711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:59558/0 (socket says 172.21.15.104:59558) 2023-07-14T11:55:30.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54b1601700 1 -- 172.21.15.104:0/4257599326 learned_addr learned my addr 172.21.15.104:0/4257599326 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:30.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54b0e00700 1 --2- 172.21.15.104:0/4257599326 >> v2:172.21.15.121:3300/0 conn(0x7f54ac0afd80 0x7f54ac15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54b1e02700 1 --2- 172.21.15.104:0/4257599326 >> v2:172.21.15.104:3300/0 conn(0x7f54ac0c8950 0x7f54ac0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54a3fff700 1 -- 172.21.15.104:0/4257599326 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 694241616 0 0) 0x7f54a8001110 con 0x7f54ac15df10 2023-07-14T11:55:30.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54a3fff700 1 -- 172.21.15.104:0/4257599326 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54980035d0 con 0x7f54ac15df10 2023-07-14T11:55:30.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54b1e02700 1 -- 172.21.15.104:0/4257599326 >> v2:172.21.15.121:3300/0 conn(0x7f54ac0afd80 msgr2=0x7f54ac15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:30.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54b1e02700 1 --2- 172.21.15.104:0/4257599326 >> v2:172.21.15.121:3300/0 conn(0x7f54ac0afd80 0x7f54ac15e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54b1e02700 1 -- 172.21.15.104:0/4257599326 >> v1:172.21.15.121:6789/0 conn(0x7f54ac15df10 legacy=0x7f54ac0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54b1e02700 1 -- 172.21.15.104:0/4257599326 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac0b0210 con 0x7f54ac0c8950 2023-07-14T11:55:30.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54b1e02700 1 --2- 172.21.15.104:0/4257599326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54ac0c8950 0x7f54ac0c8d50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f54ac06bef0 tx=0x7f54a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f562a74e6a3efae server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:30.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54a3fff700 1 -- 172.21.15.104:0/4257599326 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a400e070 con 0x7f54ac0c8950 2023-07-14T11:55:30.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54a3fff700 1 -- 172.21.15.104:0/4257599326 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a400bb20 con 0x7f54ac0c8950 2023-07-14T11:55:30.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.450+0000 7f54a3fff700 1 -- 172.21.15.104:0/4257599326 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4012430 con 0x7f54ac0c8950 2023-07-14T11:55:30.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.454+0000 7f54b3865700 1 -- 172.21.15.104:0/4257599326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54ac0c8950 msgr2=0x7f54ac0c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.454+0000 7f54b3865700 1 --2- 172.21.15.104:0/4257599326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54ac0c8950 0x7f54ac0c8d50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f54ac06bef0 tx=0x7f54a400b330 comp rx=0 tx=0).stop 2023-07-14T11:55:30.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.454+0000 7f54b3865700 1 -- 172.21.15.104:0/4257599326 shutdown_connections 2023-07-14T11:55:30.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.454+0000 7f54b3865700 1 --2- 172.21.15.104:0/4257599326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54ac0c8950 0x7f54ac0c8d50 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.454+0000 7f54b3865700 1 --2- 172.21.15.104:0/4257599326 >> v2:172.21.15.121:3300/0 conn(0x7f54ac0afd80 0x7f54ac15e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.454+0000 7f54b3865700 1 -- 172.21.15.104:0/4257599326 >> 172.21.15.104:0/4257599326 conn(0x7f54ac0cad80 msgr2=0x7f54ac0cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:30.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.454+0000 7f54b3865700 1 -- 172.21.15.104:0/4257599326 shutdown_connections 2023-07-14T11:55:30.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.454+0000 7f54b3865700 1 -- 172.21.15.104:0/4257599326 wait complete. 2023-07-14T11:55:30.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b3865700 1 Processor -- start 2023-07-14T11:55:30.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b3865700 1 -- start start 2023-07-14T11:55:30.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b3865700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54ac0afd80 0x7f54ac154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b3865700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54ac0c8950 0x7f54ac15c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b3865700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54ac15cb10 0x7f54ac0cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b3865700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac0c5f80 con 0x7f54ac0afd80 2023-07-14T11:55:30.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b3865700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac15cfb0 con 0x7f54ac15cb10 2023-07-14T11:55:30.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b3865700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac0c53d0 con 0x7f54ac0c8950 2023-07-14T11:55:30.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b1601700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54ac0afd80 0x7f54ac154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b1601700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54ac0afd80 0x7f54ac154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:51482/0 (socket says 172.21.15.104:51482) 2023-07-14T11:55:30.447 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b1601700 1 -- 172.21.15.104:0/1024704087 learned_addr learned my addr 172.21.15.104:0/1024704087 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:30.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b1e02700 1 --2- 172.21.15.104:0/1024704087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54ac15cb10 0x7f54ac0cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b0e00700 1 --2- 172.21.15.104:0/1024704087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54ac0c8950 0x7f54ac15c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b1601700 1 -- 172.21.15.104:0/1024704087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54ac0c8950 msgr2=0x7f54ac15c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b1601700 1 --2- 172.21.15.104:0/1024704087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54ac0c8950 0x7f54ac15c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b1601700 1 -- 172.21.15.104:0/1024704087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54ac15cb10 msgr2=0x7f54ac0cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.449 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b1601700 1 --2- 172.21.15.104:0/1024704087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54ac15cb10 0x7f54ac0cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.449 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b1601700 1 -- 172.21.15.104:0/1024704087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a4009a70 con 0x7f54ac0afd80 2023-07-14T11:55:30.449 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.458+0000 7f54b1601700 1 --2- 172.21.15.104:0/1024704087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54ac0afd80 0x7f54ac154770 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f54a8001270 tx=0x7f54a800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:30.449 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.462+0000 7f54a1ffb700 1 -- 172.21.15.104:0/1024704087 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8005d90 con 0x7f54ac0afd80 2023-07-14T11:55:30.449 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.462+0000 7f54b3865700 1 -- 172.21.15.104:0/1024704087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ac0c55e0 con 0x7f54ac0afd80 2023-07-14T11:55:30.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.462+0000 7f54a1ffb700 1 -- 172.21.15.104:0/1024704087 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a80070d0 con 0x7f54ac0afd80 2023-07-14T11:55:30.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.462+0000 7f54a1ffb700 1 -- 172.21.15.104:0/1024704087 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8007480 con 0x7f54ac0afd80 2023-07-14T11:55:30.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.462+0000 7f54b3865700 1 -- 172.21.15.104:0/1024704087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ac0c5ba0 con 0x7f54ac0afd80 2023-07-14T11:55:30.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.462+0000 7f54b3865700 1 -- 172.21.15.104:0/1024704087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ac0c60c0 con 0x7f54ac0afd80 2023-07-14T11:55:30.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.462+0000 7f54a1ffb700 1 -- 172.21.15.104:0/1024704087 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f54a80075e0 con 0x7f54ac0afd80 2023-07-14T11:55:30.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.466+0000 7f54a1ffb700 1 --2- 172.21.15.104:0/1024704087 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f54900662f0 0x7f54900687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.466+0000 7f54a1ffb700 1 -- 172.21.15.104:0/1024704087 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f54a8083420 con 0x7f54ac0afd80 2023-07-14T11:55:30.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.466+0000 7f54b0e00700 1 --2- 172.21.15.104:0/1024704087 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f54900662f0 0x7f54900687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.466+0000 7f54b0e00700 1 --2- 172.21.15.104:0/1024704087 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f54900662f0 0x7f54900687a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0c9260 tx=0x7f549c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:30.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.470+0000 7f54a1ffb700 1 -- 172.21.15.104:0/1024704087 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f54ac0c60c0 con 0x7f54ac0afd80 2023-07-14T11:55:30.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.642+0000 7f54b3865700 1 -- 172.21.15.104:0/1024704087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f54ac0c9290 con 0x7f54ac0afd80 2023-07-14T11:55:30.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.642+0000 7f54a1ffb700 1 -- 172.21.15.104:0/1024704087 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v45) v1 ==== 97+0+11 (secure 0 0 0) 0x7f54ac0c9290 con 0x7f54ac0afd80 2023-07-14T11:55:30.629 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:55:30.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.646+0000 7f54b3865700 1 -- 172.21.15.104:0/1024704087 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f54900662f0 msgr2=0x7f54900687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.646+0000 7f54b3865700 1 --2- 172.21.15.104:0/1024704087 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f54900662f0 0x7f54900687a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0c9260 tx=0x7f549c008040 comp rx=0 tx=0).stop 2023-07-14T11:55:30.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.646+0000 7f54b3865700 1 -- 172.21.15.104:0/1024704087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54ac0afd80 msgr2=0x7f54ac154770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.646+0000 7f54b3865700 1 --2- 172.21.15.104:0/1024704087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54ac0afd80 0x7f54ac154770 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f54a8001270 tx=0x7f54a800ca90 comp rx=0 tx=0).stop 2023-07-14T11:55:30.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.646+0000 7f54b3865700 1 -- 172.21.15.104:0/1024704087 shutdown_connections 2023-07-14T11:55:30.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.646+0000 7f54b3865700 1 --2- 172.21.15.104:0/1024704087 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f54900662f0 0x7f54900687a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.646+0000 7f54b3865700 1 --2- 172.21.15.104:0/1024704087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54ac15cb10 0x7f54ac0cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.646+0000 7f54b3865700 1 --2- 172.21.15.104:0/1024704087 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54ac0c8950 0x7f54ac15c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.646+0000 7f54b3865700 1 --2- 172.21.15.104:0/1024704087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54ac0afd80 0x7f54ac154770 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.646+0000 7f54b3865700 1 -- 172.21.15.104:0/1024704087 >> 172.21.15.104:0/1024704087 conn(0x7f54ac0cad80 msgr2=0x7f54ac0af590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:30.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.646+0000 7f54b3865700 1 -- 172.21.15.104:0/1024704087 shutdown_connections 2023-07-14T11:55:30.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.646+0000 7f54b3865700 1 -- 172.21.15.104:0/1024704087 wait complete. 2023-07-14T11:55:30.652 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:55:30.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.838+0000 7feb65051700 1 Processor -- start 2023-07-14T11:55:30.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.838+0000 7feb65051700 1 -- start start 2023-07-14T11:55:30.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.838+0000 7feb65051700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb600afd30 0x7feb600b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.838+0000 7feb65051700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb600c8900 con 0x7feb600cddb0 2023-07-14T11:55:30.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.838+0000 7feb65051700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb600caae0 con 0x7feb60160640 2023-07-14T11:55:30.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.838+0000 7feb65051700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb600c8fc0 con 0x7feb600afd30 2023-07-14T11:55:30.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.838+0000 7feb57fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7feb600cddb0 0x7feb60070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:45176/0 (socket says 172.21.15.104:45176) 2023-07-14T11:55:30.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.838+0000 7feb57fff700 1 -- 172.21.15.104:0/1552276528 learned_addr learned my addr 172.21.15.104:0/1552276528 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:30.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.838+0000 7feb5f59e700 1 --2- 172.21.15.104:0/1552276528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb600afd30 0x7feb600b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647895755 0 0) 0x7feb600c8900 con 0x7feb600cddb0 2023-07-14T11:55:30.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb4c0035d0 con 0x7feb600cddb0 2023-07-14T11:55:30.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905748474 0 0) 0x7feb600caae0 con 0x7feb60160640 2023-07-14T11:55:30.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb600c8900 con 0x7feb60160640 2023-07-14T11:55:30.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3911300563 0 0) 0x7feb4c0035d0 con 0x7feb600cddb0 2023-07-14T11:55:30.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb600afd30 msgr2=0x7feb600b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:30.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 --2- 172.21.15.104:0/1552276528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb600afd30 0x7feb600b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 >> v1:172.21.15.121:6789/0 conn(0x7feb60160640 legacy=0x7feb60164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb600ca520 con 0x7feb600cddb0 2023-07-14T11:55:30.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7feb48001670 con 0x7feb600cddb0 2023-07-14T11:55:30.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 >> v1:172.21.15.104:6789/0 conn(0x7feb600cddb0 legacy=0x7feb60070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 --2- 172.21.15.104:0/1552276528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb4c004c40 0x7feb4c005030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 --2- 172.21.15.104:0/1552276528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb4c003810 0x7feb4c003c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 --2- 172.21.15.104:0/1552276528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb600afd30 0x7feb4c0041c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb600c8fc0 con 0x7feb4c003810 2023-07-14T11:55:30.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c00bb10 con 0x7feb4c004c40 2023-07-14T11:55:30.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c00bcd0 con 0x7feb600afd30 2023-07-14T11:55:30.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb57fff700 1 --2- 172.21.15.104:0/1552276528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb4c003810 0x7feb4c003c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5ed9d700 1 --2- 172.21.15.104:0/1552276528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb4c004c40 0x7feb4c005030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5f59e700 1 --2- 172.21.15.104:0/1552276528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb600afd30 0x7feb4c0041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb57fff700 1 -- 172.21.15.104:0/1552276528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb600afd30 msgr2=0x7feb4c0041c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:30.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb57fff700 1 --2- 172.21.15.104:0/1552276528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb600afd30 0x7feb4c0041c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb57fff700 1 -- 172.21.15.104:0/1552276528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb4c004c40 msgr2=0x7feb4c005030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb57fff700 1 --2- 172.21.15.104:0/1552276528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb4c004c40 0x7feb4c005030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb57fff700 1 -- 172.21.15.104:0/1552276528 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c0035d0 con 0x7feb4c003810 2023-07-14T11:55:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb57fff700 1 --2- 172.21.15.104:0/1552276528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb4c003810 0x7feb4c003c80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7feb600ce1e0 tx=0x7feb4c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e99755b3f6bdb1e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4800cd80 con 0x7feb4c003810 2023-07-14T11:55:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb48005980 con 0x7feb4c003810 2023-07-14T11:55:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.842+0000 7feb5e59c700 1 -- 172.21.15.104:0/1552276528 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48005d30 con 0x7feb4c003810 2023-07-14T11:55:30.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.846+0000 7feb65051700 1 -- 172.21.15.104:0/1552276528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb4c003810 msgr2=0x7feb4c003c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.846+0000 7feb65051700 1 --2- 172.21.15.104:0/1552276528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb4c003810 0x7feb4c003c80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7feb600ce1e0 tx=0x7feb4c0058a0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.846+0000 7feb5f59e700 1 -- 172.21.15.104:0/1552276528 reap_dead start 2023-07-14T11:55:30.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.846+0000 7feb65051700 1 -- 172.21.15.104:0/1552276528 shutdown_connections 2023-07-14T11:55:30.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.846+0000 7feb65051700 1 -- 172.21.15.104:0/1552276528 >> 172.21.15.104:0/1552276528 conn(0x7feb600cad30 msgr2=0x7feb600cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:30.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.846+0000 7feb65051700 1 -- 172.21.15.104:0/1552276528 shutdown_connections 2023-07-14T11:55:30.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.846+0000 7feb65051700 1 -- 172.21.15.104:0/1552276528 wait complete. 2023-07-14T11:55:30.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb65051700 1 Processor -- start 2023-07-14T11:55:30.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb65051700 1 -- start start 2023-07-14T11:55:30.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb65051700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb4c003810 0x7feb60157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb65051700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb60161870 0x7feb60161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb65051700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb600b10e0 0x7feb600b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb65051700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb60070540 con 0x7feb60161870 2023-07-14T11:55:30.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb65051700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb600b1a90 con 0x7feb4c003810 2023-07-14T11:55:30.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb65051700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb60162ef0 con 0x7feb600b10e0 2023-07-14T11:55:30.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb57fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb60161870 0x7feb60161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb57fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb60161870 0x7feb60161ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:51494/0 (socket says 172.21.15.104:51494) 2023-07-14T11:55:30.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb57fff700 1 -- 172.21.15.104:0/2689419041 learned_addr learned my addr 172.21.15.104:0/2689419041 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:30.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb5ed9d700 1 --2- 172.21.15.104:0/2689419041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb4c003810 0x7feb60157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb5f59e700 1 --2- 172.21.15.104:0/2689419041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb600b10e0 0x7feb600b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb57fff700 1 -- 172.21.15.104:0/2689419041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb600b10e0 msgr2=0x7feb600b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb57fff700 1 --2- 172.21.15.104:0/2689419041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb600b10e0 0x7feb600b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb57fff700 1 -- 172.21.15.104:0/2689419041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb4c003810 msgr2=0x7feb60157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:30.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb57fff700 1 --2- 172.21.15.104:0/2689419041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb4c003810 0x7feb60157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:30.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb57fff700 1 -- 172.21.15.104:0/2689419041 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb601622e0 con 0x7feb60161870 2023-07-14T11:55:30.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb5ed9d700 1 --2- 172.21.15.104:0/2689419041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb4c003810 0x7feb60157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:30.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb57fff700 1 --2- 172.21.15.104:0/2689419041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb60161870 0x7feb60161ce0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7feb4800c7d0 tx=0x7feb48001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:30.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb577fe700 1 -- 172.21.15.104:0/2689419041 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48005d00 con 0x7feb60161870 2023-07-14T11:55:30.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb577fe700 1 -- 172.21.15.104:0/2689419041 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb48005e60 con 0x7feb60161870 2023-07-14T11:55:30.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb65051700 1 -- 172.21.15.104:0/2689419041 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb601625c0 con 0x7feb60161870 2023-07-14T11:55:30.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb577fe700 1 -- 172.21.15.104:0/2689419041 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48004ad0 con 0x7feb60161870 2023-07-14T11:55:30.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.850+0000 7feb65051700 1 -- 172.21.15.104:0/2689419041 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb60162b80 con 0x7feb60161870 2023-07-14T11:55:30.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.854+0000 7feb65051700 1 -- 172.21.15.104:0/2689419041 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb6004dcb0 con 0x7feb60161870 2023-07-14T11:55:30.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.854+0000 7feb577fe700 1 -- 172.21.15.104:0/2689419041 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7feb48004c30 con 0x7feb60161870 2023-07-14T11:55:30.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.858+0000 7feb577fe700 1 --2- 172.21.15.104:0/2689419041 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feb400662f0 0x7feb400687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:30.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.858+0000 7feb577fe700 1 -- 172.21.15.104:0/2689419041 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7feb48085190 con 0x7feb60161870 2023-07-14T11:55:30.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.858+0000 7feb5ed9d700 1 --2- 172.21.15.104:0/2689419041 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feb400662f0 0x7feb400687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:30.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.858+0000 7feb5ed9d700 1 --2- 172.21.15.104:0/2689419041 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feb400662f0 0x7feb400687a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feb50007fd0 tx=0x7feb50009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:30.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:30.862+0000 7feb577fe700 1 -- 172.21.15.104:0/2689419041 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7feb48052330 con 0x7feb60161870 2023-07-14T11:55:31.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:31.050+0000 7feb65051700 1 -- 172.21.15.104:0/2689419041 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feb600cb9f0 con 0x7feb60161870 2023-07-14T11:55:31.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:31.050+0000 7feb577fe700 1 -- 172.21.15.104:0/2689419041 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) v1 ==== 80+0+0 (secure 0 0 0) 0x7feb48007d30 con 0x7feb60161870 2023-07-14T11:55:31.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:31.054+0000 7feb65051700 1 -- 172.21.15.104:0/2689419041 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feb400662f0 msgr2=0x7feb400687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:31.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:31.054+0000 7feb65051700 1 --2- 172.21.15.104:0/2689419041 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feb400662f0 0x7feb400687a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feb50007fd0 tx=0x7feb50009040 comp rx=0 tx=0).stop 2023-07-14T11:55:31.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:31.054+0000 7feb65051700 1 -- 172.21.15.104:0/2689419041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb60161870 msgr2=0x7feb60161ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:31.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:31.054+0000 7feb65051700 1 --2- 172.21.15.104:0/2689419041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb60161870 0x7feb60161ce0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7feb4800c7d0 tx=0x7feb48001980 comp rx=0 tx=0).stop 2023-07-14T11:55:31.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:31.054+0000 7feb65051700 1 -- 172.21.15.104:0/2689419041 shutdown_connections 2023-07-14T11:55:31.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:31.054+0000 7feb65051700 1 --2- 172.21.15.104:0/2689419041 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feb400662f0 0x7feb400687a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:31.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:31.054+0000 7feb65051700 1 --2- 172.21.15.104:0/2689419041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb60161870 0x7feb60161ce0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:31.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:31.054+0000 7feb65051700 1 --2- 172.21.15.104:0/2689419041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb600b10e0 0x7feb600b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:31.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:31.054+0000 7feb65051700 1 --2- 172.21.15.104:0/2689419041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb4c003810 0x7feb60157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:31.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:31.054+0000 7feb65051700 1 -- 172.21.15.104:0/2689419041 >> 172.21.15.104:0/2689419041 conn(0x7feb600cad30 msgr2=0x7feb600add10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:31.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:31.054+0000 7feb65051700 1 -- 172.21.15.104:0/2689419041 shutdown_connections 2023-07-14T11:55:31.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:31.054+0000 7feb65051700 1 -- 172.21.15.104:0/2689419041 wait complete. 2023-07-14T11:55:31.043 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:55:31.059 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi104.front.sepia.ceph.com... 2023-07-14T11:55:31.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:31.060 DEBUG:teuthology.orchestra.run.smithi104:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:31.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[47662]: fuse finished with error 0 and tester_r 0 2023-07-14T11:55:31.091 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:55:37.193 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:55:37.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:37.194 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-14T11:55:37.215 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rwp' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2023-07-14T11:55:37.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.458+0000 7f80fca78700 1 Processor -- start 2023-07-14T11:55:37.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.458+0000 7f80fca78700 1 -- start start 2023-07-14T11:55:37.449 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.462+0000 7f80fca78700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f80f80b4130 0x7f80f80d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:37.449 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.462+0000 7f80fca78700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f80b1130 0x7f80f806fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:37.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.462+0000 7f80fca78700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f80b0ca0 con 0x7f80f815c1a0 2023-07-14T11:55:37.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.462+0000 7f80fca78700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f80d2530 con 0x7f80f80b4130 2023-07-14T11:55:37.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.462+0000 7f80fca78700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f80b03b0 con 0x7f80f80b1130 2023-07-14T11:55:37.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.462+0000 7f80f659c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f80f80b4130 0x7f80f80d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:37.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.462+0000 7f80f6d9d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f80b1130 0x7f80f806fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:37.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.462+0000 7f80f6d9d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f80b1130 0x7f80f806fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:35206/0 (socket says 172.21.15.104:35206) 2023-07-14T11:55:37.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.462+0000 7f80f659c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f80f80b4130 0x7f80f80d1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:54508/0 (socket says 172.21.15.104:54508) 2023-07-14T11:55:37.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.462+0000 7f80f6d9d700 1 -- 172.21.15.104:0/937622695 learned_addr learned my addr 172.21.15.104:0/937622695 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:37.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80f559a700 1 -- 172.21.15.104:0/937622695 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595242288 0 0) 0x7f80f80b0ca0 con 0x7f80f815c1a0 2023-07-14T11:55:37.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80f6d9d700 1 -- 172.21.15.104:0/937622695 >> v2:172.21.15.121:3300/0 conn(0x7f80f80b4130 msgr2=0x7f80f80d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:37.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80f6d9d700 1 --2- 172.21.15.104:0/937622695 >> v2:172.21.15.121:3300/0 conn(0x7f80f80b4130 0x7f80f80d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:37.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80f6d9d700 1 -- 172.21.15.104:0/937622695 >> v1:172.21.15.121:6789/0 conn(0x7f80f815c1a0 legacy=0x7f80f80b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:37.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80f6d9d700 1 -- 172.21.15.104:0/937622695 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f80b0650 con 0x7f80f80b1130 2023-07-14T11:55:37.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80f6d9d700 1 --2- 172.21.15.104:0/937622695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f80b1130 0x7f80f806fdd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009e50 tx=0x7f80f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3b8fe6bddae38e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:37.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80f559a700 1 -- 172.21.15.104:0/937622695 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f000e070 con 0x7f80f80b1130 2023-07-14T11:55:37.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80f559a700 1 -- 172.21.15.104:0/937622695 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f000b910 con 0x7f80f80b1130 2023-07-14T11:55:37.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80f559a700 1 -- 172.21.15.104:0/937622695 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f000bcc0 con 0x7f80f80b1130 2023-07-14T11:55:37.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80fca78700 1 -- 172.21.15.104:0/937622695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f80b1130 msgr2=0x7f80f806fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:37.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80fca78700 1 --2- 172.21.15.104:0/937622695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f80b1130 0x7f80f806fdd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009e50 tx=0x7f80f000b330 comp rx=0 tx=0).stop 2023-07-14T11:55:37.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80fca78700 1 -- 172.21.15.104:0/937622695 shutdown_connections 2023-07-14T11:55:37.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80fca78700 1 --2- 172.21.15.104:0/937622695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f80b1130 0x7f80f806fdd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:37.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80fca78700 1 --2- 172.21.15.104:0/937622695 >> v2:172.21.15.121:3300/0 conn(0x7f80f80b4130 0x7f80f80d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:37.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80fca78700 1 -- 172.21.15.104:0/937622695 >> 172.21.15.104:0/937622695 conn(0x7f80f809ba10 msgr2=0x7f80f80d26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:37.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80fca78700 1 -- 172.21.15.104:0/937622695 shutdown_connections 2023-07-14T11:55:37.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.466+0000 7f80fca78700 1 -- 172.21.15.104:0/937622695 wait complete. 2023-07-14T11:55:37.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80fca78700 1 Processor -- start 2023-07-14T11:55:37.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80fca78700 1 -- start start 2023-07-14T11:55:37.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80fca78700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80f80b1130 0x7f80f81559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:37.457 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80fca78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80f80b4130 0x7f80f80ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:37.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80fca78700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f815c1a0 0x7f80f80b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:37.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80fca78700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f80cc5d0 con 0x7f80f80b1130 2023-07-14T11:55:37.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80fca78700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f8155f00 con 0x7f80f80b4130 2023-07-14T11:55:37.458 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80fca78700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f80cb530 con 0x7f80f815c1a0 2023-07-14T11:55:37.459 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80f659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80f80b1130 0x7f80f81559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:37.459 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80f659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80f80b1130 0x7f80f81559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:51506/0 (socket says 172.21.15.104:51506) 2023-07-14T11:55:37.459 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80f659c700 1 -- 172.21.15.104:0/1567622656 learned_addr learned my addr 172.21.15.104:0/1567622656 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:37.459 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80f5d9b700 1 --2- 172.21.15.104:0/1567622656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80f80b4130 0x7f80f80ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:37.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80f6d9d700 1 --2- 172.21.15.104:0/1567622656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f815c1a0 0x7f80f80b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:37.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80f659c700 1 -- 172.21.15.104:0/1567622656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f815c1a0 msgr2=0x7f80f80b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:37.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80f659c700 1 --2- 172.21.15.104:0/1567622656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f815c1a0 0x7f80f80b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:37.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80f659c700 1 -- 172.21.15.104:0/1567622656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80f80b4130 msgr2=0x7f80f80ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:37.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.470+0000 7f80f659c700 1 --2- 172.21.15.104:0/1567622656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80f80b4130 0x7f80f80ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:37.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.474+0000 7f80f659c700 1 -- 172.21.15.104:0/1567622656 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f0009b30 con 0x7f80f80b1130 2023-07-14T11:55:37.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.474+0000 7f80f659c700 1 --2- 172.21.15.104:0/1567622656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80f80b1130 0x7f80f81559c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f80e800e9b0 tx=0x7f80e800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:37.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.474+0000 7f80e6ffd700 1 -- 172.21.15.104:0/1567622656 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e8009670 con 0x7f80f80b1130 2023-07-14T11:55:37.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.474+0000 7f80e6ffd700 1 -- 172.21.15.104:0/1567622656 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e800c850 con 0x7f80f80b1130 2023-07-14T11:55:37.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.474+0000 7f80fca78700 1 -- 172.21.15.104:0/1567622656 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f80cb810 con 0x7f80f80b1130 2023-07-14T11:55:37.462 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.474+0000 7f80e6ffd700 1 -- 172.21.15.104:0/1567622656 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e800cbc0 con 0x7f80f80b1130 2023-07-14T11:55:37.462 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.474+0000 7f80fca78700 1 -- 172.21.15.104:0/1567622656 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f80cbdd0 con 0x7f80f80b1130 2023-07-14T11:55:37.462 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.474+0000 7f80fca78700 1 -- 172.21.15.104:0/1567622656 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f804dcb0 con 0x7f80f80b1130 2023-07-14T11:55:37.463 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.478+0000 7f80e6ffd700 1 -- 172.21.15.104:0/1567622656 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f80e800cd20 con 0x7f80f80b1130 2023-07-14T11:55:37.465 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.478+0000 7f80e6ffd700 1 --2- 172.21.15.104:0/1567622656 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f80d80662f0 0x7f80d80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:37.465 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.478+0000 7f80e6ffd700 1 -- 172.21.15.104:0/1567622656 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f80e8084510 con 0x7f80f80b1130 2023-07-14T11:55:37.465 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.478+0000 7f80f5d9b700 1 --2- 172.21.15.104:0/1567622656 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f80d80662f0 0x7f80d80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:37.466 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.478+0000 7f80f5d9b700 1 --2- 172.21.15.104:0/1567622656 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f80d80662f0 0x7f80d80687a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f80f81589c0 tx=0x7f80ec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:37.472 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.486+0000 7f80e6ffd700 1 -- 172.21.15.104:0/1567622656 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f80e80515a0 con 0x7f80f80b1130 2023-07-14T11:55:37.712 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.726+0000 7f80fca78700 1 -- 172.21.15.104:0/1567622656 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7f80f80b0900 con 0x7f80f80b1130 2023-07-14T11:55:37.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.734+0000 7f80e6ffd700 1 -- 172.21.15.104:0/1567622656 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v15) v1 ==== 211+0+168 (secure 0 0 0) 0x7f80e8054bc0 con 0x7f80f80b1130 2023-07-14T11:55:37.722 INFO:teuthology.orchestra.run.smithi104.stdout:[client.0] 2023-07-14T11:55:37.722 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDrNrFkbrzjERAA+9BBzuauwUdhALUcDtd80A== 2023-07-14T11:55:37.722 INFO:teuthology.orchestra.run.smithi104.stdout: caps mds = "allow rwp" 2023-07-14T11:55:37.722 INFO:teuthology.orchestra.run.smithi104.stdout: caps mon = "allow r" 2023-07-14T11:55:37.722 INFO:teuthology.orchestra.run.smithi104.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2023-07-14T11:55:37.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.738+0000 7f80fca78700 1 -- 172.21.15.104:0/1567622656 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f80d80662f0 msgr2=0x7f80d80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:37.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.738+0000 7f80fca78700 1 --2- 172.21.15.104:0/1567622656 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f80d80662f0 0x7f80d80687a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f80f81589c0 tx=0x7f80ec009040 comp rx=0 tx=0).stop 2023-07-14T11:55:37.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.738+0000 7f80fca78700 1 -- 172.21.15.104:0/1567622656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80f80b1130 msgr2=0x7f80f81559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:37.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.738+0000 7f80fca78700 1 --2- 172.21.15.104:0/1567622656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80f80b1130 0x7f80f81559c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f80e800e9b0 tx=0x7f80e800ed70 comp rx=0 tx=0).stop 2023-07-14T11:55:37.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.738+0000 7f80fca78700 1 -- 172.21.15.104:0/1567622656 shutdown_connections 2023-07-14T11:55:37.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.738+0000 7f80fca78700 1 --2- 172.21.15.104:0/1567622656 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f80d80662f0 0x7f80d80687a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:37.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.738+0000 7f80fca78700 1 --2- 172.21.15.104:0/1567622656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f80f815c1a0 0x7f80f80b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:37.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.738+0000 7f80fca78700 1 --2- 172.21.15.104:0/1567622656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80f80b4130 0x7f80f80ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:37.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.738+0000 7f80fca78700 1 --2- 172.21.15.104:0/1567622656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80f80b1130 0x7f80f81559c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:37.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.738+0000 7f80fca78700 1 -- 172.21.15.104:0/1567622656 >> 172.21.15.104:0/1567622656 conn(0x7f80f809ba10 msgr2=0x7f80f80afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:37.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.738+0000 7f80fca78700 1 -- 172.21.15.104:0/1567622656 shutdown_connections 2023-07-14T11:55:37.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.738+0000 7f80fca78700 1 -- 172.21.15.104:0/1567622656 wait complete. 2023-07-14T11:55:37.727 INFO:teuthology.orchestra.run.smithi104.stderr:updated caps for client.0 2023-07-14T11:55:37.747 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-14T11:55:37.916 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.930+0000 7fc1eea85700 1 Processor -- start 2023-07-14T11:55:37.916 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.930+0000 7fc1eea85700 1 -- start start 2023-07-14T11:55:37.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.930+0000 7fc1eea85700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc1e81599d0 0x7fc1e8071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:37.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.930+0000 7fc1eea85700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e80cde10 con 0x7fc1e815ccf0 2023-07-14T11:55:37.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.930+0000 7fc1eea85700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e80cd520 con 0x7fc1e80caa20 2023-07-14T11:55:37.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.930+0000 7fc1eea85700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e815d190 con 0x7fc1e81599d0 2023-07-14T11:55:37.921 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.930+0000 7fc1ed282700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fc1e815ccf0 0x7fc1e80c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:45190/0 (socket says 172.21.15.104:45190) 2023-07-14T11:55:37.921 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1eda83700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc1e81599d0 0x7fc1e8071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:37.921 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1ed282700 1 -- 172.21.15.104:0/3879140466 learned_addr learned my addr 172.21.15.104:0/3879140466 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:37.921 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1eda83700 1 -- 172.21.15.104:0/3879140466 >> v1:172.21.15.121:6789/0 conn(0x7fc1e80caa20 legacy=0x7fc1e80cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:37.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1eda83700 1 -- 172.21.15.104:0/3879140466 >> v1:172.21.15.104:6789/0 conn(0x7fc1e815ccf0 legacy=0x7fc1e80c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:37.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1eda83700 1 -- 172.21.15.104:0/3879140466 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e80715e0 con 0x7fc1e81599d0 2023-07-14T11:55:37.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1eda83700 1 --2- 172.21.15.104:0/3879140466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1e81599d0 0x7fc1e8071000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009850 tx=0x7fc1e4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fd3b0ca86ab6b2b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:37.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1eca81700 1 -- 172.21.15.104:0/3879140466 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4011830 con 0x7fc1e81599d0 2023-07-14T11:55:37.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1eca81700 1 -- 172.21.15.104:0/3879140466 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e4011990 con 0x7fc1e81599d0 2023-07-14T11:55:37.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1eca81700 1 -- 172.21.15.104:0/3879140466 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4011d20 con 0x7fc1e81599d0 2023-07-14T11:55:37.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1eea85700 1 -- 172.21.15.104:0/3879140466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1e81599d0 msgr2=0x7fc1e8071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:37.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1eea85700 1 --2- 172.21.15.104:0/3879140466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1e81599d0 0x7fc1e8071000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009850 tx=0x7fc1e4009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:37.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1eea85700 1 -- 172.21.15.104:0/3879140466 shutdown_connections 2023-07-14T11:55:37.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1eea85700 1 --2- 172.21.15.104:0/3879140466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1e81599d0 0x7fc1e8071000 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:37.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1eea85700 1 -- 172.21.15.104:0/3879140466 >> 172.21.15.104:0/3879140466 conn(0x7fc1e806e960 msgr2=0x7fc1e806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:37.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1eea85700 1 -- 172.21.15.104:0/3879140466 shutdown_connections 2023-07-14T11:55:37.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.934+0000 7fc1eea85700 1 -- 172.21.15.104:0/3879140466 wait complete. 2023-07-14T11:55:37.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.938+0000 7fc1eea85700 1 Processor -- start 2023-07-14T11:55:37.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.938+0000 7fc1eea85700 1 -- start start 2023-07-14T11:55:37.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.938+0000 7fc1eea85700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1e80caa20 0x7fc1e8154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:37.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.938+0000 7fc1eea85700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e81599d0 0x7fc1e80b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:37.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.938+0000 7fc1eea85700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1e815ccf0 0x7fc1e815f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:37.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.938+0000 7fc1eea85700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e80af610 con 0x7fc1e815ccf0 2023-07-14T11:55:37.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.938+0000 7fc1eea85700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e80b1cc0 con 0x7fc1e80caa20 2023-07-14T11:55:37.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.938+0000 7fc1eea85700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e80b2bc0 con 0x7fc1e81599d0 2023-07-14T11:55:37.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1ee284700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1e815ccf0 0x7fc1e815f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:37.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1ee284700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1e815ccf0 0x7fc1e815f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:51512/0 (socket says 172.21.15.104:51512) 2023-07-14T11:55:37.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1ee284700 1 -- 172.21.15.104:0/2829613592 learned_addr learned my addr 172.21.15.104:0/2829613592 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:37.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1eda83700 1 --2- 172.21.15.104:0/2829613592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1e80caa20 0x7fc1e8154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:37.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1ed282700 1 --2- 172.21.15.104:0/2829613592 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e81599d0 0x7fc1e80b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:37.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1ee284700 1 -- 172.21.15.104:0/2829613592 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e81599d0 msgr2=0x7fc1e80b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:37.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1ee284700 1 --2- 172.21.15.104:0/2829613592 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e81599d0 0x7fc1e80b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:37.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1ee284700 1 -- 172.21.15.104:0/2829613592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1e80caa20 msgr2=0x7fc1e8154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:37.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1ee284700 1 --2- 172.21.15.104:0/2829613592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1e80caa20 0x7fc1e8154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:37.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1ee284700 1 -- 172.21.15.104:0/2829613592 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e40094b0 con 0x7fc1e815ccf0 2023-07-14T11:55:37.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1ee284700 1 --2- 172.21.15.104:0/2829613592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1e815ccf0 0x7fc1e815f4a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000cc50 tx=0x7fc1e000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:37.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1da7fc700 1 -- 172.21.15.104:0/2829613592 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0003c20 con 0x7fc1e815ccf0 2023-07-14T11:55:37.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1da7fc700 1 -- 172.21.15.104:0/2829613592 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e0003d80 con 0x7fc1e815ccf0 2023-07-14T11:55:37.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1eea85700 1 -- 172.21.15.104:0/2829613592 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e80b2ea0 con 0x7fc1e815ccf0 2023-07-14T11:55:37.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1da7fc700 1 -- 172.21.15.104:0/2829613592 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e000ab60 con 0x7fc1e815ccf0 2023-07-14T11:55:37.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1eea85700 1 -- 172.21.15.104:0/2829613592 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e80b3460 con 0x7fc1e815ccf0 2023-07-14T11:55:37.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.942+0000 7fc1eea85700 1 -- 172.21.15.104:0/2829613592 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e80ce280 con 0x7fc1e815ccf0 2023-07-14T11:55:37.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.946+0000 7fc1da7fc700 1 -- 172.21.15.104:0/2829613592 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc1e000acc0 con 0x7fc1e815ccf0 2023-07-14T11:55:37.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.946+0000 7fc1da7fc700 1 --2- 172.21.15.104:0/2829613592 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc1cc0662f0 0x7fc1cc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:37.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.946+0000 7fc1da7fc700 1 -- 172.21.15.104:0/2829613592 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fc1e0084030 con 0x7fc1e815ccf0 2023-07-14T11:55:37.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.946+0000 7fc1eda83700 1 --2- 172.21.15.104:0/2829613592 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc1cc0662f0 0x7fc1cc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:37.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.950+0000 7fc1eda83700 1 --2- 172.21.15.104:0/2829613592 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc1cc0662f0 0x7fc1cc0687a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009fd0 tx=0x7fc1e4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:37.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:37.954+0000 7fc1da7fc700 1 -- 172.21.15.104:0/2829613592 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fc1e00b4720 con 0x7fc1e815ccf0 2023-07-14T11:55:37.959 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:55:37.967 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:55:38.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:38.166+0000 7fc1eea85700 1 -- 172.21.15.104:0/2829613592 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc1e804e500 con 0x7fc1e815ccf0 2023-07-14T11:55:38.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:38.166+0000 7fc1da7fc700 1 -- 172.21.15.104:0/2829613592 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+82 (secure 0 0 0) 0x7fc1e80ce280 con 0x7fc1e815ccf0 2023-07-14T11:55:38.155 INFO:teuthology.orchestra.run.smithi104.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2023-07-14T11:55:38.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:38.170+0000 7fc1eea85700 1 -- 172.21.15.104:0/2829613592 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc1cc0662f0 msgr2=0x7fc1cc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:38.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:38.170+0000 7fc1eea85700 1 --2- 172.21.15.104:0/2829613592 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc1cc0662f0 0x7fc1cc0687a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009fd0 tx=0x7fc1e4008030 comp rx=0 tx=0).stop 2023-07-14T11:55:38.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:38.170+0000 7fc1eea85700 1 -- 172.21.15.104:0/2829613592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1e815ccf0 msgr2=0x7fc1e815f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:38.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:38.170+0000 7fc1eea85700 1 --2- 172.21.15.104:0/2829613592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1e815ccf0 0x7fc1e815f4a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000cc50 tx=0x7fc1e000cf60 comp rx=0 tx=0).stop 2023-07-14T11:55:38.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:38.170+0000 7fc1eea85700 1 -- 172.21.15.104:0/2829613592 shutdown_connections 2023-07-14T11:55:38.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:38.170+0000 7fc1eea85700 1 --2- 172.21.15.104:0/2829613592 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc1cc0662f0 0x7fc1cc0687a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:38.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:38.170+0000 7fc1eea85700 1 --2- 172.21.15.104:0/2829613592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1e815ccf0 0x7fc1e815f4a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:38.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:38.170+0000 7fc1eea85700 1 --2- 172.21.15.104:0/2829613592 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e81599d0 0x7fc1e80b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:38.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:38.170+0000 7fc1eea85700 1 --2- 172.21.15.104:0/2829613592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1e80caa20 0x7fc1e8154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:38.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:38.170+0000 7fc1eea85700 1 -- 172.21.15.104:0/2829613592 >> 172.21.15.104:0/2829613592 conn(0x7fc1e806e960 msgr2=0x7fc1e80ad260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:38.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:38.170+0000 7fc1eea85700 1 -- 172.21.15.104:0/2829613592 shutdown_connections 2023-07-14T11:55:38.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:38.170+0000 7fc1eea85700 1 -- 172.21.15.104:0/2829613592 wait complete. 2023-07-14T11:55:38.177 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-14T11:55:38.177 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T11:55:38.178 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2023-07-14T11:55:38.178 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-14T11:55:38.178 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:38.178 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-14T11:55:38.179 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T11:55:38.179 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T11:55:38.179 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:55:38.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:38.181 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2023-07-14T11:55:38.198 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:55:38.199 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:55:38.199 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:55:38.199 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:38.199 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2023-07-14T11:55:38.199 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:38.199 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:55:38.200 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:38.200 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2023-07-14T11:55:38.200 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:55:38.200 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:38.200 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2023-07-14T11:55:38.200 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:55:38.200 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:38.201 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T11:55:38.201 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:38.201 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-07-14T11:55:38.201 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42258sec preferred_lft 42258sec 2023-07-14T11:55:38.201 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2023-07-14T11:55:38.201 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:38.201 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-07-14T11:55:38.201 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:a2:a1:53:16 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:38.202 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-07-14T11:55:38.202 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:38.202 INFO:teuthology.orchestra.run.smithi104.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-14T11:55:38.202 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:b8:27:80:0d:ab brd ff:ff:ff:ff:ff:ff 2023-07-14T11:55:38.202 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-14T11:55:38.202 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:38.202 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5020:39ff:fe23:761d/64 scope link 2023-07-14T11:55:38.202 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:38.203 INFO:teuthology.orchestra.run.smithi104.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-14T11:55:38.203 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:b8:27:80:0d:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-14T11:55:38.203 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::e8b8:27ff:fe80:dab/64 scope link 2023-07-14T11:55:38.203 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:38.203 INFO:teuthology.orchestra.run.smithi104.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-14T11:55:38.203 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:e6:23:3b:72:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-14T11:55:38.203 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::e8e6:23ff:fe3b:7247/64 scope link 2023-07-14T11:55:38.203 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:55:38.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:38.205 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T11:55:38.283 INFO:teuthology.orchestra.run.smithi104.stdout:1 2023-07-14T11:55:38.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:38.287 DEBUG:teuthology.orchestra.run.smithi104:> route 2023-07-14T11:55:38.346 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2023-07-14T11:55:38.346 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T11:55:38.346 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-07-14T11:55:38.347 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T11:55:38.347 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-07-14T11:55:38.347 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-07-14T11:55:38.347 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-07-14T11:55:38.347 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T11:55:38.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:38.348 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:55:38.348 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:55:38.349 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T11:55:38.349 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T11:55:38.349 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T11:55:38.350 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:55:38.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:38.485 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2023-07-14T11:55:38.503 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-14T11:55:38.504 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-14T11:55:38.505 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:55:38.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:38.505 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2023-07-14T11:55:38.582 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-14T11:55:38.582 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:55:38.583 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:38.640 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:55:38.641 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:55:38.642 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:38.700 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:55:38.780 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:55:38.781 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:55:38.860 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:55:38.862 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:55:38.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:38.863 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:55:38.925 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-14T11:55:38.925 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-14T11:55:38.978 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:55:39.013 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:55:39.015 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:55:39.087 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:55:39.090 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:55:39.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:39.091 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:55:39.116 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:2023-07-14T11:55:39.130+0000 7fb425ddf640 -1 init, newargv = 0x55d7f602ce10 newargc=15 2023-07-14T11:55:39.116 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[48185]: starting ceph client 2023-07-14T11:55:39.126 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[48185]: starting fuse 2023-07-14T11:55:39.130 INFO:teuthology.orchestra.run.smithi104.stdout:49 2023-07-14T11:55:39.131 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-07-14T11:55:39.131 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T11:55:39.132 DEBUG:teuthology.orchestra.run.smithi104:> import glob 2023-07-14T11:55:39.132 DEBUG:teuthology.orchestra.run.smithi104:> import re 2023-07-14T11:55:39.133 DEBUG:teuthology.orchestra.run.smithi104:> import os 2023-07-14T11:55:39.133 DEBUG:teuthology.orchestra.run.smithi104:> import subprocess 2023-07-14T11:55:39.133 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:55:39.134 DEBUG:teuthology.orchestra.run.smithi104:> def _find_admin_socket(client_name): 2023-07-14T11:55:39.134 DEBUG:teuthology.orchestra.run.smithi104:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-14T11:55:39.135 DEBUG:teuthology.orchestra.run.smithi104:> files = glob.glob(asok_path) 2023-07-14T11:55:39.135 DEBUG:teuthology.orchestra.run.smithi104:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-14T11:55:39.135 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:55:39.136 DEBUG:teuthology.orchestra.run.smithi104:> # Given a non-glob path, it better be there 2023-07-14T11:55:39.136 DEBUG:teuthology.orchestra.run.smithi104:> if "*" not in asok_path: 2023-07-14T11:55:39.136 DEBUG:teuthology.orchestra.run.smithi104:> assert(len(files) == 1) 2023-07-14T11:55:39.136 DEBUG:teuthology.orchestra.run.smithi104:> return files[0] 2023-07-14T11:55:39.137 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:55:39.137 DEBUG:teuthology.orchestra.run.smithi104:> for f in files: 2023-07-14T11:55:39.137 DEBUG:teuthology.orchestra.run.smithi104:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-14T11:55:39.138 DEBUG:teuthology.orchestra.run.smithi104:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-14T11:55:39.138 DEBUG:teuthology.orchestra.run.smithi104:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-14T11:55:39.138 DEBUG:teuthology.orchestra.run.smithi104:> contents = proc_f.read() 2023-07-14T11:55:39.139 DEBUG:teuthology.orchestra.run.smithi104:> if mountpoint in contents: 2023-07-14T11:55:39.139 DEBUG:teuthology.orchestra.run.smithi104:> return f 2023-07-14T11:55:39.139 DEBUG:teuthology.orchestra.run.smithi104:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-14T11:55:39.139 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:55:39.140 DEBUG:teuthology.orchestra.run.smithi104:> print(_find_admin_socket("client.0")) 2023-07-14T11:55:39.140 DEBUG:teuthology.orchestra.run.smithi104:> ' 2023-07-14T11:55:39.289 INFO:teuthology.orchestra.run.smithi104.stdout:/var/run/ceph/ceph-client.0.48185.asok 2023-07-14T11:55:39.473 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48185.asok 2023-07-14T11:55:39.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:39.474 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48185.asok status 2023-07-14T11:55:39.669 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:55:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: "metadata": { 2023-07-14T11:55:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-14T11:55:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:55:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: "entity_id": "0", 2023-07-14T11:55:39.670 INFO:teuthology.orchestra.run.smithi104.stdout: "hostname": "smithi104", 2023-07-14T11:55:39.670 INFO:teuthology.orchestra.run.smithi104.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-14T11:55:39.670 INFO:teuthology.orchestra.run.smithi104.stdout: "pid": "48185", 2023-07-14T11:55:39.670 INFO:teuthology.orchestra.run.smithi104.stdout: "root": "/" 2023-07-14T11:55:39.670 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:55:39.670 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_count": 0, 2023-07-14T11:55:39.670 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_pinned_count": 0, 2023-07-14T11:55:39.671 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 6045, 2023-07-14T11:55:39.671 INFO:teuthology.orchestra.run.smithi104.stdout: "inst": { 2023-07-14T11:55:39.671 INFO:teuthology.orchestra.run.smithi104.stdout: "name": { 2023-07-14T11:55:39.671 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "client", 2023-07-14T11:55:39.671 INFO:teuthology.orchestra.run.smithi104.stdout: "num": 6045 2023-07-14T11:55:39.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:55:39.671 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:55:39.671 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:55:39.672 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:55:39.672 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2799252046 2023-07-14T11:55:39.672 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:55:39.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:55:39.672 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:55:39.672 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:55:39.672 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:55:39.673 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2799252046 2023-07-14T11:55:39.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:55:39.673 INFO:teuthology.orchestra.run.smithi104.stdout: "inst_str": "client.6045 192.168.0.1:0/2799252046", 2023-07-14T11:55:39.673 INFO:teuthology.orchestra.run.smithi104.stdout: "addr_str": "192.168.0.1:0/2799252046", 2023-07-14T11:55:39.673 INFO:teuthology.orchestra.run.smithi104.stdout: "inode_count": 1, 2023-07-14T11:55:39.673 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_epoch": 19, 2023-07-14T11:55:39.673 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch": 45, 2023-07-14T11:55:39.673 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch_barrier": 0, 2023-07-14T11:55:39.674 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklisted": false, 2023-07-14T11:55:39.674 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs" 2023-07-14T11:55:39.674 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:55:39.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:39.685 DEBUG:teuthology.orchestra.run.smithi104:> stat --file-system '--printf=%T 2023-07-14T11:55:39.685 DEBUG:teuthology.orchestra.run.smithi104:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:39.704 INFO:teuthology.orchestra.run.smithi104.stdout:fuseblk 2023-07-14T11:55:39.704 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:39.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:39.705 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:39.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:39.781 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2023-07-14T11:55:39.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:39.848 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2023-07-14T11:55:39.922 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:55:39.922 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:55:39.982 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:55:40.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.218+0000 7fed7dccb700 1 Processor -- start 2023-07-14T11:55:40.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.218+0000 7fed7dccb700 1 -- start start 2023-07-14T11:55:40.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.222+0000 7fed7dccb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fed7815bff0 0x7fed780ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:40.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.222+0000 7fed7dccb700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed780af0a0 con 0x7fed780cd070 2023-07-14T11:55:40.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.222+0000 7fed7dccb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed780cf720 con 0x7fed780cf1c0 2023-07-14T11:55:40.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.222+0000 7fed7dccb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed780cd4f0 con 0x7fed7815bff0 2023-07-14T11:55:40.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.222+0000 7fed77fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fed780cd070 0x7fed780ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:45200/0 (socket says 172.21.15.104:45200) 2023-07-14T11:55:40.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.222+0000 7fed77fff700 1 -- 172.21.15.104:0/1173706793 learned_addr learned my addr 172.21.15.104:0/1173706793 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:40.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.222+0000 7fed7ccc9700 1 --2- 172.21.15.104:0/1173706793 >> v2:172.21.15.121:3300/0 conn(0x7fed7815bff0 0x7fed780ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:40.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.222+0000 7fed777fe700 1 -- 172.21.15.104:0/1173706793 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180644638 0 0) 0x7fed780af0a0 con 0x7fed780cd070 2023-07-14T11:55:40.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.222+0000 7fed777fe700 1 -- 172.21.15.104:0/1173706793 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed600035d0 con 0x7fed780cd070 2023-07-14T11:55:40.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.222+0000 7fed777fe700 1 -- 172.21.15.104:0/1173706793 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3601749684 0 0) 0x7fed780cf720 con 0x7fed780cf1c0 2023-07-14T11:55:40.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.222+0000 7fed777fe700 1 -- 172.21.15.104:0/1173706793 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed780af0a0 con 0x7fed780cf1c0 2023-07-14T11:55:40.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.222+0000 7fed7ccc9700 1 -- 172.21.15.104:0/1173706793 >> v1:172.21.15.121:6789/0 conn(0x7fed780cf1c0 legacy=0x7fed780afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:40.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.222+0000 7fed7ccc9700 1 -- 172.21.15.104:0/1173706793 >> v1:172.21.15.104:6789/0 conn(0x7fed780cd070 legacy=0x7fed780ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:40.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.222+0000 7fed7ccc9700 1 -- 172.21.15.104:0/1173706793 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed780af2f0 con 0x7fed7815bff0 2023-07-14T11:55:40.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.222+0000 7fed7ccc9700 1 --2- 172.21.15.104:0/1173706793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed7815bff0 0x7fed780ccb30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fed68009850 tx=0x7fed68009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=735e8db43bda1243 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:40.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.226+0000 7fed777fe700 1 -- 172.21.15.104:0/1173706793 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68011830 con 0x7fed7815bff0 2023-07-14T11:55:40.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.226+0000 7fed777fe700 1 -- 172.21.15.104:0/1173706793 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed68011990 con 0x7fed7815bff0 2023-07-14T11:55:40.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.226+0000 7fed777fe700 1 -- 172.21.15.104:0/1173706793 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68011d40 con 0x7fed7815bff0 2023-07-14T11:55:40.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.226+0000 7fed7dccb700 1 -- 172.21.15.104:0/1173706793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed7815bff0 msgr2=0x7fed780ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:40.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.226+0000 7fed7dccb700 1 --2- 172.21.15.104:0/1173706793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed7815bff0 0x7fed780ccb30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fed68009850 tx=0x7fed68009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:40.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.226+0000 7fed7dccb700 1 -- 172.21.15.104:0/1173706793 shutdown_connections 2023-07-14T11:55:40.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.226+0000 7fed7dccb700 1 --2- 172.21.15.104:0/1173706793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed7815bff0 0x7fed780ccb30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.226+0000 7fed7dccb700 1 -- 172.21.15.104:0/1173706793 >> 172.21.15.104:0/1173706793 conn(0x7fed78060c40 msgr2=0x7fed7809b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:40.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.226+0000 7fed7dccb700 1 -- 172.21.15.104:0/1173706793 shutdown_connections 2023-07-14T11:55:40.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.226+0000 7fed7dccb700 1 -- 172.21.15.104:0/1173706793 wait complete. 2023-07-14T11:55:40.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.226+0000 7fed7dccb700 1 Processor -- start 2023-07-14T11:55:40.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.226+0000 7fed7dccb700 1 -- start start 2023-07-14T11:55:40.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7dccb700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed780cd070 0x7fed781f1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:40.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7dccb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed780cf1c0 0x7fed780b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:40.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7dccb700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed7815bff0 0x7fed781f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:40.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7dccb700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7815e150 con 0x7fed780cd070 2023-07-14T11:55:40.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7dccb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed780b6620 con 0x7fed780cf1c0 2023-07-14T11:55:40.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7dccb700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed780c9850 con 0x7fed7815bff0 2023-07-14T11:55:40.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7ccc9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed780cd070 0x7fed781f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:40.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7ccc9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed780cd070 0x7fed781f1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:51514/0 (socket says 172.21.15.104:51514) 2023-07-14T11:55:40.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7ccc9700 1 -- 172.21.15.104:0/377110215 learned_addr learned my addr 172.21.15.104:0/377110215 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:40.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed77fff700 1 --2- 172.21.15.104:0/377110215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed780cf1c0 0x7fed780b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:40.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7d4ca700 1 --2- 172.21.15.104:0/377110215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed7815bff0 0x7fed781f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:40.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7ccc9700 1 -- 172.21.15.104:0/377110215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed7815bff0 msgr2=0x7fed781f1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:40.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7ccc9700 1 --2- 172.21.15.104:0/377110215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed7815bff0 0x7fed781f1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7ccc9700 1 -- 172.21.15.104:0/377110215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed780cf1c0 msgr2=0x7fed780b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:40.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7ccc9700 1 --2- 172.21.15.104:0/377110215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed780cf1c0 0x7fed780b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7ccc9700 1 -- 172.21.15.104:0/377110215 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed680094b0 con 0x7fed780cd070 2023-07-14T11:55:40.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7ccc9700 1 --2- 172.21.15.104:0/377110215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed780cd070 0x7fed781f1240 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fed68009ba0 tx=0x7fed68004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:40.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed757fa700 1 -- 172.21.15.104:0/377110215 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68019980 con 0x7fed780cd070 2023-07-14T11:55:40.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed757fa700 1 -- 172.21.15.104:0/377110215 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6801e070 con 0x7fed780cd070 2023-07-14T11:55:40.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed757fa700 1 -- 172.21.15.104:0/377110215 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68019d40 con 0x7fed780cd070 2023-07-14T11:55:40.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7dccb700 1 -- 172.21.15.104:0/377110215 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed780b5a10 con 0x7fed780cd070 2023-07-14T11:55:40.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.230+0000 7fed7dccb700 1 -- 172.21.15.104:0/377110215 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed780b5fd0 con 0x7fed780cd070 2023-07-14T11:55:40.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.234+0000 7fed7dccb700 1 -- 172.21.15.104:0/377110215 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed78159290 con 0x7fed780cd070 2023-07-14T11:55:40.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.234+0000 7fed757fa700 1 -- 172.21.15.104:0/377110215 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fed6802d450 con 0x7fed780cd070 2023-07-14T11:55:40.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.238+0000 7fed757fa700 1 --2- 172.21.15.104:0/377110215 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fed640662f0 0x7fed640687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:40.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.238+0000 7fed757fa700 1 -- 172.21.15.104:0/377110215 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fed68084e30 con 0x7fed780cd070 2023-07-14T11:55:40.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.238+0000 7fed77fff700 1 --2- 172.21.15.104:0/377110215 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fed640662f0 0x7fed640687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:40.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.238+0000 7fed77fff700 1 --2- 172.21.15.104:0/377110215 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fed640662f0 0x7fed640687a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fed780728b0 tx=0x7fed6c007e20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:40.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.242+0000 7fed757fa700 1 -- 172.21.15.104:0/377110215 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fed68051fd0 con 0x7fed780cd070 2023-07-14T11:55:40.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.418+0000 7fed7dccb700 1 -- 172.21.15.104:0/377110215 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed7804e500 con 0x7fed780cd070 2023-07-14T11:55:40.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.422+0000 7fed757fa700 1 -- 172.21.15.104:0/377110215 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+20311 (secure 0 0 0) 0x7fed68020090 con 0x7fed780cd070 2023-07-14T11:55:40.411 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:40.411 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":45,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:55:10.654581+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:54:56.370313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-14T11:54:57.608947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"data_new","create_time":"2023-07-14T11:55:08.099546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:55:40.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.426+0000 7fed7dccb700 1 -- 172.21.15.104:0/377110215 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fed640662f0 msgr2=0x7fed640687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:40.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.426+0000 7fed7dccb700 1 --2- 172.21.15.104:0/377110215 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fed640662f0 0x7fed640687a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fed780728b0 tx=0x7fed6c007e20 comp rx=0 tx=0).stop 2023-07-14T11:55:40.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.426+0000 7fed7dccb700 1 -- 172.21.15.104:0/377110215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed780cd070 msgr2=0x7fed781f1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:40.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.426+0000 7fed7dccb700 1 --2- 172.21.15.104:0/377110215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed780cd070 0x7fed781f1240 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fed68009ba0 tx=0x7fed68004770 comp rx=0 tx=0).stop 2023-07-14T11:55:40.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.430+0000 7fed7dccb700 1 -- 172.21.15.104:0/377110215 shutdown_connections 2023-07-14T11:55:40.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.430+0000 7fed7dccb700 1 --2- 172.21.15.104:0/377110215 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fed640662f0 0x7fed640687a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.430+0000 7fed7dccb700 1 --2- 172.21.15.104:0/377110215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed7815bff0 0x7fed781f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.430+0000 7fed7dccb700 1 --2- 172.21.15.104:0/377110215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed780cf1c0 0x7fed780b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.430+0000 7fed7dccb700 1 --2- 172.21.15.104:0/377110215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed780cd070 0x7fed781f1240 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.430+0000 7fed7dccb700 1 -- 172.21.15.104:0/377110215 >> 172.21.15.104:0/377110215 conn(0x7fed78060c40 msgr2=0x7fed780d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:40.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.430+0000 7fed7dccb700 1 -- 172.21.15.104:0/377110215 shutdown_connections 2023-07-14T11:55:40.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.430+0000 7fed7dccb700 1 -- 172.21.15.104:0/377110215 wait complete. 2023-07-14T11:55:40.436 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:54:56.370313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:54:57.608947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'data_new', 'create_time': '2023-07-14T11:55:08.099546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:55:40.437 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:55:40.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.630+0000 7f938fcc6700 1 Processor -- start 2023-07-14T11:55:40.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.630+0000 7f938fcc6700 1 -- start start 2023-07-14T11:55:40.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.630+0000 7f938fcc6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9388160240 0x7f93880cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:40.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.630+0000 7f938fcc6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9388160610 0x7f93880ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:40.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938fcc6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f93880afd80 0x7f93880b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:40.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938fcc6700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9388163fe0 con 0x7f93880afd80 2023-07-14T11:55:40.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938fcc6700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93880b0700 con 0x7f9388160610 2023-07-14T11:55:40.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938fcc6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93880c8c30 con 0x7f9388160240 2023-07-14T11:55:40.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938e263700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f93880afd80 0x7f93880b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:40.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938e263700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f93880afd80 0x7f93880b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41878/0 (socket says 172.21.15.104:41878) 2023-07-14T11:55:40.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938e263700 1 -- 172.21.15.104:0/3153370892 learned_addr learned my addr 172.21.15.104:0/3153370892 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:40.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938e263700 1 -- 172.21.15.104:0/3153370892 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9388160240 msgr2=0x7f93880cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:40.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938e263700 1 --2- 172.21.15.104:0/3153370892 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9388160240 0x7f93880cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938e263700 1 -- 172.21.15.104:0/3153370892 >> v2:172.21.15.121:3300/0 conn(0x7f9388160610 msgr2=0x7f93880ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:40.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938e263700 1 --2- 172.21.15.104:0/3153370892 >> v2:172.21.15.121:3300/0 conn(0x7f9388160610 0x7f93880ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938e263700 1 -- 172.21.15.104:0/3153370892 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93880ca9e0 con 0x7f93880afd80 2023-07-14T11:55:40.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938e263700 1 --2- 172.21.15.104:0/3153370892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93880afd80 0x7f93880b01c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f938806bc50 tx=0x7f938000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dddfd7632defa157 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:40.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938ca60700 1 -- 172.21.15.104:0/3153370892 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938000e070 con 0x7f93880afd80 2023-07-14T11:55:40.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938ca60700 1 -- 172.21.15.104:0/3153370892 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938000b9c0 con 0x7f93880afd80 2023-07-14T11:55:40.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938ca60700 1 -- 172.21.15.104:0/3153370892 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938000bd70 con 0x7f93880afd80 2023-07-14T11:55:40.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938fcc6700 1 -- 172.21.15.104:0/3153370892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93880afd80 msgr2=0x7f93880b01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:40.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938fcc6700 1 --2- 172.21.15.104:0/3153370892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93880afd80 0x7f93880b01c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f938806bc50 tx=0x7f938000b330 comp rx=0 tx=0).stop 2023-07-14T11:55:40.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938fcc6700 1 -- 172.21.15.104:0/3153370892 shutdown_connections 2023-07-14T11:55:40.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938fcc6700 1 --2- 172.21.15.104:0/3153370892 >> v2:172.21.15.121:3300/0 conn(0x7f9388160610 0x7f93880ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938fcc6700 1 --2- 172.21.15.104:0/3153370892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93880afd80 0x7f93880b01c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938fcc6700 1 --2- 172.21.15.104:0/3153370892 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9388160240 0x7f93880cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938fcc6700 1 -- 172.21.15.104:0/3153370892 >> 172.21.15.104:0/3153370892 conn(0x7f93880cad80 msgr2=0x7f93880cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:40.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938fcc6700 1 -- 172.21.15.104:0/3153370892 shutdown_connections 2023-07-14T11:55:40.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.634+0000 7f938fcc6700 1 -- 172.21.15.104:0/3153370892 wait complete. 2023-07-14T11:55:40.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.638+0000 7f938fcc6700 1 Processor -- start 2023-07-14T11:55:40.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.638+0000 7f938fcc6700 1 -- start start 2023-07-14T11:55:40.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.638+0000 7f938fcc6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93880afd80 0x7f93880cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:40.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.638+0000 7f938fcc6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9388160240 0x7f93880c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:40.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.638+0000 7f938fcc6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9388160610 0x7f93881f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:40.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.638+0000 7f938fcc6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93880c79b0 con 0x7f93880afd80 2023-07-14T11:55:40.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.638+0000 7f938fcc6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938815d000 con 0x7f9388160610 2023-07-14T11:55:40.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.638+0000 7f938fcc6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93880b38d0 con 0x7f9388160240 2023-07-14T11:55:40.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.638+0000 7f938da62700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93880afd80 0x7f93880cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:40.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.638+0000 7f938da62700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93880afd80 0x7f93880cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41884/0 (socket says 172.21.15.104:41884) 2023-07-14T11:55:40.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.638+0000 7f938da62700 1 -- 172.21.15.104:0/1729275712 learned_addr learned my addr 172.21.15.104:0/1729275712 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:40.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.638+0000 7f938d261700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9388160240 0x7f93880c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:40.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.638+0000 7f938e263700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9388160610 0x7f93881f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:40.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.642+0000 7f938da62700 1 -- 172.21.15.104:0/1729275712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9388160240 msgr2=0x7f93880c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:40.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.642+0000 7f938da62700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9388160240 0x7f93880c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.642+0000 7f938da62700 1 -- 172.21.15.104:0/1729275712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9388160610 msgr2=0x7f93881f5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:40.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.642+0000 7f938da62700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9388160610 0x7f93881f5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.642+0000 7f938da62700 1 -- 172.21.15.104:0/1729275712 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9380009b30 con 0x7f93880afd80 2023-07-14T11:55:40.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.642+0000 7f938e263700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9388160610 0x7f93881f5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:40.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.642+0000 7f938d261700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9388160240 0x7f93880c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:40.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.642+0000 7f938da62700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93880afd80 0x7f93880cd050 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f938400b550 tx=0x7f938400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:40.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.642+0000 7f937e7fc700 1 -- 172.21.15.104:0/1729275712 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384011830 con 0x7f93880afd80 2023-07-14T11:55:40.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.642+0000 7f937e7fc700 1 -- 172.21.15.104:0/1729275712 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9384011990 con 0x7f93880afd80 2023-07-14T11:55:40.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.642+0000 7f937e7fc700 1 -- 172.21.15.104:0/1729275712 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384011d40 con 0x7f93880afd80 2023-07-14T11:55:40.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.642+0000 7f938fcc6700 1 -- 172.21.15.104:0/1729275712 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93880b3bb0 con 0x7f93880afd80 2023-07-14T11:55:40.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.642+0000 7f938fcc6700 1 -- 172.21.15.104:0/1729275712 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93880c8380 con 0x7f93880afd80 2023-07-14T11:55:40.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.642+0000 7f938fcc6700 1 -- 172.21.15.104:0/1729275712 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f938804dcb0 con 0x7f93880afd80 2023-07-14T11:55:40.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.646+0000 7f937e7fc700 1 -- 172.21.15.104:0/1729275712 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f938401a400 con 0x7f93880afd80 2023-07-14T11:55:40.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.646+0000 7f937e7fc700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f936c0662f0 0x7f936c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:40.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.646+0000 7f937e7fc700 1 -- 172.21.15.104:0/1729275712 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f9384084050 con 0x7f93880afd80 2023-07-14T11:55:40.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.646+0000 7f938d261700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f936c0662f0 0x7f936c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:40.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.646+0000 7f938d261700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f936c0662f0 0x7f936c0687a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9378006fd0 tx=0x7f9378008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:40.639 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.654+0000 7f937e7fc700 1 -- 172.21.15.104:0/1729275712 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f9384051210 con 0x7f93880afd80 2023-07-14T11:55:40.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.822+0000 7f938fcc6700 1 -- 172.21.15.104:0/1729275712 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9388163d00 con 0x7f93880afd80 2023-07-14T11:55:40.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.822+0000 7f937e7fc700 1 -- 172.21.15.104:0/1729275712 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9384054830 con 0x7f93880afd80 2023-07-14T11:55:40.811 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:55:40.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.826+0000 7f938fcc6700 1 -- 172.21.15.104:0/1729275712 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f936c0662f0 msgr2=0x7f936c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:40.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.826+0000 7f938fcc6700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f936c0662f0 0x7f936c0687a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9378006fd0 tx=0x7f9378008040 comp rx=0 tx=0).stop 2023-07-14T11:55:40.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.826+0000 7f938fcc6700 1 -- 172.21.15.104:0/1729275712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93880afd80 msgr2=0x7f93880cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:40.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.826+0000 7f938fcc6700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93880afd80 0x7f93880cd050 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f938400b550 tx=0x7f938400b910 comp rx=0 tx=0).stop 2023-07-14T11:55:40.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.826+0000 7f938fcc6700 1 -- 172.21.15.104:0/1729275712 shutdown_connections 2023-07-14T11:55:40.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.826+0000 7f938fcc6700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f936c0662f0 0x7f936c0687a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.826+0000 7f938fcc6700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9388160610 0x7f93881f5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.826+0000 7f938fcc6700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9388160240 0x7f93880c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.826+0000 7f938fcc6700 1 --2- 172.21.15.104:0/1729275712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93880afd80 0x7f93880cd050 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:40.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.826+0000 7f938fcc6700 1 -- 172.21.15.104:0/1729275712 >> 172.21.15.104:0/1729275712 conn(0x7f93880cad80 msgr2=0x7f93880712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:40.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.826+0000 7f938fcc6700 1 -- 172.21.15.104:0/1729275712 shutdown_connections 2023-07-14T11:55:40.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:40.826+0000 7f938fcc6700 1 -- 172.21.15.104:0/1729275712 wait complete. 2023-07-14T11:55:40.835 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:55:41.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.026+0000 7ff9d1dbf700 1 Processor -- start 2023-07-14T11:55:41.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.026+0000 7ff9d1dbf700 1 -- start start 2023-07-14T11:55:41.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.030+0000 7ff9d1dbf700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9cc0c9270 0x7ff9cc071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.030+0000 7ff9d1dbf700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff9cc0afdf0 0x7ff9cc0b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.030+0000 7ff9d1dbf700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9cc158970 con 0x7ff9cc0cdcb0 2023-07-14T11:55:41.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.030+0000 7ff9d1dbf700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9cc0ce080 con 0x7ff9cc0afdf0 2023-07-14T11:55:41.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.030+0000 7ff9d1dbf700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc158bf0 con 0x7ff9cc0c9270 2023-07-14T11:55:41.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.030+0000 7ff9cb7fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ff9cc0cdcb0 0x7ff9cc0c8b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:54604/0 (socket says 172.21.15.104:54604) 2023-07-14T11:55:41.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.030+0000 7ff9cb7fe700 1 -- 172.21.15.104:0/3457992805 learned_addr learned my addr 172.21.15.104:0/3457992805 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:41.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.030+0000 7ff9c3fff700 1 --2- 172.21.15.104:0/3457992805 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9cc0c9270 0x7ff9cc071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.030+0000 7ff9cbfff700 1 --2- 172.21.15.104:0/3457992805 >> v2:172.21.15.121:3300/0 conn(0x7ff9cc0afdf0 0x7ff9cc0b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.030+0000 7ff9caffd700 1 -- 172.21.15.104:0/3457992805 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714934864 0 0) 0x7ff9cc158970 con 0x7ff9cc0cdcb0 2023-07-14T11:55:41.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9caffd700 1 -- 172.21.15.104:0/3457992805 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9b80035d0 con 0x7ff9cc0cdcb0 2023-07-14T11:55:41.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9c3fff700 1 -- 172.21.15.104:0/3457992805 >> v2:172.21.15.121:3300/0 conn(0x7ff9cc0afdf0 msgr2=0x7ff9cc0b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:41.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9c3fff700 1 --2- 172.21.15.104:0/3457992805 >> v2:172.21.15.121:3300/0 conn(0x7ff9cc0afdf0 0x7ff9cc0b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9c3fff700 1 -- 172.21.15.104:0/3457992805 >> v1:172.21.15.104:6789/0 conn(0x7ff9cc0cdcb0 legacy=0x7ff9cc0c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9c3fff700 1 -- 172.21.15.104:0/3457992805 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc158ed0 con 0x7ff9cc0c9270 2023-07-14T11:55:41.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9c3fff700 1 --2- 172.21.15.104:0/3457992805 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9cc0c9270 0x7ff9cc071440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009850 tx=0x7ff9b4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3df4f0b44028185 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:41.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9caffd700 1 -- 172.21.15.104:0/3457992805 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4011830 con 0x7ff9cc0c9270 2023-07-14T11:55:41.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9caffd700 1 -- 172.21.15.104:0/3457992805 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b4011990 con 0x7ff9cc0c9270 2023-07-14T11:55:41.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9caffd700 1 -- 172.21.15.104:0/3457992805 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4011d40 con 0x7ff9cc0c9270 2023-07-14T11:55:41.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9d1dbf700 1 -- 172.21.15.104:0/3457992805 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9cc0c9270 msgr2=0x7ff9cc071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9d1dbf700 1 --2- 172.21.15.104:0/3457992805 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9cc0c9270 0x7ff9cc071440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009850 tx=0x7ff9b4009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:41.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9d1dbf700 1 -- 172.21.15.104:0/3457992805 shutdown_connections 2023-07-14T11:55:41.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9d1dbf700 1 --2- 172.21.15.104:0/3457992805 >> v2:172.21.15.121:3300/0 conn(0x7ff9cc0afdf0 0x7ff9cc0b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9d1dbf700 1 --2- 172.21.15.104:0/3457992805 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9cc0c9270 0x7ff9cc071440 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9d1dbf700 1 -- 172.21.15.104:0/3457992805 >> 172.21.15.104:0/3457992805 conn(0x7ff9cc0cad80 msgr2=0x7ff9cc0cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:41.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9d1dbf700 1 -- 172.21.15.104:0/3457992805 shutdown_connections 2023-07-14T11:55:41.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.034+0000 7ff9d1dbf700 1 -- 172.21.15.104:0/3457992805 wait complete. 2023-07-14T11:55:41.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.038+0000 7ff9d1dbf700 1 Processor -- start 2023-07-14T11:55:41.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.038+0000 7ff9d1dbf700 1 -- start start 2023-07-14T11:55:41.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9d1dbf700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9cc0afdf0 0x7ff9cc1e3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9d1dbf700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9cc0c9270 0x7ff9cc151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9d1dbf700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9cc0cdcb0 0x7ff9cc158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9d1dbf700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc1e3bb0 con 0x7ff9cc0afdf0 2023-07-14T11:55:41.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9d1dbf700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc1e2c90 con 0x7ff9cc0c9270 2023-07-14T11:55:41.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9d1dbf700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc152850 con 0x7ff9cc0cdcb0 2023-07-14T11:55:41.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9cb7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9cc0afdf0 0x7ff9cc1e3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9cb7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9cc0afdf0 0x7ff9cc1e3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41888/0 (socket says 172.21.15.104:41888) 2023-07-14T11:55:41.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9cb7fe700 1 -- 172.21.15.104:0/2702550539 learned_addr learned my addr 172.21.15.104:0/2702550539 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:41.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9cbfff700 1 --2- 172.21.15.104:0/2702550539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9cc0cdcb0 0x7ff9cc158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9c3fff700 1 --2- 172.21.15.104:0/2702550539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9cc0c9270 0x7ff9cc151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9cb7fe700 1 -- 172.21.15.104:0/2702550539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9cc0cdcb0 msgr2=0x7ff9cc158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9cb7fe700 1 --2- 172.21.15.104:0/2702550539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9cc0cdcb0 0x7ff9cc158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9cb7fe700 1 -- 172.21.15.104:0/2702550539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9cc0c9270 msgr2=0x7ff9cc151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9cb7fe700 1 --2- 172.21.15.104:0/2702550539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9cc0c9270 0x7ff9cc151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9cb7fe700 1 -- 172.21.15.104:0/2702550539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b40094b0 con 0x7ff9cc0afdf0 2023-07-14T11:55:41.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9cb7fe700 1 --2- 172.21.15.104:0/2702550539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9cc0afdf0 0x7ff9cc1e3fd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00c700 tx=0x7ff9bc00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:41.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9c8ff9700 1 -- 172.21.15.104:0/2702550539 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc005d90 con 0x7ff9cc0afdf0 2023-07-14T11:55:41.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9c8ff9700 1 -- 172.21.15.104:0/2702550539 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9bc0070d0 con 0x7ff9cc0afdf0 2023-07-14T11:55:41.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9d1dbf700 1 -- 172.21.15.104:0/2702550539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc1e2f30 con 0x7ff9cc0afdf0 2023-07-14T11:55:41.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9c8ff9700 1 -- 172.21.15.104:0/2702550539 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc007480 con 0x7ff9cc0afdf0 2023-07-14T11:55:41.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.042+0000 7ff9d1dbf700 1 -- 172.21.15.104:0/2702550539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc1e34f0 con 0x7ff9cc0afdf0 2023-07-14T11:55:41.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.046+0000 7ff9d1dbf700 1 -- 172.21.15.104:0/2702550539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9cc061060 con 0x7ff9cc0afdf0 2023-07-14T11:55:41.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.046+0000 7ff9c8ff9700 1 -- 172.21.15.104:0/2702550539 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff9bc0075e0 con 0x7ff9cc0afdf0 2023-07-14T11:55:41.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.046+0000 7ff9c8ff9700 1 --2- 172.21.15.104:0/2702550539 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9ac0662f0 0x7ff9ac0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.046+0000 7ff9c8ff9700 1 -- 172.21.15.104:0/2702550539 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7ff9bc0833e0 con 0x7ff9cc0afdf0 2023-07-14T11:55:41.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.050+0000 7ff9c3fff700 1 --2- 172.21.15.104:0/2702550539 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9ac0662f0 0x7ff9ac0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.050+0000 7ff9c3fff700 1 --2- 172.21.15.104:0/2702550539 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9ac0662f0 0x7ff9ac0687a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40090b0 tx=0x7ff9b4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:41.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.054+0000 7ff9c8ff9700 1 -- 172.21.15.104:0/2702550539 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff9bc050590 con 0x7ff9cc0afdf0 2023-07-14T11:55:41.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.226+0000 7ff9d1dbf700 1 -- 172.21.15.104:0/2702550539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff9cc159460 con 0x7ff9cc0afdf0 2023-07-14T11:55:41.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.226+0000 7ff9c8ff9700 1 -- 172.21.15.104:0/2702550539 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff9bc0170a0 con 0x7ff9cc0afdf0 2023-07-14T11:55:41.212 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:55:41.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.226+0000 7ff9d1dbf700 1 -- 172.21.15.104:0/2702550539 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9ac0662f0 msgr2=0x7ff9ac0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.226+0000 7ff9d1dbf700 1 --2- 172.21.15.104:0/2702550539 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9ac0662f0 0x7ff9ac0687a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40090b0 tx=0x7ff9b4008030 comp rx=0 tx=0).stop 2023-07-14T11:55:41.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.226+0000 7ff9d1dbf700 1 -- 172.21.15.104:0/2702550539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9cc0afdf0 msgr2=0x7ff9cc1e3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.226+0000 7ff9d1dbf700 1 --2- 172.21.15.104:0/2702550539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9cc0afdf0 0x7ff9cc1e3fd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00c700 tx=0x7ff9bc00ca10 comp rx=0 tx=0).stop 2023-07-14T11:55:41.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.230+0000 7ff9d1dbf700 1 -- 172.21.15.104:0/2702550539 shutdown_connections 2023-07-14T11:55:41.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.230+0000 7ff9d1dbf700 1 --2- 172.21.15.104:0/2702550539 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9ac0662f0 0x7ff9ac0687a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.230+0000 7ff9d1dbf700 1 --2- 172.21.15.104:0/2702550539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9cc0cdcb0 0x7ff9cc158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.230+0000 7ff9d1dbf700 1 --2- 172.21.15.104:0/2702550539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9cc0c9270 0x7ff9cc151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.230+0000 7ff9d1dbf700 1 --2- 172.21.15.104:0/2702550539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9cc0afdf0 0x7ff9cc1e3fd0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.230+0000 7ff9d1dbf700 1 -- 172.21.15.104:0/2702550539 >> 172.21.15.104:0/2702550539 conn(0x7ff9cc0cad80 msgr2=0x7ff9cc159b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:41.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.230+0000 7ff9d1dbf700 1 -- 172.21.15.104:0/2702550539 shutdown_connections 2023-07-14T11:55:41.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.230+0000 7ff9d1dbf700 1 -- 172.21.15.104:0/2702550539 wait complete. 2023-07-14T11:55:41.235 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:55:41.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.426+0000 7f9d1c3a7700 1 Processor -- start 2023-07-14T11:55:41.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.426+0000 7f9d1c3a7700 1 -- start start 2023-07-14T11:55:41.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.430+0000 7f9d1c3a7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9d140afd80 0x7f9d1415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.430+0000 7f9d1c3a7700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d14070a30 con 0x7f9d140c8950 2023-07-14T11:55:41.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.430+0000 7f9d1c3a7700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d140b0210 con 0x7f9d1415df10 2023-07-14T11:55:41.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.430+0000 7f9d1c3a7700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1415e830 con 0x7f9d140afd80 2023-07-14T11:55:41.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.430+0000 7f9d1a944700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f9d140c8950 0x7f9d1406d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:54612/0 (socket says 172.21.15.104:54612) 2023-07-14T11:55:41.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.430+0000 7f9d1a944700 1 -- 172.21.15.104:0/4243118661 learned_addr learned my addr 172.21.15.104:0/4243118661 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:41.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.430+0000 7f9d19942700 1 --2- 172.21.15.104:0/4243118661 >> v2:172.21.15.121:3300/0 conn(0x7f9d140afd80 0x7f9d1415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.430+0000 7f9d19141700 1 -- 172.21.15.104:0/4243118661 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357009954 0 0) 0x7f9d14070a30 con 0x7f9d140c8950 2023-07-14T11:55:41.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.430+0000 7f9d19141700 1 -- 172.21.15.104:0/4243118661 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d000035d0 con 0x7f9d140c8950 2023-07-14T11:55:41.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.430+0000 7f9d19942700 1 -- 172.21.15.104:0/4243118661 >> v1:172.21.15.121:6789/0 conn(0x7f9d1415df10 legacy=0x7f9d140711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.430+0000 7f9d19942700 1 -- 172.21.15.104:0/4243118661 >> v1:172.21.15.104:6789/0 conn(0x7f9d140c8950 legacy=0x7f9d1406d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.434+0000 7f9d19942700 1 -- 172.21.15.104:0/4243118661 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d140b04f0 con 0x7f9d140afd80 2023-07-14T11:55:41.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.434+0000 7f9d19942700 1 --2- 172.21.15.104:0/4243118661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d140afd80 0x7f9d1415e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009850 tx=0x7f9d04009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bc41af7c07dc65b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:41.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.434+0000 7f9d19141700 1 -- 172.21.15.104:0/4243118661 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04011830 con 0x7f9d140afd80 2023-07-14T11:55:41.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.434+0000 7f9d19141700 1 -- 172.21.15.104:0/4243118661 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d04011990 con 0x7f9d140afd80 2023-07-14T11:55:41.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.434+0000 7f9d19141700 1 -- 172.21.15.104:0/4243118661 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04011d40 con 0x7f9d140afd80 2023-07-14T11:55:41.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.434+0000 7f9d1c3a7700 1 -- 172.21.15.104:0/4243118661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d140afd80 msgr2=0x7f9d1415e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.434+0000 7f9d1c3a7700 1 --2- 172.21.15.104:0/4243118661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d140afd80 0x7f9d1415e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009850 tx=0x7f9d04009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:41.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.434+0000 7f9d1c3a7700 1 -- 172.21.15.104:0/4243118661 shutdown_connections 2023-07-14T11:55:41.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.434+0000 7f9d1c3a7700 1 --2- 172.21.15.104:0/4243118661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d140afd80 0x7f9d1415e2f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.434+0000 7f9d1c3a7700 1 -- 172.21.15.104:0/4243118661 >> 172.21.15.104:0/4243118661 conn(0x7f9d140cad80 msgr2=0x7f9d140cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:41.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.434+0000 7f9d1c3a7700 1 -- 172.21.15.104:0/4243118661 shutdown_connections 2023-07-14T11:55:41.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.434+0000 7f9d1c3a7700 1 -- 172.21.15.104:0/4243118661 wait complete. 2023-07-14T11:55:41.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1c3a7700 1 Processor -- start 2023-07-14T11:55:41.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1c3a7700 1 -- start start 2023-07-14T11:55:41.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1c3a7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d140afd80 0x7f9d14154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1c3a7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d140c8950 0x7f9d140b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1c3a7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d14156090 0x7f9d141585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1c3a7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d14156a70 con 0x7f9d140afd80 2023-07-14T11:55:41.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1c3a7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d14156d50 con 0x7f9d140c8950 2023-07-14T11:55:41.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1c3a7700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d14157030 con 0x7f9d14156090 2023-07-14T11:55:41.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1a143700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d140afd80 0x7f9d14154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1a143700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d140afd80 0x7f9d14154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41898/0 (socket says 172.21.15.104:41898) 2023-07-14T11:55:41.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1a143700 1 -- 172.21.15.104:0/1336987111 learned_addr learned my addr 172.21.15.104:0/1336987111 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:41.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d19942700 1 --2- 172.21.15.104:0/1336987111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d140c8950 0x7f9d140b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1a944700 1 --2- 172.21.15.104:0/1336987111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d14156090 0x7f9d141585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1a143700 1 -- 172.21.15.104:0/1336987111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d14156090 msgr2=0x7f9d141585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1a143700 1 --2- 172.21.15.104:0/1336987111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d14156090 0x7f9d141585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1a143700 1 -- 172.21.15.104:0/1336987111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d140c8950 msgr2=0x7f9d140b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1a143700 1 --2- 172.21.15.104:0/1336987111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d140c8950 0x7f9d140b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d1a143700 1 -- 172.21.15.104:0/1336987111 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d040094b0 con 0x7f9d140afd80 2023-07-14T11:55:41.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.438+0000 7f9d19942700 1 --2- 172.21.15.104:0/1336987111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d140c8950 0x7f9d140b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:41.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.442+0000 7f9d1a143700 1 --2- 172.21.15.104:0/1336987111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d140afd80 0x7f9d14154ab0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000c500 tx=0x7f9d1000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:41.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.442+0000 7f9d0affd700 1 -- 172.21.15.104:0/1336987111 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d100052d0 con 0x7f9d140afd80 2023-07-14T11:55:41.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.442+0000 7f9d0affd700 1 -- 172.21.15.104:0/1336987111 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d10005430 con 0x7f9d140afd80 2023-07-14T11:55:41.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.442+0000 7f9d0affd700 1 -- 172.21.15.104:0/1336987111 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d100055a0 con 0x7f9d140afd80 2023-07-14T11:55:41.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.442+0000 7f9d1c3a7700 1 -- 172.21.15.104:0/1336987111 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d140abfe0 con 0x7f9d140afd80 2023-07-14T11:55:41.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.442+0000 7f9d1c3a7700 1 -- 172.21.15.104:0/1336987111 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d140b1fc0 con 0x7f9d140afd80 2023-07-14T11:55:41.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.442+0000 7f9d1c3a7700 1 -- 172.21.15.104:0/1336987111 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d140b2150 con 0x7f9d140afd80 2023-07-14T11:55:41.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.446+0000 7f9d0affd700 1 -- 172.21.15.104:0/1336987111 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9d100038e0 con 0x7f9d140afd80 2023-07-14T11:55:41.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.446+0000 7f9d0affd700 1 --2- 172.21.15.104:0/1336987111 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cf8066320 0x7f9cf80687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.446+0000 7f9d0affd700 1 -- 172.21.15.104:0/1336987111 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f9d10083580 con 0x7f9d140afd80 2023-07-14T11:55:41.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.446+0000 7f9d19942700 1 --2- 172.21.15.104:0/1336987111 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cf8066320 0x7f9cf80687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.446+0000 7f9d19942700 1 --2- 172.21.15.104:0/1336987111 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cf8066320 0x7f9cf80687d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9d040090b0 tx=0x7f9d0401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:41.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.450+0000 7f9d0affd700 1 -- 172.21.15.104:0/1336987111 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f9d10050690 con 0x7f9d140afd80 2023-07-14T11:55:41.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.630+0000 7f9d1c3a7700 1 -- 172.21.15.104:0/1336987111 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d1404dcb0 con 0x7f9d140afd80 2023-07-14T11:55:41.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.630+0000 7f9d0affd700 1 -- 172.21.15.104:0/1336987111 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d10053cb0 con 0x7f9d140afd80 2023-07-14T11:55:41.616 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:55:41.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.634+0000 7f9d1c3a7700 1 -- 172.21.15.104:0/1336987111 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cf8066320 msgr2=0x7f9cf80687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.634+0000 7f9d1c3a7700 1 --2- 172.21.15.104:0/1336987111 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cf8066320 0x7f9cf80687d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9d040090b0 tx=0x7f9d0401c040 comp rx=0 tx=0).stop 2023-07-14T11:55:41.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.634+0000 7f9d1c3a7700 1 -- 172.21.15.104:0/1336987111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d140afd80 msgr2=0x7f9d14154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.634+0000 7f9d1c3a7700 1 --2- 172.21.15.104:0/1336987111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d140afd80 0x7f9d14154ab0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000c500 tx=0x7f9d1000c8c0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.634+0000 7f9d1c3a7700 1 -- 172.21.15.104:0/1336987111 shutdown_connections 2023-07-14T11:55:41.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.634+0000 7f9d1c3a7700 1 --2- 172.21.15.104:0/1336987111 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9cf8066320 0x7f9cf80687d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.634+0000 7f9d1c3a7700 1 --2- 172.21.15.104:0/1336987111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d14156090 0x7f9d141585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.634+0000 7f9d1c3a7700 1 --2- 172.21.15.104:0/1336987111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d140c8950 0x7f9d140b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.634+0000 7f9d1c3a7700 1 --2- 172.21.15.104:0/1336987111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d140afd80 0x7f9d14154ab0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.634+0000 7f9d1c3a7700 1 -- 172.21.15.104:0/1336987111 >> 172.21.15.104:0/1336987111 conn(0x7f9d140cad80 msgr2=0x7f9d1406c120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:41.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.634+0000 7f9d1c3a7700 1 -- 172.21.15.104:0/1336987111 shutdown_connections 2023-07-14T11:55:41.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.634+0000 7f9d1c3a7700 1 -- 172.21.15.104:0/1336987111 wait complete. 2023-07-14T11:55:41.642 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:55:41.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.830+0000 7fc66ef8b700 1 Processor -- start 2023-07-14T11:55:41.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.830+0000 7fc66ef8b700 1 -- start start 2023-07-14T11:55:41.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66ef8b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc66815fa10 0x7fc66815fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66ef8b700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6680d0930 con 0x7fc6680b01c0 2023-07-14T11:55:41.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66ef8b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6680b4ae0 con 0x7fc668160320 2023-07-14T11:55:41.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66ef8b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680ce560 con 0x7fc66815fa10 2023-07-14T11:55:41.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66e78a700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fc6680b01c0 0x7fc6680cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:54628/0 (socket says 172.21.15.104:54628) 2023-07-14T11:55:41.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66e78a700 1 -- 172.21.15.104:0/1592630939 learned_addr learned my addr 172.21.15.104:0/1592630939 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:41.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66df89700 1 --2- 172.21.15.104:0/1592630939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc66815fa10 0x7fc66815fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66cf87700 1 -- 172.21.15.104:0/1592630939 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794844288 0 0) 0x7fc6680d0930 con 0x7fc6680b01c0 2023-07-14T11:55:41.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66cf87700 1 -- 172.21.15.104:0/1592630939 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6540035d0 con 0x7fc6680b01c0 2023-07-14T11:55:41.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66cf87700 1 -- 172.21.15.104:0/1592630939 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264980639 0 0) 0x7fc658001110 con 0x7fc668160320 2023-07-14T11:55:41.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66df89700 1 -- 172.21.15.104:0/1592630939 >> v1:172.21.15.121:6789/0 conn(0x7fc668160320 legacy=0x7fc6680ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66df89700 1 -- 172.21.15.104:0/1592630939 >> v1:172.21.15.104:6789/0 conn(0x7fc6680b01c0 legacy=0x7fc6680cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66df89700 1 -- 172.21.15.104:0/1592630939 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6680cd990 con 0x7fc66815fa10 2023-07-14T11:55:41.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66df89700 1 --2- 172.21.15.104:0/1592630939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc66815fa10 0x7fc66815fde0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc664009870 tx=0x7fc664009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82ffaaf532651bbc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:41.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66cf87700 1 -- 172.21.15.104:0/1592630939 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc664005e50 con 0x7fc66815fa10 2023-07-14T11:55:41.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66cf87700 1 -- 172.21.15.104:0/1592630939 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc664006030 con 0x7fc66815fa10 2023-07-14T11:55:41.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.838+0000 7fc66cf87700 1 -- 172.21.15.104:0/1592630939 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6640063c0 con 0x7fc66815fa10 2023-07-14T11:55:41.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.842+0000 7fc66ef8b700 1 -- 172.21.15.104:0/1592630939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc66815fa10 msgr2=0x7fc66815fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.842+0000 7fc66ef8b700 1 --2- 172.21.15.104:0/1592630939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc66815fa10 0x7fc66815fde0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc664009870 tx=0x7fc664009b80 comp rx=0 tx=0).stop 2023-07-14T11:55:41.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.842+0000 7fc66ef8b700 1 -- 172.21.15.104:0/1592630939 shutdown_connections 2023-07-14T11:55:41.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.842+0000 7fc66ef8b700 1 --2- 172.21.15.104:0/1592630939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc66815fa10 0x7fc66815fde0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.842+0000 7fc66ef8b700 1 -- 172.21.15.104:0/1592630939 >> 172.21.15.104:0/1592630939 conn(0x7fc66809b850 msgr2=0x7fc6680d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:41.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.842+0000 7fc66ef8b700 1 -- 172.21.15.104:0/1592630939 shutdown_connections 2023-07-14T11:55:41.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.842+0000 7fc66ef8b700 1 -- 172.21.15.104:0/1592630939 wait complete. 2023-07-14T11:55:41.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.842+0000 7fc66ef8b700 1 Processor -- start 2023-07-14T11:55:41.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.842+0000 7fc66ef8b700 1 -- start start 2023-07-14T11:55:41.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66ef8b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6680b01c0 0x7fc6680b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66ef8b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc66815fa10 0x7fc6680b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66ef8b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc668160320 0x7fc6680b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66ef8b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680b6090 con 0x7fc668160320 2023-07-14T11:55:41.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66ef8b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66815b380 con 0x7fc66815fa10 2023-07-14T11:55:41.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66ef8b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680b5410 con 0x7fc6680b01c0 2023-07-14T11:55:41.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66e78a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc668160320 0x7fc6680b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66e78a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc668160320 0x7fc6680b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41910/0 (socket says 172.21.15.104:41910) 2023-07-14T11:55:41.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66d788700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc66815fa10 0x7fc6680b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66e78a700 1 -- 172.21.15.104:0/477597790 learned_addr learned my addr 172.21.15.104:0/477597790 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:41.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66df89700 1 --2- 172.21.15.104:0/477597790 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6680b01c0 0x7fc6680b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66d788700 1 -- 172.21.15.104:0/477597790 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6680b01c0 msgr2=0x7fc6680b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66d788700 1 --2- 172.21.15.104:0/477597790 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6680b01c0 0x7fc6680b6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66d788700 1 -- 172.21.15.104:0/477597790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc668160320 msgr2=0x7fc6680b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:41.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66d788700 1 --2- 172.21.15.104:0/477597790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc668160320 0x7fc6680b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:41.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66d788700 1 -- 172.21.15.104:0/477597790 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc664009450 con 0x7fc66815fa10 2023-07-14T11:55:41.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66e78a700 1 --2- 172.21.15.104:0/477597790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc668160320 0x7fc6680b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:55:41.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66d788700 1 --2- 172.21.15.104:0/477597790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc66815fa10 0x7fc6680b72c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc65800c750 tx=0x7fc65800ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:41.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc65e7fc700 1 -- 172.21.15.104:0/477597790 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6580054e0 con 0x7fc66815fa10 2023-07-14T11:55:41.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc65e7fc700 1 -- 172.21.15.104:0/477597790 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6580036a0 con 0x7fc66815fa10 2023-07-14T11:55:41.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66ef8b700 1 -- 172.21.15.104:0/477597790 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6680b56f0 con 0x7fc66815fa10 2023-07-14T11:55:41.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc65e7fc700 1 -- 172.21.15.104:0/477597790 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658003a50 con 0x7fc66815fa10 2023-07-14T11:55:41.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.846+0000 7fc66ef8b700 1 -- 172.21.15.104:0/477597790 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6680b5cb0 con 0x7fc66815fa10 2023-07-14T11:55:41.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.850+0000 7fc66ef8b700 1 -- 172.21.15.104:0/477597790 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6680d3430 con 0x7fc66815fa10 2023-07-14T11:55:41.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.850+0000 7fc65e7fc700 1 -- 172.21.15.104:0/477597790 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc658003bb0 con 0x7fc66815fa10 2023-07-14T11:55:41.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.850+0000 7fc65e7fc700 1 --2- 172.21.15.104:0/477597790 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc64c0662f0 0x7fc64c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:41.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.850+0000 7fc65e7fc700 1 -- 172.21.15.104:0/477597790 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fc658083280 con 0x7fc66815fa10 2023-07-14T11:55:41.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.850+0000 7fc66df89700 1 --2- 172.21.15.104:0/477597790 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc64c0662f0 0x7fc64c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:41.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.854+0000 7fc66df89700 1 --2- 172.21.15.104:0/477597790 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc64c0662f0 0x7fc64c0687a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc664013040 tx=0x7fc664008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:41.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:41.858+0000 7fc65e7fc700 1 -- 172.21.15.104:0/477597790 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fc6580503a0 con 0x7fc66815fa10 2023-07-14T11:55:42.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.030+0000 7fc66ef8b700 1 -- 172.21.15.104:0/477597790 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6680b4ae0 con 0x7fc66815fa10 2023-07-14T11:55:42.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.030+0000 7fc65e7fc700 1 -- 172.21.15.104:0/477597790 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc6680b4ae0 con 0x7fc66815fa10 2023-07-14T11:55:42.018 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:55:42.020 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.034+0000 7fc66ef8b700 1 -- 172.21.15.104:0/477597790 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc64c0662f0 msgr2=0x7fc64c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:42.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.034+0000 7fc66ef8b700 1 --2- 172.21.15.104:0/477597790 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc64c0662f0 0x7fc64c0687a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc664013040 tx=0x7fc664008070 comp rx=0 tx=0).stop 2023-07-14T11:55:42.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.034+0000 7fc66ef8b700 1 -- 172.21.15.104:0/477597790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc66815fa10 msgr2=0x7fc6680b72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:42.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.034+0000 7fc66ef8b700 1 --2- 172.21.15.104:0/477597790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc66815fa10 0x7fc6680b72c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc65800c750 tx=0x7fc65800ca60 comp rx=0 tx=0).stop 2023-07-14T11:55:42.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.034+0000 7fc66ef8b700 1 -- 172.21.15.104:0/477597790 shutdown_connections 2023-07-14T11:55:42.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.034+0000 7fc66ef8b700 1 --2- 172.21.15.104:0/477597790 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc64c0662f0 0x7fc64c0687a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.034+0000 7fc66ef8b700 1 --2- 172.21.15.104:0/477597790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc668160320 0x7fc6680b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.034+0000 7fc66ef8b700 1 --2- 172.21.15.104:0/477597790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc66815fa10 0x7fc6680b72c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.034+0000 7fc66ef8b700 1 --2- 172.21.15.104:0/477597790 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6680b01c0 0x7fc6680b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.034+0000 7fc66ef8b700 1 -- 172.21.15.104:0/477597790 >> 172.21.15.104:0/477597790 conn(0x7fc66809b850 msgr2=0x7fc6680cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:42.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.034+0000 7fc66ef8b700 1 -- 172.21.15.104:0/477597790 shutdown_connections 2023-07-14T11:55:42.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.034+0000 7fc66ef8b700 1 -- 172.21.15.104:0/477597790 wait complete. 2023-07-14T11:55:42.042 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-07-14T11:55:42.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.238+0000 7fa545de8700 1 Processor -- start 2023-07-14T11:55:42.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.238+0000 7fa545de8700 1 -- start start 2023-07-14T11:55:42.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa545de8700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5401603d0 0x7fa54006dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:42.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa545de8700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa5400cd7f0 0x7fa5400c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:42.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa545de8700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5400cdfb0 con 0x7fa540160000 2023-07-14T11:55:42.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa545de8700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa540070e40 con 0x7fa5400cd7f0 2023-07-14T11:55:42.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa545de8700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400cdcd0 con 0x7fa5401603d0 2023-07-14T11:55:42.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa544de6700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fa540160000 0x7fa5400c9e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:54632/0 (socket says 172.21.15.104:54632) 2023-07-14T11:55:42.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa544de6700 1 -- 172.21.15.104:0/3689349449 learned_addr learned my addr 172.21.15.104:0/3689349449 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:42.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa5455e7700 1 --2- 172.21.15.104:0/3689349449 >> v2:172.21.15.104:3300/0 conn(0x7fa5400cd7f0 0x7fa5400c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:42.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa53ffff700 1 --2- 172.21.15.104:0/3689349449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5401603d0 0x7fa54006dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:42.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa53f7fe700 1 -- 172.21.15.104:0/3689349449 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247149952 0 0) 0x7fa534001110 con 0x7fa540160000 2023-07-14T11:55:42.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa53f7fe700 1 -- 172.21.15.104:0/3689349449 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5300035d0 con 0x7fa540160000 2023-07-14T11:55:42.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa53ffff700 1 -- 172.21.15.104:0/3689349449 >> v2:172.21.15.104:3300/0 conn(0x7fa5400cd7f0 msgr2=0x7fa5400c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:42.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa53ffff700 1 --2- 172.21.15.104:0/3689349449 >> v2:172.21.15.104:3300/0 conn(0x7fa5400cd7f0 0x7fa5400c0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa53ffff700 1 -- 172.21.15.104:0/3689349449 >> v1:172.21.15.104:6789/0 conn(0x7fa540160000 legacy=0x7fa5400c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:42.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa53ffff700 1 -- 172.21.15.104:0/3689349449 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa540070780 con 0x7fa5401603d0 2023-07-14T11:55:42.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa5455e7700 1 --2- 172.21.15.104:0/3689349449 >> v2:172.21.15.104:3300/0 conn(0x7fa5400cd7f0 0x7fa5400c0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:55:42.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa53ffff700 1 --2- 172.21.15.104:0/3689349449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5401603d0 0x7fa54006dd10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009850 tx=0x7fa52c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fe0b99ff5f453aa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:42.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa53f7fe700 1 -- 172.21.15.104:0/3689349449 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c011830 con 0x7fa5401603d0 2023-07-14T11:55:42.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.242+0000 7fa53f7fe700 1 -- 172.21.15.104:0/3689349449 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c011990 con 0x7fa5401603d0 2023-07-14T11:55:42.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.246+0000 7fa53f7fe700 1 -- 172.21.15.104:0/3689349449 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c011d00 con 0x7fa5401603d0 2023-07-14T11:55:42.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.246+0000 7fa545de8700 1 -- 172.21.15.104:0/3689349449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5401603d0 msgr2=0x7fa54006dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:42.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.246+0000 7fa545de8700 1 --2- 172.21.15.104:0/3689349449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5401603d0 0x7fa54006dd10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009850 tx=0x7fa52c009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:42.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.246+0000 7fa545de8700 1 -- 172.21.15.104:0/3689349449 shutdown_connections 2023-07-14T11:55:42.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.246+0000 7fa545de8700 1 --2- 172.21.15.104:0/3689349449 >> v2:172.21.15.104:3300/0 conn(0x7fa5400cd7f0 0x7fa5400c0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.246+0000 7fa545de8700 1 --2- 172.21.15.104:0/3689349449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5401603d0 0x7fa54006dd10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.246+0000 7fa545de8700 1 -- 172.21.15.104:0/3689349449 >> 172.21.15.104:0/3689349449 conn(0x7fa5400afc60 msgr2=0x7fa5400b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:42.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.246+0000 7fa545de8700 1 -- 172.21.15.104:0/3689349449 shutdown_connections 2023-07-14T11:55:42.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.246+0000 7fa545de8700 1 -- 172.21.15.104:0/3689349449 wait complete. 2023-07-14T11:55:42.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.246+0000 7fa545de8700 1 Processor -- start 2023-07-14T11:55:42.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.246+0000 7fa545de8700 1 -- start start 2023-07-14T11:55:42.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa545de8700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5400cd7f0 0x7fa5401f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:42.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa545de8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa540160000 0x7fa5400b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:42.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa545de8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5401603d0 0x7fa5400b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:42.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa545de8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5401f6610 con 0x7fa540160000 2023-07-14T11:55:42.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa545de8700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400b2100 con 0x7fa5401603d0 2023-07-14T11:55:42.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa545de8700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400b23e0 con 0x7fa5400cd7f0 2023-07-14T11:55:42.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa53ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa540160000 0x7fa5400b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:42.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa53ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa540160000 0x7fa5400b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41916/0 (socket says 172.21.15.104:41916) 2023-07-14T11:55:42.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa53ffff700 1 -- 172.21.15.104:0/2173313918 learned_addr learned my addr 172.21.15.104:0/2173313918 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:42.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa544de6700 1 --2- 172.21.15.104:0/2173313918 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5400cd7f0 0x7fa5401f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:42.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa5455e7700 1 --2- 172.21.15.104:0/2173313918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5401603d0 0x7fa5400b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:42.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa53ffff700 1 -- 172.21.15.104:0/2173313918 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5400cd7f0 msgr2=0x7fa5401f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:42.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa53ffff700 1 --2- 172.21.15.104:0/2173313918 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5400cd7f0 0x7fa5401f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa53ffff700 1 -- 172.21.15.104:0/2173313918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5401603d0 msgr2=0x7fa5400b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:42.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa53ffff700 1 --2- 172.21.15.104:0/2173313918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5401603d0 0x7fa5400b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.239 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa53ffff700 1 -- 172.21.15.104:0/2173313918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa538009a70 con 0x7fa540160000 2023-07-14T11:55:42.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa544de6700 1 --2- 172.21.15.104:0/2173313918 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5400cd7f0 0x7fa5401f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:42.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa53ffff700 1 --2- 172.21.15.104:0/2173313918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa540160000 0x7fa5400b3270 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009ba0 tx=0x7fa52c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:42.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa53d7fa700 1 -- 172.21.15.104:0/2173313918 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0036c0 con 0x7fa540160000 2023-07-14T11:55:42.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa53d7fa700 1 -- 172.21.15.104:0/2173313918 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c01e070 con 0x7fa540160000 2023-07-14T11:55:42.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa53d7fa700 1 -- 172.21.15.104:0/2173313918 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c003a60 con 0x7fa540160000 2023-07-14T11:55:42.240 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa545de8700 1 -- 172.21.15.104:0/2173313918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52c0094b0 con 0x7fa540160000 2023-07-14T11:55:42.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.250+0000 7fa545de8700 1 -- 172.21.15.104:0/2173313918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5400b29a0 con 0x7fa540160000 2023-07-14T11:55:42.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.254+0000 7fa545de8700 1 -- 172.21.15.104:0/2173313918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5401f6780 con 0x7fa540160000 2023-07-14T11:55:42.241 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.254+0000 7fa53d7fa700 1 -- 172.21.15.104:0/2173313918 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa52c003bc0 con 0x7fa540160000 2023-07-14T11:55:42.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.254+0000 7fa53d7fa700 1 --2- 172.21.15.104:0/2173313918 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa5240662f0 0x7fa5240687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:42.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.254+0000 7fa53d7fa700 1 -- 172.21.15.104:0/2173313918 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fa52c084f40 con 0x7fa540160000 2023-07-14T11:55:42.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.254+0000 7fa544de6700 1 --2- 172.21.15.104:0/2173313918 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa5240662f0 0x7fa5240687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:42.243 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.258+0000 7fa544de6700 1 --2- 172.21.15.104:0/2173313918 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa5240662f0 0x7fa5240687a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa534005930 tx=0x7fa534009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:42.248 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.262+0000 7fa53d7fa700 1 -- 172.21.15.104:0/2173313918 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fa52c052100 con 0x7fa540160000 2023-07-14T11:55:42.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.434+0000 7fa545de8700 1 -- 172.21.15.104:0/2173313918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fa54004e500 con 0x7fa540160000 2023-07-14T11:55:42.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.434+0000 7fa53d7fa700 1 -- 172.21.15.104:0/2173313918 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v45) v1 ==== 97+0+11 (secure 0 0 0) 0x7fa52c055720 con 0x7fa540160000 2023-07-14T11:55:42.421 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:55:42.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.434+0000 7fa545de8700 1 -- 172.21.15.104:0/2173313918 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa5240662f0 msgr2=0x7fa5240687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:42.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.434+0000 7fa545de8700 1 --2- 172.21.15.104:0/2173313918 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa5240662f0 0x7fa5240687a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa534005930 tx=0x7fa534009040 comp rx=0 tx=0).stop 2023-07-14T11:55:42.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.434+0000 7fa545de8700 1 -- 172.21.15.104:0/2173313918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa540160000 msgr2=0x7fa5400b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:42.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.434+0000 7fa545de8700 1 --2- 172.21.15.104:0/2173313918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa540160000 0x7fa5400b3270 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009ba0 tx=0x7fa52c004770 comp rx=0 tx=0).stop 2023-07-14T11:55:42.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.438+0000 7fa545de8700 1 -- 172.21.15.104:0/2173313918 shutdown_connections 2023-07-14T11:55:42.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.438+0000 7fa545de8700 1 --2- 172.21.15.104:0/2173313918 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa5240662f0 0x7fa5240687a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.438+0000 7fa545de8700 1 --2- 172.21.15.104:0/2173313918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa540160000 0x7fa5400b3270 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.438+0000 7fa545de8700 1 --2- 172.21.15.104:0/2173313918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5401603d0 0x7fa5400b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.438+0000 7fa545de8700 1 --2- 172.21.15.104:0/2173313918 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5400cd7f0 0x7fa5401f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.438+0000 7fa545de8700 1 -- 172.21.15.104:0/2173313918 >> 172.21.15.104:0/2173313918 conn(0x7fa5400afc60 msgr2=0x7fa54015f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:42.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.438+0000 7fa545de8700 1 -- 172.21.15.104:0/2173313918 shutdown_connections 2023-07-14T11:55:42.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.438+0000 7fa545de8700 1 -- 172.21.15.104:0/2173313918 wait complete. 2023-07-14T11:55:42.444 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:55:42.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.630+0000 7f1758089700 1 Processor -- start 2023-07-14T11:55:42.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.630+0000 7f1758089700 1 -- start start 2023-07-14T11:55:42.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1758089700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f175015c070 0x7f17500ad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:42.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1758089700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f17500ad8b0 0x7f17500afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:42.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1758089700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17500b06d0 con 0x7f17500b0270 2023-07-14T11:55:42.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1758089700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17500cd530 con 0x7f17500ad8b0 2023-07-14T11:55:42.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1758089700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175015c440 con 0x7f175015c070 2023-07-14T11:55:42.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1755624700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f17500ad8b0 0x7f17500afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:42.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1756626700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f17500b0270 0x7f175015fb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:54638/0 (socket says 172.21.15.104:54638) 2023-07-14T11:55:42.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1756626700 1 -- 172.21.15.104:0/2470836426 learned_addr learned my addr 172.21.15.104:0/2470836426 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:42.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1755e25700 1 --2- 172.21.15.104:0/2470836426 >> v2:172.21.15.121:3300/0 conn(0x7f175015c070 0x7f17500ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:42.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1755624700 1 -- 172.21.15.104:0/2470836426 >> v2:172.21.15.121:3300/0 conn(0x7f175015c070 msgr2=0x7f17500ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:42.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1755624700 1 --2- 172.21.15.104:0/2470836426 >> v2:172.21.15.121:3300/0 conn(0x7f175015c070 0x7f17500ad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1755624700 1 -- 172.21.15.104:0/2470836426 >> v1:172.21.15.104:6789/0 conn(0x7f17500b0270 legacy=0x7f175015fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:42.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1755624700 1 -- 172.21.15.104:0/2470836426 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17500cc1b0 con 0x7f17500ad8b0 2023-07-14T11:55:42.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1754e23700 1 -- 172.21.15.104:0/2470836426 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472202766 0 0) 0x7f17500b06d0 con 0x7f17500b0270 2023-07-14T11:55:42.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1755624700 1 --2- 172.21.15.104:0/2470836426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17500ad8b0 0x7f17500afd30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f175006ba30 tx=0x7f1740009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30d803f4a8aa7c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:42.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1754e23700 1 -- 172.21.15.104:0/2470836426 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740011830 con 0x7f17500ad8b0 2023-07-14T11:55:42.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1754e23700 1 -- 172.21.15.104:0/2470836426 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1740011990 con 0x7f17500ad8b0 2023-07-14T11:55:42.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.634+0000 7f1754e23700 1 -- 172.21.15.104:0/2470836426 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740011d20 con 0x7f17500ad8b0 2023-07-14T11:55:42.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.638+0000 7f1758089700 1 -- 172.21.15.104:0/2470836426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17500ad8b0 msgr2=0x7f17500afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:42.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.638+0000 7f1758089700 1 --2- 172.21.15.104:0/2470836426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17500ad8b0 0x7f17500afd30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f175006ba30 tx=0x7f1740009b30 comp rx=0 tx=0).stop 2023-07-14T11:55:42.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.638+0000 7f1758089700 1 -- 172.21.15.104:0/2470836426 shutdown_connections 2023-07-14T11:55:42.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.638+0000 7f1758089700 1 --2- 172.21.15.104:0/2470836426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17500ad8b0 0x7f17500afd30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.638+0000 7f1758089700 1 --2- 172.21.15.104:0/2470836426 >> v2:172.21.15.121:3300/0 conn(0x7f175015c070 0x7f17500ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.638+0000 7f1758089700 1 -- 172.21.15.104:0/2470836426 >> 172.21.15.104:0/2470836426 conn(0x7f17500cad30 msgr2=0x7f17500cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:42.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.638+0000 7f1758089700 1 -- 172.21.15.104:0/2470836426 shutdown_connections 2023-07-14T11:55:42.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.638+0000 7f1758089700 1 -- 172.21.15.104:0/2470836426 wait complete. 2023-07-14T11:55:42.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.638+0000 7f1758089700 1 Processor -- start 2023-07-14T11:55:42.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.638+0000 7f1758089700 1 -- start start 2023-07-14T11:55:42.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1758089700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17500ad8b0 0x7f1750152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:42.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1758089700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17500b0270 0x7f17500c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:42.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1758089700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175015c070 0x7f175015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:42.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1758089700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17500c4520 con 0x7f17500b0270 2023-07-14T11:55:42.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1758089700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17500ae770 con 0x7f17500ad8b0 2023-07-14T11:55:42.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1758089700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17500ade40 con 0x7f175015c070 2023-07-14T11:55:42.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1755624700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17500b0270 0x7f17500c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:42.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1755624700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17500b0270 0x7f17500c3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41934/0 (socket says 172.21.15.104:41934) 2023-07-14T11:55:42.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1755624700 1 -- 172.21.15.104:0/2691015817 learned_addr learned my addr 172.21.15.104:0/2691015817 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:42.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1755e25700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17500ad8b0 0x7f1750152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:42.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1756626700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175015c070 0x7f175015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:42.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1755624700 1 -- 172.21.15.104:0/2691015817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175015c070 msgr2=0x7f175015ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:42.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1755624700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175015c070 0x7f175015ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1755624700 1 -- 172.21.15.104:0/2691015817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17500ad8b0 msgr2=0x7f1750152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:42.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1755624700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17500ad8b0 0x7f1750152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1755624700 1 -- 172.21.15.104:0/2691015817 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17400094b0 con 0x7f17500b0270 2023-07-14T11:55:42.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1755e25700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17500ad8b0 0x7f1750152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:42.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1756626700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175015c070 0x7f175015ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:55:42.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1755624700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17500b0270 0x7f17500c3bd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1740009fd0 tx=0x7f1740004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:42.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f17467fc700 1 -- 172.21.15.104:0/2691015817 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17400045b0 con 0x7f17500b0270 2023-07-14T11:55:42.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f17467fc700 1 -- 172.21.15.104:0/2691015817 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174001f070 con 0x7f17500b0270 2023-07-14T11:55:42.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1758089700 1 -- 172.21.15.104:0/2691015817 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17500ae120 con 0x7f17500b0270 2023-07-14T11:55:42.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f17467fc700 1 -- 172.21.15.104:0/2691015817 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17400038e0 con 0x7f17500b0270 2023-07-14T11:55:42.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.642+0000 7f1758089700 1 -- 172.21.15.104:0/2691015817 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1750152d60 con 0x7f17500b0270 2023-07-14T11:55:42.632 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.646+0000 7f1758089700 1 -- 172.21.15.104:0/2691015817 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17500ae8e0 con 0x7f17500b0270 2023-07-14T11:55:42.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.646+0000 7f17467fc700 1 -- 172.21.15.104:0/2691015817 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1740021030 con 0x7f17500b0270 2023-07-14T11:55:42.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.646+0000 7f17467fc700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1734066320 0x7f17340687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:42.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.646+0000 7f17467fc700 1 -- 172.21.15.104:0/2691015817 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f1740085450 con 0x7f17500b0270 2023-07-14T11:55:42.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.646+0000 7f1755e25700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1734066320 0x7f17340687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:42.634 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.650+0000 7f1755e25700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1734066320 0x7f17340687d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f174c009fd0 tx=0x7f174c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:42.637 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.650+0000 7f17467fc700 1 -- 172.21.15.104:0/2691015817 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f17400525f0 con 0x7f17500b0270 2023-07-14T11:55:42.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.834+0000 7f1758089700 1 -- 172.21.15.104:0/2691015817 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f17500cca20 con 0x7f17500b0270 2023-07-14T11:55:42.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.834+0000 7f17467fc700 1 -- 172.21.15.104:0/2691015817 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1740055c10 con 0x7f17500b0270 2023-07-14T11:55:42.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.838+0000 7f1758089700 1 -- 172.21.15.104:0/2691015817 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1734066320 msgr2=0x7f17340687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:42.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.838+0000 7f1758089700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1734066320 0x7f17340687d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f174c009fd0 tx=0x7f174c00b040 comp rx=0 tx=0).stop 2023-07-14T11:55:42.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.838+0000 7f1758089700 1 -- 172.21.15.104:0/2691015817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17500b0270 msgr2=0x7f17500c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:42.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.838+0000 7f1758089700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17500b0270 0x7f17500c3bd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1740009fd0 tx=0x7f1740004770 comp rx=0 tx=0).stop 2023-07-14T11:55:42.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.838+0000 7f1758089700 1 -- 172.21.15.104:0/2691015817 shutdown_connections 2023-07-14T11:55:42.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.838+0000 7f1758089700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1734066320 0x7f17340687d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.838+0000 7f1758089700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17500b0270 0x7f17500c3bd0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.838+0000 7f1758089700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f175015c070 0x7f175015ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.838+0000 7f1758089700 1 --2- 172.21.15.104:0/2691015817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17500ad8b0 0x7f1750152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:42.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.838+0000 7f1758089700 1 -- 172.21.15.104:0/2691015817 >> 172.21.15.104:0/2691015817 conn(0x7f17500cad30 msgr2=0x7f17500c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:42.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.838+0000 7f1758089700 1 -- 172.21.15.104:0/2691015817 shutdown_connections 2023-07-14T11:55:42.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:42.838+0000 7f1758089700 1 -- 172.21.15.104:0/2691015817 wait complete. 2023-07-14T11:55:42.826 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:55:42.844 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:55:42.845 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:55:42.860 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:55:43.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.086+0000 7f0e8143b700 1 Processor -- start 2023-07-14T11:55:43.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.086+0000 7f0e8143b700 1 -- start start 2023-07-14T11:55:43.074 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.086+0000 7f0e8143b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0e7c159b70 0x7f0e7c06ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.074 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.086+0000 7f0e8143b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e7c0cb4d0 0x7f0e7c06f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.074 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.086+0000 7f0e8143b700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7c0cba60 con 0x7f0e7c159f40 2023-07-14T11:55:43.074 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.086+0000 7f0e8143b700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7c0cbd40 con 0x7f0e7c159b70 2023-07-14T11:55:43.074 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.086+0000 7f0e8143b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0cc020 con 0x7f0e7c0cb4d0 2023-07-14T11:55:43.075 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.086+0000 7f0e7a7fc700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f0e7c159f40 0x7f0e7c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:54652/0 (socket says 172.21.15.104:54652) 2023-07-14T11:55:43.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.086+0000 7f0e7a7fc700 1 -- 172.21.15.104:0/3279729226 learned_addr learned my addr 172.21.15.104:0/3279729226 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:43.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.086+0000 7f0e7affd700 1 --2- 172.21.15.104:0/3279729226 >> v2:172.21.15.104:3300/0 conn(0x7f0e7c159b70 0x7f0e7c06ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e7bfff700 1 --2- 172.21.15.104:0/3279729226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e7c0cb4d0 0x7f0e7c06f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e79ffb700 1 -- 172.21.15.104:0/3279729226 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812449420 0 0) 0x7f0e7c0cba60 con 0x7f0e7c159f40 2023-07-14T11:55:43.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e79ffb700 1 -- 172.21.15.104:0/3279729226 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e640035d0 con 0x7f0e7c159f40 2023-07-14T11:55:43.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e7affd700 1 -- 172.21.15.104:0/3279729226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e7c0cb4d0 msgr2=0x7f0e7c06f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:43.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e7affd700 1 --2- 172.21.15.104:0/3279729226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e7c0cb4d0 0x7f0e7c06f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e7affd700 1 -- 172.21.15.104:0/3279729226 >> v1:172.21.15.104:6789/0 conn(0x7f0e7c159f40 legacy=0x7f0e7c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e7affd700 1 -- 172.21.15.104:0/3279729226 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c06d7d0 con 0x7f0e7c159b70 2023-07-14T11:55:43.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e7affd700 1 --2- 172.21.15.104:0/3279729226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e7c159b70 0x7f0e7c06ec00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009850 tx=0x7f0e6c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589e55c323a93683 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:43.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e79ffb700 1 -- 172.21.15.104:0/3279729226 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c011830 con 0x7f0e7c159b70 2023-07-14T11:55:43.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e79ffb700 1 -- 172.21.15.104:0/3279729226 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c011990 con 0x7f0e7c159b70 2023-07-14T11:55:43.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e79ffb700 1 -- 172.21.15.104:0/3279729226 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c011d00 con 0x7f0e7c159b70 2023-07-14T11:55:43.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e8143b700 1 -- 172.21.15.104:0/3279729226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e7c159b70 msgr2=0x7f0e7c06ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e8143b700 1 --2- 172.21.15.104:0/3279729226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e7c159b70 0x7f0e7c06ec00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009850 tx=0x7f0e6c009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:43.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e8143b700 1 -- 172.21.15.104:0/3279729226 shutdown_connections 2023-07-14T11:55:43.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e8143b700 1 --2- 172.21.15.104:0/3279729226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e7c0cb4d0 0x7f0e7c06f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e8143b700 1 --2- 172.21.15.104:0/3279729226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e7c159b70 0x7f0e7c06ec00 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e8143b700 1 -- 172.21.15.104:0/3279729226 >> 172.21.15.104:0/3279729226 conn(0x7f0e7c0cc500 msgr2=0x7f0e7c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:43.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e8143b700 1 -- 172.21.15.104:0/3279729226 shutdown_connections 2023-07-14T11:55:43.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.090+0000 7f0e8143b700 1 -- 172.21.15.104:0/3279729226 wait complete. 2023-07-14T11:55:43.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.094+0000 7f0e8143b700 1 Processor -- start 2023-07-14T11:55:43.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.094+0000 7f0e8143b700 1 -- start start 2023-07-14T11:55:43.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.094+0000 7f0e8143b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e7c0cb4d0 0x7f0e7c147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.094+0000 7f0e8143b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e7c159b70 0x7f0e7c1488f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.094+0000 7f0e8143b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e7c159f40 0x7f0e7c0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.094+0000 7f0e8143b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0c42b0 con 0x7f0e7c0cb4d0 2023-07-14T11:55:43.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.094+0000 7f0e8143b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0c36e0 con 0x7f0e7c159f40 2023-07-14T11:55:43.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.094+0000 7f0e8143b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0c3980 con 0x7f0e7c159b70 2023-07-14T11:55:43.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.094+0000 7f0e7affd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e7c0cb4d0 0x7f0e7c147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.094+0000 7f0e7affd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e7c0cb4d0 0x7f0e7c147d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41944/0 (socket says 172.21.15.104:41944) 2023-07-14T11:55:43.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.094+0000 7f0e7affd700 1 -- 172.21.15.104:0/2603906398 learned_addr learned my addr 172.21.15.104:0/2603906398 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:43.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.094+0000 7f0e7a7fc700 1 --2- 172.21.15.104:0/2603906398 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e7c159b70 0x7f0e7c1488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.094+0000 7f0e7bfff700 1 --2- 172.21.15.104:0/2603906398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e7c159f40 0x7f0e7c0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.098+0000 7f0e7affd700 1 -- 172.21.15.104:0/2603906398 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e7c159b70 msgr2=0x7f0e7c1488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.098+0000 7f0e7affd700 1 --2- 172.21.15.104:0/2603906398 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e7c159b70 0x7f0e7c1488f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.098+0000 7f0e7affd700 1 -- 172.21.15.104:0/2603906398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e7c159f40 msgr2=0x7f0e7c0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.098+0000 7f0e7affd700 1 --2- 172.21.15.104:0/2603906398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e7c159f40 0x7f0e7c0b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.098+0000 7f0e7affd700 1 -- 172.21.15.104:0/2603906398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c0094b0 con 0x7f0e7c0cb4d0 2023-07-14T11:55:43.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.098+0000 7f0e7bfff700 1 --2- 172.21.15.104:0/2603906398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e7c159f40 0x7f0e7c0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:55:43.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.098+0000 7f0e7affd700 1 --2- 172.21.15.104:0/2603906398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e7c0cb4d0 0x7f0e7c147d70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0090b0 tx=0x7f0e6c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:43.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.098+0000 7f0e637fe700 1 -- 172.21.15.104:0/2603906398 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c004610 con 0x7f0e7c0cb4d0 2023-07-14T11:55:43.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.098+0000 7f0e637fe700 1 -- 172.21.15.104:0/2603906398 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c01f070 con 0x7f0e7c0cb4d0 2023-07-14T11:55:43.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.098+0000 7f0e637fe700 1 -- 172.21.15.104:0/2603906398 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0038e0 con 0x7f0e7c0cb4d0 2023-07-14T11:55:43.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.098+0000 7f0e8143b700 1 -- 172.21.15.104:0/2603906398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7c0c3c60 con 0x7f0e7c0cb4d0 2023-07-14T11:55:43.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.098+0000 7f0e8143b700 1 -- 172.21.15.104:0/2603906398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7c1566a0 con 0x7f0e7c0cb4d0 2023-07-14T11:55:43.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.098+0000 7f0e8143b700 1 -- 172.21.15.104:0/2603906398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7c04dcb0 con 0x7f0e7c0cb4d0 2023-07-14T11:55:43.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.098+0000 7f0e637fe700 1 -- 172.21.15.104:0/2603906398 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f0e6c003a40 con 0x7f0e7c0cb4d0 2023-07-14T11:55:43.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.102+0000 7f0e637fe700 1 --2- 172.21.15.104:0/2603906398 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0e68066320 0x7f0e680687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.102+0000 7f0e637fe700 1 -- 172.21.15.104:0/2603906398 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f0e6c021070 con 0x7f0e7c0cb4d0 2023-07-14T11:55:43.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.102+0000 7f0e7a7fc700 1 --2- 172.21.15.104:0/2603906398 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0e68066320 0x7f0e680687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.102+0000 7f0e7a7fc700 1 --2- 172.21.15.104:0/2603906398 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0e68066320 0x7f0e680687d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c15cdf0 tx=0x7f0e70009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:43.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.106+0000 7f0e637fe700 1 -- 172.21.15.104:0/2603906398 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f0e6c052970 con 0x7f0e7c0cb4d0 2023-07-14T11:55:43.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.282+0000 7f0e8143b700 1 -- 172.21.15.104:0/2603906398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e7c0b2420 con 0x7f0e7c0cb4d0 2023-07-14T11:55:43.273 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.286+0000 7f0e637fe700 1 -- 172.21.15.104:0/2603906398 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+20311 (secure 0 0 0) 0x7f0e6c055f90 con 0x7f0e7c0cb4d0 2023-07-14T11:55:43.273 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:43.274 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":45,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:55:10.654581+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:54:56.370313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-14T11:54:57.608947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"data_new","create_time":"2023-07-14T11:55:08.099546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:55:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.290+0000 7f0e8143b700 1 -- 172.21.15.104:0/2603906398 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0e68066320 msgr2=0x7f0e680687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.290+0000 7f0e8143b700 1 --2- 172.21.15.104:0/2603906398 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0e68066320 0x7f0e680687d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c15cdf0 tx=0x7f0e70009040 comp rx=0 tx=0).stop 2023-07-14T11:55:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.290+0000 7f0e8143b700 1 -- 172.21.15.104:0/2603906398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e7c0cb4d0 msgr2=0x7f0e7c147d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.290+0000 7f0e8143b700 1 --2- 172.21.15.104:0/2603906398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e7c0cb4d0 0x7f0e7c147d70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0090b0 tx=0x7f0e6c004770 comp rx=0 tx=0).stop 2023-07-14T11:55:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.290+0000 7f0e8143b700 1 -- 172.21.15.104:0/2603906398 shutdown_connections 2023-07-14T11:55:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.290+0000 7f0e8143b700 1 --2- 172.21.15.104:0/2603906398 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f0e68066320 0x7f0e680687d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.290+0000 7f0e8143b700 1 --2- 172.21.15.104:0/2603906398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e7c159f40 0x7f0e7c0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.290+0000 7f0e8143b700 1 --2- 172.21.15.104:0/2603906398 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e7c159b70 0x7f0e7c1488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.290+0000 7f0e8143b700 1 --2- 172.21.15.104:0/2603906398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e7c0cb4d0 0x7f0e7c147d70 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.290+0000 7f0e8143b700 1 -- 172.21.15.104:0/2603906398 >> 172.21.15.104:0/2603906398 conn(0x7f0e7c0cc500 msgr2=0x7f0e7c156fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.290+0000 7f0e8143b700 1 -- 172.21.15.104:0/2603906398 shutdown_connections 2023-07-14T11:55:43.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.290+0000 7f0e8143b700 1 -- 172.21.15.104:0/2603906398 wait complete. 2023-07-14T11:55:43.300 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:54:56.370313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:54:57.608947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'data_new', 'create_time': '2023-07-14T11:55:08.099546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:55:43.302 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:55:43.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.490+0000 7febca92b700 1 Processor -- start 2023-07-14T11:55:43.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.490+0000 7febca92b700 1 -- start start 2023-07-14T11:55:43.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febca92b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febc40b0ee0 0x7febc40ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febca92b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7febc40d07e0 0x7febc40af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febca92b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc40cd4e0 con 0x7febc40cd090 2023-07-14T11:55:43.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febca92b700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc40d0c60 con 0x7febc40d07e0 2023-07-14T11:55:43.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febca92b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc40d1a60 con 0x7febc40b0ee0 2023-07-14T11:55:43.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febca12a700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7febc40d07e0 0x7febc40af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febca12a700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7febc40d07e0 0x7febc40af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41948/0 (socket says 172.21.15.104:41948) 2023-07-14T11:55:43.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febca12a700 1 -- 172.21.15.104:0/2288293937 learned_addr learned my addr 172.21.15.104:0/2288293937 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:43.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febc9929700 1 --2- 172.21.15.104:0/2288293937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febc40b0ee0 0x7febc40ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febca12a700 1 -- 172.21.15.104:0/2288293937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febc40b0ee0 msgr2=0x7febc40ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:43.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febca12a700 1 --2- 172.21.15.104:0/2288293937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febc40b0ee0 0x7febc40ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febca12a700 1 -- 172.21.15.104:0/2288293937 >> v1:172.21.15.121:6789/0 conn(0x7febc40cd090 legacy=0x7febc40d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febca12a700 1 -- 172.21.15.104:0/2288293937 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc40d1ef0 con 0x7febc40d07e0 2023-07-14T11:55:43.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febca12a700 1 --2- 172.21.15.104:0/2288293937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febc40d07e0 0x7febc40af0c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7febbc009e80 tx=0x7febbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5022c390f454e0c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:43.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febc8927700 1 -- 172.21.15.104:0/2288293937 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc00e070 con 0x7febc40d07e0 2023-07-14T11:55:43.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febc8927700 1 -- 172.21.15.104:0/2288293937 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc00b8d0 con 0x7febc40d07e0 2023-07-14T11:55:43.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febc8927700 1 -- 172.21.15.104:0/2288293937 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc00bc80 con 0x7febc40d07e0 2023-07-14T11:55:43.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febca92b700 1 -- 172.21.15.104:0/2288293937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febc40d07e0 msgr2=0x7febc40af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.494+0000 7febca92b700 1 --2- 172.21.15.104:0/2288293937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febc40d07e0 0x7febc40af0c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7febbc009e80 tx=0x7febbc00b330 comp rx=0 tx=0).stop 2023-07-14T11:55:43.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.498+0000 7febca92b700 1 -- 172.21.15.104:0/2288293937 shutdown_connections 2023-07-14T11:55:43.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.498+0000 7febca92b700 1 --2- 172.21.15.104:0/2288293937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febc40d07e0 0x7febc40af0c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.498+0000 7febca92b700 1 --2- 172.21.15.104:0/2288293937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febc40b0ee0 0x7febc40ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.498+0000 7febca92b700 1 -- 172.21.15.104:0/2288293937 >> 172.21.15.104:0/2288293937 conn(0x7febc4060c40 msgr2=0x7febc409b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:43.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.498+0000 7febca92b700 1 -- 172.21.15.104:0/2288293937 shutdown_connections 2023-07-14T11:55:43.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.498+0000 7febca92b700 1 -- 172.21.15.104:0/2288293937 wait complete. 2023-07-14T11:55:43.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.498+0000 7febca92b700 1 Processor -- start 2023-07-14T11:55:43.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.498+0000 7febca92b700 1 -- start start 2023-07-14T11:55:43.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.498+0000 7febca92b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febc40b0ee0 0x7febc4153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.498+0000 7febca92b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febc40cd090 0x7febc40b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.498+0000 7febca92b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febc40d07e0 0x7febc40ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.498+0000 7febca92b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc40b58f0 con 0x7febc40cd090 2023-07-14T11:55:43.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.498+0000 7febca92b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc40b48d0 con 0x7febc40d07e0 2023-07-14T11:55:43.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.498+0000 7febca92b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc40b4bb0 con 0x7febc40b0ee0 2023-07-14T11:55:43.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febc9128700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febc40cd090 0x7febc40b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febc9128700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febc40cd090 0x7febc40b7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41958/0 (socket says 172.21.15.104:41958) 2023-07-14T11:55:43.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febc9128700 1 -- 172.21.15.104:0/3445003332 learned_addr learned my addr 172.21.15.104:0/3445003332 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:43.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febc9929700 1 --2- 172.21.15.104:0/3445003332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febc40b0ee0 0x7febc4153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febca12a700 1 --2- 172.21.15.104:0/3445003332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febc40d07e0 0x7febc40ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febc9128700 1 -- 172.21.15.104:0/3445003332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febc40b0ee0 msgr2=0x7febc4153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febc9128700 1 --2- 172.21.15.104:0/3445003332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febc40b0ee0 0x7febc4153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febc9128700 1 -- 172.21.15.104:0/3445003332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febc40d07e0 msgr2=0x7febc40ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febc9128700 1 --2- 172.21.15.104:0/3445003332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febc40d07e0 0x7febc40ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febc9128700 1 -- 172.21.15.104:0/3445003332 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc009b30 con 0x7febc40cd090 2023-07-14T11:55:43.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febc9128700 1 --2- 172.21.15.104:0/3445003332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febc40cd090 0x7febc40b7c00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7febb400c500 tx=0x7febb400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:43.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febba7fc700 1 -- 172.21.15.104:0/3445003332 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb40052d0 con 0x7febc40cd090 2023-07-14T11:55:43.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febba7fc700 1 -- 172.21.15.104:0/3445003332 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb4005430 con 0x7febc40cd090 2023-07-14T11:55:43.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febca92b700 1 -- 172.21.15.104:0/3445003332 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc40b5580 con 0x7febc40cd090 2023-07-14T11:55:43.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febba7fc700 1 -- 172.21.15.104:0/3445003332 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb40055a0 con 0x7febc40cd090 2023-07-14T11:55:43.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febca92b700 1 -- 172.21.15.104:0/3445003332 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc40b8140 con 0x7febc40cd090 2023-07-14T11:55:43.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.502+0000 7febca92b700 1 -- 172.21.15.104:0/3445003332 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc40b53e0 con 0x7febc40cd090 2023-07-14T11:55:43.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.506+0000 7febba7fc700 1 -- 172.21.15.104:0/3445003332 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7febb40036a0 con 0x7febc40cd090 2023-07-14T11:55:43.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.506+0000 7febba7fc700 1 --2- 172.21.15.104:0/3445003332 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feba80662f0 0x7feba80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.506+0000 7febba7fc700 1 -- 172.21.15.104:0/3445003332 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7febb40830c0 con 0x7febc40cd090 2023-07-14T11:55:43.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.506+0000 7febc9929700 1 --2- 172.21.15.104:0/3445003332 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feba80662f0 0x7feba80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.510+0000 7febc9929700 1 --2- 172.21.15.104:0/3445003332 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feba80662f0 0x7feba80687a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7febc0009b50 tx=0x7febc0009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:43.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.514+0000 7febba7fc700 1 -- 172.21.15.104:0/3445003332 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7febc40b53e0 con 0x7febc40cd090 2023-07-14T11:55:43.671 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.686+0000 7febca92b700 1 -- 172.21.15.104:0/3445003332 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7febc40cd4e0 con 0x7febc40cd090 2023-07-14T11:55:43.672 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.686+0000 7febba7fc700 1 -- 172.21.15.104:0/3445003332 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7febc40cd4e0 con 0x7febc40cd090 2023-07-14T11:55:43.672 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:55:43.677 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.690+0000 7febca92b700 1 -- 172.21.15.104:0/3445003332 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feba80662f0 msgr2=0x7feba80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.677 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.690+0000 7febca92b700 1 --2- 172.21.15.104:0/3445003332 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feba80662f0 0x7feba80687a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7febc0009b50 tx=0x7febc0009240 comp rx=0 tx=0).stop 2023-07-14T11:55:43.677 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.690+0000 7febca92b700 1 -- 172.21.15.104:0/3445003332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febc40cd090 msgr2=0x7febc40b7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.690+0000 7febca92b700 1 --2- 172.21.15.104:0/3445003332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febc40cd090 0x7febc40b7c00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7febb400c500 tx=0x7febb400c810 comp rx=0 tx=0).stop 2023-07-14T11:55:43.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.690+0000 7febca92b700 1 -- 172.21.15.104:0/3445003332 shutdown_connections 2023-07-14T11:55:43.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.690+0000 7febca92b700 1 --2- 172.21.15.104:0/3445003332 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feba80662f0 0x7feba80687a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.690+0000 7febca92b700 1 --2- 172.21.15.104:0/3445003332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febc40cd090 0x7febc40b7c00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.690+0000 7febca92b700 1 --2- 172.21.15.104:0/3445003332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febc40d07e0 0x7febc40ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.690+0000 7febca92b700 1 --2- 172.21.15.104:0/3445003332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febc40b0ee0 0x7febc4153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.690+0000 7febca92b700 1 -- 172.21.15.104:0/3445003332 >> 172.21.15.104:0/3445003332 conn(0x7febc4060c40 msgr2=0x7febc415ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:43.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.690+0000 7febca92b700 1 -- 172.21.15.104:0/3445003332 shutdown_connections 2023-07-14T11:55:43.679 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.690+0000 7febca92b700 1 -- 172.21.15.104:0/3445003332 wait complete. 2023-07-14T11:55:43.699 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:55:43.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.890+0000 7f965c824700 1 Processor -- start 2023-07-14T11:55:43.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.890+0000 7f965c824700 1 -- start start 2023-07-14T11:55:43.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.890+0000 7f965c824700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9658160000 0x7f965806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.879 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.890+0000 7f965c824700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96580cdfb0 con 0x7f96580cd7f0 2023-07-14T11:55:43.879 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.890+0000 7f965c824700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96580cdcd0 con 0x7f96581603d0 2023-07-14T11:55:43.879 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.890+0000 7f965c824700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9658070e40 con 0x7f9658160000 2023-07-14T11:55:43.879 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f965759e700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f96580cd7f0 0x7f96580c0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:54654/0 (socket says 172.21.15.104:54654) 2023-07-14T11:55:43.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f965759e700 1 -- 172.21.15.104:0/1212433432 learned_addr learned my addr 172.21.15.104:0/1212433432 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:43.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9656d9d700 1 --2- 172.21.15.104:0/1212433432 >> v2:172.21.15.121:3300/0 conn(0x7f9658160000 0x7f965806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9655d9b700 1 -- 172.21.15.104:0/1212433432 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384796175 0 0) 0x7f9650001490 con 0x7f96580cd7f0 2023-07-14T11:55:43.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9655d9b700 1 -- 172.21.15.104:0/1212433432 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96400035d0 con 0x7f96580cd7f0 2023-07-14T11:55:43.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9655d9b700 1 -- 172.21.15.104:0/1212433432 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3883539967 0 0) 0x7f96400035d0 con 0x7f96580cd7f0 2023-07-14T11:55:43.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9655d9b700 1 -- 172.21.15.104:0/1212433432 >> v2:172.21.15.121:3300/0 conn(0x7f9658160000 msgr2=0x7f965806dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:43.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9655d9b700 1 --2- 172.21.15.104:0/1212433432 >> v2:172.21.15.121:3300/0 conn(0x7f9658160000 0x7f965806dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9655d9b700 1 -- 172.21.15.104:0/1212433432 >> v1:172.21.15.121:6789/0 conn(0x7f96581603d0 legacy=0x7f96580c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9655d9b700 1 -- 172.21.15.104:0/1212433432 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9658070780 con 0x7f96580cd7f0 2023-07-14T11:55:43.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9655d9b700 1 -- 172.21.15.104:0/1212433432 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f9650001b80 con 0x7f96580cd7f0 2023-07-14T11:55:43.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9655d9b700 1 -- 172.21.15.104:0/1212433432 >> v1:172.21.15.104:6789/0 conn(0x7f96580cd7f0 legacy=0x7f96580c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9655d9b700 1 --2- 172.21.15.104:0/1212433432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9640003a20 0x7f9640003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9655d9b700 1 --2- 172.21.15.104:0/1212433432 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9640004870 0x7f9640004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9655d9b700 1 --2- 172.21.15.104:0/1212433432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9640005220 0x7f964000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9655d9b700 1 -- 172.21.15.104:0/1212433432 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9658070e40 con 0x7f9640005220 2023-07-14T11:55:43.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9655d9b700 1 -- 172.21.15.104:0/1212433432 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964000c0a0 con 0x7f9640003a20 2023-07-14T11:55:43.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9655d9b700 1 -- 172.21.15.104:0/1212433432 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964000c260 con 0x7f9640004870 2023-07-14T11:55:43.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f965759e700 1 --2- 172.21.15.104:0/1212433432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9640005220 0x7f964000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f9656d9d700 1 --2- 172.21.15.104:0/1212433432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9640003a20 0x7f9640003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f965659c700 1 --2- 172.21.15.104:0/1212433432 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9640004870 0x7f9640004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f965759e700 1 -- 172.21.15.104:0/1212433432 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9640004870 msgr2=0x7f9640004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:43.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f965759e700 1 --2- 172.21.15.104:0/1212433432 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9640004870 0x7f9640004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f965759e700 1 -- 172.21.15.104:0/1212433432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9640003a20 msgr2=0x7f9640003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:43.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f965759e700 1 --2- 172.21.15.104:0/1212433432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9640003a20 0x7f9640003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f965759e700 1 -- 172.21.15.104:0/1212433432 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400035d0 con 0x7f9640005220 2023-07-14T11:55:43.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f965759e700 1 -- 172.21.15.104:0/1212433432 reap_dead start 2023-07-14T11:55:43.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.894+0000 7f965759e700 1 --2- 172.21.15.104:0/1212433432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9640005220 0x7f964000ba30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f965000a830 tx=0x7f96400058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bde61ecda6700ef server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:43.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.898+0000 7f9655d9b700 1 -- 172.21.15.104:0/1212433432 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965000a490 con 0x7f9640005220 2023-07-14T11:55:43.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.898+0000 7f9655d9b700 1 -- 172.21.15.104:0/1212433432 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96500072f0 con 0x7f9640005220 2023-07-14T11:55:43.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.898+0000 7f9655d9b700 1 -- 172.21.15.104:0/1212433432 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96500076a0 con 0x7f9640005220 2023-07-14T11:55:43.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.898+0000 7f965c824700 1 -- 172.21.15.104:0/1212433432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9640005220 msgr2=0x7f964000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.898+0000 7f965c824700 1 --2- 172.21.15.104:0/1212433432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9640005220 0x7f964000ba30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f965000a830 tx=0x7f96400058a0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.898+0000 7f965c824700 1 -- 172.21.15.104:0/1212433432 shutdown_connections 2023-07-14T11:55:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.898+0000 7f965c824700 1 --2- 172.21.15.104:0/1212433432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9640005220 0x7f964000ba30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.898+0000 7f965c824700 1 -- 172.21.15.104:0/1212433432 >> 172.21.15.104:0/1212433432 conn(0x7f96580afc60 msgr2=0x7f96580b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.898+0000 7f965c824700 1 -- 172.21.15.104:0/1212433432 shutdown_connections 2023-07-14T11:55:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.898+0000 7f965c824700 1 -- 172.21.15.104:0/1212433432 wait complete. 2023-07-14T11:55:43.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965c824700 1 Processor -- start 2023-07-14T11:55:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965c824700 1 -- start start 2023-07-14T11:55:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965c824700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9640005220 0x7f96581521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965c824700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96581607a0 0x7f9658160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965c824700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658163a40 0x7f9658163eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965c824700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96580c5220 con 0x7f96581607a0 2023-07-14T11:55:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965c824700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9658161150 con 0x7f9658163a40 2023-07-14T11:55:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965c824700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9658152b70 con 0x7f9640005220 2023-07-14T11:55:43.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96581607a0 0x7f9658160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96581607a0 0x7f9658160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41976/0 (socket says 172.21.15.104:41976) 2023-07-14T11:55:43.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965659c700 1 -- 172.21.15.104:0/3833521922 learned_addr learned my addr 172.21.15.104:0/3833521922 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:43.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f9656d9d700 1 --2- 172.21.15.104:0/3833521922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9640005220 0x7f96581521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965759e700 1 --2- 172.21.15.104:0/3833521922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658163a40 0x7f9658163eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965659c700 1 -- 172.21.15.104:0/3833521922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9640005220 msgr2=0x7f96581521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965659c700 1 --2- 172.21.15.104:0/3833521922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9640005220 0x7f96581521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965659c700 1 -- 172.21.15.104:0/3833521922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658163a40 msgr2=0x7f9658163eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:43.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965659c700 1 --2- 172.21.15.104:0/3833521922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658163a40 0x7f9658163eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:43.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965659c700 1 -- 172.21.15.104:0/3833521922 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96581f5790 con 0x7f96581607a0 2023-07-14T11:55:43.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f9656d9d700 1 --2- 172.21.15.104:0/3833521922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9640005220 0x7f96581521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:43.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965659c700 1 --2- 172.21.15.104:0/3833521922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96581607a0 0x7f9658160c10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f964c00e5e0 tx=0x7f964c00e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:43.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f963f7fe700 1 -- 172.21.15.104:0/3833521922 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c0052e0 con 0x7f96581607a0 2023-07-14T11:55:43.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f963f7fe700 1 -- 172.21.15.104:0/3833521922 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c005440 con 0x7f96581607a0 2023-07-14T11:55:43.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965c824700 1 -- 172.21.15.104:0/3833521922 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9658161380 con 0x7f96581607a0 2023-07-14T11:55:43.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f963f7fe700 1 -- 172.21.15.104:0/3833521922 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c0038c0 con 0x7f96581607a0 2023-07-14T11:55:43.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.902+0000 7f965c824700 1 -- 172.21.15.104:0/3833521922 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96580c5430 con 0x7f96581607a0 2023-07-14T11:55:43.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.906+0000 7f965c824700 1 -- 172.21.15.104:0/3833521922 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965804e500 con 0x7f96581607a0 2023-07-14T11:55:43.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.906+0000 7f963f7fe700 1 -- 172.21.15.104:0/3833521922 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f964c003a20 con 0x7f96581607a0 2023-07-14T11:55:43.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.910+0000 7f963f7fe700 1 --2- 172.21.15.104:0/3833521922 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9644066320 0x7f96440687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:43.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.910+0000 7f963f7fe700 1 -- 172.21.15.104:0/3833521922 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f964c083640 con 0x7f96581607a0 2023-07-14T11:55:43.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.910+0000 7f9656d9d700 1 --2- 172.21.15.104:0/3833521922 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9644066320 0x7f96440687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:43.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.910+0000 7f9656d9d700 1 --2- 172.21.15.104:0/3833521922 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9644066320 0x7f96440687d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9648009bf0 tx=0x7f9648008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:43.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:43.914+0000 7f963f7fe700 1 -- 172.21.15.104:0/3833521922 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f964c0508c0 con 0x7f96581607a0 2023-07-14T11:55:44.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.086+0000 7f965c824700 1 -- 172.21.15.104:0/3833521922 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f96580b40d0 con 0x7f96581607a0 2023-07-14T11:55:44.074 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.086+0000 7f963f7fe700 1 -- 172.21.15.104:0/3833521922 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f964c053ee0 con 0x7f96581607a0 2023-07-14T11:55:44.074 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:55:44.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.090+0000 7f965c824700 1 -- 172.21.15.104:0/3833521922 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9644066320 msgr2=0x7f96440687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:44.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.090+0000 7f965c824700 1 --2- 172.21.15.104:0/3833521922 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9644066320 0x7f96440687d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9648009bf0 tx=0x7f9648008040 comp rx=0 tx=0).stop 2023-07-14T11:55:44.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.090+0000 7f965c824700 1 -- 172.21.15.104:0/3833521922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96581607a0 msgr2=0x7f9658160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:44.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.090+0000 7f965c824700 1 --2- 172.21.15.104:0/3833521922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96581607a0 0x7f9658160c10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f964c00e5e0 tx=0x7f964c00e8f0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.090+0000 7f965c824700 1 -- 172.21.15.104:0/3833521922 shutdown_connections 2023-07-14T11:55:44.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.090+0000 7f965c824700 1 --2- 172.21.15.104:0/3833521922 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9644066320 0x7f96440687d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.090+0000 7f965c824700 1 --2- 172.21.15.104:0/3833521922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96581607a0 0x7f9658160c10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.090+0000 7f965c824700 1 --2- 172.21.15.104:0/3833521922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658163a40 0x7f9658163eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.090+0000 7f965c824700 1 --2- 172.21.15.104:0/3833521922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9640005220 0x7f96581521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.090+0000 7f965c824700 1 -- 172.21.15.104:0/3833521922 >> 172.21.15.104:0/3833521922 conn(0x7f96580afc60 msgr2=0x7f96580adbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:44.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.090+0000 7f965c824700 1 -- 172.21.15.104:0/3833521922 shutdown_connections 2023-07-14T11:55:44.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.090+0000 7f965c824700 1 -- 172.21.15.104:0/3833521922 wait complete. 2023-07-14T11:55:44.098 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:55:44.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.290+0000 7f6e59333700 1 Processor -- start 2023-07-14T11:55:44.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.290+0000 7f6e59333700 1 -- start start 2023-07-14T11:55:44.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.290+0000 7f6e59333700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6e540b0ee0 0x7f6e540ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:44.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.290+0000 7f6e59333700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e540cd4e0 con 0x7f6e540d07e0 2023-07-14T11:55:44.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.290+0000 7f6e59333700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e540d1a60 con 0x7f6e540cd090 2023-07-14T11:55:44.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.290+0000 7f6e59333700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e540d0c60 con 0x7f6e540b0ee0 2023-07-14T11:55:44.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.290+0000 7f6e52ffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6e540b0ee0 0x7f6e540ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:44.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.290+0000 7f6e52ffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6e540b0ee0 0x7f6e540ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41984/0 (socket says 172.21.15.104:41984) 2023-07-14T11:55:44.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.290+0000 7f6e537fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f6e540d07e0 0x7f6e540af0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:54666/0 (socket says 172.21.15.104:54666) 2023-07-14T11:55:44.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.290+0000 7f6e52ffd700 1 -- 172.21.15.104:0/2148193889 learned_addr learned my addr 172.21.15.104:0/2148193889 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:44.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.294+0000 7f6e527fc700 1 -- 172.21.15.104:0/2148193889 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930685036 0 0) 0x7f6e4c001490 con 0x7f6e540d07e0 2023-07-14T11:55:44.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.294+0000 7f6e52ffd700 1 -- 172.21.15.104:0/2148193889 >> v1:172.21.15.121:6789/0 conn(0x7f6e540cd090 legacy=0x7f6e540d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:44.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.294+0000 7f6e52ffd700 1 -- 172.21.15.104:0/2148193889 >> v1:172.21.15.104:6789/0 conn(0x7f6e540d07e0 legacy=0x7f6e540af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:44.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.294+0000 7f6e52ffd700 1 -- 172.21.15.104:0/2148193889 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e540d1ef0 con 0x7f6e540b0ee0 2023-07-14T11:55:44.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.294+0000 7f6e52ffd700 1 --2- 172.21.15.104:0/2148193889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e540b0ee0 0x7f6e540ccb50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009910 tx=0x7f6e44009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ea73761cdbfaf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:44.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.294+0000 7f6e527fc700 1 -- 172.21.15.104:0/2148193889 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4400ba60 con 0x7f6e540b0ee0 2023-07-14T11:55:44.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.294+0000 7f6e527fc700 1 -- 172.21.15.104:0/2148193889 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4400bbc0 con 0x7f6e540b0ee0 2023-07-14T11:55:44.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.294+0000 7f6e527fc700 1 -- 172.21.15.104:0/2148193889 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44009e60 con 0x7f6e540b0ee0 2023-07-14T11:55:44.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.294+0000 7f6e59333700 1 -- 172.21.15.104:0/2148193889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e540b0ee0 msgr2=0x7f6e540ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:44.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.294+0000 7f6e59333700 1 --2- 172.21.15.104:0/2148193889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e540b0ee0 0x7f6e540ccb50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009910 tx=0x7f6e44009c20 comp rx=0 tx=0).stop 2023-07-14T11:55:44.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.294+0000 7f6e59333700 1 -- 172.21.15.104:0/2148193889 shutdown_connections 2023-07-14T11:55:44.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.294+0000 7f6e59333700 1 --2- 172.21.15.104:0/2148193889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e540b0ee0 0x7f6e540ccb50 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.294+0000 7f6e59333700 1 -- 172.21.15.104:0/2148193889 >> 172.21.15.104:0/2148193889 conn(0x7f6e54060c40 msgr2=0x7f6e5409b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:44.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.294+0000 7f6e59333700 1 -- 172.21.15.104:0/2148193889 shutdown_connections 2023-07-14T11:55:44.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.294+0000 7f6e59333700 1 -- 172.21.15.104:0/2148193889 wait complete. 2023-07-14T11:55:44.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.298+0000 7f6e59333700 1 Processor -- start 2023-07-14T11:55:44.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.298+0000 7f6e59333700 1 -- start start 2023-07-14T11:55:44.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.298+0000 7f6e59333700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e540b0ee0 0x7f6e540b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:44.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.298+0000 7f6e59333700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e540cd090 0x7f6e5415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:44.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.298+0000 7f6e59333700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e540d07e0 0x7f6e540b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:44.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.298+0000 7f6e59333700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5415f180 con 0x7f6e540cd090 2023-07-14T11:55:44.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.298+0000 7f6e59333700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540b20d0 con 0x7f6e540b0ee0 2023-07-14T11:55:44.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.298+0000 7f6e59333700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540b23b0 con 0x7f6e540d07e0 2023-07-14T11:55:44.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e4bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e540cd090 0x7f6e5415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:44.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e4bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e540cd090 0x7f6e5415ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41988/0 (socket says 172.21.15.104:41988) 2023-07-14T11:55:44.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e4bfff700 1 -- 172.21.15.104:0/2794234035 learned_addr learned my addr 172.21.15.104:0/2794234035 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:44.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e52ffd700 1 --2- 172.21.15.104:0/2794234035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e540b0ee0 0x7f6e540b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:44.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e537fe700 1 --2- 172.21.15.104:0/2794234035 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e540d07e0 0x7f6e540b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:44.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e4bfff700 1 -- 172.21.15.104:0/2794234035 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e540d07e0 msgr2=0x7f6e540b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:44.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e4bfff700 1 --2- 172.21.15.104:0/2794234035 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e540d07e0 0x7f6e540b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e4bfff700 1 -- 172.21.15.104:0/2794234035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e540b0ee0 msgr2=0x7f6e540b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:44.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e4bfff700 1 --2- 172.21.15.104:0/2794234035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e540b0ee0 0x7f6e540b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e4bfff700 1 -- 172.21.15.104:0/2794234035 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e44009570 con 0x7f6e540cd090 2023-07-14T11:55:44.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e4bfff700 1 --2- 172.21.15.104:0/2794234035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e540cd090 0x7f6e5415ec40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00c500 tx=0x7f6e3c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:44.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e4b7fe700 1 -- 172.21.15.104:0/2794234035 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0052d0 con 0x7f6e540cd090 2023-07-14T11:55:44.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e4b7fe700 1 -- 172.21.15.104:0/2794234035 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c005430 con 0x7f6e540cd090 2023-07-14T11:55:44.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e59333700 1 -- 172.21.15.104:0/2794234035 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e540b2690 con 0x7f6e540cd090 2023-07-14T11:55:44.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e4b7fe700 1 -- 172.21.15.104:0/2794234035 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0055a0 con 0x7f6e540cd090 2023-07-14T11:55:44.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.302+0000 7f6e59333700 1 -- 172.21.15.104:0/2794234035 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e540b2c50 con 0x7f6e540cd090 2023-07-14T11:55:44.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.306+0000 7f6e59333700 1 -- 172.21.15.104:0/2794234035 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e540b2de0 con 0x7f6e540cd090 2023-07-14T11:55:44.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.306+0000 7f6e4b7fe700 1 -- 172.21.15.104:0/2794234035 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6e3c0038e0 con 0x7f6e540cd090 2023-07-14T11:55:44.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.306+0000 7f6e4b7fe700 1 --2- 172.21.15.104:0/2794234035 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6e340662f0 0x7f6e340687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:44.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.306+0000 7f6e4b7fe700 1 -- 172.21.15.104:0/2794234035 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f6e3c083580 con 0x7f6e540cd090 2023-07-14T11:55:44.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.306+0000 7f6e52ffd700 1 --2- 172.21.15.104:0/2794234035 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6e340662f0 0x7f6e340687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:44.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.310+0000 7f6e52ffd700 1 --2- 172.21.15.104:0/2794234035 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6e340662f0 0x7f6e340687a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6e44006f80 tx=0x7f6e4401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:44.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.314+0000 7f6e4b7fe700 1 -- 172.21.15.104:0/2794234035 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f6e3c050690 con 0x7f6e540cd090 2023-07-14T11:55:44.473 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.486+0000 7f6e59333700 1 -- 172.21.15.104:0/2794234035 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e5404e500 con 0x7f6e540cd090 2023-07-14T11:55:44.474 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.486+0000 7f6e4b7fe700 1 -- 172.21.15.104:0/2794234035 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6e3c053cb0 con 0x7f6e540cd090 2023-07-14T11:55:44.475 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:55:44.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.490+0000 7f6e59333700 1 -- 172.21.15.104:0/2794234035 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6e340662f0 msgr2=0x7f6e340687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:44.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.490+0000 7f6e59333700 1 --2- 172.21.15.104:0/2794234035 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6e340662f0 0x7f6e340687a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6e44006f80 tx=0x7f6e4401c040 comp rx=0 tx=0).stop 2023-07-14T11:55:44.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.490+0000 7f6e59333700 1 -- 172.21.15.104:0/2794234035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e540cd090 msgr2=0x7f6e5415ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:44.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.490+0000 7f6e59333700 1 --2- 172.21.15.104:0/2794234035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e540cd090 0x7f6e5415ec40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00c500 tx=0x7f6e3c00c8c0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.494+0000 7f6e59333700 1 -- 172.21.15.104:0/2794234035 shutdown_connections 2023-07-14T11:55:44.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.494+0000 7f6e59333700 1 --2- 172.21.15.104:0/2794234035 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6e340662f0 0x7f6e340687a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.494+0000 7f6e59333700 1 --2- 172.21.15.104:0/2794234035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e540cd090 0x7f6e5415ec40 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.494+0000 7f6e59333700 1 --2- 172.21.15.104:0/2794234035 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e540d07e0 0x7f6e540b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.494+0000 7f6e59333700 1 --2- 172.21.15.104:0/2794234035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e540b0ee0 0x7f6e540b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.494+0000 7f6e59333700 1 -- 172.21.15.104:0/2794234035 >> 172.21.15.104:0/2794234035 conn(0x7f6e54060c40 msgr2=0x7f6e540cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:44.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.494+0000 7f6e59333700 1 -- 172.21.15.104:0/2794234035 shutdown_connections 2023-07-14T11:55:44.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.494+0000 7f6e59333700 1 -- 172.21.15.104:0/2794234035 wait complete. 2023-07-14T11:55:44.502 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:55:44.676 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.690+0000 7fe6490c4700 1 Processor -- start 2023-07-14T11:55:44.677 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.690+0000 7fe6490c4700 1 -- start start 2023-07-14T11:55:44.680 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe6490c4700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6441603d0 0x7fe64406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:44.680 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe6490c4700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe6440cd7f0 0x7fe6440c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:44.680 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe6490c4700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6440cdfb0 con 0x7fe644160000 2023-07-14T11:55:44.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe6490c4700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe644070e40 con 0x7fe6440cd7f0 2023-07-14T11:55:44.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe6490c4700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6440cdcd0 con 0x7fe6441603d0 2023-07-14T11:55:44.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe643fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fe644160000 0x7fe6440c9e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:54680/0 (socket says 172.21.15.104:54680) 2023-07-14T11:55:44.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe643fff700 1 -- 172.21.15.104:0/441606364 learned_addr learned my addr 172.21.15.104:0/441606364 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:44.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe6488c3700 1 --2- 172.21.15.104:0/441606364 >> v2:172.21.15.104:3300/0 conn(0x7fe6440cd7f0 0x7fe6440c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:44.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe63bfff700 1 --2- 172.21.15.104:0/441606364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6441603d0 0x7fe64406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:44.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe6437fe700 1 -- 172.21.15.104:0/441606364 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399336072 0 0) 0x7fe634001110 con 0x7fe644160000 2023-07-14T11:55:44.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe6437fe700 1 -- 172.21.15.104:0/441606364 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6300035d0 con 0x7fe644160000 2023-07-14T11:55:44.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe63bfff700 1 -- 172.21.15.104:0/441606364 >> v2:172.21.15.104:3300/0 conn(0x7fe6440cd7f0 msgr2=0x7fe6440c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:44.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe63bfff700 1 --2- 172.21.15.104:0/441606364 >> v2:172.21.15.104:3300/0 conn(0x7fe6440cd7f0 0x7fe6440c0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe63bfff700 1 -- 172.21.15.104:0/441606364 >> v1:172.21.15.104:6789/0 conn(0x7fe644160000 legacy=0x7fe6440c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:44.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe63bfff700 1 -- 172.21.15.104:0/441606364 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe644070780 con 0x7fe6441603d0 2023-07-14T11:55:44.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe63bfff700 1 --2- 172.21.15.104:0/441606364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6441603d0 0x7fe64406dd10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009850 tx=0x7fe62c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c53f635cf42eaab3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:44.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe6437fe700 1 -- 172.21.15.104:0/441606364 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c011830 con 0x7fe6441603d0 2023-07-14T11:55:44.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe6437fe700 1 -- 172.21.15.104:0/441606364 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62c011990 con 0x7fe6441603d0 2023-07-14T11:55:44.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.694+0000 7fe6437fe700 1 -- 172.21.15.104:0/441606364 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c011d40 con 0x7fe6441603d0 2023-07-14T11:55:44.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.698+0000 7fe6490c4700 1 -- 172.21.15.104:0/441606364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6441603d0 msgr2=0x7fe64406dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:44.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.698+0000 7fe6490c4700 1 --2- 172.21.15.104:0/441606364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6441603d0 0x7fe64406dd10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009850 tx=0x7fe62c009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:44.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.698+0000 7fe6490c4700 1 -- 172.21.15.104:0/441606364 shutdown_connections 2023-07-14T11:55:44.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.698+0000 7fe6490c4700 1 --2- 172.21.15.104:0/441606364 >> v2:172.21.15.104:3300/0 conn(0x7fe6440cd7f0 0x7fe6440c0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.698+0000 7fe6490c4700 1 --2- 172.21.15.104:0/441606364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6441603d0 0x7fe64406dd10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.698+0000 7fe6490c4700 1 -- 172.21.15.104:0/441606364 >> 172.21.15.104:0/441606364 conn(0x7fe6440afc60 msgr2=0x7fe6440b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:44.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.698+0000 7fe6490c4700 1 -- 172.21.15.104:0/441606364 shutdown_connections 2023-07-14T11:55:44.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.698+0000 7fe6490c4700 1 -- 172.21.15.104:0/441606364 wait complete. 2023-07-14T11:55:44.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.698+0000 7fe6490c4700 1 Processor -- start 2023-07-14T11:55:44.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.698+0000 7fe6490c4700 1 -- start start 2023-07-14T11:55:44.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe6490c4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6440cd7f0 0x7fe6441f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:44.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe6490c4700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe644160000 0x7fe644161f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:44.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe6490c4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6441603d0 0x7fe6440aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:44.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe6490c4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe644153ef0 con 0x7fe6440cd7f0 2023-07-14T11:55:44.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe6490c4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6441f9b70 con 0x7fe6441603d0 2023-07-14T11:55:44.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe6490c4700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6441f8c80 con 0x7fe644160000 2023-07-14T11:55:44.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe643fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6440cd7f0 0x7fe6441f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:44.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe643fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6440cd7f0 0x7fe6441f5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41996/0 (socket says 172.21.15.104:41996) 2023-07-14T11:55:44.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe643fff700 1 -- 172.21.15.104:0/950017736 learned_addr learned my addr 172.21.15.104:0/950017736 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:44.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe63bfff700 1 --2- 172.21.15.104:0/950017736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe644160000 0x7fe644161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:44.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe643fff700 1 -- 172.21.15.104:0/950017736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe644160000 msgr2=0x7fe644161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:44.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe643fff700 1 --2- 172.21.15.104:0/950017736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe644160000 0x7fe644161f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe643fff700 1 -- 172.21.15.104:0/950017736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6441603d0 msgr2=0x7fe6440aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:44.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe643fff700 1 --2- 172.21.15.104:0/950017736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6441603d0 0x7fe6440aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe643fff700 1 -- 172.21.15.104:0/950017736 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62c0094b0 con 0x7fe6440cd7f0 2023-07-14T11:55:44.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe643fff700 1 --2- 172.21.15.104:0/950017736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6440cd7f0 0x7fe6441f5190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe634001270 tx=0x7fe63400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:44.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe6417fa700 1 -- 172.21.15.104:0/950017736 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634005d90 con 0x7fe6440cd7f0 2023-07-14T11:55:44.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe6417fa700 1 -- 172.21.15.104:0/950017736 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6340070d0 con 0x7fe6440cd7f0 2023-07-14T11:55:44.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe6490c4700 1 -- 172.21.15.104:0/950017736 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6441f8f60 con 0x7fe6440cd7f0 2023-07-14T11:55:44.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe6417fa700 1 -- 172.21.15.104:0/950017736 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634007480 con 0x7fe6440cd7f0 2023-07-14T11:55:44.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.702+0000 7fe6490c4700 1 -- 172.21.15.104:0/950017736 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6441f9520 con 0x7fe6440cd7f0 2023-07-14T11:55:44.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.706+0000 7fe6490c4700 1 -- 172.21.15.104:0/950017736 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe64404e500 con 0x7fe6440cd7f0 2023-07-14T11:55:44.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.706+0000 7fe6417fa700 1 -- 172.21.15.104:0/950017736 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe6340075e0 con 0x7fe6440cd7f0 2023-07-14T11:55:44.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.706+0000 7fe6417fa700 1 --2- 172.21.15.104:0/950017736 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe624066370 0x7fe624068820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:44.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.706+0000 7fe6417fa700 1 -- 172.21.15.104:0/950017736 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fe634083420 con 0x7fe6440cd7f0 2023-07-14T11:55:44.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.706+0000 7fe63bfff700 1 --2- 172.21.15.104:0/950017736 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe624066370 0x7fe624068820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:44.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.710+0000 7fe63bfff700 1 --2- 172.21.15.104:0/950017736 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe624066370 0x7fe624068820 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe644164ac0 tx=0x7fe62c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:44.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.710+0000 7fe6417fa700 1 -- 172.21.15.104:0/950017736 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe6340505b0 con 0x7fe6440cd7f0 2023-07-14T11:55:44.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.882+0000 7fe6490c4700 1 -- 172.21.15.104:0/950017736 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe6440cdfb0 con 0x7fe6440cd7f0 2023-07-14T11:55:44.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.882+0000 7fe6417fa700 1 -- 172.21.15.104:0/950017736 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe6440cdfb0 con 0x7fe6440cd7f0 2023-07-14T11:55:44.870 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:55:44.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.886+0000 7fe6490c4700 1 -- 172.21.15.104:0/950017736 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe624066370 msgr2=0x7fe624068820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:44.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.886+0000 7fe6490c4700 1 --2- 172.21.15.104:0/950017736 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe624066370 0x7fe624068820 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe644164ac0 tx=0x7fe62c008030 comp rx=0 tx=0).stop 2023-07-14T11:55:44.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.886+0000 7fe6490c4700 1 -- 172.21.15.104:0/950017736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6440cd7f0 msgr2=0x7fe6441f5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:44.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.886+0000 7fe6490c4700 1 --2- 172.21.15.104:0/950017736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6440cd7f0 0x7fe6441f5190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe634001270 tx=0x7fe63400ca90 comp rx=0 tx=0).stop 2023-07-14T11:55:44.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.886+0000 7fe6490c4700 1 -- 172.21.15.104:0/950017736 shutdown_connections 2023-07-14T11:55:44.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.886+0000 7fe6490c4700 1 --2- 172.21.15.104:0/950017736 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe624066370 0x7fe624068820 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.886+0000 7fe6490c4700 1 --2- 172.21.15.104:0/950017736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6441603d0 0x7fe6440aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.886+0000 7fe6490c4700 1 --2- 172.21.15.104:0/950017736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe644160000 0x7fe644161f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.886+0000 7fe6490c4700 1 --2- 172.21.15.104:0/950017736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6440cd7f0 0x7fe6441f5190 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:44.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.886+0000 7fe6490c4700 1 -- 172.21.15.104:0/950017736 >> 172.21.15.104:0/950017736 conn(0x7fe6440afc60 msgr2=0x7fe64406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:44.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.886+0000 7fe6490c4700 1 -- 172.21.15.104:0/950017736 shutdown_connections 2023-07-14T11:55:44.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:44.886+0000 7fe6490c4700 1 -- 172.21.15.104:0/950017736 wait complete. 2023-07-14T11:55:44.895 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-07-14T11:55:45.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.090+0000 7f3409d3e700 1 Processor -- start 2023-07-14T11:55:45.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.090+0000 7f3409d3e700 1 -- start start 2023-07-14T11:55:45.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f3409d3e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f34040b0ee0 0x7f34040ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:45.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f3409d3e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34040cd090 0x7f34040d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:45.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f3409d3e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f34040d0610 0x7f340415f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:45.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f3409d3e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34040cd4e0 con 0x7f34040d0610 2023-07-14T11:55:45.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f3409d3e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34040d1a60 con 0x7f34040b0ee0 2023-07-14T11:55:45.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f3409d3e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34040d1ef0 con 0x7f34040cd090 2023-07-14T11:55:45.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f340953d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f34040d0610 0x7f340415f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:45.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f340953d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f34040d0610 0x7f340415f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42002/0 (socket says 172.21.15.104:42002) 2023-07-14T11:55:45.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f340953d700 1 -- 172.21.15.104:0/3674507483 learned_addr learned my addr 172.21.15.104:0/3674507483 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:45.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f3403fff700 1 --2- 172.21.15.104:0/3674507483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34040cd090 0x7f34040d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:45.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f3408d3c700 1 --2- 172.21.15.104:0/3674507483 >> v2:172.21.15.121:3300/0 conn(0x7f34040b0ee0 0x7f34040ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:45.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f340953d700 1 -- 172.21.15.104:0/3674507483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34040cd090 msgr2=0x7f34040d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:45.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f340953d700 1 --2- 172.21.15.104:0/3674507483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34040cd090 0x7f34040d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f340953d700 1 -- 172.21.15.104:0/3674507483 >> v2:172.21.15.121:3300/0 conn(0x7f34040b0ee0 msgr2=0x7f34040ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:45.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f340953d700 1 --2- 172.21.15.104:0/3674507483 >> v2:172.21.15.121:3300/0 conn(0x7f34040b0ee0 0x7f34040ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f340953d700 1 -- 172.21.15.104:0/3674507483 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34040d0e50 con 0x7f34040d0610 2023-07-14T11:55:45.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.094+0000 7f340953d700 1 --2- 172.21.15.104:0/3674507483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34040d0610 0x7f340415f850 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009e50 tx=0x7f33fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513452c1bf7a8bf9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:45.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.098+0000 7f34037fe700 1 -- 172.21.15.104:0/3674507483 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc00e070 con 0x7f34040d0610 2023-07-14T11:55:45.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.098+0000 7f34037fe700 1 -- 172.21.15.104:0/3674507483 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc00b8d0 con 0x7f34040d0610 2023-07-14T11:55:45.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.098+0000 7f34037fe700 1 -- 172.21.15.104:0/3674507483 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc00bc80 con 0x7f34040d0610 2023-07-14T11:55:45.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.098+0000 7f3409d3e700 1 -- 172.21.15.104:0/3674507483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34040d0610 msgr2=0x7f340415f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:45.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.098+0000 7f3409d3e700 1 --2- 172.21.15.104:0/3674507483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34040d0610 0x7f340415f850 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009e50 tx=0x7f33fc00b330 comp rx=0 tx=0).stop 2023-07-14T11:55:45.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.098+0000 7f3409d3e700 1 -- 172.21.15.104:0/3674507483 shutdown_connections 2023-07-14T11:55:45.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.098+0000 7f3409d3e700 1 --2- 172.21.15.104:0/3674507483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34040d0610 0x7f340415f850 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.098+0000 7f3409d3e700 1 --2- 172.21.15.104:0/3674507483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34040cd090 0x7f34040d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.098+0000 7f3409d3e700 1 --2- 172.21.15.104:0/3674507483 >> v2:172.21.15.121:3300/0 conn(0x7f34040b0ee0 0x7f34040ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.098+0000 7f3409d3e700 1 -- 172.21.15.104:0/3674507483 >> 172.21.15.104:0/3674507483 conn(0x7f3404060c40 msgr2=0x7f340409b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:45.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.098+0000 7f3409d3e700 1 -- 172.21.15.104:0/3674507483 shutdown_connections 2023-07-14T11:55:45.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.098+0000 7f3409d3e700 1 -- 172.21.15.104:0/3674507483 wait complete. 2023-07-14T11:55:45.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3409d3e700 1 Processor -- start 2023-07-14T11:55:45.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3409d3e700 1 -- start start 2023-07-14T11:55:45.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3409d3e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34040b0ee0 0x7f340415c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:45.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3409d3e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34040cd090 0x7f34040cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:45.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3409d3e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34040d0610 0x7f3404159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:45.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3409d3e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34040cbaa0 con 0x7f34040cd090 2023-07-14T11:55:45.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3409d3e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34040cc710 con 0x7f34040b0ee0 2023-07-14T11:55:45.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3409d3e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34040b2800 con 0x7f34040d0610 2023-07-14T11:55:45.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3403fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34040cd090 0x7f34040cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:45.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3403fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34040cd090 0x7f34040cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42006/0 (socket says 172.21.15.104:42006) 2023-07-14T11:55:45.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3403fff700 1 -- 172.21.15.104:0/2767284012 learned_addr learned my addr 172.21.15.104:0/2767284012 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:45.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3408d3c700 1 --2- 172.21.15.104:0/2767284012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34040b0ee0 0x7f340415c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:45.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f340953d700 1 --2- 172.21.15.104:0/2767284012 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34040d0610 0x7f3404159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:45.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3403fff700 1 -- 172.21.15.104:0/2767284012 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34040d0610 msgr2=0x7f3404159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:45.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3403fff700 1 --2- 172.21.15.104:0/2767284012 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34040d0610 0x7f3404159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3403fff700 1 -- 172.21.15.104:0/2767284012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34040b0ee0 msgr2=0x7f340415c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:45.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.102+0000 7f3403fff700 1 --2- 172.21.15.104:0/2767284012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34040b0ee0 0x7f340415c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.106+0000 7f3403fff700 1 -- 172.21.15.104:0/2767284012 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc009b30 con 0x7f34040cd090 2023-07-14T11:55:45.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.106+0000 7f3403fff700 1 --2- 172.21.15.104:0/2767284012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34040cd090 0x7f34040cc1d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f33f000e8a0 tx=0x7f33f000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:45.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.106+0000 7f34017fa700 1 -- 172.21.15.104:0/2767284012 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f00095c0 con 0x7f34040cd090 2023-07-14T11:55:45.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.106+0000 7f34017fa700 1 -- 172.21.15.104:0/2767284012 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f0013680 con 0x7f34040cd090 2023-07-14T11:55:45.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.106+0000 7f34017fa700 1 -- 172.21.15.104:0/2767284012 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0013a10 con 0x7f34040cd090 2023-07-14T11:55:45.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.106+0000 7f3409d3e700 1 -- 172.21.15.104:0/2767284012 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34040cb4e0 con 0x7f34040cd090 2023-07-14T11:55:45.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.106+0000 7f3409d3e700 1 -- 172.21.15.104:0/2767284012 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34040b2490 con 0x7f34040cd090 2023-07-14T11:55:45.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.106+0000 7f3409d3e700 1 -- 172.21.15.104:0/2767284012 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34040b22f0 con 0x7f34040cd090 2023-07-14T11:55:45.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.110+0000 7f34017fa700 1 -- 172.21.15.104:0/2767284012 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f33f0013b70 con 0x7f34040cd090 2023-07-14T11:55:45.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.110+0000 7f34017fa700 1 --2- 172.21.15.104:0/2767284012 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f33e80662f0 0x7f33e80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:45.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.110+0000 7f34017fa700 1 -- 172.21.15.104:0/2767284012 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f33f0083e90 con 0x7f34040cd090 2023-07-14T11:55:45.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.110+0000 7f3408d3c700 1 --2- 172.21.15.104:0/2767284012 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f33e80662f0 0x7f33e80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:45.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.110+0000 7f3408d3c700 1 --2- 172.21.15.104:0/2767284012 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f33e80662f0 0x7f33e80687a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009ad0 tx=0x7f33f8009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:45.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.114+0000 7f34017fa700 1 -- 172.21.15.104:0/2767284012 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f34040b22f0 con 0x7f34040cd090 2023-07-14T11:55:45.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.290+0000 7f3409d3e700 1 -- 172.21.15.104:0/2767284012 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f34040b22f0 con 0x7f34040cd090 2023-07-14T11:55:45.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.290+0000 7f34017fa700 1 -- 172.21.15.104:0/2767284012 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v45) v1 ==== 97+0+11 (secure 0 0 0) 0x7f34040b22f0 con 0x7f34040cd090 2023-07-14T11:55:45.277 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:55:45.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.294+0000 7f3409d3e700 1 -- 172.21.15.104:0/2767284012 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f33e80662f0 msgr2=0x7f33e80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:45.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.294+0000 7f3409d3e700 1 --2- 172.21.15.104:0/2767284012 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f33e80662f0 0x7f33e80687a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009ad0 tx=0x7f33f8009240 comp rx=0 tx=0).stop 2023-07-14T11:55:45.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.294+0000 7f3409d3e700 1 -- 172.21.15.104:0/2767284012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34040cd090 msgr2=0x7f34040cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:45.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.294+0000 7f3409d3e700 1 --2- 172.21.15.104:0/2767284012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34040cd090 0x7f34040cc1d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f33f000e8a0 tx=0x7f33f000ebb0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.294+0000 7f3409d3e700 1 -- 172.21.15.104:0/2767284012 shutdown_connections 2023-07-14T11:55:45.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.294+0000 7f3409d3e700 1 --2- 172.21.15.104:0/2767284012 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f33e80662f0 0x7f33e80687a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.294+0000 7f3409d3e700 1 --2- 172.21.15.104:0/2767284012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34040cd090 0x7f34040cc1d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.294+0000 7f3409d3e700 1 --2- 172.21.15.104:0/2767284012 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34040d0610 0x7f3404159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.294+0000 7f3409d3e700 1 --2- 172.21.15.104:0/2767284012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34040b0ee0 0x7f340415c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.294+0000 7f3409d3e700 1 -- 172.21.15.104:0/2767284012 >> 172.21.15.104:0/2767284012 conn(0x7f3404060c40 msgr2=0x7f3404071510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:45.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.294+0000 7f3409d3e700 1 -- 172.21.15.104:0/2767284012 shutdown_connections 2023-07-14T11:55:45.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.294+0000 7f3409d3e700 1 -- 172.21.15.104:0/2767284012 wait complete. 2023-07-14T11:55:45.304 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:55:45.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.494+0000 7f47ad411700 1 Processor -- start 2023-07-14T11:55:45.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.494+0000 7f47ad411700 1 -- start start 2023-07-14T11:55:45.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47ad411700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47a80b1500 0x7f47a80cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:45.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47ad411700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a815d5a0 con 0x7f47a806e960 2023-07-14T11:55:45.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47ad411700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a815ca20 con 0x7f47a8158d10 2023-07-14T11:55:45.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47ad411700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a815cd00 con 0x7f47a80b1500 2023-07-14T11:55:45.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a7fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f47a806e960 0x7f47a8159520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:54692/0 (socket says 172.21.15.104:54692) 2023-07-14T11:55:45.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a7fff700 1 -- 172.21.15.104:0/1658439528 learned_addr learned my addr 172.21.15.104:0/1658439528 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:45.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a6ffd700 1 --2- 172.21.15.104:0/1658439528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47a80b1500 0x7f47a80cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:45.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a5ffb700 1 -- 172.21.15.104:0/1658439528 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162680408 0 0) 0x7f47a815d5a0 con 0x7f47a806e960 2023-07-14T11:55:45.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a5ffb700 1 -- 172.21.15.104:0/1658439528 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47900035d0 con 0x7f47a806e960 2023-07-14T11:55:45.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a5ffb700 1 -- 172.21.15.104:0/1658439528 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4139857305 0 0) 0x7f47900035d0 con 0x7f47a806e960 2023-07-14T11:55:45.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a5ffb700 1 -- 172.21.15.104:0/1658439528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47a80b1500 msgr2=0x7f47a80cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:45.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a5ffb700 1 --2- 172.21.15.104:0/1658439528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47a80b1500 0x7f47a80cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a5ffb700 1 -- 172.21.15.104:0/1658439528 >> v1:172.21.15.121:6789/0 conn(0x7f47a8158d10 legacy=0x7f47a80b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:45.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a5ffb700 1 -- 172.21.15.104:0/1658439528 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a815cfe0 con 0x7f47a806e960 2023-07-14T11:55:45.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a5ffb700 1 -- 172.21.15.104:0/1658439528 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f47a00019f0 con 0x7f47a806e960 2023-07-14T11:55:45.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a5ffb700 1 -- 172.21.15.104:0/1658439528 >> v1:172.21.15.104:6789/0 conn(0x7f47a806e960 legacy=0x7f47a8159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:45.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a5ffb700 1 --2- 172.21.15.104:0/1658439528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4790003a50 0x7f4790003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:45.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a5ffb700 1 --2- 172.21.15.104:0/1658439528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47900043f0 0x7f47900048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:45.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a5ffb700 1 --2- 172.21.15.104:0/1658439528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47a80b1500 0x7f4790005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:45.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a5ffb700 1 -- 172.21.15.104:0/1658439528 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a815cd00 con 0x7f4790003a50 2023-07-14T11:55:45.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a5ffb700 1 -- 172.21.15.104:0/1658439528 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479000dc40 con 0x7f47900043f0 2023-07-14T11:55:45.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a5ffb700 1 -- 172.21.15.104:0/1658439528 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479000ddd0 con 0x7f47a80b1500 2023-07-14T11:55:45.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a6ffd700 1 --2- 172.21.15.104:0/1658439528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4790003a50 0x7f4790003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:45.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a67fc700 1 --2- 172.21.15.104:0/1658439528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47900043f0 0x7f47900048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:45.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a7fff700 1 --2- 172.21.15.104:0/1658439528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47a80b1500 0x7f4790005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:45.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a6ffd700 1 -- 172.21.15.104:0/1658439528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47a80b1500 msgr2=0x7f4790005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:45.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a6ffd700 1 --2- 172.21.15.104:0/1658439528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47a80b1500 0x7f4790005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a6ffd700 1 -- 172.21.15.104:0/1658439528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47900043f0 msgr2=0x7f47900048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:45.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.498+0000 7f47a6ffd700 1 --2- 172.21.15.104:0/1658439528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47900043f0 0x7f47900048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.502+0000 7f47a6ffd700 1 -- 172.21.15.104:0/1658439528 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47900035d0 con 0x7f4790003a50 2023-07-14T11:55:45.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.502+0000 7f47a6ffd700 1 --2- 172.21.15.104:0/1658439528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4790003a50 0x7f4790003e20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4798009b70 tx=0x7f47900058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5d6cf6b5c662cc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:45.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.502+0000 7f47a5ffb700 1 -- 172.21.15.104:0/1658439528 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479800e6c0 con 0x7f4790003a50 2023-07-14T11:55:45.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.502+0000 7f47a5ffb700 1 -- 172.21.15.104:0/1658439528 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479800e820 con 0x7f4790003a50 2023-07-14T11:55:45.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.502+0000 7f47a5ffb700 1 -- 172.21.15.104:0/1658439528 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479800eb90 con 0x7f4790003a50 2023-07-14T11:55:45.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.502+0000 7f47ad411700 1 -- 172.21.15.104:0/1658439528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4790003a50 msgr2=0x7f4790003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:45.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.502+0000 7f47ad411700 1 --2- 172.21.15.104:0/1658439528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4790003a50 0x7f4790003e20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4798009b70 tx=0x7f47900058a0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.502+0000 7f47a7fff700 1 -- 172.21.15.104:0/1658439528 reap_dead start 2023-07-14T11:55:45.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.502+0000 7f47ad411700 1 -- 172.21.15.104:0/1658439528 shutdown_connections 2023-07-14T11:55:45.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.502+0000 7f47ad411700 1 -- 172.21.15.104:0/1658439528 >> 172.21.15.104:0/1658439528 conn(0x7f47a80cc500 msgr2=0x7f47a80cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:45.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.502+0000 7f47ad411700 1 -- 172.21.15.104:0/1658439528 shutdown_connections 2023-07-14T11:55:45.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.502+0000 7f47ad411700 1 -- 172.21.15.104:0/1658439528 wait complete. 2023-07-14T11:55:45.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.506+0000 7f47ad411700 1 Processor -- start 2023-07-14T11:55:45.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.506+0000 7f47ad411700 1 -- start start 2023-07-14T11:55:45.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.506+0000 7f47ad411700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a81f48d0 0x7f47a80b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:45.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.506+0000 7f47ad411700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47a815b440 0x7f47a815b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:45.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.506+0000 7f47ad411700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47a80b2a00 0x7f47a80b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:45.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.506+0000 7f47ad411700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a80b5850 con 0x7f47a81f48d0 2023-07-14T11:55:45.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.506+0000 7f47ad411700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a80b33b0 con 0x7f47a80b2a00 2023-07-14T11:55:45.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.506+0000 7f47ad411700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8154620 con 0x7f47a815b440 2023-07-14T11:55:45.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.506+0000 7f47a6ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a81f48d0 0x7f47a80b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:45.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.506+0000 7f47a6ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a81f48d0 0x7f47a80b5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42032/0 (socket says 172.21.15.104:42032) 2023-07-14T11:55:45.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.506+0000 7f47a6ffd700 1 -- 172.21.15.104:0/2972564448 learned_addr learned my addr 172.21.15.104:0/2972564448 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:45.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.506+0000 7f47a7fff700 1 --2- 172.21.15.104:0/2972564448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47a80b2a00 0x7f47a80b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:45.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.506+0000 7f47a67fc700 1 --2- 172.21.15.104:0/2972564448 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47a815b440 0x7f47a815b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:45.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.510+0000 7f47a6ffd700 1 -- 172.21.15.104:0/2972564448 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47a815b440 msgr2=0x7f47a815b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:45.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.510+0000 7f47a6ffd700 1 --2- 172.21.15.104:0/2972564448 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47a815b440 0x7f47a815b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.510+0000 7f47a6ffd700 1 -- 172.21.15.104:0/2972564448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47a80b2a00 msgr2=0x7f47a80b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:45.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.510+0000 7f47a6ffd700 1 --2- 172.21.15.104:0/2972564448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47a80b2a00 0x7f47a80b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.510+0000 7f47a6ffd700 1 -- 172.21.15.104:0/2972564448 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a8154900 con 0x7f47a81f48d0 2023-07-14T11:55:45.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.510+0000 7f47a6ffd700 1 --2- 172.21.15.104:0/2972564448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a81f48d0 0x7f47a80b5310 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4790005820 tx=0x7f47980047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:45.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.510+0000 7f478f7fe700 1 -- 172.21.15.104:0/2972564448 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479800e6c0 con 0x7f47a81f48d0 2023-07-14T11:55:45.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.510+0000 7f478f7fe700 1 -- 172.21.15.104:0/2972564448 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4798003ad0 con 0x7f47a81f48d0 2023-07-14T11:55:45.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.510+0000 7f478f7fe700 1 -- 172.21.15.104:0/2972564448 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798004610 con 0x7f47a81f48d0 2023-07-14T11:55:45.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.510+0000 7f47ad411700 1 -- 172.21.15.104:0/2972564448 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a8154be0 con 0x7f47a81f48d0 2023-07-14T11:55:45.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.510+0000 7f47ad411700 1 -- 172.21.15.104:0/2972564448 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a81f29b0 con 0x7f47a81f48d0 2023-07-14T11:55:45.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.510+0000 7f47ad411700 1 -- 172.21.15.104:0/2972564448 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a804dcb0 con 0x7f47a81f48d0 2023-07-14T11:55:45.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.514+0000 7f478f7fe700 1 -- 172.21.15.104:0/2972564448 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f4798013470 con 0x7f47a81f48d0 2023-07-14T11:55:45.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.514+0000 7f478f7fe700 1 --2- 172.21.15.104:0/2972564448 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4794066320 0x7f47940687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:45.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.514+0000 7f478f7fe700 1 -- 172.21.15.104:0/2972564448 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f47980854d0 con 0x7f47a81f48d0 2023-07-14T11:55:45.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.514+0000 7f47a67fc700 1 --2- 172.21.15.104:0/2972564448 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4794066320 0x7f47940687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:45.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.514+0000 7f47a67fc700 1 --2- 172.21.15.104:0/2972564448 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4794066320 0x7f47940687d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f47a8155f30 tx=0x7f479c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:45.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.522+0000 7f478f7fe700 1 -- 172.21.15.104:0/2972564448 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f4798052790 con 0x7f47a81f48d0 2023-07-14T11:55:45.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.710+0000 7f47ad411700 1 -- 172.21.15.104:0/2972564448 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f47a806ff00 con 0x7f47a81f48d0 2023-07-14T11:55:45.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.710+0000 7f478f7fe700 1 -- 172.21.15.104:0/2972564448 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4798000e60 con 0x7f47a81f48d0 2023-07-14T11:55:45.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.714+0000 7f47ad411700 1 -- 172.21.15.104:0/2972564448 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4794066320 msgr2=0x7f47940687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:45.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.714+0000 7f47ad411700 1 --2- 172.21.15.104:0/2972564448 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4794066320 0x7f47940687d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f47a8155f30 tx=0x7f479c009040 comp rx=0 tx=0).stop 2023-07-14T11:55:45.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.714+0000 7f47ad411700 1 -- 172.21.15.104:0/2972564448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a81f48d0 msgr2=0x7f47a80b5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:45.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.714+0000 7f47ad411700 1 --2- 172.21.15.104:0/2972564448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a81f48d0 0x7f47a80b5310 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4790005820 tx=0x7f47980047e0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.714+0000 7f47ad411700 1 -- 172.21.15.104:0/2972564448 shutdown_connections 2023-07-14T11:55:45.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.714+0000 7f47ad411700 1 --2- 172.21.15.104:0/2972564448 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4794066320 0x7f47940687d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.714+0000 7f47ad411700 1 --2- 172.21.15.104:0/2972564448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47a80b2a00 0x7f47a80b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.714+0000 7f47ad411700 1 --2- 172.21.15.104:0/2972564448 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47a815b440 0x7f47a815b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.714+0000 7f47ad411700 1 --2- 172.21.15.104:0/2972564448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a81f48d0 0x7f47a80b5310 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:45.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.714+0000 7f47ad411700 1 -- 172.21.15.104:0/2972564448 >> 172.21.15.104:0/2972564448 conn(0x7f47a80cc500 msgr2=0x7f47a80cd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:45.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.714+0000 7f47ad411700 1 -- 172.21.15.104:0/2972564448 shutdown_connections 2023-07-14T11:55:45.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:45.714+0000 7f47ad411700 1 -- 172.21.15.104:0/2972564448 wait complete. 2023-07-14T11:55:45.704 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:55:45.723 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi104.front.sepia.ceph.com... 2023-07-14T11:55:45.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:45.724 DEBUG:teuthology.orchestra.run.smithi104:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-14T11:55:45.753 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[48185]: fuse finished with error 0 and tester_r 0 2023-07-14T11:55:45.756 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:55:51.858 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:55:51.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:51.859 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-14T11:55:51.881 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2023-07-14T11:55:52.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.129+0000 7f11fd9ae700 1 Processor -- start 2023-07-14T11:55:52.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.129+0000 7f11fd9ae700 1 -- start start 2023-07-14T11:55:52.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.133+0000 7f11fd9ae700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f11f80b1c50 0x7f11f8071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:52.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.133+0000 7f11fd9ae700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f11f80cf710 0x7f11f80d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:52.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.133+0000 7f11fd9ae700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f815c420 con 0x7f11f80d0210 2023-07-14T11:55:52.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.133+0000 7f11fd9ae700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f8070840 con 0x7f11f80cf710 2023-07-14T11:55:52.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.133+0000 7f11fd9ae700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f8070b20 con 0x7f11f80b1c50 2023-07-14T11:55:52.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.133+0000 7f11f77fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f11f80cf710 0x7f11f80d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:52.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.133+0000 7f11f67fc700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f11f80d0210 0x7f11f80cf000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:39876/0 (socket says 172.21.15.104:39876) 2023-07-14T11:55:52.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.133+0000 7f11f67fc700 1 -- 172.21.15.104:0/3030566185 learned_addr learned my addr 172.21.15.104:0/3030566185 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:52.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.133+0000 7f11f6ffd700 1 --2- 172.21.15.104:0/3030566185 >> v2:172.21.15.121:3300/0 conn(0x7f11f80b1c50 0x7f11f8071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:52.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11f5ffb700 1 -- 172.21.15.104:0/3030566185 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305334437 0 0) 0x7f11f815c420 con 0x7f11f80d0210 2023-07-14T11:55:52.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11f5ffb700 1 -- 172.21.15.104:0/3030566185 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11e00035d0 con 0x7f11f80d0210 2023-07-14T11:55:52.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11f77fe700 1 -- 172.21.15.104:0/3030566185 >> v2:172.21.15.121:3300/0 conn(0x7f11f80b1c50 msgr2=0x7f11f8071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:52.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11f77fe700 1 --2- 172.21.15.104:0/3030566185 >> v2:172.21.15.121:3300/0 conn(0x7f11f80b1c50 0x7f11f8071600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:52.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11f77fe700 1 -- 172.21.15.104:0/3030566185 >> v1:172.21.15.121:6789/0 conn(0x7f11f80d0210 legacy=0x7f11f80cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:52.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11f77fe700 1 -- 172.21.15.104:0/3030566185 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f8070e00 con 0x7f11f80cf710 2023-07-14T11:55:52.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11f77fe700 1 --2- 172.21.15.104:0/3030566185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f80cf710 0x7f11f80d0640 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f11f808a000 tx=0x7f11f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5625ed0452a2ac server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:52.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11f5ffb700 1 -- 172.21.15.104:0/3030566185 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f000e070 con 0x7f11f80cf710 2023-07-14T11:55:52.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11f5ffb700 1 -- 172.21.15.104:0/3030566185 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f000bb20 con 0x7f11f80cf710 2023-07-14T11:55:52.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11f5ffb700 1 -- 172.21.15.104:0/3030566185 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0012430 con 0x7f11f80cf710 2023-07-14T11:55:52.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11fd9ae700 1 -- 172.21.15.104:0/3030566185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f80cf710 msgr2=0x7f11f80d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:52.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11fd9ae700 1 --2- 172.21.15.104:0/3030566185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f80cf710 0x7f11f80d0640 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f11f808a000 tx=0x7f11f000b330 comp rx=0 tx=0).stop 2023-07-14T11:55:52.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11fd9ae700 1 -- 172.21.15.104:0/3030566185 shutdown_connections 2023-07-14T11:55:52.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11fd9ae700 1 --2- 172.21.15.104:0/3030566185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f80cf710 0x7f11f80d0640 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:52.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11fd9ae700 1 --2- 172.21.15.104:0/3030566185 >> v2:172.21.15.121:3300/0 conn(0x7f11f80b1c50 0x7f11f8071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:52.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11fd9ae700 1 -- 172.21.15.104:0/3030566185 >> 172.21.15.104:0/3030566185 conn(0x7f11f809b9c0 msgr2=0x7f11f80d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:52.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11fd9ae700 1 -- 172.21.15.104:0/3030566185 shutdown_connections 2023-07-14T11:55:52.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.137+0000 7f11fd9ae700 1 -- 172.21.15.104:0/3030566185 wait complete. 2023-07-14T11:55:52.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.141+0000 7f11fd9ae700 1 Processor -- start 2023-07-14T11:55:52.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.141+0000 7f11fd9ae700 1 -- start start 2023-07-14T11:55:52.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11fd9ae700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11f80b1c50 0x7f11f809e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:52.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11fd9ae700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f80cf710 0x7f11f8159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:52.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11fd9ae700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f80d0210 0x7f11f81e2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:52.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11fd9ae700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f815a440 con 0x7f11f80d0210 2023-07-14T11:55:52.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11fd9ae700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f809eb50 con 0x7f11f80cf710 2023-07-14T11:55:52.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11fd9ae700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f809ee30 con 0x7f11f80b1c50 2023-07-14T11:55:52.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11f77fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f80d0210 0x7f11f81e2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:52.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11f77fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f80d0210 0x7f11f81e2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35176/0 (socket says 172.21.15.104:35176) 2023-07-14T11:55:52.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11f77fe700 1 -- 172.21.15.104:0/3971802316 learned_addr learned my addr 172.21.15.104:0/3971802316 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:52.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11f67fc700 1 --2- 172.21.15.104:0/3971802316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f80cf710 0x7f11f8159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:52.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11f6ffd700 1 --2- 172.21.15.104:0/3971802316 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11f80b1c50 0x7f11f809e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:52.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11f77fe700 1 -- 172.21.15.104:0/3971802316 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11f80b1c50 msgr2=0x7f11f809e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:52.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11f77fe700 1 --2- 172.21.15.104:0/3971802316 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11f80b1c50 0x7f11f809e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:52.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11f77fe700 1 -- 172.21.15.104:0/3971802316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f80cf710 msgr2=0x7f11f8159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:52.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11f77fe700 1 --2- 172.21.15.104:0/3971802316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f80cf710 0x7f11f8159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:52.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11f77fe700 1 -- 172.21.15.104:0/3971802316 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f0009a70 con 0x7f11f80d0210 2023-07-14T11:55:52.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11f67fc700 1 --2- 172.21.15.104:0/3971802316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f80cf710 0x7f11f8159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:52.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11f77fe700 1 --2- 172.21.15.104:0/3971802316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f80d0210 0x7f11f81e2b60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f11f0014040 tx=0x7f11f0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:52.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11dffff700 1 -- 172.21.15.104:0/3971802316 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0012430 con 0x7f11f80d0210 2023-07-14T11:55:52.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11dffff700 1 -- 172.21.15.104:0/3971802316 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f0020070 con 0x7f11f80d0210 2023-07-14T11:55:52.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11fd9ae700 1 -- 172.21.15.104:0/3971802316 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f809f110 con 0x7f11f80d0210 2023-07-14T11:55:52.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11dffff700 1 -- 172.21.15.104:0/3971802316 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0004280 con 0x7f11f80d0210 2023-07-14T11:55:52.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.145+0000 7f11fd9ae700 1 -- 172.21.15.104:0/3971802316 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f809f6d0 con 0x7f11f80d0210 2023-07-14T11:55:52.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.149+0000 7f11fd9ae700 1 -- 172.21.15.104:0/3971802316 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f804dcb0 con 0x7f11f80d0210 2023-07-14T11:55:52.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.149+0000 7f11dffff700 1 -- 172.21.15.104:0/3971802316 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f11f00043e0 con 0x7f11f80d0210 2023-07-14T11:55:52.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.149+0000 7f11dffff700 1 --2- 172.21.15.104:0/3971802316 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f11e4066320 0x7f11e40687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:52.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.153+0000 7f11dffff700 1 -- 172.21.15.104:0/3971802316 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f11f0022070 con 0x7f11f80d0210 2023-07-14T11:55:52.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.153+0000 7f11f6ffd700 1 --2- 172.21.15.104:0/3971802316 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f11e4066320 0x7f11e40687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:52.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.153+0000 7f11f6ffd700 1 --2- 172.21.15.104:0/3971802316 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f11e4066320 0x7f11e40687d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f11f81e6420 tx=0x7f11e8009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:52.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.157+0000 7f11dffff700 1 -- 172.21.15.104:0/3971802316 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f11f0052c10 con 0x7f11f80d0210 2023-07-14T11:55:52.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.393+0000 7f11fd9ae700 1 -- 172.21.15.104:0/3971802316 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f11f8070600 con 0x7f11f80d0210 2023-07-14T11:55:52.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.401+0000 7f11dffff700 1 -- 172.21.15.104:0/3971802316 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v16) v1 ==== 184+0+141 (secure 0 0 0) 0x7f11f0056230 con 0x7f11f80d0210 2023-07-14T11:55:52.387 INFO:teuthology.orchestra.run.smithi104.stdout:[client.0] 2023-07-14T11:55:52.388 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDrNrFkbrzjERAA+9BBzuauwUdhALUcDtd80A== 2023-07-14T11:55:52.388 INFO:teuthology.orchestra.run.smithi104.stdout: caps mds = "allow" 2023-07-14T11:55:52.388 INFO:teuthology.orchestra.run.smithi104.stdout: caps mon = "allow r" 2023-07-14T11:55:52.388 INFO:teuthology.orchestra.run.smithi104.stdout: caps osd = "allow rw pool=cephfs_data" 2023-07-14T11:55:52.391 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.405+0000 7f11fd9ae700 1 -- 172.21.15.104:0/3971802316 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f11e4066320 msgr2=0x7f11e40687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:52.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.405+0000 7f11fd9ae700 1 --2- 172.21.15.104:0/3971802316 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f11e4066320 0x7f11e40687d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f11f81e6420 tx=0x7f11e8009240 comp rx=0 tx=0).stop 2023-07-14T11:55:52.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.405+0000 7f11fd9ae700 1 -- 172.21.15.104:0/3971802316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f80d0210 msgr2=0x7f11f81e2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:52.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.405+0000 7f11fd9ae700 1 --2- 172.21.15.104:0/3971802316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f80d0210 0x7f11f81e2b60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f11f0014040 tx=0x7f11f0004af0 comp rx=0 tx=0).stop 2023-07-14T11:55:52.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.405+0000 7f11fd9ae700 1 -- 172.21.15.104:0/3971802316 shutdown_connections 2023-07-14T11:55:52.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.405+0000 7f11fd9ae700 1 --2- 172.21.15.104:0/3971802316 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f11e4066320 0x7f11e40687d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:52.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.405+0000 7f11fd9ae700 1 --2- 172.21.15.104:0/3971802316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f80d0210 0x7f11f81e2b60 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:52.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.405+0000 7f11fd9ae700 1 --2- 172.21.15.104:0/3971802316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f80cf710 0x7f11f8159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:52.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.405+0000 7f11fd9ae700 1 --2- 172.21.15.104:0/3971802316 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11f80b1c50 0x7f11f809e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:52.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.405+0000 7f11fd9ae700 1 -- 172.21.15.104:0/3971802316 >> 172.21.15.104:0/3971802316 conn(0x7f11f809b9c0 msgr2=0x7f11f806fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:52.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.405+0000 7f11fd9ae700 1 -- 172.21.15.104:0/3971802316 shutdown_connections 2023-07-14T11:55:52.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:52.405+0000 7f11fd9ae700 1 -- 172.21.15.104:0/3971802316 wait complete. 2023-07-14T11:55:52.394 INFO:teuthology.orchestra.run.smithi104.stderr:updated caps for client.0 2023-07-14T11:55:52.412 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-14T11:55:52.413 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-14T11:55:52.454 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-14T11:55:52.497 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-14T11:55:52.497 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:55:52.498 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:55:52.514 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352340k,nr_inodes=4088085,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275392k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275388k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275392k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-07-14T11:55:52.515 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:55:52.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:52.515 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-14T11:55:52.574 INFO:teuthology.orchestra.run.smithi104.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-07-14T11:55:52.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:55:52.576 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:55:52.576 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:55:52.594 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352176k,nr_inodes=4088044,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18484 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275352k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-07-14T11:55:52.595 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:55:52.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:52.596 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-14T11:55:52.652 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-07-14T11:55:52.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:55:52.655 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:55:52.655 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:55:52.673 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352340k,nr_inodes=4088085,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275392k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275388k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275392k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-07-14T11:55:52.674 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:55:52.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:52.675 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-14T11:55:52.734 INFO:teuthology.orchestra.run.smithi104.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-14T11:55:52.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:55:52.736 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:55:52.736 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:55:52.754 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352176k,nr_inodes=4088044,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18484 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275352k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-07-14T11:55:52.754 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:55:52.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:55:52.755 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-14T11:55:52.815 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-14T11:55:52.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:55:52.818 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:55:52.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.001+0000 7fe322016700 1 Processor -- start 2023-07-14T11:55:52.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe322016700 1 -- start start 2023-07-14T11:55:52.993 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe322016700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe31c0c9d90 0x7fe31c0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:52.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe322016700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe31c159f20 0x7fe31c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:52.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe322016700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe31c0caf70 0x7fe31c0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:52.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe322016700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31c0b0ef0 con 0x7fe31c0caf70 2023-07-14T11:55:52.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe322016700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31c0ce9c0 con 0x7fe31c0c9d90 2023-07-14T11:55:52.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe322016700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0cee50 con 0x7fe31c159f20 2023-07-14T11:55:52.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe321815700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe31c0caf70 0x7fe31c0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:52.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe320813700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe31c159f20 0x7fe31c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:52.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe321815700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe31c0caf70 0x7fe31c0c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35190/0 (socket says 172.21.15.104:35190) 2023-07-14T11:55:52.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe321815700 1 -- 172.21.15.104:0/2081870285 learned_addr learned my addr 172.21.15.104:0/2081870285 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:52.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe321014700 1 --2- 172.21.15.104:0/2081870285 >> v2:172.21.15.121:3300/0 conn(0x7fe31c0c9d90 0x7fe31c0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:52.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe321815700 1 -- 172.21.15.104:0/2081870285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe31c159f20 msgr2=0x7fe31c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:52.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe321815700 1 --2- 172.21.15.104:0/2081870285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe31c159f20 0x7fe31c0caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:52.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe321815700 1 -- 172.21.15.104:0/2081870285 >> v2:172.21.15.121:3300/0 conn(0x7fe31c0c9d90 msgr2=0x7fe31c0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:52.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe321815700 1 --2- 172.21.15.104:0/2081870285 >> v2:172.21.15.121:3300/0 conn(0x7fe31c0c9d90 0x7fe31c0ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:52.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.005+0000 7fe321815700 1 -- 172.21.15.104:0/2081870285 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c0ca6a0 con 0x7fe31c0caf70 2023-07-14T11:55:52.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.009+0000 7fe321815700 1 --2- 172.21.15.104:0/2081870285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe31c0caf70 0x7fe31c0c86b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0cdf30 tx=0x7fe31400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f31cad2df41a3a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:52.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.009+0000 7fe313fff700 1 -- 172.21.15.104:0/2081870285 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31400e070 con 0x7fe31c0caf70 2023-07-14T11:55:52.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.009+0000 7fe313fff700 1 -- 172.21.15.104:0/2081870285 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31400b910 con 0x7fe31c0caf70 2023-07-14T11:55:52.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.009+0000 7fe313fff700 1 -- 172.21.15.104:0/2081870285 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31400bcc0 con 0x7fe31c0caf70 2023-07-14T11:55:52.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.009+0000 7fe322016700 1 -- 172.21.15.104:0/2081870285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe31c0caf70 msgr2=0x7fe31c0c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:52.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.009+0000 7fe322016700 1 --2- 172.21.15.104:0/2081870285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe31c0caf70 0x7fe31c0c86b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0cdf30 tx=0x7fe31400b330 comp rx=0 tx=0).stop 2023-07-14T11:55:52.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.009+0000 7fe322016700 1 -- 172.21.15.104:0/2081870285 shutdown_connections 2023-07-14T11:55:52.998 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.009+0000 7fe322016700 1 --2- 172.21.15.104:0/2081870285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe31c0caf70 0x7fe31c0c86b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:52.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.009+0000 7fe322016700 1 --2- 172.21.15.104:0/2081870285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe31c159f20 0x7fe31c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:52.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.009+0000 7fe322016700 1 --2- 172.21.15.104:0/2081870285 >> v2:172.21.15.121:3300/0 conn(0x7fe31c0c9d90 0x7fe31c0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:52.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.009+0000 7fe322016700 1 -- 172.21.15.104:0/2081870285 >> 172.21.15.104:0/2081870285 conn(0x7fe31c06e960 msgr2=0x7fe31c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:52.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.009+0000 7fe322016700 1 -- 172.21.15.104:0/2081870285 shutdown_connections 2023-07-14T11:55:52.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.009+0000 7fe322016700 1 -- 172.21.15.104:0/2081870285 wait complete. 2023-07-14T11:55:53.000 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.013+0000 7fe322016700 1 Processor -- start 2023-07-14T11:55:53.000 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.013+0000 7fe322016700 1 -- start start 2023-07-14T11:55:53.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.013+0000 7fe322016700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe31c0c9d90 0x7fe31c1f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.013+0000 7fe322016700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe31c0caf70 0x7fe31c1581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.013+0000 7fe322016700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe31c159f20 0x7fe31c154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.013+0000 7fe322016700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c1f17e0 con 0x7fe31c159f20 2023-07-14T11:55:53.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.013+0000 7fe322016700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c158ac0 con 0x7fe31c0c9d90 2023-07-14T11:55:53.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.013+0000 7fe322016700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c158da0 con 0x7fe31c0caf70 2023-07-14T11:55:53.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.013+0000 7fe321815700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe31c159f20 0x7fe31c154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.013+0000 7fe321815700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe31c159f20 0x7fe31c154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35198/0 (socket says 172.21.15.104:35198) 2023-07-14T11:55:53.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.013+0000 7fe321815700 1 -- 172.21.15.104:0/330781430 learned_addr learned my addr 172.21.15.104:0/330781430 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:53.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.013+0000 7fe321014700 1 --2- 172.21.15.104:0/330781430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe31c0c9d90 0x7fe31c1f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.017+0000 7fe320813700 1 --2- 172.21.15.104:0/330781430 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe31c0caf70 0x7fe31c1581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.017+0000 7fe321815700 1 -- 172.21.15.104:0/330781430 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe31c0caf70 msgr2=0x7fe31c1581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.017+0000 7fe321815700 1 --2- 172.21.15.104:0/330781430 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe31c0caf70 0x7fe31c1581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.017+0000 7fe321815700 1 -- 172.21.15.104:0/330781430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe31c0c9d90 msgr2=0x7fe31c1f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.017+0000 7fe321815700 1 --2- 172.21.15.104:0/330781430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe31c0c9d90 0x7fe31c1f1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.017+0000 7fe321815700 1 -- 172.21.15.104:0/330781430 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe314009b30 con 0x7fe31c159f20 2023-07-14T11:55:53.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.017+0000 7fe321815700 1 --2- 172.21.15.104:0/330781430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe31c159f20 0x7fe31c154170 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe31c06a6f0 tx=0x7fe314004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:53.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.017+0000 7fe311ffb700 1 -- 172.21.15.104:0/330781430 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3140099b0 con 0x7fe31c159f20 2023-07-14T11:55:53.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.017+0000 7fe311ffb700 1 -- 172.21.15.104:0/330781430 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3140092e0 con 0x7fe31c159f20 2023-07-14T11:55:53.005 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.017+0000 7fe311ffb700 1 -- 172.21.15.104:0/330781430 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314004470 con 0x7fe31c159f20 2023-07-14T11:55:53.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.017+0000 7fe322016700 1 -- 172.21.15.104:0/330781430 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c1f1400 con 0x7fe31c159f20 2023-07-14T11:55:53.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.017+0000 7fe322016700 1 -- 172.21.15.104:0/330781430 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c1586f0 con 0x7fe31c159f20 2023-07-14T11:55:53.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.017+0000 7fe322016700 1 -- 172.21.15.104:0/330781430 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c1f1a50 con 0x7fe31c159f20 2023-07-14T11:55:53.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.021+0000 7fe311ffb700 1 -- 172.21.15.104:0/330781430 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe3140045d0 con 0x7fe31c159f20 2023-07-14T11:55:53.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.021+0000 7fe311ffb700 1 --2- 172.21.15.104:0/330781430 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe3000662f0 0x7fe3000687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.021+0000 7fe311ffb700 1 -- 172.21.15.104:0/330781430 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fe314085430 con 0x7fe31c159f20 2023-07-14T11:55:53.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.021+0000 7fe321014700 1 --2- 172.21.15.104:0/330781430 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe3000662f0 0x7fe3000687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.021+0000 7fe321014700 1 --2- 172.21.15.104:0/330781430 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe3000662f0 0x7fe3000687a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe3180045f0 tx=0x7fe318008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:53.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.029+0000 7fe311ffb700 1 -- 172.21.15.104:0/330781430 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe314055fe0 con 0x7fe31c159f20 2023-07-14T11:55:53.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.233+0000 7fe322016700 1 -- 172.21.15.104:0/330781430 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe31c04e500 con 0x7fe31c159f20 2023-07-14T11:55:53.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.233+0000 7fe311ffb700 1 -- 172.21.15.104:0/330781430 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fe314055b70 con 0x7fe31c159f20 2023-07-14T11:55:53.222 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:53.223 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3696540262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3696540262},{"type":"v1","addr":"172.21.15.104:6835","nonce":3696540262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/684052037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":684052037},{"type":"v1","addr":"172.21.15.121:6835","nonce":684052037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/1267091306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":1267091306},{"type":"v1","addr":"172.21.15.121:6833","nonce":1267091306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:54:58.600122+0000","modified":"2023-07-14T11:55:09.633871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5616},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5616":{"gid":5616,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6837/398018673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":398018673},{"type":"v1","addr":"172.21.15.104:6837","nonce":398018673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-14T11:55:53.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.237+0000 7fe322016700 1 -- 172.21.15.104:0/330781430 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe3000662f0 msgr2=0x7fe3000687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.237+0000 7fe322016700 1 --2- 172.21.15.104:0/330781430 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe3000662f0 0x7fe3000687a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe3180045f0 tx=0x7fe318008040 comp rx=0 tx=0).stop 2023-07-14T11:55:53.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.237+0000 7fe322016700 1 -- 172.21.15.104:0/330781430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe31c159f20 msgr2=0x7fe31c154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.237+0000 7fe322016700 1 --2- 172.21.15.104:0/330781430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe31c159f20 0x7fe31c154170 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe31c06a6f0 tx=0x7fe314004af0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.237+0000 7fe322016700 1 -- 172.21.15.104:0/330781430 shutdown_connections 2023-07-14T11:55:53.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.237+0000 7fe322016700 1 --2- 172.21.15.104:0/330781430 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe3000662f0 0x7fe3000687a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.237+0000 7fe322016700 1 --2- 172.21.15.104:0/330781430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe31c159f20 0x7fe31c154170 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.237+0000 7fe322016700 1 --2- 172.21.15.104:0/330781430 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe31c0caf70 0x7fe31c1581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.237+0000 7fe322016700 1 --2- 172.21.15.104:0/330781430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe31c0c9d90 0x7fe31c1f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.237+0000 7fe322016700 1 -- 172.21.15.104:0/330781430 >> 172.21.15.104:0/330781430 conn(0x7fe31c06e960 msgr2=0x7fe31c15a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:53.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.237+0000 7fe322016700 1 -- 172.21.15.104:0/330781430 shutdown_connections 2023-07-14T11:55:53.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.237+0000 7fe322016700 1 -- 172.21.15.104:0/330781430 wait complete. 2023-07-14T11:55:53.227 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 19 2023-07-14T11:55:53.247 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:55:53.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.429+0000 7fc27f7bf700 1 Processor -- start 2023-07-14T11:55:53.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.429+0000 7fc27f7bf700 1 -- start start 2023-07-14T11:55:53.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.433+0000 7fc27f7bf700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc27815bf20 0x7fc2780b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.433+0000 7fc27f7bf700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc27815c2f0 con 0x7fc2780d1430 2023-07-14T11:55:53.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.433+0000 7fc27f7bf700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc278070830 con 0x7fc2780b1530 2023-07-14T11:55:53.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.433+0000 7fc27f7bf700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc278070b10 con 0x7fc27815bf20 2023-07-14T11:55:53.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.433+0000 7fc27d55b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc27815bf20 0x7fc2780b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.433+0000 7fc27d55b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc27815bf20 0x7fc2780b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:47894/0 (socket says 172.21.15.104:47894) 2023-07-14T11:55:53.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.433+0000 7fc27d55b700 1 -- 172.21.15.104:0/3493466040 learned_addr learned my addr 172.21.15.104:0/3493466040 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:53.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.433+0000 7fc27dd5c700 1 --1- 172.21.15.104:0/3493466040 >> v1:172.21.15.104:6789/0 conn(0x7fc2780d1430 0x7fc2780b4050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56166/0 (socket says 172.21.15.104:56166) 2023-07-14T11:55:53.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.433+0000 7fc27d55b700 1 -- 172.21.15.104:0/3493466040 >> v1:172.21.15.121:6789/0 conn(0x7fc2780b1530 legacy=0x7fc2780d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.433+0000 7fc27d55b700 1 -- 172.21.15.104:0/3493466040 >> v1:172.21.15.104:6789/0 conn(0x7fc2780d1430 legacy=0x7fc2780b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.433+0000 7fc27d55b700 1 -- 172.21.15.104:0/3493466040 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2780cccc0 con 0x7fc27815bf20 2023-07-14T11:55:53.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.437+0000 7fc27d55b700 1 --2- 172.21.15.104:0/3493466040 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc27815bf20 0x7fc2780b5420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc274009e80 tx=0x7fc27400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4a1451abcfe97df server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:53.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.437+0000 7fc26ffff700 1 -- 172.21.15.104:0/3493466040 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27400e070 con 0x7fc27815bf20 2023-07-14T11:55:53.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.437+0000 7fc26ffff700 1 -- 172.21.15.104:0/3493466040 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27400b9c0 con 0x7fc27815bf20 2023-07-14T11:55:53.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.437+0000 7fc26ffff700 1 -- 172.21.15.104:0/3493466040 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27400bd70 con 0x7fc27815bf20 2023-07-14T11:55:53.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.437+0000 7fc27f7bf700 1 -- 172.21.15.104:0/3493466040 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc27815bf20 msgr2=0x7fc2780b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.437+0000 7fc27f7bf700 1 --2- 172.21.15.104:0/3493466040 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc27815bf20 0x7fc2780b5420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc274009e80 tx=0x7fc27400b330 comp rx=0 tx=0).stop 2023-07-14T11:55:53.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.437+0000 7fc27f7bf700 1 -- 172.21.15.104:0/3493466040 shutdown_connections 2023-07-14T11:55:53.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.437+0000 7fc27f7bf700 1 --2- 172.21.15.104:0/3493466040 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc27815bf20 0x7fc2780b5420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.437+0000 7fc27f7bf700 1 -- 172.21.15.104:0/3493466040 >> 172.21.15.104:0/3493466040 conn(0x7fc27806fc80 msgr2=0x7fc278070080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:53.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.437+0000 7fc27f7bf700 1 -- 172.21.15.104:0/3493466040 shutdown_connections 2023-07-14T11:55:53.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.437+0000 7fc27f7bf700 1 -- 172.21.15.104:0/3493466040 wait complete. 2023-07-14T11:55:53.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27f7bf700 1 Processor -- start 2023-07-14T11:55:53.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27f7bf700 1 -- start start 2023-07-14T11:55:53.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27f7bf700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2780b1530 0x7fc278143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27f7bf700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2780d1430 0x7fc278144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27f7bf700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc27815bf20 0x7fc2780ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27f7bf700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2780cc3d0 con 0x7fc2780d1430 2023-07-14T11:55:53.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27f7bf700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781448b0 con 0x7fc2780b1530 2023-07-14T11:55:53.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27f7bf700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc278144b90 con 0x7fc27815bf20 2023-07-14T11:55:53.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27cd5a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2780d1430 0x7fc278144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27d55b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2780b1530 0x7fc278143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27cd5a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2780d1430 0x7fc278144290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35200/0 (socket says 172.21.15.104:35200) 2023-07-14T11:55:53.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27cd5a700 1 -- 172.21.15.104:0/3574149321 learned_addr learned my addr 172.21.15.104:0/3574149321 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:53.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27dd5c700 1 --2- 172.21.15.104:0/3574149321 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc27815bf20 0x7fc2780ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27cd5a700 1 -- 172.21.15.104:0/3574149321 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc27815bf20 msgr2=0x7fc2780ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27cd5a700 1 --2- 172.21.15.104:0/3574149321 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc27815bf20 0x7fc2780ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27cd5a700 1 -- 172.21.15.104:0/3574149321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2780b1530 msgr2=0x7fc278143d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27cd5a700 1 --2- 172.21.15.104:0/3574149321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2780b1530 0x7fc278143d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.441+0000 7fc27cd5a700 1 -- 172.21.15.104:0/3574149321 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274009b30 con 0x7fc2780d1430 2023-07-14T11:55:53.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.445+0000 7fc27cd5a700 1 --2- 172.21.15.104:0/3574149321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2780d1430 0x7fc278144290 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc26800c500 tx=0x7fc26800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:53.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.445+0000 7fc26dffb700 1 -- 172.21.15.104:0/3574149321 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2680052d0 con 0x7fc2780d1430 2023-07-14T11:55:53.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.445+0000 7fc26dffb700 1 -- 172.21.15.104:0/3574149321 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc268005430 con 0x7fc2780d1430 2023-07-14T11:55:53.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.445+0000 7fc26dffb700 1 -- 172.21.15.104:0/3574149321 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2680055a0 con 0x7fc2780d1430 2023-07-14T11:55:53.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.445+0000 7fc27f7bf700 1 -- 172.21.15.104:0/3574149321 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc278144e70 con 0x7fc2780d1430 2023-07-14T11:55:53.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.445+0000 7fc27f7bf700 1 -- 172.21.15.104:0/3574149321 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2780cee00 con 0x7fc2780d1430 2023-07-14T11:55:53.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.445+0000 7fc27f7bf700 1 -- 172.21.15.104:0/3574149321 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27804dcb0 con 0x7fc2780d1430 2023-07-14T11:55:53.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.449+0000 7fc26dffb700 1 -- 172.21.15.104:0/3574149321 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc2680038e0 con 0x7fc2780d1430 2023-07-14T11:55:53.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.449+0000 7fc26dffb700 1 --2- 172.21.15.104:0/3574149321 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc25c0663c0 0x7fc25c068870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.449+0000 7fc26dffb700 1 -- 172.21.15.104:0/3574149321 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fc268083350 con 0x7fc2780d1430 2023-07-14T11:55:53.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.449+0000 7fc27d55b700 1 --2- 172.21.15.104:0/3574149321 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc25c0663c0 0x7fc25c068870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.449+0000 7fc27d55b700 1 --2- 172.21.15.104:0/3574149321 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc25c0663c0 0x7fc25c068870 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc274000f80 tx=0x7fc274009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:53.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.453+0000 7fc26dffb700 1 -- 172.21.15.104:0/3574149321 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fc2680504e0 con 0x7fc2780d1430 2023-07-14T11:55:53.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.629+0000 7fc27f7bf700 1 -- 172.21.15.104:0/3574149321 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2780d1f40 con 0x7fc2780d1430 2023-07-14T11:55:53.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.629+0000 7fc26dffb700 1 -- 172.21.15.104:0/3574149321 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+20311 (secure 0 0 0) 0x7fc268053b00 con 0x7fc2780d1430 2023-07-14T11:55:53.618 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:53.619 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":45,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:55:10.654581+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:54:56.370313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-14T11:54:57.608947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"data_new","create_time":"2023-07-14T11:55:08.099546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:55:53.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.633+0000 7fc27f7bf700 1 -- 172.21.15.104:0/3574149321 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc25c0663c0 msgr2=0x7fc25c068870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.633+0000 7fc27f7bf700 1 --2- 172.21.15.104:0/3574149321 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc25c0663c0 0x7fc25c068870 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc274000f80 tx=0x7fc274009000 comp rx=0 tx=0).stop 2023-07-14T11:55:53.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.633+0000 7fc27f7bf700 1 -- 172.21.15.104:0/3574149321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2780d1430 msgr2=0x7fc278144290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.633+0000 7fc27f7bf700 1 --2- 172.21.15.104:0/3574149321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2780d1430 0x7fc278144290 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc26800c500 tx=0x7fc26800c810 comp rx=0 tx=0).stop 2023-07-14T11:55:53.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.633+0000 7fc27f7bf700 1 -- 172.21.15.104:0/3574149321 shutdown_connections 2023-07-14T11:55:53.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.633+0000 7fc27f7bf700 1 --2- 172.21.15.104:0/3574149321 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc25c0663c0 0x7fc25c068870 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.633+0000 7fc27f7bf700 1 --2- 172.21.15.104:0/3574149321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2780d1430 0x7fc278144290 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.633+0000 7fc27f7bf700 1 --2- 172.21.15.104:0/3574149321 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc27815bf20 0x7fc2780ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.633+0000 7fc27f7bf700 1 --2- 172.21.15.104:0/3574149321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2780b1530 0x7fc278143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.633+0000 7fc27f7bf700 1 -- 172.21.15.104:0/3574149321 >> 172.21.15.104:0/3574149321 conn(0x7fc27806fc80 msgr2=0x7fc2780b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:53.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.633+0000 7fc27f7bf700 1 -- 172.21.15.104:0/3574149321 shutdown_connections 2023-07-14T11:55:53.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.633+0000 7fc27f7bf700 1 -- 172.21.15.104:0/3574149321 wait complete. 2023-07-14T11:55:53.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:54:56.370313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:54:57.608947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'data_new', 'create_time': '2023-07-14T11:55:08.099546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:55:53.646 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:55:53.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.829+0000 7ff9b659e700 1 Processor -- start 2023-07-14T11:55:53.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.829+0000 7ff9b659e700 1 -- start start 2023-07-14T11:55:53.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.829+0000 7ff9b659e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff9b0160000 0x7ff9b006dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.829+0000 7ff9b659e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff9b01603d0 0x7ff9b00c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.829+0000 7ff9b659e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b00cdfb0 con 0x7ff9b00cd7f0 2023-07-14T11:55:53.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.829+0000 7ff9b659e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b0070e40 con 0x7ff9b01603d0 2023-07-14T11:55:53.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.829+0000 7ff9b659e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b00cdcd0 con 0x7ff9b0160000 2023-07-14T11:55:53.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b4d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff9b01603d0 0x7ff9b00c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b559c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff9b0160000 0x7ff9b006dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b5d9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff9b00cd7f0 0x7ff9b00c0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:39900/0 (socket says 172.21.15.104:39900) 2023-07-14T11:55:53.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b5d9d700 1 -- 172.21.15.104:0/4006681366 learned_addr learned my addr 172.21.15.104:0/4006681366 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:53.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b4d9b700 1 -- 172.21.15.104:0/4006681366 >> v2:172.21.15.121:3300/0 conn(0x7ff9b0160000 msgr2=0x7ff9b006dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:53.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b4d9b700 1 --2- 172.21.15.104:0/4006681366 >> v2:172.21.15.121:3300/0 conn(0x7ff9b0160000 0x7ff9b006dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b4d9b700 1 -- 172.21.15.104:0/4006681366 >> v1:172.21.15.121:6789/0 conn(0x7ff9b00cd7f0 legacy=0x7ff9b00c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b4d9b700 1 -- 172.21.15.104:0/4006681366 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b0070780 con 0x7ff9b01603d0 2023-07-14T11:55:53.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9a7fff700 1 -- 172.21.15.104:0/4006681366 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250748112 0 0) 0x7ff9a8001490 con 0x7ff9b00cd7f0 2023-07-14T11:55:53.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b4d9b700 1 --2- 172.21.15.104:0/4006681366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b01603d0 0x7ff9b00c9e40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff9b006b3b0 tx=0x7ff9a0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e882c9ee2326e96e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:53.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9a7fff700 1 -- 172.21.15.104:0/4006681366 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0011830 con 0x7ff9b01603d0 2023-07-14T11:55:53.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9a7fff700 1 -- 172.21.15.104:0/4006681366 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a0011990 con 0x7ff9b01603d0 2023-07-14T11:55:53.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9a7fff700 1 -- 172.21.15.104:0/4006681366 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0011d20 con 0x7ff9b01603d0 2023-07-14T11:55:53.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b659e700 1 -- 172.21.15.104:0/4006681366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b01603d0 msgr2=0x7ff9b00c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b659e700 1 --2- 172.21.15.104:0/4006681366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b01603d0 0x7ff9b00c9e40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff9b006b3b0 tx=0x7ff9a0009b30 comp rx=0 tx=0).stop 2023-07-14T11:55:53.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b659e700 1 -- 172.21.15.104:0/4006681366 shutdown_connections 2023-07-14T11:55:53.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b659e700 1 --2- 172.21.15.104:0/4006681366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b01603d0 0x7ff9b00c9e40 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b659e700 1 --2- 172.21.15.104:0/4006681366 >> v2:172.21.15.121:3300/0 conn(0x7ff9b0160000 0x7ff9b006dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b659e700 1 -- 172.21.15.104:0/4006681366 >> 172.21.15.104:0/4006681366 conn(0x7ff9b00afc60 msgr2=0x7ff9b00b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:53.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b659e700 1 -- 172.21.15.104:0/4006681366 shutdown_connections 2023-07-14T11:55:53.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.833+0000 7ff9b659e700 1 -- 172.21.15.104:0/4006681366 wait complete. 2023-07-14T11:55:53.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b659e700 1 Processor -- start 2023-07-14T11:55:53.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b659e700 1 -- start start 2023-07-14T11:55:53.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b659e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b00cd7f0 0x7ff9b0158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b659e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9b0160000 0x7ff9b0162530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b659e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9b01603d0 0x7ff9b0162a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b659e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00b2950 con 0x7ff9b00cd7f0 2023-07-14T11:55:53.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b659e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b0155d50 con 0x7ff9b01603d0 2023-07-14T11:55:53.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b659e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00b1dd0 con 0x7ff9b0160000 2023-07-14T11:55:53.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b559c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b00cd7f0 0x7ff9b0158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b559c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b00cd7f0 0x7ff9b0158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35226/0 (socket says 172.21.15.104:35226) 2023-07-14T11:55:53.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b559c700 1 -- 172.21.15.104:0/2053713276 learned_addr learned my addr 172.21.15.104:0/2053713276 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:53.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b4d9b700 1 --2- 172.21.15.104:0/2053713276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9b0160000 0x7ff9b0162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b5d9d700 1 --2- 172.21.15.104:0/2053713276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9b01603d0 0x7ff9b0162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b559c700 1 -- 172.21.15.104:0/2053713276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9b0160000 msgr2=0x7ff9b0162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b559c700 1 --2- 172.21.15.104:0/2053713276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9b0160000 0x7ff9b0162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b559c700 1 -- 172.21.15.104:0/2053713276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9b01603d0 msgr2=0x7ff9b0162a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b559c700 1 --2- 172.21.15.104:0/2053713276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9b01603d0 0x7ff9b0162a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b559c700 1 -- 172.21.15.104:0/2053713276 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a00094b0 con 0x7ff9b00cd7f0 2023-07-14T11:55:53.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b4d9b700 1 --2- 172.21.15.104:0/2053713276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9b0160000 0x7ff9b0162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:53.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.837+0000 7ff9b559c700 1 --2- 172.21.15.104:0/2053713276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b00cd7f0 0x7ff9b0158080 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00e9d0 tx=0x7ff9ac00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:53.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.841+0000 7ff9a5ffb700 1 -- 172.21.15.104:0/2053713276 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac009720 con 0x7ff9b00cd7f0 2023-07-14T11:55:53.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.841+0000 7ff9a5ffb700 1 -- 172.21.15.104:0/2053713276 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac00c850 con 0x7ff9b00cd7f0 2023-07-14T11:55:53.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.841+0000 7ff9b659e700 1 -- 172.21.15.104:0/2053713276 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b00b20b0 con 0x7ff9b00cd7f0 2023-07-14T11:55:53.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.841+0000 7ff9a5ffb700 1 -- 172.21.15.104:0/2053713276 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac00cbc0 con 0x7ff9b00cd7f0 2023-07-14T11:55:53.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.841+0000 7ff9b659e700 1 -- 172.21.15.104:0/2053713276 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b00b2670 con 0x7ff9b00cd7f0 2023-07-14T11:55:53.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.841+0000 7ff9b659e700 1 -- 172.21.15.104:0/2053713276 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b0155ec0 con 0x7ff9b00cd7f0 2023-07-14T11:55:53.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.841+0000 7ff9a5ffb700 1 -- 172.21.15.104:0/2053713276 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff9ac00cd20 con 0x7ff9b00cd7f0 2023-07-14T11:55:53.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.845+0000 7ff9a5ffb700 1 --2- 172.21.15.104:0/2053713276 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff994066320 0x7ff9940687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.845+0000 7ff9a5ffb700 1 -- 172.21.15.104:0/2053713276 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7ff9ac084090 con 0x7ff9b00cd7f0 2023-07-14T11:55:53.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.845+0000 7ff9b4d9b700 1 --2- 172.21.15.104:0/2053713276 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff994066320 0x7ff9940687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.845+0000 7ff9b4d9b700 1 --2- 172.21.15.104:0/2053713276 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff994066320 0x7ff9940687d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0000c00 tx=0x7ff9a0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:53.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:53.849+0000 7ff9a5ffb700 1 -- 172.21.15.104:0/2053713276 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff9b0155ec0 con 0x7ff9b00cd7f0 2023-07-14T11:55:54.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.021+0000 7ff9b659e700 1 -- 172.21.15.104:0/2053713276 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9b0155ec0 con 0x7ff9b00cd7f0 2023-07-14T11:55:54.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.021+0000 7ff9a5ffb700 1 -- 172.21.15.104:0/2053713276 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9b0155ec0 con 0x7ff9b00cd7f0 2023-07-14T11:55:54.008 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:55:54.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.025+0000 7ff9b659e700 1 -- 172.21.15.104:0/2053713276 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff994066320 msgr2=0x7ff9940687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:54.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.025+0000 7ff9b659e700 1 --2- 172.21.15.104:0/2053713276 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff994066320 0x7ff9940687d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0000c00 tx=0x7ff9a0008030 comp rx=0 tx=0).stop 2023-07-14T11:55:54.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.025+0000 7ff9b659e700 1 -- 172.21.15.104:0/2053713276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b00cd7f0 msgr2=0x7ff9b0158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:54.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.025+0000 7ff9b659e700 1 --2- 172.21.15.104:0/2053713276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b00cd7f0 0x7ff9b0158080 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00e9d0 tx=0x7ff9ac00ed90 comp rx=0 tx=0).stop 2023-07-14T11:55:54.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.025+0000 7ff9b659e700 1 -- 172.21.15.104:0/2053713276 shutdown_connections 2023-07-14T11:55:54.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.025+0000 7ff9b659e700 1 --2- 172.21.15.104:0/2053713276 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff994066320 0x7ff9940687d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.025+0000 7ff9b659e700 1 --2- 172.21.15.104:0/2053713276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9b01603d0 0x7ff9b0162a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.025+0000 7ff9b659e700 1 --2- 172.21.15.104:0/2053713276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9b0160000 0x7ff9b0162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.025+0000 7ff9b659e700 1 --2- 172.21.15.104:0/2053713276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b00cd7f0 0x7ff9b0158080 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.025+0000 7ff9b659e700 1 -- 172.21.15.104:0/2053713276 >> 172.21.15.104:0/2053713276 conn(0x7ff9b00afc60 msgr2=0x7ff9b006c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:54.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.025+0000 7ff9b659e700 1 -- 172.21.15.104:0/2053713276 shutdown_connections 2023-07-14T11:55:54.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.025+0000 7ff9b659e700 1 -- 172.21.15.104:0/2053713276 wait complete. 2023-07-14T11:55:54.033 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:55:54.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.225+0000 7f4889a7d700 1 Processor -- start 2023-07-14T11:55:54.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.225+0000 7f4889a7d700 1 -- start start 2023-07-14T11:55:54.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.229+0000 7f4889a7d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f48840ad710 0x7f48840adae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:54.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.229+0000 7f4889a7d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f488406a2e0 0x7f488406a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:54.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.229+0000 7f4889a7d700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4884069e50 con 0x7f48840b35f0 2023-07-14T11:55:54.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.229+0000 7f4889a7d700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48840ae020 con 0x7f488406a2e0 2023-07-14T11:55:54.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.229+0000 7f4889a7d700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48840b1cc0 con 0x7f48840ad710 2023-07-14T11:55:54.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.229+0000 7f4883fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f488406a2e0 0x7f488406a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:54.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.229+0000 7f4883fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f488406a2e0 0x7f488406a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35242/0 (socket says 172.21.15.104:35242) 2023-07-14T11:55:54.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.229+0000 7f4883fff700 1 -- 172.21.15.104:0/1787964843 learned_addr learned my addr 172.21.15.104:0/1787964843 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:54.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.229+0000 7f4888a7b700 1 --2- 172.21.15.104:0/1787964843 >> v2:172.21.15.121:3300/0 conn(0x7f48840ad710 0x7f48840adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:54.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.229+0000 7f4883fff700 1 -- 172.21.15.104:0/1787964843 >> v2:172.21.15.121:3300/0 conn(0x7f48840ad710 msgr2=0x7f48840adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:54.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.229+0000 7f4883fff700 1 --2- 172.21.15.104:0/1787964843 >> v2:172.21.15.121:3300/0 conn(0x7f48840ad710 0x7f48840adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.229+0000 7f4883fff700 1 -- 172.21.15.104:0/1787964843 >> v1:172.21.15.121:6789/0 conn(0x7f48840b35f0 legacy=0x7f488406e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:54.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.229+0000 7f4883fff700 1 -- 172.21.15.104:0/1787964843 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48840b1fa0 con 0x7f488406a2e0 2023-07-14T11:55:54.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.229+0000 7f4883fff700 1 --2- 172.21.15.104:0/1787964843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f488406a2e0 0x7f488406a730 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f488406bf30 tx=0x7f4870009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ea4741c0317671 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:54.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.233+0000 7f48837fe700 1 -- 172.21.15.104:0/1787964843 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870004230 con 0x7f488406a2e0 2023-07-14T11:55:54.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.233+0000 7f48837fe700 1 -- 172.21.15.104:0/1787964843 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4870004390 con 0x7f488406a2e0 2023-07-14T11:55:54.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.233+0000 7f48837fe700 1 -- 172.21.15.104:0/1787964843 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700036c0 con 0x7f488406a2e0 2023-07-14T11:55:54.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.233+0000 7f4889a7d700 1 -- 172.21.15.104:0/1787964843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f488406a2e0 msgr2=0x7f488406a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:54.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.233+0000 7f4889a7d700 1 --2- 172.21.15.104:0/1787964843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f488406a2e0 0x7f488406a730 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f488406bf30 tx=0x7f4870009bf0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.233+0000 7f4889a7d700 1 -- 172.21.15.104:0/1787964843 shutdown_connections 2023-07-14T11:55:54.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.233+0000 7f4889a7d700 1 --2- 172.21.15.104:0/1787964843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f488406a2e0 0x7f488406a730 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.233+0000 7f4889a7d700 1 --2- 172.21.15.104:0/1787964843 >> v2:172.21.15.121:3300/0 conn(0x7f48840ad710 0x7f48840adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.233+0000 7f4889a7d700 1 -- 172.21.15.104:0/1787964843 >> 172.21.15.104:0/1787964843 conn(0x7f48840b05e0 msgr2=0x7f48840b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:54.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.233+0000 7f4889a7d700 1 -- 172.21.15.104:0/1787964843 shutdown_connections 2023-07-14T11:55:54.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.233+0000 7f4889a7d700 1 -- 172.21.15.104:0/1787964843 wait complete. 2023-07-14T11:55:54.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.233+0000 7f4889a7d700 1 Processor -- start 2023-07-14T11:55:54.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f4889a7d700 1 -- start start 2023-07-14T11:55:54.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f4889a7d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f488406a2e0 0x7f4884151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:54.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f4889a7d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48840ad710 0x7f4884151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:54.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f4889a7d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48840b35f0 0x7f488415a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:54.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f4889a7d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48840c3850 con 0x7f48840b35f0 2023-07-14T11:55:54.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f4889a7d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4884152d90 con 0x7f48840ad710 2023-07-14T11:55:54.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f4889a7d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4884153070 con 0x7f488406a2e0 2023-07-14T11:55:54.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f488927c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48840b35f0 0x7f488415a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:54.226 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f488927c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48840b35f0 0x7f488415a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35254/0 (socket says 172.21.15.104:35254) 2023-07-14T11:55:54.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f488927c700 1 -- 172.21.15.104:0/3629102399 learned_addr learned my addr 172.21.15.104:0/3629102399 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:54.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f4883fff700 1 --2- 172.21.15.104:0/3629102399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48840ad710 0x7f4884151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:54.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f4888a7b700 1 --2- 172.21.15.104:0/3629102399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f488406a2e0 0x7f4884151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:54.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f488927c700 1 -- 172.21.15.104:0/3629102399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f488406a2e0 msgr2=0x7f4884151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:54.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f488927c700 1 --2- 172.21.15.104:0/3629102399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f488406a2e0 0x7f4884151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f488927c700 1 -- 172.21.15.104:0/3629102399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48840ad710 msgr2=0x7f4884151580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:54.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f488927c700 1 --2- 172.21.15.104:0/3629102399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48840ad710 0x7f4884151580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f488927c700 1 -- 172.21.15.104:0/3629102399 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4870009570 con 0x7f48840b35f0 2023-07-14T11:55:54.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f4888a7b700 1 --2- 172.21.15.104:0/3629102399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f488406a2e0 0x7f4884151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:54.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.237+0000 7f488927c700 1 --2- 172.21.15.104:0/3629102399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48840b35f0 0x7f488415a950 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f487c00b970 tx=0x7f487c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:54.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.241+0000 7f48817fa700 1 -- 172.21.15.104:0/3629102399 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c00c930 con 0x7f48840b35f0 2023-07-14T11:55:54.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.241+0000 7f48817fa700 1 -- 172.21.15.104:0/3629102399 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487c00cad0 con 0x7f48840b35f0 2023-07-14T11:55:54.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.241+0000 7f48817fa700 1 -- 172.21.15.104:0/3629102399 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c00ce80 con 0x7f48840b35f0 2023-07-14T11:55:54.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.241+0000 7f4889a7d700 1 -- 172.21.15.104:0/3629102399 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4884153350 con 0x7f48840b35f0 2023-07-14T11:55:54.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.241+0000 7f4889a7d700 1 -- 172.21.15.104:0/3629102399 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48840cbd90 con 0x7f48840b35f0 2023-07-14T11:55:54.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.241+0000 7f4889a7d700 1 -- 172.21.15.104:0/3629102399 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48840606e0 con 0x7f48840b35f0 2023-07-14T11:55:54.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.245+0000 7f48817fa700 1 -- 172.21.15.104:0/3629102399 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f487c014440 con 0x7f48840b35f0 2023-07-14T11:55:54.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.245+0000 7f48817fa700 1 --2- 172.21.15.104:0/3629102399 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f48680662f0 0x7f48680687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:54.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.245+0000 7f48817fa700 1 -- 172.21.15.104:0/3629102399 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f487c083be0 con 0x7f48840b35f0 2023-07-14T11:55:54.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.245+0000 7f4888a7b700 1 --2- 172.21.15.104:0/3629102399 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f48680662f0 0x7f48680687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:54.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.245+0000 7f4888a7b700 1 --2- 172.21.15.104:0/3629102399 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f48680662f0 0x7f48680687a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f48840cc940 tx=0x7f4878008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:54.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.249+0000 7f48817fa700 1 -- 172.21.15.104:0/3629102399 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f487c050da0 con 0x7f48840b35f0 2023-07-14T11:55:54.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.421+0000 7f4889a7d700 1 -- 172.21.15.104:0/3629102399 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f488404e500 con 0x7f48840b35f0 2023-07-14T11:55:54.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.421+0000 7f48817fa700 1 -- 172.21.15.104:0/3629102399 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f487c0190a0 con 0x7f48840b35f0 2023-07-14T11:55:54.410 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:55:54.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.425+0000 7f4889a7d700 1 -- 172.21.15.104:0/3629102399 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f48680662f0 msgr2=0x7f48680687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:54.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.425+0000 7f4889a7d700 1 --2- 172.21.15.104:0/3629102399 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f48680662f0 0x7f48680687a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f48840cc940 tx=0x7f4878008040 comp rx=0 tx=0).stop 2023-07-14T11:55:54.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.425+0000 7f4889a7d700 1 -- 172.21.15.104:0/3629102399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48840b35f0 msgr2=0x7f488415a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:54.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.425+0000 7f4889a7d700 1 --2- 172.21.15.104:0/3629102399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48840b35f0 0x7f488415a950 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f487c00b970 tx=0x7f487c00bc80 comp rx=0 tx=0).stop 2023-07-14T11:55:54.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.425+0000 7f4889a7d700 1 -- 172.21.15.104:0/3629102399 shutdown_connections 2023-07-14T11:55:54.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.425+0000 7f4889a7d700 1 --2- 172.21.15.104:0/3629102399 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f48680662f0 0x7f48680687a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.425+0000 7f4889a7d700 1 --2- 172.21.15.104:0/3629102399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48840b35f0 0x7f488415a950 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.425+0000 7f4889a7d700 1 --2- 172.21.15.104:0/3629102399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48840ad710 0x7f4884151580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.425+0000 7f4889a7d700 1 --2- 172.21.15.104:0/3629102399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f488406a2e0 0x7f4884151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.425+0000 7f4889a7d700 1 -- 172.21.15.104:0/3629102399 >> 172.21.15.104:0/3629102399 conn(0x7f48840b05e0 msgr2=0x7f48840ab8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:54.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.425+0000 7f4889a7d700 1 -- 172.21.15.104:0/3629102399 shutdown_connections 2023-07-14T11:55:54.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.425+0000 7f4889a7d700 1 -- 172.21.15.104:0/3629102399 wait complete. 2023-07-14T11:55:54.435 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:55:54.608 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.621+0000 7f7c47530700 1 Processor -- start 2023-07-14T11:55:54.609 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.621+0000 7f7c47530700 1 -- start start 2023-07-14T11:55:54.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.621+0000 7f7c47530700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c4015d050 0x7f7c400cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:54.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.621+0000 7f7c47530700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7c4006f070 0x7f7c400cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:54.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.621+0000 7f7c47530700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c4015d420 con 0x7f7c400cab70 2023-07-14T11:55:54.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.621+0000 7f7c47530700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c400cc260 con 0x7f7c4006f070 2023-07-14T11:55:54.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.621+0000 7f7c47530700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c400cb5c0 con 0x7f7c4015d050 2023-07-14T11:55:54.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c46d2f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7c4006f070 0x7f7c400cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:54.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c46d2f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7c4006f070 0x7f7c400cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35268/0 (socket says 172.21.15.104:35268) 2023-07-14T11:55:54.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c46d2f700 1 -- 172.21.15.104:0/2730497234 learned_addr learned my addr 172.21.15.104:0/2730497234 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:54.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c4652e700 1 --2- 172.21.15.104:0/2730497234 >> v2:172.21.15.121:3300/0 conn(0x7f7c4015d050 0x7f7c400cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:54.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c46d2f700 1 -- 172.21.15.104:0/2730497234 >> v2:172.21.15.121:3300/0 conn(0x7f7c4015d050 msgr2=0x7f7c400cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:54.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c46d2f700 1 --2- 172.21.15.104:0/2730497234 >> v2:172.21.15.121:3300/0 conn(0x7f7c4015d050 0x7f7c400cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c46d2f700 1 -- 172.21.15.104:0/2730497234 >> v1:172.21.15.121:6789/0 conn(0x7f7c400cab70 legacy=0x7f7c4006e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:54.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c46d2f700 1 -- 172.21.15.104:0/2730497234 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c400cb8a0 con 0x7f7c4006f070 2023-07-14T11:55:54.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c46d2f700 1 --2- 172.21.15.104:0/2730497234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c4006f070 0x7f7c400cafd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7c4006f780 tx=0x7f7c3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6bebce90e099d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:54.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c4552c700 1 -- 172.21.15.104:0/2730497234 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800e070 con 0x7f7c4006f070 2023-07-14T11:55:54.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c4552c700 1 -- 172.21.15.104:0/2730497234 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c38003a40 con 0x7f7c4006f070 2023-07-14T11:55:54.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c4552c700 1 -- 172.21.15.104:0/2730497234 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c38003df0 con 0x7f7c4006f070 2023-07-14T11:55:54.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c47530700 1 -- 172.21.15.104:0/2730497234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c4006f070 msgr2=0x7f7c400cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:54.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c47530700 1 --2- 172.21.15.104:0/2730497234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c4006f070 0x7f7c400cafd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7c4006f780 tx=0x7f7c3800b330 comp rx=0 tx=0).stop 2023-07-14T11:55:54.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c47530700 1 -- 172.21.15.104:0/2730497234 shutdown_connections 2023-07-14T11:55:54.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c47530700 1 --2- 172.21.15.104:0/2730497234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c4006f070 0x7f7c400cafd0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c47530700 1 --2- 172.21.15.104:0/2730497234 >> v2:172.21.15.121:3300/0 conn(0x7f7c4015d050 0x7f7c400cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c47530700 1 -- 172.21.15.104:0/2730497234 >> 172.21.15.104:0/2730497234 conn(0x7f7c4006fe60 msgr2=0x7f7c40070260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:54.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c47530700 1 -- 172.21.15.104:0/2730497234 shutdown_connections 2023-07-14T11:55:54.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.625+0000 7f7c47530700 1 -- 172.21.15.104:0/2730497234 wait complete. 2023-07-14T11:55:54.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.629+0000 7f7c47530700 1 Processor -- start 2023-07-14T11:55:54.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.629+0000 7f7c47530700 1 -- start start 2023-07-14T11:55:54.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.629+0000 7f7c47530700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c4006f070 0x7f7c4006a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:54.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.629+0000 7f7c47530700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c400cab70 0x7f7c401503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:54.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.629+0000 7f7c47530700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c4015d050 0x7f7c400b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:54.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c47530700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c40150df0 con 0x7f7c4015d050 2023-07-14T11:55:54.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c47530700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c40150210 con 0x7f7c400cab70 2023-07-14T11:55:54.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c47530700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c401594b0 con 0x7f7c4006f070 2023-07-14T11:55:54.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c46d2f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c4015d050 0x7f7c400b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:54.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c46d2f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c4015d050 0x7f7c400b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35274/0 (socket says 172.21.15.104:35274) 2023-07-14T11:55:54.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c46d2f700 1 -- 172.21.15.104:0/4270658699 learned_addr learned my addr 172.21.15.104:0/4270658699 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:54.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c4652e700 1 --2- 172.21.15.104:0/4270658699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c4006f070 0x7f7c4006a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:54.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c45d2d700 1 --2- 172.21.15.104:0/4270658699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c400cab70 0x7f7c401503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:54.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c46d2f700 1 -- 172.21.15.104:0/4270658699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c4006f070 msgr2=0x7f7c4006a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:54.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c46d2f700 1 --2- 172.21.15.104:0/4270658699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c4006f070 0x7f7c4006a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c46d2f700 1 -- 172.21.15.104:0/4270658699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c400cab70 msgr2=0x7f7c401503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:54.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c46d2f700 1 --2- 172.21.15.104:0/4270658699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c400cab70 0x7f7c401503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c46d2f700 1 -- 172.21.15.104:0/4270658699 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c38009b30 con 0x7f7c4015d050 2023-07-14T11:55:54.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c4652e700 1 --2- 172.21.15.104:0/4270658699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c4006f070 0x7f7c4006a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:54.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c46d2f700 1 --2- 172.21.15.104:0/4270658699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c4015d050 0x7f7c400b4220 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7c4006f7b0 tx=0x7f7c38004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:54.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c36ffd700 1 -- 172.21.15.104:0/4270658699 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c38004440 con 0x7f7c4015d050 2023-07-14T11:55:54.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c36ffd700 1 -- 172.21.15.104:0/4270658699 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c380045c0 con 0x7f7c4015d050 2023-07-14T11:55:54.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c36ffd700 1 -- 172.21.15.104:0/4270658699 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c38004970 con 0x7f7c4015d050 2023-07-14T11:55:54.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c47530700 1 -- 172.21.15.104:0/4270658699 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4006aea0 con 0x7f7c4015d050 2023-07-14T11:55:54.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c47530700 1 -- 172.21.15.104:0/4270658699 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4006b460 con 0x7f7c4015d050 2023-07-14T11:55:54.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.633+0000 7f7c47530700 1 -- 172.21.15.104:0/4270658699 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c40159680 con 0x7f7c4015d050 2023-07-14T11:55:54.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.637+0000 7f7c36ffd700 1 -- 172.21.15.104:0/4270658699 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7c38019040 con 0x7f7c4015d050 2023-07-14T11:55:54.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.637+0000 7f7c36ffd700 1 --2- 172.21.15.104:0/4270658699 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c240662f0 0x7f7c240687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:54.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.637+0000 7f7c36ffd700 1 -- 172.21.15.104:0/4270658699 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f7c38084790 con 0x7f7c4015d050 2023-07-14T11:55:54.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.641+0000 7f7c4652e700 1 --2- 172.21.15.104:0/4270658699 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c240662f0 0x7f7c240687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:54.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.641+0000 7f7c4652e700 1 --2- 172.21.15.104:0/4270658699 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c240662f0 0x7f7c240687a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009c40 tx=0x7f7c3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:54.633 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.645+0000 7f7c36ffd700 1 -- 172.21.15.104:0/4270658699 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f7c380518b0 con 0x7f7c4015d050 2023-07-14T11:55:54.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.825+0000 7f7c47530700 1 -- 172.21.15.104:0/4270658699 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7c4004e500 con 0x7f7c4015d050 2023-07-14T11:55:54.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.825+0000 7f7c36ffd700 1 -- 172.21.15.104:0/4270658699 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7c38054ed0 con 0x7f7c4015d050 2023-07-14T11:55:54.812 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:55:54.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.829+0000 7f7c47530700 1 -- 172.21.15.104:0/4270658699 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c240662f0 msgr2=0x7f7c240687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:54.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.829+0000 7f7c47530700 1 --2- 172.21.15.104:0/4270658699 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c240662f0 0x7f7c240687a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009c40 tx=0x7f7c3c008040 comp rx=0 tx=0).stop 2023-07-14T11:55:54.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.829+0000 7f7c47530700 1 -- 172.21.15.104:0/4270658699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c4015d050 msgr2=0x7f7c400b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:54.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.829+0000 7f7c47530700 1 --2- 172.21.15.104:0/4270658699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c4015d050 0x7f7c400b4220 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7c4006f7b0 tx=0x7f7c38004af0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.829+0000 7f7c47530700 1 -- 172.21.15.104:0/4270658699 shutdown_connections 2023-07-14T11:55:54.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.829+0000 7f7c47530700 1 --2- 172.21.15.104:0/4270658699 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7c240662f0 0x7f7c240687a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.829+0000 7f7c47530700 1 --2- 172.21.15.104:0/4270658699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c4015d050 0x7f7c400b4220 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.829+0000 7f7c47530700 1 --2- 172.21.15.104:0/4270658699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c400cab70 0x7f7c401503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.829+0000 7f7c47530700 1 --2- 172.21.15.104:0/4270658699 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c4006f070 0x7f7c4006a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:54.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.829+0000 7f7c47530700 1 -- 172.21.15.104:0/4270658699 >> 172.21.15.104:0/4270658699 conn(0x7f7c4006fe60 msgr2=0x7f7c40070230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:54.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.829+0000 7f7c47530700 1 -- 172.21.15.104:0/4270658699 shutdown_connections 2023-07-14T11:55:54.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:54.829+0000 7f7c47530700 1 -- 172.21.15.104:0/4270658699 wait complete. 2023-07-14T11:55:54.836 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:55:55.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.025+0000 7f9e2964a700 1 Processor -- start 2023-07-14T11:55:55.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.025+0000 7f9e2964a700 1 -- start start 2023-07-14T11:55:55.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e2964a700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9e240c8d90 0x7f9e240c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e2964a700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e2406d860 con 0x7f9e240b0490 2023-07-14T11:55:55.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e2964a700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e2406dcf0 con 0x7f9e2415c410 2023-07-14T11:55:55.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e2964a700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e240ca710 con 0x7f9e240c8d90 2023-07-14T11:55:55.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e227fc700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f9e240b0490 0x7f9e240cbed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56176/0 (socket says 172.21.15.104:56176) 2023-07-14T11:55:55.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e227fc700 1 -- 172.21.15.104:0/638203929 learned_addr learned my addr 172.21.15.104:0/638203929 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:55.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e237fe700 1 --2- 172.21.15.104:0/638203929 >> v2:172.21.15.121:3300/0 conn(0x7f9e240c8d90 0x7f9e240c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e21ffb700 1 -- 172.21.15.104:0/638203929 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220498644 0 0) 0x7f9e2406d860 con 0x7f9e240b0490 2023-07-14T11:55:55.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e21ffb700 1 -- 172.21.15.104:0/638203929 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e0c0035d0 con 0x7f9e240b0490 2023-07-14T11:55:55.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e21ffb700 1 -- 172.21.15.104:0/638203929 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2066653289 0 0) 0x7f9e0c0035d0 con 0x7f9e240b0490 2023-07-14T11:55:55.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e21ffb700 1 -- 172.21.15.104:0/638203929 >> v2:172.21.15.121:3300/0 conn(0x7f9e240c8d90 msgr2=0x7f9e240c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:55.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e21ffb700 1 --2- 172.21.15.104:0/638203929 >> v2:172.21.15.121:3300/0 conn(0x7f9e240c8d90 0x7f9e240c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e21ffb700 1 -- 172.21.15.104:0/638203929 >> v1:172.21.15.121:6789/0 conn(0x7f9e2415c410 legacy=0x7f9e240afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e21ffb700 1 -- 172.21.15.104:0/638203929 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2406cc50 con 0x7f9e240b0490 2023-07-14T11:55:55.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e21ffb700 1 -- 172.21.15.104:0/638203929 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f9e18001670 con 0x7f9e240b0490 2023-07-14T11:55:55.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e21ffb700 1 -- 172.21.15.104:0/638203929 >> v1:172.21.15.104:6789/0 conn(0x7f9e240b0490 legacy=0x7f9e240cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e21ffb700 1 --2- 172.21.15.104:0/638203929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e0c003a20 0x7f9e0c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e21ffb700 1 --2- 172.21.15.104:0/638203929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e0c004410 0x7f9e0c0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e21ffb700 1 --2- 172.21.15.104:0/638203929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e0c0050d0 0x7f9e0c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e21ffb700 1 -- 172.21.15.104:0/638203929 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240ca710 con 0x7f9e0c0050d0 2023-07-14T11:55:55.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e21ffb700 1 -- 172.21.15.104:0/638203929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c00c020 con 0x7f9e0c003a20 2023-07-14T11:55:55.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e21ffb700 1 -- 172.21.15.104:0/638203929 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c00c1e0 con 0x7f9e0c004410 2023-07-14T11:55:55.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e237fe700 1 --2- 172.21.15.104:0/638203929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e0c0050d0 0x7f9e0c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e237fe700 1 -- 172.21.15.104:0/638203929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e0c004410 msgr2=0x7f9e0c0048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:55.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e22ffd700 1 --2- 172.21.15.104:0/638203929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e0c003a20 0x7f9e0c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e227fc700 1 --2- 172.21.15.104:0/638203929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e0c004410 0x7f9e0c0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e237fe700 1 --2- 172.21.15.104:0/638203929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e0c004410 0x7f9e0c0048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e237fe700 1 -- 172.21.15.104:0/638203929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e0c003a20 msgr2=0x7f9e0c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:55.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e237fe700 1 --2- 172.21.15.104:0/638203929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e0c003a20 0x7f9e0c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e237fe700 1 -- 172.21.15.104:0/638203929 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c0035d0 con 0x7f9e0c0050d0 2023-07-14T11:55:55.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e22ffd700 1 --2- 172.21.15.104:0/638203929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e0c003a20 0x7f9e0c003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:55:55.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e227fc700 1 --2- 172.21.15.104:0/638203929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e0c004410 0x7f9e0c0048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:55:55.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.029+0000 7f9e237fe700 1 -- 172.21.15.104:0/638203929 reap_dead start 2023-07-14T11:55:55.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.033+0000 7f9e237fe700 1 --2- 172.21.15.104:0/638203929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e0c0050d0 0x7f9e0c00ba30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009fc0 tx=0x7f9e0c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80d92eb661e68ae server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:55.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.033+0000 7f9e21ffb700 1 -- 172.21.15.104:0/638203929 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c003a40 con 0x7f9e0c0050d0 2023-07-14T11:55:55.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.033+0000 7f9e21ffb700 1 -- 172.21.15.104:0/638203929 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1c00f040 con 0x7f9e0c0050d0 2023-07-14T11:55:55.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.033+0000 7f9e21ffb700 1 -- 172.21.15.104:0/638203929 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c003e00 con 0x7f9e0c0050d0 2023-07-14T11:55:55.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.033+0000 7f9e2964a700 1 -- 172.21.15.104:0/638203929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e0c0050d0 msgr2=0x7f9e0c00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.033+0000 7f9e2964a700 1 --2- 172.21.15.104:0/638203929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e0c0050d0 0x7f9e0c00ba30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009fc0 tx=0x7f9e0c0058a0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.033+0000 7f9e2964a700 1 -- 172.21.15.104:0/638203929 shutdown_connections 2023-07-14T11:55:55.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.033+0000 7f9e2964a700 1 --2- 172.21.15.104:0/638203929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e0c0050d0 0x7f9e0c00ba30 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.033+0000 7f9e2964a700 1 -- 172.21.15.104:0/638203929 >> 172.21.15.104:0/638203929 conn(0x7f9e240cad80 msgr2=0x7f9e240cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:55.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.033+0000 7f9e2964a700 1 -- 172.21.15.104:0/638203929 shutdown_connections 2023-07-14T11:55:55.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.033+0000 7f9e2964a700 1 -- 172.21.15.104:0/638203929 wait complete. 2023-07-14T11:55:55.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.033+0000 7f9e2964a700 1 Processor -- start 2023-07-14T11:55:55.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e2964a700 1 -- start start 2023-07-14T11:55:55.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e2964a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e0c0050d0 0x7f9e241531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e2964a700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e240ac9e0 0x7f9e240ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e2964a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e240c6420 0x7f9e240c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e2964a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2414b8d0 con 0x7f9e0c0050d0 2023-07-14T11:55:55.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e2964a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240a9bb0 con 0x7f9e240c6420 2023-07-14T11:55:55.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e2964a700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240ad390 con 0x7f9e240ac9e0 2023-07-14T11:55:55.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e22ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e0c0050d0 0x7f9e241531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e22ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e0c0050d0 0x7f9e241531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35302/0 (socket says 172.21.15.104:35302) 2023-07-14T11:55:55.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e22ffd700 1 -- 172.21.15.104:0/2333289445 learned_addr learned my addr 172.21.15.104:0/2333289445 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:55.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e227fc700 1 --2- 172.21.15.104:0/2333289445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e240ac9e0 0x7f9e240ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e237fe700 1 --2- 172.21.15.104:0/2333289445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e240c6420 0x7f9e240c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e22ffd700 1 -- 172.21.15.104:0/2333289445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e240ac9e0 msgr2=0x7f9e240ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e22ffd700 1 --2- 172.21.15.104:0/2333289445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e240ac9e0 0x7f9e240ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e22ffd700 1 -- 172.21.15.104:0/2333289445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e240c6420 msgr2=0x7f9e240c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.027 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e22ffd700 1 --2- 172.21.15.104:0/2333289445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e240c6420 0x7f9e240c6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e22ffd700 1 -- 172.21.15.104:0/2333289445 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e240c81d0 con 0x7f9e0c0050d0 2023-07-14T11:55:55.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e22ffd700 1 --2- 172.21.15.104:0/2333289445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e0c0050d0 0x7f9e241531e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400d7a0 tx=0x7f9e1400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:55.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e0b7fe700 1 -- 172.21.15.104:0/2333289445 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1400b070 con 0x7f9e0c0050d0 2023-07-14T11:55:55.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e0b7fe700 1 -- 172.21.15.104:0/2333289445 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e140054e0 con 0x7f9e0c0050d0 2023-07-14T11:55:55.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e0b7fe700 1 -- 172.21.15.104:0/2333289445 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e140038e0 con 0x7f9e0c0050d0 2023-07-14T11:55:55.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e2964a700 1 -- 172.21.15.104:0/2333289445 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e240a9030 con 0x7f9e0c0050d0 2023-07-14T11:55:55.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.037+0000 7f9e2964a700 1 -- 172.21.15.104:0/2333289445 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e240a98d0 con 0x7f9e0c0050d0 2023-07-14T11:55:55.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.041+0000 7f9e2964a700 1 -- 172.21.15.104:0/2333289445 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e240a9430 con 0x7f9e0c0050d0 2023-07-14T11:55:55.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.041+0000 7f9e0b7fe700 1 -- 172.21.15.104:0/2333289445 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9e14003a40 con 0x7f9e0c0050d0 2023-07-14T11:55:55.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.041+0000 7f9e0b7fe700 1 --2- 172.21.15.104:0/2333289445 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e100662f0 0x7f9e100687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.041+0000 7f9e0b7fe700 1 -- 172.21.15.104:0/2333289445 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f9e14083730 con 0x7f9e0c0050d0 2023-07-14T11:55:55.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.041+0000 7f9e227fc700 1 --2- 172.21.15.104:0/2333289445 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e100662f0 0x7f9e100687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.045+0000 7f9e227fc700 1 --2- 172.21.15.104:0/2333289445 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e100662f0 0x7f9e100687a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9e18001200 tx=0x7f9e1800a040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:55.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.049+0000 7f9e0b7fe700 1 -- 172.21.15.104:0/2333289445 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f9e14050950 con 0x7f9e0c0050d0 2023-07-14T11:55:55.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.221+0000 7f9e2964a700 1 -- 172.21.15.104:0/2333289445 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e2404dcb0 con 0x7f9e0c0050d0 2023-07-14T11:55:55.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.221+0000 7f9e0b7fe700 1 -- 172.21.15.104:0/2333289445 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e14053f70 con 0x7f9e0c0050d0 2023-07-14T11:55:55.209 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:55:55.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.225+0000 7f9e2964a700 1 -- 172.21.15.104:0/2333289445 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e100662f0 msgr2=0x7f9e100687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.225+0000 7f9e2964a700 1 --2- 172.21.15.104:0/2333289445 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e100662f0 0x7f9e100687a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9e18001200 tx=0x7f9e1800a040 comp rx=0 tx=0).stop 2023-07-14T11:55:55.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.225+0000 7f9e2964a700 1 -- 172.21.15.104:0/2333289445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e0c0050d0 msgr2=0x7f9e241531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.225+0000 7f9e2964a700 1 --2- 172.21.15.104:0/2333289445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e0c0050d0 0x7f9e241531e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400d7a0 tx=0x7f9e1400dab0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.225+0000 7f9e2964a700 1 -- 172.21.15.104:0/2333289445 shutdown_connections 2023-07-14T11:55:55.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.225+0000 7f9e2964a700 1 --2- 172.21.15.104:0/2333289445 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e100662f0 0x7f9e100687a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.225+0000 7f9e2964a700 1 --2- 172.21.15.104:0/2333289445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e240c6420 0x7f9e240c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.225+0000 7f9e2964a700 1 --2- 172.21.15.104:0/2333289445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e240ac9e0 0x7f9e240ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.225+0000 7f9e2964a700 1 --2- 172.21.15.104:0/2333289445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e0c0050d0 0x7f9e241531e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.225+0000 7f9e2964a700 1 -- 172.21.15.104:0/2333289445 >> 172.21.15.104:0/2333289445 conn(0x7f9e240cad80 msgr2=0x7f9e240cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:55.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.225+0000 7f9e2964a700 1 -- 172.21.15.104:0/2333289445 shutdown_connections 2023-07-14T11:55:55.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.225+0000 7f9e2964a700 1 -- 172.21.15.104:0/2333289445 wait complete. 2023-07-14T11:55:55.233 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-07-14T11:55:55.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.425+0000 7fbef6903700 1 Processor -- start 2023-07-14T11:55:55.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.425+0000 7fbef6903700 1 -- start start 2023-07-14T11:55:55.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.429+0000 7fbef6903700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbef01603d0 0x7fbef006dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.429+0000 7fbef6903700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef00cdfb0 con 0x7fbef00cd7f0 2023-07-14T11:55:55.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.429+0000 7fbef6903700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef00cdcd0 con 0x7fbef0160000 2023-07-14T11:55:55.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.429+0000 7fbef6903700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef0070e40 con 0x7fbef01603d0 2023-07-14T11:55:55.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.429+0000 7fbef5901700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fbef0160000 0x7fbef00c9e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:39944/0 (socket says 172.21.15.104:39944) 2023-07-14T11:55:55.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.429+0000 7fbef5901700 1 -- 172.21.15.104:0/235850640 learned_addr learned my addr 172.21.15.104:0/235850640 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:55.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.429+0000 7fbef5100700 1 --2- 172.21.15.104:0/235850640 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbef01603d0 0x7fbef006dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.429+0000 7fbef6102700 1 --1- 172.21.15.104:0/235850640 >> v1:172.21.15.104:6789/0 conn(0x7fbef00cd7f0 0x7fbef00c0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56188/0 (socket says 172.21.15.104:56188) 2023-07-14T11:55:55.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.429+0000 7fbef5100700 1 -- 172.21.15.104:0/235850640 >> v1:172.21.15.121:6789/0 conn(0x7fbef0160000 legacy=0x7fbef00c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.429+0000 7fbef5100700 1 -- 172.21.15.104:0/235850640 >> v1:172.21.15.104:6789/0 conn(0x7fbef00cd7f0 legacy=0x7fbef00c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.429+0000 7fbef48ff700 1 -- 172.21.15.104:0/235850640 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627170969 0 0) 0x7fbee8001490 con 0x7fbef00cd7f0 2023-07-14T11:55:55.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.429+0000 7fbef5100700 1 -- 172.21.15.104:0/235850640 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef0070780 con 0x7fbef01603d0 2023-07-14T11:55:55.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.429+0000 7fbef5100700 1 --2- 172.21.15.104:0/235850640 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbef01603d0 0x7fbef006dd10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009800 tx=0x7fbee0009b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95be88fcc92f7547 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:55.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.433+0000 7fbef48ff700 1 -- 172.21.15.104:0/235850640 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0004230 con 0x7fbef01603d0 2023-07-14T11:55:55.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.433+0000 7fbef48ff700 1 -- 172.21.15.104:0/235850640 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee0004390 con 0x7fbef01603d0 2023-07-14T11:55:55.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.433+0000 7fbef48ff700 1 -- 172.21.15.104:0/235850640 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee00036c0 con 0x7fbef01603d0 2023-07-14T11:55:55.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.433+0000 7fbef6903700 1 -- 172.21.15.104:0/235850640 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbef01603d0 msgr2=0x7fbef006dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.433+0000 7fbef6903700 1 --2- 172.21.15.104:0/235850640 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbef01603d0 0x7fbef006dd10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009800 tx=0x7fbee0009b10 comp rx=0 tx=0).stop 2023-07-14T11:55:55.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.433+0000 7fbef6903700 1 -- 172.21.15.104:0/235850640 shutdown_connections 2023-07-14T11:55:55.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.433+0000 7fbef6903700 1 --2- 172.21.15.104:0/235850640 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbef01603d0 0x7fbef006dd10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.433+0000 7fbef6903700 1 -- 172.21.15.104:0/235850640 >> 172.21.15.104:0/235850640 conn(0x7fbef00afc60 msgr2=0x7fbef00b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:55.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.433+0000 7fbef6903700 1 -- 172.21.15.104:0/235850640 shutdown_connections 2023-07-14T11:55:55.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.433+0000 7fbef6903700 1 -- 172.21.15.104:0/235850640 wait complete. 2023-07-14T11:55:55.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef6903700 1 Processor -- start 2023-07-14T11:55:55.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef6903700 1 -- start start 2023-07-14T11:55:55.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef6903700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef00cd7f0 0x7fbef01611d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef6903700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbef0160000 0x7fbef0157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef6903700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbef01603d0 0x7fbef006c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef6903700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef0162430 con 0x7fbef00cd7f0 2023-07-14T11:55:55.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef6903700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01617b0 con 0x7fbef0160000 2023-07-14T11:55:55.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef6903700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef0161a90 con 0x7fbef01603d0 2023-07-14T11:55:55.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef5901700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef00cd7f0 0x7fbef01611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef5901700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef00cd7f0 0x7fbef01611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35308/0 (socket says 172.21.15.104:35308) 2023-07-14T11:55:55.427 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef5901700 1 -- 172.21.15.104:0/582078815 learned_addr learned my addr 172.21.15.104:0/582078815 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:55.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef6102700 1 --2- 172.21.15.104:0/582078815 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbef01603d0 0x7fbef006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef5100700 1 --2- 172.21.15.104:0/582078815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbef0160000 0x7fbef0157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef5901700 1 -- 172.21.15.104:0/582078815 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbef01603d0 msgr2=0x7fbef006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef5901700 1 --2- 172.21.15.104:0/582078815 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbef01603d0 0x7fbef006c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef5901700 1 -- 172.21.15.104:0/582078815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbef0160000 msgr2=0x7fbef0157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.428 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef5901700 1 --2- 172.21.15.104:0/582078815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbef0160000 0x7fbef0157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.437+0000 7fbef5901700 1 -- 172.21.15.104:0/582078815 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee00094b0 con 0x7fbef00cd7f0 2023-07-14T11:55:55.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.441+0000 7fbef5901700 1 --2- 172.21.15.104:0/582078815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef00cd7f0 0x7fbef01611d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00c780 tx=0x7fbeec00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:55.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.441+0000 7fbee67fc700 1 -- 172.21.15.104:0/582078815 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec004ca0 con 0x7fbef00cd7f0 2023-07-14T11:55:55.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.441+0000 7fbee67fc700 1 -- 172.21.15.104:0/582078815 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec004e00 con 0x7fbef00cd7f0 2023-07-14T11:55:55.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.441+0000 7fbef6903700 1 -- 172.21.15.104:0/582078815 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef0161d70 con 0x7fbef00cd7f0 2023-07-14T11:55:55.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.441+0000 7fbee67fc700 1 -- 172.21.15.104:0/582078815 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec007310 con 0x7fbef00cd7f0 2023-07-14T11:55:55.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.441+0000 7fbef6903700 1 -- 172.21.15.104:0/582078815 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef00b3b70 con 0x7fbef00cd7f0 2023-07-14T11:55:55.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.441+0000 7fbef6903700 1 -- 172.21.15.104:0/582078815 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef00c52d0 con 0x7fbef00cd7f0 2023-07-14T11:55:55.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.441+0000 7fbee67fc700 1 -- 172.21.15.104:0/582078815 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fbeec007470 con 0x7fbef00cd7f0 2023-07-14T11:55:55.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.445+0000 7fbee67fc700 1 --2- 172.21.15.104:0/582078815 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbed40662f0 0x7fbed40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.445+0000 7fbee67fc700 1 -- 172.21.15.104:0/582078815 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6835+0+0 (secure 0 0 0) 0x7fbeec083ea0 con 0x7fbef00cd7f0 2023-07-14T11:55:55.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.445+0000 7fbef5100700 1 --2- 172.21.15.104:0/582078815 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbed40662f0 0x7fbed40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.445+0000 7fbef5100700 1 --2- 172.21.15.104:0/582078815 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbed40662f0 0x7fbed40687a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009800 tx=0x7fbee0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:55.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.449+0000 7fbee67fc700 1 -- 172.21.15.104:0/582078815 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fbeec050e60 con 0x7fbef00cd7f0 2023-07-14T11:55:55.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.625+0000 7fbef6903700 1 -- 172.21.15.104:0/582078815 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fbef0071240 con 0x7fbef00cd7f0 2023-07-14T11:55:55.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.625+0000 7fbee67fc700 1 -- 172.21.15.104:0/582078815 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v46) v1 ==== 97+0+11 (secure 0 0 0) 0x7fbef0071240 con 0x7fbef00cd7f0 2023-07-14T11:55:55.613 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:55:55.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.629+0000 7fbef6903700 1 -- 172.21.15.104:0/582078815 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbed40662f0 msgr2=0x7fbed40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.629+0000 7fbef6903700 1 --2- 172.21.15.104:0/582078815 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbed40662f0 0x7fbed40687a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009800 tx=0x7fbee0008030 comp rx=0 tx=0).stop 2023-07-14T11:55:55.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.629+0000 7fbef6903700 1 -- 172.21.15.104:0/582078815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef00cd7f0 msgr2=0x7fbef01611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.629+0000 7fbef6903700 1 --2- 172.21.15.104:0/582078815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef00cd7f0 0x7fbef01611d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00c780 tx=0x7fbeec00ca90 comp rx=0 tx=0).stop 2023-07-14T11:55:55.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.629+0000 7fbef6903700 1 -- 172.21.15.104:0/582078815 shutdown_connections 2023-07-14T11:55:55.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.629+0000 7fbef6903700 1 --2- 172.21.15.104:0/582078815 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbed40662f0 0x7fbed40687a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.629+0000 7fbef6903700 1 --2- 172.21.15.104:0/582078815 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbef01603d0 0x7fbef006c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.629+0000 7fbef6903700 1 --2- 172.21.15.104:0/582078815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbef0160000 0x7fbef0157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.629+0000 7fbef6903700 1 --2- 172.21.15.104:0/582078815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef00cd7f0 0x7fbef01611d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.629+0000 7fbef6903700 1 -- 172.21.15.104:0/582078815 >> 172.21.15.104:0/582078815 conn(0x7fbef00afc60 msgr2=0x7fbef00ce150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:55.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.629+0000 7fbef6903700 1 -- 172.21.15.104:0/582078815 shutdown_connections 2023-07-14T11:55:55.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.629+0000 7fbef6903700 1 -- 172.21.15.104:0/582078815 wait complete. 2023-07-14T11:55:55.635 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:55:55.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.817+0000 7f870c025700 1 Processor -- start 2023-07-14T11:55:55.806 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.817+0000 7f870c025700 1 -- start start 2023-07-14T11:55:55.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f870c025700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8704160640 0x7f8704164000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f870c025700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87040cddb0 0x7f87040afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f870c025700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87040c8900 con 0x7f8704164540 2023-07-14T11:55:55.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f870c025700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87040c8fc0 con 0x7f8704160640 2023-07-14T11:55:55.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f870c025700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8704070540 con 0x7f87040cddb0 2023-07-14T11:55:55.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f8709dc1700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8704160640 0x7f8704164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f8709dc1700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8704160640 0x7f8704164000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35314/0 (socket says 172.21.15.104:35314) 2023-07-14T11:55:55.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f8709dc1700 1 -- 172.21.15.104:0/885080278 learned_addr learned my addr 172.21.15.104:0/885080278 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:55.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f870a5c2700 1 --2- 172.21.15.104:0/885080278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87040cddb0 0x7f87040afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.810 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f8709dc1700 1 -- 172.21.15.104:0/885080278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87040cddb0 msgr2=0x7f87040afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:55.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f8709dc1700 1 --2- 172.21.15.104:0/885080278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87040cddb0 0x7f87040afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f8709dc1700 1 -- 172.21.15.104:0/885080278 >> v1:172.21.15.121:6789/0 conn(0x7f8704164540 legacy=0x7f8704070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f8709dc1700 1 -- 172.21.15.104:0/885080278 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87040caae0 con 0x7f8704160640 2023-07-14T11:55:55.811 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f8708dbf700 1 -- 172.21.15.104:0/885080278 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1463686871 0 0) 0x7f87040c8900 con 0x7f8704164540 2023-07-14T11:55:55.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f8709dc1700 1 --2- 172.21.15.104:0/885080278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8704160640 0x7f8704164000 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8700009e50 tx=0x7f870000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff96a979fa55cdc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:55.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.821+0000 7f8708dbf700 1 -- 172.21.15.104:0/885080278 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870000e070 con 0x7f8704160640 2023-07-14T11:55:55.812 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.825+0000 7f8708dbf700 1 -- 172.21.15.104:0/885080278 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870000b910 con 0x7f8704160640 2023-07-14T11:55:55.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.825+0000 7f8708dbf700 1 -- 172.21.15.104:0/885080278 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870000bcc0 con 0x7f8704160640 2023-07-14T11:55:55.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.825+0000 7f870c025700 1 -- 172.21.15.104:0/885080278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8704160640 msgr2=0x7f8704164000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.825+0000 7f870c025700 1 --2- 172.21.15.104:0/885080278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8704160640 0x7f8704164000 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8700009e50 tx=0x7f870000b330 comp rx=0 tx=0).stop 2023-07-14T11:55:55.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.825+0000 7f870c025700 1 -- 172.21.15.104:0/885080278 shutdown_connections 2023-07-14T11:55:55.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.825+0000 7f870c025700 1 --2- 172.21.15.104:0/885080278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87040cddb0 0x7f87040afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.825+0000 7f870c025700 1 --2- 172.21.15.104:0/885080278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8704160640 0x7f8704164000 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.825+0000 7f870c025700 1 -- 172.21.15.104:0/885080278 >> 172.21.15.104:0/885080278 conn(0x7f87040cad30 msgr2=0x7f87040cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:55.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.825+0000 7f870c025700 1 -- 172.21.15.104:0/885080278 shutdown_connections 2023-07-14T11:55:55.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.825+0000 7f870c025700 1 -- 172.21.15.104:0/885080278 wait complete. 2023-07-14T11:55:55.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f870c025700 1 Processor -- start 2023-07-14T11:55:55.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f870c025700 1 -- start start 2023-07-14T11:55:55.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f870c025700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87040cddb0 0x7f8704155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f870c025700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8704160640 0x7f8704154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f870c025700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8704164540 0x7f87041569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f870c025700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87040c5520 con 0x7f87040cddb0 2023-07-14T11:55:55.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f870c025700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87041559f0 con 0x7f8704160640 2023-07-14T11:55:55.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f870c025700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8704155cd0 con 0x7f8704164540 2023-07-14T11:55:55.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f8709dc1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87040cddb0 0x7f8704155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f8709dc1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87040cddb0 0x7f8704155160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35318/0 (socket says 172.21.15.104:35318) 2023-07-14T11:55:55.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f8709dc1700 1 -- 172.21.15.104:0/2299109655 learned_addr learned my addr 172.21.15.104:0/2299109655 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:55.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f87095c0700 1 --2- 172.21.15.104:0/2299109655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8704160640 0x7f8704154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f870a5c2700 1 --2- 172.21.15.104:0/2299109655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8704164540 0x7f87041569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f8709dc1700 1 -- 172.21.15.104:0/2299109655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8704164540 msgr2=0x7f87041569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f8709dc1700 1 --2- 172.21.15.104:0/2299109655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8704164540 0x7f87041569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f8709dc1700 1 -- 172.21.15.104:0/2299109655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8704160640 msgr2=0x7f8704154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:55.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f8709dc1700 1 --2- 172.21.15.104:0/2299109655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8704160640 0x7f8704154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:55.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f8709dc1700 1 -- 172.21.15.104:0/2299109655 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8700009b30 con 0x7f87040cddb0 2023-07-14T11:55:55.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.829+0000 7f8709dc1700 1 --2- 172.21.15.104:0/2299109655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87040cddb0 0x7f8704155160 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8704071920 tx=0x7f8700004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:55.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.833+0000 7f86fa7fc700 1 -- 172.21.15.104:0/2299109655 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87000099b0 con 0x7f87040cddb0 2023-07-14T11:55:55.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.833+0000 7f86fa7fc700 1 -- 172.21.15.104:0/2299109655 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87000092e0 con 0x7f87040cddb0 2023-07-14T11:55:55.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.833+0000 7f870c025700 1 -- 172.21.15.104:0/2299109655 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8704155fb0 con 0x7f87040cddb0 2023-07-14T11:55:55.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.833+0000 7f86fa7fc700 1 -- 172.21.15.104:0/2299109655 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870001c640 con 0x7f87040cddb0 2023-07-14T11:55:55.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.833+0000 7f870c025700 1 -- 172.21.15.104:0/2299109655 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87041556a0 con 0x7f87040cddb0 2023-07-14T11:55:55.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.833+0000 7f870c025700 1 -- 172.21.15.104:0/2299109655 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870404dcb0 con 0x7f87040cddb0 2023-07-14T11:55:55.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.833+0000 7f86fa7fc700 1 -- 172.21.15.104:0/2299109655 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f870001c7a0 con 0x7f87040cddb0 2023-07-14T11:55:55.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.837+0000 7f86fa7fc700 1 --2- 172.21.15.104:0/2299109655 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86e80662f0 0x7f86e80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:55.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.837+0000 7f86fa7fc700 1 -- 172.21.15.104:0/2299109655 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f8700085af0 con 0x7f87040cddb0 2023-07-14T11:55:55.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.837+0000 7f87095c0700 1 --2- 172.21.15.104:0/2299109655 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86e80662f0 0x7f86e80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:55.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.837+0000 7f87095c0700 1 --2- 172.21.15.104:0/2299109655 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86e80662f0 0x7f86e80687a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f870406da30 tx=0x7f86f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:55.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:55.841+0000 7f86fa7fc700 1 -- 172.21.15.104:0/2299109655 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f8700052c50 con 0x7f87040cddb0 2023-07-14T11:55:56.050 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.061+0000 7f870c025700 1 -- 172.21.15.104:0/2299109655 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87040cb9f0 con 0x7f87040cddb0 2023-07-14T11:55:56.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.065+0000 7f86fa7fc700 1 -- 172.21.15.104:0/2299109655 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f8700056270 con 0x7f87040cddb0 2023-07-14T11:55:56.053 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:56.053 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3696540262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3696540262},{"type":"v1","addr":"172.21.15.104:6835","nonce":3696540262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/684052037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":684052037},{"type":"v1","addr":"172.21.15.121:6835","nonce":684052037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/1267091306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":1267091306},{"type":"v1","addr":"172.21.15.121:6833","nonce":1267091306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:54:58.600122+0000","modified":"2023-07-14T11:55:09.633871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5616},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5616":{"gid":5616,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6837/398018673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":398018673},{"type":"v1","addr":"172.21.15.104:6837","nonce":398018673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-14T11:55:56.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.069+0000 7f870c025700 1 -- 172.21.15.104:0/2299109655 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86e80662f0 msgr2=0x7f86e80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:56.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.069+0000 7f870c025700 1 --2- 172.21.15.104:0/2299109655 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86e80662f0 0x7f86e80687a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f870406da30 tx=0x7f86f4009040 comp rx=0 tx=0).stop 2023-07-14T11:55:56.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.069+0000 7f870c025700 1 -- 172.21.15.104:0/2299109655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87040cddb0 msgr2=0x7f8704155160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:56.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.069+0000 7f870c025700 1 --2- 172.21.15.104:0/2299109655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87040cddb0 0x7f8704155160 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8704071920 tx=0x7f8700004af0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.069+0000 7f870c025700 1 -- 172.21.15.104:0/2299109655 shutdown_connections 2023-07-14T11:55:56.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.069+0000 7f870c025700 1 --2- 172.21.15.104:0/2299109655 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86e80662f0 0x7f86e80687a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.069+0000 7f870c025700 1 --2- 172.21.15.104:0/2299109655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8704164540 0x7f87041569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.069+0000 7f870c025700 1 --2- 172.21.15.104:0/2299109655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8704160640 0x7f8704154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.069+0000 7f870c025700 1 --2- 172.21.15.104:0/2299109655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87040cddb0 0x7f8704155160 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.069+0000 7f870c025700 1 -- 172.21.15.104:0/2299109655 >> 172.21.15.104:0/2299109655 conn(0x7f87040cad30 msgr2=0x7f87040cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:56.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.069+0000 7f870c025700 1 -- 172.21.15.104:0/2299109655 shutdown_connections 2023-07-14T11:55:56.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.069+0000 7f870c025700 1 -- 172.21.15.104:0/2299109655 wait complete. 2023-07-14T11:55:56.059 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 19 2023-07-14T11:55:56.078 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:55:56.252 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.265+0000 7f55ee769700 1 Processor -- start 2023-07-14T11:55:56.252 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.265+0000 7f55ee769700 1 -- start start 2023-07-14T11:55:56.253 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.265+0000 7f55ee769700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f55e8160640 0x7f55e8164000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:56.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.265+0000 7f55ee769700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55e8164540 0x7f55e8070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:56.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.265+0000 7f55ee769700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55e80cddb0 0x7f55e8070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:56.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.265+0000 7f55ee769700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e80c8fc0 con 0x7f55e8160640 2023-07-14T11:55:56.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.265+0000 7f55ee769700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e8070540 con 0x7f55e80cddb0 2023-07-14T11:55:56.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.265+0000 7f55ee769700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e80c8900 con 0x7f55e8164540 2023-07-14T11:55:56.255 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.265+0000 7f55e7fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f55e8160640 0x7f55e8164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:56.255 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.265+0000 7f55ecd06700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55e80cddb0 0x7f55e8070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:56.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.265+0000 7f55e7fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f55e8160640 0x7f55e8164000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35322/0 (socket says 172.21.15.104:35322) 2023-07-14T11:55:56.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.265+0000 7f55dffff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55e8164540 0x7f55e8070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:56.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.265+0000 7f55e7fff700 1 -- 172.21.15.104:0/797891987 learned_addr learned my addr 172.21.15.104:0/797891987 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:56.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55ecd06700 1 -- 172.21.15.104:0/797891987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55e8164540 msgr2=0x7f55e8070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:56.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55ecd06700 1 --2- 172.21.15.104:0/797891987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55e8164540 0x7f55e8070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55ecd06700 1 -- 172.21.15.104:0/797891987 >> v2:172.21.15.104:3300/0 conn(0x7f55e8160640 msgr2=0x7f55e8164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:56.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55ecd06700 1 --2- 172.21.15.104:0/797891987 >> v2:172.21.15.104:3300/0 conn(0x7f55e8160640 0x7f55e8164000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.257 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55ecd06700 1 -- 172.21.15.104:0/797891987 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e80caae0 con 0x7f55e80cddb0 2023-07-14T11:55:56.257 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55e7fff700 1 --2- 172.21.15.104:0/797891987 >> v2:172.21.15.104:3300/0 conn(0x7f55e8160640 0x7f55e8164000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:55:56.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55ecd06700 1 --2- 172.21.15.104:0/797891987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e80cddb0 0x7f55e8070ea0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f55e81650c0 tx=0x7f55e000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebb1b2e58feceabd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:56.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55e77fe700 1 -- 172.21.15.104:0/797891987 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e000e070 con 0x7f55e80cddb0 2023-07-14T11:55:56.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55e77fe700 1 -- 172.21.15.104:0/797891987 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e000bb20 con 0x7f55e80cddb0 2023-07-14T11:55:56.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55e77fe700 1 -- 172.21.15.104:0/797891987 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0012430 con 0x7f55e80cddb0 2023-07-14T11:55:56.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55ee769700 1 -- 172.21.15.104:0/797891987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e80cddb0 msgr2=0x7f55e8070ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:56.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55ee769700 1 --2- 172.21.15.104:0/797891987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e80cddb0 0x7f55e8070ea0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f55e81650c0 tx=0x7f55e000b330 comp rx=0 tx=0).stop 2023-07-14T11:55:56.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55ee769700 1 -- 172.21.15.104:0/797891987 shutdown_connections 2023-07-14T11:55:56.259 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55ee769700 1 --2- 172.21.15.104:0/797891987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e80cddb0 0x7f55e8070ea0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.259 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55ee769700 1 --2- 172.21.15.104:0/797891987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55e8164540 0x7f55e8070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.259 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55ee769700 1 --2- 172.21.15.104:0/797891987 >> v2:172.21.15.104:3300/0 conn(0x7f55e8160640 0x7f55e8164000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.259 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55ee769700 1 -- 172.21.15.104:0/797891987 >> 172.21.15.104:0/797891987 conn(0x7f55e80cad30 msgr2=0x7f55e80cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:56.259 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55ee769700 1 -- 172.21.15.104:0/797891987 shutdown_connections 2023-07-14T11:55:56.260 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.269+0000 7f55ee769700 1 -- 172.21.15.104:0/797891987 wait complete. 2023-07-14T11:55:56.260 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.273+0000 7f55ee769700 1 Processor -- start 2023-07-14T11:55:56.261 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.273+0000 7f55ee769700 1 -- start start 2023-07-14T11:55:56.262 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.273+0000 7f55ee769700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55e80cddb0 0x7f55e80b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:56.262 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.273+0000 7f55ee769700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e8160640 0x7f55e80b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:56.262 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.273+0000 7f55ee769700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55e8164540 0x7f55e80b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:56.262 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.273+0000 7f55ee769700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e80c5c90 con 0x7f55e80cddb0 2023-07-14T11:55:56.263 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.273+0000 7f55ee769700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e80b35e0 con 0x7f55e8160640 2023-07-14T11:55:56.263 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.273+0000 7f55ee769700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e8160d80 con 0x7f55e8164540 2023-07-14T11:55:56.263 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55e7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55e80cddb0 0x7f55e80b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:56.264 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55e7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55e80cddb0 0x7f55e80b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35328/0 (socket says 172.21.15.104:35328) 2023-07-14T11:55:56.264 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55e7fff700 1 -- 172.21.15.104:0/2487569782 learned_addr learned my addr 172.21.15.104:0/2487569782 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:56.264 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55dffff700 1 --2- 172.21.15.104:0/2487569782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e8160640 0x7f55e80b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:56.264 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55ecd06700 1 --2- 172.21.15.104:0/2487569782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55e8164540 0x7f55e80b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:56.264 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55e7fff700 1 -- 172.21.15.104:0/2487569782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55e8164540 msgr2=0x7f55e80b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:56.264 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55e7fff700 1 --2- 172.21.15.104:0/2487569782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55e8164540 0x7f55e80b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55e7fff700 1 -- 172.21.15.104:0/2487569782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e8160640 msgr2=0x7f55e80b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:56.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55e7fff700 1 --2- 172.21.15.104:0/2487569782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e8160640 0x7f55e80b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55e7fff700 1 -- 172.21.15.104:0/2487569782 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d8009570 con 0x7f55e80cddb0 2023-07-14T11:55:56.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55dffff700 1 --2- 172.21.15.104:0/2487569782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e8160640 0x7f55e80b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:56.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55e7fff700 1 --2- 172.21.15.104:0/2487569782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55e80cddb0 0x7f55e80b3860 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f55d800ebb0 tx=0x7f55d800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:56.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55e57fa700 1 -- 172.21.15.104:0/2487569782 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8004030 con 0x7f55e80cddb0 2023-07-14T11:55:56.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55e57fa700 1 -- 172.21.15.104:0/2487569782 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d8004190 con 0x7f55e80cddb0 2023-07-14T11:55:56.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55ee769700 1 -- 172.21.15.104:0/2487569782 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e0009a70 con 0x7f55e80cddb0 2023-07-14T11:55:56.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55e57fa700 1 -- 172.21.15.104:0/2487569782 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8004520 con 0x7f55e80cddb0 2023-07-14T11:55:56.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.277+0000 7f55ee769700 1 -- 172.21.15.104:0/2487569782 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e8161340 con 0x7f55e80cddb0 2023-07-14T11:55:56.267 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.281+0000 7f55ee769700 1 -- 172.21.15.104:0/2487569782 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e804dcb0 con 0x7f55e80cddb0 2023-07-14T11:55:56.268 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.281+0000 7f55e57fa700 1 -- 172.21.15.104:0/2487569782 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f55d800c850 con 0x7f55e80cddb0 2023-07-14T11:55:56.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.281+0000 7f55e57fa700 1 --2- 172.21.15.104:0/2487569782 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f55c80662f0 0x7f55c80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:56.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.281+0000 7f55e57fa700 1 -- 172.21.15.104:0/2487569782 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f55d8083ec0 con 0x7f55e80cddb0 2023-07-14T11:55:56.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.281+0000 7f55dffff700 1 --2- 172.21.15.104:0/2487569782 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f55c80662f0 0x7f55c80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:56.271 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.285+0000 7f55dffff700 1 --2- 172.21.15.104:0/2487569782 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f55c80662f0 0x7f55c80687a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f55e81f8aa0 tx=0x7f55d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:56.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.289+0000 7f55e57fa700 1 -- 172.21.15.104:0/2487569782 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f55d8051020 con 0x7f55e80cddb0 2023-07-14T11:55:56.447 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.461+0000 7f55ee769700 1 -- 172.21.15.104:0/2487569782 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55e80cb9f0 con 0x7f55e80cddb0 2023-07-14T11:55:56.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.465+0000 7f55e57fa700 1 -- 172.21.15.104:0/2487569782 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+20462 (secure 0 0 0) 0x7f55d8054640 con 0x7f55e80cddb0 2023-07-14T11:55:56.452 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:56.452 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":47,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:55:55.683019+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:54:56.370313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-14T11:54:57.608947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"data_new","create_time":"2023-07-14T11:55:08.099546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":5}]},{"pgid":"7.36","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:55:56.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.465+0000 7f55ee769700 1 -- 172.21.15.104:0/2487569782 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f55c80662f0 msgr2=0x7f55c80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:56.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.465+0000 7f55ee769700 1 --2- 172.21.15.104:0/2487569782 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f55c80662f0 0x7f55c80687a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f55e81f8aa0 tx=0x7f55d0008040 comp rx=0 tx=0).stop 2023-07-14T11:55:56.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.465+0000 7f55ee769700 1 -- 172.21.15.104:0/2487569782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55e80cddb0 msgr2=0x7f55e80b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:56.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.465+0000 7f55ee769700 1 --2- 172.21.15.104:0/2487569782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55e80cddb0 0x7f55e80b3860 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f55d800ebb0 tx=0x7f55d800eec0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.465+0000 7f55ee769700 1 -- 172.21.15.104:0/2487569782 shutdown_connections 2023-07-14T11:55:56.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.465+0000 7f55ee769700 1 --2- 172.21.15.104:0/2487569782 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f55c80662f0 0x7f55c80687a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.465+0000 7f55ee769700 1 --2- 172.21.15.104:0/2487569782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55e8164540 0x7f55e80b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.465+0000 7f55ee769700 1 --2- 172.21.15.104:0/2487569782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e8160640 0x7f55e80b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.465+0000 7f55ee769700 1 --2- 172.21.15.104:0/2487569782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55e80cddb0 0x7f55e80b3860 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.465+0000 7f55ee769700 1 -- 172.21.15.104:0/2487569782 >> 172.21.15.104:0/2487569782 conn(0x7f55e80cad30 msgr2=0x7f55e8070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:56.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.469+0000 7f55ee769700 1 -- 172.21.15.104:0/2487569782 shutdown_connections 2023-07-14T11:55:56.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.469+0000 7f55ee769700 1 -- 172.21.15.104:0/2487569782 wait complete. 2023-07-14T11:55:56.476 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-14T11:55:56.477 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:55:56.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.657+0000 7f6d5d871700 1 Processor -- start 2023-07-14T11:55:56.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.657+0000 7f6d5d871700 1 -- start start 2023-07-14T11:55:56.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.657+0000 7f6d5d871700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6d58159f40 0x7f6d5806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:56.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.657+0000 7f6d5d871700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5806f310 con 0x7f6d5806e230 2023-07-14T11:55:56.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.657+0000 7f6d5d871700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d580cba60 con 0x7f6d58159b70 2023-07-14T11:55:56.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.657+0000 7f6d5d871700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d580cbd40 con 0x7f6d58159f40 2023-07-14T11:55:56.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.661+0000 7f6d57fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f6d5806e230 0x7f6d5806f510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56196/0 (socket says 172.21.15.104:56196) 2023-07-14T11:55:56.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.661+0000 7f6d57fff700 1 -- 172.21.15.104:0/954125835 learned_addr learned my addr 172.21.15.104:0/954125835 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:56.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.661+0000 7f6d567fc700 1 --2- 172.21.15.104:0/954125835 >> v2:172.21.15.121:3300/0 conn(0x7f6d58159f40 0x7f6d5806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:56.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.661+0000 7f6d55ffb700 1 -- 172.21.15.104:0/954125835 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373455763 0 0) 0x7f6d5806f310 con 0x7f6d5806e230 2023-07-14T11:55:56.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.661+0000 7f6d55ffb700 1 -- 172.21.15.104:0/954125835 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d440035d0 con 0x7f6d5806e230 2023-07-14T11:55:56.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.661+0000 7f6d567fc700 1 -- 172.21.15.104:0/954125835 >> v1:172.21.15.121:6789/0 conn(0x7f6d58159b70 legacy=0x7f6d5806ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:56.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.661+0000 7f6d567fc700 1 -- 172.21.15.104:0/954125835 >> v1:172.21.15.104:6789/0 conn(0x7f6d5806e230 legacy=0x7f6d5806f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:56.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.661+0000 7f6d567fc700 1 -- 172.21.15.104:0/954125835 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d580cc020 con 0x7f6d58159f40 2023-07-14T11:55:56.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.661+0000 7f6d567fc700 1 --2- 172.21.15.104:0/954125835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d58159f40 0x7f6d5806dcf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009840 tx=0x7f6d40009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5112840868ebddb7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:56.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.661+0000 7f6d55ffb700 1 -- 172.21.15.104:0/954125835 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40005e50 con 0x7f6d58159f40 2023-07-14T11:55:56.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.661+0000 7f6d55ffb700 1 -- 172.21.15.104:0/954125835 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d40006030 con 0x7f6d58159f40 2023-07-14T11:55:56.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.661+0000 7f6d55ffb700 1 -- 172.21.15.104:0/954125835 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d400063c0 con 0x7f6d58159f40 2023-07-14T11:55:56.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.661+0000 7f6d5d871700 1 -- 172.21.15.104:0/954125835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d58159f40 msgr2=0x7f6d5806dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:56.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.661+0000 7f6d5d871700 1 --2- 172.21.15.104:0/954125835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d58159f40 0x7f6d5806dcf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009840 tx=0x7f6d40009b50 comp rx=0 tx=0).stop 2023-07-14T11:55:56.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.665+0000 7f6d5d871700 1 -- 172.21.15.104:0/954125835 shutdown_connections 2023-07-14T11:55:56.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.665+0000 7f6d5d871700 1 --2- 172.21.15.104:0/954125835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d58159f40 0x7f6d5806dcf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.665+0000 7f6d5d871700 1 -- 172.21.15.104:0/954125835 >> 172.21.15.104:0/954125835 conn(0x7f6d580cc500 msgr2=0x7f6d580cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:56.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.665+0000 7f6d5d871700 1 -- 172.21.15.104:0/954125835 shutdown_connections 2023-07-14T11:55:56.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.665+0000 7f6d5d871700 1 -- 172.21.15.104:0/954125835 wait complete. 2023-07-14T11:55:56.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.665+0000 7f6d5d871700 1 Processor -- start 2023-07-14T11:55:56.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.665+0000 7f6d5d871700 1 -- start start 2023-07-14T11:55:56.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d5d871700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5806e230 0x7f6d580c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:56.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d5d871700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d58159b70 0x7f6d581f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:56.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d5d871700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d58159f40 0x7f6d580b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:56.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d5d871700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d580c6fa0 con 0x7f6d5806e230 2023-07-14T11:55:56.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d5d871700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d581f4d10 con 0x7f6d58159f40 2023-07-14T11:55:56.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d5d871700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d581f4fb0 con 0x7f6d58159b70 2023-07-14T11:55:56.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d56ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5806e230 0x7f6d580c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:56.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d56ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5806e230 0x7f6d580c9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35344/0 (socket says 172.21.15.104:35344) 2023-07-14T11:55:56.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d56ffd700 1 -- 172.21.15.104:0/410158043 learned_addr learned my addr 172.21.15.104:0/410158043 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:56.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d57fff700 1 --2- 172.21.15.104:0/410158043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d58159f40 0x7f6d580b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:56.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d567fc700 1 --2- 172.21.15.104:0/410158043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d58159b70 0x7f6d581f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:56.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d56ffd700 1 -- 172.21.15.104:0/410158043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d58159b70 msgr2=0x7f6d581f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:56.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d56ffd700 1 --2- 172.21.15.104:0/410158043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d58159b70 0x7f6d581f47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d56ffd700 1 -- 172.21.15.104:0/410158043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d58159f40 msgr2=0x7f6d580b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:56.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d56ffd700 1 --2- 172.21.15.104:0/410158043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d58159f40 0x7f6d580b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d56ffd700 1 -- 172.21.15.104:0/410158043 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d40009450 con 0x7f6d5806e230 2023-07-14T11:55:56.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d567fc700 1 --2- 172.21.15.104:0/410158043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d58159b70 0x7f6d581f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:56.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d56ffd700 1 --2- 172.21.15.104:0/410158043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5806e230 0x7f6d580c9600 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800c500 tx=0x7f6d4800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:56.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d4ffff700 1 -- 172.21.15.104:0/410158043 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d480052d0 con 0x7f6d5806e230 2023-07-14T11:55:56.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d4ffff700 1 -- 172.21.15.104:0/410158043 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d48005430 con 0x7f6d5806e230 2023-07-14T11:55:56.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d4ffff700 1 -- 172.21.15.104:0/410158043 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d480055a0 con 0x7f6d5806e230 2023-07-14T11:55:56.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d5d871700 1 -- 172.21.15.104:0/410158043 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d581f5290 con 0x7f6d5806e230 2023-07-14T11:55:56.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.669+0000 7f6d5d871700 1 -- 172.21.15.104:0/410158043 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d580c71b0 con 0x7f6d5806e230 2023-07-14T11:55:56.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.673+0000 7f6d5d871700 1 -- 172.21.15.104:0/410158043 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d580c73d0 con 0x7f6d5806e230 2023-07-14T11:55:56.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.673+0000 7f6d4ffff700 1 -- 172.21.15.104:0/410158043 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6d480038e0 con 0x7f6d5806e230 2023-07-14T11:55:56.663 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.677+0000 7f6d4ffff700 1 --2- 172.21.15.104:0/410158043 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6d380662f0 0x7f6d380687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:56.664 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.677+0000 7f6d4ffff700 1 -- 172.21.15.104:0/410158043 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f6d48083480 con 0x7f6d5806e230 2023-07-14T11:55:56.664 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.677+0000 7f6d567fc700 1 --2- 172.21.15.104:0/410158043 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6d380662f0 0x7f6d380687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:56.664 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.677+0000 7f6d567fc700 1 --2- 172.21.15.104:0/410158043 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6d380662f0 0x7f6d380687a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009840 tx=0x7f6d40008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:56.669 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.681+0000 7f6d4ffff700 1 -- 172.21.15.104:0/410158043 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f6d480505e0 con 0x7f6d5806e230 2023-07-14T11:55:56.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.885+0000 7f6d5d871700 1 -- 172.21.15.104:0/410158043 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d5804dcb0 con 0x7f6d5806e230 2023-07-14T11:55:56.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.885+0000 7f6d4ffff700 1 -- 172.21.15.104:0/410158043 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f6d48053c00 con 0x7f6d5806e230 2023-07-14T11:55:56.873 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:56.873 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3696540262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3696540262},{"type":"v1","addr":"172.21.15.104:6835","nonce":3696540262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/684052037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":684052037},{"type":"v1","addr":"172.21.15.121:6835","nonce":684052037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/1267091306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":1267091306},{"type":"v1","addr":"172.21.15.121:6833","nonce":1267091306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:54:58.600122+0000","modified":"2023-07-14T11:55:09.633871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5616},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5616":{"gid":5616,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6837/398018673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":398018673},{"type":"v1","addr":"172.21.15.104:6837","nonce":398018673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-14T11:55:56.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.889+0000 7f6d5d871700 1 -- 172.21.15.104:0/410158043 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6d380662f0 msgr2=0x7f6d380687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:56.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.889+0000 7f6d5d871700 1 --2- 172.21.15.104:0/410158043 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6d380662f0 0x7f6d380687a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009840 tx=0x7f6d40008070 comp rx=0 tx=0).stop 2023-07-14T11:55:56.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.889+0000 7f6d5d871700 1 -- 172.21.15.104:0/410158043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5806e230 msgr2=0x7f6d580c9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:56.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.889+0000 7f6d5d871700 1 --2- 172.21.15.104:0/410158043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5806e230 0x7f6d580c9600 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800c500 tx=0x7f6d4800c810 comp rx=0 tx=0).stop 2023-07-14T11:55:56.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.889+0000 7f6d5d871700 1 -- 172.21.15.104:0/410158043 shutdown_connections 2023-07-14T11:55:56.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.889+0000 7f6d5d871700 1 --2- 172.21.15.104:0/410158043 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6d380662f0 0x7f6d380687a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.889+0000 7f6d5d871700 1 --2- 172.21.15.104:0/410158043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d58159f40 0x7f6d580b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.889+0000 7f6d5d871700 1 --2- 172.21.15.104:0/410158043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d58159b70 0x7f6d581f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.889+0000 7f6d5d871700 1 --2- 172.21.15.104:0/410158043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5806e230 0x7f6d580c9600 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:56.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.889+0000 7f6d5d871700 1 -- 172.21.15.104:0/410158043 >> 172.21.15.104:0/410158043 conn(0x7f6d580cc500 msgr2=0x7f6d580cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:56.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.889+0000 7f6d5d871700 1 -- 172.21.15.104:0/410158043 shutdown_connections 2023-07-14T11:55:56.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:56.889+0000 7f6d5d871700 1 -- 172.21.15.104:0/410158043 wait complete. 2023-07-14T11:55:56.878 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 19 2023-07-14T11:55:56.895 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:55:57.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.085+0000 7f9321af2700 1 Processor -- start 2023-07-14T11:55:57.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.085+0000 7f9321af2700 1 -- start start 2023-07-14T11:55:57.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.089+0000 7f9321af2700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f931c071000 0x7f931c15ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.089+0000 7f9321af2700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c15d230 con 0x7f931c1599d0 2023-07-14T11:55:57.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.089+0000 7f9321af2700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c0cde10 con 0x7f931c0caa20 2023-07-14T11:55:57.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.089+0000 7f9321af2700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c0cd520 con 0x7f931c071000 2023-07-14T11:55:57.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.089+0000 7f9313fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f931c071000 0x7f931c15ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.089+0000 7f9320af0700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f931c1599d0 0x7f931c0c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56202/0 (socket says 172.21.15.104:56202) 2023-07-14T11:55:57.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.089+0000 7f9320af0700 1 -- 172.21.15.104:0/1901651906 learned_addr learned my addr 172.21.15.104:0/1901651906 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:57.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.089+0000 7f9313fff700 1 -- 172.21.15.104:0/1901651906 >> v1:172.21.15.121:6789/0 conn(0x7f931c0caa20 legacy=0x7f931c0cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.089+0000 7f9313fff700 1 -- 172.21.15.104:0/1901651906 >> v1:172.21.15.104:6789/0 conn(0x7f931c1599d0 legacy=0x7f931c0c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.089+0000 7f9313fff700 1 -- 172.21.15.104:0/1901651906 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c0715e0 con 0x7f931c071000 2023-07-14T11:55:57.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.089+0000 7f9313fff700 1 --2- 172.21.15.104:0/1901651906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f931c071000 0x7f931c15ccf0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f930c009850 tx=0x7f930c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4a0613cc8355b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:57.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.089+0000 7f93137fe700 1 -- 172.21.15.104:0/1901651906 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c011830 con 0x7f931c071000 2023-07-14T11:55:57.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.089+0000 7f93137fe700 1 -- 172.21.15.104:0/1901651906 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c011990 con 0x7f931c071000 2023-07-14T11:55:57.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.089+0000 7f93137fe700 1 -- 172.21.15.104:0/1901651906 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c011d20 con 0x7f931c071000 2023-07-14T11:55:57.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.093+0000 7f9321af2700 1 -- 172.21.15.104:0/1901651906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f931c071000 msgr2=0x7f931c15ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.093+0000 7f9321af2700 1 --2- 172.21.15.104:0/1901651906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f931c071000 0x7f931c15ccf0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f930c009850 tx=0x7f930c009b60 comp rx=0 tx=0).stop 2023-07-14T11:55:57.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.093+0000 7f9321af2700 1 -- 172.21.15.104:0/1901651906 shutdown_connections 2023-07-14T11:55:57.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.093+0000 7f9321af2700 1 --2- 172.21.15.104:0/1901651906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f931c071000 0x7f931c15ccf0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.093+0000 7f9321af2700 1 -- 172.21.15.104:0/1901651906 >> 172.21.15.104:0/1901651906 conn(0x7f931c06e960 msgr2=0x7f931c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:57.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.093+0000 7f9321af2700 1 -- 172.21.15.104:0/1901651906 shutdown_connections 2023-07-14T11:55:57.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.093+0000 7f9321af2700 1 -- 172.21.15.104:0/1901651906 wait complete. 2023-07-14T11:55:57.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9321af2700 1 Processor -- start 2023-07-14T11:55:57.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9321af2700 1 -- start start 2023-07-14T11:55:57.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9321af2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f931c071000 0x7f931c0c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9321af2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f931c0caa20 0x7f931c1513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9321af2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f931c1599d0 0x7f931c0acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9321af2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0aeff0 con 0x7f931c071000 2023-07-14T11:55:57.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9321af2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0add00 con 0x7f931c0caa20 2023-07-14T11:55:57.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9321af2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0adf50 con 0x7f931c1599d0 2023-07-14T11:55:57.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9320af0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f931c071000 0x7f931c0c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9320af0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f931c071000 0x7f931c0c29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35358/0 (socket says 172.21.15.104:35358) 2023-07-14T11:55:57.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9320af0700 1 -- 172.21.15.104:0/1866406119 learned_addr learned my addr 172.21.15.104:0/1866406119 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:57.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f93212f1700 1 --2- 172.21.15.104:0/1866406119 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f931c1599d0 0x7f931c0acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9313fff700 1 --2- 172.21.15.104:0/1866406119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f931c0caa20 0x7f931c1513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9320af0700 1 -- 172.21.15.104:0/1866406119 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f931c1599d0 msgr2=0x7f931c0acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9320af0700 1 --2- 172.21.15.104:0/1866406119 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f931c1599d0 0x7f931c0acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9320af0700 1 -- 172.21.15.104:0/1866406119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f931c0caa20 msgr2=0x7f931c1513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9320af0700 1 --2- 172.21.15.104:0/1866406119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f931c0caa20 0x7f931c1513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9320af0700 1 -- 172.21.15.104:0/1866406119 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930c0094b0 con 0x7f931c071000 2023-07-14T11:55:57.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f93212f1700 1 --2- 172.21.15.104:0/1866406119 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f931c1599d0 0x7f931c0acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:57.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.097+0000 7f9320af0700 1 --2- 172.21.15.104:0/1866406119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f931c071000 0x7f931c0c29b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f931800c4d0 tx=0x7f931800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:57.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.101+0000 7f93117fa700 1 -- 172.21.15.104:0/1866406119 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318005d90 con 0x7f931c071000 2023-07-14T11:55:57.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.101+0000 7f9321af2700 1 -- 172.21.15.104:0/1866406119 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c0ae230 con 0x7f931c071000 2023-07-14T11:55:57.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.101+0000 7f93117fa700 1 -- 172.21.15.104:0/1866406119 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93180070d0 con 0x7f931c071000 2023-07-14T11:55:57.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.101+0000 7f93117fa700 1 -- 172.21.15.104:0/1866406119 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318007440 con 0x7f931c071000 2023-07-14T11:55:57.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.101+0000 7f9321af2700 1 -- 172.21.15.104:0/1866406119 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c0ae7f0 con 0x7f931c071000 2023-07-14T11:55:57.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.101+0000 7f9321af2700 1 -- 172.21.15.104:0/1866406119 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931c04e500 con 0x7f931c071000 2023-07-14T11:55:57.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.101+0000 7f93117fa700 1 -- 172.21.15.104:0/1866406119 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f93180075a0 con 0x7f931c071000 2023-07-14T11:55:57.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.101+0000 7f93117fa700 1 --2- 172.21.15.104:0/1866406119 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f93080662f0 0x7f93080687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.105+0000 7f93117fa700 1 -- 172.21.15.104:0/1866406119 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f9318082ff0 con 0x7f931c071000 2023-07-14T11:55:57.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.105+0000 7f9313fff700 1 --2- 172.21.15.104:0/1866406119 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f93080662f0 0x7f93080687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.105+0000 7f9313fff700 1 --2- 172.21.15.104:0/1866406119 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f93080662f0 0x7f93080687a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f930c009fd0 tx=0x7f930c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:57.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.109+0000 7f93117fa700 1 -- 172.21.15.104:0/1866406119 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f93180501e0 con 0x7f931c071000 2023-07-14T11:55:57.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.313+0000 7f9321af2700 1 -- 172.21.15.104:0/1866406119 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f931c070de0 con 0x7f931c071000 2023-07-14T11:55:57.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.313+0000 7f93117fa700 1 -- 172.21.15.104:0/1866406119 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f9318053800 con 0x7f931c071000 2023-07-14T11:55:57.302 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:57.302 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3696540262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3696540262},{"type":"v1","addr":"172.21.15.104:6835","nonce":3696540262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/684052037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":684052037},{"type":"v1","addr":"172.21.15.121:6835","nonce":684052037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/1267091306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":1267091306},{"type":"v1","addr":"172.21.15.121:6833","nonce":1267091306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:54:58.600122+0000","modified":"2023-07-14T11:55:09.633871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5616},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5616":{"gid":5616,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6837/398018673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":398018673},{"type":"v1","addr":"172.21.15.104:6837","nonce":398018673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-14T11:55:57.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.317+0000 7f9321af2700 1 -- 172.21.15.104:0/1866406119 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f93080662f0 msgr2=0x7f93080687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.317+0000 7f9321af2700 1 --2- 172.21.15.104:0/1866406119 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f93080662f0 0x7f93080687a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f930c009fd0 tx=0x7f930c008030 comp rx=0 tx=0).stop 2023-07-14T11:55:57.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.317+0000 7f9321af2700 1 -- 172.21.15.104:0/1866406119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f931c071000 msgr2=0x7f931c0c29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.317+0000 7f9321af2700 1 --2- 172.21.15.104:0/1866406119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f931c071000 0x7f931c0c29b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f931800c4d0 tx=0x7f931800c890 comp rx=0 tx=0).stop 2023-07-14T11:55:57.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.321+0000 7f9321af2700 1 -- 172.21.15.104:0/1866406119 shutdown_connections 2023-07-14T11:55:57.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.321+0000 7f9321af2700 1 --2- 172.21.15.104:0/1866406119 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f93080662f0 0x7f93080687a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.321+0000 7f9321af2700 1 --2- 172.21.15.104:0/1866406119 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f931c1599d0 0x7f931c0acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.321+0000 7f9321af2700 1 --2- 172.21.15.104:0/1866406119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f931c0caa20 0x7f931c1513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.321+0000 7f9321af2700 1 --2- 172.21.15.104:0/1866406119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f931c071000 0x7f931c0c29b0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.321+0000 7f9321af2700 1 -- 172.21.15.104:0/1866406119 >> 172.21.15.104:0/1866406119 conn(0x7f931c06e960 msgr2=0x7f931c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:57.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.321+0000 7f9321af2700 1 -- 172.21.15.104:0/1866406119 shutdown_connections 2023-07-14T11:55:57.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.321+0000 7f9321af2700 1 -- 172.21.15.104:0/1866406119 wait complete. 2023-07-14T11:55:57.310 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 19 2023-07-14T11:55:57.332 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:55:57.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.513+0000 7f38fa003700 1 Processor -- start 2023-07-14T11:55:57.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.513+0000 7f38fa003700 1 -- start start 2023-07-14T11:55:57.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.517+0000 7f38fa003700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f38f415ffa0 0x7f38f406d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.517+0000 7f38fa003700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f38f406dc50 0x7f38f40caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.517+0000 7f38fa003700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f40ca8a0 con 0x7f38f4160370 2023-07-14T11:55:57.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.517+0000 7f38fa003700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f40cbcd0 con 0x7f38f415ffa0 2023-07-14T11:55:57.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.517+0000 7f38fa003700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40cbf70 con 0x7f38f406dc50 2023-07-14T11:55:57.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.517+0000 7f38f8800700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f38f406dc50 0x7f38f40caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.517+0000 7f38f8800700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f38f406dc50 0x7f38f40caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:48010/0 (socket says 172.21.15.104:48010) 2023-07-14T11:55:57.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.517+0000 7f38f8800700 1 -- 172.21.15.104:0/714049720 learned_addr learned my addr 172.21.15.104:0/714049720 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:57.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.517+0000 7f38f9001700 1 --2- 172.21.15.104:0/714049720 >> v2:172.21.15.121:3300/0 conn(0x7f38f415ffa0 0x7f38f406d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.517+0000 7f38f8800700 1 -- 172.21.15.104:0/714049720 >> v2:172.21.15.121:3300/0 conn(0x7f38f415ffa0 msgr2=0x7f38f406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:57.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.517+0000 7f38f8800700 1 --2- 172.21.15.104:0/714049720 >> v2:172.21.15.121:3300/0 conn(0x7f38f415ffa0 0x7f38f406d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.517+0000 7f38f8800700 1 -- 172.21.15.104:0/714049720 >> v1:172.21.15.121:6789/0 conn(0x7f38f4160370 legacy=0x7f38f40ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.517+0000 7f38f8800700 1 -- 172.21.15.104:0/714049720 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f40cc250 con 0x7f38f406dc50 2023-07-14T11:55:57.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.521+0000 7f38f8800700 1 --2- 172.21.15.104:0/714049720 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f38f406dc50 0x7f38f40caa70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009910 tx=0x7f38e4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69fc21b928251f6f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:57.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.521+0000 7f38ebfff700 1 -- 172.21.15.104:0/714049720 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4011830 con 0x7f38f406dc50 2023-07-14T11:55:57.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.521+0000 7f38ebfff700 1 -- 172.21.15.104:0/714049720 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e4011990 con 0x7f38f406dc50 2023-07-14T11:55:57.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.521+0000 7f38ebfff700 1 -- 172.21.15.104:0/714049720 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4011d40 con 0x7f38f406dc50 2023-07-14T11:55:57.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.521+0000 7f38fa003700 1 -- 172.21.15.104:0/714049720 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f38f406dc50 msgr2=0x7f38f40caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.521+0000 7f38fa003700 1 --2- 172.21.15.104:0/714049720 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f38f406dc50 0x7f38f40caa70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009910 tx=0x7f38e4009c20 comp rx=0 tx=0).stop 2023-07-14T11:55:57.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.521+0000 7f38fa003700 1 -- 172.21.15.104:0/714049720 shutdown_connections 2023-07-14T11:55:57.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.521+0000 7f38fa003700 1 --2- 172.21.15.104:0/714049720 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f38f406dc50 0x7f38f40caa70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.521+0000 7f38fa003700 1 --2- 172.21.15.104:0/714049720 >> v2:172.21.15.121:3300/0 conn(0x7f38f415ffa0 0x7f38f406d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.521+0000 7f38fa003700 1 -- 172.21.15.104:0/714049720 >> 172.21.15.104:0/714049720 conn(0x7f38f406e960 msgr2=0x7f38f406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:57.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.521+0000 7f38fa003700 1 -- 172.21.15.104:0/714049720 shutdown_connections 2023-07-14T11:55:57.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.521+0000 7f38fa003700 1 -- 172.21.15.104:0/714049720 wait complete. 2023-07-14T11:55:57.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38fa003700 1 Processor -- start 2023-07-14T11:55:57.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38fa003700 1 -- start start 2023-07-14T11:55:57.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38fa003700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38f406dc50 0x7f38f40c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38fa003700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f38f415ffa0 0x7f38f40c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38fa003700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38f4160370 0x7f38f415d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38fa003700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40c5cc0 con 0x7f38f4160370 2023-07-14T11:55:57.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38fa003700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40bd090 con 0x7f38f406dc50 2023-07-14T11:55:57.514 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38fa003700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40c5f90 con 0x7f38f415ffa0 2023-07-14T11:55:57.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38f9802700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38f4160370 0x7f38f415d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38f9802700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38f4160370 0x7f38f415d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35364/0 (socket says 172.21.15.104:35364) 2023-07-14T11:55:57.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38f9802700 1 -- 172.21.15.104:0/4173474187 learned_addr learned my addr 172.21.15.104:0/4173474187 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:57.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38f8800700 1 --2- 172.21.15.104:0/4173474187 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f38f415ffa0 0x7f38f40c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38f9001700 1 --2- 172.21.15.104:0/4173474187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38f406dc50 0x7f38f40c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38f9802700 1 -- 172.21.15.104:0/4173474187 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f38f415ffa0 msgr2=0x7f38f40c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38f9802700 1 --2- 172.21.15.104:0/4173474187 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f38f415ffa0 0x7f38f40c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38f9802700 1 -- 172.21.15.104:0/4173474187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38f406dc50 msgr2=0x7f38f40c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38f9802700 1 --2- 172.21.15.104:0/4173474187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38f406dc50 0x7f38f40c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38f9802700 1 -- 172.21.15.104:0/4173474187 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e4009570 con 0x7f38f4160370 2023-07-14T11:55:57.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.525+0000 7f38f8800700 1 --2- 172.21.15.104:0/4173474187 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f38f415ffa0 0x7f38f40c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:57.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.529+0000 7f38f9802700 1 --2- 172.21.15.104:0/4173474187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38f4160370 0x7f38f415d9f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00cb50 tx=0x7f38ec00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:57.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.529+0000 7f38e9ffb700 1 -- 172.21.15.104:0/4173474187 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec003c20 con 0x7f38f4160370 2023-07-14T11:55:57.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.529+0000 7f38e9ffb700 1 -- 172.21.15.104:0/4173474187 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec003d80 con 0x7f38f4160370 2023-07-14T11:55:57.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.529+0000 7f38fa003700 1 -- 172.21.15.104:0/4173474187 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f40c6270 con 0x7f38f4160370 2023-07-14T11:55:57.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.529+0000 7f38e9ffb700 1 -- 172.21.15.104:0/4173474187 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec00ab70 con 0x7f38f4160370 2023-07-14T11:55:57.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.529+0000 7f38fa003700 1 -- 172.21.15.104:0/4173474187 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f415df30 con 0x7f38f4160370 2023-07-14T11:55:57.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.529+0000 7f38fa003700 1 -- 172.21.15.104:0/4173474187 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f40c6790 con 0x7f38f4160370 2023-07-14T11:55:57.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.529+0000 7f38e9ffb700 1 -- 172.21.15.104:0/4173474187 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f38ec00acd0 con 0x7f38f4160370 2023-07-14T11:55:57.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.533+0000 7f38e9ffb700 1 --2- 172.21.15.104:0/4173474187 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f38d80662f0 0x7f38d80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.533+0000 7f38e9ffb700 1 -- 172.21.15.104:0/4173474187 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f38ec083e50 con 0x7f38f4160370 2023-07-14T11:55:57.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.533+0000 7f38f9001700 1 --2- 172.21.15.104:0/4173474187 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f38d80662f0 0x7f38d80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.521 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.533+0000 7f38f9001700 1 --2- 172.21.15.104:0/4173474187 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f38d80662f0 0x7f38d80687a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009ae0 tx=0x7f38f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:57.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.537+0000 7f38e9ffb700 1 -- 172.21.15.104:0/4173474187 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f38f40c6790 con 0x7f38f4160370 2023-07-14T11:55:57.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.713+0000 7f38fa003700 1 -- 172.21.15.104:0/4173474187 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38f40ca8a0 con 0x7f38f4160370 2023-07-14T11:55:57.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.717+0000 7f38e9ffb700 1 -- 172.21.15.104:0/4173474187 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+20462 (secure 0 0 0) 0x7f38f40ca8a0 con 0x7f38f4160370 2023-07-14T11:55:57.705 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:55:57.705 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":47,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:55:55.683019+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:54:56.370313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-14T11:54:57.608947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"data_new","create_time":"2023-07-14T11:55:08.099546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":5}]},{"pgid":"7.36","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:55:57.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.721+0000 7f38fa003700 1 -- 172.21.15.104:0/4173474187 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f38d80662f0 msgr2=0x7f38d80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.721+0000 7f38fa003700 1 --2- 172.21.15.104:0/4173474187 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f38d80662f0 0x7f38d80687a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009ae0 tx=0x7f38f0008040 comp rx=0 tx=0).stop 2023-07-14T11:55:57.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.721+0000 7f38fa003700 1 -- 172.21.15.104:0/4173474187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38f4160370 msgr2=0x7f38f415d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.721+0000 7f38fa003700 1 --2- 172.21.15.104:0/4173474187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38f4160370 0x7f38f415d9f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00cb50 tx=0x7f38ec00cf10 comp rx=0 tx=0).stop 2023-07-14T11:55:57.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.721+0000 7f38fa003700 1 -- 172.21.15.104:0/4173474187 shutdown_connections 2023-07-14T11:55:57.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.721+0000 7f38fa003700 1 --2- 172.21.15.104:0/4173474187 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f38d80662f0 0x7f38d80687a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.721+0000 7f38fa003700 1 --2- 172.21.15.104:0/4173474187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38f4160370 0x7f38f415d9f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.721+0000 7f38fa003700 1 --2- 172.21.15.104:0/4173474187 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f38f415ffa0 0x7f38f40c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.721+0000 7f38fa003700 1 --2- 172.21.15.104:0/4173474187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38f406dc50 0x7f38f40c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.721+0000 7f38fa003700 1 -- 172.21.15.104:0/4173474187 >> 172.21.15.104:0/4173474187 conn(0x7f38f406e960 msgr2=0x7f38f40b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:57.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.721+0000 7f38fa003700 1 -- 172.21.15.104:0/4173474187 shutdown_connections 2023-07-14T11:55:57.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.721+0000 7f38fa003700 1 -- 172.21.15.104:0/4173474187 wait complete. 2023-07-14T11:55:57.728 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-14T11:55:57.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.913+0000 7fb6e92a2700 1 Processor -- start 2023-07-14T11:55:57.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.913+0000 7fb6e92a2700 1 -- start start 2023-07-14T11:55:57.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.913+0000 7fb6e92a2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb6e4162b10 0x7fb6e4164ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.913+0000 7fb6e92a2700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e40b0ef0 con 0x7fb6e4159f20 2023-07-14T11:55:57.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.913+0000 7fb6e92a2700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e40ce9c0 con 0x7fb6e40c9d90 2023-07-14T11:55:57.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.913+0000 7fb6e92a2700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e40cee50 con 0x7fb6e4162b10 2023-07-14T11:55:57.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.913+0000 7fb6e27fc700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb6e4159f20 0x7fb6e40caa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56204/0 (socket says 172.21.15.104:56204) 2023-07-14T11:55:57.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.913+0000 7fb6e27fc700 1 -- 172.21.15.104:0/1258654408 learned_addr learned my addr 172.21.15.104:0/1258654408 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:57.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e37fe700 1 --2- 172.21.15.104:0/1258654408 >> v2:172.21.15.121:3300/0 conn(0x7fb6e4162b10 0x7fb6e4164ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e1ffb700 1 -- 172.21.15.104:0/1258654408 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039866405 0 0) 0x7fb6e40b0ef0 con 0x7fb6e4159f20 2023-07-14T11:55:57.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e1ffb700 1 -- 172.21.15.104:0/1258654408 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6cc0035d0 con 0x7fb6e4159f20 2023-07-14T11:55:57.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e37fe700 1 -- 172.21.15.104:0/1258654408 >> v1:172.21.15.121:6789/0 conn(0x7fb6e40c9d90 legacy=0x7fb6e40ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e37fe700 1 -- 172.21.15.104:0/1258654408 >> v1:172.21.15.104:6789/0 conn(0x7fb6e4159f20 legacy=0x7fb6e40caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e37fe700 1 -- 172.21.15.104:0/1258654408 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e40cb4f0 con 0x7fb6e4162b10 2023-07-14T11:55:57.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e37fe700 1 --2- 172.21.15.104:0/1258654408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6e4162b10 0x7fb6e4164ef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009d90 tx=0x7fb6dc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e0024b1ded263f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:57.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e1ffb700 1 -- 172.21.15.104:0/1258654408 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00e070 con 0x7fb6e4162b10 2023-07-14T11:55:57.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e1ffb700 1 -- 172.21.15.104:0/1258654408 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc00bb20 con 0x7fb6e4162b10 2023-07-14T11:55:57.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e1ffb700 1 -- 172.21.15.104:0/1258654408 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc012430 con 0x7fb6e4162b10 2023-07-14T11:55:57.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e92a2700 1 -- 172.21.15.104:0/1258654408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6e4162b10 msgr2=0x7fb6e4164ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e92a2700 1 --2- 172.21.15.104:0/1258654408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6e4162b10 0x7fb6e4164ef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009d90 tx=0x7fb6dc00b330 comp rx=0 tx=0).stop 2023-07-14T11:55:57.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e92a2700 1 -- 172.21.15.104:0/1258654408 shutdown_connections 2023-07-14T11:55:57.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e92a2700 1 --2- 172.21.15.104:0/1258654408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6e4162b10 0x7fb6e4164ef0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e92a2700 1 -- 172.21.15.104:0/1258654408 >> 172.21.15.104:0/1258654408 conn(0x7fb6e406e960 msgr2=0x7fb6e406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:57.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e92a2700 1 -- 172.21.15.104:0/1258654408 shutdown_connections 2023-07-14T11:55:57.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.917+0000 7fb6e92a2700 1 -- 172.21.15.104:0/1258654408 wait complete. 2023-07-14T11:55:57.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.921+0000 7fb6e92a2700 1 Processor -- start 2023-07-14T11:55:57.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.921+0000 7fb6e92a2700 1 -- start start 2023-07-14T11:55:57.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.921+0000 7fb6e92a2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6e40c9d90 0x7fb6e415c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.921+0000 7fb6e92a2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6e4159f20 0x7fb6e413f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.921+0000 7fb6e92a2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6e4162b10 0x7fb6e40b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.921+0000 7fb6e92a2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e415b750 con 0x7fb6e4162b10 2023-07-14T11:55:57.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.921+0000 7fb6e92a2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e40b1b30 con 0x7fb6e40c9d90 2023-07-14T11:55:57.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.921+0000 7fb6e92a2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e415b470 con 0x7fb6e4159f20 2023-07-14T11:55:57.911 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.921+0000 7fb6e37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6e4162b10 0x7fb6e40b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.911 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.921+0000 7fb6e37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6e4162b10 0x7fb6e40b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35366/0 (socket says 172.21.15.104:35366) 2023-07-14T11:55:57.911 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.921+0000 7fb6e37fe700 1 -- 172.21.15.104:0/2051720653 learned_addr learned my addr 172.21.15.104:0/2051720653 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:57.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.921+0000 7fb6e2ffd700 1 --2- 172.21.15.104:0/2051720653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6e40c9d90 0x7fb6e415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.921+0000 7fb6e27fc700 1 --2- 172.21.15.104:0/2051720653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6e4159f20 0x7fb6e413f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.925+0000 7fb6e37fe700 1 -- 172.21.15.104:0/2051720653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6e4159f20 msgr2=0x7fb6e413f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.925+0000 7fb6e37fe700 1 --2- 172.21.15.104:0/2051720653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6e4159f20 0x7fb6e413f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.925+0000 7fb6e37fe700 1 -- 172.21.15.104:0/2051720653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6e40c9d90 msgr2=0x7fb6e415c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:57.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.925+0000 7fb6e37fe700 1 --2- 172.21.15.104:0/2051720653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6e40c9d90 0x7fb6e415c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:57.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.925+0000 7fb6e37fe700 1 -- 172.21.15.104:0/2051720653 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc009a70 con 0x7fb6e4162b10 2023-07-14T11:55:57.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.925+0000 7fb6e37fe700 1 --2- 172.21.15.104:0/2051720653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6e4162b10 0x7fb6e40b15f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb6e406a6f0 tx=0x7fb6dc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:57.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.925+0000 7fb6cb7fe700 1 -- 172.21.15.104:0/2051720653 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc012430 con 0x7fb6e4162b10 2023-07-14T11:55:57.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.925+0000 7fb6cb7fe700 1 -- 172.21.15.104:0/2051720653 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc020070 con 0x7fb6e4162b10 2023-07-14T11:55:57.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.925+0000 7fb6cb7fe700 1 -- 172.21.15.104:0/2051720653 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc004280 con 0x7fb6e4162b10 2023-07-14T11:55:57.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.925+0000 7fb6e92a2700 1 -- 172.21.15.104:0/2051720653 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e4153b10 con 0x7fb6e4162b10 2023-07-14T11:55:57.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.925+0000 7fb6e92a2700 1 -- 172.21.15.104:0/2051720653 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e41540d0 con 0x7fb6e4162b10 2023-07-14T11:55:57.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.925+0000 7fb6e92a2700 1 -- 172.21.15.104:0/2051720653 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e4153970 con 0x7fb6e4162b10 2023-07-14T11:55:57.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.929+0000 7fb6cb7fe700 1 -- 172.21.15.104:0/2051720653 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb6dc0043e0 con 0x7fb6e4162b10 2023-07-14T11:55:57.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.929+0000 7fb6cb7fe700 1 --2- 172.21.15.104:0/2051720653 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb6d00662f0 0x7fb6d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:57.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.929+0000 7fb6cb7fe700 1 -- 172.21.15.104:0/2051720653 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6835+0+0 (secure 0 0 0) 0x7fb6dc022070 con 0x7fb6e4162b10 2023-07-14T11:55:57.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.929+0000 7fb6e2ffd700 1 --2- 172.21.15.104:0/2051720653 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb6d00662f0 0x7fb6d00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:57.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.933+0000 7fb6e2ffd700 1 --2- 172.21.15.104:0/2051720653 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb6d00662f0 0x7fb6d00687a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4000f60 tx=0x7fb6d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:57.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:57.937+0000 7fb6cb7fe700 1 -- 172.21.15.104:0/2051720653 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb6e4153970 con 0x7fb6e4162b10 2023-07-14T11:55:58.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.137+0000 7fb6e92a2700 1 -- 172.21.15.104:0/2051720653 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb6e4153970 con 0x7fb6e4162b10 2023-07-14T11:55:58.127 INFO:tasks.ceph.mon.a.smithi104.stderr:2023-07-14T11:55:58.141+0000 7fbde97f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T11:55:58.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.153+0000 7fb6cb7fe700 1 -- 172.21.15.104:0/2051720653 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb6e4153970 con 0x7fb6e4162b10 2023-07-14T11:55:58.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.157+0000 7fb6e92a2700 1 -- 172.21.15.104:0/2051720653 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb6d00662f0 msgr2=0x7fb6d00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:58.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.157+0000 7fb6e92a2700 1 --2- 172.21.15.104:0/2051720653 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb6d00662f0 0x7fb6d00687a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4000f60 tx=0x7fb6d4009040 comp rx=0 tx=0).stop 2023-07-14T11:55:58.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.157+0000 7fb6e92a2700 1 -- 172.21.15.104:0/2051720653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6e4162b10 msgr2=0x7fb6e40b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:58.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.157+0000 7fb6e92a2700 1 --2- 172.21.15.104:0/2051720653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6e4162b10 0x7fb6e40b15f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb6e406a6f0 tx=0x7fb6dc004af0 comp rx=0 tx=0).stop 2023-07-14T11:55:58.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.157+0000 7fb6e92a2700 1 -- 172.21.15.104:0/2051720653 shutdown_connections 2023-07-14T11:55:58.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.157+0000 7fb6e92a2700 1 --2- 172.21.15.104:0/2051720653 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb6d00662f0 0x7fb6d00687a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:58.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.157+0000 7fb6e92a2700 1 --2- 172.21.15.104:0/2051720653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6e4162b10 0x7fb6e40b15f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:58.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.157+0000 7fb6e92a2700 1 --2- 172.21.15.104:0/2051720653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6e4159f20 0x7fb6e413f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:58.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.157+0000 7fb6e92a2700 1 --2- 172.21.15.104:0/2051720653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6e40c9d90 0x7fb6e415c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:58.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.157+0000 7fb6e92a2700 1 -- 172.21.15.104:0/2051720653 >> 172.21.15.104:0/2051720653 conn(0x7fb6e406e960 msgr2=0x7fb6e414f530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:58.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.157+0000 7fb6e92a2700 1 -- 172.21.15.104:0/2051720653 shutdown_connections 2023-07-14T11:55:58.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.157+0000 7fb6e92a2700 1 -- 172.21.15.104:0/2051720653 wait complete. 2023-07-14T11:55:58.146 INFO:teuthology.orchestra.run.smithi104.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-14T11:55:58.148 INFO:tasks.ceph.mds.a.smithi104.stderr: -3926> 2023-07-14T11:55:02.095+0000 7fcb90b2d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-14T11:55:58.163 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-14T11:55:58.208 INFO:tasks.ceph.mds.a.smithi104.stdout:starting mds.a at 2023-07-14T11:55:58.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.345+0000 7fa8c9a8c700 1 Processor -- start 2023-07-14T11:55:58.333 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.345+0000 7fa8c9a8c700 1 -- start start 2023-07-14T11:55:58.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.345+0000 7fa8c9a8c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa8c4159c60 0x7fa8c406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:58.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.345+0000 7fa8c9a8c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa8c40b0770 0x7fa8c4070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:58.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.345+0000 7fa8c9a8c700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c406da00 con 0x7fa8c40cab00 2023-07-14T11:55:58.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.345+0000 7fa8c9a8c700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c406dce0 con 0x7fa8c4159c60 2023-07-14T11:55:58.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.345+0000 7fa8c9a8c700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c406dfc0 con 0x7fa8c40b0770 2023-07-14T11:55:58.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.345+0000 7fa8c8a8a700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa8c4159c60 0x7fa8c406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:58.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.345+0000 7fa8c928b700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa8c40cab00 0x7fa8c40c1ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:39996/0 (socket says 172.21.15.104:39996) 2023-07-14T11:55:58.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.345+0000 7fa8c928b700 1 -- 172.21.15.104:0/2427084313 learned_addr learned my addr 172.21.15.104:0/2427084313 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:58.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.345+0000 7fa8c3fff700 1 --2- 172.21.15.104:0/2427084313 >> v2:172.21.15.121:3300/0 conn(0x7fa8c40b0770 0x7fa8c4070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:58.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c8a8a700 1 -- 172.21.15.104:0/2427084313 >> v2:172.21.15.121:3300/0 conn(0x7fa8c40b0770 msgr2=0x7fa8c4070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:55:58.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c8a8a700 1 --2- 172.21.15.104:0/2427084313 >> v2:172.21.15.121:3300/0 conn(0x7fa8c40b0770 0x7fa8c4070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:58.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c8a8a700 1 -- 172.21.15.104:0/2427084313 >> v1:172.21.15.121:6789/0 conn(0x7fa8c40cab00 legacy=0x7fa8c40c1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:58.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c37fe700 1 -- 172.21.15.104:0/2427084313 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314854987 0 0) 0x7fa8c406da00 con 0x7fa8c40cab00 2023-07-14T11:55:58.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c8a8a700 1 -- 172.21.15.104:0/2427084313 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c40705e0 con 0x7fa8c4159c60 2023-07-14T11:55:58.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c3fff700 1 --2- 172.21.15.104:0/2427084313 >> v2:172.21.15.121:3300/0 conn(0x7fa8c40b0770 0x7fa8c4070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:55:58.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c8a8a700 1 --2- 172.21.15.104:0/2427084313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8c4159c60 0x7fa8c406d3b0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa8c415d7d0 tx=0x7fa8b8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497b7b67edd93609 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:58.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c37fe700 1 -- 172.21.15.104:0/2427084313 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8011830 con 0x7fa8c4159c60 2023-07-14T11:55:58.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c37fe700 1 -- 172.21.15.104:0/2427084313 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b8011990 con 0x7fa8c4159c60 2023-07-14T11:55:58.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c37fe700 1 -- 172.21.15.104:0/2427084313 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8011d40 con 0x7fa8c4159c60 2023-07-14T11:55:58.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c9a8c700 1 -- 172.21.15.104:0/2427084313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8c4159c60 msgr2=0x7fa8c406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:58.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c9a8c700 1 --2- 172.21.15.104:0/2427084313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8c4159c60 0x7fa8c406d3b0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa8c415d7d0 tx=0x7fa8b8009b30 comp rx=0 tx=0).stop 2023-07-14T11:55:58.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c9a8c700 1 -- 172.21.15.104:0/2427084313 shutdown_connections 2023-07-14T11:55:58.338 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c9a8c700 1 --2- 172.21.15.104:0/2427084313 >> v2:172.21.15.121:3300/0 conn(0x7fa8c40b0770 0x7fa8c4070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:58.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c9a8c700 1 --2- 172.21.15.104:0/2427084313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8c4159c60 0x7fa8c406d3b0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:58.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c9a8c700 1 -- 172.21.15.104:0/2427084313 >> 172.21.15.104:0/2427084313 conn(0x7fa8c40afc60 msgr2=0x7fa8c40b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:58.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c9a8c700 1 -- 172.21.15.104:0/2427084313 shutdown_connections 2023-07-14T11:55:58.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.349+0000 7fa8c9a8c700 1 -- 172.21.15.104:0/2427084313 wait complete. 2023-07-14T11:55:58.339 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c9a8c700 1 Processor -- start 2023-07-14T11:55:58.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c9a8c700 1 -- start start 2023-07-14T11:55:58.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c9a8c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8c40b0770 0x7fa8c41eec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:58.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c9a8c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8c40cab00 0x7fa8c415c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:58.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c9a8c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8c4159c60 0x7fa8c40cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:58.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c9a8c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41f32d0 con 0x7fa8c40b0770 2023-07-14T11:55:58.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c9a8c700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41ef190 con 0x7fa8c4159c60 2023-07-14T11:55:58.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c9a8c700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41f2930 con 0x7fa8c40cab00 2023-07-14T11:55:58.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c8a8a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8c40b0770 0x7fa8c41eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:58.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c8a8a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8c40b0770 0x7fa8c41eec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35416/0 (socket says 172.21.15.104:35416) 2023-07-14T11:55:58.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c8a8a700 1 -- 172.21.15.104:0/2687816321 learned_addr learned my addr 172.21.15.104:0/2687816321 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:58.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c3fff700 1 --2- 172.21.15.104:0/2687816321 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8c40cab00 0x7fa8c415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:58.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c928b700 1 --2- 172.21.15.104:0/2687816321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8c4159c60 0x7fa8c40cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:58.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c8a8a700 1 -- 172.21.15.104:0/2687816321 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8c40cab00 msgr2=0x7fa8c415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:58.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c8a8a700 1 --2- 172.21.15.104:0/2687816321 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8c40cab00 0x7fa8c415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:58.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c8a8a700 1 -- 172.21.15.104:0/2687816321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8c4159c60 msgr2=0x7fa8c40cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:58.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c8a8a700 1 --2- 172.21.15.104:0/2687816321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8c4159c60 0x7fa8c40cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:58.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.353+0000 7fa8c8a8a700 1 -- 172.21.15.104:0/2687816321 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b80094b0 con 0x7fa8c40b0770 2023-07-14T11:55:58.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.357+0000 7fa8c8a8a700 1 --2- 172.21.15.104:0/2687816321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8c40b0770 0x7fa8c41eec50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8014040 tx=0x7fa8b8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:58.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.357+0000 7fa8c17fa700 1 -- 172.21.15.104:0/2687816321 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8019980 con 0x7fa8c40b0770 2023-07-14T11:55:58.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.357+0000 7fa8c17fa700 1 -- 172.21.15.104:0/2687816321 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b801e070 con 0x7fa8c40b0770 2023-07-14T11:55:58.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.357+0000 7fa8c9a8c700 1 -- 172.21.15.104:0/2687816321 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c41f2c10 con 0x7fa8c40b0770 2023-07-14T11:55:58.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.357+0000 7fa8c17fa700 1 -- 172.21.15.104:0/2687816321 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8019d40 con 0x7fa8c40b0770 2023-07-14T11:55:58.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.357+0000 7fa8c9a8c700 1 -- 172.21.15.104:0/2687816321 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c415c640 con 0x7fa8c40b0770 2023-07-14T11:55:58.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.357+0000 7fa8c9a8c700 1 -- 172.21.15.104:0/2687816321 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c404e500 con 0x7fa8c40b0770 2023-07-14T11:55:58.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.361+0000 7fa8c17fa700 1 -- 172.21.15.104:0/2687816321 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa8b801a420 con 0x7fa8c40b0770 2023-07-14T11:55:58.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.361+0000 7fa8c17fa700 1 --2- 172.21.15.104:0/2687816321 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa8a80662f0 0x7fa8a80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:58.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.361+0000 7fa8c17fa700 1 -- 172.21.15.104:0/2687816321 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fa8b8085070 con 0x7fa8c40b0770 2023-07-14T11:55:58.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.361+0000 7fa8c3fff700 1 --2- 172.21.15.104:0/2687816321 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa8a80662f0 0x7fa8a80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:58.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.361+0000 7fa8c3fff700 1 --2- 172.21.15.104:0/2687816321 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa8a80662f0 0x7fa8a80687a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40c8370 tx=0x7fa8b0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:58.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.365+0000 7fa8c17fa700 1 -- 172.21.15.104:0/2687816321 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fa8b80b5720 con 0x7fa8c40b0770 2023-07-14T11:55:58.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:58.573+0000 7fa8c9a8c700 1 -- 172.21.15.104:0/2687816321 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa8c41571d0 con 0x7fa8c40b0770 2023-07-14T11:55:59.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.153+0000 7fa8c17fa700 1 -- 172.21.15.104:0/2687816321 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa8b80150a0 con 0x7fa8c40b0770 2023-07-14T11:55:59.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.157+0000 7fa8c9a8c700 1 -- 172.21.15.104:0/2687816321 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa8a80662f0 msgr2=0x7fa8a80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:59.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.157+0000 7fa8c9a8c700 1 --2- 172.21.15.104:0/2687816321 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa8a80662f0 0x7fa8a80687a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40c8370 tx=0x7fa8b0008040 comp rx=0 tx=0).stop 2023-07-14T11:55:59.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.157+0000 7fa8c9a8c700 1 -- 172.21.15.104:0/2687816321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8c40b0770 msgr2=0x7fa8c41eec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:59.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.157+0000 7fa8c9a8c700 1 --2- 172.21.15.104:0/2687816321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8c40b0770 0x7fa8c41eec50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8014040 tx=0x7fa8b8004770 comp rx=0 tx=0).stop 2023-07-14T11:55:59.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.161+0000 7fa8c9a8c700 1 -- 172.21.15.104:0/2687816321 shutdown_connections 2023-07-14T11:55:59.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.161+0000 7fa8c9a8c700 1 --2- 172.21.15.104:0/2687816321 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa8a80662f0 0x7fa8a80687a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:59.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.161+0000 7fa8c9a8c700 1 --2- 172.21.15.104:0/2687816321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8c4159c60 0x7fa8c40cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:59.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.161+0000 7fa8c9a8c700 1 --2- 172.21.15.104:0/2687816321 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8c40cab00 0x7fa8c415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:59.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.161+0000 7fa8c9a8c700 1 --2- 172.21.15.104:0/2687816321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8c40b0770 0x7fa8c41eec50 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:59.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.161+0000 7fa8c9a8c700 1 -- 172.21.15.104:0/2687816321 >> 172.21.15.104:0/2687816321 conn(0x7fa8c40afc60 msgr2=0x7fa8c41579d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:59.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.161+0000 7fa8c9a8c700 1 -- 172.21.15.104:0/2687816321 shutdown_connections 2023-07-14T11:55:59.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.161+0000 7fa8c9a8c700 1 -- 172.21.15.104:0/2687816321 wait complete. 2023-07-14T11:55:59.170 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-14T11:55:59.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.405+0000 7f8985365700 1 Processor -- start 2023-07-14T11:55:59.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.405+0000 7f8985365700 1 -- start start 2023-07-14T11:55:59.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.409+0000 7f8985365700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89800cd260 0x7f89800d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:59.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.409+0000 7f8985365700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89800d1a60 con 0x7f89800b0ee0 2023-07-14T11:55:59.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.409+0000 7f8985365700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89800d1ef0 con 0x7f89800d0610 2023-07-14T11:55:59.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.409+0000 7f8985365700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89800d0e50 con 0x7f89800cd260 2023-07-14T11:55:59.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.409+0000 7f897effd700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f89800b0ee0 0x7f89800ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:56232/0 (socket says 172.21.15.104:56232) 2023-07-14T11:55:59.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.409+0000 7f897effd700 1 -- 172.21.15.104:0/1935246062 learned_addr learned my addr 172.21.15.104:0/1935246062 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:55:59.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.409+0000 7f897e7fc700 1 --2- 172.21.15.104:0/1935246062 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89800cd260 0x7f89800d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:59.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.409+0000 7f897dffb700 1 -- 172.21.15.104:0/1935246062 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129068386 0 0) 0x7f89800d1a60 con 0x7f89800b0ee0 2023-07-14T11:55:59.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.409+0000 7f897dffb700 1 -- 172.21.15.104:0/1935246062 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89680035d0 con 0x7f89800b0ee0 2023-07-14T11:55:59.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.409+0000 7f897dffb700 1 -- 172.21.15.104:0/1935246062 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271754570 0 0) 0x7f89800d1ef0 con 0x7f89800d0610 2023-07-14T11:55:59.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.409+0000 7f897dffb700 1 -- 172.21.15.104:0/1935246062 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89800d1a60 con 0x7f89800d0610 2023-07-14T11:55:59.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.409+0000 7f897e7fc700 1 -- 172.21.15.104:0/1935246062 >> v1:172.21.15.121:6789/0 conn(0x7f89800d0610 legacy=0x7f89800af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:59.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.409+0000 7f897e7fc700 1 -- 172.21.15.104:0/1935246062 >> v1:172.21.15.104:6789/0 conn(0x7f89800b0ee0 legacy=0x7f89800ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:59.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.409+0000 7f897e7fc700 1 -- 172.21.15.104:0/1935246062 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89800d1130 con 0x7f89800cd260 2023-07-14T11:55:59.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.413+0000 7f897e7fc700 1 --2- 172.21.15.104:0/1935246062 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89800cd260 0x7f89800d00d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8974009880 tx=0x7f8974009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b923493792ad5ba3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:59.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.413+0000 7f897dffb700 1 -- 172.21.15.104:0/1935246062 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974011830 con 0x7f89800cd260 2023-07-14T11:55:59.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.413+0000 7f897dffb700 1 -- 172.21.15.104:0/1935246062 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8974011990 con 0x7f89800cd260 2023-07-14T11:55:59.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.413+0000 7f897dffb700 1 -- 172.21.15.104:0/1935246062 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974011d20 con 0x7f89800cd260 2023-07-14T11:55:59.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.413+0000 7f8985365700 1 -- 172.21.15.104:0/1935246062 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89800cd260 msgr2=0x7f89800d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:59.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.413+0000 7f8985365700 1 --2- 172.21.15.104:0/1935246062 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89800cd260 0x7f89800d00d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8974009880 tx=0x7f8974009b90 comp rx=0 tx=0).stop 2023-07-14T11:55:59.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.413+0000 7f8985365700 1 -- 172.21.15.104:0/1935246062 shutdown_connections 2023-07-14T11:55:59.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.413+0000 7f8985365700 1 --2- 172.21.15.104:0/1935246062 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89800cd260 0x7f89800d00d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:59.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.413+0000 7f8985365700 1 -- 172.21.15.104:0/1935246062 >> 172.21.15.104:0/1935246062 conn(0x7f8980060c40 msgr2=0x7f898009b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:59.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.413+0000 7f8985365700 1 -- 172.21.15.104:0/1935246062 shutdown_connections 2023-07-14T11:55:59.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.413+0000 7f8985365700 1 -- 172.21.15.104:0/1935246062 wait complete. 2023-07-14T11:55:59.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.417+0000 7f8985365700 1 Processor -- start 2023-07-14T11:55:59.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.417+0000 7f8985365700 1 -- start start 2023-07-14T11:55:59.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.417+0000 7f8985365700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89800b0ee0 0x7f89800c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:59.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.417+0000 7f8985365700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89800cd260 0x7f898014a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:59.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.417+0000 7f8985365700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89800d0610 0x7f89800c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:59.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.417+0000 7f8985365700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89800cc730 con 0x7f89800b0ee0 2023-07-14T11:55:59.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.417+0000 7f8985365700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89800c9180 con 0x7f89800cd260 2023-07-14T11:55:59.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.417+0000 7f8985365700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89801f21a0 con 0x7f89800d0610 2023-07-14T11:55:59.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.417+0000 7f897effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89800b0ee0 0x7f89800c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:59.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.417+0000 7f897e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89800cd260 0x7f898014a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:59.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.417+0000 7f897f7fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89800d0610 0x7f89800c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:59.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.417+0000 7f897e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89800cd260 0x7f898014a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:43826/0 (socket says 172.21.15.104:43826) 2023-07-14T11:55:59.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.417+0000 7f897f7fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89800d0610 0x7f89800c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:48074/0 (socket says 172.21.15.104:48074) 2023-07-14T11:55:59.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.417+0000 7f897effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89800b0ee0 0x7f89800c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35426/0 (socket says 172.21.15.104:35426) 2023-07-14T11:55:59.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.417+0000 7f897e7fc700 1 -- 172.21.15.104:0/2012972330 learned_addr learned my addr 172.21.15.104:0/2012972330 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:55:59.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.421+0000 7f897f7fe700 1 -- 172.21.15.104:0/2012972330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89800cd260 msgr2=0x7f898014a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:59.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.421+0000 7f897f7fe700 1 --2- 172.21.15.104:0/2012972330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89800cd260 0x7f898014a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:59.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.421+0000 7f897f7fe700 1 -- 172.21.15.104:0/2012972330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89800b0ee0 msgr2=0x7f89800c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:59.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.421+0000 7f897f7fe700 1 --2- 172.21.15.104:0/2012972330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89800b0ee0 0x7f89800c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:59.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.421+0000 7f897f7fe700 1 -- 172.21.15.104:0/2012972330 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89740094b0 con 0x7f89800d0610 2023-07-14T11:55:59.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.421+0000 7f897effd700 1 --2- 172.21.15.104:0/2012972330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89800b0ee0 0x7f89800c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:55:59.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.421+0000 7f897f7fe700 1 --2- 172.21.15.104:0/2012972330 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89800d0610 0x7f89800c6010 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f897800cbc0 tx=0x7f897800ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:59.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.421+0000 7f89677fe700 1 -- 172.21.15.104:0/2012972330 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978003c20 con 0x7f89800d0610 2023-07-14T11:55:59.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.421+0000 7f89677fe700 1 -- 172.21.15.104:0/2012972330 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8978003d80 con 0x7f89800d0610 2023-07-14T11:55:59.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.421+0000 7f89677fe700 1 -- 172.21.15.104:0/2012972330 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897800aa80 con 0x7f89800d0610 2023-07-14T11:55:59.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.421+0000 7f8985365700 1 -- 172.21.15.104:0/2012972330 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89800c8160 con 0x7f89800d0610 2023-07-14T11:55:59.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.421+0000 7f8985365700 1 -- 172.21.15.104:0/2012972330 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89800c7e10 con 0x7f89800d0610 2023-07-14T11:55:59.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.421+0000 7f8985365700 1 -- 172.21.15.104:0/2012972330 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89800c7fa0 con 0x7f89800d0610 2023-07-14T11:55:59.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.425+0000 7f89677fe700 1 -- 172.21.15.104:0/2012972330 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f897800abe0 con 0x7f89800d0610 2023-07-14T11:55:59.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.425+0000 7f89677fe700 1 --2- 172.21.15.104:0/2012972330 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f896c066340 0x7f896c0687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:59.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.425+0000 7f89677fe700 1 -- 172.21.15.104:0/2012972330 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f8978083f70 con 0x7f89800d0610 2023-07-14T11:55:59.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.425+0000 7f897effd700 1 --2- 172.21.15.104:0/2012972330 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f896c066340 0x7f896c0687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:59.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.429+0000 7f897effd700 1 --2- 172.21.15.104:0/2012972330 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f896c066340 0x7f896c0687f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8970007fd0 tx=0x7f8970004b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:59.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.433+0000 7f89677fe700 1 -- 172.21.15.104:0/2012972330 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f8978051100 con 0x7f89800d0610 2023-07-14T11:55:59.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:55:59.609+0000 7f8985365700 1 -- 172.21.15.104:0/2012972330 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f898004e500 con 0x7f89800d0610 2023-07-14T11:56:00.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.161+0000 7f89677fe700 1 -- 172.21.15.104:0/2012972330 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8978054720 con 0x7f89800d0610 2023-07-14T11:56:00.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.165+0000 7f8985365700 1 -- 172.21.15.104:0/2012972330 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f896c066340 msgr2=0x7f896c0687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:00.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.165+0000 7f8985365700 1 --2- 172.21.15.104:0/2012972330 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f896c066340 0x7f896c0687f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8970007fd0 tx=0x7f8970004b50 comp rx=0 tx=0).stop 2023-07-14T11:56:00.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.165+0000 7f8985365700 1 -- 172.21.15.104:0/2012972330 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89800d0610 msgr2=0x7f89800c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:00.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.165+0000 7f8985365700 1 --2- 172.21.15.104:0/2012972330 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89800d0610 0x7f89800c6010 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f897800cbc0 tx=0x7f897800ced0 comp rx=0 tx=0).stop 2023-07-14T11:56:00.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.165+0000 7f8985365700 1 -- 172.21.15.104:0/2012972330 shutdown_connections 2023-07-14T11:56:00.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.165+0000 7f8985365700 1 --2- 172.21.15.104:0/2012972330 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f896c066340 0x7f896c0687f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:00.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.165+0000 7f8985365700 1 --2- 172.21.15.104:0/2012972330 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89800d0610 0x7f89800c6010 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:00.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.165+0000 7f8985365700 1 --2- 172.21.15.104:0/2012972330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89800cd260 0x7f898014a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:00.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.165+0000 7f8985365700 1 --2- 172.21.15.104:0/2012972330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89800b0ee0 0x7f89800c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:00.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.165+0000 7f8985365700 1 -- 172.21.15.104:0/2012972330 >> 172.21.15.104:0/2012972330 conn(0x7f8980060c40 msgr2=0x7f89800b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:00.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.165+0000 7f8985365700 1 -- 172.21.15.104:0/2012972330 shutdown_connections 2023-07-14T11:56:00.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.169+0000 7f8985365700 1 -- 172.21.15.104:0/2012972330 wait complete. 2023-07-14T11:56:00.159 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'cephfs_metadata' removed 2023-07-14T11:56:00.176 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-14T11:56:00.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.369+0000 7fa134562700 1 Processor -- start 2023-07-14T11:56:00.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa134562700 1 -- start start 2023-07-14T11:56:00.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa134562700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa12c0cac60 0x7fa12c160410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:00.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa134562700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa12c0cb030 0x7fa12c070600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:00.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa134562700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa12c070b40 0x7fa12c070fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:00.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa134562700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12c0ca2e0 con 0x7fa12c070b40 2023-07-14T11:56:00.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa134562700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12c160950 con 0x7fa12c0cb030 2023-07-14T11:56:00.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa134562700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0cd920 con 0x7fa12c0cac60 2023-07-14T11:56:00.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa132aff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa12c070b40 0x7fa12c070fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:00.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa132aff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa12c070b40 0x7fa12c070fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45642/0 (socket says 172.21.15.104:45642) 2023-07-14T11:56:00.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa132aff700 1 -- 172.21.15.104:0/1372033464 learned_addr learned my addr 172.21.15.104:0/1372033464 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:00.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa131afd700 1 --2- 172.21.15.104:0/1372033464 >> v2:172.21.15.121:3300/0 conn(0x7fa12c0cb030 0x7fa12c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:00.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa1322fe700 1 --2- 172.21.15.104:0/1372033464 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa12c0cac60 0x7fa12c160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:00.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa131afd700 1 -- 172.21.15.104:0/1372033464 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa12c0cac60 msgr2=0x7fa12c160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:00.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa131afd700 1 --2- 172.21.15.104:0/1372033464 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa12c0cac60 0x7fa12c160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:00.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa131afd700 1 -- 172.21.15.104:0/1372033464 >> v2:172.21.15.104:3300/0 conn(0x7fa12c070b40 msgr2=0x7fa12c070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:00.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa131afd700 1 --2- 172.21.15.104:0/1372033464 >> v2:172.21.15.104:3300/0 conn(0x7fa12c070b40 0x7fa12c070fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:00.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa131afd700 1 -- 172.21.15.104:0/1372033464 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c0ad460 con 0x7fa12c0cb030 2023-07-14T11:56:00.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.373+0000 7fa132aff700 1 --2- 172.21.15.104:0/1372033464 >> v2:172.21.15.104:3300/0 conn(0x7fa12c070b40 0x7fa12c070fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:00.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.377+0000 7fa131afd700 1 --2- 172.21.15.104:0/1372033464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa12c0cb030 0x7fa12c070600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0730d0 tx=0x7fa11c009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4be337ad3563d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:00.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.377+0000 7fa1312fc700 1 -- 172.21.15.104:0/1372033464 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c011830 con 0x7fa12c0cb030 2023-07-14T11:56:00.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.377+0000 7fa1312fc700 1 -- 172.21.15.104:0/1372033464 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c011990 con 0x7fa12c0cb030 2023-07-14T11:56:00.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.377+0000 7fa1312fc700 1 -- 172.21.15.104:0/1372033464 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c011d20 con 0x7fa12c0cb030 2023-07-14T11:56:00.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.377+0000 7fa134562700 1 -- 172.21.15.104:0/1372033464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa12c0cb030 msgr2=0x7fa12c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:00.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.377+0000 7fa134562700 1 --2- 172.21.15.104:0/1372033464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa12c0cb030 0x7fa12c070600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0730d0 tx=0x7fa11c009b30 comp rx=0 tx=0).stop 2023-07-14T11:56:00.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.377+0000 7fa134562700 1 -- 172.21.15.104:0/1372033464 shutdown_connections 2023-07-14T11:56:00.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.377+0000 7fa134562700 1 --2- 172.21.15.104:0/1372033464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa12c0cb030 0x7fa12c070600 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:00.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.377+0000 7fa134562700 1 --2- 172.21.15.104:0/1372033464 >> v2:172.21.15.104:3300/0 conn(0x7fa12c070b40 0x7fa12c070fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:00.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.377+0000 7fa134562700 1 --2- 172.21.15.104:0/1372033464 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa12c0cac60 0x7fa12c160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:00.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.377+0000 7fa134562700 1 -- 172.21.15.104:0/1372033464 >> 172.21.15.104:0/1372033464 conn(0x7fa12c0afd60 msgr2=0x7fa12c0b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:00.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.377+0000 7fa134562700 1 -- 172.21.15.104:0/1372033464 shutdown_connections 2023-07-14T11:56:00.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.377+0000 7fa134562700 1 -- 172.21.15.104:0/1372033464 wait complete. 2023-07-14T11:56:00.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.377+0000 7fa134562700 1 Processor -- start 2023-07-14T11:56:00.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa134562700 1 -- start start 2023-07-14T11:56:00.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa134562700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa12c070b40 0x7fa12c1f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:00.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa134562700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa12c0cac60 0x7fa12c1f1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:00.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa134562700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa12c0cb030 0x7fa12c1f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:00.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa134562700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0c2f60 con 0x7fa12c0cb030 2023-07-14T11:56:00.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa134562700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0c2c50 con 0x7fa12c070b40 2023-07-14T11:56:00.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa134562700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c1f5280 con 0x7fa12c0cac60 2023-07-14T11:56:00.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa132aff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa12c0cb030 0x7fa12c1f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:00.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa132aff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa12c0cb030 0x7fa12c1f4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45650/0 (socket says 172.21.15.104:45650) 2023-07-14T11:56:00.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa132aff700 1 -- 172.21.15.104:0/1729193206 learned_addr learned my addr 172.21.15.104:0/1729193206 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:00.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa1322fe700 1 --2- 172.21.15.104:0/1729193206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa12c070b40 0x7fa12c1f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:00.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa131afd700 1 --2- 172.21.15.104:0/1729193206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa12c0cac60 0x7fa12c1f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:00.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa132aff700 1 -- 172.21.15.104:0/1729193206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa12c0cac60 msgr2=0x7fa12c1f1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:00.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa132aff700 1 --2- 172.21.15.104:0/1729193206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa12c0cac60 0x7fa12c1f1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:00.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa132aff700 1 -- 172.21.15.104:0/1729193206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa12c070b40 msgr2=0x7fa12c1f2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:00.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa132aff700 1 --2- 172.21.15.104:0/1729193206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa12c070b40 0x7fa12c1f2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:00.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa132aff700 1 -- 172.21.15.104:0/1729193206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa124009b30 con 0x7fa12c0cb030 2023-07-14T11:56:00.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa132aff700 1 --2- 172.21.15.104:0/1729193206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa12c0cb030 0x7fa12c1f4d40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa12400eed0 tx=0x7fa12400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:00.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa122ffd700 1 -- 172.21.15.104:0/1729193206 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124018070 con 0x7fa12c0cb030 2023-07-14T11:56:00.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa122ffd700 1 -- 172.21.15.104:0/1729193206 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1240092e0 con 0x7fa12c0cb030 2023-07-14T11:56:00.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa134562700 1 -- 172.21.15.104:0/1729193206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c0094b0 con 0x7fa12c0cb030 2023-07-14T11:56:00.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa122ffd700 1 -- 172.21.15.104:0/1729193206 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124010040 con 0x7fa12c0cb030 2023-07-14T11:56:00.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.381+0000 7fa134562700 1 -- 172.21.15.104:0/1729193206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12c1f57e0 con 0x7fa12c0cb030 2023-07-14T11:56:00.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.385+0000 7fa134562700 1 -- 172.21.15.104:0/1729193206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12c0c30a0 con 0x7fa12c0cb030 2023-07-14T11:56:00.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.385+0000 7fa122ffd700 1 -- 172.21.15.104:0/1729193206 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa12400b870 con 0x7fa12c0cb030 2023-07-14T11:56:00.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.389+0000 7fa122ffd700 1 --2- 172.21.15.104:0/1729193206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa1100662f0 0x7fa1100687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:00.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.389+0000 7fa122ffd700 1 -- 172.21.15.104:0/1729193206 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fa124083b90 con 0x7fa12c0cb030 2023-07-14T11:56:00.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.389+0000 7fa1322fe700 1 --2- 172.21.15.104:0/1729193206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa1100662f0 0x7fa1100687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:00.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.389+0000 7fa1322fe700 1 --2- 172.21.15.104:0/1729193206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa1100662f0 0x7fa1100687a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0b2600 tx=0x7fa128009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:00.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.393+0000 7fa122ffd700 1 -- 172.21.15.104:0/1729193206 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fa12c0c30a0 con 0x7fa12c0cb030 2023-07-14T11:56:00.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:00.569+0000 7fa134562700 1 -- 172.21.15.104:0/1729193206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa12c0c30a0 con 0x7fa12c0cb030 2023-07-14T11:56:01.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.169+0000 7fa122ffd700 1 -- 172.21.15.104:0/1729193206 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa12c0c30a0 con 0x7fa12c0cb030 2023-07-14T11:56:01.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.173+0000 7fa134562700 1 -- 172.21.15.104:0/1729193206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa1100662f0 msgr2=0x7fa1100687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:01.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.173+0000 7fa134562700 1 --2- 172.21.15.104:0/1729193206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa1100662f0 0x7fa1100687a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0b2600 tx=0x7fa128009240 comp rx=0 tx=0).stop 2023-07-14T11:56:01.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.173+0000 7fa134562700 1 -- 172.21.15.104:0/1729193206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa12c0cb030 msgr2=0x7fa12c1f4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:01.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.173+0000 7fa134562700 1 --2- 172.21.15.104:0/1729193206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa12c0cb030 0x7fa12c1f4d40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa12400eed0 tx=0x7fa12400b390 comp rx=0 tx=0).stop 2023-07-14T11:56:01.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.173+0000 7fa134562700 1 -- 172.21.15.104:0/1729193206 shutdown_connections 2023-07-14T11:56:01.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.173+0000 7fa134562700 1 --2- 172.21.15.104:0/1729193206 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa1100662f0 0x7fa1100687a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:01.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.173+0000 7fa134562700 1 --2- 172.21.15.104:0/1729193206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa12c0cb030 0x7fa12c1f4d40 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:01.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.173+0000 7fa134562700 1 --2- 172.21.15.104:0/1729193206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa12c0cac60 0x7fa12c1f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:01.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.173+0000 7fa134562700 1 --2- 172.21.15.104:0/1729193206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa12c070b40 0x7fa12c1f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:01.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.173+0000 7fa134562700 1 -- 172.21.15.104:0/1729193206 >> 172.21.15.104:0/1729193206 conn(0x7fa12c0afd60 msgr2=0x7fa12c0c9670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:01.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.173+0000 7fa134562700 1 -- 172.21.15.104:0/1729193206 shutdown_connections 2023-07-14T11:56:01.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.173+0000 7fa134562700 1 -- 172.21.15.104:0/1729193206 wait complete. 2023-07-14T11:56:01.164 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'cephfs_data' removed 2023-07-14T11:56:01.185 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2023-07-14T11:56:01.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.385+0000 7fbc42ff7700 1 Processor -- start 2023-07-14T11:56:01.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.385+0000 7fbc42ff7700 1 -- start start 2023-07-14T11:56:01.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc42ff7700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbc3c0cac60 0x7fbc3c160410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:01.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc42ff7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbc3c0cb030 0x7fbc3c070600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:01.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc42ff7700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3c160950 con 0x7fbc3c070b40 2023-07-14T11:56:01.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc42ff7700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc3c0ca2e0 con 0x7fbc3c0cac60 2023-07-14T11:56:01.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc42ff7700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc3c0cd920 con 0x7fbc3c0cb030 2023-07-14T11:56:01.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc40d93700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbc3c0cac60 0x7fbc3c160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:01.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc40d93700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbc3c0cac60 0x7fbc3c160410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45652/0 (socket says 172.21.15.104:45652) 2023-07-14T11:56:01.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc41594700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fbc3c070b40 0x7fbc3c0ab9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43704/0 (socket says 172.21.15.104:43704) 2023-07-14T11:56:01.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc40d93700 1 -- 172.21.15.104:0/2724927267 learned_addr learned my addr 172.21.15.104:0/2724927267 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:01.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc3bfff700 1 --2- 172.21.15.104:0/2724927267 >> v2:172.21.15.121:3300/0 conn(0x7fbc3c0cb030 0x7fbc3c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:01.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc40d93700 1 -- 172.21.15.104:0/2724927267 >> v2:172.21.15.121:3300/0 conn(0x7fbc3c0cb030 msgr2=0x7fbc3c070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:01.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc40d93700 1 --2- 172.21.15.104:0/2724927267 >> v2:172.21.15.121:3300/0 conn(0x7fbc3c0cb030 0x7fbc3c070600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:01.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc40d93700 1 -- 172.21.15.104:0/2724927267 >> v1:172.21.15.104:6789/0 conn(0x7fbc3c070b40 legacy=0x7fbc3c0ab9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:01.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc3b7fe700 1 -- 172.21.15.104:0/2724927267 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915699127 0 0) 0x7fbc34001490 con 0x7fbc3c070b40 2023-07-14T11:56:01.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc40d93700 1 -- 172.21.15.104:0/2724927267 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c0ad460 con 0x7fbc3c0cac60 2023-07-14T11:56:01.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc40d93700 1 --2- 172.21.15.104:0/2724927267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc3c0cac60 0x7fbc3c160410 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c06bcf0 tx=0x7fbc30009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29e0fcc23c94d16 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:01.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.389+0000 7fbc3b7fe700 1 -- 172.21.15.104:0/2724927267 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3000b8a0 con 0x7fbc3c0cac60 2023-07-14T11:56:01.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.393+0000 7fbc3b7fe700 1 -- 172.21.15.104:0/2724927267 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3000ba00 con 0x7fbc3c0cac60 2023-07-14T11:56:01.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.393+0000 7fbc3b7fe700 1 -- 172.21.15.104:0/2724927267 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3000bd90 con 0x7fbc3c0cac60 2023-07-14T11:56:01.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.393+0000 7fbc42ff7700 1 -- 172.21.15.104:0/2724927267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc3c0cac60 msgr2=0x7fbc3c160410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:01.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.393+0000 7fbc42ff7700 1 --2- 172.21.15.104:0/2724927267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc3c0cac60 0x7fbc3c160410 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c06bcf0 tx=0x7fbc30009bf0 comp rx=0 tx=0).stop 2023-07-14T11:56:01.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.393+0000 7fbc42ff7700 1 -- 172.21.15.104:0/2724927267 shutdown_connections 2023-07-14T11:56:01.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.393+0000 7fbc42ff7700 1 --2- 172.21.15.104:0/2724927267 >> v2:172.21.15.121:3300/0 conn(0x7fbc3c0cb030 0x7fbc3c070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:01.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.393+0000 7fbc42ff7700 1 --2- 172.21.15.104:0/2724927267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc3c0cac60 0x7fbc3c160410 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:01.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.393+0000 7fbc42ff7700 1 -- 172.21.15.104:0/2724927267 >> 172.21.15.104:0/2724927267 conn(0x7fbc3c0afd60 msgr2=0x7fbc3c0b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:01.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.393+0000 7fbc42ff7700 1 -- 172.21.15.104:0/2724927267 shutdown_connections 2023-07-14T11:56:01.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.393+0000 7fbc42ff7700 1 -- 172.21.15.104:0/2724927267 wait complete. 2023-07-14T11:56:01.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.393+0000 7fbc42ff7700 1 Processor -- start 2023-07-14T11:56:01.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.393+0000 7fbc42ff7700 1 -- start start 2023-07-14T11:56:01.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc42ff7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc3c070b40 0x7fbc3c0b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:01.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc42ff7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc3c0cac60 0x7fbc3c14c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:01.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc42ff7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc3c0cb030 0x7fbc3c1f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:01.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc42ff7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0b3f80 con 0x7fbc3c0cac60 2023-07-14T11:56:01.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc42ff7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0c3a50 con 0x7fbc3c070b40 2023-07-14T11:56:01.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc42ff7700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c14cc10 con 0x7fbc3c0cb030 2023-07-14T11:56:01.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc3bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc3c0cac60 0x7fbc3c14c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:01.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc3bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc3c0cac60 0x7fbc3c14c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45666/0 (socket says 172.21.15.104:45666) 2023-07-14T11:56:01.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc3bfff700 1 -- 172.21.15.104:0/423427666 learned_addr learned my addr 172.21.15.104:0/423427666 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:01.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc40d93700 1 --2- 172.21.15.104:0/423427666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc3c070b40 0x7fbc3c0b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:01.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc3bfff700 1 -- 172.21.15.104:0/423427666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc3c0cb030 msgr2=0x7fbc3c1f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:01.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc3bfff700 1 --2- 172.21.15.104:0/423427666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc3c0cb030 0x7fbc3c1f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:01.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc3bfff700 1 -- 172.21.15.104:0/423427666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc3c070b40 msgr2=0x7fbc3c0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:01.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc3bfff700 1 --2- 172.21.15.104:0/423427666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc3c070b40 0x7fbc3c0b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:01.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc3bfff700 1 -- 172.21.15.104:0/423427666 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc30009570 con 0x7fbc3c0cac60 2023-07-14T11:56:01.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc3bfff700 1 --2- 172.21.15.104:0/423427666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc3c0cac60 0x7fbc3c14c6d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800e790 tx=0x7fbc2800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:01.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc397fa700 1 -- 172.21.15.104:0/423427666 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc28013610 con 0x7fbc3c0cac60 2023-07-14T11:56:01.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc397fa700 1 -- 172.21.15.104:0/423427666 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc28013770 con 0x7fbc3c0cac60 2023-07-14T11:56:01.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc397fa700 1 -- 172.21.15.104:0/423427666 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc28013ae0 con 0x7fbc3c0cac60 2023-07-14T11:56:01.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc42ff7700 1 -- 172.21.15.104:0/423427666 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3c0c2e40 con 0x7fbc3c0cac60 2023-07-14T11:56:01.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.397+0000 7fbc42ff7700 1 -- 172.21.15.104:0/423427666 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3c0c3400 con 0x7fbc3c0cac60 2023-07-14T11:56:01.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.401+0000 7fbc42ff7700 1 -- 172.21.15.104:0/423427666 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3c04dcb0 con 0x7fbc3c0cac60 2023-07-14T11:56:01.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.401+0000 7fbc397fa700 1 -- 172.21.15.104:0/423427666 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fbc28013c40 con 0x7fbc3c0cac60 2023-07-14T11:56:01.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.401+0000 7fbc397fa700 1 --2- 172.21.15.104:0/423427666 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbc200662f0 0x7fbc200687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:01.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.401+0000 7fbc397fa700 1 -- 172.21.15.104:0/423427666 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6011+0+0 (secure 0 0 0) 0x7fbc28083a90 con 0x7fbc3c0cac60 2023-07-14T11:56:01.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.401+0000 7fbc40d93700 1 --2- 172.21.15.104:0/423427666 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbc200662f0 0x7fbc200687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:01.391 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.405+0000 7fbc40d93700 1 --2- 172.21.15.104:0/423427666 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbc200662f0 0x7fbc200687a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbc30014040 tx=0x7fbc30008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:01.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.409+0000 7fbc397fa700 1 -- 172.21.15.104:0/423427666 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fbc28050eb0 con 0x7fbc3c0cac60 2023-07-14T11:56:01.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:01.581+0000 7fbc42ff7700 1 -- 172.21.15.104:0/423427666 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbc3c0b0940 con 0x7fbc3c0cac60 2023-07-14T11:56:02.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.181+0000 7fbc397fa700 1 -- 172.21.15.104:0/423427666 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v51) v1 ==== 160+0+0 (secure 0 0 0) 0x7fbc28021070 con 0x7fbc3c0cac60 2023-07-14T11:56:02.176 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.189+0000 7fbc42ff7700 1 -- 172.21.15.104:0/423427666 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbc200662f0 msgr2=0x7fbc200687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:02.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.189+0000 7fbc42ff7700 1 --2- 172.21.15.104:0/423427666 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbc200662f0 0x7fbc200687a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbc30014040 tx=0x7fbc30008030 comp rx=0 tx=0).stop 2023-07-14T11:56:02.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.189+0000 7fbc42ff7700 1 -- 172.21.15.104:0/423427666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc3c0cac60 msgr2=0x7fbc3c14c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:02.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.189+0000 7fbc42ff7700 1 --2- 172.21.15.104:0/423427666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc3c0cac60 0x7fbc3c14c6d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800e790 tx=0x7fbc2800eb50 comp rx=0 tx=0).stop 2023-07-14T11:56:02.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.189+0000 7fbc42ff7700 1 -- 172.21.15.104:0/423427666 shutdown_connections 2023-07-14T11:56:02.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.189+0000 7fbc42ff7700 1 --2- 172.21.15.104:0/423427666 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbc200662f0 0x7fbc200687a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.189+0000 7fbc42ff7700 1 --2- 172.21.15.104:0/423427666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc3c0cac60 0x7fbc3c14c6d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.189+0000 7fbc42ff7700 1 --2- 172.21.15.104:0/423427666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc3c0cb030 0x7fbc3c1f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.189+0000 7fbc42ff7700 1 --2- 172.21.15.104:0/423427666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc3c070b40 0x7fbc3c0b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.189+0000 7fbc42ff7700 1 -- 172.21.15.104:0/423427666 >> 172.21.15.104:0/423427666 conn(0x7fbc3c0afd60 msgr2=0x7fbc3c072300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:02.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.189+0000 7fbc42ff7700 1 -- 172.21.15.104:0/423427666 shutdown_connections 2023-07-14T11:56:02.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.189+0000 7fbc42ff7700 1 -- 172.21.15.104:0/423427666 wait complete. 2023-07-14T11:56:02.178 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'data_new' removed 2023-07-14T11:56:02.197 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-14T11:56:02.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.381+0000 7f59aa216700 1 Processor -- start 2023-07-14T11:56:02.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.381+0000 7f59aa216700 1 -- start start 2023-07-14T11:56:02.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.385+0000 7f59aa216700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f59a4160000 0x7f59a406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:02.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.385+0000 7f59aa216700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f59a40cd7f0 0x7f59a40c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:02.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.385+0000 7f59aa216700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a40cdfb0 con 0x7f59a41603d0 2023-07-14T11:56:02.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.385+0000 7f59aa216700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a4070e40 con 0x7f59a4160000 2023-07-14T11:56:02.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.385+0000 7f59aa216700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a40cdcd0 con 0x7f59a40cd7f0 2023-07-14T11:56:02.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.385+0000 7f59a9214700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f59a4160000 0x7f59a406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:02.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.385+0000 7f59a9214700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f59a4160000 0x7f59a406dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45668/0 (socket says 172.21.15.104:45668) 2023-07-14T11:56:02.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.385+0000 7f59a9214700 1 -- 172.21.15.104:0/748844784 learned_addr learned my addr 172.21.15.104:0/748844784 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:02.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.385+0000 7f59a9a15700 1 --2- 172.21.15.104:0/748844784 >> v2:172.21.15.121:3300/0 conn(0x7f59a40cd7f0 0x7f59a40c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:02.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.385+0000 7f59a9a15700 1 -- 172.21.15.104:0/748844784 >> v2:172.21.15.104:3300/0 conn(0x7f59a4160000 msgr2=0x7f59a406dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:02.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.385+0000 7f59a9a15700 1 --2- 172.21.15.104:0/748844784 >> v2:172.21.15.104:3300/0 conn(0x7f59a4160000 0x7f59a406dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.385+0000 7f59a9a15700 1 -- 172.21.15.104:0/748844784 >> v1:172.21.15.121:6789/0 conn(0x7f59a41603d0 legacy=0x7f59a40c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:02.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.385+0000 7f59a9a15700 1 -- 172.21.15.104:0/748844784 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a4070780 con 0x7f59a40cd7f0 2023-07-14T11:56:02.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.385+0000 7f59a9a15700 1 --2- 172.21.15.104:0/748844784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59a40cd7f0 0x7f59a40c0d50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f59a4163c20 tx=0x7f599c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2942db5d23cd183 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:02.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.389+0000 7f599bfff700 1 -- 172.21.15.104:0/748844784 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c00e070 con 0x7f59a40cd7f0 2023-07-14T11:56:02.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.389+0000 7f599bfff700 1 -- 172.21.15.104:0/748844784 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599c00bb20 con 0x7f59a40cd7f0 2023-07-14T11:56:02.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.389+0000 7f599bfff700 1 -- 172.21.15.104:0/748844784 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c012430 con 0x7f59a40cd7f0 2023-07-14T11:56:02.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.389+0000 7f59aa216700 1 -- 172.21.15.104:0/748844784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59a40cd7f0 msgr2=0x7f59a40c0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:02.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.389+0000 7f59aa216700 1 --2- 172.21.15.104:0/748844784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59a40cd7f0 0x7f59a40c0d50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f59a4163c20 tx=0x7f599c00b330 comp rx=0 tx=0).stop 2023-07-14T11:56:02.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.389+0000 7f59aa216700 1 -- 172.21.15.104:0/748844784 shutdown_connections 2023-07-14T11:56:02.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.389+0000 7f59aa216700 1 --2- 172.21.15.104:0/748844784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59a40cd7f0 0x7f59a40c0d50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.389+0000 7f59aa216700 1 --2- 172.21.15.104:0/748844784 >> v2:172.21.15.104:3300/0 conn(0x7f59a4160000 0x7f59a406dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.389+0000 7f59aa216700 1 -- 172.21.15.104:0/748844784 >> 172.21.15.104:0/748844784 conn(0x7f59a40afc60 msgr2=0x7f59a40b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:02.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.389+0000 7f59aa216700 1 -- 172.21.15.104:0/748844784 shutdown_connections 2023-07-14T11:56:02.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.389+0000 7f59aa216700 1 -- 172.21.15.104:0/748844784 wait complete. 2023-07-14T11:56:02.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59aa216700 1 Processor -- start 2023-07-14T11:56:02.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59aa216700 1 -- start start 2023-07-14T11:56:02.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59aa216700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59a40cd7f0 0x7f59a40b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:02.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59aa216700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59a4160000 0x7f59a4162200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:02.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59aa216700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59a41603d0 0x7f59a41f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:02.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59aa216700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a41f5890 con 0x7f59a41603d0 2023-07-14T11:56:02.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59aa216700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a40b2f40 con 0x7f59a40cd7f0 2023-07-14T11:56:02.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59aa216700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a40b3220 con 0x7f59a4160000 2023-07-14T11:56:02.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59a9a15700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59a41603d0 0x7f59a41f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:02.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59a9a15700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59a41603d0 0x7f59a41f5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45682/0 (socket says 172.21.15.104:45682) 2023-07-14T11:56:02.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59a9a15700 1 -- 172.21.15.104:0/3672015690 learned_addr learned my addr 172.21.15.104:0/3672015690 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:02.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59a9214700 1 --2- 172.21.15.104:0/3672015690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59a40cd7f0 0x7f59a40b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:02.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59a8a13700 1 --2- 172.21.15.104:0/3672015690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59a4160000 0x7f59a4162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:02.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59a9a15700 1 -- 172.21.15.104:0/3672015690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59a4160000 msgr2=0x7f59a4162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:02.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59a9a15700 1 --2- 172.21.15.104:0/3672015690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59a4160000 0x7f59a4162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59a9a15700 1 -- 172.21.15.104:0/3672015690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59a40cd7f0 msgr2=0x7f59a40b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:02.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59a9a15700 1 --2- 172.21.15.104:0/3672015690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59a40cd7f0 0x7f59a40b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59a9a15700 1 -- 172.21.15.104:0/3672015690 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c009a70 con 0x7f59a41603d0 2023-07-14T11:56:02.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59a9a15700 1 --2- 172.21.15.104:0/3672015690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59a41603d0 0x7f59a41f5350 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f599c014040 tx=0x7f599c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:02.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f5999ffb700 1 -- 172.21.15.104:0/3672015690 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c012430 con 0x7f59a41603d0 2023-07-14T11:56:02.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f5999ffb700 1 -- 172.21.15.104:0/3672015690 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599c020070 con 0x7f59a41603d0 2023-07-14T11:56:02.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f59aa216700 1 -- 172.21.15.104:0/3672015690 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a40b3500 con 0x7f59a41603d0 2023-07-14T11:56:02.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.393+0000 7f5999ffb700 1 -- 172.21.15.104:0/3672015690 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c004280 con 0x7f59a41603d0 2023-07-14T11:56:02.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.397+0000 7f59aa216700 1 -- 172.21.15.104:0/3672015690 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a4155d80 con 0x7f59a41603d0 2023-07-14T11:56:02.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.397+0000 7f59aa216700 1 -- 172.21.15.104:0/3672015690 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a404e500 con 0x7f59a41603d0 2023-07-14T11:56:02.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.397+0000 7f5999ffb700 1 -- 172.21.15.104:0/3672015690 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f599c0043e0 con 0x7f59a41603d0 2023-07-14T11:56:02.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.401+0000 7f5999ffb700 1 --2- 172.21.15.104:0/3672015690 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f59880662f0 0x7f59880687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:02.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.401+0000 7f5999ffb700 1 -- 172.21.15.104:0/3672015690 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f599c022070 con 0x7f59a41603d0 2023-07-14T11:56:02.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.401+0000 7f59a9214700 1 --2- 172.21.15.104:0/3672015690 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f59880662f0 0x7f59880687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:02.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.401+0000 7f59a9214700 1 --2- 172.21.15.104:0/3672015690 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f59880662f0 0x7f59880687a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009ba0 tx=0x7f59a0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:02.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.405+0000 7f5999ffb700 1 -- 172.21.15.104:0/3672015690 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f599c0533a0 con 0x7f59a41603d0 2023-07-14T11:56:02.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.577+0000 7f59aa216700 1 -- 172.21.15.104:0/3672015690 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f59a4163a00 con 0x7f59a41603d0 2023-07-14T11:56:02.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.589+0000 7f5999ffb700 1 -- 172.21.15.104:0/3672015690 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f599c0569c0 con 0x7f59a41603d0 2023-07-14T11:56:02.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.593+0000 7f59aa216700 1 -- 172.21.15.104:0/3672015690 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f59880662f0 msgr2=0x7f59880687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:02.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.593+0000 7f59aa216700 1 --2- 172.21.15.104:0/3672015690 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f59880662f0 0x7f59880687a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009ba0 tx=0x7f59a0008040 comp rx=0 tx=0).stop 2023-07-14T11:56:02.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.593+0000 7f59aa216700 1 -- 172.21.15.104:0/3672015690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59a41603d0 msgr2=0x7f59a41f5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:02.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.593+0000 7f59aa216700 1 --2- 172.21.15.104:0/3672015690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59a41603d0 0x7f59a41f5350 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f599c014040 tx=0x7f599c004af0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.593+0000 7f59aa216700 1 -- 172.21.15.104:0/3672015690 shutdown_connections 2023-07-14T11:56:02.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.593+0000 7f59aa216700 1 --2- 172.21.15.104:0/3672015690 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f59880662f0 0x7f59880687a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.593+0000 7f59aa216700 1 --2- 172.21.15.104:0/3672015690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59a41603d0 0x7f59a41f5350 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.593+0000 7f59aa216700 1 --2- 172.21.15.104:0/3672015690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59a4160000 0x7f59a4162200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.593+0000 7f59aa216700 1 --2- 172.21.15.104:0/3672015690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59a40cd7f0 0x7f59a40b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.593+0000 7f59aa216700 1 -- 172.21.15.104:0/3672015690 >> 172.21.15.104:0/3672015690 conn(0x7f59a40afc60 msgr2=0x7f59a40cab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:02.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.593+0000 7f59aa216700 1 -- 172.21.15.104:0/3672015690 shutdown_connections 2023-07-14T11:56:02.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.593+0000 7f59aa216700 1 -- 172.21.15.104:0/3672015690 wait complete. 2023-07-14T11:56:02.602 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2023-07-14T11:56:02.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.785+0000 7f9e49a43700 1 Processor -- start 2023-07-14T11:56:02.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.785+0000 7f9e49a43700 1 -- start start 2023-07-14T11:56:02.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.785+0000 7f9e49a43700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9e440afd30 0x7f9e440b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:02.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.785+0000 7f9e49a43700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9e440703a0 0x7f9e44070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:02.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.785+0000 7f9e49a43700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e440cd530 con 0x7f9e4415c070 2023-07-14T11:56:02.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.785+0000 7f9e49a43700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e440b06e0 con 0x7f9e440afd30 2023-07-14T11:56:02.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.785+0000 7f9e49a43700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4415c440 con 0x7f9e440703a0 2023-07-14T11:56:02.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.785+0000 7f9e3a7fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9e440afd30 0x7f9e440b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:02.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.785+0000 7f9e3a7fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9e440afd30 0x7f9e440b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45696/0 (socket says 172.21.15.104:45696) 2023-07-14T11:56:02.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.785+0000 7f9e3a7fc700 1 -- 172.21.15.104:0/694045000 learned_addr learned my addr 172.21.15.104:0/694045000 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:02.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.785+0000 7f9e42ffd700 1 --1- 172.21.15.104:0/694045000 >> v1:172.21.15.104:6789/0 conn(0x7f9e4415c070 0x7f9e440ad370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43708/0 (socket says 172.21.15.104:43708) 2023-07-14T11:56:02.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.785+0000 7f9e437fe700 1 --2- 172.21.15.104:0/694045000 >> v2:172.21.15.121:3300/0 conn(0x7f9e440703a0 0x7f9e44070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:02.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e3a7fc700 1 -- 172.21.15.104:0/694045000 >> v2:172.21.15.121:3300/0 conn(0x7f9e440703a0 msgr2=0x7f9e44070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:02.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e3a7fc700 1 --2- 172.21.15.104:0/694045000 >> v2:172.21.15.121:3300/0 conn(0x7f9e440703a0 0x7f9e44070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e3a7fc700 1 -- 172.21.15.104:0/694045000 >> v1:172.21.15.104:6789/0 conn(0x7f9e4415c070 legacy=0x7f9e440ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:02.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e3a7fc700 1 -- 172.21.15.104:0/694045000 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e440cc1b0 con 0x7f9e440afd30 2023-07-14T11:56:02.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e3a7fc700 1 --2- 172.21.15.104:0/694045000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e440afd30 0x7f9e440b01a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9e44071900 tx=0x7f9e2c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fda23af7552da35 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:02.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e427fc700 1 -- 172.21.15.104:0/694045000 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c00b8a0 con 0x7f9e440afd30 2023-07-14T11:56:02.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e427fc700 1 -- 172.21.15.104:0/694045000 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c00ba00 con 0x7f9e440afd30 2023-07-14T11:56:02.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e427fc700 1 -- 172.21.15.104:0/694045000 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c00bd90 con 0x7f9e440afd30 2023-07-14T11:56:02.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e49a43700 1 -- 172.21.15.104:0/694045000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e440afd30 msgr2=0x7f9e440b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:02.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e49a43700 1 --2- 172.21.15.104:0/694045000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e440afd30 0x7f9e440b01a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9e44071900 tx=0x7f9e2c009bf0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e49a43700 1 -- 172.21.15.104:0/694045000 shutdown_connections 2023-07-14T11:56:02.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e49a43700 1 --2- 172.21.15.104:0/694045000 >> v2:172.21.15.121:3300/0 conn(0x7f9e440703a0 0x7f9e44070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e49a43700 1 --2- 172.21.15.104:0/694045000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e440afd30 0x7f9e440b01a0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e49a43700 1 -- 172.21.15.104:0/694045000 >> 172.21.15.104:0/694045000 conn(0x7f9e440cad30 msgr2=0x7f9e440cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:02.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e49a43700 1 -- 172.21.15.104:0/694045000 shutdown_connections 2023-07-14T11:56:02.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.789+0000 7f9e49a43700 1 -- 172.21.15.104:0/694045000 wait complete. 2023-07-14T11:56:02.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e49a43700 1 Processor -- start 2023-07-14T11:56:02.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e49a43700 1 -- start start 2023-07-14T11:56:02.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e49a43700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e440703a0 0x7f9e44148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:02.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e49a43700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e440afd30 0x7f9e440ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:02.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e49a43700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e4415c070 0x7f9e4415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:02.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e49a43700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440ad330 con 0x7f9e4415c070 2023-07-14T11:56:02.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e49a43700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e44148ca0 con 0x7f9e440703a0 2023-07-14T11:56:02.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e49a43700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440af340 con 0x7f9e440afd30 2023-07-14T11:56:02.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e437fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e4415c070 0x7f9e4415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:02.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e437fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e4415c070 0x7f9e4415a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45702/0 (socket says 172.21.15.104:45702) 2023-07-14T11:56:02.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e437fe700 1 -- 172.21.15.104:0/2715229638 learned_addr learned my addr 172.21.15.104:0/2715229638 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:02.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e42ffd700 1 --2- 172.21.15.104:0/2715229638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e440703a0 0x7f9e44148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:02.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e3a7fc700 1 --2- 172.21.15.104:0/2715229638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e440afd30 0x7f9e440ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:02.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e437fe700 1 -- 172.21.15.104:0/2715229638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e440afd30 msgr2=0x7f9e440ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:02.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e437fe700 1 --2- 172.21.15.104:0/2715229638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e440afd30 0x7f9e440ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e437fe700 1 -- 172.21.15.104:0/2715229638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e440703a0 msgr2=0x7f9e44148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:02.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e437fe700 1 --2- 172.21.15.104:0/2715229638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e440703a0 0x7f9e44148760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:02.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e437fe700 1 -- 172.21.15.104:0/2715229638 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2c009570 con 0x7f9e4415c070 2023-07-14T11:56:02.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.793+0000 7f9e3a7fc700 1 --2- 172.21.15.104:0/2715229638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e440afd30 0x7f9e440ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:02.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.797+0000 7f9e437fe700 1 --2- 172.21.15.104:0/2715229638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e4415c070 0x7f9e4415a6b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c007e40 tx=0x7f9e3c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:02.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.797+0000 7f9e40ff9700 1 -- 172.21.15.104:0/2715229638 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c00eea0 con 0x7f9e4415c070 2023-07-14T11:56:02.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.797+0000 7f9e40ff9700 1 -- 172.21.15.104:0/2715229638 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c014420 con 0x7f9e4415c070 2023-07-14T11:56:02.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.797+0000 7f9e40ff9700 1 -- 172.21.15.104:0/2715229638 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0147b0 con 0x7f9e4415c070 2023-07-14T11:56:02.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.797+0000 7f9e49a43700 1 -- 172.21.15.104:0/2715229638 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e440ad540 con 0x7f9e4415c070 2023-07-14T11:56:02.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.797+0000 7f9e49a43700 1 -- 172.21.15.104:0/2715229638 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e440adb00 con 0x7f9e4415c070 2023-07-14T11:56:02.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.797+0000 7f9e49a43700 1 -- 172.21.15.104:0/2715229638 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4404dcb0 con 0x7f9e4415c070 2023-07-14T11:56:02.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.797+0000 7f9e40ff9700 1 -- 172.21.15.104:0/2715229638 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9e3c014910 con 0x7f9e4415c070 2023-07-14T11:56:02.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.801+0000 7f9e40ff9700 1 --2- 172.21.15.104:0/2715229638 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e240662f0 0x7f9e240687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:02.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.801+0000 7f9e40ff9700 1 -- 172.21.15.104:0/2715229638 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f9e3c084400 con 0x7f9e4415c070 2023-07-14T11:56:02.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.801+0000 7f9e42ffd700 1 --2- 172.21.15.104:0/2715229638 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e240662f0 0x7f9e240687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:02.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.801+0000 7f9e42ffd700 1 --2- 172.21.15.104:0/2715229638 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e240662f0 0x7f9e240687a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9e4406f920 tx=0x7f9e34007d90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:02.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:02.805+0000 7f9e40ff9700 1 -- 172.21.15.104:0/2715229638 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f9e3c0519c0 con 0x7f9e4415c070 2023-07-14T11:56:03.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.033+0000 7f9e49a43700 1 -- 172.21.15.104:0/2715229638 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f9e440c85e0 con 0x7f9e4415c070 2023-07-14T11:56:03.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.189+0000 7f9e40ff9700 1 -- 172.21.15.104:0/2715229638 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v175) v1 ==== 147+0+0 (secure 0 0 0) 0x7f9e3c054fe0 con 0x7f9e4415c070 2023-07-14T11:56:03.180 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.193+0000 7f9e49a43700 1 -- 172.21.15.104:0/2715229638 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e240662f0 msgr2=0x7f9e240687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:03.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.193+0000 7f9e49a43700 1 --2- 172.21.15.104:0/2715229638 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e240662f0 0x7f9e240687a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9e4406f920 tx=0x7f9e34007d90 comp rx=0 tx=0).stop 2023-07-14T11:56:03.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.193+0000 7f9e49a43700 1 -- 172.21.15.104:0/2715229638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e4415c070 msgr2=0x7f9e4415a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:03.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.193+0000 7f9e49a43700 1 --2- 172.21.15.104:0/2715229638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e4415c070 0x7f9e4415a6b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c007e40 tx=0x7f9e3c00e3c0 comp rx=0 tx=0).stop 2023-07-14T11:56:03.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.193+0000 7f9e49a43700 1 -- 172.21.15.104:0/2715229638 shutdown_connections 2023-07-14T11:56:03.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.193+0000 7f9e49a43700 1 --2- 172.21.15.104:0/2715229638 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9e240662f0 0x7f9e240687a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:03.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.193+0000 7f9e49a43700 1 --2- 172.21.15.104:0/2715229638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e4415c070 0x7f9e4415a6b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:03.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.193+0000 7f9e49a43700 1 --2- 172.21.15.104:0/2715229638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e440afd30 0x7f9e440ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:03.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.193+0000 7f9e49a43700 1 --2- 172.21.15.104:0/2715229638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e440703a0 0x7f9e44148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:03.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.193+0000 7f9e49a43700 1 -- 172.21.15.104:0/2715229638 >> 172.21.15.104:0/2715229638 conn(0x7f9e440cad30 msgr2=0x7f9e44155420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:03.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.193+0000 7f9e49a43700 1 -- 172.21.15.104:0/2715229638 shutdown_connections 2023-07-14T11:56:03.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.193+0000 7f9e49a43700 1 -- 172.21.15.104:0/2715229638 wait complete. 2023-07-14T11:56:03.209 INFO:tasks.cephfs_test_runner:test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ok 2023-07-14T11:56:03.209 INFO:tasks.cephfs_test_runner:Starting test: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-07-14T11:56:03.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2023-07-14T11:56:03.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.401+0000 7f86f14bc700 1 Processor -- start 2023-07-14T11:56:03.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.401+0000 7f86f14bc700 1 -- start start 2023-07-14T11:56:03.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.401+0000 7f86f14bc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f86ec0c9d90 0x7f86ec0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:03.391 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86f14bc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f86ec15d7f0 0x7f86ec15dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:03.391 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86f14bc700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86ec0b0ef0 con 0x7f86ec159f20 2023-07-14T11:56:03.391 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86f14bc700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86ec0ce9c0 con 0x7f86ec0c9d90 2023-07-14T11:56:03.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86f14bc700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86ec0cee50 con 0x7f86ec15d7f0 2023-07-14T11:56:03.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86eaffd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f86ec0c9d90 0x7f86ec0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:03.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86ea7fc700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f86ec159f20 0x7f86ec0caa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43722/0 (socket says 172.21.15.104:43722) 2023-07-14T11:56:03.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86ea7fc700 1 -- 172.21.15.104:0/3879749057 learned_addr learned my addr 172.21.15.104:0/3879749057 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:03.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86eb7fe700 1 --2- 172.21.15.104:0/3879749057 >> v2:172.21.15.121:3300/0 conn(0x7f86ec15d7f0 0x7f86ec15dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:03.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86eaffd700 1 -- 172.21.15.104:0/3879749057 >> v2:172.21.15.121:3300/0 conn(0x7f86ec15d7f0 msgr2=0x7f86ec15dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:03.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86eaffd700 1 --2- 172.21.15.104:0/3879749057 >> v2:172.21.15.121:3300/0 conn(0x7f86ec15d7f0 0x7f86ec15dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:03.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86eaffd700 1 -- 172.21.15.104:0/3879749057 >> v1:172.21.15.104:6789/0 conn(0x7f86ec159f20 legacy=0x7f86ec0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:03.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86eaffd700 1 -- 172.21.15.104:0/3879749057 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec0ca6a0 con 0x7f86ec0c9d90 2023-07-14T11:56:03.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86e9ffb700 1 -- 172.21.15.104:0/3879749057 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884922492 0 0) 0x7f86ec0b0ef0 con 0x7f86ec159f20 2023-07-14T11:56:03.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86eaffd700 1 --2- 172.21.15.104:0/3879749057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86ec0c9d90 0x7f86ec0ca160 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0ca950 tx=0x7f86dc009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60651a4919ae9f39 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:03.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86e9ffb700 1 -- 172.21.15.104:0/3879749057 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc011830 con 0x7f86ec0c9d90 2023-07-14T11:56:03.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86e9ffb700 1 -- 172.21.15.104:0/3879749057 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc011990 con 0x7f86ec0c9d90 2023-07-14T11:56:03.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86e9ffb700 1 -- 172.21.15.104:0/3879749057 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc011d40 con 0x7f86ec0c9d90 2023-07-14T11:56:03.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86f14bc700 1 -- 172.21.15.104:0/3879749057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86ec0c9d90 msgr2=0x7f86ec0ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:03.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86f14bc700 1 --2- 172.21.15.104:0/3879749057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86ec0c9d90 0x7f86ec0ca160 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0ca950 tx=0x7f86dc009b30 comp rx=0 tx=0).stop 2023-07-14T11:56:03.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86f14bc700 1 -- 172.21.15.104:0/3879749057 shutdown_connections 2023-07-14T11:56:03.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86f14bc700 1 --2- 172.21.15.104:0/3879749057 >> v2:172.21.15.121:3300/0 conn(0x7f86ec15d7f0 0x7f86ec15dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:03.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86f14bc700 1 --2- 172.21.15.104:0/3879749057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86ec0c9d90 0x7f86ec0ca160 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:03.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.405+0000 7f86f14bc700 1 -- 172.21.15.104:0/3879749057 >> 172.21.15.104:0/3879749057 conn(0x7f86ec06e960 msgr2=0x7f86ec06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:03.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.409+0000 7f86f14bc700 1 -- 172.21.15.104:0/3879749057 shutdown_connections 2023-07-14T11:56:03.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.409+0000 7f86f14bc700 1 -- 172.21.15.104:0/3879749057 wait complete. 2023-07-14T11:56:03.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.409+0000 7f86f14bc700 1 Processor -- start 2023-07-14T11:56:03.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86f14bc700 1 -- start start 2023-07-14T11:56:03.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86f14bc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86ec0c9d90 0x7f86ec142300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:03.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86f14bc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86ec159f20 0x7f86ec0c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:03.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86f14bc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86ec15d7f0 0x7f86ec0c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:03.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86f14bc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec0c4c00 con 0x7f86ec15d7f0 2023-07-14T11:56:03.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86f14bc700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec152e20 con 0x7f86ec159f20 2023-07-14T11:56:03.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86f14bc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec0c4640 con 0x7f86ec0c9d90 2023-07-14T11:56:03.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86eb7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86ec15d7f0 0x7f86ec0c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:03.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86eb7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86ec15d7f0 0x7f86ec0c3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45722/0 (socket says 172.21.15.104:45722) 2023-07-14T11:56:03.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86eb7fe700 1 -- 172.21.15.104:0/193099683 learned_addr learned my addr 172.21.15.104:0/193099683 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:03.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86eaffd700 1 --2- 172.21.15.104:0/193099683 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86ec0c9d90 0x7f86ec142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:03.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86ea7fc700 1 --2- 172.21.15.104:0/193099683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86ec159f20 0x7f86ec0c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:03.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86eb7fe700 1 -- 172.21.15.104:0/193099683 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86ec0c9d90 msgr2=0x7f86ec142300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:03.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86eb7fe700 1 --2- 172.21.15.104:0/193099683 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86ec0c9d90 0x7f86ec142300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:03.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86eb7fe700 1 -- 172.21.15.104:0/193099683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86ec159f20 msgr2=0x7f86ec0c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:03.403 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86eb7fe700 1 --2- 172.21.15.104:0/193099683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86ec159f20 0x7f86ec0c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:03.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86eb7fe700 1 -- 172.21.15.104:0/193099683 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc0094b0 con 0x7f86ec15d7f0 2023-07-14T11:56:03.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86eb7fe700 1 --2- 172.21.15.104:0/193099683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86ec15d7f0 0x7f86ec0c3fe0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f86e4007e40 tx=0x7f86e400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:03.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86d37fe700 1 -- 172.21.15.104:0/193099683 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e400ede0 con 0x7f86ec15d7f0 2023-07-14T11:56:03.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86d37fe700 1 -- 172.21.15.104:0/193099683 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e4014420 con 0x7f86ec15d7f0 2023-07-14T11:56:03.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86f14bc700 1 -- 172.21.15.104:0/193099683 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86ec0c4920 con 0x7f86ec15d7f0 2023-07-14T11:56:03.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86d37fe700 1 -- 172.21.15.104:0/193099683 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40147d0 con 0x7f86ec15d7f0 2023-07-14T11:56:03.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.413+0000 7f86f14bc700 1 -- 172.21.15.104:0/193099683 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ec13f700 con 0x7f86ec15d7f0 2023-07-14T11:56:03.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.417+0000 7f86f14bc700 1 -- 172.21.15.104:0/193099683 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86ec153300 con 0x7f86ec15d7f0 2023-07-14T11:56:03.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.417+0000 7f86d37fe700 1 -- 172.21.15.104:0/193099683 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f86e4014930 con 0x7f86ec15d7f0 2023-07-14T11:56:03.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.421+0000 7f86d37fe700 1 --2- 172.21.15.104:0/193099683 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86d80662f0 0x7f86d80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:03.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.421+0000 7f86d37fe700 1 -- 172.21.15.104:0/193099683 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f86e4084240 con 0x7f86ec15d7f0 2023-07-14T11:56:03.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.421+0000 7f86eaffd700 1 --2- 172.21.15.104:0/193099683 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86d80662f0 0x7f86d80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:03.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.421+0000 7f86eaffd700 1 --2- 172.21.15.104:0/193099683 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86d80662f0 0x7f86d80687a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0090b0 tx=0x7f86dc01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:03.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.425+0000 7f86d37fe700 1 -- 172.21.15.104:0/193099683 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f86ec153300 con 0x7f86ec15d7f0 2023-07-14T11:56:03.642 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:03.653+0000 7f86f14bc700 1 -- 172.21.15.104:0/193099683 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7f86ec153300 con 0x7f86ec15d7f0 2023-07-14T11:56:04.180 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.193+0000 7f86d37fe700 1 -- 172.21.15.104:0/193099683 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v176) v1 ==== 137+0+0 (secure 0 0 0) 0x7f86ec153300 con 0x7f86ec15d7f0 2023-07-14T11:56:04.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.197+0000 7f86f14bc700 1 -- 172.21.15.104:0/193099683 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86d80662f0 msgr2=0x7f86d80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:04.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.197+0000 7f86f14bc700 1 --2- 172.21.15.104:0/193099683 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86d80662f0 0x7f86d80687a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0090b0 tx=0x7f86dc01c040 comp rx=0 tx=0).stop 2023-07-14T11:56:04.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.197+0000 7f86f14bc700 1 -- 172.21.15.104:0/193099683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86ec15d7f0 msgr2=0x7f86ec0c3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:04.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.197+0000 7f86f14bc700 1 --2- 172.21.15.104:0/193099683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86ec15d7f0 0x7f86ec0c3fe0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f86e4007e40 tx=0x7f86e400e3c0 comp rx=0 tx=0).stop 2023-07-14T11:56:04.186 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.197+0000 7f86f14bc700 1 -- 172.21.15.104:0/193099683 shutdown_connections 2023-07-14T11:56:04.186 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.197+0000 7f86f14bc700 1 --2- 172.21.15.104:0/193099683 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86d80662f0 0x7f86d80687a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:04.187 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.197+0000 7f86f14bc700 1 --2- 172.21.15.104:0/193099683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86ec15d7f0 0x7f86ec0c3fe0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:04.187 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.197+0000 7f86f14bc700 1 --2- 172.21.15.104:0/193099683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86ec159f20 0x7f86ec0c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:04.187 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.197+0000 7f86f14bc700 1 --2- 172.21.15.104:0/193099683 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86ec0c9d90 0x7f86ec142300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:04.187 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.197+0000 7f86f14bc700 1 -- 172.21.15.104:0/193099683 >> 172.21.15.104:0/193099683 conn(0x7f86ec06e960 msgr2=0x7f86ec15c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:04.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.197+0000 7f86f14bc700 1 -- 172.21.15.104:0/193099683 shutdown_connections 2023-07-14T11:56:04.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.197+0000 7f86f14bc700 1 -- 172.21.15.104:0/193099683 wait complete. 2023-07-14T11:56:04.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-14T11:56:04.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.401+0000 7f191d719700 1 Processor -- start 2023-07-14T11:56:04.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.401+0000 7f191d719700 1 -- start start 2023-07-14T11:56:04.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.405+0000 7f191d719700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f19180d07e0 0x7f19181669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:04.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.405+0000 7f191d719700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19180d1a60 con 0x7f19180cd260 2023-07-14T11:56:04.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.405+0000 7f191d719700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19180d1ef0 con 0x7f19180b0ee0 2023-07-14T11:56:04.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.405+0000 7f191d719700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19180d0c60 con 0x7f19180d07e0 2023-07-14T11:56:04.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.405+0000 7f19167fc700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f19180cd260 0x7f19180d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43736/0 (socket says 172.21.15.104:43736) 2023-07-14T11:56:04.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.405+0000 7f19167fc700 1 -- 172.21.15.104:0/393118669 learned_addr learned my addr 172.21.15.104:0/393118669 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:04.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.405+0000 7f19177fe700 1 --2- 172.21.15.104:0/393118669 >> v2:172.21.15.121:3300/0 conn(0x7f19180d07e0 0x7f19181669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:04.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.405+0000 7f1915ffb700 1 -- 172.21.15.104:0/393118669 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3196210241 0 0) 0x7f19180d1a60 con 0x7f19180cd260 2023-07-14T11:56:04.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.405+0000 7f1915ffb700 1 -- 172.21.15.104:0/393118669 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19000035d0 con 0x7f19180cd260 2023-07-14T11:56:04.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f1915ffb700 1 -- 172.21.15.104:0/393118669 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512437538 0 0) 0x7f19180d1ef0 con 0x7f19180b0ee0 2023-07-14T11:56:04.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f1915ffb700 1 -- 172.21.15.104:0/393118669 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19180d1a60 con 0x7f19180b0ee0 2023-07-14T11:56:04.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f19177fe700 1 -- 172.21.15.104:0/393118669 >> v1:172.21.15.121:6789/0 conn(0x7f19180b0ee0 legacy=0x7f19180ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:04.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f19177fe700 1 -- 172.21.15.104:0/393118669 >> v1:172.21.15.104:6789/0 conn(0x7f19180cd260 legacy=0x7f19180d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:04.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f19177fe700 1 -- 172.21.15.104:0/393118669 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19180d0e50 con 0x7f19180d07e0 2023-07-14T11:56:04.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f19177fe700 1 --2- 172.21.15.104:0/393118669 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19180d07e0 0x7f19181669b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1910009d90 tx=0x7f191000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf053aea84ac6c20 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:04.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f1915ffb700 1 -- 172.21.15.104:0/393118669 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191000e070 con 0x7f19180d07e0 2023-07-14T11:56:04.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f1915ffb700 1 -- 172.21.15.104:0/393118669 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191000bb20 con 0x7f19180d07e0 2023-07-14T11:56:04.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f1915ffb700 1 -- 172.21.15.104:0/393118669 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910012430 con 0x7f19180d07e0 2023-07-14T11:56:04.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f191d719700 1 -- 172.21.15.104:0/393118669 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19180d07e0 msgr2=0x7f19181669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:04.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f191d719700 1 --2- 172.21.15.104:0/393118669 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19180d07e0 0x7f19181669b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1910009d90 tx=0x7f191000b330 comp rx=0 tx=0).stop 2023-07-14T11:56:04.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f191d719700 1 -- 172.21.15.104:0/393118669 shutdown_connections 2023-07-14T11:56:04.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f191d719700 1 --2- 172.21.15.104:0/393118669 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19180d07e0 0x7f19181669b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:04.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f191d719700 1 -- 172.21.15.104:0/393118669 >> 172.21.15.104:0/393118669 conn(0x7f1918060c40 msgr2=0x7f191809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:04.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f191d719700 1 -- 172.21.15.104:0/393118669 shutdown_connections 2023-07-14T11:56:04.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.409+0000 7f191d719700 1 -- 172.21.15.104:0/393118669 wait complete. 2023-07-14T11:56:04.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f191d719700 1 Processor -- start 2023-07-14T11:56:04.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f191d719700 1 -- start start 2023-07-14T11:56:04.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f191d719700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19180b0ee0 0x7f19181f1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:04.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f191d719700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19180cd260 0x7f19180c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:04.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f191d719700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19180d07e0 0x7f19180c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:04.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f191d719700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19180c8fb0 con 0x7f19180d07e0 2023-07-14T11:56:04.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f191d719700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19180c8330 con 0x7f19180cd260 2023-07-14T11:56:04.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f191d719700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19180c8610 con 0x7f19180b0ee0 2023-07-14T11:56:04.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f19177fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19180d07e0 0x7f19180c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:04.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f19177fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19180d07e0 0x7f19180c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45734/0 (socket says 172.21.15.104:45734) 2023-07-14T11:56:04.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f19177fe700 1 -- 172.21.15.104:0/1716994024 learned_addr learned my addr 172.21.15.104:0/1716994024 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:04.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f19167fc700 1 --2- 172.21.15.104:0/1716994024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19180cd260 0x7f19180c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:04.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f1916ffd700 1 --2- 172.21.15.104:0/1716994024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19180b0ee0 0x7f19181f1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:04.404 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f19177fe700 1 -- 172.21.15.104:0/1716994024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19180b0ee0 msgr2=0x7f19181f1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:04.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f19177fe700 1 --2- 172.21.15.104:0/1716994024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19180b0ee0 0x7f19181f1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:04.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f19177fe700 1 -- 172.21.15.104:0/1716994024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19180cd260 msgr2=0x7f19180c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:04.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f19177fe700 1 --2- 172.21.15.104:0/1716994024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19180cd260 0x7f19180c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:04.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f19177fe700 1 -- 172.21.15.104:0/1716994024 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1910009a70 con 0x7f19180d07e0 2023-07-14T11:56:04.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f19167fc700 1 --2- 172.21.15.104:0/1716994024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19180cd260 0x7f19180c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:04.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.413+0000 7f19177fe700 1 --2- 172.21.15.104:0/1716994024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19180d07e0 0x7f19180c9b10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f191000b370 tx=0x7f1910004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:04.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.417+0000 7f18ff7fe700 1 -- 172.21.15.104:0/1716994024 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19100048e0 con 0x7f19180d07e0 2023-07-14T11:56:04.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.417+0000 7f18ff7fe700 1 -- 172.21.15.104:0/1716994024 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1910020070 con 0x7f19180d07e0 2023-07-14T11:56:04.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.417+0000 7f18ff7fe700 1 -- 172.21.15.104:0/1716994024 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19100276c0 con 0x7f19180d07e0 2023-07-14T11:56:04.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.417+0000 7f191d719700 1 -- 172.21.15.104:0/1716994024 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19180c88f0 con 0x7f19180d07e0 2023-07-14T11:56:04.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.417+0000 7f191d719700 1 -- 172.21.15.104:0/1716994024 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19180c7600 con 0x7f19180d07e0 2023-07-14T11:56:04.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.417+0000 7f191d719700 1 -- 172.21.15.104:0/1716994024 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1918148460 con 0x7f19180d07e0 2023-07-14T11:56:04.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.421+0000 7f18ff7fe700 1 -- 172.21.15.104:0/1716994024 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1910012430 con 0x7f19180d07e0 2023-07-14T11:56:04.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.421+0000 7f18ff7fe700 1 --2- 172.21.15.104:0/1716994024 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19040662f0 0x7f19040687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:04.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.421+0000 7f18ff7fe700 1 -- 172.21.15.104:0/1716994024 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f1910022070 con 0x7f19180d07e0 2023-07-14T11:56:04.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.421+0000 7f1916ffd700 1 --2- 172.21.15.104:0/1716994024 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19040662f0 0x7f19040687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:04.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.421+0000 7f1916ffd700 1 --2- 172.21.15.104:0/1716994024 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19040662f0 0x7f19040687a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1908007fd0 tx=0x7f1908009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:04.415 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.429+0000 7f18ff7fe700 1 -- 172.21.15.104:0/1716994024 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f1918148460 con 0x7f19180d07e0 2023-07-14T11:56:04.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.597+0000 7f191d719700 1 -- 172.21.15.104:0/1716994024 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1918148460 con 0x7f19180d07e0 2023-07-14T11:56:04.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.605+0000 7f18ff7fe700 1 -- 172.21.15.104:0/1716994024 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1918148460 con 0x7f19180d07e0 2023-07-14T11:56:04.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.609+0000 7f191d719700 1 -- 172.21.15.104:0/1716994024 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19040662f0 msgr2=0x7f19040687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:04.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.609+0000 7f191d719700 1 --2- 172.21.15.104:0/1716994024 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19040662f0 0x7f19040687a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1908007fd0 tx=0x7f1908009040 comp rx=0 tx=0).stop 2023-07-14T11:56:04.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.609+0000 7f191d719700 1 -- 172.21.15.104:0/1716994024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19180d07e0 msgr2=0x7f19180c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:04.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.609+0000 7f191d719700 1 --2- 172.21.15.104:0/1716994024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19180d07e0 0x7f19180c9b10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f191000b370 tx=0x7f1910004af0 comp rx=0 tx=0).stop 2023-07-14T11:56:04.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.609+0000 7f191d719700 1 -- 172.21.15.104:0/1716994024 shutdown_connections 2023-07-14T11:56:04.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.609+0000 7f191d719700 1 --2- 172.21.15.104:0/1716994024 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19040662f0 0x7f19040687a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:04.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.609+0000 7f191d719700 1 --2- 172.21.15.104:0/1716994024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19180d07e0 0x7f19180c9b10 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:04.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.609+0000 7f191d719700 1 --2- 172.21.15.104:0/1716994024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19180cd260 0x7f19180c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:04.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.609+0000 7f191d719700 1 --2- 172.21.15.104:0/1716994024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19180b0ee0 0x7f19181f1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:04.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.609+0000 7f191d719700 1 -- 172.21.15.104:0/1716994024 >> 172.21.15.104:0/1716994024 conn(0x7f1918060c40 msgr2=0x7f19180b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:04.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.609+0000 7f191d719700 1 -- 172.21.15.104:0/1716994024 shutdown_connections 2023-07-14T11:56:04.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:04.609+0000 7f191d719700 1 -- 172.21.15.104:0/1716994024 wait complete. 2023-07-14T11:56:04.616 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-14T11:56:04.617 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-14T11:56:04.661 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-14T11:56:04.705 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-14T11:56:04.706 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:56:04.706 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:56:04.723 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352340k,nr_inodes=4088085,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275392k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275388k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275392k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-07-14T11:56:04.724 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:56:04.724 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:56:04.770 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352176k,nr_inodes=4088044,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18484 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275352k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-07-14T11:56:04.770 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:56:04.771 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:56:04.789 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352340k,nr_inodes=4088085,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275392k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275388k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275392k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-07-14T11:56:04.789 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:56:04.790 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:56:04.831 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352176k,nr_inodes=4088044,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18484 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275352k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-07-14T11:56:04.832 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-14T11:56:04.832 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:56:05.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.021+0000 7f2c99199700 1 Processor -- start 2023-07-14T11:56:05.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.021+0000 7f2c99199700 1 -- start start 2023-07-14T11:56:05.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.021+0000 7f2c99199700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2c940b1530 0x7f2c940d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:05.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c99199700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c940d1260 0x7f2c940b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:05.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c99199700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c9415c2f0 con 0x7f2c9415bf20 2023-07-14T11:56:05.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c99199700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c940b5b30 con 0x7f2c940b1530 2023-07-14T11:56:05.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c99199700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c94070830 con 0x7f2c940d1260 2023-07-14T11:56:05.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c8a59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2c940b1530 0x7f2c940d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:05.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c8a59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2c940b1530 0x7f2c940d0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45738/0 (socket says 172.21.15.104:45738) 2023-07-14T11:56:05.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c8a59c700 1 -- 172.21.15.104:0/2961370464 learned_addr learned my addr 172.21.15.104:0/2961370464 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:05.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c9359e700 1 --2- 172.21.15.104:0/2961370464 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c940d1260 0x7f2c940b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:05.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c9359e700 1 -- 172.21.15.104:0/2961370464 >> v2:172.21.15.104:3300/0 conn(0x7f2c940b1530 msgr2=0x7f2c940d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:05.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c9359e700 1 --2- 172.21.15.104:0/2961370464 >> v2:172.21.15.104:3300/0 conn(0x7f2c940b1530 0x7f2c940d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:05.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c9359e700 1 -- 172.21.15.104:0/2961370464 >> v1:172.21.15.121:6789/0 conn(0x7f2c9415bf20 legacy=0x7f2c940b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:05.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c9359e700 1 -- 172.21.15.104:0/2961370464 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c94070b10 con 0x7f2c940d1260 2023-07-14T11:56:05.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c8a59c700 1 --2- 172.21.15.104:0/2961370464 >> v2:172.21.15.104:3300/0 conn(0x7f2c940b1530 0x7f2c940d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:05.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c9359e700 1 --2- 172.21.15.104:0/2961370464 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c940d1260 0x7f2c940b4050 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009d90 tx=0x7f2c8c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f680b07f1ba02d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:05.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c9259c700 1 -- 172.21.15.104:0/2961370464 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00e070 con 0x7f2c940d1260 2023-07-14T11:56:05.014 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c9259c700 1 -- 172.21.15.104:0/2961370464 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c00bb20 con 0x7f2c940d1260 2023-07-14T11:56:05.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c9259c700 1 -- 172.21.15.104:0/2961370464 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c012430 con 0x7f2c940d1260 2023-07-14T11:56:05.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c99199700 1 -- 172.21.15.104:0/2961370464 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c940d1260 msgr2=0x7f2c940b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:05.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c99199700 1 --2- 172.21.15.104:0/2961370464 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c940d1260 0x7f2c940b4050 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009d90 tx=0x7f2c8c00b330 comp rx=0 tx=0).stop 2023-07-14T11:56:05.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c99199700 1 -- 172.21.15.104:0/2961370464 shutdown_connections 2023-07-14T11:56:05.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c99199700 1 --2- 172.21.15.104:0/2961370464 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c940d1260 0x7f2c940b4050 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:05.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c99199700 1 --2- 172.21.15.104:0/2961370464 >> v2:172.21.15.104:3300/0 conn(0x7f2c940b1530 0x7f2c940d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:05.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.025+0000 7f2c99199700 1 -- 172.21.15.104:0/2961370464 >> 172.21.15.104:0/2961370464 conn(0x7f2c9406fc80 msgr2=0x7f2c94070080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:05.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.029+0000 7f2c99199700 1 -- 172.21.15.104:0/2961370464 shutdown_connections 2023-07-14T11:56:05.016 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.029+0000 7f2c99199700 1 -- 172.21.15.104:0/2961370464 wait complete. 2023-07-14T11:56:05.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.029+0000 7f2c99199700 1 Processor -- start 2023-07-14T11:56:05.017 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.029+0000 7f2c99199700 1 -- start start 2023-07-14T11:56:05.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.029+0000 7f2c99199700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c940b1530 0x7f2c941f4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:05.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.029+0000 7f2c99199700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c940d1260 0x7f2c941f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:05.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.029+0000 7f2c99199700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c9415bf20 0x7f2c9415e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:05.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.029+0000 7f2c99199700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9415f440 con 0x7f2c940b1530 2023-07-14T11:56:05.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.029+0000 7f2c99199700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c940c8fb0 con 0x7f2c940d1260 2023-07-14T11:56:05.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.029+0000 7f2c99199700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9415f160 con 0x7f2c9415bf20 2023-07-14T11:56:05.020 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c92d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c940b1530 0x7f2c941f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:05.020 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c92d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c940b1530 0x7f2c941f4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45752/0 (socket says 172.21.15.104:45752) 2023-07-14T11:56:05.020 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c92d9d700 1 -- 172.21.15.104:0/644970510 learned_addr learned my addr 172.21.15.104:0/644970510 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:05.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c8a59c700 1 --2- 172.21.15.104:0/644970510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c940d1260 0x7f2c941f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:05.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c9359e700 1 --2- 172.21.15.104:0/644970510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c9415bf20 0x7f2c9415e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:05.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c92d9d700 1 -- 172.21.15.104:0/644970510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c9415bf20 msgr2=0x7f2c9415e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:05.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c92d9d700 1 --2- 172.21.15.104:0/644970510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c9415bf20 0x7f2c9415e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:05.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c92d9d700 1 -- 172.21.15.104:0/644970510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c940d1260 msgr2=0x7f2c941f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:05.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c92d9d700 1 --2- 172.21.15.104:0/644970510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c940d1260 0x7f2c941f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:05.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c92d9d700 1 -- 172.21.15.104:0/644970510 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7c009570 con 0x7f2c940b1530 2023-07-14T11:56:05.021 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c92d9d700 1 --2- 172.21.15.104:0/644970510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c940b1530 0x7f2c941f4730 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400c500 tx=0x7f2c8400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:05.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c8bfff700 1 -- 172.21.15.104:0/644970510 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c840052d0 con 0x7f2c940b1530 2023-07-14T11:56:05.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c8bfff700 1 -- 172.21.15.104:0/644970510 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c84005430 con 0x7f2c940b1530 2023-07-14T11:56:05.022 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c99199700 1 -- 172.21.15.104:0/644970510 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8c009a70 con 0x7f2c940b1530 2023-07-14T11:56:05.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c8bfff700 1 -- 172.21.15.104:0/644970510 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c840055a0 con 0x7f2c940b1530 2023-07-14T11:56:05.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c99199700 1 -- 172.21.15.104:0/644970510 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c940b3290 con 0x7f2c940b1530 2023-07-14T11:56:05.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.033+0000 7f2c99199700 1 -- 172.21.15.104:0/644970510 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c940ccf80 con 0x7f2c940b1530 2023-07-14T11:56:05.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.037+0000 7f2c8bfff700 1 -- 172.21.15.104:0/644970510 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2c840038e0 con 0x7f2c940b1530 2023-07-14T11:56:05.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.037+0000 7f2c8bfff700 1 --2- 172.21.15.104:0/644970510 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2c74066320 0x7f2c740687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:05.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.037+0000 7f2c8bfff700 1 -- 172.21.15.104:0/644970510 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f2c84082fe0 con 0x7f2c940b1530 2023-07-14T11:56:05.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.037+0000 7f2c8a59c700 1 --2- 172.21.15.104:0/644970510 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2c74066320 0x7f2c740687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:05.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.037+0000 7f2c8a59c700 1 --2- 172.21.15.104:0/644970510 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2c74066320 0x7f2c740687d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2c940b5020 tx=0x7f2c7c0092e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:05.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.041+0000 7f2c8bfff700 1 -- 172.21.15.104:0/644970510 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f2c940ccf80 con 0x7f2c940b1530 2023-07-14T11:56:05.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.245+0000 7f2c99199700 1 -- 172.21.15.104:0/644970510 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c940ccf80 con 0x7f2c940b1530 2023-07-14T11:56:05.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.245+0000 7f2c8bfff700 1 -- 172.21.15.104:0/644970510 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f2c84017df0 con 0x7f2c940b1530 2023-07-14T11:56:05.234 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:05.234 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3696540262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3696540262},{"type":"v1","addr":"172.21.15.104:6835","nonce":3696540262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/684052037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":684052037},{"type":"v1","addr":"172.21.15.121:6835","nonce":684052037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/1267091306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":1267091306},{"type":"v1","addr":"172.21.15.121:6833","nonce":1267091306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":6237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2933017090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2933017090},{"type":"v1","addr":"172.21.15.104:6837","nonce":2933017090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[]} 2023-07-14T11:56:05.236 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.249+0000 7f2c99199700 1 -- 172.21.15.104:0/644970510 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2c74066320 msgr2=0x7f2c740687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:05.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.249+0000 7f2c99199700 1 --2- 172.21.15.104:0/644970510 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2c74066320 0x7f2c740687d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2c940b5020 tx=0x7f2c7c0092e0 comp rx=0 tx=0).stop 2023-07-14T11:56:05.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.249+0000 7f2c99199700 1 -- 172.21.15.104:0/644970510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c940b1530 msgr2=0x7f2c941f4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:05.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.249+0000 7f2c99199700 1 --2- 172.21.15.104:0/644970510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c940b1530 0x7f2c941f4730 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400c500 tx=0x7f2c8400c810 comp rx=0 tx=0).stop 2023-07-14T11:56:05.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.249+0000 7f2c99199700 1 -- 172.21.15.104:0/644970510 shutdown_connections 2023-07-14T11:56:05.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.249+0000 7f2c99199700 1 --2- 172.21.15.104:0/644970510 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2c74066320 0x7f2c740687d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:05.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.249+0000 7f2c99199700 1 --2- 172.21.15.104:0/644970510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c9415bf20 0x7f2c9415e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:05.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.249+0000 7f2c99199700 1 --2- 172.21.15.104:0/644970510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c940d1260 0x7f2c941f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:05.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.249+0000 7f2c99199700 1 --2- 172.21.15.104:0/644970510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c940b1530 0x7f2c941f4730 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:05.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.249+0000 7f2c99199700 1 -- 172.21.15.104:0/644970510 >> 172.21.15.104:0/644970510 conn(0x7f2c9406fc80 msgr2=0x7f2c94070050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:05.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.249+0000 7f2c99199700 1 -- 172.21.15.104:0/644970510 shutdown_connections 2023-07-14T11:56:05.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.249+0000 7f2c99199700 1 -- 172.21.15.104:0/644970510 wait complete. 2023-07-14T11:56:05.240 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 21 2023-07-14T11:56:05.259 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-14T11:56:05.259 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-14T11:56:05.260 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-14T11:56:05.260 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:56:05.260 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-14T11:56:05.260 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-14T11:56:05.261 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-14T11:56:05.261 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:56:05.261 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-14T11:56:05.261 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-14T11:56:05.261 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-14T11:56:05.262 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:56:05.262 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-14T11:56:05.262 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-14T11:56:05.262 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-14T11:56:05.262 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:56:05.282 INFO:tasks.ceph.mds.a:Stopped 2023-07-14T11:56:05.283 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-14T11:56:05.285 INFO:tasks.ceph.mds.b:Stopped 2023-07-14T11:56:05.285 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-14T11:56:05.286 INFO:tasks.ceph.mds.c:Stopped 2023-07-14T11:56:05.286 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-14T11:56:05.287 INFO:tasks.ceph.mds.d:Stopped 2023-07-14T11:56:05.287 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-14T11:56:05.336 INFO:tasks.ceph.mds.a:Started 2023-07-14T11:56:05.337 DEBUG:teuthology.parallel:result is None 2023-07-14T11:56:05.341 INFO:tasks.ceph.mds.b:Started 2023-07-14T11:56:05.342 DEBUG:teuthology.parallel:result is None 2023-07-14T11:56:05.343 INFO:tasks.ceph.mds.c:Started 2023-07-14T11:56:05.343 DEBUG:teuthology.parallel:result is None 2023-07-14T11:56:05.345 INFO:tasks.ceph.mds.d:Started 2023-07-14T11:56:05.345 DEBUG:teuthology.parallel:result is None 2023-07-14T11:56:05.346 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-14T11:56:05.454 INFO:tasks.ceph.mds.c.smithi104.stdout:starting mds.c at 2023-07-14T11:56:05.455 INFO:tasks.ceph.mds.a.smithi104.stdout:starting mds.a at 2023-07-14T11:56:05.459 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2023-07-14T11:56:05.467 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2023-07-14T11:56:05.521 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.533+0000 7ff7fd6b1700 1 Processor -- start 2023-07-14T11:56:05.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.533+0000 7ff7fd6b1700 1 -- start start 2023-07-14T11:56:05.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.537+0000 7ff7fd6b1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff7f815bff0 0x7ff7f80ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:05.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.537+0000 7ff7fd6b1700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff7f80cf1c0 0x7ff7f80afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:05.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.537+0000 7ff7fd6b1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f80af0a0 con 0x7ff7f80cd070 2023-07-14T11:56:05.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.537+0000 7ff7fd6b1700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f80cd4f0 con 0x7ff7f80cf1c0 2023-07-14T11:56:05.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.537+0000 7ff7fd6b1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f80cf720 con 0x7ff7f815bff0 2023-07-14T11:56:05.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.537+0000 7ff7f77fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff7f80cf1c0 0x7ff7f80afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:05.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.537+0000 7ff7f77fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff7f80cf1c0 0x7ff7f80afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45780/0 (socket says 172.21.15.104:45780) 2023-07-14T11:56:05.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.537+0000 7ff7f77fe700 1 -- 172.21.15.104:0/246453634 learned_addr learned my addr 172.21.15.104:0/246453634 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:05.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.537+0000 7ff7f6ffd700 1 --2- 172.21.15.104:0/246453634 >> v2:172.21.15.121:3300/0 conn(0x7ff7f815bff0 0x7ff7f80ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:05.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.537+0000 7ff7f77fe700 1 -- 172.21.15.104:0/246453634 >> v2:172.21.15.121:3300/0 conn(0x7ff7f815bff0 msgr2=0x7ff7f80ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:05.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.537+0000 7ff7f77fe700 1 --2- 172.21.15.104:0/246453634 >> v2:172.21.15.121:3300/0 conn(0x7ff7f815bff0 0x7ff7f80ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:05.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.537+0000 7ff7f77fe700 1 -- 172.21.15.104:0/246453634 >> v1:172.21.15.121:6789/0 conn(0x7ff7f80cd070 legacy=0x7ff7f80ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:05.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.537+0000 7ff7f77fe700 1 -- 172.21.15.104:0/246453634 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f80af2f0 con 0x7ff7f80cf1c0 2023-07-14T11:56:05.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.537+0000 7ff7f6ffd700 1 --2- 172.21.15.104:0/246453634 >> v2:172.21.15.121:3300/0 conn(0x7ff7f815bff0 0x7ff7f80ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:05.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.537+0000 7ff7f77fe700 1 --2- 172.21.15.104:0/246453634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f80cf1c0 0x7ff7f80afdd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff7f808ac70 tx=0x7ff7f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9289aa12cdfd03e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:05.528 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.541+0000 7ff7f67fc700 1 -- 172.21.15.104:0/246453634 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f000e070 con 0x7ff7f80cf1c0 2023-07-14T11:56:05.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.541+0000 7ff7f67fc700 1 -- 172.21.15.104:0/246453634 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f0003a40 con 0x7ff7f80cf1c0 2023-07-14T11:56:05.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.541+0000 7ff7f67fc700 1 -- 172.21.15.104:0/246453634 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0003df0 con 0x7ff7f80cf1c0 2023-07-14T11:56:05.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.541+0000 7ff7fd6b1700 1 -- 172.21.15.104:0/246453634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f80cf1c0 msgr2=0x7ff7f80afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:05.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.541+0000 7ff7fd6b1700 1 --2- 172.21.15.104:0/246453634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f80cf1c0 0x7ff7f80afdd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff7f808ac70 tx=0x7ff7f000b330 comp rx=0 tx=0).stop 2023-07-14T11:56:05.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.541+0000 7ff7fd6b1700 1 -- 172.21.15.104:0/246453634 shutdown_connections 2023-07-14T11:56:05.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.541+0000 7ff7fd6b1700 1 --2- 172.21.15.104:0/246453634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f80cf1c0 0x7ff7f80afdd0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:05.529 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.541+0000 7ff7fd6b1700 1 --2- 172.21.15.104:0/246453634 >> v2:172.21.15.121:3300/0 conn(0x7ff7f815bff0 0x7ff7f80ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:05.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.541+0000 7ff7fd6b1700 1 -- 172.21.15.104:0/246453634 >> 172.21.15.104:0/246453634 conn(0x7ff7f8060c40 msgr2=0x7ff7f809b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:05.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.541+0000 7ff7fd6b1700 1 -- 172.21.15.104:0/246453634 shutdown_connections 2023-07-14T11:56:05.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.541+0000 7ff7fd6b1700 1 -- 172.21.15.104:0/246453634 wait complete. 2023-07-14T11:56:05.530 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7fd6b1700 1 Processor -- start 2023-07-14T11:56:05.531 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7fd6b1700 1 -- start start 2023-07-14T11:56:05.532 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7fd6b1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f80cd070 0x7ff7f81f2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:05.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7fd6b1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80cf1c0 0x7ff7f80b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:05.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7fd6b1700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f815bff0 0x7ff7f81eead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:05.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7fd6b1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f80b5850 con 0x7ff7f80cd070 2023-07-14T11:56:05.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7fd6b1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f81ef010 con 0x7ff7f80cf1c0 2023-07-14T11:56:05.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7fd6b1700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f80b5a40 con 0x7ff7f815bff0 2023-07-14T11:56:05.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7f6ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f80cd070 0x7ff7f81f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:05.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7f6ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f80cd070 0x7ff7f81f2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45782/0 (socket says 172.21.15.104:45782) 2023-07-14T11:56:05.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7f6ffd700 1 -- 172.21.15.104:0/1555334138 learned_addr learned my addr 172.21.15.104:0/1555334138 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:05.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7f77fe700 1 --2- 172.21.15.104:0/1555334138 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f815bff0 0x7ff7f81eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:05.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7ee7fc700 1 --2- 172.21.15.104:0/1555334138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80cf1c0 0x7ff7f80b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:05.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7f6ffd700 1 -- 172.21.15.104:0/1555334138 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f815bff0 msgr2=0x7ff7f81eead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:05.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7f6ffd700 1 --2- 172.21.15.104:0/1555334138 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f815bff0 0x7ff7f81eead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:05.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7f6ffd700 1 -- 172.21.15.104:0/1555334138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80cf1c0 msgr2=0x7ff7f80b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:05.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7f6ffd700 1 --2- 172.21.15.104:0/1555334138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80cf1c0 0x7ff7f80b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:05.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7f6ffd700 1 -- 172.21.15.104:0/1555334138 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f0009b30 con 0x7ff7f80cd070 2023-07-14T11:56:05.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7f6ffd700 1 --2- 172.21.15.104:0/1555334138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f80cd070 0x7ff7f81f2190 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800c7c0 tx=0x7ff7e800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:05.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7effff700 1 -- 172.21.15.104:0/1555334138 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8011800 con 0x7ff7f80cd070 2023-07-14T11:56:05.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7effff700 1 -- 172.21.15.104:0/1555334138 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e8011960 con 0x7ff7f80cd070 2023-07-14T11:56:05.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7effff700 1 -- 172.21.15.104:0/1555334138 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8011cf0 con 0x7ff7f80cd070 2023-07-14T11:56:05.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.545+0000 7ff7fd6b1700 1 -- 172.21.15.104:0/1555334138 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f80b5d20 con 0x7ff7f80cd070 2023-07-14T11:56:05.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.549+0000 7ff7fd6b1700 1 -- 172.21.15.104:0/1555334138 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f80b62e0 con 0x7ff7f80cd070 2023-07-14T11:56:05.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.549+0000 7ff7fd6b1700 1 -- 172.21.15.104:0/1555334138 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f80ccb30 con 0x7ff7f80cd070 2023-07-14T11:56:05.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.549+0000 7ff7effff700 1 -- 172.21.15.104:0/1555334138 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff7e8021350 con 0x7ff7f80cd070 2023-07-14T11:56:05.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.553+0000 7ff7effff700 1 --2- 172.21.15.104:0/1555334138 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff7d80662f0 0x7ff7d80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:05.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.553+0000 7ff7effff700 1 -- 172.21.15.104:0/1555334138 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5616+0+0 (secure 0 0 0) 0x7ff7e8083ad0 con 0x7ff7f80cd070 2023-07-14T11:56:05.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.553+0000 7ff7ee7fc700 1 --2- 172.21.15.104:0/1555334138 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff7d80662f0 0x7ff7d80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:05.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.553+0000 7ff7ee7fc700 1 --2- 172.21.15.104:0/1555334138 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff7d80662f0 0x7ff7d80687a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff7f815f490 tx=0x7ff7e0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:05.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.557+0000 7ff7effff700 1 -- 172.21.15.104:0/1555334138 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff7e8054bb0 con 0x7ff7f80cd070 2023-07-14T11:56:05.730 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:05.741+0000 7ff7fd6b1700 1 -- 172.21.15.104:0/1555334138 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff7f804e500 con 0x7ff7f80cd070 2023-07-14T11:56:06.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.217+0000 7ff7effff700 1 -- 172.21.15.104:0/1555334138 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v53) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff7e8054740 con 0x7ff7f80cd070 2023-07-14T11:56:06.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.221+0000 7ff7fd6b1700 1 -- 172.21.15.104:0/1555334138 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff7d80662f0 msgr2=0x7ff7d80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:06.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.221+0000 7ff7fd6b1700 1 --2- 172.21.15.104:0/1555334138 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff7d80662f0 0x7ff7d80687a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff7f815f490 tx=0x7ff7e0008040 comp rx=0 tx=0).stop 2023-07-14T11:56:06.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.221+0000 7ff7fd6b1700 1 -- 172.21.15.104:0/1555334138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f80cd070 msgr2=0x7ff7f81f2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:06.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.221+0000 7ff7fd6b1700 1 --2- 172.21.15.104:0/1555334138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f80cd070 0x7ff7f81f2190 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800c7c0 tx=0x7ff7e800cb80 comp rx=0 tx=0).stop 2023-07-14T11:56:06.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.225+0000 7ff7fd6b1700 1 -- 172.21.15.104:0/1555334138 shutdown_connections 2023-07-14T11:56:06.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.225+0000 7ff7fd6b1700 1 --2- 172.21.15.104:0/1555334138 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff7d80662f0 0x7ff7d80687a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.225+0000 7ff7fd6b1700 1 --2- 172.21.15.104:0/1555334138 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7f815bff0 0x7ff7f81eead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.225+0000 7ff7fd6b1700 1 --2- 172.21.15.104:0/1555334138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80cf1c0 0x7ff7f80b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.225+0000 7ff7fd6b1700 1 --2- 172.21.15.104:0/1555334138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f80cd070 0x7ff7f81f2190 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.225+0000 7ff7fd6b1700 1 -- 172.21.15.104:0/1555334138 >> 172.21.15.104:0/1555334138 conn(0x7ff7f8060c40 msgr2=0x7ff7f80b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:06.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.225+0000 7ff7fd6b1700 1 -- 172.21.15.104:0/1555334138 shutdown_connections 2023-07-14T11:56:06.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.225+0000 7ff7fd6b1700 1 -- 172.21.15.104:0/1555334138 wait complete. 2023-07-14T11:56:06.214 INFO:teuthology.orchestra.run.smithi104.stderr: removed all blocklist entries 2023-07-14T11:56:06.234 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-14T11:56:06.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.417+0000 7fb17bf13700 1 Processor -- start 2023-07-14T11:56:06.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.417+0000 7fb17bf13700 1 -- start start 2023-07-14T11:56:06.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.421+0000 7fb17bf13700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb1740afd30 0x7fb1740b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:06.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.421+0000 7fb17bf13700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1740cd530 con 0x7fb1740703a0 2023-07-14T11:56:06.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.421+0000 7fb17bf13700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17415c440 con 0x7fb17415c070 2023-07-14T11:56:06.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.421+0000 7fb17bf13700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1740b06e0 con 0x7fb1740afd30 2023-07-14T11:56:06.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.421+0000 7fb17a4b0700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb1740703a0 0x7fb174070820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43754/0 (socket says 172.21.15.104:43754) 2023-07-14T11:56:06.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.421+0000 7fb17a4b0700 1 -- 172.21.15.104:0/2079741161 learned_addr learned my addr 172.21.15.104:0/2079741161 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:06.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.421+0000 7fb1794ae700 1 --2- 172.21.15.104:0/2079741161 >> v2:172.21.15.121:3300/0 conn(0x7fb1740afd30 0x7fb1740b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:06.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.421+0000 7fb178cad700 1 -- 172.21.15.104:0/2079741161 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163454156 0 0) 0x7fb1740cd530 con 0x7fb1740703a0 2023-07-14T11:56:06.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.421+0000 7fb178cad700 1 -- 172.21.15.104:0/2079741161 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1600035d0 con 0x7fb1740703a0 2023-07-14T11:56:06.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.421+0000 7fb1794ae700 1 -- 172.21.15.104:0/2079741161 >> v1:172.21.15.121:6789/0 conn(0x7fb17415c070 legacy=0x7fb1740ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:06.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.421+0000 7fb1794ae700 1 -- 172.21.15.104:0/2079741161 >> v1:172.21.15.104:6789/0 conn(0x7fb1740703a0 legacy=0x7fb174070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:06.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.421+0000 7fb1794ae700 1 -- 172.21.15.104:0/2079741161 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1740cc1b0 con 0x7fb1740afd30 2023-07-14T11:56:06.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.421+0000 7fb1794ae700 1 --2- 172.21.15.104:0/2079741161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740afd30 0x7fb1740b01a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb164009850 tx=0x7fb164009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7848f9479ca968cb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:06.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.421+0000 7fb178cad700 1 -- 172.21.15.104:0/2079741161 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164011830 con 0x7fb1740afd30 2023-07-14T11:56:06.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.425+0000 7fb178cad700 1 -- 172.21.15.104:0/2079741161 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb164011990 con 0x7fb1740afd30 2023-07-14T11:56:06.411 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.425+0000 7fb178cad700 1 -- 172.21.15.104:0/2079741161 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164011d40 con 0x7fb1740afd30 2023-07-14T11:56:06.412 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.425+0000 7fb17bf13700 1 -- 172.21.15.104:0/2079741161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740afd30 msgr2=0x7fb1740b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:06.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.425+0000 7fb17bf13700 1 --2- 172.21.15.104:0/2079741161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740afd30 0x7fb1740b01a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb164009850 tx=0x7fb164009b60 comp rx=0 tx=0).stop 2023-07-14T11:56:06.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.425+0000 7fb17bf13700 1 -- 172.21.15.104:0/2079741161 shutdown_connections 2023-07-14T11:56:06.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.425+0000 7fb17bf13700 1 --2- 172.21.15.104:0/2079741161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740afd30 0x7fb1740b01a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.425+0000 7fb17bf13700 1 -- 172.21.15.104:0/2079741161 >> 172.21.15.104:0/2079741161 conn(0x7fb1740cad30 msgr2=0x7fb1740cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:06.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.425+0000 7fb17bf13700 1 -- 172.21.15.104:0/2079741161 shutdown_connections 2023-07-14T11:56:06.413 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.425+0000 7fb17bf13700 1 -- 172.21.15.104:0/2079741161 wait complete. 2023-07-14T11:56:06.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb17bf13700 1 Processor -- start 2023-07-14T11:56:06.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb17bf13700 1 -- start start 2023-07-14T11:56:06.417 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb17bf13700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740703a0 0x7fb174157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:06.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb17bf13700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1740afd30 0x7fb17415c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:06.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb17bf13700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb17415c070 0x7fb17415cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:06.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb17bf13700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17415de50 con 0x7fb1740afd30 2023-07-14T11:56:06.418 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb17bf13700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1740b2620 con 0x7fb1740703a0 2023-07-14T11:56:06.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb17bf13700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb174157c90 con 0x7fb17415c070 2023-07-14T11:56:06.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb1794ae700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1740afd30 0x7fb17415c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:06.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb1794ae700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1740afd30 0x7fb17415c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45788/0 (socket says 172.21.15.104:45788) 2023-07-14T11:56:06.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb1794ae700 1 -- 172.21.15.104:0/3137909468 learned_addr learned my addr 172.21.15.104:0/3137909468 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:06.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb179caf700 1 --2- 172.21.15.104:0/3137909468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740703a0 0x7fb174157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:06.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb17a4b0700 1 --2- 172.21.15.104:0/3137909468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb17415c070 0x7fb17415cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:06.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb1794ae700 1 -- 172.21.15.104:0/3137909468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb17415c070 msgr2=0x7fb17415cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:06.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb1794ae700 1 --2- 172.21.15.104:0/3137909468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb17415c070 0x7fb17415cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.422 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb1794ae700 1 -- 172.21.15.104:0/3137909468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740703a0 msgr2=0x7fb174157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:06.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.429+0000 7fb1794ae700 1 --2- 172.21.15.104:0/3137909468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740703a0 0x7fb174157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.433+0000 7fb1794ae700 1 -- 172.21.15.104:0/3137909468 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1640094b0 con 0x7fb1740afd30 2023-07-14T11:56:06.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.433+0000 7fb17a4b0700 1 --2- 172.21.15.104:0/3137909468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb17415c070 0x7fb17415cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:06.423 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.433+0000 7fb1794ae700 1 --2- 172.21.15.104:0/3137909468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1740afd30 0x7fb17415c660 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb164000c00 tx=0x7fb164004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:06.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.433+0000 7fb16a7fc700 1 -- 172.21.15.104:0/3137909468 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16401aa10 con 0x7fb1740afd30 2023-07-14T11:56:06.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.433+0000 7fb16a7fc700 1 -- 172.21.15.104:0/3137909468 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16401f070 con 0x7fb1740afd30 2023-07-14T11:56:06.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.433+0000 7fb17bf13700 1 -- 172.21.15.104:0/3137909468 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17415d240 con 0x7fb1740afd30 2023-07-14T11:56:06.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.433+0000 7fb16a7fc700 1 -- 172.21.15.104:0/3137909468 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16401adb0 con 0x7fb1740afd30 2023-07-14T11:56:06.424 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.433+0000 7fb17bf13700 1 -- 172.21.15.104:0/3137909468 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17415d800 con 0x7fb1740afd30 2023-07-14T11:56:06.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.433+0000 7fb17bf13700 1 -- 172.21.15.104:0/3137909468 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17415dfc0 con 0x7fb1740afd30 2023-07-14T11:56:06.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.437+0000 7fb16a7fc700 1 -- 172.21.15.104:0/3137909468 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb16401b430 con 0x7fb1740afd30 2023-07-14T11:56:06.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.437+0000 7fb16a7fc700 1 --2- 172.21.15.104:0/3137909468 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb158066320 0x7fb1580687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:06.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.437+0000 7fb16a7fc700 1 -- 172.21.15.104:0/3137909468 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fb164084cb0 con 0x7fb1740afd30 2023-07-14T11:56:06.425 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.437+0000 7fb179caf700 1 --2- 172.21.15.104:0/3137909468 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb158066320 0x7fb1580687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:06.426 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.437+0000 7fb179caf700 1 --2- 172.21.15.104:0/3137909468 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb158066320 0x7fb1580687d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb1740af4b0 tx=0x7fb170009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:06.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.445+0000 7fb16a7fc700 1 -- 172.21.15.104:0/3137909468 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb164052330 con 0x7fb1740afd30 2023-07-14T11:56:06.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.673+0000 7fb17bf13700 1 -- 172.21.15.104:0/3137909468 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb17404dcb0 con 0x7fb1740afd30 2023-07-14T11:56:06.662 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.673+0000 7fb16a7fc700 1 -- 172.21.15.104:0/3137909468 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5298 (secure 0 0 0) 0x7fb164021090 con 0x7fb1740afd30 2023-07-14T11:56:06.663 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:06.663 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:56:06.663 INFO:teuthology.orchestra.run.smithi104.stdout: "auth_dump": [ 2023-07-14T11:56:06.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.664 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "mds.a", 2023-07-14T11:56:06.664 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDqNrFkusAwKBAAd0vgoEvYzWoPG+qOPSx7xg==", 2023-07-14T11:56:06.664 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.665 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:56:06.665 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow *", 2023-07-14T11:56:06.665 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow *", 2023-07-14T11:56:06.665 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:56:06.666 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.667 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "mds.b", 2023-07-14T11:56:06.667 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDqNrFkjoJoOxAA4nNPKT2i5JIO7aF42mYIzg==", 2023-07-14T11:56:06.667 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.667 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:56:06.667 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow *", 2023-07-14T11:56:06.668 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow *", 2023-07-14T11:56:06.668 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:56:06.668 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.668 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.668 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "mds.c", 2023-07-14T11:56:06.669 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDqNrFkRtpAMxAAq3jyn4VjOGmZrpCJVy6BYw==", 2023-07-14T11:56:06.669 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.669 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:56:06.669 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow *", 2023-07-14T11:56:06.669 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow *", 2023-07-14T11:56:06.670 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:56:06.670 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.670 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.670 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.670 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "mds.d", 2023-07-14T11:56:06.670 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDrNrFkADXNChAA5ndYnLdDxZQwMJBBT4XBtQ==", 2023-07-14T11:56:06.670 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.671 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:56:06.671 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow *", 2023-07-14T11:56:06.671 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow *", 2023-07-14T11:56:06.671 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:56:06.671 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.672 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.0", 2023-07-14T11:56:06.672 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDuNrFkvPfmNhAABqDrwKAMelrRtze4ZCW4fw==", 2023-07-14T11:56:06.672 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.672 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:56:06.673 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:56:06.673 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:56:06.673 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.673 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.673 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.1", 2023-07-14T11:56:06.673 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDyNrFkNslaCRAAntw0JcsmGTwVodNy8b7LeQ==", 2023-07-14T11:56:06.674 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.674 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:56:06.674 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:56:06.674 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:56:06.674 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.675 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.675 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.675 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.2", 2023-07-14T11:56:06.675 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQD1NrFkTkx9FhAAoxh/ll+DixRrVsqpDPdKZA==", 2023-07-14T11:56:06.676 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.676 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:56:06.676 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:56:06.676 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:56:06.676 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.676 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.677 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.677 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.3", 2023-07-14T11:56:06.677 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQD4NrFkiGAHJRAA0o2NVYbQ5uGbsLaZmojHBA==", 2023-07-14T11:56:06.677 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.677 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:56:06.677 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:56:06.678 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:56:06.678 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.678 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.678 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.678 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.4", 2023-07-14T11:56:06.679 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQD+NrFkZ+13OhAAKNDNbO4EgSTYmT1sujX2TQ==", 2023-07-14T11:56:06.679 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.679 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:56:06.679 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:56:06.680 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:56:06.680 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.680 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.680 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.680 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.5", 2023-07-14T11:56:06.680 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQACN7FkY69aDBAA6qbtQcxmW84CHeizN6bCvw==", 2023-07-14T11:56:06.680 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.681 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:56:06.681 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:56:06.681 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:56:06.681 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.681 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.681 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.681 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.6", 2023-07-14T11:56:06.682 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQAFN7Fk6roFGRAA2A0Zt8ivszAYsaGf0wQH0Q==", 2023-07-14T11:56:06.682 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.682 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:56:06.682 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:56:06.682 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:56:06.682 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.682 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.683 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.683 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "osd.7", 2023-07-14T11:56:06.683 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQAIN7FkEVH+JRAAc5EcTI6vOzyNM1iXiESbjQ==", 2023-07-14T11:56:06.683 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.683 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow profile osd", 2023-07-14T11:56:06.683 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile osd", 2023-07-14T11:56:06.683 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:56:06.684 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.684 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.684 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.684 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.0", 2023-07-14T11:56:06.684 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDrNrFkbrzjERAA+9BBzuauwUdhALUcDtd80A==", 2023-07-14T11:56:06.684 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.684 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:56:06.685 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow r", 2023-07-14T11:56:06.685 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow rw pool=cephfs_data" 2023-07-14T11:56:06.685 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.685 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.685 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.685 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.1", 2023-07-14T11:56:06.685 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDrNrFk9gt2HRAAZ0izobs6vpgfpG6isDFRbQ==", 2023-07-14T11:56:06.686 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.686 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:56:06.686 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow r", 2023-07-14T11:56:06.686 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow rw", 2023-07-14T11:56:06.686 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow rwx" 2023-07-14T11:56:06.686 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.687 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.687 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.687 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.2", 2023-07-14T11:56:06.687 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDrNrFkNmBZFxAA96kq+aaMbx6FQIJGUdN6xQ==", 2023-07-14T11:56:06.687 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.688 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:56:06.688 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow r", 2023-07-14T11:56:06.688 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow rw", 2023-07-14T11:56:06.688 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow rwx" 2023-07-14T11:56:06.688 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.688 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.689 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.689 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.3", 2023-07-14T11:56:06.689 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDrNrFkiTBxIxAAfEZF/Xe7Kc/QZHkBmHFShg==", 2023-07-14T11:56:06.689 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.690 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow", 2023-07-14T11:56:06.690 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow r", 2023-07-14T11:56:06.690 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow rw", 2023-07-14T11:56:06.690 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow rwx" 2023-07-14T11:56:06.691 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.691 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.691 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.691 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "client.admin", 2023-07-14T11:56:06.692 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDpNrFkagugORAAjrkuHLbV3wPXHZC4Y4tgag==", 2023-07-14T11:56:06.692 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.692 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow *", 2023-07-14T11:56:06.692 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr": "allow *", 2023-07-14T11:56:06.692 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow *", 2023-07-14T11:56:06.693 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:56:06.693 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.693 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.693 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.693 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "mgr.x", 2023-07-14T11:56:06.694 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDqNrFkqhD3IBAAE14Ks6MpY1PyMfpbQcUgjQ==", 2023-07-14T11:56:06.694 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.694 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow *", 2023-07-14T11:56:06.694 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile mgr", 2023-07-14T11:56:06.694 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:56:06.694 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.694 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:06.694 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:06.695 INFO:teuthology.orchestra.run.smithi104.stdout: "entity": "mgr.y", 2023-07-14T11:56:06.695 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "AQDqNrFkmLyZGhAAe2/pYD9LAp8Y6JR+IxqDuA==", 2023-07-14T11:56:06.695 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": { 2023-07-14T11:56:06.695 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": "allow *", 2023-07-14T11:56:06.695 INFO:teuthology.orchestra.run.smithi104.stdout: "mon": "allow profile mgr", 2023-07-14T11:56:06.695 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": "allow *" 2023-07-14T11:56:06.695 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.696 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:06.696 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2023-07-14T11:56:06.696 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:56:06.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.677+0000 7fb17bf13700 1 -- 172.21.15.104:0/3137909468 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb158066320 msgr2=0x7fb1580687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:06.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.681+0000 7fb17bf13700 1 --2- 172.21.15.104:0/3137909468 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb158066320 0x7fb1580687d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb1740af4b0 tx=0x7fb170009040 comp rx=0 tx=0).stop 2023-07-14T11:56:06.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.681+0000 7fb17bf13700 1 -- 172.21.15.104:0/3137909468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1740afd30 msgr2=0x7fb17415c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:06.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.681+0000 7fb17bf13700 1 --2- 172.21.15.104:0/3137909468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1740afd30 0x7fb17415c660 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb164000c00 tx=0x7fb164004770 comp rx=0 tx=0).stop 2023-07-14T11:56:06.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.681+0000 7fb17bf13700 1 -- 172.21.15.104:0/3137909468 shutdown_connections 2023-07-14T11:56:06.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.681+0000 7fb17bf13700 1 --2- 172.21.15.104:0/3137909468 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb158066320 0x7fb1580687d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.681+0000 7fb17bf13700 1 --2- 172.21.15.104:0/3137909468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1740afd30 0x7fb17415c660 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.681+0000 7fb17bf13700 1 --2- 172.21.15.104:0/3137909468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb17415c070 0x7fb17415cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.681+0000 7fb17bf13700 1 --2- 172.21.15.104:0/3137909468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740703a0 0x7fb174157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.681+0000 7fb17bf13700 1 -- 172.21.15.104:0/3137909468 >> 172.21.15.104:0/3137909468 conn(0x7fb1740cad30 msgr2=0x7fb1740ca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:06.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.681+0000 7fb17bf13700 1 -- 172.21.15.104:0/3137909468 shutdown_connections 2023-07-14T11:56:06.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.681+0000 7fb17bf13700 1 -- 172.21.15.104:0/3137909468 wait complete. 2023-07-14T11:56:06.700 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:56:06.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.885+0000 7fb28ad5b700 1 Processor -- start 2023-07-14T11:56:06.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.885+0000 7fb28ad5b700 1 -- start start 2023-07-14T11:56:06.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.885+0000 7fb28ad5b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb28406be70 0x7fb2840cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:06.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.885+0000 7fb28ad5b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb2840c1e00 0x7fb2840cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:06.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.885+0000 7fb28ad5b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2840cc1b0 con 0x7fb28406d860 2023-07-14T11:56:06.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.885+0000 7fb28ad5b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2840cd690 con 0x7fb2840c1e00 2023-07-14T11:56:06.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.885+0000 7fb28ad5b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840cd970 con 0x7fb28406be70 2023-07-14T11:56:06.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.885+0000 7fb2892f8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb2840c1e00 0x7fb2840cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:06.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.885+0000 7fb2892f8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb2840c1e00 0x7fb2840cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:37360/0 (socket says 172.21.15.104:37360) 2023-07-14T11:56:06.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.885+0000 7fb2892f8700 1 -- 172.21.15.104:0/3496178467 learned_addr learned my addr 172.21.15.104:0/3496178467 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:06.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.885+0000 7fb27bfff700 1 --2- 172.21.15.104:0/3496178467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb28406be70 0x7fb2840cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:06.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb2892f8700 1 -- 172.21.15.104:0/3496178467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb28406be70 msgr2=0x7fb2840cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:06.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb2892f8700 1 --2- 172.21.15.104:0/3496178467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb28406be70 0x7fb2840cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb2892f8700 1 -- 172.21.15.104:0/3496178467 >> v1:172.21.15.121:6789/0 conn(0x7fb28406d860 legacy=0x7fb28406dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:06.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb2892f8700 1 -- 172.21.15.104:0/3496178467 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2840cdc50 con 0x7fb2840c1e00 2023-07-14T11:56:06.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb2892f8700 1 --2- 172.21.15.104:0/3496178467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2840c1e00 0x7fb2840cc640 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009e80 tx=0x7fb27c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a591316d2845c787 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:06.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb27b7fe700 1 -- 172.21.15.104:0/3496178467 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c00e070 con 0x7fb2840c1e00 2023-07-14T11:56:06.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb27b7fe700 1 -- 172.21.15.104:0/3496178467 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27c00b8d0 con 0x7fb2840c1e00 2023-07-14T11:56:06.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb27b7fe700 1 -- 172.21.15.104:0/3496178467 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c00bc80 con 0x7fb2840c1e00 2023-07-14T11:56:06.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb28ad5b700 1 -- 172.21.15.104:0/3496178467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2840c1e00 msgr2=0x7fb2840cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:06.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb28ad5b700 1 --2- 172.21.15.104:0/3496178467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2840c1e00 0x7fb2840cc640 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009e80 tx=0x7fb27c00b330 comp rx=0 tx=0).stop 2023-07-14T11:56:06.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb28ad5b700 1 -- 172.21.15.104:0/3496178467 shutdown_connections 2023-07-14T11:56:06.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb28ad5b700 1 --2- 172.21.15.104:0/3496178467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2840c1e00 0x7fb2840cc640 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb28ad5b700 1 --2- 172.21.15.104:0/3496178467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb28406be70 0x7fb2840cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb28ad5b700 1 -- 172.21.15.104:0/3496178467 >> 172.21.15.104:0/3496178467 conn(0x7fb2840cad30 msgr2=0x7fb2840cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:06.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb28ad5b700 1 -- 172.21.15.104:0/3496178467 shutdown_connections 2023-07-14T11:56:06.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.889+0000 7fb28ad5b700 1 -- 172.21.15.104:0/3496178467 wait complete. 2023-07-14T11:56:06.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.893+0000 7fb28ad5b700 1 Processor -- start 2023-07-14T11:56:06.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.893+0000 7fb28ad5b700 1 -- start start 2023-07-14T11:56:06.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb28ad5b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb28406be70 0x7fb2841f0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:06.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb28ad5b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb28406d860 0x7fb2840c2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:06.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb28ad5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2840c1e00 0x7fb2840c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:06.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb28ad5b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840ab8f0 con 0x7fb28406be70 2023-07-14T11:56:06.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb28ad5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2841f11c0 con 0x7fb2840c1e00 2023-07-14T11:56:06.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb28ad5b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28415a820 con 0x7fb28406d860 2023-07-14T11:56:06.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb288af7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb28406be70 0x7fb2841f0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:06.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb288af7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb28406be70 0x7fb2841f0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45794/0 (socket says 172.21.15.104:45794) 2023-07-14T11:56:06.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb288af7700 1 -- 172.21.15.104:0/3692888242 learned_addr learned my addr 172.21.15.104:0/3692888242 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:06.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb2892f8700 1 --2- 172.21.15.104:0/3692888242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2840c1e00 0x7fb2840c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:06.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb27bfff700 1 --2- 172.21.15.104:0/3692888242 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb28406d860 0x7fb2840c2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:06.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb288af7700 1 -- 172.21.15.104:0/3692888242 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb28406d860 msgr2=0x7fb2840c2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:06.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb288af7700 1 --2- 172.21.15.104:0/3692888242 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb28406d860 0x7fb2840c2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb288af7700 1 -- 172.21.15.104:0/3692888242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2840c1e00 msgr2=0x7fb2840c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:06.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb288af7700 1 --2- 172.21.15.104:0/3692888242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2840c1e00 0x7fb2840c2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:06.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb288af7700 1 -- 172.21.15.104:0/3692888242 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c009b30 con 0x7fb28406be70 2023-07-14T11:56:06.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb288af7700 1 --2- 172.21.15.104:0/3692888242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb28406be70 0x7fb2841f0c80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb28000c530 tx=0x7fb28000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:06.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb2797fa700 1 -- 172.21.15.104:0/3692888242 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2800052d0 con 0x7fb28406be70 2023-07-14T11:56:06.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb2797fa700 1 -- 172.21.15.104:0/3692888242 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb280005430 con 0x7fb28406be70 2023-07-14T11:56:06.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb2797fa700 1 -- 172.21.15.104:0/3692888242 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2800055a0 con 0x7fb28406be70 2023-07-14T11:56:06.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb28ad5b700 1 -- 172.21.15.104:0/3692888242 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2840c9750 con 0x7fb28406be70 2023-07-14T11:56:06.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.897+0000 7fb28ad5b700 1 -- 172.21.15.104:0/3692888242 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2840c9d10 con 0x7fb28406be70 2023-07-14T11:56:06.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.901+0000 7fb28ad5b700 1 -- 172.21.15.104:0/3692888242 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2840c9ed0 con 0x7fb28406be70 2023-07-14T11:56:06.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.901+0000 7fb2797fa700 1 -- 172.21.15.104:0/3692888242 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb2800038e0 con 0x7fb28406be70 2023-07-14T11:56:06.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.901+0000 7fb2797fa700 1 --2- 172.21.15.104:0/3692888242 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2700662f0 0x7fb2700687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:06.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.901+0000 7fb2797fa700 1 -- 172.21.15.104:0/3692888242 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fb280082f30 con 0x7fb28406be70 2023-07-14T11:56:06.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.901+0000 7fb27bfff700 1 --2- 172.21.15.104:0/3692888242 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2700662f0 0x7fb2700687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:06.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.905+0000 7fb27bfff700 1 --2- 172.21.15.104:0/3692888242 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2700662f0 0x7fb2700687a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb2840ad4e0 tx=0x7fb274008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:06.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:06.909+0000 7fb2797fa700 1 -- 172.21.15.104:0/3692888242 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb2800505b0 con 0x7fb28406be70 2023-07-14T11:56:07.071 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.085+0000 7fb28ad5b700 1 -- 172.21.15.104:0/3692888242 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb28404dcb0 con 0x7fb28406be70 2023-07-14T11:56:07.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.085+0000 7fb2797fa700 1 -- 172.21.15.104:0/3692888242 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+14880 (secure 0 0 0) 0x7fb280053bd0 con 0x7fb28406be70 2023-07-14T11:56:07.074 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:07.075 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":55,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:56:06.232219+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:56:07.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.089+0000 7fb28ad5b700 1 -- 172.21.15.104:0/3692888242 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2700662f0 msgr2=0x7fb2700687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:07.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.089+0000 7fb28ad5b700 1 --2- 172.21.15.104:0/3692888242 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2700662f0 0x7fb2700687a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb2840ad4e0 tx=0x7fb274008040 comp rx=0 tx=0).stop 2023-07-14T11:56:07.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.089+0000 7fb28ad5b700 1 -- 172.21.15.104:0/3692888242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb28406be70 msgr2=0x7fb2841f0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:07.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.089+0000 7fb28ad5b700 1 --2- 172.21.15.104:0/3692888242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb28406be70 0x7fb2841f0c80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb28000c530 tx=0x7fb28000c840 comp rx=0 tx=0).stop 2023-07-14T11:56:07.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.089+0000 7fb28ad5b700 1 -- 172.21.15.104:0/3692888242 shutdown_connections 2023-07-14T11:56:07.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.089+0000 7fb28ad5b700 1 --2- 172.21.15.104:0/3692888242 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb2700662f0 0x7fb2700687a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.089+0000 7fb28ad5b700 1 --2- 172.21.15.104:0/3692888242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2840c1e00 0x7fb2840c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.089+0000 7fb28ad5b700 1 --2- 172.21.15.104:0/3692888242 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb28406d860 0x7fb2840c2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.089+0000 7fb28ad5b700 1 --2- 172.21.15.104:0/3692888242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb28406be70 0x7fb2841f0c80 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.089+0000 7fb28ad5b700 1 -- 172.21.15.104:0/3692888242 >> 172.21.15.104:0/3692888242 conn(0x7fb2840cad30 msgr2=0x7fb2841586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:07.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.089+0000 7fb28ad5b700 1 -- 172.21.15.104:0/3692888242 shutdown_connections 2023-07-14T11:56:07.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.093+0000 7fb28ad5b700 1 -- 172.21.15.104:0/3692888242 wait complete. 2023-07-14T11:56:07.101 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:56:07.102 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:56:07.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.297+0000 7ff2cfbdc700 1 Processor -- start 2023-07-14T11:56:07.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.297+0000 7ff2cfbdc700 1 -- start start 2023-07-14T11:56:07.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.301+0000 7ff2cfbdc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff2c8160240 0x7ff2c80cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:07.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.301+0000 7ff2cfbdc700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c8163fe0 con 0x7ff2c80b0490 2023-07-14T11:56:07.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.301+0000 7ff2cfbdc700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c80c8c30 con 0x7ff2c8160610 2023-07-14T11:56:07.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.301+0000 7ff2cfbdc700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c80ca9e0 con 0x7ff2c8160240 2023-07-14T11:56:07.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.301+0000 7ff2cd978700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff2c8160240 0x7ff2c80cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:07.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.301+0000 7ff2cd978700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff2c8160240 0x7ff2c80cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45808/0 (socket says 172.21.15.104:45808) 2023-07-14T11:56:07.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.301+0000 7ff2cd978700 1 -- 172.21.15.104:0/3423961042 learned_addr learned my addr 172.21.15.104:0/3423961042 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:07.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.301+0000 7ff2ce179700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ff2c80b0490 0x7ff2c806cf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43764/0 (socket says 172.21.15.104:43764) 2023-07-14T11:56:07.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.301+0000 7ff2cd978700 1 -- 172.21.15.104:0/3423961042 >> v1:172.21.15.121:6789/0 conn(0x7ff2c8160610 legacy=0x7ff2c80afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:07.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.301+0000 7ff2cd978700 1 -- 172.21.15.104:0/3423961042 >> v1:172.21.15.104:6789/0 conn(0x7ff2c80b0490 legacy=0x7ff2c806cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:07.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.301+0000 7ff2cd978700 1 -- 172.21.15.104:0/3423961042 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c8164230 con 0x7ff2c8160240 2023-07-14T11:56:07.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.301+0000 7ff2cd978700 1 --2- 172.21.15.104:0/3423961042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2c8160240 0x7ff2c80cdc20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009910 tx=0x7ff2c4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3176312547511b60 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:07.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.301+0000 7ff2cc976700 1 -- 172.21.15.104:0/3423961042 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c400ba60 con 0x7ff2c8160240 2023-07-14T11:56:07.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.301+0000 7ff2cc976700 1 -- 172.21.15.104:0/3423961042 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c400bbc0 con 0x7ff2c8160240 2023-07-14T11:56:07.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.301+0000 7ff2cc976700 1 -- 172.21.15.104:0/3423961042 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4009e60 con 0x7ff2c8160240 2023-07-14T11:56:07.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.305+0000 7ff2cfbdc700 1 -- 172.21.15.104:0/3423961042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2c8160240 msgr2=0x7ff2c80cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:07.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.305+0000 7ff2cfbdc700 1 --2- 172.21.15.104:0/3423961042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2c8160240 0x7ff2c80cdc20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009910 tx=0x7ff2c4009c20 comp rx=0 tx=0).stop 2023-07-14T11:56:07.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.305+0000 7ff2cfbdc700 1 -- 172.21.15.104:0/3423961042 shutdown_connections 2023-07-14T11:56:07.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.305+0000 7ff2cfbdc700 1 --2- 172.21.15.104:0/3423961042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2c8160240 0x7ff2c80cdc20 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.305+0000 7ff2cfbdc700 1 -- 172.21.15.104:0/3423961042 >> 172.21.15.104:0/3423961042 conn(0x7ff2c80cad80 msgr2=0x7ff2c80cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:07.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.305+0000 7ff2cfbdc700 1 -- 172.21.15.104:0/3423961042 shutdown_connections 2023-07-14T11:56:07.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.305+0000 7ff2cfbdc700 1 -- 172.21.15.104:0/3423961042 wait complete. 2023-07-14T11:56:07.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cfbdc700 1 Processor -- start 2023-07-14T11:56:07.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cfbdc700 1 -- start start 2023-07-14T11:56:07.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cfbdc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2c80b0490 0x7ff2c815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:07.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cfbdc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2c8160240 0x7ff2c8162e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:07.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cfbdc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2c8160610 0x7ff2c80b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:07.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cfbdc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c815af30 con 0x7ff2c8160240 2023-07-14T11:56:07.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cfbdc700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c8161d60 con 0x7ff2c8160610 2023-07-14T11:56:07.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cfbdc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c8162040 con 0x7ff2c80b0490 2023-07-14T11:56:07.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cd177700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2c8160240 0x7ff2c8162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:07.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cd177700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2c8160240 0x7ff2c8162e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45824/0 (socket says 172.21.15.104:45824) 2023-07-14T11:56:07.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cd177700 1 -- 172.21.15.104:0/2649922969 learned_addr learned my addr 172.21.15.104:0/2649922969 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:07.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cd978700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2c80b0490 0x7ff2c815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:07.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2ce179700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2c8160610 0x7ff2c80b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:07.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cd177700 1 -- 172.21.15.104:0/2649922969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2c80b0490 msgr2=0x7ff2c815a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:07.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cd177700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2c80b0490 0x7ff2c815a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cd177700 1 -- 172.21.15.104:0/2649922969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2c8160610 msgr2=0x7ff2c80b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:07.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cd177700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2c8160610 0x7ff2c80b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cd177700 1 -- 172.21.15.104:0/2649922969 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c4009570 con 0x7ff2c8160240 2023-07-14T11:56:07.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2cd978700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2c80b0490 0x7ff2c815a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:07.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.309+0000 7ff2ce179700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2c8160610 0x7ff2c80b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:07.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.313+0000 7ff2cd177700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2c8160240 0x7ff2c8162e00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800c500 tx=0x7ff2b800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:07.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.313+0000 7ff2be7fc700 1 -- 172.21.15.104:0/2649922969 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80052d0 con 0x7ff2c8160240 2023-07-14T11:56:07.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.313+0000 7ff2be7fc700 1 -- 172.21.15.104:0/2649922969 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b8005430 con 0x7ff2c8160240 2023-07-14T11:56:07.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.313+0000 7ff2cfbdc700 1 -- 172.21.15.104:0/2649922969 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c8162320 con 0x7ff2c8160240 2023-07-14T11:56:07.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.313+0000 7ff2be7fc700 1 -- 172.21.15.104:0/2649922969 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80055a0 con 0x7ff2c8160240 2023-07-14T11:56:07.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.313+0000 7ff2cfbdc700 1 -- 172.21.15.104:0/2649922969 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c81628e0 con 0x7ff2c8160240 2023-07-14T11:56:07.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.313+0000 7ff2cfbdc700 1 -- 172.21.15.104:0/2649922969 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c804dcb0 con 0x7ff2c8160240 2023-07-14T11:56:07.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.313+0000 7ff2be7fc700 1 -- 172.21.15.104:0/2649922969 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff2b80038e0 con 0x7ff2c8160240 2023-07-14T11:56:07.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.317+0000 7ff2be7fc700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2ac0662f0 0x7ff2ac0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:07.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.317+0000 7ff2be7fc700 1 -- 172.21.15.104:0/2649922969 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5530+0+0 (secure 0 0 0) 0x7ff2b8082f70 con 0x7ff2c8160240 2023-07-14T11:56:07.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.317+0000 7ff2cd978700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2ac0662f0 0x7ff2ac0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:07.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.317+0000 7ff2cd978700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2ac0662f0 0x7ff2ac0687a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4006f80 tx=0x7ff2c4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.325+0000 7ff2be7fc700 1 -- 172.21.15.104:0/2649922969 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff2b80b3720 con 0x7ff2c8160240 2023-07-14T11:56:07.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.493+0000 7ff2cfbdc700 1 -- 172.21.15.104:0/2649922969 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2c80c8830 con 0x7ff2c8160240 2023-07-14T11:56:07.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.497+0000 7ff2be7fc700 1 -- 172.21.15.104:0/2649922969 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2b8050690 con 0x7ff2c8160240 2023-07-14T11:56:07.483 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:56:07.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.497+0000 7ff2cfbdc700 1 -- 172.21.15.104:0/2649922969 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2ac0662f0 msgr2=0x7ff2ac0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:07.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.497+0000 7ff2cfbdc700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2ac0662f0 0x7ff2ac0687a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4006f80 tx=0x7ff2c4008030 comp rx=0 tx=0).stop 2023-07-14T11:56:07.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.497+0000 7ff2cfbdc700 1 -- 172.21.15.104:0/2649922969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2c8160240 msgr2=0x7ff2c8162e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:07.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.497+0000 7ff2cfbdc700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2c8160240 0x7ff2c8162e00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800c500 tx=0x7ff2b800c8c0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.497+0000 7ff2cfbdc700 1 -- 172.21.15.104:0/2649922969 shutdown_connections 2023-07-14T11:56:07.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.497+0000 7ff2cfbdc700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2ac0662f0 0x7ff2ac0687a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.497+0000 7ff2cfbdc700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2c8160240 0x7ff2c8162e00 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.497+0000 7ff2cfbdc700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2c8160610 0x7ff2c80b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.497+0000 7ff2cfbdc700 1 --2- 172.21.15.104:0/2649922969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2c80b0490 0x7ff2c815a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.497+0000 7ff2cfbdc700 1 -- 172.21.15.104:0/2649922969 >> 172.21.15.104:0/2649922969 conn(0x7ff2c80cad80 msgr2=0x7ff2c806c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:07.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.497+0000 7ff2cfbdc700 1 -- 172.21.15.104:0/2649922969 shutdown_connections 2023-07-14T11:56:07.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.497+0000 7ff2cfbdc700 1 -- 172.21.15.104:0/2649922969 wait complete. 2023-07-14T11:56:07.506 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:56:07.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.721+0000 7f219f59e700 1 Processor -- start 2023-07-14T11:56:07.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.721+0000 7f219f59e700 1 -- start start 2023-07-14T11:56:07.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.721+0000 7f219f59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f21a00cd260 0x7f21a00d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:07.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.721+0000 7f219f59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21a00d0610 0x7f21a00af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:07.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.721+0000 7f219f59e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a00d1a60 con 0x7f21a00b0ee0 2023-07-14T11:56:07.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.721+0000 7f219f59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a00d1ef0 con 0x7f21a00cd260 2023-07-14T11:56:07.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.721+0000 7f219f59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00d0e50 con 0x7f21a00d0610 2023-07-14T11:56:07.714 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219e59c700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f21a00b0ee0 0x7f21a00ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43766/0 (socket says 172.21.15.104:43766) 2023-07-14T11:56:07.715 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219e59c700 1 -- 172.21.15.104:0/72387919 learned_addr learned my addr 172.21.15.104:0/72387919 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:07.715 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219dd9b700 1 --2- 172.21.15.104:0/72387919 >> v2:172.21.15.121:3300/0 conn(0x7f21a00cd260 0x7f21a00d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:07.715 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219ed9d700 1 --2- 172.21.15.104:0/72387919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21a00d0610 0x7f21a00af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:07.716 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 -- 172.21.15.104:0/72387919 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3100206672 0 0) 0x7f21a00d1a60 con 0x7f21a00b0ee0 2023-07-14T11:56:07.716 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 -- 172.21.15.104:0/72387919 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21880035d0 con 0x7f21a00b0ee0 2023-07-14T11:56:07.716 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 -- 172.21.15.104:0/72387919 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3297821019 0 0) 0x7f21880035d0 con 0x7f21a00b0ee0 2023-07-14T11:56:07.717 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 -- 172.21.15.104:0/72387919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21a00d0610 msgr2=0x7f21a00af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:07.717 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 --2- 172.21.15.104:0/72387919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21a00d0610 0x7f21a00af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.717 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 -- 172.21.15.104:0/72387919 >> v2:172.21.15.121:3300/0 conn(0x7f21a00cd260 msgr2=0x7f21a00d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:07.717 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 --2- 172.21.15.104:0/72387919 >> v2:172.21.15.121:3300/0 conn(0x7f21a00cd260 0x7f21a00d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.717 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 -- 172.21.15.104:0/72387919 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a00d1130 con 0x7f21a00b0ee0 2023-07-14T11:56:07.718 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 -- 172.21.15.104:0/72387919 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f2190001670 con 0x7f21a00b0ee0 2023-07-14T11:56:07.718 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 -- 172.21.15.104:0/72387919 >> v1:172.21.15.104:6789/0 conn(0x7f21a00b0ee0 legacy=0x7f21a00ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:07.718 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 --2- 172.21.15.104:0/72387919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2188003a20 0x7f2188003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:07.718 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 --2- 172.21.15.104:0/72387919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21a00d0610 0x7f2188004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:07.718 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 --2- 172.21.15.104:0/72387919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2188004db0 0x7f2188005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:07.718 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 -- 172.21.15.104:0/72387919 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00d1ef0 con 0x7f2188003a20 2023-07-14T11:56:07.718 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 -- 172.21.15.104:0/72387919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00d0e50 con 0x7f2188004db0 2023-07-14T11:56:07.718 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 -- 172.21.15.104:0/72387919 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218800ba30 con 0x7f21a00d0610 2023-07-14T11:56:07.719 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219e59c700 1 --2- 172.21.15.104:0/72387919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2188003a20 0x7f2188003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:07.719 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219dd9b700 1 --2- 172.21.15.104:0/72387919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21a00d0610 0x7f2188004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:07.719 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219ed9d700 1 --2- 172.21.15.104:0/72387919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2188004db0 0x7f2188005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:07.719 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219e59c700 1 -- 172.21.15.104:0/72387919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21a00d0610 msgr2=0x7f2188004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:07.719 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219e59c700 1 --2- 172.21.15.104:0/72387919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21a00d0610 0x7f2188004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.719 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219e59c700 1 -- 172.21.15.104:0/72387919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2188004db0 msgr2=0x7f2188005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:07.719 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219e59c700 1 --2- 172.21.15.104:0/72387919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2188004db0 0x7f2188005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.720 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219e59c700 1 -- 172.21.15.104:0/72387919 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21880035d0 con 0x7f2188003a20 2023-07-14T11:56:07.720 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219e59c700 1 --2- 172.21.15.104:0/72387919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2188003a20 0x7f2188003e30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f219000c320 tx=0x7f21880058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a410179828bc4f6a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:07.720 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 -- 172.21.15.104:0/72387919 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219000ce60 con 0x7f2188003a20 2023-07-14T11:56:07.720 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 -- 172.21.15.104:0/72387919 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2190005980 con 0x7f2188003a20 2023-07-14T11:56:07.720 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.725+0000 7f219d59a700 1 -- 172.21.15.104:0/72387919 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2190005d10 con 0x7f2188003a20 2023-07-14T11:56:07.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.729+0000 7f219f59e700 1 -- 172.21.15.104:0/72387919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2188003a20 msgr2=0x7f2188003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:07.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.729+0000 7f219f59e700 1 --2- 172.21.15.104:0/72387919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2188003a20 0x7f2188003e30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f219000c320 tx=0x7f21880058a0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.729+0000 7f219ed9d700 1 -- 172.21.15.104:0/72387919 reap_dead start 2023-07-14T11:56:07.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.729+0000 7f219f59e700 1 -- 172.21.15.104:0/72387919 shutdown_connections 2023-07-14T11:56:07.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.729+0000 7f219f59e700 1 -- 172.21.15.104:0/72387919 >> 172.21.15.104:0/72387919 conn(0x7f21a0060c40 msgr2=0x7f21a009b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:07.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.729+0000 7f219f59e700 1 -- 172.21.15.104:0/72387919 shutdown_connections 2023-07-14T11:56:07.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.729+0000 7f219f59e700 1 -- 172.21.15.104:0/72387919 wait complete. 2023-07-14T11:56:07.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219f59e700 1 Processor -- start 2023-07-14T11:56:07.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219f59e700 1 -- start start 2023-07-14T11:56:07.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219f59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2188003a20 0x7f21a00b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:07.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21a00b3220 0x7f21a015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:07.724 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219f59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21a00b47e0 0x7f21a009f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:07.724 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219f59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00b2180 con 0x7f21a00b47e0 2023-07-14T11:56:07.724 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219f59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00b2460 con 0x7f21a00b3220 2023-07-14T11:56:07.724 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219f59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00b2740 con 0x7f2188003a20 2023-07-14T11:56:07.724 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219ed9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21a00b47e0 0x7f21a009f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:07.724 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219ed9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21a00b47e0 0x7f21a009f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45842/0 (socket says 172.21.15.104:45842) 2023-07-14T11:56:07.724 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219ed9d700 1 -- 172.21.15.104:0/2225123307 learned_addr learned my addr 172.21.15.104:0/2225123307 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:07.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219dd9b700 1 --2- 172.21.15.104:0/2225123307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21a00b3220 0x7f21a015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:07.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219e59c700 1 --2- 172.21.15.104:0/2225123307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2188003a20 0x7f21a00b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:07.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219ed9d700 1 -- 172.21.15.104:0/2225123307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2188003a20 msgr2=0x7f21a00b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:07.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219ed9d700 1 --2- 172.21.15.104:0/2225123307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2188003a20 0x7f21a00b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219ed9d700 1 -- 172.21.15.104:0/2225123307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21a00b3220 msgr2=0x7f21a015ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:07.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219ed9d700 1 --2- 172.21.15.104:0/2225123307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21a00b3220 0x7f21a015ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219ed9d700 1 -- 172.21.15.104:0/2225123307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a00b2a20 con 0x7f21a00b47e0 2023-07-14T11:56:07.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219dd9b700 1 --2- 172.21.15.104:0/2225123307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21a00b3220 0x7f21a015ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:07.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219ed9d700 1 --2- 172.21.15.104:0/2225123307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21a00b47e0 0x7f21a009f7c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f21a00d09e0 tx=0x7f2198011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:07.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f218effd700 1 -- 172.21.15.104:0/2225123307 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219800e040 con 0x7f21a00b47e0 2023-07-14T11:56:07.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f218effd700 1 -- 172.21.15.104:0/2225123307 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219800f040 con 0x7f21a00b47e0 2023-07-14T11:56:07.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f218effd700 1 -- 172.21.15.104:0/2225123307 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219800fdb0 con 0x7f21a00b47e0 2023-07-14T11:56:07.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219f59e700 1 -- 172.21.15.104:0/2225123307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a00b2d00 con 0x7f21a00b47e0 2023-07-14T11:56:07.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.733+0000 7f219f59e700 1 -- 172.21.15.104:0/2225123307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a009fd00 con 0x7f21a00b47e0 2023-07-14T11:56:07.727 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.737+0000 7f219f59e700 1 -- 172.21.15.104:0/2225123307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a004e500 con 0x7f21a00b47e0 2023-07-14T11:56:07.727 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.737+0000 7f218effd700 1 -- 172.21.15.104:0/2225123307 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2198022460 con 0x7f21a00b47e0 2023-07-14T11:56:07.727 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.737+0000 7f218effd700 1 --2- 172.21.15.104:0/2225123307 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2180066320 0x7f21800687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:07.727 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.737+0000 7f218effd700 1 -- 172.21.15.104:0/2225123307 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f21980840e0 con 0x7f21a00b47e0 2023-07-14T11:56:07.728 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.737+0000 7f219e59c700 1 --2- 172.21.15.104:0/2225123307 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2180066320 0x7f21800687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:07.728 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.741+0000 7f219e59c700 1 --2- 172.21.15.104:0/2225123307 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2180066320 0x7f21800687d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2190015040 tx=0x7f219000a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:07.732 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.745+0000 7f218effd700 1 -- 172.21.15.104:0/2225123307 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f21980516d0 con 0x7f21a00b47e0 2023-07-14T11:56:07.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.913+0000 7f219f59e700 1 -- 172.21.15.104:0/2225123307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f21a00b13b0 con 0x7f21a00b47e0 2023-07-14T11:56:07.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.917+0000 7f218effd700 1 -- 172.21.15.104:0/2225123307 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2198054cf0 con 0x7f21a00b47e0 2023-07-14T11:56:07.903 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:56:07.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.921+0000 7f219f59e700 1 -- 172.21.15.104:0/2225123307 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2180066320 msgr2=0x7f21800687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.921+0000 7f219f59e700 1 --2- 172.21.15.104:0/2225123307 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2180066320 0x7f21800687d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2190015040 tx=0x7f219000a000 comp rx=0 tx=0).stop 2023-07-14T11:56:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.921+0000 7f219f59e700 1 -- 172.21.15.104:0/2225123307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21a00b47e0 msgr2=0x7f21a009f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.921+0000 7f219f59e700 1 --2- 172.21.15.104:0/2225123307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21a00b47e0 0x7f21a009f7c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f21a00d09e0 tx=0x7f2198011f80 comp rx=0 tx=0).stop 2023-07-14T11:56:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.921+0000 7f219f59e700 1 -- 172.21.15.104:0/2225123307 shutdown_connections 2023-07-14T11:56:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.921+0000 7f219f59e700 1 --2- 172.21.15.104:0/2225123307 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2180066320 0x7f21800687d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.921+0000 7f219f59e700 1 --2- 172.21.15.104:0/2225123307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21a00b47e0 0x7f21a009f7c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.921+0000 7f219f59e700 1 --2- 172.21.15.104:0/2225123307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21a00b3220 0x7f21a015ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.921+0000 7f219f59e700 1 --2- 172.21.15.104:0/2225123307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2188003a20 0x7f21a00b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:07.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.921+0000 7f219f59e700 1 -- 172.21.15.104:0/2225123307 >> 172.21.15.104:0/2225123307 conn(0x7f21a0060c40 msgr2=0x7f21a00cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:07.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.921+0000 7f219f59e700 1 -- 172.21.15.104:0/2225123307 shutdown_connections 2023-07-14T11:56:07.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:07.921+0000 7f219f59e700 1 -- 172.21.15.104:0/2225123307 wait complete. 2023-07-14T11:56:07.928 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-14T11:56:08.046 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:56:08.053 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:56:08.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.097+0000 7f6c25708700 1 Processor -- start 2023-07-14T11:56:08.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.097+0000 7f6c25708700 1 -- start start 2023-07-14T11:56:08.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c25708700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c200715a0 0x7f6c201605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:08.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c25708700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6c200cd360 0x7f6c200cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:08.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c25708700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c20160b10 con 0x7f6c200ce180 2023-07-14T11:56:08.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c25708700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c200d0230 con 0x7f6c200cd360 2023-07-14T11:56:08.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c25708700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c200cfc70 con 0x7f6c200715a0 2023-07-14T11:56:08.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c1f7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6c200cd360 0x7f6c200cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:08.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c1f7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6c200cd360 0x7f6c200cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45848/0 (socket says 172.21.15.104:45848) 2023-07-14T11:56:08.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c1f7fe700 1 -- 172.21.15.104:0/3989469174 learned_addr learned my addr 172.21.15.104:0/3989469174 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:08.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c1e7fc700 1 --1- 172.21.15.104:0/3989469174 >> v1:172.21.15.121:6789/0 conn(0x7f6c200ce180 0x7f6c200ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:52370/0 (socket says 172.21.15.104:52370) 2023-07-14T11:56:08.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c1effd700 1 --2- 172.21.15.104:0/3989469174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c200715a0 0x7f6c201605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:08.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c1f7fe700 1 -- 172.21.15.104:0/3989469174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c200715a0 msgr2=0x7f6c201605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:08.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c1f7fe700 1 --2- 172.21.15.104:0/3989469174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c200715a0 0x7f6c201605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:08.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c1f7fe700 1 -- 172.21.15.104:0/3989469174 >> v1:172.21.15.121:6789/0 conn(0x7f6c200ce180 legacy=0x7f6c200ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:08.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c1f7fe700 1 -- 172.21.15.104:0/3989469174 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c200cff50 con 0x7f6c200cd360 2023-07-14T11:56:08.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c1f7fe700 1 --2- 172.21.15.104:0/3989469174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c200cd360 0x7f6c200cb3c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009e50 tx=0x7f6c1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58906e1782418ac server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:08.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.101+0000 7f6c1dffb700 1 -- 172.21.15.104:0/3989469174 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1800e070 con 0x7f6c200cd360 2023-07-14T11:56:08.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.105+0000 7f6c1dffb700 1 -- 172.21.15.104:0/3989469174 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1800b910 con 0x7f6c200cd360 2023-07-14T11:56:08.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.105+0000 7f6c1dffb700 1 -- 172.21.15.104:0/3989469174 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1800bcc0 con 0x7f6c200cd360 2023-07-14T11:56:08.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.105+0000 7f6c25708700 1 -- 172.21.15.104:0/3989469174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c200cd360 msgr2=0x7f6c200cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:08.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.105+0000 7f6c25708700 1 --2- 172.21.15.104:0/3989469174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c200cd360 0x7f6c200cb3c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009e50 tx=0x7f6c1800b330 comp rx=0 tx=0).stop 2023-07-14T11:56:08.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.105+0000 7f6c25708700 1 -- 172.21.15.104:0/3989469174 shutdown_connections 2023-07-14T11:56:08.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.105+0000 7f6c25708700 1 --2- 172.21.15.104:0/3989469174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c200cd360 0x7f6c200cb3c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:08.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.105+0000 7f6c25708700 1 --2- 172.21.15.104:0/3989469174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c200715a0 0x7f6c201605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:08.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.105+0000 7f6c25708700 1 -- 172.21.15.104:0/3989469174 >> 172.21.15.104:0/3989469174 conn(0x7f6c20060320 msgr2=0x7f6c20060720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:08.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.105+0000 7f6c25708700 1 -- 172.21.15.104:0/3989469174 shutdown_connections 2023-07-14T11:56:08.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.105+0000 7f6c25708700 1 -- 172.21.15.104:0/3989469174 wait complete. 2023-07-14T11:56:08.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.105+0000 7f6c25708700 1 Processor -- start 2023-07-14T11:56:08.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.105+0000 7f6c25708700 1 -- start start 2023-07-14T11:56:08.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c25708700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c200715a0 0x7f6c200c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:08.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c25708700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c200cd360 0x7f6c200c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:08.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c25708700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c200ce180 0x7f6c2015db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:08.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c25708700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c200c7020 con 0x7f6c200715a0 2023-07-14T11:56:08.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c25708700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2015e080 con 0x7f6c200cd360 2023-07-14T11:56:08.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c25708700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2015e320 con 0x7f6c200ce180 2023-07-14T11:56:08.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c1effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c200715a0 0x7f6c200c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:08.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c1effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c200715a0 0x7f6c200c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45850/0 (socket says 172.21.15.104:45850) 2023-07-14T11:56:08.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c1effd700 1 -- 172.21.15.104:0/1256281786 learned_addr learned my addr 172.21.15.104:0/1256281786 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:08.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c1effd700 1 -- 172.21.15.104:0/1256281786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c200ce180 msgr2=0x7f6c2015db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:08.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c1effd700 1 --2- 172.21.15.104:0/1256281786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c200ce180 0x7f6c2015db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c1effd700 1 -- 172.21.15.104:0/1256281786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c200cd360 msgr2=0x7f6c200c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c1effd700 1 --2- 172.21.15.104:0/1256281786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c200cd360 0x7f6c200c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c1effd700 1 -- 172.21.15.104:0/1256281786 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c18009b30 con 0x7f6c200715a0 2023-07-14T11:56:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c1effd700 1 --2- 172.21.15.104:0/1256281786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c200715a0 0x7f6c200c7f40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000e840 tx=0x7f6c1000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c077fe700 1 -- 172.21.15.104:0/1256281786 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c100095a0 con 0x7f6c200715a0 2023-07-14T11:56:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c077fe700 1 -- 172.21.15.104:0/1256281786 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1000c690 con 0x7f6c200715a0 2023-07-14T11:56:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c077fe700 1 -- 172.21.15.104:0/1256281786 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1000ca00 con 0x7f6c200715a0 2023-07-14T11:56:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c25708700 1 -- 172.21.15.104:0/1256281786 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2015e600 con 0x7f6c200715a0 2023-07-14T11:56:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c25708700 1 -- 172.21.15.104:0/1256281786 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c201f4dd0 con 0x7f6c200715a0 2023-07-14T11:56:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c25708700 1 -- 172.21.15.104:0/1256281786 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2004dcb0 con 0x7f6c200715a0 2023-07-14T11:56:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.109+0000 7f6c077fe700 1 -- 172.21.15.104:0/1256281786 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6c1000cb60 con 0x7f6c200715a0 2023-07-14T11:56:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.113+0000 7f6c077fe700 1 --2- 172.21.15.104:0/1256281786 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c0c0662f0 0x7f6c0c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.113+0000 7f6c077fe700 1 -- 172.21.15.104:0/1256281786 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f6c10083b70 con 0x7f6c200715a0 2023-07-14T11:56:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.113+0000 7f6c1e7fc700 1 --2- 172.21.15.104:0/1256281786 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c0c0662f0 0x7f6c0c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:08.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.113+0000 7f6c1e7fc700 1 --2- 172.21.15.104:0/1256281786 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c0c0662f0 0x7f6c0c0687a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6c201f4d30 tx=0x7f6c14007a70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:08.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.117+0000 7f6c077fe700 1 -- 172.21.15.104:0/1256281786 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f6c10054bf0 con 0x7f6c200715a0 2023-07-14T11:56:08.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.289+0000 7f6c25708700 1 -- 172.21.15.104:0/1256281786 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6c200cbb20 con 0x7f6c200715a0 2023-07-14T11:56:08.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.289+0000 7f6c077fe700 1 -- 172.21.15.104:0/1256281786 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) v1 ==== 84+0+126 (secure 0 0 0) 0x7f6c10021090 con 0x7f6c200715a0 2023-07-14T11:56:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:08.277 INFO:teuthology.orchestra.run.smithi104.stdout:[ 2023-07-14T11:56:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: "poolnum": 1, 2023-07-14T11:56:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: "poolname": ".mgr" 2023-07-14T11:56:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: { 2023-07-14T11:56:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: "poolnum": 2, 2023-07-14T11:56:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: "poolname": "rbd" 2023-07-14T11:56:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:08.278 INFO:teuthology.orchestra.run.smithi104.stdout:] 2023-07-14T11:56:08.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.293+0000 7f6c25708700 1 -- 172.21.15.104:0/1256281786 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c0c0662f0 msgr2=0x7f6c0c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:08.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.293+0000 7f6c25708700 1 --2- 172.21.15.104:0/1256281786 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c0c0662f0 0x7f6c0c0687a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6c201f4d30 tx=0x7f6c14007a70 comp rx=0 tx=0).stop 2023-07-14T11:56:08.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.293+0000 7f6c25708700 1 -- 172.21.15.104:0/1256281786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c200715a0 msgr2=0x7f6c200c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:08.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.293+0000 7f6c25708700 1 --2- 172.21.15.104:0/1256281786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c200715a0 0x7f6c200c7f40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000e840 tx=0x7f6c1000ec00 comp rx=0 tx=0).stop 2023-07-14T11:56:08.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.293+0000 7f6c25708700 1 -- 172.21.15.104:0/1256281786 shutdown_connections 2023-07-14T11:56:08.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.293+0000 7f6c25708700 1 --2- 172.21.15.104:0/1256281786 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f6c0c0662f0 0x7f6c0c0687a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:08.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.293+0000 7f6c25708700 1 --2- 172.21.15.104:0/1256281786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c200ce180 0x7f6c2015db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:08.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.293+0000 7f6c25708700 1 --2- 172.21.15.104:0/1256281786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c200cd360 0x7f6c200c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:08.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.293+0000 7f6c25708700 1 --2- 172.21.15.104:0/1256281786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c200715a0 0x7f6c200c7f40 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:08.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.293+0000 7f6c25708700 1 -- 172.21.15.104:0/1256281786 >> 172.21.15.104:0/1256281786 conn(0x7f6c20060320 msgr2=0x7f6c200606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:08.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.293+0000 7f6c25708700 1 -- 172.21.15.104:0/1256281786 shutdown_connections 2023-07-14T11:56:08.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.293+0000 7f6c25708700 1 -- 172.21.15.104:0/1256281786 wait complete. 2023-07-14T11:56:08.301 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-14T11:56:08.301 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-14T11:56:08.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.493+0000 7fd473e3e700 1 Processor -- start 2023-07-14T11:56:08.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.493+0000 7fd473e3e700 1 -- start start 2023-07-14T11:56:08.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.493+0000 7fd473e3e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46c160190 0x7fd46c160560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:08.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.493+0000 7fd473e3e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd46c0cc530 0x7fd46c164980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:08.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.493+0000 7fd473e3e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46c0c8cd0 con 0x7fd46c0c88a0 2023-07-14T11:56:08.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.493+0000 7fd473e3e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd46c06da60 con 0x7fd46c0cc530 2023-07-14T11:56:08.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.493+0000 7fd473e3e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c164ec0 con 0x7fd46c160190 2023-07-14T11:56:08.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.493+0000 7fd4723db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd46c0cc530 0x7fd46c164980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:08.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.493+0000 7fd471bda700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46c160190 0x7fd46c160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:08.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.493+0000 7fd4723db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd46c0cc530 0x7fd46c164980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:37424/0 (socket says 172.21.15.104:37424) 2023-07-14T11:56:08.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.493+0000 7fd4723db700 1 -- 172.21.15.104:0/116923012 learned_addr learned my addr 172.21.15.104:0/116923012 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:08.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd4723db700 1 -- 172.21.15.104:0/116923012 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46c160190 msgr2=0x7fd46c160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:08.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd4723db700 1 --2- 172.21.15.104:0/116923012 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46c160190 0x7fd46c160560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:08.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd4723db700 1 -- 172.21.15.104:0/116923012 >> v1:172.21.15.121:6789/0 conn(0x7fd46c0c88a0 legacy=0x7fd46c0cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:08.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd4723db700 1 -- 172.21.15.104:0/116923012 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c0cdd80 con 0x7fd46c0cc530 2023-07-14T11:56:08.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd4723db700 1 --2- 172.21.15.104:0/116923012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd46c0cc530 0x7fd46c164980 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd464009e80 tx=0x7fd46400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf4a5186f3370b24 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:08.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd470bd8700 1 -- 172.21.15.104:0/116923012 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46400e070 con 0x7fd46c0cc530 2023-07-14T11:56:08.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd470bd8700 1 -- 172.21.15.104:0/116923012 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46400b8d0 con 0x7fd46c0cc530 2023-07-14T11:56:08.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd470bd8700 1 -- 172.21.15.104:0/116923012 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46400bc80 con 0x7fd46c0cc530 2023-07-14T11:56:08.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd473e3e700 1 -- 172.21.15.104:0/116923012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd46c0cc530 msgr2=0x7fd46c164980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:08.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd473e3e700 1 --2- 172.21.15.104:0/116923012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd46c0cc530 0x7fd46c164980 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd464009e80 tx=0x7fd46400b330 comp rx=0 tx=0).stop 2023-07-14T11:56:08.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd473e3e700 1 -- 172.21.15.104:0/116923012 shutdown_connections 2023-07-14T11:56:08.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd473e3e700 1 --2- 172.21.15.104:0/116923012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd46c0cc530 0x7fd46c164980 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:08.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd473e3e700 1 --2- 172.21.15.104:0/116923012 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46c160190 0x7fd46c160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:08.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd473e3e700 1 -- 172.21.15.104:0/116923012 >> 172.21.15.104:0/116923012 conn(0x7fd46c0afd60 msgr2=0x7fd46c0b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:08.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd473e3e700 1 -- 172.21.15.104:0/116923012 shutdown_connections 2023-07-14T11:56:08.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.497+0000 7fd473e3e700 1 -- 172.21.15.104:0/116923012 wait complete. 2023-07-14T11:56:08.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.501+0000 7fd473e3e700 1 Processor -- start 2023-07-14T11:56:08.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.501+0000 7fd473e3e700 1 -- start start 2023-07-14T11:56:08.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd473e3e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd46c0c88a0 0x7fd46c0c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:08.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd473e3e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46c0cc530 0x7fd46c0c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:08.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd473e3e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd46c160190 0x7fd46c0ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:08.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd473e3e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c0b2d30 con 0x7fd46c0c88a0 2023-07-14T11:56:08.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd473e3e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c0b2400 con 0x7fd46c160190 2023-07-14T11:56:08.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd473e3e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c15cdb0 con 0x7fd46c0cc530 2023-07-14T11:56:08.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd471bda700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd46c0c88a0 0x7fd46c0c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:08.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd471bda700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd46c0c88a0 0x7fd46c0c5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45856/0 (socket says 172.21.15.104:45856) 2023-07-14T11:56:08.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd471bda700 1 -- 172.21.15.104:0/759835207 learned_addr learned my addr 172.21.15.104:0/759835207 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:08.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd4723db700 1 --2- 172.21.15.104:0/759835207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd46c160190 0x7fd46c0ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:08.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd4713d9700 1 --2- 172.21.15.104:0/759835207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46c0cc530 0x7fd46c0c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:08.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd471bda700 1 -- 172.21.15.104:0/759835207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46c0cc530 msgr2=0x7fd46c0c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:08.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd471bda700 1 --2- 172.21.15.104:0/759835207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46c0cc530 0x7fd46c0c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:08.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd471bda700 1 -- 172.21.15.104:0/759835207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd46c160190 msgr2=0x7fd46c0ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:08.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd471bda700 1 --2- 172.21.15.104:0/759835207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd46c160190 0x7fd46c0ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:08.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd471bda700 1 -- 172.21.15.104:0/759835207 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd464009b30 con 0x7fd46c0c88a0 2023-07-14T11:56:08.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd471bda700 1 --2- 172.21.15.104:0/759835207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd46c0c88a0 0x7fd46c0c5320 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd46800e840 tx=0x7fd46800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:08.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd4627fc700 1 -- 172.21.15.104:0/759835207 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680095a0 con 0x7fd46c0c88a0 2023-07-14T11:56:08.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd4627fc700 1 -- 172.21.15.104:0/759835207 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd468013680 con 0x7fd46c0c88a0 2023-07-14T11:56:08.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd4627fc700 1 -- 172.21.15.104:0/759835207 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680139f0 con 0x7fd46c0c88a0 2023-07-14T11:56:08.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd473e3e700 1 -- 172.21.15.104:0/759835207 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46c0b2670 con 0x7fd46c0c88a0 2023-07-14T11:56:08.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.505+0000 7fd473e3e700 1 -- 172.21.15.104:0/759835207 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46c0c8550 con 0x7fd46c0c88a0 2023-07-14T11:56:08.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.509+0000 7fd473e3e700 1 -- 172.21.15.104:0/759835207 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46c04dcb0 con 0x7fd46c0c88a0 2023-07-14T11:56:08.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.509+0000 7fd4627fc700 1 -- 172.21.15.104:0/759835207 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd468013b50 con 0x7fd46c0c88a0 2023-07-14T11:56:08.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.513+0000 7fd4627fc700 1 --2- 172.21.15.104:0/759835207 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd4500662f0 0x7fd4500687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:08.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.513+0000 7fd4627fc700 1 -- 172.21.15.104:0/759835207 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd4680839a0 con 0x7fd46c0c88a0 2023-07-14T11:56:08.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.513+0000 7fd4713d9700 1 --2- 172.21.15.104:0/759835207 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd4500662f0 0x7fd4500687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:08.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.513+0000 7fd4713d9700 1 --2- 172.21.15.104:0/759835207 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd4500662f0 0x7fd4500687a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0c5b20 tx=0x7fd45c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:08.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.517+0000 7fd4627fc700 1 -- 172.21.15.104:0/759835207 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fd468054b50 con 0x7fd46c0c88a0 2023-07-14T11:56:08.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:08.689+0000 7fd473e3e700 1 -- 172.21.15.104:0/759835207 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd46c0cda00 con 0x7fd46c0c88a0 2023-07-14T11:56:09.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.657+0000 7fd4627fc700 1 -- 172.21.15.104:0/759835207 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v56) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd4680546e0 con 0x7fd46c0c88a0 2023-07-14T11:56:09.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.661+0000 7fd473e3e700 1 -- 172.21.15.104:0/759835207 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd4500662f0 msgr2=0x7fd4500687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:09.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.661+0000 7fd473e3e700 1 --2- 172.21.15.104:0/759835207 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd4500662f0 0x7fd4500687a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0c5b20 tx=0x7fd45c009040 comp rx=0 tx=0).stop 2023-07-14T11:56:09.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.661+0000 7fd473e3e700 1 -- 172.21.15.104:0/759835207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd46c0c88a0 msgr2=0x7fd46c0c5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:09.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.661+0000 7fd473e3e700 1 --2- 172.21.15.104:0/759835207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd46c0c88a0 0x7fd46c0c5320 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd46800e840 tx=0x7fd46800eb50 comp rx=0 tx=0).stop 2023-07-14T11:56:09.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.661+0000 7fd473e3e700 1 -- 172.21.15.104:0/759835207 shutdown_connections 2023-07-14T11:56:09.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.661+0000 7fd473e3e700 1 --2- 172.21.15.104:0/759835207 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd4500662f0 0x7fd4500687a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:09.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.661+0000 7fd473e3e700 1 --2- 172.21.15.104:0/759835207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd46c160190 0x7fd46c0ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:09.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.661+0000 7fd473e3e700 1 --2- 172.21.15.104:0/759835207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd46c0cc530 0x7fd46c0c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:09.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.661+0000 7fd473e3e700 1 --2- 172.21.15.104:0/759835207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd46c0c88a0 0x7fd46c0c5320 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:09.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.661+0000 7fd473e3e700 1 -- 172.21.15.104:0/759835207 >> 172.21.15.104:0/759835207 conn(0x7fd46c0afd60 msgr2=0x7fd46c15f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:09.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.661+0000 7fd473e3e700 1 -- 172.21.15.104:0/759835207 shutdown_connections 2023-07-14T11:56:09.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.665+0000 7fd473e3e700 1 -- 172.21.15.104:0/759835207 wait complete. 2023-07-14T11:56:09.653 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'cephfs_metadata' created 2023-07-14T11:56:09.672 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-14T11:56:09.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.865+0000 7f4d6eba0700 1 Processor -- start 2023-07-14T11:56:09.855 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.865+0000 7f4d6eba0700 1 -- start start 2023-07-14T11:56:09.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.869+0000 7f4d6eba0700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f4d680d0210 0x7f4d680d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:09.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.869+0000 7f4d6eba0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d680cf000 0x7f4d680cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:09.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.869+0000 7f4d6eba0700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d68071d10 con 0x7f4d680b1c50 2023-07-14T11:56:09.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.869+0000 7f4d6eba0700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d680cf9c0 con 0x7f4d680d0210 2023-07-14T11:56:09.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.869+0000 7f4d6eba0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6815c420 con 0x7f4d680cf000 2023-07-14T11:56:09.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.869+0000 7f4d67fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f4d680d0210 0x7f4d680d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:09.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.869+0000 7f4d6c93c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4d680b1c50 0x7f4d68071600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:52384/0 (socket says 172.21.15.104:52384) 2023-07-14T11:56:09.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.869+0000 7f4d67fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f4d680d0210 0x7f4d680d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45860/0 (socket says 172.21.15.104:45860) 2023-07-14T11:56:09.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.869+0000 7f4d6d13d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d680cf000 0x7f4d680cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:09.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.869+0000 7f4d6c93c700 1 -- 172.21.15.104:0/4208343378 learned_addr learned my addr 172.21.15.104:0/4208343378 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:09.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.869+0000 7f4d67fff700 1 -- 172.21.15.104:0/4208343378 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d680cf000 msgr2=0x7f4d680cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:09.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.869+0000 7f4d67fff700 1 --2- 172.21.15.104:0/4208343378 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d680cf000 0x7f4d680cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:09.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.869+0000 7f4d67fff700 1 -- 172.21.15.104:0/4208343378 >> v1:172.21.15.121:6789/0 conn(0x7f4d680b1c50 legacy=0x7f4d68071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:09.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.869+0000 7f4d67fff700 1 -- 172.21.15.104:0/4208343378 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d68070840 con 0x7f4d680d0210 2023-07-14T11:56:09.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.869+0000 7f4d67fff700 1 --2- 172.21.15.104:0/4208343378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d680d0210 0x7f4d680d0640 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009850 tx=0x7f4d54009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfe6e8fde210ca3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:09.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.873+0000 7f4d677fe700 1 -- 172.21.15.104:0/4208343378 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54011830 con 0x7f4d680d0210 2023-07-14T11:56:09.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.873+0000 7f4d677fe700 1 -- 172.21.15.104:0/4208343378 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d54011990 con 0x7f4d680d0210 2023-07-14T11:56:09.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.873+0000 7f4d677fe700 1 -- 172.21.15.104:0/4208343378 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54011d40 con 0x7f4d680d0210 2023-07-14T11:56:09.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.873+0000 7f4d6eba0700 1 -- 172.21.15.104:0/4208343378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d680d0210 msgr2=0x7f4d680d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:09.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.873+0000 7f4d6eba0700 1 --2- 172.21.15.104:0/4208343378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d680d0210 0x7f4d680d0640 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009850 tx=0x7f4d54009b60 comp rx=0 tx=0).stop 2023-07-14T11:56:09.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.873+0000 7f4d6eba0700 1 -- 172.21.15.104:0/4208343378 shutdown_connections 2023-07-14T11:56:09.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.873+0000 7f4d6eba0700 1 --2- 172.21.15.104:0/4208343378 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d680cf000 0x7f4d680cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:09.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.873+0000 7f4d6eba0700 1 --2- 172.21.15.104:0/4208343378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d680d0210 0x7f4d680d0640 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:09.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.873+0000 7f4d6eba0700 1 -- 172.21.15.104:0/4208343378 >> 172.21.15.104:0/4208343378 conn(0x7f4d6809b9c0 msgr2=0x7f4d680d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:09.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.873+0000 7f4d6eba0700 1 -- 172.21.15.104:0/4208343378 shutdown_connections 2023-07-14T11:56:09.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.873+0000 7f4d6eba0700 1 -- 172.21.15.104:0/4208343378 wait complete. 2023-07-14T11:56:09.863 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6eba0700 1 Processor -- start 2023-07-14T11:56:09.863 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6eba0700 1 -- start start 2023-07-14T11:56:09.864 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6eba0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d680b1c50 0x7f4d681e5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:09.865 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6eba0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d680cf000 0x7f4d68150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:09.865 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6eba0700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d680d0210 0x7f4d681e4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:09.865 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6eba0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d681e6a20 con 0x7f4d680b1c50 2023-07-14T11:56:09.865 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6eba0700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d681e6180 con 0x7f4d680d0210 2023-07-14T11:56:09.865 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6eba0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d681e6460 con 0x7f4d680cf000 2023-07-14T11:56:09.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6c93c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d680b1c50 0x7f4d681e5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:09.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6c93c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d680b1c50 0x7f4d681e5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45868/0 (socket says 172.21.15.104:45868) 2023-07-14T11:56:09.866 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6c93c700 1 -- 172.21.15.104:0/3713785126 learned_addr learned my addr 172.21.15.104:0/3713785126 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:09.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d67fff700 1 --2- 172.21.15.104:0/3713785126 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d680cf000 0x7f4d68150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:09.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6d13d700 1 --2- 172.21.15.104:0/3713785126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d680d0210 0x7f4d681e4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:09.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6c93c700 1 -- 172.21.15.104:0/3713785126 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d680cf000 msgr2=0x7f4d68150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:09.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6c93c700 1 --2- 172.21.15.104:0/3713785126 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d680cf000 0x7f4d68150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:09.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6c93c700 1 -- 172.21.15.104:0/3713785126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d680d0210 msgr2=0x7f4d681e4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:09.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6c93c700 1 --2- 172.21.15.104:0/3713785126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d680d0210 0x7f4d681e4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:09.867 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6c93c700 1 -- 172.21.15.104:0/3713785126 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d540094b0 con 0x7f4d680b1c50 2023-07-14T11:56:09.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6d13d700 1 --2- 172.21.15.104:0/3713785126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d680d0210 0x7f4d681e4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:09.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.877+0000 7f4d6c93c700 1 --2- 172.21.15.104:0/3713785126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d680b1c50 0x7f4d681e5b30 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00c700 tx=0x7f4d5c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:09.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.881+0000 7f4d657fa700 1 -- 172.21.15.104:0/3713785126 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c005d90 con 0x7f4d680b1c50 2023-07-14T11:56:09.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.881+0000 7f4d657fa700 1 -- 172.21.15.104:0/3713785126 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c0070d0 con 0x7f4d680b1c50 2023-07-14T11:56:09.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.881+0000 7f4d657fa700 1 -- 172.21.15.104:0/3713785126 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c007440 con 0x7f4d680b1c50 2023-07-14T11:56:09.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.881+0000 7f4d6eba0700 1 -- 172.21.15.104:0/3713785126 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d681e6740 con 0x7f4d680b1c50 2023-07-14T11:56:09.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.881+0000 7f4d6eba0700 1 -- 172.21.15.104:0/3713785126 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d681e4560 con 0x7f4d680b1c50 2023-07-14T11:56:09.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.881+0000 7f4d6eba0700 1 -- 172.21.15.104:0/3713785126 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d680b2cd0 con 0x7f4d680b1c50 2023-07-14T11:56:09.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.885+0000 7f4d657fa700 1 -- 172.21.15.104:0/3713785126 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f4d5c0075a0 con 0x7f4d680b1c50 2023-07-14T11:56:09.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.885+0000 7f4d657fa700 1 --2- 172.21.15.104:0/3713785126 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4d4c066320 0x7f4d4c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:09.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.885+0000 7f4d657fa700 1 -- 172.21.15.104:0/3713785126 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f4d5c083600 con 0x7f4d680b1c50 2023-07-14T11:56:09.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.885+0000 7f4d67fff700 1 --2- 172.21.15.104:0/3713785126 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4d4c066320 0x7f4d4c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:09.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.885+0000 7f4d67fff700 1 --2- 172.21.15.104:0/3713785126 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4d4c066320 0x7f4d4c0687d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4d540090b0 tx=0x7f4d5401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:09.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:09.889+0000 7f4d657fa700 1 -- 172.21.15.104:0/3713785126 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f4d5c050b30 con 0x7f4d680b1c50 2023-07-14T11:56:10.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.069+0000 7f4d6eba0700 1 -- 172.21.15.104:0/3713785126 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4d68071d10 con 0x7f4d680b1c50 2023-07-14T11:56:10.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.669+0000 7f4d657fa700 1 -- 172.21.15.104:0/3713785126 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v57) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4d68071d10 con 0x7f4d680b1c50 2023-07-14T11:56:10.665 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.677+0000 7f4d6eba0700 1 -- 172.21.15.104:0/3713785126 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4d4c066320 msgr2=0x7f4d4c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:10.666 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.677+0000 7f4d6eba0700 1 --2- 172.21.15.104:0/3713785126 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4d4c066320 0x7f4d4c0687d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4d540090b0 tx=0x7f4d5401c040 comp rx=0 tx=0).stop 2023-07-14T11:56:10.666 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.677+0000 7f4d6eba0700 1 -- 172.21.15.104:0/3713785126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d680b1c50 msgr2=0x7f4d681e5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:10.666 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.677+0000 7f4d6eba0700 1 --2- 172.21.15.104:0/3713785126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d680b1c50 0x7f4d681e5b30 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00c700 tx=0x7f4d5c00ca10 comp rx=0 tx=0).stop 2023-07-14T11:56:10.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.677+0000 7f4d6eba0700 1 -- 172.21.15.104:0/3713785126 shutdown_connections 2023-07-14T11:56:10.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.677+0000 7f4d6eba0700 1 --2- 172.21.15.104:0/3713785126 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4d4c066320 0x7f4d4c0687d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.677+0000 7f4d6eba0700 1 --2- 172.21.15.104:0/3713785126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d680d0210 0x7f4d681e4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.677+0000 7f4d6eba0700 1 --2- 172.21.15.104:0/3713785126 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d680cf000 0x7f4d68150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.677+0000 7f4d6eba0700 1 --2- 172.21.15.104:0/3713785126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d680b1c50 0x7f4d681e5b30 secure :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00c700 tx=0x7f4d5c00ca10 comp rx=0 tx=0).stop 2023-07-14T11:56:10.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.677+0000 7f4d6eba0700 1 -- 172.21.15.104:0/3713785126 >> 172.21.15.104:0/3713785126 conn(0x7f4d6809b9c0 msgr2=0x7f4d680b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:10.668 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.681+0000 7f4d6eba0700 1 -- 172.21.15.104:0/3713785126 shutdown_connections 2023-07-14T11:56:10.672 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.681+0000 7f4d6eba0700 1 -- 172.21.15.104:0/3713785126 wait complete. 2023-07-14T11:56:10.676 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'cephfs_data' created 2023-07-14T11:56:10.698 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-14T11:56:10.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.885+0000 7ff8c7fff700 1 Processor -- start 2023-07-14T11:56:10.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.885+0000 7ff8c7fff700 1 -- start start 2023-07-14T11:56:10.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.885+0000 7ff8c7fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff8c81601b0 0x7ff8c80c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:10.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.885+0000 7ff8c7fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff8c8160580 0x7ff8c80c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:10.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.885+0000 7ff8c7fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c80c5ba0 0x7ff8c8163f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:10.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.885+0000 7ff8c7fff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c80d0420 con 0x7ff8c8160580 2023-07-14T11:56:10.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.885+0000 7ff8c7fff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c809bda0 con 0x7ff8c81601b0 2023-07-14T11:56:10.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.885+0000 7ff8c7fff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c80ce590 con 0x7ff8c80c5ba0 2023-07-14T11:56:10.876 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c67fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff8c8160580 0x7ff8c80c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:10.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c67fc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff8c8160580 0x7ff8c80c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50300/0 (socket says 172.21.15.104:50300) 2023-07-14T11:56:10.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c67fc700 1 -- 172.21.15.104:0/1516332178 learned_addr learned my addr 172.21.15.104:0/1516332178 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:10.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c77fe700 1 --2- 172.21.15.104:0/1516332178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c80c5ba0 0x7ff8c8163f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:10.877 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c6ffd700 1 --2- 172.21.15.104:0/1516332178 >> v2:172.21.15.121:3300/0 conn(0x7ff8c81601b0 0x7ff8c80c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:10.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c67fc700 1 -- 172.21.15.104:0/1516332178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c80c5ba0 msgr2=0x7ff8c8163f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:10.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c67fc700 1 --2- 172.21.15.104:0/1516332178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c80c5ba0 0x7ff8c8163f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.879 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c67fc700 1 -- 172.21.15.104:0/1516332178 >> v2:172.21.15.121:3300/0 conn(0x7ff8c81601b0 msgr2=0x7ff8c80c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:10.879 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c67fc700 1 --2- 172.21.15.104:0/1516332178 >> v2:172.21.15.121:3300/0 conn(0x7ff8c81601b0 0x7ff8c80c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.879 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c67fc700 1 -- 172.21.15.104:0/1516332178 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c80ce870 con 0x7ff8c8160580 2023-07-14T11:56:10.879 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c67fc700 1 --2- 172.21.15.104:0/1516332178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c8160580 0x7ff8c80c5660 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff8c806f700 tx=0x7ff8bc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9114dd7f9d9672c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:10.879 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c5ffb700 1 -- 172.21.15.104:0/1516332178 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc011830 con 0x7ff8c8160580 2023-07-14T11:56:10.879 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c5ffb700 1 -- 172.21.15.104:0/1516332178 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc011990 con 0x7ff8c8160580 2023-07-14T11:56:10.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c5ffb700 1 -- 172.21.15.104:0/1516332178 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc011d20 con 0x7ff8c8160580 2023-07-14T11:56:10.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c7fff700 1 -- 172.21.15.104:0/1516332178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c8160580 msgr2=0x7ff8c80c5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:10.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c7fff700 1 --2- 172.21.15.104:0/1516332178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c8160580 0x7ff8c80c5660 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff8c806f700 tx=0x7ff8bc009bf0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c7fff700 1 -- 172.21.15.104:0/1516332178 shutdown_connections 2023-07-14T11:56:10.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c7fff700 1 --2- 172.21.15.104:0/1516332178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c80c5ba0 0x7ff8c8163f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c7fff700 1 --2- 172.21.15.104:0/1516332178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c8160580 0x7ff8c80c5660 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c7fff700 1 --2- 172.21.15.104:0/1516332178 >> v2:172.21.15.121:3300/0 conn(0x7ff8c81601b0 0x7ff8c80c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c7fff700 1 -- 172.21.15.104:0/1516332178 >> 172.21.15.104:0/1516332178 conn(0x7ff8c809b850 msgr2=0x7ff8c80d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:10.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c7fff700 1 -- 172.21.15.104:0/1516332178 shutdown_connections 2023-07-14T11:56:10.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.889+0000 7ff8c7fff700 1 -- 172.21.15.104:0/1516332178 wait complete. 2023-07-14T11:56:10.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.893+0000 7ff8c7fff700 1 Processor -- start 2023-07-14T11:56:10.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.893+0000 7ff8c7fff700 1 -- start start 2023-07-14T11:56:10.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c7fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c80c5ba0 0x7ff8c806c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:10.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c81601b0 0x7ff8c806c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:10.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c8160580 0x7ff8c80b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:10.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c7fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c814e680 con 0x7ff8c8160580 2023-07-14T11:56:10.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c7fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c80b6ad0 con 0x7ff8c81601b0 2023-07-14T11:56:10.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c7fff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c80b6db0 con 0x7ff8c80c5ba0 2023-07-14T11:56:10.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c77fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c8160580 0x7ff8c80b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:10.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c77fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c8160580 0x7ff8c80b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50310/0 (socket says 172.21.15.104:50310) 2023-07-14T11:56:10.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c77fe700 1 -- 172.21.15.104:0/717070495 learned_addr learned my addr 172.21.15.104:0/717070495 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:10.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c6ffd700 1 --2- 172.21.15.104:0/717070495 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c80c5ba0 0x7ff8c806c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:10.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c67fc700 1 --2- 172.21.15.104:0/717070495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c81601b0 0x7ff8c806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:10.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c77fe700 1 -- 172.21.15.104:0/717070495 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c80c5ba0 msgr2=0x7ff8c806c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:10.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c77fe700 1 --2- 172.21.15.104:0/717070495 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c80c5ba0 0x7ff8c806c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c77fe700 1 -- 172.21.15.104:0/717070495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c81601b0 msgr2=0x7ff8c806c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:10.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c77fe700 1 --2- 172.21.15.104:0/717070495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c81601b0 0x7ff8c806c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c77fe700 1 -- 172.21.15.104:0/717070495 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc009570 con 0x7ff8c8160580 2023-07-14T11:56:10.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c77fe700 1 --2- 172.21.15.104:0/717070495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c8160580 0x7ff8c80b5d00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000eaf0 tx=0x7ff8c000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:10.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8affff700 1 -- 172.21.15.104:0/717070495 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0010040 con 0x7ff8c8160580 2023-07-14T11:56:10.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8affff700 1 -- 172.21.15.104:0/717070495 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c000b950 con 0x7ff8c8160580 2023-07-14T11:56:10.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8affff700 1 -- 172.21.15.104:0/717070495 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c000bd00 con 0x7ff8c8160580 2023-07-14T11:56:10.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.897+0000 7ff8c7fff700 1 -- 172.21.15.104:0/717070495 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c80b7090 con 0x7ff8c8160580 2023-07-14T11:56:10.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.901+0000 7ff8c7fff700 1 -- 172.21.15.104:0/717070495 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c81f5730 con 0x7ff8c8160580 2023-07-14T11:56:10.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.901+0000 7ff8c7fff700 1 -- 172.21.15.104:0/717070495 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c814e7c0 con 0x7ff8c8160580 2023-07-14T11:56:10.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.901+0000 7ff8affff700 1 -- 172.21.15.104:0/717070495 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff8c001d400 con 0x7ff8c8160580 2023-07-14T11:56:10.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.905+0000 7ff8affff700 1 --2- 172.21.15.104:0/717070495 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff8b4066320 0x7ff8b40687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:10.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.905+0000 7ff8affff700 1 -- 172.21.15.104:0/717070495 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6282+0+0 (secure 0 0 0) 0x7ff8c0084770 con 0x7ff8c8160580 2023-07-14T11:56:10.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.905+0000 7ff8c6ffd700 1 --2- 172.21.15.104:0/717070495 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff8b4066320 0x7ff8b40687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:10.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.905+0000 7ff8c6ffd700 1 --2- 172.21.15.104:0/717070495 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff8b4066320 0x7ff8b40687d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80d1600 tx=0x7ff8b8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:10.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:10.909+0000 7ff8affff700 1 -- 172.21.15.104:0/717070495 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff8c0051af0 con 0x7ff8c8160580 2023-07-14T11:56:11.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.117+0000 7ff8c7fff700 1 -- 172.21.15.104:0/717070495 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff8c814e7c0 con 0x7ff8c8160580 2023-07-14T11:56:11.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.681+0000 7ff8affff700 1 -- 172.21.15.104:0/717070495 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v24) v1 ==== 173+0+0 (secure 0 0 0) 0x7ff8c814e7c0 con 0x7ff8c8160580 2023-07-14T11:56:11.672 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.685+0000 7ff8c7fff700 1 -- 172.21.15.104:0/717070495 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff8b4066320 msgr2=0x7ff8b40687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:11.672 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.685+0000 7ff8c7fff700 1 --2- 172.21.15.104:0/717070495 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff8b4066320 0x7ff8b40687d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80d1600 tx=0x7ff8b8008040 comp rx=0 tx=0).stop 2023-07-14T11:56:11.672 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.685+0000 7ff8c7fff700 1 -- 172.21.15.104:0/717070495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c8160580 msgr2=0x7ff8c80b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:11.672 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.685+0000 7ff8c7fff700 1 --2- 172.21.15.104:0/717070495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c8160580 0x7ff8c80b5d00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000eaf0 tx=0x7ff8c000ee00 comp rx=0 tx=0).stop 2023-07-14T11:56:11.673 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.685+0000 7ff8c7fff700 1 -- 172.21.15.104:0/717070495 shutdown_connections 2023-07-14T11:56:11.673 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.685+0000 7ff8c7fff700 1 --2- 172.21.15.104:0/717070495 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff8b4066320 0x7ff8b40687d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:11.673 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.685+0000 7ff8c7fff700 1 --2- 172.21.15.104:0/717070495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8c8160580 0x7ff8c80b5d00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:11.673 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.685+0000 7ff8c7fff700 1 --2- 172.21.15.104:0/717070495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c81601b0 0x7ff8c806c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:11.674 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.685+0000 7ff8c7fff700 1 --2- 172.21.15.104:0/717070495 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c80c5ba0 0x7ff8c806c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:11.674 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.685+0000 7ff8c7fff700 1 -- 172.21.15.104:0/717070495 >> 172.21.15.104:0/717070495 conn(0x7ff8c809b850 msgr2=0x7ff8c80c7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:11.674 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.685+0000 7ff8c7fff700 1 -- 172.21.15.104:0/717070495 shutdown_connections 2023-07-14T11:56:11.674 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.685+0000 7ff8c7fff700 1 -- 172.21.15.104:0/717070495 wait complete. 2023-07-14T11:56:11.680 INFO:teuthology.orchestra.run.smithi104.stderr:new fs with metadata pool 9 and data pool 10 2023-07-14T11:56:11.710 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:56:11.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5e0df65700 1 Processor -- start 2023-07-14T11:56:11.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5e0df65700 1 -- start start 2023-07-14T11:56:11.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5e0df65700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e0806d860 0x7f5e080cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:11.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5e0df65700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5e080c1e00 0x7f5e080cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:11.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5e0df65700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e080cc1b0 con 0x7f5e0806be70 2023-07-14T11:56:11.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5e0df65700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e080cd690 con 0x7f5e080c1e00 2023-07-14T11:56:11.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5e0df65700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e080cd970 con 0x7f5e0806d860 2023-07-14T11:56:11.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5dfffff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f5e0806be70 0x7f5e0806dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:47882/0 (socket says 172.21.15.104:47882) 2023-07-14T11:56:11.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5dfffff700 1 -- 172.21.15.104:0/3294192222 learned_addr learned my addr 172.21.15.104:0/3294192222 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:11.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5e077fe700 1 --2- 172.21.15.104:0/3294192222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e0806d860 0x7f5e080cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:11.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5e07fff700 1 --2- 172.21.15.104:0/3294192222 >> v2:172.21.15.121:3300/0 conn(0x7f5e080c1e00 0x7f5e080cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:11.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5e06ffd700 1 -- 172.21.15.104:0/3294192222 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277195258 0 0) 0x7f5e080cc1b0 con 0x7f5e0806be70 2023-07-14T11:56:11.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5e06ffd700 1 -- 172.21.15.104:0/3294192222 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5df40035d0 con 0x7f5e0806be70 2023-07-14T11:56:11.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5e077fe700 1 -- 172.21.15.104:0/3294192222 >> v2:172.21.15.121:3300/0 conn(0x7f5e080c1e00 msgr2=0x7f5e080cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:11.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5e077fe700 1 --2- 172.21.15.104:0/3294192222 >> v2:172.21.15.121:3300/0 conn(0x7f5e080c1e00 0x7f5e080cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:11.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5e077fe700 1 -- 172.21.15.104:0/3294192222 >> v1:172.21.15.104:6789/0 conn(0x7f5e0806be70 legacy=0x7f5e0806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:11.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.897+0000 7f5e077fe700 1 -- 172.21.15.104:0/3294192222 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e080cdc50 con 0x7f5e0806d860 2023-07-14T11:56:11.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.901+0000 7f5e077fe700 1 --2- 172.21.15.104:0/3294192222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e0806d860 0x7f5e080cb840 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009880 tx=0x7f5df8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=221927523a64f9fa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:11.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.901+0000 7f5e06ffd700 1 -- 172.21.15.104:0/3294192222 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8011830 con 0x7f5e0806d860 2023-07-14T11:56:11.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.901+0000 7f5e06ffd700 1 -- 172.21.15.104:0/3294192222 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df8011990 con 0x7f5e0806d860 2023-07-14T11:56:11.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.901+0000 7f5e06ffd700 1 -- 172.21.15.104:0/3294192222 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8011d00 con 0x7f5e0806d860 2023-07-14T11:56:11.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.901+0000 7f5e0df65700 1 -- 172.21.15.104:0/3294192222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e0806d860 msgr2=0x7f5e080cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:11.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.901+0000 7f5e0df65700 1 --2- 172.21.15.104:0/3294192222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e0806d860 0x7f5e080cb840 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009880 tx=0x7f5df8009b90 comp rx=0 tx=0).stop 2023-07-14T11:56:11.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.901+0000 7f5e0df65700 1 -- 172.21.15.104:0/3294192222 shutdown_connections 2023-07-14T11:56:11.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.901+0000 7f5e0df65700 1 --2- 172.21.15.104:0/3294192222 >> v2:172.21.15.121:3300/0 conn(0x7f5e080c1e00 0x7f5e080cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:11.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.901+0000 7f5e0df65700 1 --2- 172.21.15.104:0/3294192222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e0806d860 0x7f5e080cb840 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:11.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.901+0000 7f5e0df65700 1 -- 172.21.15.104:0/3294192222 >> 172.21.15.104:0/3294192222 conn(0x7f5e080cad30 msgr2=0x7f5e080cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:11.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.901+0000 7f5e0df65700 1 -- 172.21.15.104:0/3294192222 shutdown_connections 2023-07-14T11:56:11.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.901+0000 7f5e0df65700 1 -- 172.21.15.104:0/3294192222 wait complete. 2023-07-14T11:56:11.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e0df65700 1 Processor -- start 2023-07-14T11:56:11.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e0df65700 1 -- start start 2023-07-14T11:56:11.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e0df65700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e0806be70 0x7f5e08155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:11.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e0df65700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e0806d860 0x7f5e08157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:11.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e0df65700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e080c1e00 0x7f5e08158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:11.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e0df65700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e080af890 con 0x7f5e080c1e00 2023-07-14T11:56:11.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e0df65700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e080afb70 con 0x7f5e0806d860 2023-07-14T11:56:11.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e0df65700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e080afe50 con 0x7f5e0806be70 2023-07-14T11:56:11.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e07fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e080c1e00 0x7f5e08158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:11.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e07fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e080c1e00 0x7f5e08158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50326/0 (socket says 172.21.15.104:50326) 2023-07-14T11:56:11.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e07fff700 1 -- 172.21.15.104:0/4171707312 learned_addr learned my addr 172.21.15.104:0/4171707312 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:11.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e077fe700 1 --2- 172.21.15.104:0/4171707312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e0806be70 0x7f5e08155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:11.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5dfffff700 1 --2- 172.21.15.104:0/4171707312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e0806d860 0x7f5e08157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:11.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e07fff700 1 -- 172.21.15.104:0/4171707312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e0806be70 msgr2=0x7f5e08155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:11.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e07fff700 1 --2- 172.21.15.104:0/4171707312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e0806be70 0x7f5e08155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:11.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e07fff700 1 -- 172.21.15.104:0/4171707312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e0806d860 msgr2=0x7f5e08157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:11.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e07fff700 1 --2- 172.21.15.104:0/4171707312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e0806d860 0x7f5e08157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:11.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e07fff700 1 -- 172.21.15.104:0/4171707312 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df80094b0 con 0x7f5e080c1e00 2023-07-14T11:56:11.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5dfffff700 1 --2- 172.21.15.104:0/4171707312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e0806d860 0x7f5e08157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:11.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e07fff700 1 --2- 172.21.15.104:0/4171707312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e080c1e00 0x7f5e08158460 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000efc0 tx=0x7f5e0000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:11.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e04ff9700 1 -- 172.21.15.104:0/4171707312 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0000cb80 con 0x7f5e080c1e00 2023-07-14T11:56:11.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e04ff9700 1 -- 172.21.15.104:0/4171707312 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0000f040 con 0x7f5e080c1e00 2023-07-14T11:56:11.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e0df65700 1 -- 172.21.15.104:0/4171707312 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e080b0130 con 0x7f5e080c1e00 2023-07-14T11:56:11.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e04ff9700 1 -- 172.21.15.104:0/4171707312 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00014480 con 0x7f5e080c1e00 2023-07-14T11:56:11.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.905+0000 7f5e0df65700 1 -- 172.21.15.104:0/4171707312 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0815b1a0 con 0x7f5e080c1e00 2023-07-14T11:56:11.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.909+0000 7f5e0df65700 1 -- 172.21.15.104:0/4171707312 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e080b0350 con 0x7f5e080c1e00 2023-07-14T11:56:11.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.909+0000 7f5e04ff9700 1 -- 172.21.15.104:0/4171707312 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5e000145e0 con 0x7f5e080c1e00 2023-07-14T11:56:11.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.913+0000 7f5e04ff9700 1 --2- 172.21.15.104:0/4171707312 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5de80662f0 0x7f5de80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:11.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.913+0000 7f5e04ff9700 1 -- 172.21.15.104:0/4171707312 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5e00084680 con 0x7f5e080c1e00 2023-07-14T11:56:11.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.913+0000 7f5e077fe700 1 --2- 172.21.15.104:0/4171707312 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5de80662f0 0x7f5de80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:11.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.913+0000 7f5e077fe700 1 --2- 172.21.15.104:0/4171707312 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5de80662f0 0x7f5de80687a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5df8000c00 tx=0x7f5df8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:11.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:11.917+0000 7f5e04ff9700 1 -- 172.21.15.104:0/4171707312 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f5e00051920 con 0x7f5e080c1e00 2023-07-14T11:56:12.074 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.085+0000 7f5e0df65700 1 -- 172.21.15.104:0/4171707312 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e080af9d0 con 0x7f5e080c1e00 2023-07-14T11:56:12.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.089+0000 7f5e04ff9700 1 -- 172.21.15.104:0/4171707312 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+18547 (secure 0 0 0) 0x7f5e080af9d0 con 0x7f5e080c1e00 2023-07-14T11:56:12.077 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:12.078 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":58,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:56:11.122673+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:56:08.755551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-14T11:56:10.132688+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:56:12.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.093+0000 7f5e0df65700 1 -- 172.21.15.104:0/4171707312 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5de80662f0 msgr2=0x7f5de80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:12.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.093+0000 7f5e0df65700 1 --2- 172.21.15.104:0/4171707312 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5de80662f0 0x7f5de80687a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5df8000c00 tx=0x7f5df8008030 comp rx=0 tx=0).stop 2023-07-14T11:56:12.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.093+0000 7f5e0df65700 1 -- 172.21.15.104:0/4171707312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e080c1e00 msgr2=0x7f5e08158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:12.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.093+0000 7f5e0df65700 1 --2- 172.21.15.104:0/4171707312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e080c1e00 0x7f5e08158460 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000efc0 tx=0x7f5e0000c0c0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.093+0000 7f5e0df65700 1 -- 172.21.15.104:0/4171707312 shutdown_connections 2023-07-14T11:56:12.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.093+0000 7f5e0df65700 1 --2- 172.21.15.104:0/4171707312 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5de80662f0 0x7f5de80687a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.093+0000 7f5e0df65700 1 --2- 172.21.15.104:0/4171707312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e080c1e00 0x7f5e08158460 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.093+0000 7f5e0df65700 1 --2- 172.21.15.104:0/4171707312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e0806d860 0x7f5e08157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.093+0000 7f5e0df65700 1 --2- 172.21.15.104:0/4171707312 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e0806be70 0x7f5e08155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.093+0000 7f5e0df65700 1 -- 172.21.15.104:0/4171707312 >> 172.21.15.104:0/4171707312 conn(0x7f5e080cad30 msgr2=0x7f5e080703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:12.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.093+0000 7f5e0df65700 1 -- 172.21.15.104:0/4171707312 shutdown_connections 2023-07-14T11:56:12.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.093+0000 7f5e0df65700 1 -- 172.21.15.104:0/4171707312 wait complete. 2023-07-14T11:56:12.104 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:56:12.271 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.281+0000 7fe288bae700 1 Processor -- start 2023-07-14T11:56:12.271 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.285+0000 7fe288bae700 1 -- start start 2023-07-14T11:56:12.273 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.285+0000 7fe288bae700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe2840afd30 0x7fe2840c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:12.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.285+0000 7fe288bae700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2840c97f0 con 0x7fe2840b0100 2023-07-14T11:56:12.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.285+0000 7fe288bae700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2840c8900 con 0x7fe2840cc1b0 2023-07-14T11:56:12.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.285+0000 7fe288bae700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2840c8be0 con 0x7fe2840afd30 2023-07-14T11:56:12.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.285+0000 7fe28259c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe2840afd30 0x7fe2840c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:12.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.285+0000 7fe281d9b700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fe2840b0100 0x7fe28406d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:47898/0 (socket says 172.21.15.104:47898) 2023-07-14T11:56:12.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.285+0000 7fe281d9b700 1 -- 172.21.15.104:0/226993066 learned_addr learned my addr 172.21.15.104:0/226993066 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:12.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.289+0000 7fe28259c700 1 -- 172.21.15.104:0/226993066 >> v1:172.21.15.121:6789/0 conn(0x7fe2840cc1b0 legacy=0x7fe2840cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:12.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.289+0000 7fe28259c700 1 -- 172.21.15.104:0/226993066 >> v1:172.21.15.104:6789/0 conn(0x7fe2840b0100 legacy=0x7fe28406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:12.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.289+0000 7fe28259c700 1 -- 172.21.15.104:0/226993066 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2840c8ec0 con 0x7fe2840afd30 2023-07-14T11:56:12.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.285+0000 7fe28159a700 1 -- 172.21.15.104:0/226993066 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226123458 0 0) 0x7fe2840c97f0 con 0x7fe2840b0100 2023-07-14T11:56:12.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.289+0000 7fe28259c700 1 --2- 172.21.15.104:0/226993066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2840afd30 0x7fe2840c9c80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe274009850 tx=0x7fe274009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c319d8741f3f4625 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:12.276 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.289+0000 7fe28159a700 1 -- 172.21.15.104:0/226993066 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274011830 con 0x7fe2840afd30 2023-07-14T11:56:12.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.289+0000 7fe28159a700 1 -- 172.21.15.104:0/226993066 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe274011990 con 0x7fe2840afd30 2023-07-14T11:56:12.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.289+0000 7fe28159a700 1 -- 172.21.15.104:0/226993066 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274011d20 con 0x7fe2840afd30 2023-07-14T11:56:12.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.289+0000 7fe288bae700 1 -- 172.21.15.104:0/226993066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2840afd30 msgr2=0x7fe2840c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:12.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.289+0000 7fe288bae700 1 --2- 172.21.15.104:0/226993066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2840afd30 0x7fe2840c9c80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe274009850 tx=0x7fe274009b60 comp rx=0 tx=0).stop 2023-07-14T11:56:12.277 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.289+0000 7fe288bae700 1 -- 172.21.15.104:0/226993066 shutdown_connections 2023-07-14T11:56:12.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.289+0000 7fe288bae700 1 --2- 172.21.15.104:0/226993066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2840afd30 0x7fe2840c9c80 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.289+0000 7fe288bae700 1 -- 172.21.15.104:0/226993066 >> 172.21.15.104:0/226993066 conn(0x7fe2840cad30 msgr2=0x7fe2840cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:12.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.289+0000 7fe288bae700 1 -- 172.21.15.104:0/226993066 shutdown_connections 2023-07-14T11:56:12.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.289+0000 7fe288bae700 1 -- 172.21.15.104:0/226993066 wait complete. 2023-07-14T11:56:12.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.293+0000 7fe288bae700 1 Processor -- start 2023-07-14T11:56:12.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.293+0000 7fe288bae700 1 -- start start 2023-07-14T11:56:12.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.293+0000 7fe288bae700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2840afd30 0x7fe284159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:12.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.293+0000 7fe288bae700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2840b0100 0x7fe284140520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:12.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.293+0000 7fe288bae700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2840cc1b0 0x7fe2840b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:12.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.293+0000 7fe288bae700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2840b1af0 con 0x7fe2840b0100 2023-07-14T11:56:12.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.293+0000 7fe288bae700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2840b2670 con 0x7fe2840afd30 2023-07-14T11:56:12.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.293+0000 7fe288bae700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2840b1d40 con 0x7fe2840cc1b0 2023-07-14T11:56:12.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe281d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2840b0100 0x7fe284140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:12.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe281d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2840b0100 0x7fe284140520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50334/0 (socket says 172.21.15.104:50334) 2023-07-14T11:56:12.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe281d9b700 1 -- 172.21.15.104:0/2657159926 learned_addr learned my addr 172.21.15.104:0/2657159926 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:12.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe28259c700 1 --2- 172.21.15.104:0/2657159926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2840afd30 0x7fe284159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:12.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe282d9d700 1 --2- 172.21.15.104:0/2657159926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2840cc1b0 0x7fe2840b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:12.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe281d9b700 1 -- 172.21.15.104:0/2657159926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2840cc1b0 msgr2=0x7fe2840b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:12.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe281d9b700 1 --2- 172.21.15.104:0/2657159926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2840cc1b0 0x7fe2840b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe281d9b700 1 -- 172.21.15.104:0/2657159926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2840afd30 msgr2=0x7fe284159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:12.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe281d9b700 1 --2- 172.21.15.104:0/2657159926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2840afd30 0x7fe284159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe281d9b700 1 -- 172.21.15.104:0/2657159926 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2740094b0 con 0x7fe2840b0100 2023-07-14T11:56:12.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe282d9d700 1 --2- 172.21.15.104:0/2657159926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2840cc1b0 0x7fe2840b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:12.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe281d9b700 1 --2- 172.21.15.104:0/2657159926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2840b0100 0x7fe284140520 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe27800c550 tx=0x7fe27800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:12.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe272ffd700 1 -- 172.21.15.104:0/2657159926 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278004ca0 con 0x7fe2840b0100 2023-07-14T11:56:12.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe272ffd700 1 -- 172.21.15.104:0/2657159926 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe278004e00 con 0x7fe2840b0100 2023-07-14T11:56:12.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe288bae700 1 -- 172.21.15.104:0/2657159926 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2840b2020 con 0x7fe2840b0100 2023-07-14T11:56:12.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe272ffd700 1 -- 172.21.15.104:0/2657159926 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278007310 con 0x7fe2840b0100 2023-07-14T11:56:12.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe288bae700 1 -- 172.21.15.104:0/2657159926 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2840c5e90 con 0x7fe2840b0100 2023-07-14T11:56:12.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.297+0000 7fe288bae700 1 -- 172.21.15.104:0/2657159926 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2840ce0b0 con 0x7fe2840b0100 2023-07-14T11:56:12.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.301+0000 7fe272ffd700 1 -- 172.21.15.104:0/2657159926 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe278007470 con 0x7fe2840b0100 2023-07-14T11:56:12.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.301+0000 7fe272ffd700 1 --2- 172.21.15.104:0/2657159926 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe2640662f0 0x7fe2640687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:12.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.301+0000 7fe272ffd700 1 -- 172.21.15.104:0/2657159926 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe278083c00 con 0x7fe2840b0100 2023-07-14T11:56:12.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.301+0000 7fe28259c700 1 --2- 172.21.15.104:0/2657159926 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe2640662f0 0x7fe2640687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:12.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.301+0000 7fe28259c700 1 --2- 172.21.15.104:0/2657159926 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe2640662f0 0x7fe2640687a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe2740090b0 tx=0x7fe27401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:12.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.309+0000 7fe272ffd700 1 -- 172.21.15.104:0/2657159926 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe278050eb0 con 0x7fe2840b0100 2023-07-14T11:56:12.470 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.481+0000 7fe288bae700 1 -- 172.21.15.104:0/2657159926 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2840b27e0 con 0x7fe2840b0100 2023-07-14T11:56:12.473 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.485+0000 7fe272ffd700 1 -- 172.21.15.104:0/2657159926 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+18547 (secure 0 0 0) 0x7fe278007720 con 0x7fe2840b0100 2023-07-14T11:56:12.473 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:12.473 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":58,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:56:11.122673+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:56:08.755551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-14T11:56:10.132688+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:56:12.475 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.489+0000 7fe288bae700 1 -- 172.21.15.104:0/2657159926 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe2640662f0 msgr2=0x7fe2640687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:12.475 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.489+0000 7fe288bae700 1 --2- 172.21.15.104:0/2657159926 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe2640662f0 0x7fe2640687a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe2740090b0 tx=0x7fe27401c040 comp rx=0 tx=0).stop 2023-07-14T11:56:12.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.489+0000 7fe288bae700 1 -- 172.21.15.104:0/2657159926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2840b0100 msgr2=0x7fe284140520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:12.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.489+0000 7fe288bae700 1 --2- 172.21.15.104:0/2657159926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2840b0100 0x7fe284140520 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe27800c550 tx=0x7fe27800c910 comp rx=0 tx=0).stop 2023-07-14T11:56:12.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.489+0000 7fe288bae700 1 -- 172.21.15.104:0/2657159926 shutdown_connections 2023-07-14T11:56:12.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.489+0000 7fe288bae700 1 --2- 172.21.15.104:0/2657159926 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe2640662f0 0x7fe2640687a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.489+0000 7fe288bae700 1 --2- 172.21.15.104:0/2657159926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2840b0100 0x7fe284140520 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.489+0000 7fe288bae700 1 --2- 172.21.15.104:0/2657159926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2840cc1b0 0x7fe2840b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.489+0000 7fe288bae700 1 --2- 172.21.15.104:0/2657159926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2840afd30 0x7fe284159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.477 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.489+0000 7fe288bae700 1 -- 172.21.15.104:0/2657159926 >> 172.21.15.104:0/2657159926 conn(0x7fe2840cad30 msgr2=0x7fe2840ca410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:12.477 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.489+0000 7fe288bae700 1 -- 172.21.15.104:0/2657159926 shutdown_connections 2023-07-14T11:56:12.477 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.489+0000 7fe288bae700 1 -- 172.21.15.104:0/2657159926 wait complete. 2023-07-14T11:56:12.500 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-14T11:56:12.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.693+0000 7f21dd24f700 1 Processor -- start 2023-07-14T11:56:12.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.693+0000 7f21dd24f700 1 -- start start 2023-07-14T11:56:12.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21dd24f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21d808a470 0x7f21d806fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:12.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21dd24f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f21d80ccb20 0x7f21d806ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:12.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21dd24f700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d80b1bf0 con 0x7f21d80b1820 2023-07-14T11:56:12.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21dd24f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d815c2d0 con 0x7f21d80ccb20 2023-07-14T11:56:12.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21dd24f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80cb8e0 con 0x7f21d808a470 2023-07-14T11:56:12.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d7fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f21d80b1820 0x7f21d80cde70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:47908/0 (socket says 172.21.15.104:47908) 2023-07-14T11:56:12.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d7fff700 1 -- 172.21.15.104:0/1478962898 learned_addr learned my addr 172.21.15.104:0/1478962898 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:12.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d77fe700 1 --2- 172.21.15.104:0/1478962898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21d808a470 0x7f21d806fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:12.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 -- 172.21.15.104:0/1478962898 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934010123 0 0) 0x7f21c8001110 con 0x7f21d80b1820 2023-07-14T11:56:12.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 -- 172.21.15.104:0/1478962898 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21c00035d0 con 0x7f21d80b1820 2023-07-14T11:56:12.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 -- 172.21.15.104:0/1478962898 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 620960719 0 0) 0x7f21c00035d0 con 0x7f21d80b1820 2023-07-14T11:56:12.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 -- 172.21.15.104:0/1478962898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21d808a470 msgr2=0x7f21d806fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:12.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 --2- 172.21.15.104:0/1478962898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21d808a470 0x7f21d806fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 -- 172.21.15.104:0/1478962898 >> v2:172.21.15.121:3300/0 conn(0x7f21d80ccb20 msgr2=0x7f21d806ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:12.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 --2- 172.21.15.104:0/1478962898 >> v2:172.21.15.121:3300/0 conn(0x7f21d80ccb20 0x7f21d806ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 -- 172.21.15.104:0/1478962898 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d80ce580 con 0x7f21d80b1820 2023-07-14T11:56:12.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 -- 172.21.15.104:0/1478962898 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f21c8001800 con 0x7f21d80b1820 2023-07-14T11:56:12.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 -- 172.21.15.104:0/1478962898 >> v1:172.21.15.104:6789/0 conn(0x7f21d80b1820 legacy=0x7f21d80cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:12.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 --2- 172.21.15.104:0/1478962898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21c0003a20 0x7f21c0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:12.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 --2- 172.21.15.104:0/1478962898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21d808a470 0x7f21c0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:12.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 --2- 172.21.15.104:0/1478962898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21c0004db0 0x7f21c0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:12.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 -- 172.21.15.104:0/1478962898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80cb8e0 con 0x7f21c0003a20 2023-07-14T11:56:12.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 -- 172.21.15.104:0/1478962898 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c000bb10 con 0x7f21c0004db0 2023-07-14T11:56:12.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d6ffd700 1 -- 172.21.15.104:0/1478962898 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c000bcd0 con 0x7f21d808a470 2023-07-14T11:56:12.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d7fff700 1 --2- 172.21.15.104:0/1478962898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21c0003a20 0x7f21c0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:12.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d7fff700 1 -- 172.21.15.104:0/1478962898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21d808a470 msgr2=0x7f21c0004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:12.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d7fff700 1 --2- 172.21.15.104:0/1478962898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21d808a470 0x7f21c0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d7fff700 1 -- 172.21.15.104:0/1478962898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21c0004db0 msgr2=0x7f21c0005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:12.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d7fff700 1 --2- 172.21.15.104:0/1478962898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21c0004db0 0x7f21c0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.697+0000 7f21d7fff700 1 -- 172.21.15.104:0/1478962898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c00035d0 con 0x7f21c0003a20 2023-07-14T11:56:12.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.701+0000 7f21d7fff700 1 --2- 172.21.15.104:0/1478962898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21c0003a20 0x7f21c0003e30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f21c800c320 tx=0x7f21c00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652e4a6a6f77b251 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:12.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.701+0000 7f21d6ffd700 1 -- 172.21.15.104:0/1478962898 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c800ce60 con 0x7f21c0003a20 2023-07-14T11:56:12.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.701+0000 7f21d6ffd700 1 -- 172.21.15.104:0/1478962898 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c8005980 con 0x7f21c0003a20 2023-07-14T11:56:12.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.701+0000 7f21d6ffd700 1 -- 172.21.15.104:0/1478962898 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8005d10 con 0x7f21c0003a20 2023-07-14T11:56:12.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.705+0000 7f21dd24f700 1 -- 172.21.15.104:0/1478962898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21c0003a20 msgr2=0x7f21c0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:12.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.705+0000 7f21dd24f700 1 --2- 172.21.15.104:0/1478962898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21c0003a20 0x7f21c0003e30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f21c800c320 tx=0x7f21c00058a0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.705+0000 7f21dca4e700 1 -- 172.21.15.104:0/1478962898 reap_dead start 2023-07-14T11:56:12.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.705+0000 7f21dd24f700 1 -- 172.21.15.104:0/1478962898 shutdown_connections 2023-07-14T11:56:12.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.705+0000 7f21dd24f700 1 -- 172.21.15.104:0/1478962898 >> 172.21.15.104:0/1478962898 conn(0x7f21d808a000 msgr2=0x7f21d80d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:12.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.705+0000 7f21dd24f700 1 -- 172.21.15.104:0/1478962898 shutdown_connections 2023-07-14T11:56:12.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.705+0000 7f21dd24f700 1 -- 172.21.15.104:0/1478962898 wait complete. 2023-07-14T11:56:12.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.705+0000 7f21dd24f700 1 Processor -- start 2023-07-14T11:56:12.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.705+0000 7f21dd24f700 1 -- start start 2023-07-14T11:56:12.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21dd24f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21c0003a20 0x7f21d80b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:12.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21dd24f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d8151300 0x7f21d80b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:12.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21dd24f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21d815e270 0x7f21d815e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:12.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21dd24f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d815ec50 con 0x7f21d815e270 2023-07-14T11:56:12.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21dd24f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d81f1090 con 0x7f21d8151300 2023-07-14T11:56:12.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21dd24f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80b60f0 con 0x7f21c0003a20 2023-07-14T11:56:12.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21dca4e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21d815e270 0x7f21d815e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:12.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21d7fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21c0003a20 0x7f21d80b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:12.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21dca4e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21d815e270 0x7f21d815e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50360/0 (socket says 172.21.15.104:50360) 2023-07-14T11:56:12.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21d7fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21c0003a20 0x7f21d80b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:54980/0 (socket says 172.21.15.104:54980) 2023-07-14T11:56:12.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21dca4e700 1 -- 172.21.15.104:0/3874606182 learned_addr learned my addr 172.21.15.104:0/3874606182 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:12.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21d77fe700 1 --2- 172.21.15.104:0/3874606182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d8151300 0x7f21d80b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:12.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21d7fff700 1 -- 172.21.15.104:0/3874606182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d8151300 msgr2=0x7f21d80b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:12.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21d7fff700 1 --2- 172.21.15.104:0/3874606182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d8151300 0x7f21d80b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21d7fff700 1 -- 172.21.15.104:0/3874606182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21d815e270 msgr2=0x7f21d815e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:12.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21d7fff700 1 --2- 172.21.15.104:0/3874606182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21d815e270 0x7f21d815e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:12.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21d7fff700 1 -- 172.21.15.104:0/3874606182 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d80b4e90 con 0x7f21c0003a20 2023-07-14T11:56:12.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21dca4e700 1 --2- 172.21.15.104:0/3874606182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21d815e270 0x7f21d815e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:12.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21d7fff700 1 --2- 172.21.15.104:0/3874606182 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21c0003a20 0x7f21d80b4860 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f21c8014000 tx=0x7f21c8001b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:12.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21d4ff9700 1 -- 172.21.15.104:0/3874606182 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8005d00 con 0x7f21c0003a20 2023-07-14T11:56:12.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21d4ff9700 1 -- 172.21.15.104:0/3874606182 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c8004890 con 0x7f21c0003a20 2023-07-14T11:56:12.706 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21dd24f700 1 -- 172.21.15.104:0/3874606182 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d81f4840 con 0x7f21c0003a20 2023-07-14T11:56:12.706 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21d4ff9700 1 -- 172.21.15.104:0/3874606182 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8005ea0 con 0x7f21c0003a20 2023-07-14T11:56:12.706 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.709+0000 7f21dd24f700 1 -- 172.21.15.104:0/3874606182 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d81f4e00 con 0x7f21c0003a20 2023-07-14T11:56:12.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.713+0000 7f21dd24f700 1 -- 172.21.15.104:0/3874606182 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d806f450 con 0x7f21c0003a20 2023-07-14T11:56:12.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.713+0000 7f21d4ff9700 1 -- 172.21.15.104:0/3874606182 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f21c8004c40 con 0x7f21c0003a20 2023-07-14T11:56:12.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.713+0000 7f21d4ff9700 1 --2- 172.21.15.104:0/3874606182 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21c4066320 0x7f21c40687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:12.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.713+0000 7f21d4ff9700 1 -- 172.21.15.104:0/3874606182 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f21c80852b0 con 0x7f21c0003a20 2023-07-14T11:56:12.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.713+0000 7f21d77fe700 1 --2- 172.21.15.104:0/3874606182 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21c4066320 0x7f21c40687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:12.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.717+0000 7f21d77fe700 1 --2- 172.21.15.104:0/3874606182 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21c4066320 0x7f21c40687d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f21d80b47f0 tx=0x7f21cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:12.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.717+0000 7f21d4ff9700 1 -- 172.21.15.104:0/3874606182 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f21c8052560 con 0x7f21c0003a20 2023-07-14T11:56:12.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:12.929+0000 7f21dd24f700 1 -- 172.21.15.104:0/3874606182 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f21d80b1bf0 con 0x7f21c0003a20 2023-07-14T11:56:13.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.717+0000 7f21d4ff9700 1 -- 172.21.15.104:0/3874606182 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) v1 ==== 118+0+0 (secure 0 0 0) 0x7f21d80b1bf0 con 0x7f21c0003a20 2023-07-14T11:56:13.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.717+0000 7f21dd24f700 1 -- 172.21.15.104:0/3874606182 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21c4066320 msgr2=0x7f21c40687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:13.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.717+0000 7f21dd24f700 1 --2- 172.21.15.104:0/3874606182 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21c4066320 0x7f21c40687d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f21d80b47f0 tx=0x7f21cc008040 comp rx=0 tx=0).stop 2023-07-14T11:56:13.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.717+0000 7f21dd24f700 1 -- 172.21.15.104:0/3874606182 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21c0003a20 msgr2=0x7f21d80b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:13.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.717+0000 7f21dd24f700 1 --2- 172.21.15.104:0/3874606182 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21c0003a20 0x7f21d80b4860 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f21c8014000 tx=0x7f21c8001b90 comp rx=0 tx=0).stop 2023-07-14T11:56:13.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.721+0000 7f21dd24f700 1 -- 172.21.15.104:0/3874606182 shutdown_connections 2023-07-14T11:56:13.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.721+0000 7f21dd24f700 1 --2- 172.21.15.104:0/3874606182 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21c4066320 0x7f21c40687d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:13.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.721+0000 7f21dd24f700 1 --2- 172.21.15.104:0/3874606182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21d815e270 0x7f21d815e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:13.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.721+0000 7f21dd24f700 1 --2- 172.21.15.104:0/3874606182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d8151300 0x7f21d80b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:13.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.721+0000 7f21dd24f700 1 --2- 172.21.15.104:0/3874606182 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21c0003a20 0x7f21d80b4860 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:13.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.721+0000 7f21dd24f700 1 -- 172.21.15.104:0/3874606182 >> 172.21.15.104:0/3874606182 conn(0x7f21d808a000 msgr2=0x7f21d806a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:13.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.721+0000 7f21dd24f700 1 -- 172.21.15.104:0/3874606182 shutdown_connections 2023-07-14T11:56:13.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.721+0000 7f21dd24f700 1 -- 172.21.15.104:0/3874606182 wait complete. 2023-07-14T11:56:13.729 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-14T11:56:13.729 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-14T11:56:13.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.929+0000 7f11a6777700 1 Processor -- start 2023-07-14T11:56:13.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.929+0000 7f11a6777700 1 -- start start 2023-07-14T11:56:13.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.929+0000 7f11a6777700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f11a015df10 0x7f11a015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:13.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.929+0000 7f11a6777700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a0070a30 con 0x7f11a00c8950 2023-07-14T11:56:13.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.929+0000 7f11a6777700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a00b0210 con 0x7f11a00afd80 2023-07-14T11:56:13.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.929+0000 7f11a6777700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a015e830 con 0x7f11a015df10 2023-07-14T11:56:13.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.929+0000 7f11a4d14700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f11a00c8950 0x7f11a006d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:47912/0 (socket says 172.21.15.104:47912) 2023-07-14T11:56:13.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.929+0000 7f11a4d14700 1 -- 172.21.15.104:0/2475477569 learned_addr learned my addr 172.21.15.104:0/2475477569 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:13.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.929+0000 7f119ffff700 1 --2- 172.21.15.104:0/2475477569 >> v2:172.21.15.121:3300/0 conn(0x7f11a015df10 0x7f11a015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:13.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.929+0000 7f119f7fe700 1 -- 172.21.15.104:0/2475477569 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031213834 0 0) 0x7f11a0070a30 con 0x7f11a00c8950 2023-07-14T11:56:13.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.929+0000 7f119f7fe700 1 -- 172.21.15.104:0/2475477569 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f118c0035d0 con 0x7f11a00c8950 2023-07-14T11:56:13.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f119f7fe700 1 -- 172.21.15.104:0/2475477569 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104066813 0 0) 0x7f11a00b0210 con 0x7f11a00afd80 2023-07-14T11:56:13.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f119f7fe700 1 -- 172.21.15.104:0/2475477569 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11a0070a30 con 0x7f11a00afd80 2023-07-14T11:56:13.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f119ffff700 1 -- 172.21.15.104:0/2475477569 >> v1:172.21.15.121:6789/0 conn(0x7f11a00afd80 legacy=0x7f11a00711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:13.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f119ffff700 1 -- 172.21.15.104:0/2475477569 >> v1:172.21.15.104:6789/0 conn(0x7f11a00c8950 legacy=0x7f11a006d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:13.920 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f119ffff700 1 -- 172.21.15.104:0/2475477569 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a00b04f0 con 0x7f11a015df10 2023-07-14T11:56:13.920 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f119ffff700 1 --2- 172.21.15.104:0/2475477569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a015df10 0x7f11a015e2f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1190009850 tx=0x7f1190009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebeee1a27087249e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:13.920 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f119f7fe700 1 -- 172.21.15.104:0/2475477569 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190011830 con 0x7f11a015df10 2023-07-14T11:56:13.921 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f119f7fe700 1 -- 172.21.15.104:0/2475477569 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1190011990 con 0x7f11a015df10 2023-07-14T11:56:13.921 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f119f7fe700 1 -- 172.21.15.104:0/2475477569 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190011d40 con 0x7f11a015df10 2023-07-14T11:56:13.921 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f11a6777700 1 -- 172.21.15.104:0/2475477569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a015df10 msgr2=0x7f11a015e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:13.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f11a6777700 1 --2- 172.21.15.104:0/2475477569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a015df10 0x7f11a015e2f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1190009850 tx=0x7f1190009b60 comp rx=0 tx=0).stop 2023-07-14T11:56:13.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f11a6777700 1 -- 172.21.15.104:0/2475477569 shutdown_connections 2023-07-14T11:56:13.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f11a6777700 1 --2- 172.21.15.104:0/2475477569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a015df10 0x7f11a015e2f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:13.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f11a6777700 1 -- 172.21.15.104:0/2475477569 >> 172.21.15.104:0/2475477569 conn(0x7f11a00cad80 msgr2=0x7f11a00cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:13.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f11a6777700 1 -- 172.21.15.104:0/2475477569 shutdown_connections 2023-07-14T11:56:13.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.933+0000 7f11a6777700 1 -- 172.21.15.104:0/2475477569 wait complete. 2023-07-14T11:56:13.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.937+0000 7f11a6777700 1 Processor -- start 2023-07-14T11:56:13.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.937+0000 7f11a6777700 1 -- start start 2023-07-14T11:56:13.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.937+0000 7f11a6777700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11a00afd80 0x7f11a00cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:13.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.937+0000 7f11a6777700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a00c8950 0x7f11a0155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:13.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.937+0000 7f11a6777700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a00cc8e0 0x7f11a00c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:13.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.937+0000 7f11a6777700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a0156470 con 0x7f11a00afd80 2023-07-14T11:56:13.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.937+0000 7f11a6777700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00cce10 con 0x7f11a00cc8e0 2023-07-14T11:56:13.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.937+0000 7f11a6777700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00cd0f0 con 0x7f11a00c8950 2023-07-14T11:56:13.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.937+0000 7f119ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11a00afd80 0x7f11a00cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:13.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.937+0000 7f119ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11a00afd80 0x7f11a00cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50370/0 (socket says 172.21.15.104:50370) 2023-07-14T11:56:13.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.937+0000 7f119ffff700 1 -- 172.21.15.104:0/1828545618 learned_addr learned my addr 172.21.15.104:0/1828545618 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:13.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.937+0000 7f11977fe700 1 --2- 172.21.15.104:0/1828545618 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a00c8950 0x7f11a0155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:13.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.937+0000 7f11a4d14700 1 --2- 172.21.15.104:0/1828545618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a00cc8e0 0x7f11a00c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:13.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.941+0000 7f119ffff700 1 -- 172.21.15.104:0/1828545618 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a00c8950 msgr2=0x7f11a0155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:13.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.941+0000 7f119ffff700 1 --2- 172.21.15.104:0/1828545618 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a00c8950 0x7f11a0155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:13.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.941+0000 7f119ffff700 1 -- 172.21.15.104:0/1828545618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a00cc8e0 msgr2=0x7f11a00c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:13.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.941+0000 7f119ffff700 1 --2- 172.21.15.104:0/1828545618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a00cc8e0 0x7f11a00c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:13.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.941+0000 7f119ffff700 1 -- 172.21.15.104:0/1828545618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11900094b0 con 0x7f11a00afd80 2023-07-14T11:56:13.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.941+0000 7f11a4d14700 1 --2- 172.21.15.104:0/1828545618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a00cc8e0 0x7f11a00c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:13.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.941+0000 7f119ffff700 1 --2- 172.21.15.104:0/1828545618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11a00afd80 0x7f11a00cdb30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f11a006e410 tx=0x7f1190004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:13.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.941+0000 7f119d7fa700 1 -- 172.21.15.104:0/1828545618 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119001aa10 con 0x7f11a00afd80 2023-07-14T11:56:13.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.941+0000 7f11a6777700 1 -- 172.21.15.104:0/1828545618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a00c6ad0 con 0x7f11a00afd80 2023-07-14T11:56:13.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.941+0000 7f119d7fa700 1 -- 172.21.15.104:0/1828545618 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119001f070 con 0x7f11a00afd80 2023-07-14T11:56:13.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.941+0000 7f119d7fa700 1 -- 172.21.15.104:0/1828545618 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119001adb0 con 0x7f11a00afd80 2023-07-14T11:56:13.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.941+0000 7f11a6777700 1 -- 172.21.15.104:0/1828545618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a0157820 con 0x7f11a00afd80 2023-07-14T11:56:13.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.941+0000 7f11a6777700 1 -- 172.21.15.104:0/1828545618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a00c6ff0 con 0x7f11a00afd80 2023-07-14T11:56:13.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.945+0000 7f119d7fa700 1 -- 172.21.15.104:0/1828545618 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f119001b430 con 0x7f11a00afd80 2023-07-14T11:56:13.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.945+0000 7f119d7fa700 1 --2- 172.21.15.104:0/1828545618 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f11800662f0 0x7f11800687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:13.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.945+0000 7f119d7fa700 1 -- 172.21.15.104:0/1828545618 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1190084f30 con 0x7f11a00afd80 2023-07-14T11:56:13.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.945+0000 7f11977fe700 1 --2- 172.21.15.104:0/1828545618 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f11800662f0 0x7f11800687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:13.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.945+0000 7f11977fe700 1 --2- 172.21.15.104:0/1828545618 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f11800662f0 0x7f11800687a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f11a00c5190 tx=0x7f1188009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:13.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:13.953+0000 7f119d7fa700 1 -- 172.21.15.104:0/1828545618 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f11a00c6ff0 con 0x7f11a00afd80 2023-07-14T11:56:14.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.169+0000 7f11a6777700 1 -- 172.21.15.104:0/1828545618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f11a004dcb0 con 0x7f11a00afd80 2023-07-14T11:56:14.716 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.729+0000 7f119d7fa700 1 -- 172.21.15.104:0/1828545618 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7f11900210c0 con 0x7f11a00afd80 2023-07-14T11:56:14.720 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.733+0000 7f11a6777700 1 -- 172.21.15.104:0/1828545618 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f11800662f0 msgr2=0x7f11800687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:14.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.733+0000 7f11a6777700 1 --2- 172.21.15.104:0/1828545618 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f11800662f0 0x7f11800687a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f11a00c5190 tx=0x7f1188009040 comp rx=0 tx=0).stop 2023-07-14T11:56:14.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.733+0000 7f11a6777700 1 -- 172.21.15.104:0/1828545618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11a00afd80 msgr2=0x7f11a00cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:14.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.733+0000 7f11a6777700 1 --2- 172.21.15.104:0/1828545618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11a00afd80 0x7f11a00cdb30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f11a006e410 tx=0x7f1190004770 comp rx=0 tx=0).stop 2023-07-14T11:56:14.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.733+0000 7f11a6777700 1 -- 172.21.15.104:0/1828545618 shutdown_connections 2023-07-14T11:56:14.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.733+0000 7f11a6777700 1 --2- 172.21.15.104:0/1828545618 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f11800662f0 0x7f11800687a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:14.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.733+0000 7f11a6777700 1 --2- 172.21.15.104:0/1828545618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a00cc8e0 0x7f11a00c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:14.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.733+0000 7f11a6777700 1 --2- 172.21.15.104:0/1828545618 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11a00c8950 0x7f11a0155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:14.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.733+0000 7f11a6777700 1 --2- 172.21.15.104:0/1828545618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11a00afd80 0x7f11a00cdb30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:14.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.733+0000 7f11a6777700 1 -- 172.21.15.104:0/1828545618 >> 172.21.15.104:0/1828545618 conn(0x7f11a00cad80 msgr2=0x7f11a006c120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:14.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.733+0000 7f11a6777700 1 -- 172.21.15.104:0/1828545618 shutdown_connections 2023-07-14T11:56:14.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.733+0000 7f11a6777700 1 -- 172.21.15.104:0/1828545618 wait complete. 2023-07-14T11:56:14.743 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-14T11:56:14.743 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:56:14.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.937+0000 7ffb61595700 1 Processor -- start 2023-07-14T11:56:14.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.941+0000 7ffb61595700 1 -- start start 2023-07-14T11:56:14.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.941+0000 7ffb61595700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffb5c0b0de0 0x7ffb5c1669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:14.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.941+0000 7ffb61595700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb5c0af0a0 con 0x7ffb5c15bff0 2023-07-14T11:56:14.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.941+0000 7ffb61595700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb5c0cf720 con 0x7ffb5c0cd240 2023-07-14T11:56:14.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.941+0000 7ffb61595700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5c0af2f0 con 0x7ffb5c0b0de0 2023-07-14T11:56:14.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.941+0000 7ffb5bfff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ffb5c15bff0 0x7ffb5c0ccb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:47922/0 (socket says 172.21.15.104:47922) 2023-07-14T11:56:14.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.941+0000 7ffb5bfff700 1 -- 172.21.15.104:0/939539056 learned_addr learned my addr 172.21.15.104:0/939539056 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:14.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.941+0000 7ffb60d94700 1 --2- 172.21.15.104:0/939539056 >> v2:172.21.15.121:3300/0 conn(0x7ffb5c0b0de0 0x7ffb5c1669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:14.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.941+0000 7ffb5b7fe700 1 -- 172.21.15.104:0/939539056 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810425491 0 0) 0x7ffb5c0af0a0 con 0x7ffb5c15bff0 2023-07-14T11:56:14.931 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.945+0000 7ffb5b7fe700 1 -- 172.21.15.104:0/939539056 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb480035d0 con 0x7ffb5c15bff0 2023-07-14T11:56:14.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.945+0000 7ffb60d94700 1 -- 172.21.15.104:0/939539056 >> v1:172.21.15.121:6789/0 conn(0x7ffb5c0cd240 legacy=0x7ffb5c0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:14.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.945+0000 7ffb60d94700 1 -- 172.21.15.104:0/939539056 >> v1:172.21.15.104:6789/0 conn(0x7ffb5c15bff0 legacy=0x7ffb5c0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:14.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.945+0000 7ffb60d94700 1 -- 172.21.15.104:0/939539056 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5c0af5d0 con 0x7ffb5c0b0de0 2023-07-14T11:56:14.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.945+0000 7ffb60d94700 1 --2- 172.21.15.104:0/939539056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb5c0b0de0 0x7ffb5c1669d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009850 tx=0x7ffb54009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=545379f4978abb45 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:14.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.945+0000 7ffb5b7fe700 1 -- 172.21.15.104:0/939539056 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54011830 con 0x7ffb5c0b0de0 2023-07-14T11:56:14.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.945+0000 7ffb5b7fe700 1 -- 172.21.15.104:0/939539056 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb54011990 con 0x7ffb5c0b0de0 2023-07-14T11:56:14.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.945+0000 7ffb5b7fe700 1 -- 172.21.15.104:0/939539056 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54011d40 con 0x7ffb5c0b0de0 2023-07-14T11:56:14.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.945+0000 7ffb61595700 1 -- 172.21.15.104:0/939539056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb5c0b0de0 msgr2=0x7ffb5c1669d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:14.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.945+0000 7ffb61595700 1 --2- 172.21.15.104:0/939539056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb5c0b0de0 0x7ffb5c1669d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009850 tx=0x7ffb54009b60 comp rx=0 tx=0).stop 2023-07-14T11:56:14.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.945+0000 7ffb61595700 1 -- 172.21.15.104:0/939539056 shutdown_connections 2023-07-14T11:56:14.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.945+0000 7ffb61595700 1 --2- 172.21.15.104:0/939539056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb5c0b0de0 0x7ffb5c1669d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:14.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.945+0000 7ffb61595700 1 -- 172.21.15.104:0/939539056 >> 172.21.15.104:0/939539056 conn(0x7ffb5c060c40 msgr2=0x7ffb5c09b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:14.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.945+0000 7ffb61595700 1 -- 172.21.15.104:0/939539056 shutdown_connections 2023-07-14T11:56:14.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.945+0000 7ffb61595700 1 -- 172.21.15.104:0/939539056 wait complete. 2023-07-14T11:56:14.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.949+0000 7ffb61595700 1 Processor -- start 2023-07-14T11:56:14.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.949+0000 7ffb61595700 1 -- start start 2023-07-14T11:56:14.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.949+0000 7ffb61595700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb5c0b0de0 0x7ffb5c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:14.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.949+0000 7ffb61595700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb5c0cd240 0x7ffb5c1576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:14.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.949+0000 7ffb61595700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb5c15bff0 0x7ffb5c157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:14.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.949+0000 7ffb61595700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c1f5640 con 0x7ffb5c0b0de0 2023-07-14T11:56:14.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.949+0000 7ffb61595700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c1f4a50 con 0x7ffb5c15bff0 2023-07-14T11:56:14.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.949+0000 7ffb61595700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c158170 con 0x7ffb5c0cd240 2023-07-14T11:56:14.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.949+0000 7ffb5bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb5c0b0de0 0x7ffb5c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:14.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.949+0000 7ffb5bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb5c0b0de0 0x7ffb5c15e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50386/0 (socket says 172.21.15.104:50386) 2023-07-14T11:56:14.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.949+0000 7ffb5bfff700 1 -- 172.21.15.104:0/3462788869 learned_addr learned my addr 172.21.15.104:0/3462788869 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:14.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.953+0000 7ffb60d94700 1 --2- 172.21.15.104:0/3462788869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb5c15bff0 0x7ffb5c157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:14.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.953+0000 7ffb537fe700 1 --2- 172.21.15.104:0/3462788869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb5c0cd240 0x7ffb5c1576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:14.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.953+0000 7ffb5bfff700 1 -- 172.21.15.104:0/3462788869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb5c0cd240 msgr2=0x7ffb5c1576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:14.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.953+0000 7ffb5bfff700 1 --2- 172.21.15.104:0/3462788869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb5c0cd240 0x7ffb5c1576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:14.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.953+0000 7ffb5bfff700 1 -- 172.21.15.104:0/3462788869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb5c15bff0 msgr2=0x7ffb5c157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:14.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.953+0000 7ffb5bfff700 1 --2- 172.21.15.104:0/3462788869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb5c15bff0 0x7ffb5c157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:14.941 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.953+0000 7ffb5bfff700 1 -- 172.21.15.104:0/3462788869 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb540094b0 con 0x7ffb5c0b0de0 2023-07-14T11:56:14.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.953+0000 7ffb5bfff700 1 --2- 172.21.15.104:0/3462788869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb5c0b0de0 0x7ffb5c15e750 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00cbf0 tx=0x7ffb4c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:14.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.953+0000 7ffb597fa700 1 -- 172.21.15.104:0/3462788869 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c007480 con 0x7ffb5c0b0de0 2023-07-14T11:56:14.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.953+0000 7ffb597fa700 1 -- 172.21.15.104:0/3462788869 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4c0075e0 con 0x7ffb5c0b0de0 2023-07-14T11:56:14.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.953+0000 7ffb597fa700 1 -- 172.21.15.104:0/3462788869 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c007970 con 0x7ffb5c0b0de0 2023-07-14T11:56:14.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.953+0000 7ffb61595700 1 -- 172.21.15.104:0/3462788869 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5c1f4ca0 con 0x7ffb5c0b0de0 2023-07-14T11:56:14.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.953+0000 7ffb61595700 1 -- 172.21.15.104:0/3462788869 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5c1f5260 con 0x7ffb5c0b0de0 2023-07-14T11:56:14.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.953+0000 7ffb61595700 1 -- 172.21.15.104:0/3462788869 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5c0ccb10 con 0x7ffb5c0b0de0 2023-07-14T11:56:14.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.957+0000 7ffb597fa700 1 -- 172.21.15.104:0/3462788869 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ffb4c01b3b0 con 0x7ffb5c0b0de0 2023-07-14T11:56:14.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.957+0000 7ffb597fa700 1 --2- 172.21.15.104:0/3462788869 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffb3c0662f0 0x7ffb3c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:14.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.957+0000 7ffb597fa700 1 -- 172.21.15.104:0/3462788869 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ffb4c0837d0 con 0x7ffb5c0b0de0 2023-07-14T11:56:14.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.957+0000 7ffb537fe700 1 --2- 172.21.15.104:0/3462788869 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffb3c0662f0 0x7ffb3c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:14.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.961+0000 7ffb537fe700 1 --2- 172.21.15.104:0/3462788869 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffb3c0662f0 0x7ffb3c0687a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c1f25d0 tx=0x7ffb44009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:14.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:14.961+0000 7ffb597fa700 1 -- 172.21.15.104:0/3462788869 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ffb4c050b00 con 0x7ffb5c0b0de0 2023-07-14T11:56:15.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.165+0000 7ffb61595700 1 -- 172.21.15.104:0/3462788869 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb5c04e500 con 0x7ffb5c0b0de0 2023-07-14T11:56:15.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.165+0000 7ffb597fa700 1 -- 172.21.15.104:0/3462788869 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4722 (secure 0 0 0) 0x7ffb4c054120 con 0x7ffb5c0b0de0 2023-07-14T11:56:15.153 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:15.153 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/4186544624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":4186544624},{"type":"v1","addr":"172.21.15.104:6837","nonce":4186544624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4234575863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4234575863},{"type":"v1","addr":"172.21.15.121:6835","nonce":4234575863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6220,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4127455794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4127455794},{"type":"v1","addr":"172.21.15.121:6833","nonce":4127455794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:56:11.123432+0000","modified":"2023-07-14T11:56:14.719624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/2705739882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2705739882},{"type":"v1","addr":"172.21.15.104:6836","nonce":2705739882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-14T11:56:15.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.169+0000 7ffb61595700 1 -- 172.21.15.104:0/3462788869 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffb3c0662f0 msgr2=0x7ffb3c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:15.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.169+0000 7ffb61595700 1 --2- 172.21.15.104:0/3462788869 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffb3c0662f0 0x7ffb3c0687a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c1f25d0 tx=0x7ffb44009040 comp rx=0 tx=0).stop 2023-07-14T11:56:15.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.169+0000 7ffb61595700 1 -- 172.21.15.104:0/3462788869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb5c0b0de0 msgr2=0x7ffb5c15e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:15.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.169+0000 7ffb61595700 1 --2- 172.21.15.104:0/3462788869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb5c0b0de0 0x7ffb5c15e750 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00cbf0 tx=0x7ffb4c00cc20 comp rx=0 tx=0).stop 2023-07-14T11:56:15.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.169+0000 7ffb61595700 1 -- 172.21.15.104:0/3462788869 shutdown_connections 2023-07-14T11:56:15.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.169+0000 7ffb61595700 1 --2- 172.21.15.104:0/3462788869 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffb3c0662f0 0x7ffb3c0687a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.169+0000 7ffb61595700 1 --2- 172.21.15.104:0/3462788869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb5c15bff0 0x7ffb5c157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.169+0000 7ffb61595700 1 --2- 172.21.15.104:0/3462788869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb5c0cd240 0x7ffb5c1576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.169+0000 7ffb61595700 1 --2- 172.21.15.104:0/3462788869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb5c0b0de0 0x7ffb5c15e750 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.169+0000 7ffb61595700 1 -- 172.21.15.104:0/3462788869 >> 172.21.15.104:0/3462788869 conn(0x7ffb5c060c40 msgr2=0x7ffb5c158a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:15.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.169+0000 7ffb61595700 1 -- 172.21.15.104:0/3462788869 shutdown_connections 2023-07-14T11:56:15.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.169+0000 7ffb61595700 1 -- 172.21.15.104:0/3462788869 wait complete. 2023-07-14T11:56:15.158 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 27 2023-07-14T11:56:15.176 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:56:15.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.357+0000 7fd91b18d700 1 Processor -- start 2023-07-14T11:56:15.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.357+0000 7fd91b18d700 1 -- start start 2023-07-14T11:56:15.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.361+0000 7fd91b18d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd9140ce100 0x7fd9140ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:15.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.361+0000 7fd91b18d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9140cd250 0x7fd91406e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:15.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.361+0000 7fd91b18d700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9141607d0 con 0x7fd9140cee20 2023-07-14T11:56:15.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.361+0000 7fd91b18d700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91406e4e0 con 0x7fd9140ce100 2023-07-14T11:56:15.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.361+0000 7fd91b18d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91415fbc0 con 0x7fd9140cd250 2023-07-14T11:56:15.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.361+0000 7fd918f29700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd9140ce100 0x7fd9140ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:15.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.361+0000 7fd918f29700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd9140ce100 0x7fd9140ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:53168/0 (socket says 172.21.15.104:53168) 2023-07-14T11:56:15.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.361+0000 7fd918f29700 1 -- 172.21.15.104:0/1633149471 learned_addr learned my addr 172.21.15.104:0/1633149471 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:15.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.361+0000 7fd913fff700 1 --2- 172.21.15.104:0/1633149471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9140cd250 0x7fd91406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:15.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.361+0000 7fd918f29700 1 -- 172.21.15.104:0/1633149471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9140cd250 msgr2=0x7fd91406e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:15.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.361+0000 7fd918f29700 1 --2- 172.21.15.104:0/1633149471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9140cd250 0x7fd91406e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.361+0000 7fd918f29700 1 -- 172.21.15.104:0/1633149471 >> v1:172.21.15.121:6789/0 conn(0x7fd9140cee20 legacy=0x7fd9140b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:15.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.361+0000 7fd918f29700 1 -- 172.21.15.104:0/1633149471 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91415fea0 con 0x7fd9140ce100 2023-07-14T11:56:15.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.361+0000 7fd913fff700 1 --2- 172.21.15.104:0/1633149471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9140cd250 0x7fd91406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:15.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.365+0000 7fd918f29700 1 --2- 172.21.15.104:0/1633149471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9140ce100 0x7fd9140ce8e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd908009910 tx=0x7fd908009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b65a6c08fb764f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:15.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.365+0000 7fd9137fe700 1 -- 172.21.15.104:0/1633149471 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908011830 con 0x7fd9140ce100 2023-07-14T11:56:15.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.365+0000 7fd9137fe700 1 -- 172.21.15.104:0/1633149471 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd908011990 con 0x7fd9140ce100 2023-07-14T11:56:15.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.365+0000 7fd9137fe700 1 -- 172.21.15.104:0/1633149471 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908011d40 con 0x7fd9140ce100 2023-07-14T11:56:15.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.365+0000 7fd91b18d700 1 -- 172.21.15.104:0/1633149471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9140ce100 msgr2=0x7fd9140ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:15.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.365+0000 7fd91b18d700 1 --2- 172.21.15.104:0/1633149471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9140ce100 0x7fd9140ce8e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd908009910 tx=0x7fd908009c20 comp rx=0 tx=0).stop 2023-07-14T11:56:15.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.365+0000 7fd91b18d700 1 -- 172.21.15.104:0/1633149471 shutdown_connections 2023-07-14T11:56:15.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.365+0000 7fd91b18d700 1 --2- 172.21.15.104:0/1633149471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9140cd250 0x7fd91406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.365+0000 7fd91b18d700 1 --2- 172.21.15.104:0/1633149471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9140ce100 0x7fd9140ce8e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.365+0000 7fd91b18d700 1 -- 172.21.15.104:0/1633149471 >> 172.21.15.104:0/1633149471 conn(0x7fd9140cbdd0 msgr2=0x7fd9140cc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:15.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.365+0000 7fd91b18d700 1 -- 172.21.15.104:0/1633149471 shutdown_connections 2023-07-14T11:56:15.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.365+0000 7fd91b18d700 1 -- 172.21.15.104:0/1633149471 wait complete. 2023-07-14T11:56:15.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.369+0000 7fd91b18d700 1 Processor -- start 2023-07-14T11:56:15.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.369+0000 7fd91b18d700 1 -- start start 2023-07-14T11:56:15.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.369+0000 7fd91b18d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9140cd250 0x7fd9140b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:15.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.369+0000 7fd91b18d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9140ce100 0x7fd9140b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:15.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.369+0000 7fd91b18d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9140cee20 0x7fd91414a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:15.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.369+0000 7fd91b18d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140b2cc0 con 0x7fd9140ce100 2023-07-14T11:56:15.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.369+0000 7fd91b18d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140b2fa0 con 0x7fd9140cee20 2023-07-14T11:56:15.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.369+0000 7fd91b18d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140b3280 con 0x7fd9140cd250 2023-07-14T11:56:15.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd913fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9140ce100 0x7fd9140b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:15.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd918f29700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9140cd250 0x7fd9140b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:15.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd913fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9140ce100 0x7fd9140b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50390/0 (socket says 172.21.15.104:50390) 2023-07-14T11:56:15.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd913fff700 1 -- 172.21.15.104:0/1490291683 learned_addr learned my addr 172.21.15.104:0/1490291683 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:15.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd91972a700 1 --2- 172.21.15.104:0/1490291683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9140cee20 0x7fd91414a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:15.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd913fff700 1 -- 172.21.15.104:0/1490291683 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9140cd250 msgr2=0x7fd9140b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:15.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd913fff700 1 --2- 172.21.15.104:0/1490291683 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9140cd250 0x7fd9140b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd913fff700 1 -- 172.21.15.104:0/1490291683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9140cee20 msgr2=0x7fd91414a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:15.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd913fff700 1 --2- 172.21.15.104:0/1490291683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9140cee20 0x7fd91414a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd913fff700 1 -- 172.21.15.104:0/1490291683 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9000094b0 con 0x7fd9140ce100 2023-07-14T11:56:15.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd913fff700 1 --2- 172.21.15.104:0/1490291683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9140ce100 0x7fd9140b41d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd90000ebb0 tx=0x7fd90000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:15.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd9117fa700 1 -- 172.21.15.104:0/1490291683 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90000c850 con 0x7fd9140ce100 2023-07-14T11:56:15.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd9117fa700 1 -- 172.21.15.104:0/1490291683 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90000c9b0 con 0x7fd9140ce100 2023-07-14T11:56:15.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd9117fa700 1 -- 172.21.15.104:0/1490291683 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90000cd40 con 0x7fd9140ce100 2023-07-14T11:56:15.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd91b18d700 1 -- 172.21.15.104:0/1490291683 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd908009570 con 0x7fd9140ce100 2023-07-14T11:56:15.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.373+0000 7fd91b18d700 1 -- 172.21.15.104:0/1490291683 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9140b1ad0 con 0x7fd9140ce100 2023-07-14T11:56:15.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.377+0000 7fd91b18d700 1 -- 172.21.15.104:0/1490291683 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91406f250 con 0x7fd9140ce100 2023-07-14T11:56:15.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.377+0000 7fd9117fa700 1 -- 172.21.15.104:0/1490291683 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd900013470 con 0x7fd9140ce100 2023-07-14T11:56:15.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.381+0000 7fd9117fa700 1 --2- 172.21.15.104:0/1490291683 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8f80662f0 0x7fd8f80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:15.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.381+0000 7fd9117fa700 1 -- 172.21.15.104:0/1490291683 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd900083ec0 con 0x7fd9140ce100 2023-07-14T11:56:15.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.381+0000 7fd918f29700 1 --2- 172.21.15.104:0/1490291683 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8f80662f0 0x7fd8f80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:15.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.381+0000 7fd918f29700 1 --2- 172.21.15.104:0/1490291683 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8f80662f0 0x7fd8f80687a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd908000c00 tx=0x7fd908008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:15.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.385+0000 7fd9117fa700 1 -- 172.21.15.104:0/1490291683 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fd9000511f0 con 0x7fd9140ce100 2023-07-14T11:56:15.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.553+0000 7fd91b18d700 1 -- 172.21.15.104:0/1490291683 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd91406de00 con 0x7fd9140ce100 2023-07-14T11:56:15.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.553+0000 7fd9117fa700 1 -- 172.21.15.104:0/1490291683 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+18534 (secure 0 0 0) 0x7fd900017090 con 0x7fd9140ce100 2023-07-14T11:56:15.543 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:15.543 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":59,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:56:12.692633+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:56:08.755551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-14T11:56:10.132688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:56:15.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.557+0000 7fd91b18d700 1 -- 172.21.15.104:0/1490291683 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8f80662f0 msgr2=0x7fd8f80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:15.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.557+0000 7fd91b18d700 1 --2- 172.21.15.104:0/1490291683 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8f80662f0 0x7fd8f80687a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd908000c00 tx=0x7fd908008030 comp rx=0 tx=0).stop 2023-07-14T11:56:15.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.557+0000 7fd91b18d700 1 -- 172.21.15.104:0/1490291683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9140ce100 msgr2=0x7fd9140b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:15.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.557+0000 7fd91b18d700 1 --2- 172.21.15.104:0/1490291683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9140ce100 0x7fd9140b41d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd90000ebb0 tx=0x7fd90000eec0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.557+0000 7fd91b18d700 1 -- 172.21.15.104:0/1490291683 shutdown_connections 2023-07-14T11:56:15.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.557+0000 7fd91b18d700 1 --2- 172.21.15.104:0/1490291683 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8f80662f0 0x7fd8f80687a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.557+0000 7fd91b18d700 1 --2- 172.21.15.104:0/1490291683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9140ce100 0x7fd9140b41d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.557+0000 7fd91b18d700 1 --2- 172.21.15.104:0/1490291683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9140cee20 0x7fd91414a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.557+0000 7fd91b18d700 1 --2- 172.21.15.104:0/1490291683 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9140cd250 0x7fd9140b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.557+0000 7fd91b18d700 1 -- 172.21.15.104:0/1490291683 >> 172.21.15.104:0/1490291683 conn(0x7fd9140cbdd0 msgr2=0x7fd91406af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:15.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.557+0000 7fd91b18d700 1 -- 172.21.15.104:0/1490291683 shutdown_connections 2023-07-14T11:56:15.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.557+0000 7fd91b18d700 1 -- 172.21.15.104:0/1490291683 wait complete. 2023-07-14T11:56:15.569 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-14T11:56:15.569 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:56:15.736 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.749+0000 7f198759e700 1 Processor -- start 2023-07-14T11:56:15.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.749+0000 7f198759e700 1 -- start start 2023-07-14T11:56:15.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.749+0000 7f198759e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f198806dc20 0x7f19880cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:15.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.749+0000 7f198759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19881596f0 0x7f19880ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:15.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.749+0000 7f198759e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1988159d60 con 0x7f19880c9820 2023-07-14T11:56:15.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.749+0000 7f198759e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19880cd250 con 0x7f198806dc20 2023-07-14T11:56:15.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.749+0000 7f198759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19880c8a60 con 0x7f19881596f0 2023-07-14T11:56:15.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.749+0000 7f198659c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f198806dc20 0x7f19880cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:15.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.749+0000 7f198659c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f198806dc20 0x7f19880cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50402/0 (socket says 172.21.15.104:50402) 2023-07-14T11:56:15.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.749+0000 7f198659c700 1 -- 172.21.15.104:0/857821277 learned_addr learned my addr 172.21.15.104:0/857821277 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:15.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.749+0000 7f1986d9d700 1 --2- 172.21.15.104:0/857821277 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19881596f0 0x7f19880ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:15.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f198659c700 1 -- 172.21.15.104:0/857821277 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19881596f0 msgr2=0x7f19880ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:15.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f198659c700 1 --2- 172.21.15.104:0/857821277 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19881596f0 0x7f19880ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f198659c700 1 -- 172.21.15.104:0/857821277 >> v1:172.21.15.121:6789/0 conn(0x7f19880c9820 legacy=0x7f1988158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:15.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f198659c700 1 -- 172.21.15.104:0/857821277 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19880c8d40 con 0x7f198806dc20 2023-07-14T11:56:15.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f1986d9d700 1 --2- 172.21.15.104:0/857821277 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19881596f0 0x7f19880ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:15.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f198659c700 1 --2- 172.21.15.104:0/857821277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f198806dc20 0x7f19880cded0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f197800bbb0 tx=0x7f197800bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b5f6b7e5db8929 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:15.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f1985d9b700 1 -- 172.21.15.104:0/857821277 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197800dd30 con 0x7f198806dc20 2023-07-14T11:56:15.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f1985d9b700 1 -- 172.21.15.104:0/857821277 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197800de90 con 0x7f198806dc20 2023-07-14T11:56:15.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f1985d9b700 1 -- 172.21.15.104:0/857821277 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978010690 con 0x7f198806dc20 2023-07-14T11:56:15.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f198759e700 1 -- 172.21.15.104:0/857821277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f198806dc20 msgr2=0x7f19880cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:15.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f198759e700 1 --2- 172.21.15.104:0/857821277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f198806dc20 0x7f19880cded0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f197800bbb0 tx=0x7f197800bec0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f198759e700 1 -- 172.21.15.104:0/857821277 shutdown_connections 2023-07-14T11:56:15.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f198759e700 1 --2- 172.21.15.104:0/857821277 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19881596f0 0x7f19880ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f198759e700 1 --2- 172.21.15.104:0/857821277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f198806dc20 0x7f19880cded0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f198759e700 1 -- 172.21.15.104:0/857821277 >> 172.21.15.104:0/857821277 conn(0x7f198806e960 msgr2=0x7f198806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:15.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f198759e700 1 -- 172.21.15.104:0/857821277 shutdown_connections 2023-07-14T11:56:15.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.753+0000 7f198759e700 1 -- 172.21.15.104:0/857821277 wait complete. 2023-07-14T11:56:15.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.757+0000 7f198759e700 1 Processor -- start 2023-07-14T11:56:15.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.757+0000 7f198759e700 1 -- start start 2023-07-14T11:56:15.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.757+0000 7f198759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198806dc20 0x7f19880adb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:15.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.757+0000 7f198759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19880c9820 0x7f19880af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:15.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.757+0000 7f198759e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19881596f0 0x7f19880c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:15.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.757+0000 7f198759e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19881f22a0 con 0x7f19881596f0 2023-07-14T11:56:15.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.757+0000 7f198759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19881f1620 con 0x7f19880c9820 2023-07-14T11:56:15.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.757+0000 7f198759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19881f1900 con 0x7f198806dc20 2023-07-14T11:56:15.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.757+0000 7f1986d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19881596f0 0x7f19880c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:15.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.757+0000 7f198659c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198806dc20 0x7f19880adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:15.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.757+0000 7f1986d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19881596f0 0x7f19880c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50414/0 (socket says 172.21.15.104:50414) 2023-07-14T11:56:15.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.757+0000 7f1986d9d700 1 -- 172.21.15.104:0/3066119346 learned_addr learned my addr 172.21.15.104:0/3066119346 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:15.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.757+0000 7f197ffff700 1 --2- 172.21.15.104:0/3066119346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19880c9820 0x7f19880af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:15.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.761+0000 7f198659c700 1 -- 172.21.15.104:0/3066119346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19880c9820 msgr2=0x7f19880af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:15.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.761+0000 7f198659c700 1 --2- 172.21.15.104:0/3066119346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19880c9820 0x7f19880af880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.761+0000 7f198659c700 1 -- 172.21.15.104:0/3066119346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19881596f0 msgr2=0x7f19880c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:15.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.761+0000 7f198659c700 1 --2- 172.21.15.104:0/3066119346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19881596f0 0x7f19880c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.761+0000 7f198659c700 1 -- 172.21.15.104:0/3066119346 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980009a70 con 0x7f198806dc20 2023-07-14T11:56:15.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.761+0000 7f1986d9d700 1 --2- 172.21.15.104:0/3066119346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19881596f0 0x7f19880c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:15.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.761+0000 7f198659c700 1 --2- 172.21.15.104:0/3066119346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198806dc20 0x7f19880adb60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1978018000 tx=0x7f1978004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:15.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.761+0000 7f197effd700 1 -- 172.21.15.104:0/3066119346 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978003c10 con 0x7f198806dc20 2023-07-14T11:56:15.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.761+0000 7f197effd700 1 -- 172.21.15.104:0/3066119346 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1978003d90 con 0x7f198806dc20 2023-07-14T11:56:15.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.761+0000 7f198759e700 1 -- 172.21.15.104:0/3066119346 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197800b810 con 0x7f198806dc20 2023-07-14T11:56:15.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.761+0000 7f197effd700 1 -- 172.21.15.104:0/3066119346 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19780106b0 con 0x7f198806dc20 2023-07-14T11:56:15.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.761+0000 7f198759e700 1 -- 172.21.15.104:0/3066119346 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19881f1ec0 con 0x7f198806dc20 2023-07-14T11:56:15.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.761+0000 7f198759e700 1 -- 172.21.15.104:0/3066119346 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198804e500 con 0x7f198806dc20 2023-07-14T11:56:15.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.765+0000 7f197effd700 1 -- 172.21.15.104:0/3066119346 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1978010810 con 0x7f198806dc20 2023-07-14T11:56:15.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.765+0000 7f197effd700 1 --2- 172.21.15.104:0/3066119346 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19680662f0 0x7f19680687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:15.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.765+0000 7f197effd700 1 -- 172.21.15.104:0/3066119346 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1978085880 con 0x7f198806dc20 2023-07-14T11:56:15.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.765+0000 7f197ffff700 1 --2- 172.21.15.104:0/3066119346 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19680662f0 0x7f19680687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:15.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.765+0000 7f197ffff700 1 --2- 172.21.15.104:0/3066119346 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19680662f0 0x7f19680687a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f19881f5a10 tx=0x7f1970009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:15.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.773+0000 7f197effd700 1 -- 172.21.15.104:0/3066119346 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f1978052c30 con 0x7f198806dc20 2023-07-14T11:56:15.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.941+0000 7f198759e700 1 -- 172.21.15.104:0/3066119346 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19880c9300 con 0x7f19680662f0 2023-07-14T11:56:15.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.957+0000 7f197effd700 1 -- 172.21.15.104:0/3066119346 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303908 (secure 0 0 0) 0x7f19700015e0 con 0x7f19680662f0 2023-07-14T11:56:15.948 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:15.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.965+0000 7f198759e700 1 -- 172.21.15.104:0/3066119346 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19680662f0 msgr2=0x7f19680687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:15.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.965+0000 7f198759e700 1 --2- 172.21.15.104:0/3066119346 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19680662f0 0x7f19680687a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f19881f5a10 tx=0x7f1970009040 comp rx=0 tx=0).stop 2023-07-14T11:56:15.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.965+0000 7f198759e700 1 -- 172.21.15.104:0/3066119346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198806dc20 msgr2=0x7f19880adb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:15.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.965+0000 7f198759e700 1 --2- 172.21.15.104:0/3066119346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198806dc20 0x7f19880adb60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1978018000 tx=0x7f1978004770 comp rx=0 tx=0).stop 2023-07-14T11:56:15.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.965+0000 7f198759e700 1 -- 172.21.15.104:0/3066119346 shutdown_connections 2023-07-14T11:56:15.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.965+0000 7f198759e700 1 --2- 172.21.15.104:0/3066119346 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19680662f0 0x7f19680687a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.965+0000 7f198759e700 1 --2- 172.21.15.104:0/3066119346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19881596f0 0x7f19880c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.965+0000 7f198759e700 1 --2- 172.21.15.104:0/3066119346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19880c9820 0x7f19880af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.965+0000 7f198759e700 1 --2- 172.21.15.104:0/3066119346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f198806dc20 0x7f19880adb60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:15.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.965+0000 7f198759e700 1 -- 172.21.15.104:0/3066119346 >> 172.21.15.104:0/3066119346 conn(0x7f198806e960 msgr2=0x7f19881588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:15.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.965+0000 7f198759e700 1 -- 172.21.15.104:0/3066119346 shutdown_connections 2023-07-14T11:56:15.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:15.965+0000 7f198759e700 1 -- 172.21.15.104:0/3066119346 wait complete. 2023-07-14T11:56:15.956 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:56:15.975 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2023-07-14T11:56:13.947706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230276,"kb_used_data":6468,"kb_used_omap":0,"kb_used_meta":223744,"kb_avail":754744444,"statfs":{"total":773094113280,"available":772858310656,"internally_reserved":0,"allocated":6623232,"data_stored":3700596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":229113856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.288477"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683147+0000","last_change":"2023-07-14T11:56:10.683147+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683147+0000","last_undegraded":"2023-07-14T11:56:10.683147+0000","last_fullsized":"2023-07-14T11:56:10.683147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679541+0000","last_change":"2023-07-14T11:56:10.679541+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679541+0000","last_undegraded":"2023-07-14T11:56:10.679541+0000","last_fullsized":"2023-07-14T11:56:10.679541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.688004+0000","last_change":"2023-07-14T11:56:10.688004+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.688004+0000","last_undegraded":"2023-07-14T11:56:10.688004+0000","last_fullsized":"2023-07-14T11:56:10.688004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683798+0000","last_change":"2023-07-14T11:56:10.683798+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683798+0000","last_undegraded":"2023-07-14T11:56:10.683798+0000","last_fullsized":"2023-07-14T11:56:10.683798+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677302+0000","last_change":"2023-07-14T11:56:10.677302+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677302+0000","last_undegraded":"2023-07-14T11:56:10.677302+0000","last_fullsized":"2023-07-14T11:56:10.677302+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.685591+0000","last_change":"2023-07-14T11:56:10.685591+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.685591+0000","last_undegraded":"2023-07-14T11:56:10.685591+0000","last_fullsized":"2023-07-14T11:56:10.685591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677725+0000","last_change":"2023-07-14T11:56:10.677725+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677725+0000","last_undegraded":"2023-07-14T11:56:10.677725+0000","last_fullsized":"2023-07-14T11:56:10.677725+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683020+0000","last_change":"2023-07-14T11:56:10.683020+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683020+0000","last_undegraded":"2023-07-14T11:56:10.683020+0000","last_fullsized":"2023-07-14T11:56:10.683020+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.689105+0000","last_change":"2023-07-14T11:56:10.689105+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.689105+0000","last_undegraded":"2023-07-14T11:56:10.689105+0000","last_fullsized":"2023-07-14T11:56:10.689105+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.689300+0000","last_change":"2023-07-14T11:56:10.689300+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.689300+0000","last_undegraded":"2023-07-14T11:56:10.689300+0000","last_fullsized":"2023-07-14T11:56:10.689300+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687590+0000","last_change":"2023-07-14T11:56:10.687590+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687590+0000","last_undegraded":"2023-07-14T11:56:10.687590+0000","last_fullsized":"2023-07-14T11:56:10.687590+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.684482+0000","last_change":"2023-07-14T11:56:10.684482+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.684482+0000","last_undegraded":"2023-07-14T11:56:10.684482+0000","last_fullsized":"2023-07-14T11:56:10.684482+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.685262+0000","last_change":"2023-07-14T11:56:10.685262+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.685262+0000","last_undegraded":"2023-07-14T11:56:10.685262+0000","last_fullsized":"2023-07-14T11:56:10.685262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687474+0000","last_change":"2023-07-14T11:56:10.687474+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687474+0000","last_undegraded":"2023-07-14T11:56:10.687474+0000","last_fullsized":"2023-07-14T11:56:10.687474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.691056+0000","last_change":"2023-07-14T11:56:10.691056+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.691056+0000","last_undegraded":"2023-07-14T11:56:10.691056+0000","last_fullsized":"2023-07-14T11:56:10.691056+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.678388+0000","last_change":"2023-07-14T11:56:10.678388+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.678388+0000","last_undegraded":"2023-07-14T11:56:10.678388+0000","last_fullsized":"2023-07-14T11:56:10.678388+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682628+0000","last_change":"2023-07-14T11:56:10.682628+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682628+0000","last_undegraded":"2023-07-14T11:56:10.682628+0000","last_fullsized":"2023-07-14T11:56:10.682628+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.685212+0000","last_change":"2023-07-14T11:56:10.685212+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.685212+0000","last_undegraded":"2023-07-14T11:56:10.685212+0000","last_fullsized":"2023-07-14T11:56:10.685212+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679247+0000","last_change":"2023-07-14T11:56:10.679247+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679247+0000","last_undegraded":"2023-07-14T11:56:10.679247+0000","last_fullsized":"2023-07-14T11:56:10.679247+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682339+0000","last_change":"2023-07-14T11:56:10.682339+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682339+0000","last_undegraded":"2023-07-14T11:56:10.682339+0000","last_fullsized":"2023-07-14T11:56:10.682339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.684054+0000","last_change":"2023-07-14T11:56:10.684054+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.684054+0000","last_undegraded":"2023-07-14T11:56:10.684054+0000","last_fullsized":"2023-07-14T11:56:10.684054+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.680085+0000","last_change":"2023-07-14T11:56:10.680085+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.680085+0000","last_undegraded":"2023-07-14T11:56:10.680085+0000","last_fullsized":"2023-07-14T11:56:10.680085+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683502+0000","last_change":"2023-07-14T11:56:10.683502+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683502+0000","last_undegraded":"2023-07-14T11:56:10.683502+0000","last_fullsized":"2023-07-14T11:56:10.683502+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686241+0000","last_change":"2023-07-14T11:56:10.686241+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686241+0000","last_undegraded":"2023-07-14T11:56:10.686241+0000","last_fullsized":"2023-07-14T11:56:10.686241+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686556+0000","last_change":"2023-07-14T11:56:10.686556+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686556+0000","last_undegraded":"2023-07-14T11:56:10.686556+0000","last_fullsized":"2023-07-14T11:56:10.686556+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686616+0000","last_change":"2023-07-14T11:56:10.686616+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686616+0000","last_undegraded":"2023-07-14T11:56:10.686616+0000","last_fullsized":"2023-07-14T11:56:10.686616+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687141+0000","last_change":"2023-07-14T11:56:10.687141+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687141+0000","last_undegraded":"2023-07-14T11:56:10.687141+0000","last_fullsized":"2023-07-14T11:56:10.687141+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.681576+0000","last_change":"2023-07-14T11:56:10.681576+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.681576+0000","last_undegraded":"2023-07-14T11:56:10.681576+0000","last_fullsized":"2023-07-14T11:56:10.681576+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.690699+0000","last_change":"2023-07-14T11:56:10.690699+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.690699+0000","last_undegraded":"2023-07-14T11:56:10.690699+0000","last_fullsized":"2023-07-14T11:56:10.690699+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683599+0000","last_change":"2023-07-14T11:56:10.683599+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683599+0000","last_undegraded":"2023-07-14T11:56:10.683599+0000","last_fullsized":"2023-07-14T11:56:10.683599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683345+0000","last_change":"2023-07-14T11:56:10.683345+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683345+0000","last_undegraded":"2023-07-14T11:56:10.683345+0000","last_fullsized":"2023-07-14T11:56:10.683345+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687533+0000","last_change":"2023-07-14T11:56:10.687533+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687533+0000","last_undegraded":"2023-07-14T11:56:10.687533+0000","last_fullsized":"2023-07-14T11:56:10.687533+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676335+0000","last_change":"2023-07-14T11:56:10.676335+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676335+0000","last_undegraded":"2023-07-14T11:56:10.676335+0000","last_fullsized":"2023-07-14T11:56:10.676335+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.702674+0000","last_change":"2023-07-14T11:56:10.696681+0000","last_active":"2023-07-14T11:56:10.702674+0000","last_peered":"2023-07-14T11:56:10.702674+0000","last_clean":"2023-07-14T11:56:10.702674+0000","last_became_active":"2023-07-14T11:56:10.696413+0000","last_became_peered":"2023-07-14T11:56:10.696413+0000","last_unstale":"2023-07-14T11:56:10.702674+0000","last_undegraded":"2023-07-14T11:56:10.702674+0000","last_fullsized":"2023-07-14T11:56:10.702674+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:34.861496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682407+0000","last_change":"2023-07-14T11:56:10.682407+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682407+0000","last_undegraded":"2023-07-14T11:56:10.682407+0000","last_fullsized":"2023-07-14T11:56:10.682407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696074+0000","last_change":"2023-07-14T11:56:10.694408+0000","last_active":"2023-07-14T11:56:10.696074+0000","last_peered":"2023-07-14T11:56:10.696074+0000","last_clean":"2023-07-14T11:56:10.696074+0000","last_became_active":"2023-07-14T11:56:10.694097+0000","last_became_peered":"2023-07-14T11:56:10.694097+0000","last_unstale":"2023-07-14T11:56:10.696074+0000","last_undegraded":"2023-07-14T11:56:10.696074+0000","last_fullsized":"2023-07-14T11:56:10.696074+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:53.935404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686577+0000","last_change":"2023-07-14T11:56:10.686577+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686577+0000","last_undegraded":"2023-07-14T11:56:10.686577+0000","last_fullsized":"2023-07-14T11:56:10.686577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.705379+0000","last_change":"2023-07-14T11:56:10.703709+0000","last_active":"2023-07-14T11:56:10.705379+0000","last_peered":"2023-07-14T11:56:10.705379+0000","last_clean":"2023-07-14T11:56:10.705379+0000","last_became_active":"2023-07-14T11:56:10.703444+0000","last_became_peered":"2023-07-14T11:56:10.703444+0000","last_unstale":"2023-07-14T11:56:10.705379+0000","last_undegraded":"2023-07-14T11:56:10.705379+0000","last_fullsized":"2023-07-14T11:56:10.705379+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:01:25.510366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.691008+0000","last_change":"2023-07-14T11:56:10.691008+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.691008+0000","last_undegraded":"2023-07-14T11:56:10.691008+0000","last_fullsized":"2023-07-14T11:56:10.691008+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703069+0000","last_change":"2023-07-14T11:56:10.701563+0000","last_active":"2023-07-14T11:56:10.703069+0000","last_peered":"2023-07-14T11:56:10.703069+0000","last_clean":"2023-07-14T11:56:10.703069+0000","last_became_active":"2023-07-14T11:56:10.701262+0000","last_became_peered":"2023-07-14T11:56:10.701262+0000","last_unstale":"2023-07-14T11:56:10.703069+0000","last_undegraded":"2023-07-14T11:56:10.703069+0000","last_fullsized":"2023-07-14T11:56:10.703069+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:14.962001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677447+0000","last_change":"2023-07-14T11:56:10.677447+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677447+0000","last_undegraded":"2023-07-14T11:56:10.677447+0000","last_fullsized":"2023-07-14T11:56:10.677447+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.697759+0000","last_change":"2023-07-14T11:56:10.694726+0000","last_active":"2023-07-14T11:56:10.697759+0000","last_peered":"2023-07-14T11:56:10.697759+0000","last_clean":"2023-07-14T11:56:10.697759+0000","last_became_active":"2023-07-14T11:56:10.694444+0000","last_became_peered":"2023-07-14T11:56:10.694444+0000","last_unstale":"2023-07-14T11:56:10.697759+0000","last_undegraded":"2023-07-14T11:56:10.697759+0000","last_fullsized":"2023-07-14T11:56:10.697759+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:58:15.607086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677746+0000","last_change":"2023-07-14T11:56:10.677746+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677746+0000","last_undegraded":"2023-07-14T11:56:10.677746+0000","last_fullsized":"2023-07-14T11:56:10.677746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.692679+0000","last_change":"2023-07-14T11:56:10.689607+0000","last_active":"2023-07-14T11:56:10.692679+0000","last_peered":"2023-07-14T11:56:10.692679+0000","last_clean":"2023-07-14T11:56:10.692679+0000","last_became_active":"2023-07-14T11:56:10.689345+0000","last_became_peered":"2023-07-14T11:56:10.689345+0000","last_unstale":"2023-07-14T11:56:10.692679+0000","last_undegraded":"2023-07-14T11:56:10.692679+0000","last_fullsized":"2023-07-14T11:56:10.692679+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:00:10.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679929+0000","last_change":"2023-07-14T11:56:10.679929+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679929+0000","last_undegraded":"2023-07-14T11:56:10.679929+0000","last_fullsized":"2023-07-14T11:56:10.679929+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.693487+0000","last_change":"2023-07-14T11:56:10.691388+0000","last_active":"2023-07-14T11:56:10.693487+0000","last_peered":"2023-07-14T11:56:10.693487+0000","last_clean":"2023-07-14T11:56:10.693487+0000","last_became_active":"2023-07-14T11:56:10.690746+0000","last_became_peered":"2023-07-14T11:56:10.690746+0000","last_unstale":"2023-07-14T11:56:10.693487+0000","last_undegraded":"2023-07-14T11:56:10.693487+0000","last_fullsized":"2023-07-14T11:56:10.693487+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:01.327056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682930+0000","last_change":"2023-07-14T11:56:10.682930+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682930+0000","last_undegraded":"2023-07-14T11:56:10.682930+0000","last_fullsized":"2023-07-14T11:56:10.682930+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.700186+0000","last_change":"2023-07-14T11:56:10.694837+0000","last_active":"2023-07-14T11:56:10.700186+0000","last_peered":"2023-07-14T11:56:10.700186+0000","last_clean":"2023-07-14T11:56:10.700186+0000","last_became_active":"2023-07-14T11:56:10.694071+0000","last_became_peered":"2023-07-14T11:56:10.694071+0000","last_unstale":"2023-07-14T11:56:10.700186+0000","last_undegraded":"2023-07-14T11:56:10.700186+0000","last_fullsized":"2023-07-14T11:56:10.700186+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:12.411429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.681039+0000","last_change":"2023-07-14T11:56:10.681039+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.681039+0000","last_undegraded":"2023-07-14T11:56:10.681039+0000","last_fullsized":"2023-07-14T11:56:10.681039+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.708387+0000","last_change":"2023-07-14T11:56:10.705201+0000","last_active":"2023-07-14T11:56:10.708387+0000","last_peered":"2023-07-14T11:56:10.708387+0000","last_clean":"2023-07-14T11:56:10.708387+0000","last_became_active":"2023-07-14T11:56:10.704922+0000","last_became_peered":"2023-07-14T11:56:10.704922+0000","last_unstale":"2023-07-14T11:56:10.708387+0000","last_undegraded":"2023-07-14T11:56:10.708387+0000","last_fullsized":"2023-07-14T11:56:10.708387+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:32:12.442228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.680746+0000","last_change":"2023-07-14T11:56:10.680746+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.680746+0000","last_undegraded":"2023-07-14T11:56:10.680746+0000","last_fullsized":"2023-07-14T11:56:10.680746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696725+0000","last_change":"2023-07-14T11:56:10.692503+0000","last_active":"2023-07-14T11:56:10.696725+0000","last_peered":"2023-07-14T11:56:10.696725+0000","last_clean":"2023-07-14T11:56:10.696725+0000","last_became_active":"2023-07-14T11:56:10.692092+0000","last_became_peered":"2023-07-14T11:56:10.692092+0000","last_unstale":"2023-07-14T11:56:10.696725+0000","last_undegraded":"2023-07-14T11:56:10.696725+0000","last_fullsized":"2023-07-14T11:56:10.696725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:05.395217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677241+0000","last_change":"2023-07-14T11:56:10.677241+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677241+0000","last_undegraded":"2023-07-14T11:56:10.677241+0000","last_fullsized":"2023-07-14T11:56:10.677241+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707828+0000","last_change":"2023-07-14T11:56:10.703832+0000","last_active":"2023-07-14T11:56:10.707828+0000","last_peered":"2023-07-14T11:56:10.707828+0000","last_clean":"2023-07-14T11:56:10.707828+0000","last_became_active":"2023-07-14T11:56:10.703548+0000","last_became_peered":"2023-07-14T11:56:10.703548+0000","last_unstale":"2023-07-14T11:56:10.707828+0000","last_undegraded":"2023-07-14T11:56:10.707828+0000","last_fullsized":"2023-07-14T11:56:10.707828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:03:55.287678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682757+0000","last_change":"2023-07-14T11:56:10.682757+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682757+0000","last_undegraded":"2023-07-14T11:56:10.682757+0000","last_fullsized":"2023-07-14T11:56:10.682757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703549+0000","last_change":"2023-07-14T11:56:10.697093+0000","last_active":"2023-07-14T11:56:10.703549+0000","last_peered":"2023-07-14T11:56:10.703549+0000","last_clean":"2023-07-14T11:56:10.703549+0000","last_became_active":"2023-07-14T11:56:10.692072+0000","last_became_peered":"2023-07-14T11:56:10.692072+0000","last_unstale":"2023-07-14T11:56:10.703549+0000","last_undegraded":"2023-07-14T11:56:10.703549+0000","last_fullsized":"2023-07-14T11:56:10.703549+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:46:05.586216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682498+0000","last_change":"2023-07-14T11:56:10.682498+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682498+0000","last_undegraded":"2023-07-14T11:56:10.682498+0000","last_fullsized":"2023-07-14T11:56:10.682498+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707805+0000","last_change":"2023-07-14T11:56:10.701400+0000","last_active":"2023-07-14T11:56:10.707805+0000","last_peered":"2023-07-14T11:56:10.707805+0000","last_clean":"2023-07-14T11:56:10.707805+0000","last_became_active":"2023-07-14T11:56:10.701104+0000","last_became_peered":"2023-07-14T11:56:10.701104+0000","last_unstale":"2023-07-14T11:56:10.707805+0000","last_undegraded":"2023-07-14T11:56:10.707805+0000","last_fullsized":"2023-07-14T11:56:10.707805+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:04.078467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683497+0000","last_change":"2023-07-14T11:56:10.683497+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683497+0000","last_undegraded":"2023-07-14T11:56:10.683497+0000","last_fullsized":"2023-07-14T11:56:10.683497+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.705209+0000","last_change":"2023-07-14T11:56:10.702066+0000","last_active":"2023-07-14T11:56:10.705209+0000","last_peered":"2023-07-14T11:56:10.705209+0000","last_clean":"2023-07-14T11:56:10.705209+0000","last_became_active":"2023-07-14T11:56:10.701720+0000","last_became_peered":"2023-07-14T11:56:10.701720+0000","last_unstale":"2023-07-14T11:56:10.705209+0000","last_undegraded":"2023-07-14T11:56:10.705209+0000","last_fullsized":"2023-07-14T11:56:10.705209+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:34:01.415077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679469+0000","last_change":"2023-07-14T11:56:10.679469+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679469+0000","last_undegraded":"2023-07-14T11:56:10.679469+0000","last_fullsized":"2023-07-14T11:56:10.679469+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.704869+0000","last_change":"2023-07-14T11:56:10.699176+0000","last_active":"2023-07-14T11:56:10.704869+0000","last_peered":"2023-07-14T11:56:10.704869+0000","last_clean":"2023-07-14T11:56:10.704869+0000","last_became_active":"2023-07-14T11:56:10.698881+0000","last_became_peered":"2023-07-14T11:56:10.698881+0000","last_unstale":"2023-07-14T11:56:10.704869+0000","last_undegraded":"2023-07-14T11:56:10.704869+0000","last_fullsized":"2023-07-14T11:56:10.704869+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:50.194031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677560+0000","last_change":"2023-07-14T11:56:10.677560+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677560+0000","last_undegraded":"2023-07-14T11:56:10.677560+0000","last_fullsized":"2023-07-14T11:56:10.677560+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.698670+0000","last_change":"2023-07-14T11:56:10.695120+0000","last_active":"2023-07-14T11:56:10.698670+0000","last_peered":"2023-07-14T11:56:10.698670+0000","last_clean":"2023-07-14T11:56:10.698670+0000","last_became_active":"2023-07-14T11:56:10.693407+0000","last_became_peered":"2023-07-14T11:56:10.693407+0000","last_unstale":"2023-07-14T11:56:10.698670+0000","last_undegraded":"2023-07-14T11:56:10.698670+0000","last_fullsized":"2023-07-14T11:56:10.698670+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:47:56.814799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.688739+0000","last_change":"2023-07-14T11:56:10.688739+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.688739+0000","last_undegraded":"2023-07-14T11:56:10.688739+0000","last_fullsized":"2023-07-14T11:56:10.688739+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703966+0000","last_change":"2023-07-14T11:56:10.700844+0000","last_active":"2023-07-14T11:56:10.703966+0000","last_peered":"2023-07-14T11:56:10.703966+0000","last_clean":"2023-07-14T11:56:10.703966+0000","last_became_active":"2023-07-14T11:56:10.700534+0000","last_became_peered":"2023-07-14T11:56:10.700534+0000","last_unstale":"2023-07-14T11:56:10.703966+0000","last_undegraded":"2023-07-14T11:56:10.703966+0000","last_fullsized":"2023-07-14T11:56:10.703966+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:45:37.602121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.678287+0000","last_change":"2023-07-14T11:56:10.678287+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.678287+0000","last_undegraded":"2023-07-14T11:56:10.678287+0000","last_fullsized":"2023-07-14T11:56:10.678287+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703169+0000","last_change":"2023-07-14T11:56:10.695559+0000","last_active":"2023-07-14T11:56:10.703169+0000","last_peered":"2023-07-14T11:56:10.703169+0000","last_clean":"2023-07-14T11:56:10.703169+0000","last_became_active":"2023-07-14T11:56:10.695312+0000","last_became_peered":"2023-07-14T11:56:10.695312+0000","last_unstale":"2023-07-14T11:56:10.703169+0000","last_undegraded":"2023-07-14T11:56:10.703169+0000","last_fullsized":"2023-07-14T11:56:10.703169+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:01.509577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686132+0000","last_change":"2023-07-14T11:56:10.686132+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686132+0000","last_undegraded":"2023-07-14T11:56:10.686132+0000","last_fullsized":"2023-07-14T11:56:10.686132+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.704530+0000","last_change":"2023-07-14T11:56:10.703244+0000","last_active":"2023-07-14T11:56:10.704530+0000","last_peered":"2023-07-14T11:56:10.704530+0000","last_clean":"2023-07-14T11:56:10.704530+0000","last_became_active":"2023-07-14T11:56:10.702957+0000","last_became_peered":"2023-07-14T11:56:10.702957+0000","last_unstale":"2023-07-14T11:56:10.704530+0000","last_undegraded":"2023-07-14T11:56:10.704530+0000","last_fullsized":"2023-07-14T11:56:10.704530+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:47:56.814799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677574+0000","last_change":"2023-07-14T11:56:10.677574+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677574+0000","last_undegraded":"2023-07-14T11:56:10.677574+0000","last_fullsized":"2023-07-14T11:56:10.677574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.700755+0000","last_change":"2023-07-14T11:56:10.695223+0000","last_active":"2023-07-14T11:56:10.700755+0000","last_peered":"2023-07-14T11:56:10.700755+0000","last_clean":"2023-07-14T11:56:10.700755+0000","last_became_active":"2023-07-14T11:56:10.691638+0000","last_became_peered":"2023-07-14T11:56:10.691638+0000","last_unstale":"2023-07-14T11:56:10.700755+0000","last_undegraded":"2023-07-14T11:56:10.700755+0000","last_fullsized":"2023-07-14T11:56:10.700755+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:06:54.458842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676708+0000","last_change":"2023-07-14T11:56:10.676708+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676708+0000","last_undegraded":"2023-07-14T11:56:10.676708+0000","last_fullsized":"2023-07-14T11:56:10.676708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.704883+0000","last_change":"2023-07-14T11:56:10.699836+0000","last_active":"2023-07-14T11:56:10.704883+0000","last_peered":"2023-07-14T11:56:10.704883+0000","last_clean":"2023-07-14T11:56:10.704883+0000","last_became_active":"2023-07-14T11:56:10.699528+0000","last_became_peered":"2023-07-14T11:56:10.699528+0000","last_unstale":"2023-07-14T11:56:10.704883+0000","last_undegraded":"2023-07-14T11:56:10.704883+0000","last_fullsized":"2023-07-14T11:56:10.704883+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:28:16.553232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682860+0000","last_change":"2023-07-14T11:56:10.682860+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682860+0000","last_undegraded":"2023-07-14T11:56:10.682860+0000","last_fullsized":"2023-07-14T11:56:10.682860+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.695725+0000","last_change":"2023-07-14T11:56:10.693503+0000","last_active":"2023-07-14T11:56:10.695725+0000","last_peered":"2023-07-14T11:56:10.695725+0000","last_clean":"2023-07-14T11:56:10.695725+0000","last_became_active":"2023-07-14T11:56:10.693197+0000","last_became_peered":"2023-07-14T11:56:10.693197+0000","last_unstale":"2023-07-14T11:56:10.695725+0000","last_undegraded":"2023-07-14T11:56:10.695725+0000","last_fullsized":"2023-07-14T11:56:10.695725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:21:50.291873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677561+0000","last_change":"2023-07-14T11:56:10.677561+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677561+0000","last_undegraded":"2023-07-14T11:56:10.677561+0000","last_fullsized":"2023-07-14T11:56:10.677561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.693506+0000","last_change":"2023-07-14T11:56:10.691758+0000","last_active":"2023-07-14T11:56:10.693506+0000","last_peered":"2023-07-14T11:56:10.693506+0000","last_clean":"2023-07-14T11:56:10.693506+0000","last_became_active":"2023-07-14T11:56:10.691130+0000","last_became_peered":"2023-07-14T11:56:10.691130+0000","last_unstale":"2023-07-14T11:56:10.693506+0000","last_undegraded":"2023-07-14T11:56:10.693506+0000","last_fullsized":"2023-07-14T11:56:10.693506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:35:21.027007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.678966+0000","last_change":"2023-07-14T11:56:10.678966+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.678966+0000","last_undegraded":"2023-07-14T11:56:10.678966+0000","last_fullsized":"2023-07-14T11:56:10.678966+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.700706+0000","last_change":"2023-07-14T11:56:10.692036+0000","last_active":"2023-07-14T11:56:10.700706+0000","last_peered":"2023-07-14T11:56:10.700706+0000","last_clean":"2023-07-14T11:56:10.700706+0000","last_became_active":"2023-07-14T11:56:10.691608+0000","last_became_peered":"2023-07-14T11:56:10.691608+0000","last_unstale":"2023-07-14T11:56:10.700706+0000","last_undegraded":"2023-07-14T11:56:10.700706+0000","last_fullsized":"2023-07-14T11:56:10.700706+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:30:54.612583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676440+0000","last_change":"2023-07-14T11:56:10.676440+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676440+0000","last_undegraded":"2023-07-14T11:56:10.676440+0000","last_fullsized":"2023-07-14T11:56:10.676440+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.692802+0000","last_change":"2023-07-14T11:56:10.690467+0000","last_active":"2023-07-14T11:56:10.692802+0000","last_peered":"2023-07-14T11:56:10.692802+0000","last_clean":"2023-07-14T11:56:10.692802+0000","last_became_active":"2023-07-14T11:56:10.690201+0000","last_became_peered":"2023-07-14T11:56:10.690201+0000","last_unstale":"2023-07-14T11:56:10.692802+0000","last_undegraded":"2023-07-14T11:56:10.692802+0000","last_fullsized":"2023-07-14T11:56:10.692802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:06:38.660785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.681506+0000","last_change":"2023-07-14T11:53:09.145201+0000","last_active":"2023-07-14T11:56:10.681506+0000","last_peered":"2023-07-14T11:56:10.681506+0000","last_clean":"2023-07-14T11:56:10.681506+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:56:10.681506+0000","last_undegraded":"2023-07-14T11:56:10.681506+0000","last_fullsized":"2023-07-14T11:56:10.681506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:16:24.795475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683131+0000","last_change":"2023-07-14T11:56:10.683131+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683131+0000","last_undegraded":"2023-07-14T11:56:10.683131+0000","last_fullsized":"2023-07-14T11:56:10.683131+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707863+0000","last_change":"2023-07-14T11:56:10.704116+0000","last_active":"2023-07-14T11:56:10.707863+0000","last_peered":"2023-07-14T11:56:10.707863+0000","last_clean":"2023-07-14T11:56:10.707863+0000","last_became_active":"2023-07-14T11:56:10.703775+0000","last_became_peered":"2023-07-14T11:56:10.703775+0000","last_unstale":"2023-07-14T11:56:10.707863+0000","last_undegraded":"2023-07-14T11:56:10.707863+0000","last_fullsized":"2023-07-14T11:56:10.707863+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:23:09.743965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.687564+0000","last_change":"2023-07-14T11:53:09.146150+0000","last_active":"2023-07-14T11:56:10.687564+0000","last_peered":"2023-07-14T11:56:10.687564+0000","last_clean":"2023-07-14T11:56:10.687564+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:56:10.687564+0000","last_undegraded":"2023-07-14T11:56:10.687564+0000","last_fullsized":"2023-07-14T11:56:10.687564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:49:39.347836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676479+0000","last_change":"2023-07-14T11:56:10.676479+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676479+0000","last_undegraded":"2023-07-14T11:56:10.676479+0000","last_fullsized":"2023-07-14T11:56:10.676479+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707751+0000","last_change":"2023-07-14T11:56:10.700575+0000","last_active":"2023-07-14T11:56:10.707751+0000","last_peered":"2023-07-14T11:56:10.707751+0000","last_clean":"2023-07-14T11:56:10.707751+0000","last_became_active":"2023-07-14T11:56:10.700293+0000","last_became_peered":"2023-07-14T11:56:10.700293+0000","last_unstale":"2023-07-14T11:56:10.707751+0000","last_undegraded":"2023-07-14T11:56:10.707751+0000","last_fullsized":"2023-07-14T11:56:10.707751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:24:16.135877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.694687+0000","last_change":"2023-07-14T11:53:09.435876+0000","last_active":"2023-07-14T11:56:10.694687+0000","last_peered":"2023-07-14T11:56:10.694687+0000","last_clean":"2023-07-14T11:56:10.694687+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:56:10.694687+0000","last_undegraded":"2023-07-14T11:56:10.694687+0000","last_fullsized":"2023-07-14T11:56:10.694687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:29:06.737180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687524+0000","last_change":"2023-07-14T11:56:10.687524+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687524+0000","last_undegraded":"2023-07-14T11:56:10.687524+0000","last_fullsized":"2023-07-14T11:56:10.687524+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707881+0000","last_change":"2023-07-14T11:56:10.702843+0000","last_active":"2023-07-14T11:56:10.707881+0000","last_peered":"2023-07-14T11:56:10.707881+0000","last_clean":"2023-07-14T11:56:10.707881+0000","last_became_active":"2023-07-14T11:56:10.702583+0000","last_became_peered":"2023-07-14T11:56:10.702583+0000","last_unstale":"2023-07-14T11:56:10.707881+0000","last_undegraded":"2023-07-14T11:56:10.707881+0000","last_fullsized":"2023-07-14T11:56:10.707881+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:08:58.730889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.687487+0000","last_change":"2023-07-14T11:53:09.434255+0000","last_active":"2023-07-14T11:56:10.687487+0000","last_peered":"2023-07-14T11:56:10.687487+0000","last_clean":"2023-07-14T11:56:10.687487+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:56:10.687487+0000","last_undegraded":"2023-07-14T11:56:10.687487+0000","last_fullsized":"2023-07-14T11:56:10.687487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:11:16.228885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677097+0000","last_change":"2023-07-14T11:56:10.677097+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677097+0000","last_undegraded":"2023-07-14T11:56:10.677097+0000","last_fullsized":"2023-07-14T11:56:10.677097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.699530+0000","last_change":"2023-07-14T11:56:10.697670+0000","last_active":"2023-07-14T11:56:10.699530+0000","last_peered":"2023-07-14T11:56:10.699530+0000","last_clean":"2023-07-14T11:56:10.699530+0000","last_became_active":"2023-07-14T11:56:10.697406+0000","last_became_peered":"2023-07-14T11:56:10.697406+0000","last_unstale":"2023-07-14T11:56:10.699530+0000","last_undegraded":"2023-07-14T11:56:10.699530+0000","last_fullsized":"2023-07-14T11:56:10.699530+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:05.395217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":84,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.694638+0000","last_change":"2023-07-14T11:53:09.147040+0000","last_active":"2023-07-14T11:56:10.694638+0000","last_peered":"2023-07-14T11:56:10.694638+0000","last_clean":"2023-07-14T11:56:10.694638+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:56:10.694638+0000","last_undegraded":"2023-07-14T11:56:10.694638+0000","last_fullsized":"2023-07-14T11:56:10.694638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:37.742068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682908+0000","last_change":"2023-07-14T11:56:10.682908+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682908+0000","last_undegraded":"2023-07-14T11:56:10.682908+0000","last_fullsized":"2023-07-14T11:56:10.682908+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.705188+0000","last_change":"2023-07-14T11:56:10.701923+0000","last_active":"2023-07-14T11:56:10.705188+0000","last_peered":"2023-07-14T11:56:10.705188+0000","last_clean":"2023-07-14T11:56:10.705188+0000","last_became_active":"2023-07-14T11:56:10.701351+0000","last_became_peered":"2023-07-14T11:56:10.701351+0000","last_unstale":"2023-07-14T11:56:10.705188+0000","last_undegraded":"2023-07-14T11:56:10.705188+0000","last_fullsized":"2023-07-14T11:56:10.705188+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:01:13.092675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":86,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.673618+0000","last_change":"2023-07-14T11:53:09.619294+0000","last_active":"2023-07-14T11:56:10.673618+0000","last_peered":"2023-07-14T11:56:10.673618+0000","last_clean":"2023-07-14T11:56:10.673618+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:56:10.673618+0000","last_undegraded":"2023-07-14T11:56:10.673618+0000","last_fullsized":"2023-07-14T11:56:10.673618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:07:12.290309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687092+0000","last_change":"2023-07-14T11:56:10.687092+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687092+0000","last_undegraded":"2023-07-14T11:56:10.687092+0000","last_fullsized":"2023-07-14T11:56:10.687092+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696108+0000","last_change":"2023-07-14T11:56:10.693854+0000","last_active":"2023-07-14T11:56:10.696108+0000","last_peered":"2023-07-14T11:56:10.696108+0000","last_clean":"2023-07-14T11:56:10.696108+0000","last_became_active":"2023-07-14T11:56:10.693588+0000","last_became_peered":"2023-07-14T11:56:10.693588+0000","last_unstale":"2023-07-14T11:56:10.696108+0000","last_undegraded":"2023-07-14T11:56:10.696108+0000","last_fullsized":"2023-07-14T11:56:10.696108+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:37:02.428527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":72,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696339+0000","last_change":"2023-07-14T11:53:09.146156+0000","last_active":"2023-07-14T11:56:10.696339+0000","last_peered":"2023-07-14T11:56:10.696339+0000","last_clean":"2023-07-14T11:56:10.696339+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:56:10.696339+0000","last_undegraded":"2023-07-14T11:56:10.696339+0000","last_fullsized":"2023-07-14T11:56:10.696339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:05:34.882240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676332+0000","last_change":"2023-07-14T11:56:10.676332+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676332+0000","last_undegraded":"2023-07-14T11:56:10.676332+0000","last_fullsized":"2023-07-14T11:56:10.676332+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.697324+0000","last_change":"2023-07-14T11:56:10.694832+0000","last_active":"2023-07-14T11:56:10.697324+0000","last_peered":"2023-07-14T11:56:10.697324+0000","last_clean":"2023-07-14T11:56:10.697324+0000","last_became_active":"2023-07-14T11:56:10.694539+0000","last_became_peered":"2023-07-14T11:56:10.694539+0000","last_unstale":"2023-07-14T11:56:10.697324+0000","last_undegraded":"2023-07-14T11:56:10.697324+0000","last_fullsized":"2023-07-14T11:56:10.697324+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:17:28.732628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":84,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.694479+0000","last_change":"2023-07-14T11:53:09.146556+0000","last_active":"2023-07-14T11:56:10.694479+0000","last_peered":"2023-07-14T11:56:10.694479+0000","last_clean":"2023-07-14T11:56:10.694479+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:56:10.694479+0000","last_undegraded":"2023-07-14T11:56:10.694479+0000","last_fullsized":"2023-07-14T11:56:10.694479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:34.689837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696481+0000","last_change":"2023-07-14T11:53:03.159818+0000","last_active":"2023-07-14T11:56:10.696481+0000","last_peered":"2023-07-14T11:56:10.696481+0000","last_clean":"2023-07-14T11:56:10.696481+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:56:10.696481+0000","last_undegraded":"2023-07-14T11:56:10.696481+0000","last_fullsized":"2023-07-14T11:56:10.696481+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_clean_scrub_stamp":"2023-07-14T11:53:02.054348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:05:58.561507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1265664,"data_stored":901013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.093,"15min":1.111},"min":{"1min":0.754,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.474,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.103,"15min":1.127},"min":{"1min":0.65100000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2430000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.4450000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1499999999999999,"15min":1.171},"min":{"1min":0.68300000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.298,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.22,"15min":1.262},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.4159999999999999,"5min":2.234,"15min":2.234},"last":0.88300000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.234,"15min":1.288},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.536,"5min":2.04,"15min":2.04},"last":1.143},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.25,"15min":1.2829999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8089999999999999,"5min":2.492,"15min":2.492},"last":1.1080000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.282,"15min":1.3360000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.665,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.2430000000000001,"15min":1.274},"min":{"1min":0.81799999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5940000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.81699999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.321,"15min":1.385},"min":{"1min":0.84399999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5660000000000001,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":1.371},{"interface":"front","average":{"1min":1.286,"5min":1.3400000000000001,"15min":1.3799999999999999},"min":{"1min":0.92500000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.76,"5min":2.399,"15min":2.399},"last":1.2490000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.284,"15min":1.3200000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6459999999999999,"5min":2.387,"15min":2.387},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.456,"15min":1.538},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.464,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":1.012}]},{"osd":6,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.5309999999999999,"15min":1.609},"min":{"1min":1.0940000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5289999999999999,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.494},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.4550000000000001,"15min":1.5229999999999999},"min":{"1min":0.95199999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.53,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.5049999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0700000000000001,"15min":1.0620000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.397,"5min":1.954,"15min":1.954},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.1160000000000001,"15min":1.1000000000000001},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.3460000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1890000000000001,"15min":1.2090000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7110000000000001,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.214,"15min":1.23},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7769999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.4770000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.171,"15min":1.1659999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4690000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2709999999999999,"15min":1.2729999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6819999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.0449999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.3140000000000001,"15min":1.3440000000000001},"min":{"1min":0.94099999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.6579999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.1899999999999999,"15min":1.1799999999999999},"min":{"1min":0.91700000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.482,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.98499999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.236,"15min":1.2529999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.625,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.133},{"interface":"front","average":{"1min":1.194,"5min":1.3129999999999999,"15min":1.3400000000000001},"min":{"1min":0.78000000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.512,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.599}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.238,"15min":1.2270000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.419},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4039999999999999,"15min":1.425},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.9139999999999999,"5min":1.984,"15min":1.984},"last":1.218}]},{"osd":7,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.4330000000000001,"15min":1.4650000000000001},"min":{"1min":0.99199999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.6519999999999999,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.643},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.365,"15min":1.379},"min":{"1min":0.92900000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.728,"5min":1.879,"15min":1.879},"last":1.571}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.1359999999999999,"15min":1.1819999999999999},"min":{"1min":0.68100000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4790000000000001,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.129,"15min":1.175},"min":{"1min":0.70299999999999996,"5min":0.622,"15min":0.622},"max":{"1min":1.474,"5min":1.581,"15min":1.581},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.167,"15min":1.1990000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.524,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.224},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.175,"15min":1.1890000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.498,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.1739999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.198,"15min":1.226},"min":{"1min":0.86699999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4550000000000001,"5min":1.839,"15min":1.839},"last":1.002},{"interface":"front","average":{"1min":1.113,"5min":1.282,"15min":1.347},"min":{"1min":0.73799999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4990000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.083}]},{"osd":3,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.272,"15min":1.3220000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.5409999999999999,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.242,"5min":1.3440000000000001,"15min":1.3919999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.5349999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.399}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2090000000000001,"15min":1.2490000000000001},"min":{"1min":0.75900000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5900000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.3540000000000001,"15min":1.3740000000000001},"min":{"1min":1.024,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.5109999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.264}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3959999999999999,"15min":1.4179999999999999},"min":{"1min":1.1459999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.544,"5min":1.99,"15min":1.99},"last":1.341},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.375,"15min":1.4159999999999999},"min":{"1min":0.86399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6779999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.3069999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4039999999999999,"15min":1.421},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.6220000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.5569999999999999,"15min":1.6160000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.5880000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.4430000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.149,"15min":1.171},"min":{"1min":0.67200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.5069999999999999,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.133,"15min":1.1599999999999999},"min":{"1min":0.71099999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.76500000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.159,"15min":1.1819999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.508,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.871},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1879999999999999,"15min":1.2090000000000001},"min":{"1min":0.75900000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.516,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.3}]},{"osd":2,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1359999999999999,"15min":1.143},"min":{"1min":0.83899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.546,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.095},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3400000000000001,"15min":1.3620000000000001},"min":{"1min":0.96599999999999997,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.581,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":1.046}]},{"osd":3,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2789999999999999,"15min":1.321},"min":{"1min":0.82199999999999995,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.681,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2270000000000001,"15min":1.2450000000000001},"min":{"1min":0.88100000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.4770000000000001,"5min":1.823,"15min":1.823},"last":1.3799999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2989999999999999,"15min":1.3420000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6299999999999999,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.421},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.89500000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.47,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.2609999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.264,"15min":1.2669999999999999},"min":{"1min":0.97799999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.522,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.476,"15min":1.532},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6759999999999999,"5min":3.5289999999999999,"15min":3.5289999999999999},"last":1.5109999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.466,"15min":1.5049999999999999},"min":{"1min":1.0589999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.6899999999999999,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.47,"15min":1.5209999999999999},"min":{"1min":1.0640000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":1.605,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":1.55}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":671744,"data_stored":310645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.84099999999999997,"15min":0.871},"min":{"1min":0.47699999999999998,"5min":0.247,"15min":0.247},"max":{"1min":1.234,"5min":1.325,"15min":1.325},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.875,"15min":0.90800000000000003},"min":{"1min":0.60599999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2490000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.96699999999999997}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92300000000000004,"15min":0.96299999999999997},"min":{"1min":0.52600000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6950000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.0600000000000001,"15min":1.111},"min":{"1min":0.54300000000000004,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.319,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.88600000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.097,"15min":1.141},"min":{"1min":0.63500000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4259999999999999,"5min":1.627,"15min":1.627},"last":1.119},{"interface":"front","average":{"1min":1.079,"5min":1.0669999999999999,"15min":1.091},"min":{"1min":0.73399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.94099999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2410000000000001,"15min":1.2849999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.53,"5min":2.335,"15min":2.335},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.2470000000000001,"15min":1.2869999999999999},"min":{"1min":0.77700000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.1659999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.323,"15min":1.359},"min":{"1min":0.94999999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.3580000000000001,"15min":1.421},"min":{"1min":0.86899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.532,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.2}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.329,"15min":1.387},"min":{"1min":0.85899999999999999,"5min":0.625,"15min":0.625},"max":{"1min":1.7010000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.4550000000000001,"15min":1.5009999999999999},"min":{"1min":1.083,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.0790000000000002,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.3819999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.419,"15min":1.4810000000000001},"min":{"1min":0.90500000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.712,"5min":3.395,"15min":3.395},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.367,"15min":1.431},"min":{"1min":1.0680000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.619,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.1850000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.874,"15min":0.90600000000000003},"min":{"1min":0.53800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.22,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.876,"15min":0.90200000000000002},"min":{"1min":0.48799999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1799999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.95499999999999996}]},{"osd":1,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.88100000000000001,"15min":0.89900000000000002},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.093,"5min":1.365,"15min":1.365},"last":0.622},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0269999999999999,"15min":1.069},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.425,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.202}]},{"osd":3,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0649999999999999,"15min":1.0960000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5089999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.018,"5min":1.026,"15min":1.052},"min":{"1min":0.65300000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.5900000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.1419999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2709999999999999,"15min":1.2989999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.718,"5min":2.097,"15min":2.097},"last":1.268},{"interface":"front","average":{"1min":1.242,"5min":1.2769999999999999,"15min":1.3149999999999999},"min":{"1min":0.67800000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.861,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.7270000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.2889999999999999,"15min":1.319},"min":{"1min":0.95699999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.4710000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.242},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3180000000000001,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.645,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.3520000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.333,"15min":1.383},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7649999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.7849999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.3919999999999999,"15min":1.425},"min":{"1min":1.1040000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.726,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.321}]},{"osd":7,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.397,"15min":1.444},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.677,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.9079999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4219999999999999,"15min":1.466},"min":{"1min":1.01,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6899999999999999,"5min":2.536,"15min":2.536},"last":1.4219999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.80100000000000005,"15min":0.83299999999999996},"min":{"1min":0.48899999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.26,"5min":1.349,"15min":1.349},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83299999999999996,"15min":0.83399999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.113}]},{"osd":2,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84799999999999998,"15min":0.85799999999999998},"min":{"1min":0.499,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.93500000000000005,"15min":0.93999999999999995},"min":{"1min":0.55900000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.81799999999999995}]},{"osd":3,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.99199999999999999,"15min":1.002},"min":{"1min":0.78500000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.399,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.272},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0169999999999999,"15min":1.0229999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.8169999999999999,"5min":1.857,"15min":1.857},"last":0.874}]},{"osd":4,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.2729999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6279999999999999,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.022},{"interface":"front","average":{"1min":1.111,"5min":1.1830000000000001,"15min":1.2170000000000001},"min":{"1min":0.754,"5min":0.745,"15min":0.745},"max":{"1min":1.6279999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.206}]},{"osd":5,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.23,"15min":1.256},"min":{"1min":0.876,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7190000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3149999999999999,"15min":1.365},"min":{"1min":0.93100000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.778,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3560000000000001,"15min":1.387},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.9119999999999999,"5min":2.633,"15min":2.633},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.335,"15min":1.359},"min":{"1min":1.091,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.873,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.367}]},{"osd":7,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3580000000000001,"15min":1.3979999999999999},"min":{"1min":1.004,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.992,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3300000000000001,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.754,"15min":0.754},"max":{"1min":2.056,"5min":2.375,"15min":2.375},"last":1.3260000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1265664,"data_stored":901013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.879,"15min":0.89300000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3100000000000001,"5min":1.581,"15min":1.581},"last":1.0329999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0049999999999999,"15min":1},"min":{"1min":0.64800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.5880000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90800000000000003,"15min":0.95099999999999996},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.1639999999999999,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.93600000000000005,"15min":0.94599999999999995},"min":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.228,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.099,"15min":1.1319999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3799999999999999,"5min":1.704,"15min":1.704},"last":1.22}]},{"osd":4,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2430000000000001,"15min":1.278},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5149999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3420000000000001,"15min":1.3720000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.837,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.345}]},{"osd":5,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.228,"15min":1.284},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9870000000000001,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.276,"15min":1.3200000000000001},"min":{"1min":0.79800000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.7749999999999999,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.0940000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.0459999999999998,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.319,"15min":1.357},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.9299999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.3300000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.335,"15min":1.3819999999999999},"min":{"1min":0.83299999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.865,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.379,"15min":1.417},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.6870000000000001,"5min":2.109,"15min":2.109},"last":1.2929999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:56:15.983 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:56:16.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.201+0000 7fb00b59e700 1 Processor -- start 2023-07-14T11:56:16.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.201+0000 7fb00b59e700 1 -- start start 2023-07-14T11:56:16.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb00b59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb00c162b10 0x7fb00c164ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:16.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb00b59e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00c0b0ef0 con 0x7fb00c159f20 2023-07-14T11:56:16.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb00b59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00c0ce9c0 con 0x7fb00c0c9d90 2023-07-14T11:56:16.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb00b59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00c0cee50 con 0x7fb00c162b10 2023-07-14T11:56:16.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb003fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb00c159f20 0x7fb00c0caa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:47938/0 (socket says 172.21.15.104:47938) 2023-07-14T11:56:16.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb003fff700 1 -- 172.21.15.104:0/147089925 learned_addr learned my addr 172.21.15.104:0/147089925 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:16.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb00ad9d700 1 --2- 172.21.15.104:0/147089925 >> v2:172.21.15.121:3300/0 conn(0x7fb00c162b10 0x7fb00c164ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:16.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb009d9b700 1 -- 172.21.15.104:0/147089925 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 617570594 0 0) 0x7fb00c0b0ef0 con 0x7fb00c159f20 2023-07-14T11:56:16.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb009d9b700 1 -- 172.21.15.104:0/147089925 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faff80035d0 con 0x7fb00c159f20 2023-07-14T11:56:16.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb009d9b700 1 -- 172.21.15.104:0/147089925 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805768545 0 0) 0x7fb00c0ce9c0 con 0x7fb00c0c9d90 2023-07-14T11:56:16.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb009d9b700 1 -- 172.21.15.104:0/147089925 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb00c0b0ef0 con 0x7fb00c0c9d90 2023-07-14T11:56:16.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb00ad9d700 1 -- 172.21.15.104:0/147089925 >> v1:172.21.15.121:6789/0 conn(0x7fb00c0c9d90 legacy=0x7fb00c0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:16.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb00ad9d700 1 -- 172.21.15.104:0/147089925 >> v1:172.21.15.104:6789/0 conn(0x7fb00c159f20 legacy=0x7fb00c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:16.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb009d9b700 1 -- 172.21.15.104:0/147089925 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3753856441 0 0) 0x7faff80035d0 con 0x7fb00c159f20 2023-07-14T11:56:16.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb00ad9d700 1 -- 172.21.15.104:0/147089925 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c0cb4f0 con 0x7fb00c162b10 2023-07-14T11:56:16.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb00ad9d700 1 --2- 172.21.15.104:0/147089925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c162b10 0x7fb00c164ef0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb004009d90 tx=0x7fb00400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=567c55981fbee53f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:16.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb009d9b700 1 -- 172.21.15.104:0/147089925 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00400e070 con 0x7fb00c162b10 2023-07-14T11:56:16.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb009d9b700 1 -- 172.21.15.104:0/147089925 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb004003a40 con 0x7fb00c162b10 2023-07-14T11:56:16.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.205+0000 7fb009d9b700 1 -- 172.21.15.104:0/147089925 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff4001670 con 0x7fb00c162b10 2023-07-14T11:56:16.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.209+0000 7fb00b59e700 1 -- 172.21.15.104:0/147089925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c162b10 msgr2=0x7fb00c164ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:16.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.209+0000 7fb00b59e700 1 --2- 172.21.15.104:0/147089925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c162b10 0x7fb00c164ef0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb004009d90 tx=0x7fb00400b330 comp rx=0 tx=0).stop 2023-07-14T11:56:16.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.209+0000 7fb00b59e700 1 -- 172.21.15.104:0/147089925 shutdown_connections 2023-07-14T11:56:16.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.209+0000 7fb00b59e700 1 --2- 172.21.15.104:0/147089925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c162b10 0x7fb00c164ef0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.209+0000 7fb00b59e700 1 -- 172.21.15.104:0/147089925 >> 172.21.15.104:0/147089925 conn(0x7fb00c06e960 msgr2=0x7fb00c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:16.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.209+0000 7fb00b59e700 1 -- 172.21.15.104:0/147089925 shutdown_connections 2023-07-14T11:56:16.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.209+0000 7fb00b59e700 1 -- 172.21.15.104:0/147089925 wait complete. 2023-07-14T11:56:16.197 INFO:tasks.ceph.mds.c.smithi104.stderr:2023-07-14T11:56:16.209+0000 7f11ba383700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-14T11:56:16.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.209+0000 7fb00b59e700 1 Processor -- start 2023-07-14T11:56:16.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.209+0000 7fb00b59e700 1 -- start start 2023-07-14T11:56:16.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00b59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0c9d90 0x7fb00c0cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:16.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00b59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c159f20 0x7fb00c0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:16.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00b59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb00c162b10 0x7fb00c145ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:16.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00b59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0ccfc0 con 0x7fb00c162b10 2023-07-14T11:56:16.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00b59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0cc340 con 0x7fb00c159f20 2023-07-14T11:56:16.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00b59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0cc620 con 0x7fb00c0c9d90 2023-07-14T11:56:16.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00ad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb00c162b10 0x7fb00c145ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:16.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00ad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb00c162b10 0x7fb00c145ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50422/0 (socket says 172.21.15.104:50422) 2023-07-14T11:56:16.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00ad9d700 1 -- 172.21.15.104:0/1429667472 learned_addr learned my addr 172.21.15.104:0/1429667472 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:16.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb003fff700 1 --2- 172.21.15.104:0/1429667472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c159f20 0x7fb00c0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:16.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00a59c700 1 --2- 172.21.15.104:0/1429667472 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0c9d90 0x7fb00c0cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:16.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00ad9d700 1 -- 172.21.15.104:0/1429667472 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0c9d90 msgr2=0x7fb00c0cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:16.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00ad9d700 1 --2- 172.21.15.104:0/1429667472 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0c9d90 0x7fb00c0cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00ad9d700 1 -- 172.21.15.104:0/1429667472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c159f20 msgr2=0x7fb00c0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:16.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00ad9d700 1 --2- 172.21.15.104:0/1429667472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c159f20 0x7fb00c0cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00ad9d700 1 -- 172.21.15.104:0/1429667472 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb004009a70 con 0x7fb00c162b10 2023-07-14T11:56:16.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00ad9d700 1 --2- 172.21.15.104:0/1429667472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb00c162b10 0x7fb00c145ec0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb00c06a6f0 tx=0x7fb004004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:16.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb002ffd700 1 -- 172.21.15.104:0/1429667472 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004004440 con 0x7fb00c162b10 2023-07-14T11:56:16.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb002ffd700 1 -- 172.21.15.104:0/1429667472 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0040045a0 con 0x7fb00c162b10 2023-07-14T11:56:16.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00b59e700 1 -- 172.21.15.104:0/1429667472 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00c0cc900 con 0x7fb00c162b10 2023-07-14T11:56:16.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb002ffd700 1 -- 172.21.15.104:0/1429667472 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004004910 con 0x7fb00c162b10 2023-07-14T11:56:16.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.213+0000 7fb00b59e700 1 -- 172.21.15.104:0/1429667472 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00c1f54b0 con 0x7fb00c162b10 2023-07-14T11:56:16.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.217+0000 7fb00b59e700 1 -- 172.21.15.104:0/1429667472 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00c0cd100 con 0x7fb00c162b10 2023-07-14T11:56:16.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.217+0000 7fb002ffd700 1 -- 172.21.15.104:0/1429667472 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb00401d470 con 0x7fb00c162b10 2023-07-14T11:56:16.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.217+0000 7fb002ffd700 1 --2- 172.21.15.104:0/1429667472 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fafec0662f0 0x7fafec0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:16.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.217+0000 7fb002ffd700 1 -- 172.21.15.104:0/1429667472 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb004085a20 con 0x7fb00c162b10 2023-07-14T11:56:16.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.217+0000 7fb00a59c700 1 --2- 172.21.15.104:0/1429667472 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fafec0662f0 0x7fafec0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:16.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.221+0000 7fb00a59c700 1 --2- 172.21.15.104:0/1429667472 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fafec0662f0 0x7fafec0687a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faffc0017b0 tx=0x7faffc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:16.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.225+0000 7fb002ffd700 1 -- 172.21.15.104:0/1429667472 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb00c0cd100 con 0x7fb00c162b10 2023-07-14T11:56:16.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.389+0000 7fb00b59e700 1 -- 172.21.15.104:0/1429667472 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb00c0ce370 con 0x7fafec0662f0 2023-07-14T11:56:16.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.401+0000 7fb002ffd700 1 -- 172.21.15.104:0/1429667472 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303908 (secure 0 0 0) 0x7faffc001c80 con 0x7fafec0662f0 2023-07-14T11:56:16.394 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:16.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.413+0000 7fb00b59e700 1 -- 172.21.15.104:0/1429667472 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fafec0662f0 msgr2=0x7fafec0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:16.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.413+0000 7fb00b59e700 1 --2- 172.21.15.104:0/1429667472 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fafec0662f0 0x7fafec0687a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faffc0017b0 tx=0x7faffc009040 comp rx=0 tx=0).stop 2023-07-14T11:56:16.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.413+0000 7fb00b59e700 1 -- 172.21.15.104:0/1429667472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb00c162b10 msgr2=0x7fb00c145ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:16.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.413+0000 7fb00b59e700 1 --2- 172.21.15.104:0/1429667472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb00c162b10 0x7fb00c145ec0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb00c06a6f0 tx=0x7fb004004af0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.413+0000 7fb00b59e700 1 -- 172.21.15.104:0/1429667472 shutdown_connections 2023-07-14T11:56:16.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.413+0000 7fb00b59e700 1 --2- 172.21.15.104:0/1429667472 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fafec0662f0 0x7fafec0687a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.413+0000 7fb00b59e700 1 --2- 172.21.15.104:0/1429667472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb00c162b10 0x7fb00c145ec0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.413+0000 7fb00b59e700 1 --2- 172.21.15.104:0/1429667472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c159f20 0x7fb00c0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.413+0000 7fb00b59e700 1 --2- 172.21.15.104:0/1429667472 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0c9d90 0x7fb00c0cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.413+0000 7fb00b59e700 1 -- 172.21.15.104:0/1429667472 >> 172.21.15.104:0/1429667472 conn(0x7fb00c06e960 msgr2=0x7fb00c0af330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:16.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.413+0000 7fb00b59e700 1 -- 172.21.15.104:0/1429667472 shutdown_connections 2023-07-14T11:56:16.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.413+0000 7fb00b59e700 1 -- 172.21.15.104:0/1429667472 wait complete. 2023-07-14T11:56:16.403 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:56:16.422 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2023-07-14T11:56:15.962209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230276,"kb_used_data":6468,"kb_used_omap":0,"kb_used_meta":223744,"kb_avail":754744444,"statfs":{"total":773094113280,"available":772858310656,"internally_reserved":0,"allocated":6623232,"data_stored":3700596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":229113856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.015474"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683147+0000","last_change":"2023-07-14T11:56:10.683147+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683147+0000","last_undegraded":"2023-07-14T11:56:10.683147+0000","last_fullsized":"2023-07-14T11:56:10.683147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679541+0000","last_change":"2023-07-14T11:56:10.679541+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679541+0000","last_undegraded":"2023-07-14T11:56:10.679541+0000","last_fullsized":"2023-07-14T11:56:10.679541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.688004+0000","last_change":"2023-07-14T11:56:10.688004+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.688004+0000","last_undegraded":"2023-07-14T11:56:10.688004+0000","last_fullsized":"2023-07-14T11:56:10.688004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683798+0000","last_change":"2023-07-14T11:56:10.683798+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683798+0000","last_undegraded":"2023-07-14T11:56:10.683798+0000","last_fullsized":"2023-07-14T11:56:10.683798+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677302+0000","last_change":"2023-07-14T11:56:10.677302+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677302+0000","last_undegraded":"2023-07-14T11:56:10.677302+0000","last_fullsized":"2023-07-14T11:56:10.677302+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.685591+0000","last_change":"2023-07-14T11:56:10.685591+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.685591+0000","last_undegraded":"2023-07-14T11:56:10.685591+0000","last_fullsized":"2023-07-14T11:56:10.685591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677725+0000","last_change":"2023-07-14T11:56:10.677725+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677725+0000","last_undegraded":"2023-07-14T11:56:10.677725+0000","last_fullsized":"2023-07-14T11:56:10.677725+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683020+0000","last_change":"2023-07-14T11:56:10.683020+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683020+0000","last_undegraded":"2023-07-14T11:56:10.683020+0000","last_fullsized":"2023-07-14T11:56:10.683020+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.689105+0000","last_change":"2023-07-14T11:56:10.689105+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.689105+0000","last_undegraded":"2023-07-14T11:56:10.689105+0000","last_fullsized":"2023-07-14T11:56:10.689105+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.689300+0000","last_change":"2023-07-14T11:56:10.689300+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.689300+0000","last_undegraded":"2023-07-14T11:56:10.689300+0000","last_fullsized":"2023-07-14T11:56:10.689300+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687590+0000","last_change":"2023-07-14T11:56:10.687590+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687590+0000","last_undegraded":"2023-07-14T11:56:10.687590+0000","last_fullsized":"2023-07-14T11:56:10.687590+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.684482+0000","last_change":"2023-07-14T11:56:10.684482+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.684482+0000","last_undegraded":"2023-07-14T11:56:10.684482+0000","last_fullsized":"2023-07-14T11:56:10.684482+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.685262+0000","last_change":"2023-07-14T11:56:10.685262+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.685262+0000","last_undegraded":"2023-07-14T11:56:10.685262+0000","last_fullsized":"2023-07-14T11:56:10.685262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687474+0000","last_change":"2023-07-14T11:56:10.687474+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687474+0000","last_undegraded":"2023-07-14T11:56:10.687474+0000","last_fullsized":"2023-07-14T11:56:10.687474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.691056+0000","last_change":"2023-07-14T11:56:10.691056+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.691056+0000","last_undegraded":"2023-07-14T11:56:10.691056+0000","last_fullsized":"2023-07-14T11:56:10.691056+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.678388+0000","last_change":"2023-07-14T11:56:10.678388+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.678388+0000","last_undegraded":"2023-07-14T11:56:10.678388+0000","last_fullsized":"2023-07-14T11:56:10.678388+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682628+0000","last_change":"2023-07-14T11:56:10.682628+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682628+0000","last_undegraded":"2023-07-14T11:56:10.682628+0000","last_fullsized":"2023-07-14T11:56:10.682628+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.685212+0000","last_change":"2023-07-14T11:56:10.685212+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.685212+0000","last_undegraded":"2023-07-14T11:56:10.685212+0000","last_fullsized":"2023-07-14T11:56:10.685212+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679247+0000","last_change":"2023-07-14T11:56:10.679247+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679247+0000","last_undegraded":"2023-07-14T11:56:10.679247+0000","last_fullsized":"2023-07-14T11:56:10.679247+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682339+0000","last_change":"2023-07-14T11:56:10.682339+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682339+0000","last_undegraded":"2023-07-14T11:56:10.682339+0000","last_fullsized":"2023-07-14T11:56:10.682339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.684054+0000","last_change":"2023-07-14T11:56:10.684054+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.684054+0000","last_undegraded":"2023-07-14T11:56:10.684054+0000","last_fullsized":"2023-07-14T11:56:10.684054+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.680085+0000","last_change":"2023-07-14T11:56:10.680085+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.680085+0000","last_undegraded":"2023-07-14T11:56:10.680085+0000","last_fullsized":"2023-07-14T11:56:10.680085+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683502+0000","last_change":"2023-07-14T11:56:10.683502+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683502+0000","last_undegraded":"2023-07-14T11:56:10.683502+0000","last_fullsized":"2023-07-14T11:56:10.683502+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686241+0000","last_change":"2023-07-14T11:56:10.686241+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686241+0000","last_undegraded":"2023-07-14T11:56:10.686241+0000","last_fullsized":"2023-07-14T11:56:10.686241+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686556+0000","last_change":"2023-07-14T11:56:10.686556+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686556+0000","last_undegraded":"2023-07-14T11:56:10.686556+0000","last_fullsized":"2023-07-14T11:56:10.686556+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686616+0000","last_change":"2023-07-14T11:56:10.686616+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686616+0000","last_undegraded":"2023-07-14T11:56:10.686616+0000","last_fullsized":"2023-07-14T11:56:10.686616+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687141+0000","last_change":"2023-07-14T11:56:10.687141+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687141+0000","last_undegraded":"2023-07-14T11:56:10.687141+0000","last_fullsized":"2023-07-14T11:56:10.687141+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.681576+0000","last_change":"2023-07-14T11:56:10.681576+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.681576+0000","last_undegraded":"2023-07-14T11:56:10.681576+0000","last_fullsized":"2023-07-14T11:56:10.681576+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.690699+0000","last_change":"2023-07-14T11:56:10.690699+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.690699+0000","last_undegraded":"2023-07-14T11:56:10.690699+0000","last_fullsized":"2023-07-14T11:56:10.690699+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683599+0000","last_change":"2023-07-14T11:56:10.683599+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683599+0000","last_undegraded":"2023-07-14T11:56:10.683599+0000","last_fullsized":"2023-07-14T11:56:10.683599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683345+0000","last_change":"2023-07-14T11:56:10.683345+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683345+0000","last_undegraded":"2023-07-14T11:56:10.683345+0000","last_fullsized":"2023-07-14T11:56:10.683345+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687533+0000","last_change":"2023-07-14T11:56:10.687533+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687533+0000","last_undegraded":"2023-07-14T11:56:10.687533+0000","last_fullsized":"2023-07-14T11:56:10.687533+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676335+0000","last_change":"2023-07-14T11:56:10.676335+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676335+0000","last_undegraded":"2023-07-14T11:56:10.676335+0000","last_fullsized":"2023-07-14T11:56:10.676335+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.702674+0000","last_change":"2023-07-14T11:56:10.696681+0000","last_active":"2023-07-14T11:56:10.702674+0000","last_peered":"2023-07-14T11:56:10.702674+0000","last_clean":"2023-07-14T11:56:10.702674+0000","last_became_active":"2023-07-14T11:56:10.696413+0000","last_became_peered":"2023-07-14T11:56:10.696413+0000","last_unstale":"2023-07-14T11:56:10.702674+0000","last_undegraded":"2023-07-14T11:56:10.702674+0000","last_fullsized":"2023-07-14T11:56:10.702674+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:34.861496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682407+0000","last_change":"2023-07-14T11:56:10.682407+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682407+0000","last_undegraded":"2023-07-14T11:56:10.682407+0000","last_fullsized":"2023-07-14T11:56:10.682407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696074+0000","last_change":"2023-07-14T11:56:10.694408+0000","last_active":"2023-07-14T11:56:10.696074+0000","last_peered":"2023-07-14T11:56:10.696074+0000","last_clean":"2023-07-14T11:56:10.696074+0000","last_became_active":"2023-07-14T11:56:10.694097+0000","last_became_peered":"2023-07-14T11:56:10.694097+0000","last_unstale":"2023-07-14T11:56:10.696074+0000","last_undegraded":"2023-07-14T11:56:10.696074+0000","last_fullsized":"2023-07-14T11:56:10.696074+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:53.935404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686577+0000","last_change":"2023-07-14T11:56:10.686577+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686577+0000","last_undegraded":"2023-07-14T11:56:10.686577+0000","last_fullsized":"2023-07-14T11:56:10.686577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.705379+0000","last_change":"2023-07-14T11:56:10.703709+0000","last_active":"2023-07-14T11:56:10.705379+0000","last_peered":"2023-07-14T11:56:10.705379+0000","last_clean":"2023-07-14T11:56:10.705379+0000","last_became_active":"2023-07-14T11:56:10.703444+0000","last_became_peered":"2023-07-14T11:56:10.703444+0000","last_unstale":"2023-07-14T11:56:10.705379+0000","last_undegraded":"2023-07-14T11:56:10.705379+0000","last_fullsized":"2023-07-14T11:56:10.705379+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:01:25.510366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.691008+0000","last_change":"2023-07-14T11:56:10.691008+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.691008+0000","last_undegraded":"2023-07-14T11:56:10.691008+0000","last_fullsized":"2023-07-14T11:56:10.691008+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703069+0000","last_change":"2023-07-14T11:56:10.701563+0000","last_active":"2023-07-14T11:56:10.703069+0000","last_peered":"2023-07-14T11:56:10.703069+0000","last_clean":"2023-07-14T11:56:10.703069+0000","last_became_active":"2023-07-14T11:56:10.701262+0000","last_became_peered":"2023-07-14T11:56:10.701262+0000","last_unstale":"2023-07-14T11:56:10.703069+0000","last_undegraded":"2023-07-14T11:56:10.703069+0000","last_fullsized":"2023-07-14T11:56:10.703069+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:14.962001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677447+0000","last_change":"2023-07-14T11:56:10.677447+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677447+0000","last_undegraded":"2023-07-14T11:56:10.677447+0000","last_fullsized":"2023-07-14T11:56:10.677447+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.697759+0000","last_change":"2023-07-14T11:56:10.694726+0000","last_active":"2023-07-14T11:56:10.697759+0000","last_peered":"2023-07-14T11:56:10.697759+0000","last_clean":"2023-07-14T11:56:10.697759+0000","last_became_active":"2023-07-14T11:56:10.694444+0000","last_became_peered":"2023-07-14T11:56:10.694444+0000","last_unstale":"2023-07-14T11:56:10.697759+0000","last_undegraded":"2023-07-14T11:56:10.697759+0000","last_fullsized":"2023-07-14T11:56:10.697759+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:58:15.607086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677746+0000","last_change":"2023-07-14T11:56:10.677746+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677746+0000","last_undegraded":"2023-07-14T11:56:10.677746+0000","last_fullsized":"2023-07-14T11:56:10.677746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.692679+0000","last_change":"2023-07-14T11:56:10.689607+0000","last_active":"2023-07-14T11:56:10.692679+0000","last_peered":"2023-07-14T11:56:10.692679+0000","last_clean":"2023-07-14T11:56:10.692679+0000","last_became_active":"2023-07-14T11:56:10.689345+0000","last_became_peered":"2023-07-14T11:56:10.689345+0000","last_unstale":"2023-07-14T11:56:10.692679+0000","last_undegraded":"2023-07-14T11:56:10.692679+0000","last_fullsized":"2023-07-14T11:56:10.692679+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:00:10.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679929+0000","last_change":"2023-07-14T11:56:10.679929+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679929+0000","last_undegraded":"2023-07-14T11:56:10.679929+0000","last_fullsized":"2023-07-14T11:56:10.679929+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.693487+0000","last_change":"2023-07-14T11:56:10.691388+0000","last_active":"2023-07-14T11:56:10.693487+0000","last_peered":"2023-07-14T11:56:10.693487+0000","last_clean":"2023-07-14T11:56:10.693487+0000","last_became_active":"2023-07-14T11:56:10.690746+0000","last_became_peered":"2023-07-14T11:56:10.690746+0000","last_unstale":"2023-07-14T11:56:10.693487+0000","last_undegraded":"2023-07-14T11:56:10.693487+0000","last_fullsized":"2023-07-14T11:56:10.693487+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:01.327056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682930+0000","last_change":"2023-07-14T11:56:10.682930+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682930+0000","last_undegraded":"2023-07-14T11:56:10.682930+0000","last_fullsized":"2023-07-14T11:56:10.682930+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.700186+0000","last_change":"2023-07-14T11:56:10.694837+0000","last_active":"2023-07-14T11:56:10.700186+0000","last_peered":"2023-07-14T11:56:10.700186+0000","last_clean":"2023-07-14T11:56:10.700186+0000","last_became_active":"2023-07-14T11:56:10.694071+0000","last_became_peered":"2023-07-14T11:56:10.694071+0000","last_unstale":"2023-07-14T11:56:10.700186+0000","last_undegraded":"2023-07-14T11:56:10.700186+0000","last_fullsized":"2023-07-14T11:56:10.700186+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:12.411429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.681039+0000","last_change":"2023-07-14T11:56:10.681039+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.681039+0000","last_undegraded":"2023-07-14T11:56:10.681039+0000","last_fullsized":"2023-07-14T11:56:10.681039+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.708387+0000","last_change":"2023-07-14T11:56:10.705201+0000","last_active":"2023-07-14T11:56:10.708387+0000","last_peered":"2023-07-14T11:56:10.708387+0000","last_clean":"2023-07-14T11:56:10.708387+0000","last_became_active":"2023-07-14T11:56:10.704922+0000","last_became_peered":"2023-07-14T11:56:10.704922+0000","last_unstale":"2023-07-14T11:56:10.708387+0000","last_undegraded":"2023-07-14T11:56:10.708387+0000","last_fullsized":"2023-07-14T11:56:10.708387+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:32:12.442228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.680746+0000","last_change":"2023-07-14T11:56:10.680746+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.680746+0000","last_undegraded":"2023-07-14T11:56:10.680746+0000","last_fullsized":"2023-07-14T11:56:10.680746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696725+0000","last_change":"2023-07-14T11:56:10.692503+0000","last_active":"2023-07-14T11:56:10.696725+0000","last_peered":"2023-07-14T11:56:10.696725+0000","last_clean":"2023-07-14T11:56:10.696725+0000","last_became_active":"2023-07-14T11:56:10.692092+0000","last_became_peered":"2023-07-14T11:56:10.692092+0000","last_unstale":"2023-07-14T11:56:10.696725+0000","last_undegraded":"2023-07-14T11:56:10.696725+0000","last_fullsized":"2023-07-14T11:56:10.696725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:05.395217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677241+0000","last_change":"2023-07-14T11:56:10.677241+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677241+0000","last_undegraded":"2023-07-14T11:56:10.677241+0000","last_fullsized":"2023-07-14T11:56:10.677241+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707828+0000","last_change":"2023-07-14T11:56:10.703832+0000","last_active":"2023-07-14T11:56:10.707828+0000","last_peered":"2023-07-14T11:56:10.707828+0000","last_clean":"2023-07-14T11:56:10.707828+0000","last_became_active":"2023-07-14T11:56:10.703548+0000","last_became_peered":"2023-07-14T11:56:10.703548+0000","last_unstale":"2023-07-14T11:56:10.707828+0000","last_undegraded":"2023-07-14T11:56:10.707828+0000","last_fullsized":"2023-07-14T11:56:10.707828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:03:55.287678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682757+0000","last_change":"2023-07-14T11:56:10.682757+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682757+0000","last_undegraded":"2023-07-14T11:56:10.682757+0000","last_fullsized":"2023-07-14T11:56:10.682757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703549+0000","last_change":"2023-07-14T11:56:10.697093+0000","last_active":"2023-07-14T11:56:10.703549+0000","last_peered":"2023-07-14T11:56:10.703549+0000","last_clean":"2023-07-14T11:56:10.703549+0000","last_became_active":"2023-07-14T11:56:10.692072+0000","last_became_peered":"2023-07-14T11:56:10.692072+0000","last_unstale":"2023-07-14T11:56:10.703549+0000","last_undegraded":"2023-07-14T11:56:10.703549+0000","last_fullsized":"2023-07-14T11:56:10.703549+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:46:05.586216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682498+0000","last_change":"2023-07-14T11:56:10.682498+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682498+0000","last_undegraded":"2023-07-14T11:56:10.682498+0000","last_fullsized":"2023-07-14T11:56:10.682498+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707805+0000","last_change":"2023-07-14T11:56:10.701400+0000","last_active":"2023-07-14T11:56:10.707805+0000","last_peered":"2023-07-14T11:56:10.707805+0000","last_clean":"2023-07-14T11:56:10.707805+0000","last_became_active":"2023-07-14T11:56:10.701104+0000","last_became_peered":"2023-07-14T11:56:10.701104+0000","last_unstale":"2023-07-14T11:56:10.707805+0000","last_undegraded":"2023-07-14T11:56:10.707805+0000","last_fullsized":"2023-07-14T11:56:10.707805+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:04.078467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683497+0000","last_change":"2023-07-14T11:56:10.683497+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683497+0000","last_undegraded":"2023-07-14T11:56:10.683497+0000","last_fullsized":"2023-07-14T11:56:10.683497+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.705209+0000","last_change":"2023-07-14T11:56:10.702066+0000","last_active":"2023-07-14T11:56:10.705209+0000","last_peered":"2023-07-14T11:56:10.705209+0000","last_clean":"2023-07-14T11:56:10.705209+0000","last_became_active":"2023-07-14T11:56:10.701720+0000","last_became_peered":"2023-07-14T11:56:10.701720+0000","last_unstale":"2023-07-14T11:56:10.705209+0000","last_undegraded":"2023-07-14T11:56:10.705209+0000","last_fullsized":"2023-07-14T11:56:10.705209+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:34:01.415077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679469+0000","last_change":"2023-07-14T11:56:10.679469+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679469+0000","last_undegraded":"2023-07-14T11:56:10.679469+0000","last_fullsized":"2023-07-14T11:56:10.679469+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.704869+0000","last_change":"2023-07-14T11:56:10.699176+0000","last_active":"2023-07-14T11:56:10.704869+0000","last_peered":"2023-07-14T11:56:10.704869+0000","last_clean":"2023-07-14T11:56:10.704869+0000","last_became_active":"2023-07-14T11:56:10.698881+0000","last_became_peered":"2023-07-14T11:56:10.698881+0000","last_unstale":"2023-07-14T11:56:10.704869+0000","last_undegraded":"2023-07-14T11:56:10.704869+0000","last_fullsized":"2023-07-14T11:56:10.704869+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:50.194031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677560+0000","last_change":"2023-07-14T11:56:10.677560+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677560+0000","last_undegraded":"2023-07-14T11:56:10.677560+0000","last_fullsized":"2023-07-14T11:56:10.677560+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.698670+0000","last_change":"2023-07-14T11:56:10.695120+0000","last_active":"2023-07-14T11:56:10.698670+0000","last_peered":"2023-07-14T11:56:10.698670+0000","last_clean":"2023-07-14T11:56:10.698670+0000","last_became_active":"2023-07-14T11:56:10.693407+0000","last_became_peered":"2023-07-14T11:56:10.693407+0000","last_unstale":"2023-07-14T11:56:10.698670+0000","last_undegraded":"2023-07-14T11:56:10.698670+0000","last_fullsized":"2023-07-14T11:56:10.698670+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:47:56.814799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.688739+0000","last_change":"2023-07-14T11:56:10.688739+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.688739+0000","last_undegraded":"2023-07-14T11:56:10.688739+0000","last_fullsized":"2023-07-14T11:56:10.688739+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703966+0000","last_change":"2023-07-14T11:56:10.700844+0000","last_active":"2023-07-14T11:56:10.703966+0000","last_peered":"2023-07-14T11:56:10.703966+0000","last_clean":"2023-07-14T11:56:10.703966+0000","last_became_active":"2023-07-14T11:56:10.700534+0000","last_became_peered":"2023-07-14T11:56:10.700534+0000","last_unstale":"2023-07-14T11:56:10.703966+0000","last_undegraded":"2023-07-14T11:56:10.703966+0000","last_fullsized":"2023-07-14T11:56:10.703966+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:45:37.602121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.678287+0000","last_change":"2023-07-14T11:56:10.678287+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.678287+0000","last_undegraded":"2023-07-14T11:56:10.678287+0000","last_fullsized":"2023-07-14T11:56:10.678287+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703169+0000","last_change":"2023-07-14T11:56:10.695559+0000","last_active":"2023-07-14T11:56:10.703169+0000","last_peered":"2023-07-14T11:56:10.703169+0000","last_clean":"2023-07-14T11:56:10.703169+0000","last_became_active":"2023-07-14T11:56:10.695312+0000","last_became_peered":"2023-07-14T11:56:10.695312+0000","last_unstale":"2023-07-14T11:56:10.703169+0000","last_undegraded":"2023-07-14T11:56:10.703169+0000","last_fullsized":"2023-07-14T11:56:10.703169+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:01.509577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686132+0000","last_change":"2023-07-14T11:56:10.686132+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686132+0000","last_undegraded":"2023-07-14T11:56:10.686132+0000","last_fullsized":"2023-07-14T11:56:10.686132+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.704530+0000","last_change":"2023-07-14T11:56:10.703244+0000","last_active":"2023-07-14T11:56:10.704530+0000","last_peered":"2023-07-14T11:56:10.704530+0000","last_clean":"2023-07-14T11:56:10.704530+0000","last_became_active":"2023-07-14T11:56:10.702957+0000","last_became_peered":"2023-07-14T11:56:10.702957+0000","last_unstale":"2023-07-14T11:56:10.704530+0000","last_undegraded":"2023-07-14T11:56:10.704530+0000","last_fullsized":"2023-07-14T11:56:10.704530+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:47:56.814799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677574+0000","last_change":"2023-07-14T11:56:10.677574+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677574+0000","last_undegraded":"2023-07-14T11:56:10.677574+0000","last_fullsized":"2023-07-14T11:56:10.677574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.700755+0000","last_change":"2023-07-14T11:56:10.695223+0000","last_active":"2023-07-14T11:56:10.700755+0000","last_peered":"2023-07-14T11:56:10.700755+0000","last_clean":"2023-07-14T11:56:10.700755+0000","last_became_active":"2023-07-14T11:56:10.691638+0000","last_became_peered":"2023-07-14T11:56:10.691638+0000","last_unstale":"2023-07-14T11:56:10.700755+0000","last_undegraded":"2023-07-14T11:56:10.700755+0000","last_fullsized":"2023-07-14T11:56:10.700755+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:06:54.458842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676708+0000","last_change":"2023-07-14T11:56:10.676708+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676708+0000","last_undegraded":"2023-07-14T11:56:10.676708+0000","last_fullsized":"2023-07-14T11:56:10.676708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.704883+0000","last_change":"2023-07-14T11:56:10.699836+0000","last_active":"2023-07-14T11:56:10.704883+0000","last_peered":"2023-07-14T11:56:10.704883+0000","last_clean":"2023-07-14T11:56:10.704883+0000","last_became_active":"2023-07-14T11:56:10.699528+0000","last_became_peered":"2023-07-14T11:56:10.699528+0000","last_unstale":"2023-07-14T11:56:10.704883+0000","last_undegraded":"2023-07-14T11:56:10.704883+0000","last_fullsized":"2023-07-14T11:56:10.704883+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:28:16.553232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682860+0000","last_change":"2023-07-14T11:56:10.682860+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682860+0000","last_undegraded":"2023-07-14T11:56:10.682860+0000","last_fullsized":"2023-07-14T11:56:10.682860+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.695725+0000","last_change":"2023-07-14T11:56:10.693503+0000","last_active":"2023-07-14T11:56:10.695725+0000","last_peered":"2023-07-14T11:56:10.695725+0000","last_clean":"2023-07-14T11:56:10.695725+0000","last_became_active":"2023-07-14T11:56:10.693197+0000","last_became_peered":"2023-07-14T11:56:10.693197+0000","last_unstale":"2023-07-14T11:56:10.695725+0000","last_undegraded":"2023-07-14T11:56:10.695725+0000","last_fullsized":"2023-07-14T11:56:10.695725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:21:50.291873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677561+0000","last_change":"2023-07-14T11:56:10.677561+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677561+0000","last_undegraded":"2023-07-14T11:56:10.677561+0000","last_fullsized":"2023-07-14T11:56:10.677561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.693506+0000","last_change":"2023-07-14T11:56:10.691758+0000","last_active":"2023-07-14T11:56:10.693506+0000","last_peered":"2023-07-14T11:56:10.693506+0000","last_clean":"2023-07-14T11:56:10.693506+0000","last_became_active":"2023-07-14T11:56:10.691130+0000","last_became_peered":"2023-07-14T11:56:10.691130+0000","last_unstale":"2023-07-14T11:56:10.693506+0000","last_undegraded":"2023-07-14T11:56:10.693506+0000","last_fullsized":"2023-07-14T11:56:10.693506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:35:21.027007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.678966+0000","last_change":"2023-07-14T11:56:10.678966+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.678966+0000","last_undegraded":"2023-07-14T11:56:10.678966+0000","last_fullsized":"2023-07-14T11:56:10.678966+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.700706+0000","last_change":"2023-07-14T11:56:10.692036+0000","last_active":"2023-07-14T11:56:10.700706+0000","last_peered":"2023-07-14T11:56:10.700706+0000","last_clean":"2023-07-14T11:56:10.700706+0000","last_became_active":"2023-07-14T11:56:10.691608+0000","last_became_peered":"2023-07-14T11:56:10.691608+0000","last_unstale":"2023-07-14T11:56:10.700706+0000","last_undegraded":"2023-07-14T11:56:10.700706+0000","last_fullsized":"2023-07-14T11:56:10.700706+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:30:54.612583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676440+0000","last_change":"2023-07-14T11:56:10.676440+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676440+0000","last_undegraded":"2023-07-14T11:56:10.676440+0000","last_fullsized":"2023-07-14T11:56:10.676440+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.692802+0000","last_change":"2023-07-14T11:56:10.690467+0000","last_active":"2023-07-14T11:56:10.692802+0000","last_peered":"2023-07-14T11:56:10.692802+0000","last_clean":"2023-07-14T11:56:10.692802+0000","last_became_active":"2023-07-14T11:56:10.690201+0000","last_became_peered":"2023-07-14T11:56:10.690201+0000","last_unstale":"2023-07-14T11:56:10.692802+0000","last_undegraded":"2023-07-14T11:56:10.692802+0000","last_fullsized":"2023-07-14T11:56:10.692802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:06:38.660785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.681506+0000","last_change":"2023-07-14T11:53:09.145201+0000","last_active":"2023-07-14T11:56:10.681506+0000","last_peered":"2023-07-14T11:56:10.681506+0000","last_clean":"2023-07-14T11:56:10.681506+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:56:10.681506+0000","last_undegraded":"2023-07-14T11:56:10.681506+0000","last_fullsized":"2023-07-14T11:56:10.681506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:16:24.795475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683131+0000","last_change":"2023-07-14T11:56:10.683131+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683131+0000","last_undegraded":"2023-07-14T11:56:10.683131+0000","last_fullsized":"2023-07-14T11:56:10.683131+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707863+0000","last_change":"2023-07-14T11:56:10.704116+0000","last_active":"2023-07-14T11:56:10.707863+0000","last_peered":"2023-07-14T11:56:10.707863+0000","last_clean":"2023-07-14T11:56:10.707863+0000","last_became_active":"2023-07-14T11:56:10.703775+0000","last_became_peered":"2023-07-14T11:56:10.703775+0000","last_unstale":"2023-07-14T11:56:10.707863+0000","last_undegraded":"2023-07-14T11:56:10.707863+0000","last_fullsized":"2023-07-14T11:56:10.707863+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:23:09.743965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.687564+0000","last_change":"2023-07-14T11:53:09.146150+0000","last_active":"2023-07-14T11:56:10.687564+0000","last_peered":"2023-07-14T11:56:10.687564+0000","last_clean":"2023-07-14T11:56:10.687564+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:56:10.687564+0000","last_undegraded":"2023-07-14T11:56:10.687564+0000","last_fullsized":"2023-07-14T11:56:10.687564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:49:39.347836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676479+0000","last_change":"2023-07-14T11:56:10.676479+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676479+0000","last_undegraded":"2023-07-14T11:56:10.676479+0000","last_fullsized":"2023-07-14T11:56:10.676479+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707751+0000","last_change":"2023-07-14T11:56:10.700575+0000","last_active":"2023-07-14T11:56:10.707751+0000","last_peered":"2023-07-14T11:56:10.707751+0000","last_clean":"2023-07-14T11:56:10.707751+0000","last_became_active":"2023-07-14T11:56:10.700293+0000","last_became_peered":"2023-07-14T11:56:10.700293+0000","last_unstale":"2023-07-14T11:56:10.707751+0000","last_undegraded":"2023-07-14T11:56:10.707751+0000","last_fullsized":"2023-07-14T11:56:10.707751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:24:16.135877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.694687+0000","last_change":"2023-07-14T11:53:09.435876+0000","last_active":"2023-07-14T11:56:10.694687+0000","last_peered":"2023-07-14T11:56:10.694687+0000","last_clean":"2023-07-14T11:56:10.694687+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:56:10.694687+0000","last_undegraded":"2023-07-14T11:56:10.694687+0000","last_fullsized":"2023-07-14T11:56:10.694687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:29:06.737180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687524+0000","last_change":"2023-07-14T11:56:10.687524+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687524+0000","last_undegraded":"2023-07-14T11:56:10.687524+0000","last_fullsized":"2023-07-14T11:56:10.687524+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707881+0000","last_change":"2023-07-14T11:56:10.702843+0000","last_active":"2023-07-14T11:56:10.707881+0000","last_peered":"2023-07-14T11:56:10.707881+0000","last_clean":"2023-07-14T11:56:10.707881+0000","last_became_active":"2023-07-14T11:56:10.702583+0000","last_became_peered":"2023-07-14T11:56:10.702583+0000","last_unstale":"2023-07-14T11:56:10.707881+0000","last_undegraded":"2023-07-14T11:56:10.707881+0000","last_fullsized":"2023-07-14T11:56:10.707881+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:08:58.730889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.687487+0000","last_change":"2023-07-14T11:53:09.434255+0000","last_active":"2023-07-14T11:56:10.687487+0000","last_peered":"2023-07-14T11:56:10.687487+0000","last_clean":"2023-07-14T11:56:10.687487+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:56:10.687487+0000","last_undegraded":"2023-07-14T11:56:10.687487+0000","last_fullsized":"2023-07-14T11:56:10.687487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:11:16.228885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677097+0000","last_change":"2023-07-14T11:56:10.677097+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677097+0000","last_undegraded":"2023-07-14T11:56:10.677097+0000","last_fullsized":"2023-07-14T11:56:10.677097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.699530+0000","last_change":"2023-07-14T11:56:10.697670+0000","last_active":"2023-07-14T11:56:10.699530+0000","last_peered":"2023-07-14T11:56:10.699530+0000","last_clean":"2023-07-14T11:56:10.699530+0000","last_became_active":"2023-07-14T11:56:10.697406+0000","last_became_peered":"2023-07-14T11:56:10.697406+0000","last_unstale":"2023-07-14T11:56:10.699530+0000","last_undegraded":"2023-07-14T11:56:10.699530+0000","last_fullsized":"2023-07-14T11:56:10.699530+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:05.395217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":84,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.694638+0000","last_change":"2023-07-14T11:53:09.147040+0000","last_active":"2023-07-14T11:56:10.694638+0000","last_peered":"2023-07-14T11:56:10.694638+0000","last_clean":"2023-07-14T11:56:10.694638+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:56:10.694638+0000","last_undegraded":"2023-07-14T11:56:10.694638+0000","last_fullsized":"2023-07-14T11:56:10.694638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:37.742068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682908+0000","last_change":"2023-07-14T11:56:10.682908+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682908+0000","last_undegraded":"2023-07-14T11:56:10.682908+0000","last_fullsized":"2023-07-14T11:56:10.682908+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.705188+0000","last_change":"2023-07-14T11:56:10.701923+0000","last_active":"2023-07-14T11:56:10.705188+0000","last_peered":"2023-07-14T11:56:10.705188+0000","last_clean":"2023-07-14T11:56:10.705188+0000","last_became_active":"2023-07-14T11:56:10.701351+0000","last_became_peered":"2023-07-14T11:56:10.701351+0000","last_unstale":"2023-07-14T11:56:10.705188+0000","last_undegraded":"2023-07-14T11:56:10.705188+0000","last_fullsized":"2023-07-14T11:56:10.705188+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:01:13.092675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":86,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.673618+0000","last_change":"2023-07-14T11:53:09.619294+0000","last_active":"2023-07-14T11:56:10.673618+0000","last_peered":"2023-07-14T11:56:10.673618+0000","last_clean":"2023-07-14T11:56:10.673618+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:56:10.673618+0000","last_undegraded":"2023-07-14T11:56:10.673618+0000","last_fullsized":"2023-07-14T11:56:10.673618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:07:12.290309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687092+0000","last_change":"2023-07-14T11:56:10.687092+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687092+0000","last_undegraded":"2023-07-14T11:56:10.687092+0000","last_fullsized":"2023-07-14T11:56:10.687092+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696108+0000","last_change":"2023-07-14T11:56:10.693854+0000","last_active":"2023-07-14T11:56:10.696108+0000","last_peered":"2023-07-14T11:56:10.696108+0000","last_clean":"2023-07-14T11:56:10.696108+0000","last_became_active":"2023-07-14T11:56:10.693588+0000","last_became_peered":"2023-07-14T11:56:10.693588+0000","last_unstale":"2023-07-14T11:56:10.696108+0000","last_undegraded":"2023-07-14T11:56:10.696108+0000","last_fullsized":"2023-07-14T11:56:10.696108+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:37:02.428527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":72,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696339+0000","last_change":"2023-07-14T11:53:09.146156+0000","last_active":"2023-07-14T11:56:10.696339+0000","last_peered":"2023-07-14T11:56:10.696339+0000","last_clean":"2023-07-14T11:56:10.696339+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:56:10.696339+0000","last_undegraded":"2023-07-14T11:56:10.696339+0000","last_fullsized":"2023-07-14T11:56:10.696339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:05:34.882240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676332+0000","last_change":"2023-07-14T11:56:10.676332+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676332+0000","last_undegraded":"2023-07-14T11:56:10.676332+0000","last_fullsized":"2023-07-14T11:56:10.676332+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.697324+0000","last_change":"2023-07-14T11:56:10.694832+0000","last_active":"2023-07-14T11:56:10.697324+0000","last_peered":"2023-07-14T11:56:10.697324+0000","last_clean":"2023-07-14T11:56:10.697324+0000","last_became_active":"2023-07-14T11:56:10.694539+0000","last_became_peered":"2023-07-14T11:56:10.694539+0000","last_unstale":"2023-07-14T11:56:10.697324+0000","last_undegraded":"2023-07-14T11:56:10.697324+0000","last_fullsized":"2023-07-14T11:56:10.697324+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:17:28.732628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":84,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.694479+0000","last_change":"2023-07-14T11:53:09.146556+0000","last_active":"2023-07-14T11:56:10.694479+0000","last_peered":"2023-07-14T11:56:10.694479+0000","last_clean":"2023-07-14T11:56:10.694479+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:56:10.694479+0000","last_undegraded":"2023-07-14T11:56:10.694479+0000","last_fullsized":"2023-07-14T11:56:10.694479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:34.689837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696481+0000","last_change":"2023-07-14T11:53:03.159818+0000","last_active":"2023-07-14T11:56:10.696481+0000","last_peered":"2023-07-14T11:56:10.696481+0000","last_clean":"2023-07-14T11:56:10.696481+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:56:10.696481+0000","last_undegraded":"2023-07-14T11:56:10.696481+0000","last_fullsized":"2023-07-14T11:56:10.696481+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_clean_scrub_stamp":"2023-07-14T11:53:02.054348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:05:58.561507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1265664,"data_stored":901013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.093,"15min":1.111},"min":{"1min":0.754,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.474,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.103,"15min":1.127},"min":{"1min":0.65100000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2430000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.4450000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1499999999999999,"15min":1.171},"min":{"1min":0.68300000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.298,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.22,"15min":1.262},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.4159999999999999,"5min":2.234,"15min":2.234},"last":0.88300000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.234,"15min":1.288},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.536,"5min":2.04,"15min":2.04},"last":1.143},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.25,"15min":1.2829999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8089999999999999,"5min":2.492,"15min":2.492},"last":1.1080000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.282,"15min":1.3360000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.665,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.2430000000000001,"15min":1.274},"min":{"1min":0.81799999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5940000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.81699999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.321,"15min":1.385},"min":{"1min":0.84399999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5660000000000001,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":1.371},{"interface":"front","average":{"1min":1.286,"5min":1.3400000000000001,"15min":1.3799999999999999},"min":{"1min":0.92500000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.76,"5min":2.399,"15min":2.399},"last":1.2490000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.284,"15min":1.3200000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6459999999999999,"5min":2.387,"15min":2.387},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.456,"15min":1.538},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.464,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":1.012}]},{"osd":6,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.5309999999999999,"15min":1.609},"min":{"1min":1.0940000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5289999999999999,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.494},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.4550000000000001,"15min":1.5229999999999999},"min":{"1min":0.95199999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.53,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.5049999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0700000000000001,"15min":1.0620000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.397,"5min":1.954,"15min":1.954},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.1160000000000001,"15min":1.1000000000000001},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.3460000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1890000000000001,"15min":1.2090000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7110000000000001,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.214,"15min":1.23},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7769999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.4770000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.171,"15min":1.1659999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4690000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2709999999999999,"15min":1.2729999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6819999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.0449999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.3140000000000001,"15min":1.3440000000000001},"min":{"1min":0.94099999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.6579999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.1899999999999999,"15min":1.1799999999999999},"min":{"1min":0.91700000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.482,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.98499999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.236,"15min":1.2529999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.625,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.133},{"interface":"front","average":{"1min":1.194,"5min":1.3129999999999999,"15min":1.3400000000000001},"min":{"1min":0.78000000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.512,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.599}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.238,"15min":1.2270000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.419},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4039999999999999,"15min":1.425},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.9139999999999999,"5min":1.984,"15min":1.984},"last":1.218}]},{"osd":7,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.4330000000000001,"15min":1.4650000000000001},"min":{"1min":0.99199999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.6519999999999999,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.643},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.365,"15min":1.379},"min":{"1min":0.92900000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.728,"5min":1.879,"15min":1.879},"last":1.571}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.1359999999999999,"15min":1.1819999999999999},"min":{"1min":0.68100000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4790000000000001,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.129,"15min":1.175},"min":{"1min":0.70299999999999996,"5min":0.622,"15min":0.622},"max":{"1min":1.474,"5min":1.581,"15min":1.581},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.167,"15min":1.1990000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.524,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.224},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.175,"15min":1.1890000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.498,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.1739999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.198,"15min":1.226},"min":{"1min":0.86699999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4550000000000001,"5min":1.839,"15min":1.839},"last":1.002},{"interface":"front","average":{"1min":1.113,"5min":1.282,"15min":1.347},"min":{"1min":0.73799999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4990000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.083}]},{"osd":3,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.272,"15min":1.3220000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.5409999999999999,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.242,"5min":1.3440000000000001,"15min":1.3919999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.5349999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.399}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2090000000000001,"15min":1.2490000000000001},"min":{"1min":0.75900000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5900000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.3540000000000001,"15min":1.3740000000000001},"min":{"1min":1.024,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.5109999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.264}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3959999999999999,"15min":1.4179999999999999},"min":{"1min":1.1459999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.544,"5min":1.99,"15min":1.99},"last":1.341},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.375,"15min":1.4159999999999999},"min":{"1min":0.86399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6779999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.3069999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4039999999999999,"15min":1.421},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.6220000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.5569999999999999,"15min":1.6160000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.5880000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.4430000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.149,"15min":1.171},"min":{"1min":0.67200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.5069999999999999,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.133,"15min":1.1599999999999999},"min":{"1min":0.71099999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.76500000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.159,"15min":1.1819999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.508,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.871},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1879999999999999,"15min":1.2090000000000001},"min":{"1min":0.75900000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.516,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.3}]},{"osd":2,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1359999999999999,"15min":1.143},"min":{"1min":0.83899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.546,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.095},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3400000000000001,"15min":1.3620000000000001},"min":{"1min":0.96599999999999997,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.581,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":1.046}]},{"osd":3,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2789999999999999,"15min":1.321},"min":{"1min":0.82199999999999995,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.681,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2270000000000001,"15min":1.2450000000000001},"min":{"1min":0.88100000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.4770000000000001,"5min":1.823,"15min":1.823},"last":1.3799999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2989999999999999,"15min":1.3420000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6299999999999999,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.421},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.89500000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.47,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.2609999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.264,"15min":1.2669999999999999},"min":{"1min":0.97799999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.522,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.476,"15min":1.532},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6759999999999999,"5min":3.5289999999999999,"15min":3.5289999999999999},"last":1.5109999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.466,"15min":1.5049999999999999},"min":{"1min":1.0589999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.6899999999999999,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.47,"15min":1.5209999999999999},"min":{"1min":1.0640000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":1.605,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":1.55}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":671744,"data_stored":310645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.84099999999999997,"15min":0.871},"min":{"1min":0.47699999999999998,"5min":0.247,"15min":0.247},"max":{"1min":1.234,"5min":1.325,"15min":1.325},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.875,"15min":0.90800000000000003},"min":{"1min":0.60599999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2490000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.96699999999999997}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92300000000000004,"15min":0.96299999999999997},"min":{"1min":0.52600000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6950000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.0600000000000001,"15min":1.111},"min":{"1min":0.54300000000000004,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.319,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.88600000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.097,"15min":1.141},"min":{"1min":0.63500000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4259999999999999,"5min":1.627,"15min":1.627},"last":1.119},{"interface":"front","average":{"1min":1.079,"5min":1.0669999999999999,"15min":1.091},"min":{"1min":0.73399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.94099999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2410000000000001,"15min":1.2849999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.53,"5min":2.335,"15min":2.335},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.2470000000000001,"15min":1.2869999999999999},"min":{"1min":0.77700000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.1659999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.323,"15min":1.359},"min":{"1min":0.94999999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.3580000000000001,"15min":1.421},"min":{"1min":0.86899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.532,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.2}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.329,"15min":1.387},"min":{"1min":0.85899999999999999,"5min":0.625,"15min":0.625},"max":{"1min":1.7010000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.4550000000000001,"15min":1.5009999999999999},"min":{"1min":1.083,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.0790000000000002,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.3819999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.419,"15min":1.4810000000000001},"min":{"1min":0.90500000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.712,"5min":3.395,"15min":3.395},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.367,"15min":1.431},"min":{"1min":1.0680000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.619,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.1850000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.874,"15min":0.90600000000000003},"min":{"1min":0.53800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.22,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.876,"15min":0.90200000000000002},"min":{"1min":0.48799999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1799999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.95499999999999996}]},{"osd":1,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.88100000000000001,"15min":0.89900000000000002},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.093,"5min":1.365,"15min":1.365},"last":0.622},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0269999999999999,"15min":1.069},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.425,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.202}]},{"osd":3,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0649999999999999,"15min":1.0960000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5089999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.018,"5min":1.026,"15min":1.052},"min":{"1min":0.65300000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.5900000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.1419999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2709999999999999,"15min":1.2989999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.718,"5min":2.097,"15min":2.097},"last":1.268},{"interface":"front","average":{"1min":1.242,"5min":1.2769999999999999,"15min":1.3149999999999999},"min":{"1min":0.67800000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.861,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.7270000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.2889999999999999,"15min":1.319},"min":{"1min":0.95699999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.4710000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.242},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3180000000000001,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.645,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.3520000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.333,"15min":1.383},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7649999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.7849999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.3919999999999999,"15min":1.425},"min":{"1min":1.1040000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.726,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.321}]},{"osd":7,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.397,"15min":1.444},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.677,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.9079999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4219999999999999,"15min":1.466},"min":{"1min":1.01,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6899999999999999,"5min":2.536,"15min":2.536},"last":1.4219999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.80100000000000005,"15min":0.83299999999999996},"min":{"1min":0.48899999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.26,"5min":1.349,"15min":1.349},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83299999999999996,"15min":0.83399999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.113}]},{"osd":2,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84799999999999998,"15min":0.85799999999999998},"min":{"1min":0.499,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.93500000000000005,"15min":0.93999999999999995},"min":{"1min":0.55900000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.81799999999999995}]},{"osd":3,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.99199999999999999,"15min":1.002},"min":{"1min":0.78500000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.399,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.272},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0169999999999999,"15min":1.0229999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.8169999999999999,"5min":1.857,"15min":1.857},"last":0.874}]},{"osd":4,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.2729999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6279999999999999,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.022},{"interface":"front","average":{"1min":1.111,"5min":1.1830000000000001,"15min":1.2170000000000001},"min":{"1min":0.754,"5min":0.745,"15min":0.745},"max":{"1min":1.6279999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.206}]},{"osd":5,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.23,"15min":1.256},"min":{"1min":0.876,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7190000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3149999999999999,"15min":1.365},"min":{"1min":0.93100000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.778,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3560000000000001,"15min":1.387},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.9119999999999999,"5min":2.633,"15min":2.633},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.335,"15min":1.359},"min":{"1min":1.091,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.873,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.367}]},{"osd":7,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3580000000000001,"15min":1.3979999999999999},"min":{"1min":1.004,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.992,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3300000000000001,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.754,"15min":0.754},"max":{"1min":2.056,"5min":2.375,"15min":2.375},"last":1.3260000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1265664,"data_stored":901013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.879,"15min":0.89300000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3100000000000001,"5min":1.581,"15min":1.581},"last":1.0329999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0049999999999999,"15min":1},"min":{"1min":0.64800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.5880000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90800000000000003,"15min":0.95099999999999996},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.1639999999999999,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.93600000000000005,"15min":0.94599999999999995},"min":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.228,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.099,"15min":1.1319999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3799999999999999,"5min":1.704,"15min":1.704},"last":1.22}]},{"osd":4,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2430000000000001,"15min":1.278},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5149999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3420000000000001,"15min":1.3720000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.837,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.345}]},{"osd":5,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.228,"15min":1.284},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9870000000000001,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.276,"15min":1.3200000000000001},"min":{"1min":0.79800000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.7749999999999999,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.0940000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.0459999999999998,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.319,"15min":1.357},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.9299999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.3300000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.335,"15min":1.3819999999999999},"min":{"1min":0.83299999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.865,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.379,"15min":1.417},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.6870000000000001,"5min":2.109,"15min":2.109},"last":1.2929999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:56:16.430 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:56:16.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.657+0000 7fc113d81700 1 Processor -- start 2023-07-14T11:56:16.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc113d81700 1 -- start start 2023-07-14T11:56:16.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc113d81700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc10c06d860 0x7fc10c0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:16.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc113d81700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc10c06be70 0x7fc10c06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:16.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc113d81700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc10c0c1e00 0x7fc10c0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:16.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc113d81700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc10c0cc1b0 con 0x7fc10c06d860 2023-07-14T11:56:16.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc113d81700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc10c06e170 con 0x7fc10c06be70 2023-07-14T11:56:16.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc113d81700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c0cd690 con 0x7fc10c0c1e00 2023-07-14T11:56:16.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc111b1d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc10c06d860 0x7fc10c0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:16.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc111b1d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc10c06d860 0x7fc10c0cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50426/0 (socket says 172.21.15.104:50426) 2023-07-14T11:56:16.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc111b1d700 1 -- 172.21.15.104:0/2888701195 learned_addr learned my addr 172.21.15.104:0/2888701195 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:16.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc11131c700 1 --2- 172.21.15.104:0/2888701195 >> v2:172.21.15.121:3300/0 conn(0x7fc10c06be70 0x7fc10c06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:16.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc11231e700 1 --2- 172.21.15.104:0/2888701195 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc10c0c1e00 0x7fc10c0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:16.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc111b1d700 1 -- 172.21.15.104:0/2888701195 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc10c0c1e00 msgr2=0x7fc10c0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:16.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc111b1d700 1 --2- 172.21.15.104:0/2888701195 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc10c0c1e00 0x7fc10c0cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc111b1d700 1 -- 172.21.15.104:0/2888701195 >> v2:172.21.15.121:3300/0 conn(0x7fc10c06be70 msgr2=0x7fc10c06dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:16.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc111b1d700 1 --2- 172.21.15.104:0/2888701195 >> v2:172.21.15.121:3300/0 conn(0x7fc10c06be70 0x7fc10c06dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc111b1d700 1 -- 172.21.15.104:0/2888701195 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c0cd970 con 0x7fc10c06d860 2023-07-14T11:56:16.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc11231e700 1 --2- 172.21.15.104:0/2888701195 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc10c0c1e00 0x7fc10c0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:16.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.661+0000 7fc11131c700 1 --2- 172.21.15.104:0/2888701195 >> v2:172.21.15.121:3300/0 conn(0x7fc10c06be70 0x7fc10c06dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:16.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.665+0000 7fc111b1d700 1 --2- 172.21.15.104:0/2888701195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc10c06d860 0x7fc10c0cb840 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0b0c60 tx=0x7fc10800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102f701f7416d9dc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:16.655 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.665+0000 7fc110b1b700 1 -- 172.21.15.104:0/2888701195 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800e070 con 0x7fc10c06d860 2023-07-14T11:56:16.655 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.665+0000 7fc110b1b700 1 -- 172.21.15.104:0/2888701195 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10800b9c0 con 0x7fc10c06d860 2023-07-14T11:56:16.655 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.665+0000 7fc110b1b700 1 -- 172.21.15.104:0/2888701195 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800bd70 con 0x7fc10c06d860 2023-07-14T11:56:16.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.665+0000 7fc113d81700 1 -- 172.21.15.104:0/2888701195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc10c06d860 msgr2=0x7fc10c0cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:16.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.665+0000 7fc113d81700 1 --2- 172.21.15.104:0/2888701195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc10c06d860 0x7fc10c0cb840 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0b0c60 tx=0x7fc10800b330 comp rx=0 tx=0).stop 2023-07-14T11:56:16.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.665+0000 7fc113d81700 1 -- 172.21.15.104:0/2888701195 shutdown_connections 2023-07-14T11:56:16.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.665+0000 7fc113d81700 1 --2- 172.21.15.104:0/2888701195 >> v2:172.21.15.121:3300/0 conn(0x7fc10c06be70 0x7fc10c06dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.665+0000 7fc113d81700 1 --2- 172.21.15.104:0/2888701195 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc10c0c1e00 0x7fc10c0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.665+0000 7fc113d81700 1 --2- 172.21.15.104:0/2888701195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc10c06d860 0x7fc10c0cb840 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.665+0000 7fc113d81700 1 -- 172.21.15.104:0/2888701195 >> 172.21.15.104:0/2888701195 conn(0x7fc10c0cad30 msgr2=0x7fc10c0cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:16.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.665+0000 7fc113d81700 1 -- 172.21.15.104:0/2888701195 shutdown_connections 2023-07-14T11:56:16.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.665+0000 7fc113d81700 1 -- 172.21.15.104:0/2888701195 wait complete. 2023-07-14T11:56:16.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.665+0000 7fc113d81700 1 Processor -- start 2023-07-14T11:56:16.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc113d81700 1 -- start start 2023-07-14T11:56:16.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc113d81700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc10c06be70 0x7fc10c0c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:16.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc113d81700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc10c06d860 0x7fc10c152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:16.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc113d81700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc10c0c1e00 0x7fc10c0aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:16.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc113d81700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c153d90 con 0x7fc10c06d860 2023-07-14T11:56:16.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc113d81700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c0c6df0 con 0x7fc10c0c1e00 2023-07-14T11:56:16.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc113d81700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c0c7280 con 0x7fc10c06be70 2023-07-14T11:56:16.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc11131c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc10c06d860 0x7fc10c152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:16.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc11131c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc10c06d860 0x7fc10c152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50440/0 (socket says 172.21.15.104:50440) 2023-07-14T11:56:16.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc11131c700 1 -- 172.21.15.104:0/227394137 learned_addr learned my addr 172.21.15.104:0/227394137 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:16.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc11131c700 1 -- 172.21.15.104:0/227394137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc10c06be70 msgr2=0x7fc10c0c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:16.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc11131c700 1 --2- 172.21.15.104:0/227394137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc10c06be70 0x7fc10c0c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc11131c700 1 -- 172.21.15.104:0/227394137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc10c0c1e00 msgr2=0x7fc10c0aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:16.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc11231e700 1 --2- 172.21.15.104:0/227394137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc10c0c1e00 0x7fc10c0aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:16.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc11131c700 1 --2- 172.21.15.104:0/227394137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc10c0c1e00 0x7fc10c0aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc11131c700 1 -- 172.21.15.104:0/227394137 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc108009b30 con 0x7fc10c06d860 2023-07-14T11:56:16.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc11231e700 1 --2- 172.21.15.104:0/227394137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc10c0c1e00 0x7fc10c0aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:16.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc11131c700 1 --2- 172.21.15.104:0/227394137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc10c06d860 0x7fc10c152f90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00d830 tx=0x7fc0fc00dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:16.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc1027fc700 1 -- 172.21.15.104:0/227394137 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc004030 con 0x7fc10c06d860 2023-07-14T11:56:16.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc1027fc700 1 -- 172.21.15.104:0/227394137 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc004190 con 0x7fc10c06d860 2023-07-14T11:56:16.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc1027fc700 1 -- 172.21.15.104:0/227394137 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc004500 con 0x7fc10c06d860 2023-07-14T11:56:16.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc113d81700 1 -- 172.21.15.104:0/227394137 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10c0c61e0 con 0x7fc10c06d860 2023-07-14T11:56:16.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.669+0000 7fc113d81700 1 -- 172.21.15.104:0/227394137 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10c0c67a0 con 0x7fc10c06d860 2023-07-14T11:56:16.662 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.673+0000 7fc113d81700 1 -- 172.21.15.104:0/227394137 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc10c04dcb0 con 0x7fc10c06d860 2023-07-14T11:56:16.662 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.673+0000 7fc1027fc700 1 -- 172.21.15.104:0/227394137 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc0fc0036c0 con 0x7fc10c06d860 2023-07-14T11:56:16.662 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.673+0000 7fc1027fc700 1 --2- 172.21.15.104:0/227394137 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc0f00662f0 0x7fc0f00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:16.663 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.673+0000 7fc1027fc700 1 -- 172.21.15.104:0/227394137 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc0fc0844b0 con 0x7fc10c06d860 2023-07-14T11:56:16.663 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.673+0000 7fc111b1d700 1 --2- 172.21.15.104:0/227394137 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc0f00662f0 0x7fc0f00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:16.663 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.677+0000 7fc111b1d700 1 --2- 172.21.15.104:0/227394137 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc0f00662f0 0x7fc0f00687a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc108009fd0 tx=0x7fc108010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:16.668 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.681+0000 7fc1027fc700 1 -- 172.21.15.104:0/227394137 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fc0fc051750 con 0x7fc10c06d860 2023-07-14T11:56:16.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.845+0000 7fc113d81700 1 -- 172.21.15.104:0/227394137 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc10c0cdf30 con 0x7fc0f00662f0 2023-07-14T11:56:16.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.861+0000 7fc1027fc700 1 -- 172.21.15.104:0/227394137 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303908 (secure 0 0 0) 0x7fc10800cd70 con 0x7fc0f00662f0 2023-07-14T11:56:16.853 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:16.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.869+0000 7fc113d81700 1 -- 172.21.15.104:0/227394137 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc0f00662f0 msgr2=0x7fc0f00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:16.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.869+0000 7fc113d81700 1 --2- 172.21.15.104:0/227394137 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc0f00662f0 0x7fc0f00687a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc108009fd0 tx=0x7fc108010000 comp rx=0 tx=0).stop 2023-07-14T11:56:16.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.869+0000 7fc113d81700 1 -- 172.21.15.104:0/227394137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc10c06d860 msgr2=0x7fc10c152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:16.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.869+0000 7fc113d81700 1 --2- 172.21.15.104:0/227394137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc10c06d860 0x7fc10c152f90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00d830 tx=0x7fc0fc00dbf0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.869+0000 7fc113d81700 1 -- 172.21.15.104:0/227394137 shutdown_connections 2023-07-14T11:56:16.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.869+0000 7fc113d81700 1 --2- 172.21.15.104:0/227394137 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc0f00662f0 0x7fc0f00687a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.869+0000 7fc113d81700 1 --2- 172.21.15.104:0/227394137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc10c06d860 0x7fc10c152f90 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.869+0000 7fc113d81700 1 --2- 172.21.15.104:0/227394137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc10c0c1e00 0x7fc10c0aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.869+0000 7fc113d81700 1 --2- 172.21.15.104:0/227394137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc10c06be70 0x7fc10c0c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:16.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.869+0000 7fc113d81700 1 -- 172.21.15.104:0/227394137 >> 172.21.15.104:0/227394137 conn(0x7fc10c0cad30 msgr2=0x7fc10c0cb100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:16.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.869+0000 7fc113d81700 1 -- 172.21.15.104:0/227394137 shutdown_connections 2023-07-14T11:56:16.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:16.869+0000 7fc113d81700 1 -- 172.21.15.104:0/227394137 wait complete. 2023-07-14T11:56:16.859 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:56:16.880 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2023-07-14T11:56:15.962209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230276,"kb_used_data":6468,"kb_used_omap":0,"kb_used_meta":223744,"kb_avail":754744444,"statfs":{"total":773094113280,"available":772858310656,"internally_reserved":0,"allocated":6623232,"data_stored":3700596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":229113856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.015474"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683147+0000","last_change":"2023-07-14T11:56:10.683147+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683147+0000","last_undegraded":"2023-07-14T11:56:10.683147+0000","last_fullsized":"2023-07-14T11:56:10.683147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679541+0000","last_change":"2023-07-14T11:56:10.679541+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679541+0000","last_undegraded":"2023-07-14T11:56:10.679541+0000","last_fullsized":"2023-07-14T11:56:10.679541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.688004+0000","last_change":"2023-07-14T11:56:10.688004+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.688004+0000","last_undegraded":"2023-07-14T11:56:10.688004+0000","last_fullsized":"2023-07-14T11:56:10.688004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683798+0000","last_change":"2023-07-14T11:56:10.683798+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683798+0000","last_undegraded":"2023-07-14T11:56:10.683798+0000","last_fullsized":"2023-07-14T11:56:10.683798+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677302+0000","last_change":"2023-07-14T11:56:10.677302+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677302+0000","last_undegraded":"2023-07-14T11:56:10.677302+0000","last_fullsized":"2023-07-14T11:56:10.677302+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.685591+0000","last_change":"2023-07-14T11:56:10.685591+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.685591+0000","last_undegraded":"2023-07-14T11:56:10.685591+0000","last_fullsized":"2023-07-14T11:56:10.685591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677725+0000","last_change":"2023-07-14T11:56:10.677725+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677725+0000","last_undegraded":"2023-07-14T11:56:10.677725+0000","last_fullsized":"2023-07-14T11:56:10.677725+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683020+0000","last_change":"2023-07-14T11:56:10.683020+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683020+0000","last_undegraded":"2023-07-14T11:56:10.683020+0000","last_fullsized":"2023-07-14T11:56:10.683020+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.689105+0000","last_change":"2023-07-14T11:56:10.689105+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.689105+0000","last_undegraded":"2023-07-14T11:56:10.689105+0000","last_fullsized":"2023-07-14T11:56:10.689105+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.689300+0000","last_change":"2023-07-14T11:56:10.689300+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.689300+0000","last_undegraded":"2023-07-14T11:56:10.689300+0000","last_fullsized":"2023-07-14T11:56:10.689300+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687590+0000","last_change":"2023-07-14T11:56:10.687590+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687590+0000","last_undegraded":"2023-07-14T11:56:10.687590+0000","last_fullsized":"2023-07-14T11:56:10.687590+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.684482+0000","last_change":"2023-07-14T11:56:10.684482+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.684482+0000","last_undegraded":"2023-07-14T11:56:10.684482+0000","last_fullsized":"2023-07-14T11:56:10.684482+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.685262+0000","last_change":"2023-07-14T11:56:10.685262+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.685262+0000","last_undegraded":"2023-07-14T11:56:10.685262+0000","last_fullsized":"2023-07-14T11:56:10.685262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687474+0000","last_change":"2023-07-14T11:56:10.687474+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687474+0000","last_undegraded":"2023-07-14T11:56:10.687474+0000","last_fullsized":"2023-07-14T11:56:10.687474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.691056+0000","last_change":"2023-07-14T11:56:10.691056+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.691056+0000","last_undegraded":"2023-07-14T11:56:10.691056+0000","last_fullsized":"2023-07-14T11:56:10.691056+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.678388+0000","last_change":"2023-07-14T11:56:10.678388+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.678388+0000","last_undegraded":"2023-07-14T11:56:10.678388+0000","last_fullsized":"2023-07-14T11:56:10.678388+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682628+0000","last_change":"2023-07-14T11:56:10.682628+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682628+0000","last_undegraded":"2023-07-14T11:56:10.682628+0000","last_fullsized":"2023-07-14T11:56:10.682628+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.685212+0000","last_change":"2023-07-14T11:56:10.685212+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.685212+0000","last_undegraded":"2023-07-14T11:56:10.685212+0000","last_fullsized":"2023-07-14T11:56:10.685212+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679247+0000","last_change":"2023-07-14T11:56:10.679247+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679247+0000","last_undegraded":"2023-07-14T11:56:10.679247+0000","last_fullsized":"2023-07-14T11:56:10.679247+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682339+0000","last_change":"2023-07-14T11:56:10.682339+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682339+0000","last_undegraded":"2023-07-14T11:56:10.682339+0000","last_fullsized":"2023-07-14T11:56:10.682339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.684054+0000","last_change":"2023-07-14T11:56:10.684054+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.684054+0000","last_undegraded":"2023-07-14T11:56:10.684054+0000","last_fullsized":"2023-07-14T11:56:10.684054+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.680085+0000","last_change":"2023-07-14T11:56:10.680085+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.680085+0000","last_undegraded":"2023-07-14T11:56:10.680085+0000","last_fullsized":"2023-07-14T11:56:10.680085+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683502+0000","last_change":"2023-07-14T11:56:10.683502+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683502+0000","last_undegraded":"2023-07-14T11:56:10.683502+0000","last_fullsized":"2023-07-14T11:56:10.683502+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686241+0000","last_change":"2023-07-14T11:56:10.686241+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686241+0000","last_undegraded":"2023-07-14T11:56:10.686241+0000","last_fullsized":"2023-07-14T11:56:10.686241+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686556+0000","last_change":"2023-07-14T11:56:10.686556+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686556+0000","last_undegraded":"2023-07-14T11:56:10.686556+0000","last_fullsized":"2023-07-14T11:56:10.686556+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686616+0000","last_change":"2023-07-14T11:56:10.686616+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686616+0000","last_undegraded":"2023-07-14T11:56:10.686616+0000","last_fullsized":"2023-07-14T11:56:10.686616+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687141+0000","last_change":"2023-07-14T11:56:10.687141+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687141+0000","last_undegraded":"2023-07-14T11:56:10.687141+0000","last_fullsized":"2023-07-14T11:56:10.687141+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.681576+0000","last_change":"2023-07-14T11:56:10.681576+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.681576+0000","last_undegraded":"2023-07-14T11:56:10.681576+0000","last_fullsized":"2023-07-14T11:56:10.681576+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.690699+0000","last_change":"2023-07-14T11:56:10.690699+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.690699+0000","last_undegraded":"2023-07-14T11:56:10.690699+0000","last_fullsized":"2023-07-14T11:56:10.690699+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683599+0000","last_change":"2023-07-14T11:56:10.683599+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683599+0000","last_undegraded":"2023-07-14T11:56:10.683599+0000","last_fullsized":"2023-07-14T11:56:10.683599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683345+0000","last_change":"2023-07-14T11:56:10.683345+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683345+0000","last_undegraded":"2023-07-14T11:56:10.683345+0000","last_fullsized":"2023-07-14T11:56:10.683345+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687533+0000","last_change":"2023-07-14T11:56:10.687533+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687533+0000","last_undegraded":"2023-07-14T11:56:10.687533+0000","last_fullsized":"2023-07-14T11:56:10.687533+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676335+0000","last_change":"2023-07-14T11:56:10.676335+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676335+0000","last_undegraded":"2023-07-14T11:56:10.676335+0000","last_fullsized":"2023-07-14T11:56:10.676335+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.702674+0000","last_change":"2023-07-14T11:56:10.696681+0000","last_active":"2023-07-14T11:56:10.702674+0000","last_peered":"2023-07-14T11:56:10.702674+0000","last_clean":"2023-07-14T11:56:10.702674+0000","last_became_active":"2023-07-14T11:56:10.696413+0000","last_became_peered":"2023-07-14T11:56:10.696413+0000","last_unstale":"2023-07-14T11:56:10.702674+0000","last_undegraded":"2023-07-14T11:56:10.702674+0000","last_fullsized":"2023-07-14T11:56:10.702674+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:34.861496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682407+0000","last_change":"2023-07-14T11:56:10.682407+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682407+0000","last_undegraded":"2023-07-14T11:56:10.682407+0000","last_fullsized":"2023-07-14T11:56:10.682407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696074+0000","last_change":"2023-07-14T11:56:10.694408+0000","last_active":"2023-07-14T11:56:10.696074+0000","last_peered":"2023-07-14T11:56:10.696074+0000","last_clean":"2023-07-14T11:56:10.696074+0000","last_became_active":"2023-07-14T11:56:10.694097+0000","last_became_peered":"2023-07-14T11:56:10.694097+0000","last_unstale":"2023-07-14T11:56:10.696074+0000","last_undegraded":"2023-07-14T11:56:10.696074+0000","last_fullsized":"2023-07-14T11:56:10.696074+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:53.935404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686577+0000","last_change":"2023-07-14T11:56:10.686577+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686577+0000","last_undegraded":"2023-07-14T11:56:10.686577+0000","last_fullsized":"2023-07-14T11:56:10.686577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.705379+0000","last_change":"2023-07-14T11:56:10.703709+0000","last_active":"2023-07-14T11:56:10.705379+0000","last_peered":"2023-07-14T11:56:10.705379+0000","last_clean":"2023-07-14T11:56:10.705379+0000","last_became_active":"2023-07-14T11:56:10.703444+0000","last_became_peered":"2023-07-14T11:56:10.703444+0000","last_unstale":"2023-07-14T11:56:10.705379+0000","last_undegraded":"2023-07-14T11:56:10.705379+0000","last_fullsized":"2023-07-14T11:56:10.705379+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:01:25.510366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.691008+0000","last_change":"2023-07-14T11:56:10.691008+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.691008+0000","last_undegraded":"2023-07-14T11:56:10.691008+0000","last_fullsized":"2023-07-14T11:56:10.691008+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703069+0000","last_change":"2023-07-14T11:56:10.701563+0000","last_active":"2023-07-14T11:56:10.703069+0000","last_peered":"2023-07-14T11:56:10.703069+0000","last_clean":"2023-07-14T11:56:10.703069+0000","last_became_active":"2023-07-14T11:56:10.701262+0000","last_became_peered":"2023-07-14T11:56:10.701262+0000","last_unstale":"2023-07-14T11:56:10.703069+0000","last_undegraded":"2023-07-14T11:56:10.703069+0000","last_fullsized":"2023-07-14T11:56:10.703069+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:14.962001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677447+0000","last_change":"2023-07-14T11:56:10.677447+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677447+0000","last_undegraded":"2023-07-14T11:56:10.677447+0000","last_fullsized":"2023-07-14T11:56:10.677447+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.697759+0000","last_change":"2023-07-14T11:56:10.694726+0000","last_active":"2023-07-14T11:56:10.697759+0000","last_peered":"2023-07-14T11:56:10.697759+0000","last_clean":"2023-07-14T11:56:10.697759+0000","last_became_active":"2023-07-14T11:56:10.694444+0000","last_became_peered":"2023-07-14T11:56:10.694444+0000","last_unstale":"2023-07-14T11:56:10.697759+0000","last_undegraded":"2023-07-14T11:56:10.697759+0000","last_fullsized":"2023-07-14T11:56:10.697759+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:58:15.607086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677746+0000","last_change":"2023-07-14T11:56:10.677746+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677746+0000","last_undegraded":"2023-07-14T11:56:10.677746+0000","last_fullsized":"2023-07-14T11:56:10.677746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.692679+0000","last_change":"2023-07-14T11:56:10.689607+0000","last_active":"2023-07-14T11:56:10.692679+0000","last_peered":"2023-07-14T11:56:10.692679+0000","last_clean":"2023-07-14T11:56:10.692679+0000","last_became_active":"2023-07-14T11:56:10.689345+0000","last_became_peered":"2023-07-14T11:56:10.689345+0000","last_unstale":"2023-07-14T11:56:10.692679+0000","last_undegraded":"2023-07-14T11:56:10.692679+0000","last_fullsized":"2023-07-14T11:56:10.692679+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:00:10.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679929+0000","last_change":"2023-07-14T11:56:10.679929+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679929+0000","last_undegraded":"2023-07-14T11:56:10.679929+0000","last_fullsized":"2023-07-14T11:56:10.679929+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.693487+0000","last_change":"2023-07-14T11:56:10.691388+0000","last_active":"2023-07-14T11:56:10.693487+0000","last_peered":"2023-07-14T11:56:10.693487+0000","last_clean":"2023-07-14T11:56:10.693487+0000","last_became_active":"2023-07-14T11:56:10.690746+0000","last_became_peered":"2023-07-14T11:56:10.690746+0000","last_unstale":"2023-07-14T11:56:10.693487+0000","last_undegraded":"2023-07-14T11:56:10.693487+0000","last_fullsized":"2023-07-14T11:56:10.693487+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:01.327056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682930+0000","last_change":"2023-07-14T11:56:10.682930+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682930+0000","last_undegraded":"2023-07-14T11:56:10.682930+0000","last_fullsized":"2023-07-14T11:56:10.682930+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.700186+0000","last_change":"2023-07-14T11:56:10.694837+0000","last_active":"2023-07-14T11:56:10.700186+0000","last_peered":"2023-07-14T11:56:10.700186+0000","last_clean":"2023-07-14T11:56:10.700186+0000","last_became_active":"2023-07-14T11:56:10.694071+0000","last_became_peered":"2023-07-14T11:56:10.694071+0000","last_unstale":"2023-07-14T11:56:10.700186+0000","last_undegraded":"2023-07-14T11:56:10.700186+0000","last_fullsized":"2023-07-14T11:56:10.700186+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:12.411429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.681039+0000","last_change":"2023-07-14T11:56:10.681039+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.681039+0000","last_undegraded":"2023-07-14T11:56:10.681039+0000","last_fullsized":"2023-07-14T11:56:10.681039+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.708387+0000","last_change":"2023-07-14T11:56:10.705201+0000","last_active":"2023-07-14T11:56:10.708387+0000","last_peered":"2023-07-14T11:56:10.708387+0000","last_clean":"2023-07-14T11:56:10.708387+0000","last_became_active":"2023-07-14T11:56:10.704922+0000","last_became_peered":"2023-07-14T11:56:10.704922+0000","last_unstale":"2023-07-14T11:56:10.708387+0000","last_undegraded":"2023-07-14T11:56:10.708387+0000","last_fullsized":"2023-07-14T11:56:10.708387+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:32:12.442228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.680746+0000","last_change":"2023-07-14T11:56:10.680746+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.680746+0000","last_undegraded":"2023-07-14T11:56:10.680746+0000","last_fullsized":"2023-07-14T11:56:10.680746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696725+0000","last_change":"2023-07-14T11:56:10.692503+0000","last_active":"2023-07-14T11:56:10.696725+0000","last_peered":"2023-07-14T11:56:10.696725+0000","last_clean":"2023-07-14T11:56:10.696725+0000","last_became_active":"2023-07-14T11:56:10.692092+0000","last_became_peered":"2023-07-14T11:56:10.692092+0000","last_unstale":"2023-07-14T11:56:10.696725+0000","last_undegraded":"2023-07-14T11:56:10.696725+0000","last_fullsized":"2023-07-14T11:56:10.696725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:05.395217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677241+0000","last_change":"2023-07-14T11:56:10.677241+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677241+0000","last_undegraded":"2023-07-14T11:56:10.677241+0000","last_fullsized":"2023-07-14T11:56:10.677241+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707828+0000","last_change":"2023-07-14T11:56:10.703832+0000","last_active":"2023-07-14T11:56:10.707828+0000","last_peered":"2023-07-14T11:56:10.707828+0000","last_clean":"2023-07-14T11:56:10.707828+0000","last_became_active":"2023-07-14T11:56:10.703548+0000","last_became_peered":"2023-07-14T11:56:10.703548+0000","last_unstale":"2023-07-14T11:56:10.707828+0000","last_undegraded":"2023-07-14T11:56:10.707828+0000","last_fullsized":"2023-07-14T11:56:10.707828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:03:55.287678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682757+0000","last_change":"2023-07-14T11:56:10.682757+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682757+0000","last_undegraded":"2023-07-14T11:56:10.682757+0000","last_fullsized":"2023-07-14T11:56:10.682757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703549+0000","last_change":"2023-07-14T11:56:10.697093+0000","last_active":"2023-07-14T11:56:10.703549+0000","last_peered":"2023-07-14T11:56:10.703549+0000","last_clean":"2023-07-14T11:56:10.703549+0000","last_became_active":"2023-07-14T11:56:10.692072+0000","last_became_peered":"2023-07-14T11:56:10.692072+0000","last_unstale":"2023-07-14T11:56:10.703549+0000","last_undegraded":"2023-07-14T11:56:10.703549+0000","last_fullsized":"2023-07-14T11:56:10.703549+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:46:05.586216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682498+0000","last_change":"2023-07-14T11:56:10.682498+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682498+0000","last_undegraded":"2023-07-14T11:56:10.682498+0000","last_fullsized":"2023-07-14T11:56:10.682498+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707805+0000","last_change":"2023-07-14T11:56:10.701400+0000","last_active":"2023-07-14T11:56:10.707805+0000","last_peered":"2023-07-14T11:56:10.707805+0000","last_clean":"2023-07-14T11:56:10.707805+0000","last_became_active":"2023-07-14T11:56:10.701104+0000","last_became_peered":"2023-07-14T11:56:10.701104+0000","last_unstale":"2023-07-14T11:56:10.707805+0000","last_undegraded":"2023-07-14T11:56:10.707805+0000","last_fullsized":"2023-07-14T11:56:10.707805+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:04.078467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683497+0000","last_change":"2023-07-14T11:56:10.683497+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683497+0000","last_undegraded":"2023-07-14T11:56:10.683497+0000","last_fullsized":"2023-07-14T11:56:10.683497+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.705209+0000","last_change":"2023-07-14T11:56:10.702066+0000","last_active":"2023-07-14T11:56:10.705209+0000","last_peered":"2023-07-14T11:56:10.705209+0000","last_clean":"2023-07-14T11:56:10.705209+0000","last_became_active":"2023-07-14T11:56:10.701720+0000","last_became_peered":"2023-07-14T11:56:10.701720+0000","last_unstale":"2023-07-14T11:56:10.705209+0000","last_undegraded":"2023-07-14T11:56:10.705209+0000","last_fullsized":"2023-07-14T11:56:10.705209+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:34:01.415077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679469+0000","last_change":"2023-07-14T11:56:10.679469+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679469+0000","last_undegraded":"2023-07-14T11:56:10.679469+0000","last_fullsized":"2023-07-14T11:56:10.679469+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.704869+0000","last_change":"2023-07-14T11:56:10.699176+0000","last_active":"2023-07-14T11:56:10.704869+0000","last_peered":"2023-07-14T11:56:10.704869+0000","last_clean":"2023-07-14T11:56:10.704869+0000","last_became_active":"2023-07-14T11:56:10.698881+0000","last_became_peered":"2023-07-14T11:56:10.698881+0000","last_unstale":"2023-07-14T11:56:10.704869+0000","last_undegraded":"2023-07-14T11:56:10.704869+0000","last_fullsized":"2023-07-14T11:56:10.704869+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:50.194031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677560+0000","last_change":"2023-07-14T11:56:10.677560+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677560+0000","last_undegraded":"2023-07-14T11:56:10.677560+0000","last_fullsized":"2023-07-14T11:56:10.677560+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.698670+0000","last_change":"2023-07-14T11:56:10.695120+0000","last_active":"2023-07-14T11:56:10.698670+0000","last_peered":"2023-07-14T11:56:10.698670+0000","last_clean":"2023-07-14T11:56:10.698670+0000","last_became_active":"2023-07-14T11:56:10.693407+0000","last_became_peered":"2023-07-14T11:56:10.693407+0000","last_unstale":"2023-07-14T11:56:10.698670+0000","last_undegraded":"2023-07-14T11:56:10.698670+0000","last_fullsized":"2023-07-14T11:56:10.698670+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:47:56.814799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.688739+0000","last_change":"2023-07-14T11:56:10.688739+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.688739+0000","last_undegraded":"2023-07-14T11:56:10.688739+0000","last_fullsized":"2023-07-14T11:56:10.688739+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703966+0000","last_change":"2023-07-14T11:56:10.700844+0000","last_active":"2023-07-14T11:56:10.703966+0000","last_peered":"2023-07-14T11:56:10.703966+0000","last_clean":"2023-07-14T11:56:10.703966+0000","last_became_active":"2023-07-14T11:56:10.700534+0000","last_became_peered":"2023-07-14T11:56:10.700534+0000","last_unstale":"2023-07-14T11:56:10.703966+0000","last_undegraded":"2023-07-14T11:56:10.703966+0000","last_fullsized":"2023-07-14T11:56:10.703966+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:45:37.602121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.678287+0000","last_change":"2023-07-14T11:56:10.678287+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.678287+0000","last_undegraded":"2023-07-14T11:56:10.678287+0000","last_fullsized":"2023-07-14T11:56:10.678287+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703169+0000","last_change":"2023-07-14T11:56:10.695559+0000","last_active":"2023-07-14T11:56:10.703169+0000","last_peered":"2023-07-14T11:56:10.703169+0000","last_clean":"2023-07-14T11:56:10.703169+0000","last_became_active":"2023-07-14T11:56:10.695312+0000","last_became_peered":"2023-07-14T11:56:10.695312+0000","last_unstale":"2023-07-14T11:56:10.703169+0000","last_undegraded":"2023-07-14T11:56:10.703169+0000","last_fullsized":"2023-07-14T11:56:10.703169+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:01.509577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686132+0000","last_change":"2023-07-14T11:56:10.686132+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686132+0000","last_undegraded":"2023-07-14T11:56:10.686132+0000","last_fullsized":"2023-07-14T11:56:10.686132+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.704530+0000","last_change":"2023-07-14T11:56:10.703244+0000","last_active":"2023-07-14T11:56:10.704530+0000","last_peered":"2023-07-14T11:56:10.704530+0000","last_clean":"2023-07-14T11:56:10.704530+0000","last_became_active":"2023-07-14T11:56:10.702957+0000","last_became_peered":"2023-07-14T11:56:10.702957+0000","last_unstale":"2023-07-14T11:56:10.704530+0000","last_undegraded":"2023-07-14T11:56:10.704530+0000","last_fullsized":"2023-07-14T11:56:10.704530+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:47:56.814799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677574+0000","last_change":"2023-07-14T11:56:10.677574+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677574+0000","last_undegraded":"2023-07-14T11:56:10.677574+0000","last_fullsized":"2023-07-14T11:56:10.677574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.700755+0000","last_change":"2023-07-14T11:56:10.695223+0000","last_active":"2023-07-14T11:56:10.700755+0000","last_peered":"2023-07-14T11:56:10.700755+0000","last_clean":"2023-07-14T11:56:10.700755+0000","last_became_active":"2023-07-14T11:56:10.691638+0000","last_became_peered":"2023-07-14T11:56:10.691638+0000","last_unstale":"2023-07-14T11:56:10.700755+0000","last_undegraded":"2023-07-14T11:56:10.700755+0000","last_fullsized":"2023-07-14T11:56:10.700755+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:06:54.458842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676708+0000","last_change":"2023-07-14T11:56:10.676708+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676708+0000","last_undegraded":"2023-07-14T11:56:10.676708+0000","last_fullsized":"2023-07-14T11:56:10.676708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.704883+0000","last_change":"2023-07-14T11:56:10.699836+0000","last_active":"2023-07-14T11:56:10.704883+0000","last_peered":"2023-07-14T11:56:10.704883+0000","last_clean":"2023-07-14T11:56:10.704883+0000","last_became_active":"2023-07-14T11:56:10.699528+0000","last_became_peered":"2023-07-14T11:56:10.699528+0000","last_unstale":"2023-07-14T11:56:10.704883+0000","last_undegraded":"2023-07-14T11:56:10.704883+0000","last_fullsized":"2023-07-14T11:56:10.704883+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:28:16.553232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682860+0000","last_change":"2023-07-14T11:56:10.682860+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682860+0000","last_undegraded":"2023-07-14T11:56:10.682860+0000","last_fullsized":"2023-07-14T11:56:10.682860+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.695725+0000","last_change":"2023-07-14T11:56:10.693503+0000","last_active":"2023-07-14T11:56:10.695725+0000","last_peered":"2023-07-14T11:56:10.695725+0000","last_clean":"2023-07-14T11:56:10.695725+0000","last_became_active":"2023-07-14T11:56:10.693197+0000","last_became_peered":"2023-07-14T11:56:10.693197+0000","last_unstale":"2023-07-14T11:56:10.695725+0000","last_undegraded":"2023-07-14T11:56:10.695725+0000","last_fullsized":"2023-07-14T11:56:10.695725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:21:50.291873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677561+0000","last_change":"2023-07-14T11:56:10.677561+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677561+0000","last_undegraded":"2023-07-14T11:56:10.677561+0000","last_fullsized":"2023-07-14T11:56:10.677561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.693506+0000","last_change":"2023-07-14T11:56:10.691758+0000","last_active":"2023-07-14T11:56:10.693506+0000","last_peered":"2023-07-14T11:56:10.693506+0000","last_clean":"2023-07-14T11:56:10.693506+0000","last_became_active":"2023-07-14T11:56:10.691130+0000","last_became_peered":"2023-07-14T11:56:10.691130+0000","last_unstale":"2023-07-14T11:56:10.693506+0000","last_undegraded":"2023-07-14T11:56:10.693506+0000","last_fullsized":"2023-07-14T11:56:10.693506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:35:21.027007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.678966+0000","last_change":"2023-07-14T11:56:10.678966+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.678966+0000","last_undegraded":"2023-07-14T11:56:10.678966+0000","last_fullsized":"2023-07-14T11:56:10.678966+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.700706+0000","last_change":"2023-07-14T11:56:10.692036+0000","last_active":"2023-07-14T11:56:10.700706+0000","last_peered":"2023-07-14T11:56:10.700706+0000","last_clean":"2023-07-14T11:56:10.700706+0000","last_became_active":"2023-07-14T11:56:10.691608+0000","last_became_peered":"2023-07-14T11:56:10.691608+0000","last_unstale":"2023-07-14T11:56:10.700706+0000","last_undegraded":"2023-07-14T11:56:10.700706+0000","last_fullsized":"2023-07-14T11:56:10.700706+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:30:54.612583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676440+0000","last_change":"2023-07-14T11:56:10.676440+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676440+0000","last_undegraded":"2023-07-14T11:56:10.676440+0000","last_fullsized":"2023-07-14T11:56:10.676440+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.692802+0000","last_change":"2023-07-14T11:56:10.690467+0000","last_active":"2023-07-14T11:56:10.692802+0000","last_peered":"2023-07-14T11:56:10.692802+0000","last_clean":"2023-07-14T11:56:10.692802+0000","last_became_active":"2023-07-14T11:56:10.690201+0000","last_became_peered":"2023-07-14T11:56:10.690201+0000","last_unstale":"2023-07-14T11:56:10.692802+0000","last_undegraded":"2023-07-14T11:56:10.692802+0000","last_fullsized":"2023-07-14T11:56:10.692802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:06:38.660785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.681506+0000","last_change":"2023-07-14T11:53:09.145201+0000","last_active":"2023-07-14T11:56:10.681506+0000","last_peered":"2023-07-14T11:56:10.681506+0000","last_clean":"2023-07-14T11:56:10.681506+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:56:10.681506+0000","last_undegraded":"2023-07-14T11:56:10.681506+0000","last_fullsized":"2023-07-14T11:56:10.681506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:16:24.795475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683131+0000","last_change":"2023-07-14T11:56:10.683131+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683131+0000","last_undegraded":"2023-07-14T11:56:10.683131+0000","last_fullsized":"2023-07-14T11:56:10.683131+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707863+0000","last_change":"2023-07-14T11:56:10.704116+0000","last_active":"2023-07-14T11:56:10.707863+0000","last_peered":"2023-07-14T11:56:10.707863+0000","last_clean":"2023-07-14T11:56:10.707863+0000","last_became_active":"2023-07-14T11:56:10.703775+0000","last_became_peered":"2023-07-14T11:56:10.703775+0000","last_unstale":"2023-07-14T11:56:10.707863+0000","last_undegraded":"2023-07-14T11:56:10.707863+0000","last_fullsized":"2023-07-14T11:56:10.707863+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:23:09.743965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.687564+0000","last_change":"2023-07-14T11:53:09.146150+0000","last_active":"2023-07-14T11:56:10.687564+0000","last_peered":"2023-07-14T11:56:10.687564+0000","last_clean":"2023-07-14T11:56:10.687564+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:56:10.687564+0000","last_undegraded":"2023-07-14T11:56:10.687564+0000","last_fullsized":"2023-07-14T11:56:10.687564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:49:39.347836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676479+0000","last_change":"2023-07-14T11:56:10.676479+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676479+0000","last_undegraded":"2023-07-14T11:56:10.676479+0000","last_fullsized":"2023-07-14T11:56:10.676479+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707751+0000","last_change":"2023-07-14T11:56:10.700575+0000","last_active":"2023-07-14T11:56:10.707751+0000","last_peered":"2023-07-14T11:56:10.707751+0000","last_clean":"2023-07-14T11:56:10.707751+0000","last_became_active":"2023-07-14T11:56:10.700293+0000","last_became_peered":"2023-07-14T11:56:10.700293+0000","last_unstale":"2023-07-14T11:56:10.707751+0000","last_undegraded":"2023-07-14T11:56:10.707751+0000","last_fullsized":"2023-07-14T11:56:10.707751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:24:16.135877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.694687+0000","last_change":"2023-07-14T11:53:09.435876+0000","last_active":"2023-07-14T11:56:10.694687+0000","last_peered":"2023-07-14T11:56:10.694687+0000","last_clean":"2023-07-14T11:56:10.694687+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:56:10.694687+0000","last_undegraded":"2023-07-14T11:56:10.694687+0000","last_fullsized":"2023-07-14T11:56:10.694687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:29:06.737180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687524+0000","last_change":"2023-07-14T11:56:10.687524+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687524+0000","last_undegraded":"2023-07-14T11:56:10.687524+0000","last_fullsized":"2023-07-14T11:56:10.687524+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707881+0000","last_change":"2023-07-14T11:56:10.702843+0000","last_active":"2023-07-14T11:56:10.707881+0000","last_peered":"2023-07-14T11:56:10.707881+0000","last_clean":"2023-07-14T11:56:10.707881+0000","last_became_active":"2023-07-14T11:56:10.702583+0000","last_became_peered":"2023-07-14T11:56:10.702583+0000","last_unstale":"2023-07-14T11:56:10.707881+0000","last_undegraded":"2023-07-14T11:56:10.707881+0000","last_fullsized":"2023-07-14T11:56:10.707881+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:08:58.730889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.687487+0000","last_change":"2023-07-14T11:53:09.434255+0000","last_active":"2023-07-14T11:56:10.687487+0000","last_peered":"2023-07-14T11:56:10.687487+0000","last_clean":"2023-07-14T11:56:10.687487+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:56:10.687487+0000","last_undegraded":"2023-07-14T11:56:10.687487+0000","last_fullsized":"2023-07-14T11:56:10.687487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:11:16.228885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677097+0000","last_change":"2023-07-14T11:56:10.677097+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677097+0000","last_undegraded":"2023-07-14T11:56:10.677097+0000","last_fullsized":"2023-07-14T11:56:10.677097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.699530+0000","last_change":"2023-07-14T11:56:10.697670+0000","last_active":"2023-07-14T11:56:10.699530+0000","last_peered":"2023-07-14T11:56:10.699530+0000","last_clean":"2023-07-14T11:56:10.699530+0000","last_became_active":"2023-07-14T11:56:10.697406+0000","last_became_peered":"2023-07-14T11:56:10.697406+0000","last_unstale":"2023-07-14T11:56:10.699530+0000","last_undegraded":"2023-07-14T11:56:10.699530+0000","last_fullsized":"2023-07-14T11:56:10.699530+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:05.395217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":84,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.694638+0000","last_change":"2023-07-14T11:53:09.147040+0000","last_active":"2023-07-14T11:56:10.694638+0000","last_peered":"2023-07-14T11:56:10.694638+0000","last_clean":"2023-07-14T11:56:10.694638+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:56:10.694638+0000","last_undegraded":"2023-07-14T11:56:10.694638+0000","last_fullsized":"2023-07-14T11:56:10.694638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:37.742068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682908+0000","last_change":"2023-07-14T11:56:10.682908+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682908+0000","last_undegraded":"2023-07-14T11:56:10.682908+0000","last_fullsized":"2023-07-14T11:56:10.682908+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.705188+0000","last_change":"2023-07-14T11:56:10.701923+0000","last_active":"2023-07-14T11:56:10.705188+0000","last_peered":"2023-07-14T11:56:10.705188+0000","last_clean":"2023-07-14T11:56:10.705188+0000","last_became_active":"2023-07-14T11:56:10.701351+0000","last_became_peered":"2023-07-14T11:56:10.701351+0000","last_unstale":"2023-07-14T11:56:10.705188+0000","last_undegraded":"2023-07-14T11:56:10.705188+0000","last_fullsized":"2023-07-14T11:56:10.705188+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:01:13.092675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":86,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.673618+0000","last_change":"2023-07-14T11:53:09.619294+0000","last_active":"2023-07-14T11:56:10.673618+0000","last_peered":"2023-07-14T11:56:10.673618+0000","last_clean":"2023-07-14T11:56:10.673618+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:56:10.673618+0000","last_undegraded":"2023-07-14T11:56:10.673618+0000","last_fullsized":"2023-07-14T11:56:10.673618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:07:12.290309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687092+0000","last_change":"2023-07-14T11:56:10.687092+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687092+0000","last_undegraded":"2023-07-14T11:56:10.687092+0000","last_fullsized":"2023-07-14T11:56:10.687092+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696108+0000","last_change":"2023-07-14T11:56:10.693854+0000","last_active":"2023-07-14T11:56:10.696108+0000","last_peered":"2023-07-14T11:56:10.696108+0000","last_clean":"2023-07-14T11:56:10.696108+0000","last_became_active":"2023-07-14T11:56:10.693588+0000","last_became_peered":"2023-07-14T11:56:10.693588+0000","last_unstale":"2023-07-14T11:56:10.696108+0000","last_undegraded":"2023-07-14T11:56:10.696108+0000","last_fullsized":"2023-07-14T11:56:10.696108+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:37:02.428527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":72,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696339+0000","last_change":"2023-07-14T11:53:09.146156+0000","last_active":"2023-07-14T11:56:10.696339+0000","last_peered":"2023-07-14T11:56:10.696339+0000","last_clean":"2023-07-14T11:56:10.696339+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:56:10.696339+0000","last_undegraded":"2023-07-14T11:56:10.696339+0000","last_fullsized":"2023-07-14T11:56:10.696339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:05:34.882240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676332+0000","last_change":"2023-07-14T11:56:10.676332+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676332+0000","last_undegraded":"2023-07-14T11:56:10.676332+0000","last_fullsized":"2023-07-14T11:56:10.676332+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.697324+0000","last_change":"2023-07-14T11:56:10.694832+0000","last_active":"2023-07-14T11:56:10.697324+0000","last_peered":"2023-07-14T11:56:10.697324+0000","last_clean":"2023-07-14T11:56:10.697324+0000","last_became_active":"2023-07-14T11:56:10.694539+0000","last_became_peered":"2023-07-14T11:56:10.694539+0000","last_unstale":"2023-07-14T11:56:10.697324+0000","last_undegraded":"2023-07-14T11:56:10.697324+0000","last_fullsized":"2023-07-14T11:56:10.697324+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:17:28.732628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":84,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.694479+0000","last_change":"2023-07-14T11:53:09.146556+0000","last_active":"2023-07-14T11:56:10.694479+0000","last_peered":"2023-07-14T11:56:10.694479+0000","last_clean":"2023-07-14T11:56:10.694479+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:56:10.694479+0000","last_undegraded":"2023-07-14T11:56:10.694479+0000","last_fullsized":"2023-07-14T11:56:10.694479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:34.689837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696481+0000","last_change":"2023-07-14T11:53:03.159818+0000","last_active":"2023-07-14T11:56:10.696481+0000","last_peered":"2023-07-14T11:56:10.696481+0000","last_clean":"2023-07-14T11:56:10.696481+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:56:10.696481+0000","last_undegraded":"2023-07-14T11:56:10.696481+0000","last_fullsized":"2023-07-14T11:56:10.696481+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_clean_scrub_stamp":"2023-07-14T11:53:02.054348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:05:58.561507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1265664,"data_stored":901013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.093,"15min":1.111},"min":{"1min":0.754,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.474,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.103,"15min":1.127},"min":{"1min":0.65100000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2430000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.4450000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1499999999999999,"15min":1.171},"min":{"1min":0.68300000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.298,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.22,"15min":1.262},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.4159999999999999,"5min":2.234,"15min":2.234},"last":0.88300000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.234,"15min":1.288},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.536,"5min":2.04,"15min":2.04},"last":1.143},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.25,"15min":1.2829999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8089999999999999,"5min":2.492,"15min":2.492},"last":1.1080000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.282,"15min":1.3360000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.665,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.2430000000000001,"15min":1.274},"min":{"1min":0.81799999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5940000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.81699999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.321,"15min":1.385},"min":{"1min":0.84399999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5660000000000001,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":1.371},{"interface":"front","average":{"1min":1.286,"5min":1.3400000000000001,"15min":1.3799999999999999},"min":{"1min":0.92500000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.76,"5min":2.399,"15min":2.399},"last":1.2490000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.284,"15min":1.3200000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6459999999999999,"5min":2.387,"15min":2.387},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.456,"15min":1.538},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.464,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":1.012}]},{"osd":6,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.5309999999999999,"15min":1.609},"min":{"1min":1.0940000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5289999999999999,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.494},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.4550000000000001,"15min":1.5229999999999999},"min":{"1min":0.95199999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.53,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.5049999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0700000000000001,"15min":1.0620000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.397,"5min":1.954,"15min":1.954},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.1160000000000001,"15min":1.1000000000000001},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.3460000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1890000000000001,"15min":1.2090000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7110000000000001,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.214,"15min":1.23},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7769999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.4770000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.171,"15min":1.1659999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4690000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2709999999999999,"15min":1.2729999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6819999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.0449999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.3140000000000001,"15min":1.3440000000000001},"min":{"1min":0.94099999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.6579999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.1899999999999999,"15min":1.1799999999999999},"min":{"1min":0.91700000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.482,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.98499999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.236,"15min":1.2529999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.625,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.133},{"interface":"front","average":{"1min":1.194,"5min":1.3129999999999999,"15min":1.3400000000000001},"min":{"1min":0.78000000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.512,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.599}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.238,"15min":1.2270000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.419},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4039999999999999,"15min":1.425},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.9139999999999999,"5min":1.984,"15min":1.984},"last":1.218}]},{"osd":7,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.4330000000000001,"15min":1.4650000000000001},"min":{"1min":0.99199999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.6519999999999999,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.643},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.365,"15min":1.379},"min":{"1min":0.92900000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.728,"5min":1.879,"15min":1.879},"last":1.571}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.1359999999999999,"15min":1.1819999999999999},"min":{"1min":0.68100000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4790000000000001,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.129,"15min":1.175},"min":{"1min":0.70299999999999996,"5min":0.622,"15min":0.622},"max":{"1min":1.474,"5min":1.581,"15min":1.581},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.167,"15min":1.1990000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.524,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.224},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.175,"15min":1.1890000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.498,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.1739999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.198,"15min":1.226},"min":{"1min":0.86699999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4550000000000001,"5min":1.839,"15min":1.839},"last":1.002},{"interface":"front","average":{"1min":1.113,"5min":1.282,"15min":1.347},"min":{"1min":0.73799999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4990000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.083}]},{"osd":3,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.272,"15min":1.3220000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.5409999999999999,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.242,"5min":1.3440000000000001,"15min":1.3919999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.5349999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.399}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2090000000000001,"15min":1.2490000000000001},"min":{"1min":0.75900000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5900000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.3540000000000001,"15min":1.3740000000000001},"min":{"1min":1.024,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.5109999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.264}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3959999999999999,"15min":1.4179999999999999},"min":{"1min":1.1459999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.544,"5min":1.99,"15min":1.99},"last":1.341},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.375,"15min":1.4159999999999999},"min":{"1min":0.86399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6779999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.3069999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4039999999999999,"15min":1.421},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.6220000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.5569999999999999,"15min":1.6160000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.5880000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.4430000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.149,"15min":1.171},"min":{"1min":0.67200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.5069999999999999,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.133,"15min":1.1599999999999999},"min":{"1min":0.71099999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.76500000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.159,"15min":1.1819999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.508,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.871},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1879999999999999,"15min":1.2090000000000001},"min":{"1min":0.75900000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.516,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.3}]},{"osd":2,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1359999999999999,"15min":1.143},"min":{"1min":0.83899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.546,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.095},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3400000000000001,"15min":1.3620000000000001},"min":{"1min":0.96599999999999997,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.581,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":1.046}]},{"osd":3,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2789999999999999,"15min":1.321},"min":{"1min":0.82199999999999995,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.681,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2270000000000001,"15min":1.2450000000000001},"min":{"1min":0.88100000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.4770000000000001,"5min":1.823,"15min":1.823},"last":1.3799999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2989999999999999,"15min":1.3420000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6299999999999999,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.421},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.89500000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.47,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.2609999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.264,"15min":1.2669999999999999},"min":{"1min":0.97799999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.522,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.476,"15min":1.532},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6759999999999999,"5min":3.5289999999999999,"15min":3.5289999999999999},"last":1.5109999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.466,"15min":1.5049999999999999},"min":{"1min":1.0589999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.6899999999999999,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.47,"15min":1.5209999999999999},"min":{"1min":1.0640000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":1.605,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":1.55}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":671744,"data_stored":310645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.84099999999999997,"15min":0.871},"min":{"1min":0.47699999999999998,"5min":0.247,"15min":0.247},"max":{"1min":1.234,"5min":1.325,"15min":1.325},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.875,"15min":0.90800000000000003},"min":{"1min":0.60599999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2490000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.96699999999999997}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92300000000000004,"15min":0.96299999999999997},"min":{"1min":0.52600000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6950000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.0600000000000001,"15min":1.111},"min":{"1min":0.54300000000000004,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.319,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.88600000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.097,"15min":1.141},"min":{"1min":0.63500000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4259999999999999,"5min":1.627,"15min":1.627},"last":1.119},{"interface":"front","average":{"1min":1.079,"5min":1.0669999999999999,"15min":1.091},"min":{"1min":0.73399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.94099999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2410000000000001,"15min":1.2849999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.53,"5min":2.335,"15min":2.335},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.2470000000000001,"15min":1.2869999999999999},"min":{"1min":0.77700000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.1659999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.323,"15min":1.359},"min":{"1min":0.94999999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.3580000000000001,"15min":1.421},"min":{"1min":0.86899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.532,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.2}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.329,"15min":1.387},"min":{"1min":0.85899999999999999,"5min":0.625,"15min":0.625},"max":{"1min":1.7010000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.4550000000000001,"15min":1.5009999999999999},"min":{"1min":1.083,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.0790000000000002,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.3819999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.419,"15min":1.4810000000000001},"min":{"1min":0.90500000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.712,"5min":3.395,"15min":3.395},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.367,"15min":1.431},"min":{"1min":1.0680000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.619,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.1850000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.874,"15min":0.90600000000000003},"min":{"1min":0.53800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.22,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.876,"15min":0.90200000000000002},"min":{"1min":0.48799999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1799999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.95499999999999996}]},{"osd":1,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.88100000000000001,"15min":0.89900000000000002},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.093,"5min":1.365,"15min":1.365},"last":0.622},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0269999999999999,"15min":1.069},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.425,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.202}]},{"osd":3,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0649999999999999,"15min":1.0960000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5089999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.018,"5min":1.026,"15min":1.052},"min":{"1min":0.65300000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.5900000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.1419999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2709999999999999,"15min":1.2989999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.718,"5min":2.097,"15min":2.097},"last":1.268},{"interface":"front","average":{"1min":1.242,"5min":1.2769999999999999,"15min":1.3149999999999999},"min":{"1min":0.67800000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.861,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.7270000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.2889999999999999,"15min":1.319},"min":{"1min":0.95699999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.4710000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.242},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3180000000000001,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.645,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.3520000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.333,"15min":1.383},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7649999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.7849999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.3919999999999999,"15min":1.425},"min":{"1min":1.1040000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.726,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.321}]},{"osd":7,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.397,"15min":1.444},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.677,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.9079999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4219999999999999,"15min":1.466},"min":{"1min":1.01,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6899999999999999,"5min":2.536,"15min":2.536},"last":1.4219999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.80100000000000005,"15min":0.83299999999999996},"min":{"1min":0.48899999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.26,"5min":1.349,"15min":1.349},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83299999999999996,"15min":0.83399999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.113}]},{"osd":2,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84799999999999998,"15min":0.85799999999999998},"min":{"1min":0.499,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.93500000000000005,"15min":0.93999999999999995},"min":{"1min":0.55900000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.81799999999999995}]},{"osd":3,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.99199999999999999,"15min":1.002},"min":{"1min":0.78500000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.399,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.272},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0169999999999999,"15min":1.0229999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.8169999999999999,"5min":1.857,"15min":1.857},"last":0.874}]},{"osd":4,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.2729999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6279999999999999,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.022},{"interface":"front","average":{"1min":1.111,"5min":1.1830000000000001,"15min":1.2170000000000001},"min":{"1min":0.754,"5min":0.745,"15min":0.745},"max":{"1min":1.6279999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.206}]},{"osd":5,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.23,"15min":1.256},"min":{"1min":0.876,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7190000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3149999999999999,"15min":1.365},"min":{"1min":0.93100000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.778,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3560000000000001,"15min":1.387},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.9119999999999999,"5min":2.633,"15min":2.633},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.335,"15min":1.359},"min":{"1min":1.091,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.873,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.367}]},{"osd":7,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3580000000000001,"15min":1.3979999999999999},"min":{"1min":1.004,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.992,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3300000000000001,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.754,"15min":0.754},"max":{"1min":2.056,"5min":2.375,"15min":2.375},"last":1.3260000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1265664,"data_stored":901013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.879,"15min":0.89300000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3100000000000001,"5min":1.581,"15min":1.581},"last":1.0329999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0049999999999999,"15min":1},"min":{"1min":0.64800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.5880000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90800000000000003,"15min":0.95099999999999996},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.1639999999999999,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.93600000000000005,"15min":0.94599999999999995},"min":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.228,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.099,"15min":1.1319999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3799999999999999,"5min":1.704,"15min":1.704},"last":1.22}]},{"osd":4,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2430000000000001,"15min":1.278},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5149999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3420000000000001,"15min":1.3720000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.837,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.345}]},{"osd":5,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.228,"15min":1.284},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9870000000000001,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.276,"15min":1.3200000000000001},"min":{"1min":0.79800000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.7749999999999999,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.0940000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.0459999999999998,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.319,"15min":1.357},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.9299999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.3300000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.335,"15min":1.3819999999999999},"min":{"1min":0.83299999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.865,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.379,"15min":1.417},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.6870000000000001,"5min":2.109,"15min":2.109},"last":1.2929999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:56:16.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2023-07-14T11:56:16.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.683147+0000', 'last_change': '2023-07-14T11:56:10.683147+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.683147+0000', 'last_undegraded': '2023-07-14T11:56:10.683147+0000', 'last_fullsized': '2023-07-14T11:56:10.683147+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-14T11:56:16.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2023-07-14T11:56:16.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.679541+0000', 'last_change': '2023-07-14T11:56:10.679541+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.679541+0000', 'last_undegraded': '2023-07-14T11:56:10.679541+0000', 'last_fullsized': '2023-07-14T11:56:10.679541+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-14T11:56:16.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2023-07-14T11:56:16.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.688004+0000', 'last_change': '2023-07-14T11:56:10.688004+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.688004+0000', 'last_undegraded': '2023-07-14T11:56:10.688004+0000', 'last_fullsized': '2023-07-14T11:56:10.688004+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-14T11:56:16.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2023-07-14T11:56:16.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.683798+0000', 'last_change': '2023-07-14T11:56:10.683798+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.683798+0000', 'last_undegraded': '2023-07-14T11:56:10.683798+0000', 'last_fullsized': '2023-07-14T11:56:10.683798+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-14T11:56:16.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2023-07-14T11:56:16.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.677302+0000', 'last_change': '2023-07-14T11:56:10.677302+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.677302+0000', 'last_undegraded': '2023-07-14T11:56:10.677302+0000', 'last_fullsized': '2023-07-14T11:56:10.677302+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-14T11:56:16.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3a is not active+clean 2023-07-14T11:56:16.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.685591+0000', 'last_change': '2023-07-14T11:56:10.685591+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.685591+0000', 'last_undegraded': '2023-07-14T11:56:10.685591+0000', 'last_fullsized': '2023-07-14T11:56:10.685591+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-14T11:56:16.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2023-07-14T11:56:16.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.677725+0000', 'last_change': '2023-07-14T11:56:10.677725+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.677725+0000', 'last_undegraded': '2023-07-14T11:56:10.677725+0000', 'last_fullsized': '2023-07-14T11:56:10.677725+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-14T11:56:16.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2023-07-14T11:56:16.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.683020+0000', 'last_change': '2023-07-14T11:56:10.683020+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.683020+0000', 'last_undegraded': '2023-07-14T11:56:10.683020+0000', 'last_fullsized': '2023-07-14T11:56:10.683020+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-14T11:56:16.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2023-07-14T11:56:16.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.689105+0000', 'last_change': '2023-07-14T11:56:10.689105+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.689105+0000', 'last_undegraded': '2023-07-14T11:56:10.689105+0000', 'last_fullsized': '2023-07-14T11:56:10.689105+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-14T11:56:16.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2023-07-14T11:56:16.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.689300+0000', 'last_change': '2023-07-14T11:56:10.689300+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.689300+0000', 'last_undegraded': '2023-07-14T11:56:10.689300+0000', 'last_fullsized': '2023-07-14T11:56:10.689300+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-14T11:56:16.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2023-07-14T11:56:16.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.687590+0000', 'last_change': '2023-07-14T11:56:10.687590+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.687590+0000', 'last_undegraded': '2023-07-14T11:56:10.687590+0000', 'last_fullsized': '2023-07-14T11:56:10.687590+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-14T11:56:16.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2023-07-14T11:56:16.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.684482+0000', 'last_change': '2023-07-14T11:56:10.684482+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.684482+0000', 'last_undegraded': '2023-07-14T11:56:10.684482+0000', 'last_fullsized': '2023-07-14T11:56:10.684482+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-14T11:56:16.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2023-07-14T11:56:16.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.685262+0000', 'last_change': '2023-07-14T11:56:10.685262+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.685262+0000', 'last_undegraded': '2023-07-14T11:56:10.685262+0000', 'last_fullsized': '2023-07-14T11:56:10.685262+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-14T11:56:16.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2023-07-14T11:56:16.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.687474+0000', 'last_change': '2023-07-14T11:56:10.687474+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.687474+0000', 'last_undegraded': '2023-07-14T11:56:10.687474+0000', 'last_fullsized': '2023-07-14T11:56:10.687474+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-14T11:56:16.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2023-07-14T11:56:16.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.691056+0000', 'last_change': '2023-07-14T11:56:10.691056+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.691056+0000', 'last_undegraded': '2023-07-14T11:56:10.691056+0000', 'last_fullsized': '2023-07-14T11:56:10.691056+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-14T11:56:16.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.30 is not active+clean 2023-07-14T11:56:16.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.678388+0000', 'last_change': '2023-07-14T11:56:10.678388+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.678388+0000', 'last_undegraded': '2023-07-14T11:56:10.678388+0000', 'last_fullsized': '2023-07-14T11:56:10.678388+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-14T11:56:16.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2023-07-14T11:56:16.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.682628+0000', 'last_change': '2023-07-14T11:56:10.682628+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.682628+0000', 'last_undegraded': '2023-07-14T11:56:10.682628+0000', 'last_fullsized': '2023-07-14T11:56:10.682628+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-14T11:56:16.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2023-07-14T11:56:16.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.685212+0000', 'last_change': '2023-07-14T11:56:10.685212+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.685212+0000', 'last_undegraded': '2023-07-14T11:56:10.685212+0000', 'last_fullsized': '2023-07-14T11:56:10.685212+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-14T11:56:16.914 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2023-07-14T11:56:16.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.679247+0000', 'last_change': '2023-07-14T11:56:10.679247+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.679247+0000', 'last_undegraded': '2023-07-14T11:56:10.679247+0000', 'last_fullsized': '2023-07-14T11:56:10.679247+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-14T11:56:16.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2c is not active+clean 2023-07-14T11:56:16.915 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.682339+0000', 'last_change': '2023-07-14T11:56:10.682339+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.682339+0000', 'last_undegraded': '2023-07-14T11:56:10.682339+0000', 'last_fullsized': '2023-07-14T11:56:10.682339+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-14T11:56:16.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2023-07-14T11:56:16.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.684054+0000', 'last_change': '2023-07-14T11:56:10.684054+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.684054+0000', 'last_undegraded': '2023-07-14T11:56:10.684054+0000', 'last_fullsized': '2023-07-14T11:56:10.684054+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-14T11:56:16.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2a is not active+clean 2023-07-14T11:56:16.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.680085+0000', 'last_change': '2023-07-14T11:56:10.680085+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.680085+0000', 'last_undegraded': '2023-07-14T11:56:10.680085+0000', 'last_fullsized': '2023-07-14T11:56:10.680085+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-14T11:56:16.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.29 is not active+clean 2023-07-14T11:56:16.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.683502+0000', 'last_change': '2023-07-14T11:56:10.683502+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.683502+0000', 'last_undegraded': '2023-07-14T11:56:10.683502+0000', 'last_fullsized': '2023-07-14T11:56:10.683502+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-14T11:56:16.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2023-07-14T11:56:16.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.686241+0000', 'last_change': '2023-07-14T11:56:10.686241+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.686241+0000', 'last_undegraded': '2023-07-14T11:56:10.686241+0000', 'last_fullsized': '2023-07-14T11:56:10.686241+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-14T11:56:16.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2023-07-14T11:56:16.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.686556+0000', 'last_change': '2023-07-14T11:56:10.686556+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.686556+0000', 'last_undegraded': '2023-07-14T11:56:10.686556+0000', 'last_fullsized': '2023-07-14T11:56:10.686556+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-14T11:56:16.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2023-07-14T11:56:16.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.686616+0000', 'last_change': '2023-07-14T11:56:10.686616+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.686616+0000', 'last_undegraded': '2023-07-14T11:56:10.686616+0000', 'last_fullsized': '2023-07-14T11:56:10.686616+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-14T11:56:16.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2023-07-14T11:56:16.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.687141+0000', 'last_change': '2023-07-14T11:56:10.687141+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.687141+0000', 'last_undegraded': '2023-07-14T11:56:10.687141+0000', 'last_fullsized': '2023-07-14T11:56:10.687141+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-14T11:56:16.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2023-07-14T11:56:16.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.681576+0000', 'last_change': '2023-07-14T11:56:10.681576+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.681576+0000', 'last_undegraded': '2023-07-14T11:56:10.681576+0000', 'last_fullsized': '2023-07-14T11:56:10.681576+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-14T11:56:16.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2023-07-14T11:56:16.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.690699+0000', 'last_change': '2023-07-14T11:56:10.690699+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.690699+0000', 'last_undegraded': '2023-07-14T11:56:10.690699+0000', 'last_fullsized': '2023-07-14T11:56:10.690699+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-14T11:56:16.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2023-07-14T11:56:16.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.683599+0000', 'last_change': '2023-07-14T11:56:10.683599+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.683599+0000', 'last_undegraded': '2023-07-14T11:56:10.683599+0000', 'last_fullsized': '2023-07-14T11:56:10.683599+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-14T11:56:16.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2023-07-14T11:56:16.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.683345+0000', 'last_change': '2023-07-14T11:56:10.683345+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.683345+0000', 'last_undegraded': '2023-07-14T11:56:10.683345+0000', 'last_fullsized': '2023-07-14T11:56:10.683345+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-14T11:56:16.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.20 is not active+clean 2023-07-14T11:56:16.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.687533+0000', 'last_change': '2023-07-14T11:56:10.687533+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.687533+0000', 'last_undegraded': '2023-07-14T11:56:10.687533+0000', 'last_fullsized': '2023-07-14T11:56:10.687533+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-14T11:56:16.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2023-07-14T11:56:16.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.676335+0000', 'last_change': '2023-07-14T11:56:10.676335+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.676335+0000', 'last_undegraded': '2023-07-14T11:56:10.676335+0000', 'last_fullsized': '2023-07-14T11:56:10.676335+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-14T11:56:16.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2023-07-14T11:56:16.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.682407+0000', 'last_change': '2023-07-14T11:56:10.682407+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.682407+0000', 'last_undegraded': '2023-07-14T11:56:10.682407+0000', 'last_fullsized': '2023-07-14T11:56:10.682407+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-14T11:56:16.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2023-07-14T11:56:16.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.686577+0000', 'last_change': '2023-07-14T11:56:10.686577+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.686577+0000', 'last_undegraded': '2023-07-14T11:56:10.686577+0000', 'last_fullsized': '2023-07-14T11:56:10.686577+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-14T11:56:16.927 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2023-07-14T11:56:16.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.691008+0000', 'last_change': '2023-07-14T11:56:10.691008+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.691008+0000', 'last_undegraded': '2023-07-14T11:56:10.691008+0000', 'last_fullsized': '2023-07-14T11:56:10.691008+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-14T11:56:16.928 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2023-07-14T11:56:16.928 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.677447+0000', 'last_change': '2023-07-14T11:56:10.677447+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.677447+0000', 'last_undegraded': '2023-07-14T11:56:10.677447+0000', 'last_fullsized': '2023-07-14T11:56:10.677447+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-14T11:56:16.928 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-07-14T11:56:16.929 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.677746+0000', 'last_change': '2023-07-14T11:56:10.677746+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.677746+0000', 'last_undegraded': '2023-07-14T11:56:10.677746+0000', 'last_fullsized': '2023-07-14T11:56:10.677746+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-14T11:56:16.929 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2023-07-14T11:56:16.929 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.679929+0000', 'last_change': '2023-07-14T11:56:10.679929+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.679929+0000', 'last_undegraded': '2023-07-14T11:56:10.679929+0000', 'last_fullsized': '2023-07-14T11:56:10.679929+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-14T11:56:16.930 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2023-07-14T11:56:16.930 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.682930+0000', 'last_change': '2023-07-14T11:56:10.682930+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.682930+0000', 'last_undegraded': '2023-07-14T11:56:10.682930+0000', 'last_fullsized': '2023-07-14T11:56:10.682930+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-14T11:56:16.931 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-07-14T11:56:16.931 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.681039+0000', 'last_change': '2023-07-14T11:56:10.681039+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.681039+0000', 'last_undegraded': '2023-07-14T11:56:10.681039+0000', 'last_fullsized': '2023-07-14T11:56:10.681039+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-14T11:56:16.931 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2023-07-14T11:56:16.932 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.680746+0000', 'last_change': '2023-07-14T11:56:10.680746+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.680746+0000', 'last_undegraded': '2023-07-14T11:56:10.680746+0000', 'last_fullsized': '2023-07-14T11:56:10.680746+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-14T11:56:16.932 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2023-07-14T11:56:16.932 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.677241+0000', 'last_change': '2023-07-14T11:56:10.677241+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.677241+0000', 'last_undegraded': '2023-07-14T11:56:10.677241+0000', 'last_fullsized': '2023-07-14T11:56:10.677241+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-14T11:56:16.933 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2023-07-14T11:56:16.933 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.682757+0000', 'last_change': '2023-07-14T11:56:10.682757+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.682757+0000', 'last_undegraded': '2023-07-14T11:56:10.682757+0000', 'last_fullsized': '2023-07-14T11:56:10.682757+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-14T11:56:16.933 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2023-07-14T11:56:16.933 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.682498+0000', 'last_change': '2023-07-14T11:56:10.682498+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.682498+0000', 'last_undegraded': '2023-07-14T11:56:10.682498+0000', 'last_fullsized': '2023-07-14T11:56:10.682498+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-14T11:56:16.934 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-07-14T11:56:16.934 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.683497+0000', 'last_change': '2023-07-14T11:56:10.683497+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.683497+0000', 'last_undegraded': '2023-07-14T11:56:10.683497+0000', 'last_fullsized': '2023-07-14T11:56:10.683497+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-14T11:56:16.934 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2023-07-14T11:56:16.935 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.679469+0000', 'last_change': '2023-07-14T11:56:10.679469+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.679469+0000', 'last_undegraded': '2023-07-14T11:56:10.679469+0000', 'last_fullsized': '2023-07-14T11:56:10.679469+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-14T11:56:16.935 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-07-14T11:56:16.935 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.677560+0000', 'last_change': '2023-07-14T11:56:10.677560+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.677560+0000', 'last_undegraded': '2023-07-14T11:56:10.677560+0000', 'last_fullsized': '2023-07-14T11:56:10.677560+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-14T11:56:16.936 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2023-07-14T11:56:16.936 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.688739+0000', 'last_change': '2023-07-14T11:56:10.688739+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.688739+0000', 'last_undegraded': '2023-07-14T11:56:10.688739+0000', 'last_fullsized': '2023-07-14T11:56:10.688739+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-14T11:56:16.936 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2023-07-14T11:56:16.937 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.678287+0000', 'last_change': '2023-07-14T11:56:10.678287+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.678287+0000', 'last_undegraded': '2023-07-14T11:56:10.678287+0000', 'last_fullsized': '2023-07-14T11:56:10.678287+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-14T11:56:16.937 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-07-14T11:56:16.937 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.686132+0000', 'last_change': '2023-07-14T11:56:10.686132+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.686132+0000', 'last_undegraded': '2023-07-14T11:56:10.686132+0000', 'last_fullsized': '2023-07-14T11:56:10.686132+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-14T11:56:16.938 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2023-07-14T11:56:16.938 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.677574+0000', 'last_change': '2023-07-14T11:56:10.677574+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.677574+0000', 'last_undegraded': '2023-07-14T11:56:10.677574+0000', 'last_fullsized': '2023-07-14T11:56:10.677574+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-14T11:56:16.938 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2023-07-14T11:56:16.939 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.676708+0000', 'last_change': '2023-07-14T11:56:10.676708+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.676708+0000', 'last_undegraded': '2023-07-14T11:56:10.676708+0000', 'last_fullsized': '2023-07-14T11:56:10.676708+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-14T11:56:16.939 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2023-07-14T11:56:16.940 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.682860+0000', 'last_change': '2023-07-14T11:56:10.682860+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.682860+0000', 'last_undegraded': '2023-07-14T11:56:10.682860+0000', 'last_fullsized': '2023-07-14T11:56:10.682860+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-14T11:56:16.940 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-07-14T11:56:16.940 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.677561+0000', 'last_change': '2023-07-14T11:56:10.677561+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.677561+0000', 'last_undegraded': '2023-07-14T11:56:10.677561+0000', 'last_fullsized': '2023-07-14T11:56:10.677561+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-14T11:56:16.941 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2023-07-14T11:56:16.941 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.678966+0000', 'last_change': '2023-07-14T11:56:10.678966+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.678966+0000', 'last_undegraded': '2023-07-14T11:56:10.678966+0000', 'last_fullsized': '2023-07-14T11:56:10.678966+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-14T11:56:16.941 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2023-07-14T11:56:16.942 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.676440+0000', 'last_change': '2023-07-14T11:56:10.676440+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.676440+0000', 'last_undegraded': '2023-07-14T11:56:10.676440+0000', 'last_fullsized': '2023-07-14T11:56:10.676440+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-14T11:56:16.942 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2023-07-14T11:56:16.942 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.683131+0000', 'last_change': '2023-07-14T11:56:10.683131+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.683131+0000', 'last_undegraded': '2023-07-14T11:56:10.683131+0000', 'last_fullsized': '2023-07-14T11:56:10.683131+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-14T11:56:16.943 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2023-07-14T11:56:16.943 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.676479+0000', 'last_change': '2023-07-14T11:56:10.676479+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.676479+0000', 'last_undegraded': '2023-07-14T11:56:10.676479+0000', 'last_fullsized': '2023-07-14T11:56:10.676479+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-14T11:56:16.943 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2023-07-14T11:56:16.944 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.687524+0000', 'last_change': '2023-07-14T11:56:10.687524+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.687524+0000', 'last_undegraded': '2023-07-14T11:56:10.687524+0000', 'last_fullsized': '2023-07-14T11:56:10.687524+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-14T11:56:16.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2023-07-14T11:56:16.944 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.677097+0000', 'last_change': '2023-07-14T11:56:10.677097+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.677097+0000', 'last_undegraded': '2023-07-14T11:56:10.677097+0000', 'last_fullsized': '2023-07-14T11:56:10.677097+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-14T11:56:16.945 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-07-14T11:56:16.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.682908+0000', 'last_change': '2023-07-14T11:56:10.682908+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.682908+0000', 'last_undegraded': '2023-07-14T11:56:10.682908+0000', 'last_fullsized': '2023-07-14T11:56:10.682908+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-14T11:56:16.945 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2023-07-14T11:56:16.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.687092+0000', 'last_change': '2023-07-14T11:56:10.687092+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.687092+0000', 'last_undegraded': '2023-07-14T11:56:10.687092+0000', 'last_fullsized': '2023-07-14T11:56:10.687092+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-14T11:56:16.946 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2023-07-14T11:56:16.946 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2023-07-14T11:56:10.676332+0000', 'last_change': '2023-07-14T11:56:10.676332+0000', 'last_active': '2023-07-14T11:56:10.656698+0000', 'last_peered': '2023-07-14T11:56:10.656698+0000', 'last_clean': '2023-07-14T11:56:10.656698+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-14T11:56:10.676332+0000', 'last_undegraded': '2023-07-14T11:56:10.676332+0000', 'last_fullsized': '2023-07-14T11:56:10.676332+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'last_clean_scrub_stamp': '2023-07-14T11:56:10.656698+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-14T11:56:16.947 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-07-14T11:56:17.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.129+0000 7fbf5359e700 1 Processor -- start 2023-07-14T11:56:17.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.129+0000 7fbf5359e700 1 -- start start 2023-07-14T11:56:17.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.129+0000 7fbf5359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf540d0860 0x7fbf540b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:17.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.129+0000 7fbf5359e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf54145d90 con 0x7fbf541459c0 2023-07-14T11:56:17.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.129+0000 7fbf5359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf540cd650 con 0x7fbf540ce6f0 2023-07-14T11:56:17.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.129+0000 7fbf5359e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf540ceb50 con 0x7fbf540d0860 2023-07-14T11:56:17.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.129+0000 7fbf5259c700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fbf541459c0 0x7fbf540ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:47946/0 (socket says 172.21.15.104:47946) 2023-07-14T11:56:17.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.129+0000 7fbf5259c700 1 -- 172.21.15.104:0/872778915 learned_addr learned my addr 172.21.15.104:0/872778915 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:17.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.129+0000 7fbf52d9d700 1 --2- 172.21.15.104:0/872778915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf540d0860 0x7fbf540b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:17.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076674625 0 0) 0x7fbf54145d90 con 0x7fbf541459c0 2023-07-14T11:56:17.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf400035d0 con 0x7fbf541459c0 2023-07-14T11:56:17.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831473284 0 0) 0x7fbf540cd650 con 0x7fbf540ce6f0 2023-07-14T11:56:17.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf54145d90 con 0x7fbf540ce6f0 2023-07-14T11:56:17.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 230631295 0 0) 0x7fbf400035d0 con 0x7fbf541459c0 2023-07-14T11:56:17.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf540d0860 msgr2=0x7fbf540b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:17.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 --2- 172.21.15.104:0/872778915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf540d0860 0x7fbf540b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 >> v1:172.21.15.121:6789/0 conn(0x7fbf540ce6f0 legacy=0x7fbf540d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:17.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf540cd930 con 0x7fbf541459c0 2023-07-14T11:56:17.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7fbf44001b70 con 0x7fbf541459c0 2023-07-14T11:56:17.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 >> v1:172.21.15.104:6789/0 conn(0x7fbf541459c0 legacy=0x7fbf540ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:17.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 --2- 172.21.15.104:0/872778915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf40003be0 0x7fbf40003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:17.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 --2- 172.21.15.104:0/872778915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf40004870 0x7fbf40004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:17.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 --2- 172.21.15.104:0/872778915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf40005220 0x7fbf4000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:17.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf540ceb50 con 0x7fbf40003be0 2023-07-14T11:56:17.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4000bff0 con 0x7fbf40004870 2023-07-14T11:56:17.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4000c1b0 con 0x7fbf40005220 2023-07-14T11:56:17.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf5259c700 1 --2- 172.21.15.104:0/872778915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf40003be0 0x7fbf40003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:17.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf49d9b700 1 --2- 172.21.15.104:0/872778915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf40004870 0x7fbf40004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:17.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf5259c700 1 -- 172.21.15.104:0/872778915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf40005220 msgr2=0x7fbf4000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:17.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf52d9d700 1 --2- 172.21.15.104:0/872778915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf40005220 0x7fbf4000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:17.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf5259c700 1 --2- 172.21.15.104:0/872778915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf40005220 0x7fbf4000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf5259c700 1 -- 172.21.15.104:0/872778915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf40004870 msgr2=0x7fbf40004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:17.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf5259c700 1 --2- 172.21.15.104:0/872778915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf40004870 0x7fbf40004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf5259c700 1 -- 172.21.15.104:0/872778915 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf400035d0 con 0x7fbf40003be0 2023-07-14T11:56:17.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf49d9b700 1 --2- 172.21.15.104:0/872778915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf40004870 0x7fbf40004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:17.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf52d9d700 1 --2- 172.21.15.104:0/872778915 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf40005220 0x7fbf4000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:17.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf5259c700 1 --2- 172.21.15.104:0/872778915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf40003be0 0x7fbf40003fd0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400a900 tx=0x7fbf400058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16437d377895f108 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:17.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44005dd0 con 0x7fbf40003be0 2023-07-14T11:56:17.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf44001ff0 con 0x7fbf40003be0 2023-07-14T11:56:17.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.133+0000 7fbf51d9b700 1 -- 172.21.15.104:0/872778915 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf440072b0 con 0x7fbf40003be0 2023-07-14T11:56:17.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.137+0000 7fbf5359e700 1 -- 172.21.15.104:0/872778915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf40003be0 msgr2=0x7fbf40003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:17.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.137+0000 7fbf5359e700 1 --2- 172.21.15.104:0/872778915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf40003be0 0x7fbf40003fd0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400a900 tx=0x7fbf400058a0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.137+0000 7fbf52d9d700 1 -- 172.21.15.104:0/872778915 reap_dead start 2023-07-14T11:56:17.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.137+0000 7fbf5359e700 1 -- 172.21.15.104:0/872778915 shutdown_connections 2023-07-14T11:56:17.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.137+0000 7fbf5359e700 1 -- 172.21.15.104:0/872778915 >> 172.21.15.104:0/872778915 conn(0x7fbf54060c40 msgr2=0x7fbf5409b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:17.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.137+0000 7fbf5359e700 1 -- 172.21.15.104:0/872778915 shutdown_connections 2023-07-14T11:56:17.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.137+0000 7fbf5359e700 1 -- 172.21.15.104:0/872778915 wait complete. 2023-07-14T11:56:17.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5359e700 1 Processor -- start 2023-07-14T11:56:17.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5359e700 1 -- start start 2023-07-14T11:56:17.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5359e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf40003be0 0x7fbf54158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:17.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf540c7bb0 0x7fbf540c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:17.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf5415ebd0 0x7fbf5415f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:17.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5359e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf540c8c40 con 0x7fbf40003be0 2023-07-14T11:56:17.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf540b66f0 con 0x7fbf5415ebd0 2023-07-14T11:56:17.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5359e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf540b7e30 con 0x7fbf540c7bb0 2023-07-14T11:56:17.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf40003be0 0x7fbf54158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:17.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf40003be0 0x7fbf54158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50460/0 (socket says 172.21.15.104:50460) 2023-07-14T11:56:17.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5259c700 1 -- 172.21.15.104:0/3640268269 learned_addr learned my addr 172.21.15.104:0/3640268269 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:17.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf52d9d700 1 --2- 172.21.15.104:0/3640268269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf5415ebd0 0x7fbf5415f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:17.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf49d9b700 1 --2- 172.21.15.104:0/3640268269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf540c7bb0 0x7fbf540c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:17.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5259c700 1 -- 172.21.15.104:0/3640268269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf540c7bb0 msgr2=0x7fbf540c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:17.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5259c700 1 --2- 172.21.15.104:0/3640268269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf540c7bb0 0x7fbf540c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5259c700 1 -- 172.21.15.104:0/3640268269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf5415ebd0 msgr2=0x7fbf5415f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:17.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5259c700 1 --2- 172.21.15.104:0/3640268269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf5415ebd0 0x7fbf5415f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5259c700 1 -- 172.21.15.104:0/3640268269 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf540c8680 con 0x7fbf40003be0 2023-07-14T11:56:17.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf49d9b700 1 --2- 172.21.15.104:0/3640268269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf540c7bb0 0x7fbf540c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:17.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5259c700 1 --2- 172.21.15.104:0/3640268269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf40003be0 0x7fbf54158040 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000bfc0 tx=0x7fbf44001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:17.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf4b7fe700 1 -- 172.21.15.104:0/3640268269 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44007500 con 0x7fbf40003be0 2023-07-14T11:56:17.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf4b7fe700 1 -- 172.21.15.104:0/3640268269 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf44007660 con 0x7fbf40003be0 2023-07-14T11:56:17.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5359e700 1 -- 172.21.15.104:0/3640268269 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf540c8960 con 0x7fbf40003be0 2023-07-14T11:56:17.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf4b7fe700 1 -- 172.21.15.104:0/3640268269 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44007a10 con 0x7fbf40003be0 2023-07-14T11:56:17.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.141+0000 7fbf5359e700 1 -- 172.21.15.104:0/3640268269 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5415f580 con 0x7fbf40003be0 2023-07-14T11:56:17.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.145+0000 7fbf5359e700 1 -- 172.21.15.104:0/3640268269 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf540b61e0 con 0x7fbf40003be0 2023-07-14T11:56:17.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.145+0000 7fbf4b7fe700 1 -- 172.21.15.104:0/3640268269 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fbf4401e430 con 0x7fbf40003be0 2023-07-14T11:56:17.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.145+0000 7fbf4b7fe700 1 --2- 172.21.15.104:0/3640268269 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbf340662f0 0x7fbf340687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:17.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.145+0000 7fbf4b7fe700 1 -- 172.21.15.104:0/3640268269 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbf44085970 con 0x7fbf40003be0 2023-07-14T11:56:17.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.149+0000 7fbf49d9b700 1 --2- 172.21.15.104:0/3640268269 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbf340662f0 0x7fbf340687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:17.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.149+0000 7fbf49d9b700 1 --2- 172.21.15.104:0/3640268269 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbf340662f0 0x7fbf340687a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbf540b5f60 tx=0x7fbf3c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:17.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.149+0000 7fbf4b7fe700 1 -- 172.21.15.104:0/3640268269 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fbf540b61e0 con 0x7fbf40003be0 2023-07-14T11:56:17.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.389+0000 7fbf5359e700 1 -- 172.21.15.104:0/3640268269 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fbf540b61e0 con 0x7fbf40003be0 2023-07-14T11:56:17.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.389+0000 7fbf4b7fe700 1 -- 172.21.15.104:0/3640268269 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1195 (secure 0 0 0) 0x7fbf540b61e0 con 0x7fbf40003be0 2023-07-14T11:56:17.378 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:17.378 INFO:teuthology.orchestra.run.smithi104.stdout:{"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":209,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":59,"num_osds":8,"num_up_osds":8,"osd_up_since":1689335580,"num_in_osds":8,"osd_in_since":1689335570,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":235802624,"bytes_avail":772858310656,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":27,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":6312}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2023-07-14T11:55:53.941870+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-07-14T11:56:17.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.393+0000 7fbf5359e700 1 -- 172.21.15.104:0/3640268269 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbf340662f0 msgr2=0x7fbf340687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:17.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.393+0000 7fbf5359e700 1 --2- 172.21.15.104:0/3640268269 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbf340662f0 0x7fbf340687a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbf540b5f60 tx=0x7fbf3c009040 comp rx=0 tx=0).stop 2023-07-14T11:56:17.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.393+0000 7fbf5359e700 1 -- 172.21.15.104:0/3640268269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf40003be0 msgr2=0x7fbf54158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:17.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.393+0000 7fbf5359e700 1 --2- 172.21.15.104:0/3640268269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf40003be0 0x7fbf54158040 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000bfc0 tx=0x7fbf44001e80 comp rx=0 tx=0).stop 2023-07-14T11:56:17.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.393+0000 7fbf5359e700 1 -- 172.21.15.104:0/3640268269 shutdown_connections 2023-07-14T11:56:17.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.393+0000 7fbf5359e700 1 --2- 172.21.15.104:0/3640268269 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbf340662f0 0x7fbf340687a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.393+0000 7fbf5359e700 1 --2- 172.21.15.104:0/3640268269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf5415ebd0 0x7fbf5415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.393+0000 7fbf5359e700 1 --2- 172.21.15.104:0/3640268269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf540c7bb0 0x7fbf540c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.393+0000 7fbf5359e700 1 --2- 172.21.15.104:0/3640268269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf40003be0 0x7fbf54158040 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.393+0000 7fbf5359e700 1 -- 172.21.15.104:0/3640268269 >> 172.21.15.104:0/3640268269 conn(0x7fbf54060c40 msgr2=0x7fbf540ccdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:17.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.393+0000 7fbf5359e700 1 -- 172.21.15.104:0/3640268269 shutdown_connections 2023-07-14T11:56:17.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.393+0000 7fbf5359e700 1 -- 172.21.15.104:0/3640268269 wait complete. 2023-07-14T11:56:17.401 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-07-14T11:56:17.402 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:56:17.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.585+0000 7f247a640700 1 Processor -- start 2023-07-14T11:56:17.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.585+0000 7f247a640700 1 -- start start 2023-07-14T11:56:17.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f247a640700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f24740cd240 0x7f24740ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:17.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f247a640700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f24740ceff0 0x7f24740afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:17.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f247a640700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24740af0a0 con 0x7f247415bff0 2023-07-14T11:56:17.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f247a640700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24740cf720 con 0x7f24740cd240 2023-07-14T11:56:17.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f247a640700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24740af2f0 con 0x7f24740ceff0 2023-07-14T11:56:17.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f2478e3d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f24740cd240 0x7f24740ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:17.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f2478e3d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f24740cd240 0x7f24740ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50464/0 (socket says 172.21.15.104:50464) 2023-07-14T11:56:17.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f2478e3d700 1 -- 172.21.15.104:0/1721329696 learned_addr learned my addr 172.21.15.104:0/1721329696 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:17.576 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f2479e3f700 1 --2- 172.21.15.104:0/1721329696 >> v2:172.21.15.121:3300/0 conn(0x7f24740ceff0 0x7f24740afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:17.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f2478e3d700 1 -- 172.21.15.104:0/1721329696 >> v2:172.21.15.121:3300/0 conn(0x7f24740ceff0 msgr2=0x7f24740afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:17.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f2478e3d700 1 --2- 172.21.15.104:0/1721329696 >> v2:172.21.15.121:3300/0 conn(0x7f24740ceff0 0x7f24740afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f2478e3d700 1 -- 172.21.15.104:0/1721329696 >> v1:172.21.15.121:6789/0 conn(0x7f247415bff0 legacy=0x7f24740ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:17.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f2478e3d700 1 -- 172.21.15.104:0/1721329696 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24740af5d0 con 0x7f24740cd240 2023-07-14T11:56:17.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f2478e3d700 1 --2- 172.21.15.104:0/1721329696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24740cd240 0x7f24740ceab0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f247408b700 tx=0x7f2464009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61c30f6f1384a92 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:17.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f246bfff700 1 -- 172.21.15.104:0/1721329696 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2464011830 con 0x7f24740cd240 2023-07-14T11:56:17.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f246bfff700 1 -- 172.21.15.104:0/1721329696 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2464011990 con 0x7f24740cd240 2023-07-14T11:56:17.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f246bfff700 1 -- 172.21.15.104:0/1721329696 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2464011d20 con 0x7f24740cd240 2023-07-14T11:56:17.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f247a640700 1 -- 172.21.15.104:0/1721329696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24740cd240 msgr2=0x7f24740ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:17.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.589+0000 7f247a640700 1 --2- 172.21.15.104:0/1721329696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24740cd240 0x7f24740ceab0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f247408b700 tx=0x7f2464009bf0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.593+0000 7f247a640700 1 -- 172.21.15.104:0/1721329696 shutdown_connections 2023-07-14T11:56:17.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.593+0000 7f247a640700 1 --2- 172.21.15.104:0/1721329696 >> v2:172.21.15.121:3300/0 conn(0x7f24740ceff0 0x7f24740afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.593+0000 7f247a640700 1 --2- 172.21.15.104:0/1721329696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24740cd240 0x7f24740ceab0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.593+0000 7f247a640700 1 -- 172.21.15.104:0/1721329696 >> 172.21.15.104:0/1721329696 conn(0x7f2474060c40 msgr2=0x7f247409b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:17.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.593+0000 7f247a640700 1 -- 172.21.15.104:0/1721329696 shutdown_connections 2023-07-14T11:56:17.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.593+0000 7f247a640700 1 -- 172.21.15.104:0/1721329696 wait complete. 2023-07-14T11:56:17.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.593+0000 7f247a640700 1 Processor -- start 2023-07-14T11:56:17.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.593+0000 7f247a640700 1 -- start start 2023-07-14T11:56:17.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f247a640700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24740cd240 0x7f247414c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:17.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f247a640700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24740ceff0 0x7f247414cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:17.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f247a640700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f247415bff0 0x7f24741493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:17.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f247a640700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24740caf20 con 0x7f24740ceff0 2023-07-14T11:56:17.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f247a640700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24740cb200 con 0x7f247415bff0 2023-07-14T11:56:17.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f247a640700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24741f0e90 con 0x7f24740cd240 2023-07-14T11:56:17.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f2478e3d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24740ceff0 0x7f247414cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:17.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f2478e3d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24740ceff0 0x7f247414cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50476/0 (socket says 172.21.15.104:50476) 2023-07-14T11:56:17.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f2478e3d700 1 -- 172.21.15.104:0/608044183 learned_addr learned my addr 172.21.15.104:0/608044183 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:17.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f2479e3f700 1 --2- 172.21.15.104:0/608044183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f247415bff0 0x7f24741493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:17.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f247963e700 1 --2- 172.21.15.104:0/608044183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24740cd240 0x7f247414c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:17.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f2478e3d700 1 -- 172.21.15.104:0/608044183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24740cd240 msgr2=0x7f247414c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:17.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f2478e3d700 1 --2- 172.21.15.104:0/608044183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24740cd240 0x7f247414c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f2478e3d700 1 -- 172.21.15.104:0/608044183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f247415bff0 msgr2=0x7f24741493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:17.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f2478e3d700 1 --2- 172.21.15.104:0/608044183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f247415bff0 0x7f24741493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f2478e3d700 1 -- 172.21.15.104:0/608044183 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2464009570 con 0x7f24740ceff0 2023-07-14T11:56:17.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f2478e3d700 1 --2- 172.21.15.104:0/608044183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24740ceff0 0x7f247414cd60 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2464000c00 tx=0x7f2464004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:17.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f2469ffb700 1 -- 172.21.15.104:0/608044183 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2464004610 con 0x7f24740ceff0 2023-07-14T11:56:17.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f2469ffb700 1 -- 172.21.15.104:0/608044183 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246401e070 con 0x7f24740ceff0 2023-07-14T11:56:17.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f247a640700 1 -- 172.21.15.104:0/608044183 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24740c9fc0 con 0x7f24740ceff0 2023-07-14T11:56:17.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f2469ffb700 1 -- 172.21.15.104:0/608044183 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24640038e0 con 0x7f24740ceff0 2023-07-14T11:56:17.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.597+0000 7f247a640700 1 -- 172.21.15.104:0/608044183 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24740ca580 con 0x7f24740ceff0 2023-07-14T11:56:17.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.601+0000 7f247a640700 1 -- 172.21.15.104:0/608044183 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24740cb060 con 0x7f24740ceff0 2023-07-14T11:56:17.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.601+0000 7f2469ffb700 1 -- 172.21.15.104:0/608044183 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2464003a40 con 0x7f24740ceff0 2023-07-14T11:56:17.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.601+0000 7f2469ffb700 1 --2- 172.21.15.104:0/608044183 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f24580662f0 0x7f24580687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:17.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.601+0000 7f2469ffb700 1 -- 172.21.15.104:0/608044183 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2464084a80 con 0x7f24740ceff0 2023-07-14T11:56:17.591 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.605+0000 7f247963e700 1 --2- 172.21.15.104:0/608044183 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f24580662f0 0x7f24580687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:17.592 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.605+0000 7f247963e700 1 --2- 172.21.15.104:0/608044183 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f24580662f0 0x7f24580687a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2470000f60 tx=0x7f2470009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:17.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.609+0000 7f2469ffb700 1 -- 172.21.15.104:0/608044183 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f24740cb060 con 0x7f24740ceff0 2023-07-14T11:56:17.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.773+0000 7f247a640700 1 -- 172.21.15.104:0/608044183 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24740d1fa0 con 0x7f24580662f0 2023-07-14T11:56:17.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.785+0000 7f2469ffb700 1 -- 172.21.15.104:0/608044183 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303908 (secure 0 0 0) 0x7f2470001340 con 0x7f24580662f0 2023-07-14T11:56:17.775 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:17.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.793+0000 7f247a640700 1 -- 172.21.15.104:0/608044183 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f24580662f0 msgr2=0x7f24580687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:17.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.793+0000 7f247a640700 1 --2- 172.21.15.104:0/608044183 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f24580662f0 0x7f24580687a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2470000f60 tx=0x7f2470009040 comp rx=0 tx=0).stop 2023-07-14T11:56:17.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.793+0000 7f247a640700 1 -- 172.21.15.104:0/608044183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24740ceff0 msgr2=0x7f247414cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:17.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.793+0000 7f247a640700 1 --2- 172.21.15.104:0/608044183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24740ceff0 0x7f247414cd60 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2464000c00 tx=0x7f2464004770 comp rx=0 tx=0).stop 2023-07-14T11:56:17.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.793+0000 7f247a640700 1 -- 172.21.15.104:0/608044183 shutdown_connections 2023-07-14T11:56:17.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.793+0000 7f247a640700 1 --2- 172.21.15.104:0/608044183 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f24580662f0 0x7f24580687a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.793+0000 7f247a640700 1 --2- 172.21.15.104:0/608044183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24740ceff0 0x7f247414cd60 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.793+0000 7f247a640700 1 --2- 172.21.15.104:0/608044183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f247415bff0 0x7f24741493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.793+0000 7f247a640700 1 --2- 172.21.15.104:0/608044183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24740cd240 0x7f247414c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:17.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.793+0000 7f247a640700 1 -- 172.21.15.104:0/608044183 >> 172.21.15.104:0/608044183 conn(0x7f2474060c40 msgr2=0x7f24740b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:17.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.793+0000 7f247a640700 1 -- 172.21.15.104:0/608044183 shutdown_connections 2023-07-14T11:56:17.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:17.793+0000 7f247a640700 1 -- 172.21.15.104:0/608044183 wait complete. 2023-07-14T11:56:17.786 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:56:17.805 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2023-07-14T11:56:15.962209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230276,"kb_used_data":6468,"kb_used_omap":0,"kb_used_meta":223744,"kb_avail":754744444,"statfs":{"total":773094113280,"available":772858310656,"internally_reserved":0,"allocated":6623232,"data_stored":3700596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":229113856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.015474"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683147+0000","last_change":"2023-07-14T11:56:10.683147+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683147+0000","last_undegraded":"2023-07-14T11:56:10.683147+0000","last_fullsized":"2023-07-14T11:56:10.683147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679541+0000","last_change":"2023-07-14T11:56:10.679541+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679541+0000","last_undegraded":"2023-07-14T11:56:10.679541+0000","last_fullsized":"2023-07-14T11:56:10.679541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.688004+0000","last_change":"2023-07-14T11:56:10.688004+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.688004+0000","last_undegraded":"2023-07-14T11:56:10.688004+0000","last_fullsized":"2023-07-14T11:56:10.688004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683798+0000","last_change":"2023-07-14T11:56:10.683798+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683798+0000","last_undegraded":"2023-07-14T11:56:10.683798+0000","last_fullsized":"2023-07-14T11:56:10.683798+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677302+0000","last_change":"2023-07-14T11:56:10.677302+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677302+0000","last_undegraded":"2023-07-14T11:56:10.677302+0000","last_fullsized":"2023-07-14T11:56:10.677302+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.685591+0000","last_change":"2023-07-14T11:56:10.685591+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.685591+0000","last_undegraded":"2023-07-14T11:56:10.685591+0000","last_fullsized":"2023-07-14T11:56:10.685591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677725+0000","last_change":"2023-07-14T11:56:10.677725+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677725+0000","last_undegraded":"2023-07-14T11:56:10.677725+0000","last_fullsized":"2023-07-14T11:56:10.677725+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683020+0000","last_change":"2023-07-14T11:56:10.683020+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683020+0000","last_undegraded":"2023-07-14T11:56:10.683020+0000","last_fullsized":"2023-07-14T11:56:10.683020+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.689105+0000","last_change":"2023-07-14T11:56:10.689105+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.689105+0000","last_undegraded":"2023-07-14T11:56:10.689105+0000","last_fullsized":"2023-07-14T11:56:10.689105+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.689300+0000","last_change":"2023-07-14T11:56:10.689300+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.689300+0000","last_undegraded":"2023-07-14T11:56:10.689300+0000","last_fullsized":"2023-07-14T11:56:10.689300+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687590+0000","last_change":"2023-07-14T11:56:10.687590+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687590+0000","last_undegraded":"2023-07-14T11:56:10.687590+0000","last_fullsized":"2023-07-14T11:56:10.687590+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.684482+0000","last_change":"2023-07-14T11:56:10.684482+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.684482+0000","last_undegraded":"2023-07-14T11:56:10.684482+0000","last_fullsized":"2023-07-14T11:56:10.684482+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.685262+0000","last_change":"2023-07-14T11:56:10.685262+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.685262+0000","last_undegraded":"2023-07-14T11:56:10.685262+0000","last_fullsized":"2023-07-14T11:56:10.685262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687474+0000","last_change":"2023-07-14T11:56:10.687474+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687474+0000","last_undegraded":"2023-07-14T11:56:10.687474+0000","last_fullsized":"2023-07-14T11:56:10.687474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.691056+0000","last_change":"2023-07-14T11:56:10.691056+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.691056+0000","last_undegraded":"2023-07-14T11:56:10.691056+0000","last_fullsized":"2023-07-14T11:56:10.691056+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.678388+0000","last_change":"2023-07-14T11:56:10.678388+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.678388+0000","last_undegraded":"2023-07-14T11:56:10.678388+0000","last_fullsized":"2023-07-14T11:56:10.678388+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682628+0000","last_change":"2023-07-14T11:56:10.682628+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682628+0000","last_undegraded":"2023-07-14T11:56:10.682628+0000","last_fullsized":"2023-07-14T11:56:10.682628+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.685212+0000","last_change":"2023-07-14T11:56:10.685212+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.685212+0000","last_undegraded":"2023-07-14T11:56:10.685212+0000","last_fullsized":"2023-07-14T11:56:10.685212+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679247+0000","last_change":"2023-07-14T11:56:10.679247+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679247+0000","last_undegraded":"2023-07-14T11:56:10.679247+0000","last_fullsized":"2023-07-14T11:56:10.679247+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682339+0000","last_change":"2023-07-14T11:56:10.682339+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682339+0000","last_undegraded":"2023-07-14T11:56:10.682339+0000","last_fullsized":"2023-07-14T11:56:10.682339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.684054+0000","last_change":"2023-07-14T11:56:10.684054+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.684054+0000","last_undegraded":"2023-07-14T11:56:10.684054+0000","last_fullsized":"2023-07-14T11:56:10.684054+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.680085+0000","last_change":"2023-07-14T11:56:10.680085+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.680085+0000","last_undegraded":"2023-07-14T11:56:10.680085+0000","last_fullsized":"2023-07-14T11:56:10.680085+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683502+0000","last_change":"2023-07-14T11:56:10.683502+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683502+0000","last_undegraded":"2023-07-14T11:56:10.683502+0000","last_fullsized":"2023-07-14T11:56:10.683502+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686241+0000","last_change":"2023-07-14T11:56:10.686241+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686241+0000","last_undegraded":"2023-07-14T11:56:10.686241+0000","last_fullsized":"2023-07-14T11:56:10.686241+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686556+0000","last_change":"2023-07-14T11:56:10.686556+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686556+0000","last_undegraded":"2023-07-14T11:56:10.686556+0000","last_fullsized":"2023-07-14T11:56:10.686556+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686616+0000","last_change":"2023-07-14T11:56:10.686616+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686616+0000","last_undegraded":"2023-07-14T11:56:10.686616+0000","last_fullsized":"2023-07-14T11:56:10.686616+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687141+0000","last_change":"2023-07-14T11:56:10.687141+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687141+0000","last_undegraded":"2023-07-14T11:56:10.687141+0000","last_fullsized":"2023-07-14T11:56:10.687141+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.681576+0000","last_change":"2023-07-14T11:56:10.681576+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.681576+0000","last_undegraded":"2023-07-14T11:56:10.681576+0000","last_fullsized":"2023-07-14T11:56:10.681576+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.690699+0000","last_change":"2023-07-14T11:56:10.690699+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.690699+0000","last_undegraded":"2023-07-14T11:56:10.690699+0000","last_fullsized":"2023-07-14T11:56:10.690699+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683599+0000","last_change":"2023-07-14T11:56:10.683599+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683599+0000","last_undegraded":"2023-07-14T11:56:10.683599+0000","last_fullsized":"2023-07-14T11:56:10.683599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683345+0000","last_change":"2023-07-14T11:56:10.683345+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683345+0000","last_undegraded":"2023-07-14T11:56:10.683345+0000","last_fullsized":"2023-07-14T11:56:10.683345+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687533+0000","last_change":"2023-07-14T11:56:10.687533+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687533+0000","last_undegraded":"2023-07-14T11:56:10.687533+0000","last_fullsized":"2023-07-14T11:56:10.687533+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676335+0000","last_change":"2023-07-14T11:56:10.676335+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676335+0000","last_undegraded":"2023-07-14T11:56:10.676335+0000","last_fullsized":"2023-07-14T11:56:10.676335+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.702674+0000","last_change":"2023-07-14T11:56:10.696681+0000","last_active":"2023-07-14T11:56:10.702674+0000","last_peered":"2023-07-14T11:56:10.702674+0000","last_clean":"2023-07-14T11:56:10.702674+0000","last_became_active":"2023-07-14T11:56:10.696413+0000","last_became_peered":"2023-07-14T11:56:10.696413+0000","last_unstale":"2023-07-14T11:56:10.702674+0000","last_undegraded":"2023-07-14T11:56:10.702674+0000","last_fullsized":"2023-07-14T11:56:10.702674+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:34.861496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682407+0000","last_change":"2023-07-14T11:56:10.682407+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682407+0000","last_undegraded":"2023-07-14T11:56:10.682407+0000","last_fullsized":"2023-07-14T11:56:10.682407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696074+0000","last_change":"2023-07-14T11:56:10.694408+0000","last_active":"2023-07-14T11:56:10.696074+0000","last_peered":"2023-07-14T11:56:10.696074+0000","last_clean":"2023-07-14T11:56:10.696074+0000","last_became_active":"2023-07-14T11:56:10.694097+0000","last_became_peered":"2023-07-14T11:56:10.694097+0000","last_unstale":"2023-07-14T11:56:10.696074+0000","last_undegraded":"2023-07-14T11:56:10.696074+0000","last_fullsized":"2023-07-14T11:56:10.696074+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:53.935404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686577+0000","last_change":"2023-07-14T11:56:10.686577+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686577+0000","last_undegraded":"2023-07-14T11:56:10.686577+0000","last_fullsized":"2023-07-14T11:56:10.686577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.705379+0000","last_change":"2023-07-14T11:56:10.703709+0000","last_active":"2023-07-14T11:56:10.705379+0000","last_peered":"2023-07-14T11:56:10.705379+0000","last_clean":"2023-07-14T11:56:10.705379+0000","last_became_active":"2023-07-14T11:56:10.703444+0000","last_became_peered":"2023-07-14T11:56:10.703444+0000","last_unstale":"2023-07-14T11:56:10.705379+0000","last_undegraded":"2023-07-14T11:56:10.705379+0000","last_fullsized":"2023-07-14T11:56:10.705379+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:01:25.510366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.691008+0000","last_change":"2023-07-14T11:56:10.691008+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.691008+0000","last_undegraded":"2023-07-14T11:56:10.691008+0000","last_fullsized":"2023-07-14T11:56:10.691008+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703069+0000","last_change":"2023-07-14T11:56:10.701563+0000","last_active":"2023-07-14T11:56:10.703069+0000","last_peered":"2023-07-14T11:56:10.703069+0000","last_clean":"2023-07-14T11:56:10.703069+0000","last_became_active":"2023-07-14T11:56:10.701262+0000","last_became_peered":"2023-07-14T11:56:10.701262+0000","last_unstale":"2023-07-14T11:56:10.703069+0000","last_undegraded":"2023-07-14T11:56:10.703069+0000","last_fullsized":"2023-07-14T11:56:10.703069+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:32:14.962001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677447+0000","last_change":"2023-07-14T11:56:10.677447+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677447+0000","last_undegraded":"2023-07-14T11:56:10.677447+0000","last_fullsized":"2023-07-14T11:56:10.677447+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.697759+0000","last_change":"2023-07-14T11:56:10.694726+0000","last_active":"2023-07-14T11:56:10.697759+0000","last_peered":"2023-07-14T11:56:10.697759+0000","last_clean":"2023-07-14T11:56:10.697759+0000","last_became_active":"2023-07-14T11:56:10.694444+0000","last_became_peered":"2023-07-14T11:56:10.694444+0000","last_unstale":"2023-07-14T11:56:10.697759+0000","last_undegraded":"2023-07-14T11:56:10.697759+0000","last_fullsized":"2023-07-14T11:56:10.697759+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:58:15.607086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677746+0000","last_change":"2023-07-14T11:56:10.677746+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677746+0000","last_undegraded":"2023-07-14T11:56:10.677746+0000","last_fullsized":"2023-07-14T11:56:10.677746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.692679+0000","last_change":"2023-07-14T11:56:10.689607+0000","last_active":"2023-07-14T11:56:10.692679+0000","last_peered":"2023-07-14T11:56:10.692679+0000","last_clean":"2023-07-14T11:56:10.692679+0000","last_became_active":"2023-07-14T11:56:10.689345+0000","last_became_peered":"2023-07-14T11:56:10.689345+0000","last_unstale":"2023-07-14T11:56:10.692679+0000","last_undegraded":"2023-07-14T11:56:10.692679+0000","last_fullsized":"2023-07-14T11:56:10.692679+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:00:10.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679929+0000","last_change":"2023-07-14T11:56:10.679929+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679929+0000","last_undegraded":"2023-07-14T11:56:10.679929+0000","last_fullsized":"2023-07-14T11:56:10.679929+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.693487+0000","last_change":"2023-07-14T11:56:10.691388+0000","last_active":"2023-07-14T11:56:10.693487+0000","last_peered":"2023-07-14T11:56:10.693487+0000","last_clean":"2023-07-14T11:56:10.693487+0000","last_became_active":"2023-07-14T11:56:10.690746+0000","last_became_peered":"2023-07-14T11:56:10.690746+0000","last_unstale":"2023-07-14T11:56:10.693487+0000","last_undegraded":"2023-07-14T11:56:10.693487+0000","last_fullsized":"2023-07-14T11:56:10.693487+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:01.327056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682930+0000","last_change":"2023-07-14T11:56:10.682930+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682930+0000","last_undegraded":"2023-07-14T11:56:10.682930+0000","last_fullsized":"2023-07-14T11:56:10.682930+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.700186+0000","last_change":"2023-07-14T11:56:10.694837+0000","last_active":"2023-07-14T11:56:10.700186+0000","last_peered":"2023-07-14T11:56:10.700186+0000","last_clean":"2023-07-14T11:56:10.700186+0000","last_became_active":"2023-07-14T11:56:10.694071+0000","last_became_peered":"2023-07-14T11:56:10.694071+0000","last_unstale":"2023-07-14T11:56:10.700186+0000","last_undegraded":"2023-07-14T11:56:10.700186+0000","last_fullsized":"2023-07-14T11:56:10.700186+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:12.411429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.681039+0000","last_change":"2023-07-14T11:56:10.681039+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.681039+0000","last_undegraded":"2023-07-14T11:56:10.681039+0000","last_fullsized":"2023-07-14T11:56:10.681039+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.708387+0000","last_change":"2023-07-14T11:56:10.705201+0000","last_active":"2023-07-14T11:56:10.708387+0000","last_peered":"2023-07-14T11:56:10.708387+0000","last_clean":"2023-07-14T11:56:10.708387+0000","last_became_active":"2023-07-14T11:56:10.704922+0000","last_became_peered":"2023-07-14T11:56:10.704922+0000","last_unstale":"2023-07-14T11:56:10.708387+0000","last_undegraded":"2023-07-14T11:56:10.708387+0000","last_fullsized":"2023-07-14T11:56:10.708387+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:32:12.442228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.680746+0000","last_change":"2023-07-14T11:56:10.680746+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.680746+0000","last_undegraded":"2023-07-14T11:56:10.680746+0000","last_fullsized":"2023-07-14T11:56:10.680746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696725+0000","last_change":"2023-07-14T11:56:10.692503+0000","last_active":"2023-07-14T11:56:10.696725+0000","last_peered":"2023-07-14T11:56:10.696725+0000","last_clean":"2023-07-14T11:56:10.696725+0000","last_became_active":"2023-07-14T11:56:10.692092+0000","last_became_peered":"2023-07-14T11:56:10.692092+0000","last_unstale":"2023-07-14T11:56:10.696725+0000","last_undegraded":"2023-07-14T11:56:10.696725+0000","last_fullsized":"2023-07-14T11:56:10.696725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:05.395217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677241+0000","last_change":"2023-07-14T11:56:10.677241+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677241+0000","last_undegraded":"2023-07-14T11:56:10.677241+0000","last_fullsized":"2023-07-14T11:56:10.677241+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707828+0000","last_change":"2023-07-14T11:56:10.703832+0000","last_active":"2023-07-14T11:56:10.707828+0000","last_peered":"2023-07-14T11:56:10.707828+0000","last_clean":"2023-07-14T11:56:10.707828+0000","last_became_active":"2023-07-14T11:56:10.703548+0000","last_became_peered":"2023-07-14T11:56:10.703548+0000","last_unstale":"2023-07-14T11:56:10.707828+0000","last_undegraded":"2023-07-14T11:56:10.707828+0000","last_fullsized":"2023-07-14T11:56:10.707828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:03:55.287678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682757+0000","last_change":"2023-07-14T11:56:10.682757+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682757+0000","last_undegraded":"2023-07-14T11:56:10.682757+0000","last_fullsized":"2023-07-14T11:56:10.682757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703549+0000","last_change":"2023-07-14T11:56:10.697093+0000","last_active":"2023-07-14T11:56:10.703549+0000","last_peered":"2023-07-14T11:56:10.703549+0000","last_clean":"2023-07-14T11:56:10.703549+0000","last_became_active":"2023-07-14T11:56:10.692072+0000","last_became_peered":"2023-07-14T11:56:10.692072+0000","last_unstale":"2023-07-14T11:56:10.703549+0000","last_undegraded":"2023-07-14T11:56:10.703549+0000","last_fullsized":"2023-07-14T11:56:10.703549+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:46:05.586216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682498+0000","last_change":"2023-07-14T11:56:10.682498+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682498+0000","last_undegraded":"2023-07-14T11:56:10.682498+0000","last_fullsized":"2023-07-14T11:56:10.682498+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707805+0000","last_change":"2023-07-14T11:56:10.701400+0000","last_active":"2023-07-14T11:56:10.707805+0000","last_peered":"2023-07-14T11:56:10.707805+0000","last_clean":"2023-07-14T11:56:10.707805+0000","last_became_active":"2023-07-14T11:56:10.701104+0000","last_became_peered":"2023-07-14T11:56:10.701104+0000","last_unstale":"2023-07-14T11:56:10.707805+0000","last_undegraded":"2023-07-14T11:56:10.707805+0000","last_fullsized":"2023-07-14T11:56:10.707805+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:15:04.078467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683497+0000","last_change":"2023-07-14T11:56:10.683497+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683497+0000","last_undegraded":"2023-07-14T11:56:10.683497+0000","last_fullsized":"2023-07-14T11:56:10.683497+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.705209+0000","last_change":"2023-07-14T11:56:10.702066+0000","last_active":"2023-07-14T11:56:10.705209+0000","last_peered":"2023-07-14T11:56:10.705209+0000","last_clean":"2023-07-14T11:56:10.705209+0000","last_became_active":"2023-07-14T11:56:10.701720+0000","last_became_peered":"2023-07-14T11:56:10.701720+0000","last_unstale":"2023-07-14T11:56:10.705209+0000","last_undegraded":"2023-07-14T11:56:10.705209+0000","last_fullsized":"2023-07-14T11:56:10.705209+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:34:01.415077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.679469+0000","last_change":"2023-07-14T11:56:10.679469+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.679469+0000","last_undegraded":"2023-07-14T11:56:10.679469+0000","last_fullsized":"2023-07-14T11:56:10.679469+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.704869+0000","last_change":"2023-07-14T11:56:10.699176+0000","last_active":"2023-07-14T11:56:10.704869+0000","last_peered":"2023-07-14T11:56:10.704869+0000","last_clean":"2023-07-14T11:56:10.704869+0000","last_became_active":"2023-07-14T11:56:10.698881+0000","last_became_peered":"2023-07-14T11:56:10.698881+0000","last_unstale":"2023-07-14T11:56:10.704869+0000","last_undegraded":"2023-07-14T11:56:10.704869+0000","last_fullsized":"2023-07-14T11:56:10.704869+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:50.194031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677560+0000","last_change":"2023-07-14T11:56:10.677560+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677560+0000","last_undegraded":"2023-07-14T11:56:10.677560+0000","last_fullsized":"2023-07-14T11:56:10.677560+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.698670+0000","last_change":"2023-07-14T11:56:10.695120+0000","last_active":"2023-07-14T11:56:10.698670+0000","last_peered":"2023-07-14T11:56:10.698670+0000","last_clean":"2023-07-14T11:56:10.698670+0000","last_became_active":"2023-07-14T11:56:10.693407+0000","last_became_peered":"2023-07-14T11:56:10.693407+0000","last_unstale":"2023-07-14T11:56:10.698670+0000","last_undegraded":"2023-07-14T11:56:10.698670+0000","last_fullsized":"2023-07-14T11:56:10.698670+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:47:56.814799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.688739+0000","last_change":"2023-07-14T11:56:10.688739+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.688739+0000","last_undegraded":"2023-07-14T11:56:10.688739+0000","last_fullsized":"2023-07-14T11:56:10.688739+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703966+0000","last_change":"2023-07-14T11:56:10.700844+0000","last_active":"2023-07-14T11:56:10.703966+0000","last_peered":"2023-07-14T11:56:10.703966+0000","last_clean":"2023-07-14T11:56:10.703966+0000","last_became_active":"2023-07-14T11:56:10.700534+0000","last_became_peered":"2023-07-14T11:56:10.700534+0000","last_unstale":"2023-07-14T11:56:10.703966+0000","last_undegraded":"2023-07-14T11:56:10.703966+0000","last_fullsized":"2023-07-14T11:56:10.703966+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:45:37.602121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.678287+0000","last_change":"2023-07-14T11:56:10.678287+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.678287+0000","last_undegraded":"2023-07-14T11:56:10.678287+0000","last_fullsized":"2023-07-14T11:56:10.678287+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.703169+0000","last_change":"2023-07-14T11:56:10.695559+0000","last_active":"2023-07-14T11:56:10.703169+0000","last_peered":"2023-07-14T11:56:10.703169+0000","last_clean":"2023-07-14T11:56:10.703169+0000","last_became_active":"2023-07-14T11:56:10.695312+0000","last_became_peered":"2023-07-14T11:56:10.695312+0000","last_unstale":"2023-07-14T11:56:10.703169+0000","last_undegraded":"2023-07-14T11:56:10.703169+0000","last_fullsized":"2023-07-14T11:56:10.703169+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:01.509577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.686132+0000","last_change":"2023-07-14T11:56:10.686132+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.686132+0000","last_undegraded":"2023-07-14T11:56:10.686132+0000","last_fullsized":"2023-07-14T11:56:10.686132+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.704530+0000","last_change":"2023-07-14T11:56:10.703244+0000","last_active":"2023-07-14T11:56:10.704530+0000","last_peered":"2023-07-14T11:56:10.704530+0000","last_clean":"2023-07-14T11:56:10.704530+0000","last_became_active":"2023-07-14T11:56:10.702957+0000","last_became_peered":"2023-07-14T11:56:10.702957+0000","last_unstale":"2023-07-14T11:56:10.704530+0000","last_undegraded":"2023-07-14T11:56:10.704530+0000","last_fullsized":"2023-07-14T11:56:10.704530+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:47:56.814799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677574+0000","last_change":"2023-07-14T11:56:10.677574+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677574+0000","last_undegraded":"2023-07-14T11:56:10.677574+0000","last_fullsized":"2023-07-14T11:56:10.677574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.700755+0000","last_change":"2023-07-14T11:56:10.695223+0000","last_active":"2023-07-14T11:56:10.700755+0000","last_peered":"2023-07-14T11:56:10.700755+0000","last_clean":"2023-07-14T11:56:10.700755+0000","last_became_active":"2023-07-14T11:56:10.691638+0000","last_became_peered":"2023-07-14T11:56:10.691638+0000","last_unstale":"2023-07-14T11:56:10.700755+0000","last_undegraded":"2023-07-14T11:56:10.700755+0000","last_fullsized":"2023-07-14T11:56:10.700755+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:06:54.458842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676708+0000","last_change":"2023-07-14T11:56:10.676708+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676708+0000","last_undegraded":"2023-07-14T11:56:10.676708+0000","last_fullsized":"2023-07-14T11:56:10.676708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.704883+0000","last_change":"2023-07-14T11:56:10.699836+0000","last_active":"2023-07-14T11:56:10.704883+0000","last_peered":"2023-07-14T11:56:10.704883+0000","last_clean":"2023-07-14T11:56:10.704883+0000","last_became_active":"2023-07-14T11:56:10.699528+0000","last_became_peered":"2023-07-14T11:56:10.699528+0000","last_unstale":"2023-07-14T11:56:10.704883+0000","last_undegraded":"2023-07-14T11:56:10.704883+0000","last_fullsized":"2023-07-14T11:56:10.704883+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:28:16.553232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682860+0000","last_change":"2023-07-14T11:56:10.682860+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682860+0000","last_undegraded":"2023-07-14T11:56:10.682860+0000","last_fullsized":"2023-07-14T11:56:10.682860+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.695725+0000","last_change":"2023-07-14T11:56:10.693503+0000","last_active":"2023-07-14T11:56:10.695725+0000","last_peered":"2023-07-14T11:56:10.695725+0000","last_clean":"2023-07-14T11:56:10.695725+0000","last_became_active":"2023-07-14T11:56:10.693197+0000","last_became_peered":"2023-07-14T11:56:10.693197+0000","last_unstale":"2023-07-14T11:56:10.695725+0000","last_undegraded":"2023-07-14T11:56:10.695725+0000","last_fullsized":"2023-07-14T11:56:10.695725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:21:50.291873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677561+0000","last_change":"2023-07-14T11:56:10.677561+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677561+0000","last_undegraded":"2023-07-14T11:56:10.677561+0000","last_fullsized":"2023-07-14T11:56:10.677561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.693506+0000","last_change":"2023-07-14T11:56:10.691758+0000","last_active":"2023-07-14T11:56:10.693506+0000","last_peered":"2023-07-14T11:56:10.693506+0000","last_clean":"2023-07-14T11:56:10.693506+0000","last_became_active":"2023-07-14T11:56:10.691130+0000","last_became_peered":"2023-07-14T11:56:10.691130+0000","last_unstale":"2023-07-14T11:56:10.693506+0000","last_undegraded":"2023-07-14T11:56:10.693506+0000","last_fullsized":"2023-07-14T11:56:10.693506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:35:21.027007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.678966+0000","last_change":"2023-07-14T11:56:10.678966+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.678966+0000","last_undegraded":"2023-07-14T11:56:10.678966+0000","last_fullsized":"2023-07-14T11:56:10.678966+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.700706+0000","last_change":"2023-07-14T11:56:10.692036+0000","last_active":"2023-07-14T11:56:10.700706+0000","last_peered":"2023-07-14T11:56:10.700706+0000","last_clean":"2023-07-14T11:56:10.700706+0000","last_became_active":"2023-07-14T11:56:10.691608+0000","last_became_peered":"2023-07-14T11:56:10.691608+0000","last_unstale":"2023-07-14T11:56:10.700706+0000","last_undegraded":"2023-07-14T11:56:10.700706+0000","last_fullsized":"2023-07-14T11:56:10.700706+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:30:54.612583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676440+0000","last_change":"2023-07-14T11:56:10.676440+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676440+0000","last_undegraded":"2023-07-14T11:56:10.676440+0000","last_fullsized":"2023-07-14T11:56:10.676440+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.692802+0000","last_change":"2023-07-14T11:56:10.690467+0000","last_active":"2023-07-14T11:56:10.692802+0000","last_peered":"2023-07-14T11:56:10.692802+0000","last_clean":"2023-07-14T11:56:10.692802+0000","last_became_active":"2023-07-14T11:56:10.690201+0000","last_became_peered":"2023-07-14T11:56:10.690201+0000","last_unstale":"2023-07-14T11:56:10.692802+0000","last_undegraded":"2023-07-14T11:56:10.692802+0000","last_fullsized":"2023-07-14T11:56:10.692802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:06:38.660785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.681506+0000","last_change":"2023-07-14T11:53:09.145201+0000","last_active":"2023-07-14T11:56:10.681506+0000","last_peered":"2023-07-14T11:56:10.681506+0000","last_clean":"2023-07-14T11:56:10.681506+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:56:10.681506+0000","last_undegraded":"2023-07-14T11:56:10.681506+0000","last_fullsized":"2023-07-14T11:56:10.681506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:16:24.795475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.683131+0000","last_change":"2023-07-14T11:56:10.683131+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.683131+0000","last_undegraded":"2023-07-14T11:56:10.683131+0000","last_fullsized":"2023-07-14T11:56:10.683131+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707863+0000","last_change":"2023-07-14T11:56:10.704116+0000","last_active":"2023-07-14T11:56:10.707863+0000","last_peered":"2023-07-14T11:56:10.707863+0000","last_clean":"2023-07-14T11:56:10.707863+0000","last_became_active":"2023-07-14T11:56:10.703775+0000","last_became_peered":"2023-07-14T11:56:10.703775+0000","last_unstale":"2023-07-14T11:56:10.707863+0000","last_undegraded":"2023-07-14T11:56:10.707863+0000","last_fullsized":"2023-07-14T11:56:10.707863+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:23:09.743965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.687564+0000","last_change":"2023-07-14T11:53:09.146150+0000","last_active":"2023-07-14T11:56:10.687564+0000","last_peered":"2023-07-14T11:56:10.687564+0000","last_clean":"2023-07-14T11:56:10.687564+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:56:10.687564+0000","last_undegraded":"2023-07-14T11:56:10.687564+0000","last_fullsized":"2023-07-14T11:56:10.687564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:49:39.347836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676479+0000","last_change":"2023-07-14T11:56:10.676479+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676479+0000","last_undegraded":"2023-07-14T11:56:10.676479+0000","last_fullsized":"2023-07-14T11:56:10.676479+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707751+0000","last_change":"2023-07-14T11:56:10.700575+0000","last_active":"2023-07-14T11:56:10.707751+0000","last_peered":"2023-07-14T11:56:10.707751+0000","last_clean":"2023-07-14T11:56:10.707751+0000","last_became_active":"2023-07-14T11:56:10.700293+0000","last_became_peered":"2023-07-14T11:56:10.700293+0000","last_unstale":"2023-07-14T11:56:10.707751+0000","last_undegraded":"2023-07-14T11:56:10.707751+0000","last_fullsized":"2023-07-14T11:56:10.707751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:24:16.135877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.694687+0000","last_change":"2023-07-14T11:53:09.435876+0000","last_active":"2023-07-14T11:56:10.694687+0000","last_peered":"2023-07-14T11:56:10.694687+0000","last_clean":"2023-07-14T11:56:10.694687+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:56:10.694687+0000","last_undegraded":"2023-07-14T11:56:10.694687+0000","last_fullsized":"2023-07-14T11:56:10.694687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:29:06.737180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687524+0000","last_change":"2023-07-14T11:56:10.687524+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687524+0000","last_undegraded":"2023-07-14T11:56:10.687524+0000","last_fullsized":"2023-07-14T11:56:10.687524+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.707881+0000","last_change":"2023-07-14T11:56:10.702843+0000","last_active":"2023-07-14T11:56:10.707881+0000","last_peered":"2023-07-14T11:56:10.707881+0000","last_clean":"2023-07-14T11:56:10.707881+0000","last_became_active":"2023-07-14T11:56:10.702583+0000","last_became_peered":"2023-07-14T11:56:10.702583+0000","last_unstale":"2023-07-14T11:56:10.707881+0000","last_undegraded":"2023-07-14T11:56:10.707881+0000","last_fullsized":"2023-07-14T11:56:10.707881+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:08:58.730889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.687487+0000","last_change":"2023-07-14T11:53:09.434255+0000","last_active":"2023-07-14T11:56:10.687487+0000","last_peered":"2023-07-14T11:56:10.687487+0000","last_clean":"2023-07-14T11:56:10.687487+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:56:10.687487+0000","last_undegraded":"2023-07-14T11:56:10.687487+0000","last_fullsized":"2023-07-14T11:56:10.687487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:11:16.228885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.677097+0000","last_change":"2023-07-14T11:56:10.677097+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.677097+0000","last_undegraded":"2023-07-14T11:56:10.677097+0000","last_fullsized":"2023-07-14T11:56:10.677097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.699530+0000","last_change":"2023-07-14T11:56:10.697670+0000","last_active":"2023-07-14T11:56:10.699530+0000","last_peered":"2023-07-14T11:56:10.699530+0000","last_clean":"2023-07-14T11:56:10.699530+0000","last_became_active":"2023-07-14T11:56:10.697406+0000","last_became_peered":"2023-07-14T11:56:10.697406+0000","last_unstale":"2023-07-14T11:56:10.699530+0000","last_undegraded":"2023-07-14T11:56:10.699530+0000","last_fullsized":"2023-07-14T11:56:10.699530+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:05.395217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":84,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.694638+0000","last_change":"2023-07-14T11:53:09.147040+0000","last_active":"2023-07-14T11:56:10.694638+0000","last_peered":"2023-07-14T11:56:10.694638+0000","last_clean":"2023-07-14T11:56:10.694638+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:56:10.694638+0000","last_undegraded":"2023-07-14T11:56:10.694638+0000","last_fullsized":"2023-07-14T11:56:10.694638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:37.742068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.682908+0000","last_change":"2023-07-14T11:56:10.682908+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.682908+0000","last_undegraded":"2023-07-14T11:56:10.682908+0000","last_fullsized":"2023-07-14T11:56:10.682908+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.705188+0000","last_change":"2023-07-14T11:56:10.701923+0000","last_active":"2023-07-14T11:56:10.705188+0000","last_peered":"2023-07-14T11:56:10.705188+0000","last_clean":"2023-07-14T11:56:10.705188+0000","last_became_active":"2023-07-14T11:56:10.701351+0000","last_became_peered":"2023-07-14T11:56:10.701351+0000","last_unstale":"2023-07-14T11:56:10.705188+0000","last_undegraded":"2023-07-14T11:56:10.705188+0000","last_fullsized":"2023-07-14T11:56:10.705188+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:01:13.092675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":86,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.673618+0000","last_change":"2023-07-14T11:53:09.619294+0000","last_active":"2023-07-14T11:56:10.673618+0000","last_peered":"2023-07-14T11:56:10.673618+0000","last_clean":"2023-07-14T11:56:10.673618+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:56:10.673618+0000","last_undegraded":"2023-07-14T11:56:10.673618+0000","last_fullsized":"2023-07-14T11:56:10.673618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:07:12.290309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.687092+0000","last_change":"2023-07-14T11:56:10.687092+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.687092+0000","last_undegraded":"2023-07-14T11:56:10.687092+0000","last_fullsized":"2023-07-14T11:56:10.687092+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696108+0000","last_change":"2023-07-14T11:56:10.693854+0000","last_active":"2023-07-14T11:56:10.696108+0000","last_peered":"2023-07-14T11:56:10.696108+0000","last_clean":"2023-07-14T11:56:10.696108+0000","last_became_active":"2023-07-14T11:56:10.693588+0000","last_became_peered":"2023-07-14T11:56:10.693588+0000","last_unstale":"2023-07-14T11:56:10.696108+0000","last_undegraded":"2023-07-14T11:56:10.696108+0000","last_fullsized":"2023-07-14T11:56:10.696108+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:37:02.428527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":72,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696339+0000","last_change":"2023-07-14T11:53:09.146156+0000","last_active":"2023-07-14T11:56:10.696339+0000","last_peered":"2023-07-14T11:56:10.696339+0000","last_clean":"2023-07-14T11:56:10.696339+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:56:10.696339+0000","last_undegraded":"2023-07-14T11:56:10.696339+0000","last_fullsized":"2023-07-14T11:56:10.696339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:05:34.882240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2023-07-14T11:56:10.676332+0000","last_change":"2023-07-14T11:56:10.676332+0000","last_active":"2023-07-14T11:56:10.656698+0000","last_peered":"2023-07-14T11:56:10.656698+0000","last_clean":"2023-07-14T11:56:10.656698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-14T11:56:10.676332+0000","last_undegraded":"2023-07-14T11:56:10.676332+0000","last_fullsized":"2023-07-14T11:56:10.676332+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.697324+0000","last_change":"2023-07-14T11:56:10.694832+0000","last_active":"2023-07-14T11:56:10.697324+0000","last_peered":"2023-07-14T11:56:10.697324+0000","last_clean":"2023-07-14T11:56:10.697324+0000","last_became_active":"2023-07-14T11:56:10.694539+0000","last_became_peered":"2023-07-14T11:56:10.694539+0000","last_unstale":"2023-07-14T11:56:10.697324+0000","last_undegraded":"2023-07-14T11:56:10.697324+0000","last_fullsized":"2023-07-14T11:56:10.697324+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:17:28.732628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":84,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.694479+0000","last_change":"2023-07-14T11:53:09.146556+0000","last_active":"2023-07-14T11:56:10.694479+0000","last_peered":"2023-07-14T11:56:10.694479+0000","last_clean":"2023-07-14T11:56:10.694479+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:56:10.694479+0000","last_undegraded":"2023-07-14T11:56:10.694479+0000","last_fullsized":"2023-07-14T11:56:10.694479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:18:34.689837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-07-14T11:56:10.696481+0000","last_change":"2023-07-14T11:53:03.159818+0000","last_active":"2023-07-14T11:56:10.696481+0000","last_peered":"2023-07-14T11:56:10.696481+0000","last_clean":"2023-07-14T11:56:10.696481+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:56:10.696481+0000","last_undegraded":"2023-07-14T11:56:10.696481+0000","last_fullsized":"2023-07-14T11:56:10.696481+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_clean_scrub_stamp":"2023-07-14T11:53:02.054348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:05:58.561507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1265664,"data_stored":901013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.093,"15min":1.111},"min":{"1min":0.754,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.474,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.103,"15min":1.127},"min":{"1min":0.65100000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2430000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.4450000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1499999999999999,"15min":1.171},"min":{"1min":0.68300000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.298,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.22,"15min":1.262},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.4159999999999999,"5min":2.234,"15min":2.234},"last":0.88300000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.234,"15min":1.288},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.536,"5min":2.04,"15min":2.04},"last":1.143},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.25,"15min":1.2829999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8089999999999999,"5min":2.492,"15min":2.492},"last":1.1080000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.282,"15min":1.3360000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.665,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.2430000000000001,"15min":1.274},"min":{"1min":0.81799999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5940000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.81699999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.321,"15min":1.385},"min":{"1min":0.84399999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5660000000000001,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":1.371},{"interface":"front","average":{"1min":1.286,"5min":1.3400000000000001,"15min":1.3799999999999999},"min":{"1min":0.92500000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.76,"5min":2.399,"15min":2.399},"last":1.2490000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.284,"15min":1.3200000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6459999999999999,"5min":2.387,"15min":2.387},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.456,"15min":1.538},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.464,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":1.012}]},{"osd":6,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.5309999999999999,"15min":1.609},"min":{"1min":1.0940000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5289999999999999,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.494},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.4550000000000001,"15min":1.5229999999999999},"min":{"1min":0.95199999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.53,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.5049999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0700000000000001,"15min":1.0620000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.397,"5min":1.954,"15min":1.954},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.1160000000000001,"15min":1.1000000000000001},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.3460000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1890000000000001,"15min":1.2090000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7110000000000001,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.214,"15min":1.23},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7769999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.4770000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.171,"15min":1.1659999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4690000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2709999999999999,"15min":1.2729999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6819999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.0449999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.3140000000000001,"15min":1.3440000000000001},"min":{"1min":0.94099999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.6579999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.1899999999999999,"15min":1.1799999999999999},"min":{"1min":0.91700000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.482,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.98499999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.236,"15min":1.2529999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.625,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.133},{"interface":"front","average":{"1min":1.194,"5min":1.3129999999999999,"15min":1.3400000000000001},"min":{"1min":0.78000000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.512,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.599}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.238,"15min":1.2270000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.419},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4039999999999999,"15min":1.425},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.9139999999999999,"5min":1.984,"15min":1.984},"last":1.218}]},{"osd":7,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.4330000000000001,"15min":1.4650000000000001},"min":{"1min":0.99199999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.6519999999999999,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.643},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.365,"15min":1.379},"min":{"1min":0.92900000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.728,"5min":1.879,"15min":1.879},"last":1.571}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.1359999999999999,"15min":1.1819999999999999},"min":{"1min":0.68100000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4790000000000001,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.129,"15min":1.175},"min":{"1min":0.70299999999999996,"5min":0.622,"15min":0.622},"max":{"1min":1.474,"5min":1.581,"15min":1.581},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.167,"15min":1.1990000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.524,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.224},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.175,"15min":1.1890000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.498,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.1739999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.198,"15min":1.226},"min":{"1min":0.86699999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4550000000000001,"5min":1.839,"15min":1.839},"last":1.002},{"interface":"front","average":{"1min":1.113,"5min":1.282,"15min":1.347},"min":{"1min":0.73799999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4990000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.083}]},{"osd":3,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.272,"15min":1.3220000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.5409999999999999,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.242,"5min":1.3440000000000001,"15min":1.3919999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.5349999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.399}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2090000000000001,"15min":1.2490000000000001},"min":{"1min":0.75900000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5900000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.3540000000000001,"15min":1.3740000000000001},"min":{"1min":1.024,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.5109999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.264}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3959999999999999,"15min":1.4179999999999999},"min":{"1min":1.1459999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.544,"5min":1.99,"15min":1.99},"last":1.341},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.375,"15min":1.4159999999999999},"min":{"1min":0.86399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6779999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.3069999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4039999999999999,"15min":1.421},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.6220000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.5569999999999999,"15min":1.6160000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.5880000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.4430000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.149,"15min":1.171},"min":{"1min":0.67200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.5069999999999999,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.133,"15min":1.1599999999999999},"min":{"1min":0.71099999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.76500000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.159,"15min":1.1819999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.508,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.871},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1879999999999999,"15min":1.2090000000000001},"min":{"1min":0.75900000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.516,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.3}]},{"osd":2,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1359999999999999,"15min":1.143},"min":{"1min":0.83899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.546,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.095},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3400000000000001,"15min":1.3620000000000001},"min":{"1min":0.96599999999999997,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.581,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":1.046}]},{"osd":3,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2789999999999999,"15min":1.321},"min":{"1min":0.82199999999999995,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.681,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2270000000000001,"15min":1.2450000000000001},"min":{"1min":0.88100000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.4770000000000001,"5min":1.823,"15min":1.823},"last":1.3799999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2989999999999999,"15min":1.3420000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6299999999999999,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.421},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.89500000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.47,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.2609999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.264,"15min":1.2669999999999999},"min":{"1min":0.97799999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.522,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.476,"15min":1.532},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6759999999999999,"5min":3.5289999999999999,"15min":3.5289999999999999},"last":1.5109999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.466,"15min":1.5049999999999999},"min":{"1min":1.0589999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.6899999999999999,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.47,"15min":1.5209999999999999},"min":{"1min":1.0640000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":1.605,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":1.55}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":671744,"data_stored":310645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.84099999999999997,"15min":0.871},"min":{"1min":0.47699999999999998,"5min":0.247,"15min":0.247},"max":{"1min":1.234,"5min":1.325,"15min":1.325},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.875,"15min":0.90800000000000003},"min":{"1min":0.60599999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2490000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.96699999999999997}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92300000000000004,"15min":0.96299999999999997},"min":{"1min":0.52600000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6950000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.0600000000000001,"15min":1.111},"min":{"1min":0.54300000000000004,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.319,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.88600000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.097,"15min":1.141},"min":{"1min":0.63500000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4259999999999999,"5min":1.627,"15min":1.627},"last":1.119},{"interface":"front","average":{"1min":1.079,"5min":1.0669999999999999,"15min":1.091},"min":{"1min":0.73399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.94099999999999995}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2410000000000001,"15min":1.2849999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.53,"5min":2.335,"15min":2.335},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.2470000000000001,"15min":1.2869999999999999},"min":{"1min":0.77700000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.1659999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.323,"15min":1.359},"min":{"1min":0.94999999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.3580000000000001,"15min":1.421},"min":{"1min":0.86899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.532,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.2}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.329,"15min":1.387},"min":{"1min":0.85899999999999999,"5min":0.625,"15min":0.625},"max":{"1min":1.7010000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.4550000000000001,"15min":1.5009999999999999},"min":{"1min":1.083,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.0790000000000002,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.3819999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.419,"15min":1.4810000000000001},"min":{"1min":0.90500000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.712,"5min":3.395,"15min":3.395},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.367,"15min":1.431},"min":{"1min":1.0680000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.619,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.1850000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.874,"15min":0.90600000000000003},"min":{"1min":0.53800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.22,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.876,"15min":0.90200000000000002},"min":{"1min":0.48799999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1799999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.95499999999999996}]},{"osd":1,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.88100000000000001,"15min":0.89900000000000002},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.093,"5min":1.365,"15min":1.365},"last":0.622},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0269999999999999,"15min":1.069},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.425,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.202}]},{"osd":3,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0649999999999999,"15min":1.0960000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5089999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.018,"5min":1.026,"15min":1.052},"min":{"1min":0.65300000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.5900000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.1419999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2709999999999999,"15min":1.2989999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.718,"5min":2.097,"15min":2.097},"last":1.268},{"interface":"front","average":{"1min":1.242,"5min":1.2769999999999999,"15min":1.3149999999999999},"min":{"1min":0.67800000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.861,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.7270000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.2889999999999999,"15min":1.319},"min":{"1min":0.95699999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.4710000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.242},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3180000000000001,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.645,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.3520000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.333,"15min":1.383},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7649999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.7849999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.3919999999999999,"15min":1.425},"min":{"1min":1.1040000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.726,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.321}]},{"osd":7,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.397,"15min":1.444},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.677,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.9079999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4219999999999999,"15min":1.466},"min":{"1min":1.01,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6899999999999999,"5min":2.536,"15min":2.536},"last":1.4219999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":684032,"data_stored":317585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.80100000000000005,"15min":0.83299999999999996},"min":{"1min":0.48899999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.26,"5min":1.349,"15min":1.349},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83299999999999996,"15min":0.83399999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.113}]},{"osd":2,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84799999999999998,"15min":0.85799999999999998},"min":{"1min":0.499,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.93500000000000005,"15min":0.93999999999999995},"min":{"1min":0.55900000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.81799999999999995}]},{"osd":3,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.99199999999999999,"15min":1.002},"min":{"1min":0.78500000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.399,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.272},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0169999999999999,"15min":1.0229999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.8169999999999999,"5min":1.857,"15min":1.857},"last":0.874}]},{"osd":4,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.2729999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6279999999999999,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.022},{"interface":"front","average":{"1min":1.111,"5min":1.1830000000000001,"15min":1.2170000000000001},"min":{"1min":0.754,"5min":0.745,"15min":0.745},"max":{"1min":1.6279999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.206}]},{"osd":5,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.23,"15min":1.256},"min":{"1min":0.876,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7190000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3149999999999999,"15min":1.365},"min":{"1min":0.93100000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.778,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3560000000000001,"15min":1.387},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.9119999999999999,"5min":2.633,"15min":2.633},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.335,"15min":1.359},"min":{"1min":1.091,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.873,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.367}]},{"osd":7,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3580000000000001,"15min":1.3979999999999999},"min":{"1min":1.004,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.992,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3300000000000001,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.754,"15min":0.754},"max":{"1min":2.056,"5min":2.375,"15min":2.375},"last":1.3260000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1265664,"data_stored":901013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.879,"15min":0.89300000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3100000000000001,"5min":1.581,"15min":1.581},"last":1.0329999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0049999999999999,"15min":1},"min":{"1min":0.64800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.5880000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90800000000000003,"15min":0.95099999999999996},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.1639999999999999,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.93600000000000005,"15min":0.94599999999999995},"min":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.228,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.099,"15min":1.1319999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3799999999999999,"5min":1.704,"15min":1.704},"last":1.22}]},{"osd":4,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2430000000000001,"15min":1.278},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5149999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3420000000000001,"15min":1.3720000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.837,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.345}]},{"osd":5,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.228,"15min":1.284},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9870000000000001,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.276,"15min":1.3200000000000001},"min":{"1min":0.79800000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.7749999999999999,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.0940000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.0459999999999998,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.319,"15min":1.357},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.9299999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.3300000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.335,"15min":1.3819999999999999},"min":{"1min":0.83299999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.865,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.379,"15min":1.417},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.6870000000000001,"5min":2.109,"15min":2.109},"last":1.2929999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:56:20.816 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:56:20.991 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.005+0000 7f531aa7d700 1 Processor -- start 2023-07-14T11:56:20.992 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.005+0000 7f531aa7d700 1 -- start start 2023-07-14T11:56:20.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.005+0000 7f531aa7d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f531415ffa0 0x7f531406d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:20.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.005+0000 7f531aa7d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53140cb180 0x7f53140ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:20.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.005+0000 7f531aa7d700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53140ca8a0 con 0x7f531406dc50 2023-07-14T11:56:20.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.005+0000 7f531aa7d700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53140cbcd0 con 0x7f531415ffa0 2023-07-14T11:56:20.994 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.005+0000 7f531aa7d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53140cbf70 con 0x7f53140cb180 2023-07-14T11:56:20.995 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.005+0000 7f5319a7b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f531415ffa0 0x7f531406d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:20.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.005+0000 7f531a27c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53140cb180 0x7f53140ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:20.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.005+0000 7f5319a7b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f531415ffa0 0x7f531406d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:41020/0 (socket says 172.21.15.104:41020) 2023-07-14T11:56:20.996 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.005+0000 7f5319a7b700 1 -- 172.21.15.104:0/2306116986 learned_addr learned my addr 172.21.15.104:0/2306116986 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:20.997 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.005+0000 7f531a27c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53140cb180 0x7f53140ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:59398/0 (socket says 172.21.15.104:59398) 2023-07-14T11:56:20.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.009+0000 7f531a27c700 1 -- 172.21.15.104:0/2306116986 >> v2:172.21.15.121:3300/0 conn(0x7f531415ffa0 msgr2=0x7f531406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:20.999 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.009+0000 7f531a27c700 1 --2- 172.21.15.104:0/2306116986 >> v2:172.21.15.121:3300/0 conn(0x7f531415ffa0 0x7f531406d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.000 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.009+0000 7f531a27c700 1 -- 172.21.15.104:0/2306116986 >> v1:172.21.15.121:6789/0 conn(0x7f531406dc50 legacy=0x7f53140caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.000 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.009+0000 7f531a27c700 1 -- 172.21.15.104:0/2306116986 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53140cc250 con 0x7f53140cb180 2023-07-14T11:56:21.000 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.009+0000 7f5319a7b700 1 --2- 172.21.15.104:0/2306116986 >> v2:172.21.15.121:3300/0 conn(0x7f531415ffa0 0x7f531406d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:21.000 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.009+0000 7f531a27c700 1 --2- 172.21.15.104:0/2306116986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53140cb180 0x7f53140ce340 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f530c009d90 tx=0x7f530c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a87737f6b41c0abf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:21.001 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.009+0000 7f5318a79700 1 -- 172.21.15.104:0/2306116986 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c00e070 con 0x7f53140cb180 2023-07-14T11:56:21.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.009+0000 7f5318a79700 1 -- 172.21.15.104:0/2306116986 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530c00bb20 con 0x7f53140cb180 2023-07-14T11:56:21.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.009+0000 7f5318a79700 1 -- 172.21.15.104:0/2306116986 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c012430 con 0x7f53140cb180 2023-07-14T11:56:21.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.009+0000 7f531aa7d700 1 -- 172.21.15.104:0/2306116986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53140cb180 msgr2=0x7f53140ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.002 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.009+0000 7f531aa7d700 1 --2- 172.21.15.104:0/2306116986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53140cb180 0x7f53140ce340 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f530c009d90 tx=0x7f530c00b330 comp rx=0 tx=0).stop 2023-07-14T11:56:21.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.009+0000 7f531aa7d700 1 -- 172.21.15.104:0/2306116986 shutdown_connections 2023-07-14T11:56:21.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.009+0000 7f531aa7d700 1 --2- 172.21.15.104:0/2306116986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53140cb180 0x7f53140ce340 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.009+0000 7f531aa7d700 1 --2- 172.21.15.104:0/2306116986 >> v2:172.21.15.121:3300/0 conn(0x7f531415ffa0 0x7f531406d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.009+0000 7f531aa7d700 1 -- 172.21.15.104:0/2306116986 >> 172.21.15.104:0/2306116986 conn(0x7f531406e960 msgr2=0x7f531406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:21.003 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.013+0000 7f531aa7d700 1 -- 172.21.15.104:0/2306116986 shutdown_connections 2023-07-14T11:56:21.004 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.013+0000 7f531aa7d700 1 -- 172.21.15.104:0/2306116986 wait complete. 2023-07-14T11:56:21.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.013+0000 7f531aa7d700 1 Processor -- start 2023-07-14T11:56:21.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.013+0000 7f531aa7d700 1 -- start start 2023-07-14T11:56:21.006 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531aa7d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f531406dc50 0x7f531415e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531aa7d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53140cb180 0x7f53141e9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531aa7d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f531415ffa0 0x7f53140aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531aa7d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53141eb0d0 con 0x7f531406dc50 2023-07-14T11:56:21.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531aa7d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53141ea030 con 0x7f531415ffa0 2023-07-14T11:56:21.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531aa7d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53141ea310 con 0x7f53140cb180 2023-07-14T11:56:21.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f5319a7b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f531406dc50 0x7f531415e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f5319a7b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f531406dc50 0x7f531415e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45942/0 (socket says 172.21.15.104:45942) 2023-07-14T11:56:21.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f5319a7b700 1 -- 172.21.15.104:0/1915317265 learned_addr learned my addr 172.21.15.104:0/1915317265 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:21.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531927a700 1 --2- 172.21.15.104:0/1915317265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53140cb180 0x7f53141e9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531a27c700 1 --2- 172.21.15.104:0/1915317265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f531415ffa0 0x7f53140aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531927a700 1 -- 172.21.15.104:0/1915317265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f531415ffa0 msgr2=0x7f53140aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531927a700 1 --2- 172.21.15.104:0/1915317265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f531415ffa0 0x7f53140aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531927a700 1 -- 172.21.15.104:0/1915317265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f531406dc50 msgr2=0x7f531415e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531927a700 1 --2- 172.21.15.104:0/1915317265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f531406dc50 0x7f531415e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531927a700 1 -- 172.21.15.104:0/1915317265 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5310009570 con 0x7f53140cb180 2023-07-14T11:56:21.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531927a700 1 --2- 172.21.15.104:0/1915317265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53140cb180 0x7f53141e9650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f531406e0d0 tx=0x7f5304001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:21.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f530a7fc700 1 -- 172.21.15.104:0/1915317265 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53040052d0 con 0x7f53140cb180 2023-07-14T11:56:21.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f530a7fc700 1 -- 172.21.15.104:0/1915317265 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5304005430 con 0x7f53140cb180 2023-07-14T11:56:21.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531aa7d700 1 -- 172.21.15.104:0/1915317265 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530c009a70 con 0x7f53140cb180 2023-07-14T11:56:21.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f530a7fc700 1 -- 172.21.15.104:0/1915317265 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53040055a0 con 0x7f53140cb180 2023-07-14T11:56:21.009 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.017+0000 7f531aa7d700 1 -- 172.21.15.104:0/1915317265 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53141ea8d0 con 0x7f53140cb180 2023-07-14T11:56:21.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.021+0000 7f531aa7d700 1 -- 172.21.15.104:0/1915317265 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5314071620 con 0x7f53140cb180 2023-07-14T11:56:21.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.021+0000 7f530a7fc700 1 -- 172.21.15.104:0/1915317265 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f53040038e0 con 0x7f53140cb180 2023-07-14T11:56:21.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.025+0000 7f530a7fc700 1 --2- 172.21.15.104:0/1915317265 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52f80662f0 0x7f52f80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.025+0000 7f530a7fc700 1 -- 172.21.15.104:0/1915317265 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5304083090 con 0x7f53140cb180 2023-07-14T11:56:21.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.025+0000 7f5319a7b700 1 --2- 172.21.15.104:0/1915317265 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52f80662f0 0x7f52f80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.025+0000 7f5319a7b700 1 --2- 172.21.15.104:0/1915317265 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52f80662f0 0x7f52f80687a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5310009e40 tx=0x7f53100092e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:21.015 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.029+0000 7f530a7fc700 1 -- 172.21.15.104:0/1915317265 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f5304050440 con 0x7f53140cb180 2023-07-14T11:56:21.187 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.201+0000 7f531aa7d700 1 -- 172.21.15.104:0/1915317265 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53140cc810 con 0x7f52f80662f0 2023-07-14T11:56:21.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.209+0000 7f530a7fc700 1 -- 172.21.15.104:0/1915317265 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309144 (secure 0 0 0) 0x7f5310011b40 con 0x7f52f80662f0 2023-07-14T11:56:21.201 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:21.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.213+0000 7f531aa7d700 1 -- 172.21.15.104:0/1915317265 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52f80662f0 msgr2=0x7f52f80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.213+0000 7f531aa7d700 1 --2- 172.21.15.104:0/1915317265 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52f80662f0 0x7f52f80687a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5310009e40 tx=0x7f53100092e0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.213+0000 7f531aa7d700 1 -- 172.21.15.104:0/1915317265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53140cb180 msgr2=0x7f53141e9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.213+0000 7f531aa7d700 1 --2- 172.21.15.104:0/1915317265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53140cb180 0x7f53141e9650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f531406e0d0 tx=0x7f5304001e70 comp rx=0 tx=0).stop 2023-07-14T11:56:21.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.217+0000 7f531aa7d700 1 -- 172.21.15.104:0/1915317265 shutdown_connections 2023-07-14T11:56:21.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.217+0000 7f531aa7d700 1 --2- 172.21.15.104:0/1915317265 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52f80662f0 0x7f52f80687a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.217+0000 7f531aa7d700 1 --2- 172.21.15.104:0/1915317265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f531415ffa0 0x7f53140aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.217+0000 7f531aa7d700 1 --2- 172.21.15.104:0/1915317265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53140cb180 0x7f53141e9650 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.217+0000 7f531aa7d700 1 --2- 172.21.15.104:0/1915317265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f531406dc50 0x7f531415e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.217+0000 7f531aa7d700 1 -- 172.21.15.104:0/1915317265 >> 172.21.15.104:0/1915317265 conn(0x7f531406e960 msgr2=0x7f531406bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:21.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.217+0000 7f531aa7d700 1 -- 172.21.15.104:0/1915317265 shutdown_connections 2023-07-14T11:56:21.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.217+0000 7f531aa7d700 1 -- 172.21.15.104:0/1915317265 wait complete. 2023-07-14T11:56:21.205 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:56:21.225 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2023-07-14T11:56:19.963588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":199,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230568,"kb_used_data":6760,"kb_used_omap":0,"kb_used_meta":223744,"kb_avail":754744152,"statfs":{"total":773094113280,"available":772858011648,"internally_reserved":0,"allocated":6922240,"data_stored":3841508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":229113856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.283596"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.942618+0000","last_change":"2023-07-14T11:56:11.712459+0000","last_active":"2023-07-14T11:56:12.942618+0000","last_peered":"2023-07-14T11:56:12.942618+0000","last_clean":"2023-07-14T11:56:12.942618+0000","last_became_active":"2023-07-14T11:56:11.712171+0000","last_became_peered":"2023-07-14T11:56:11.712171+0000","last_unstale":"2023-07-14T11:56:12.942618+0000","last_undegraded":"2023-07-14T11:56:12.942618+0000","last_fullsized":"2023-07-14T11:56:12.942618+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:58:37.674842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.942226+0000","last_change":"2023-07-14T11:56:11.710649+0000","last_active":"2023-07-14T11:56:12.942226+0000","last_peered":"2023-07-14T11:56:12.942226+0000","last_clean":"2023-07-14T11:56:12.942226+0000","last_became_active":"2023-07-14T11:56:11.710371+0000","last_became_peered":"2023-07-14T11:56:11.710371+0000","last_unstale":"2023-07-14T11:56:12.942226+0000","last_undegraded":"2023-07-14T11:56:12.942226+0000","last_fullsized":"2023-07-14T11:56:12.942226+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:00:24.735486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.945347+0000","last_change":"2023-07-14T11:56:11.714457+0000","last_active":"2023-07-14T11:56:12.945347+0000","last_peered":"2023-07-14T11:56:12.945347+0000","last_clean":"2023-07-14T11:56:12.945347+0000","last_became_active":"2023-07-14T11:56:11.714176+0000","last_became_peered":"2023-07-14T11:56:11.714176+0000","last_unstale":"2023-07-14T11:56:12.945347+0000","last_undegraded":"2023-07-14T11:56:12.945347+0000","last_fullsized":"2023-07-14T11:56:12.945347+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:35:22.034240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.704007+0000","last_change":"2023-07-14T11:56:11.712534+0000","last_active":"2023-07-14T11:56:12.704007+0000","last_peered":"2023-07-14T11:56:12.704007+0000","last_clean":"2023-07-14T11:56:12.704007+0000","last_became_active":"2023-07-14T11:56:11.712186+0000","last_became_peered":"2023-07-14T11:56:11.712186+0000","last_unstale":"2023-07-14T11:56:12.704007+0000","last_undegraded":"2023-07-14T11:56:12.704007+0000","last_fullsized":"2023-07-14T11:56:12.704007+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:48:42.232955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.704612+0000","last_change":"2023-07-14T11:56:11.710703+0000","last_active":"2023-07-14T11:56:12.704612+0000","last_peered":"2023-07-14T11:56:12.704612+0000","last_clean":"2023-07-14T11:56:12.704612+0000","last_became_active":"2023-07-14T11:56:11.710426+0000","last_became_peered":"2023-07-14T11:56:11.710426+0000","last_unstale":"2023-07-14T11:56:12.704612+0000","last_undegraded":"2023-07-14T11:56:12.704612+0000","last_fullsized":"2023-07-14T11:56:12.704612+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:13:58.850438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944288+0000","last_change":"2023-07-14T11:56:11.710978+0000","last_active":"2023-07-14T11:56:12.944288+0000","last_peered":"2023-07-14T11:56:12.944288+0000","last_clean":"2023-07-14T11:56:12.944288+0000","last_became_active":"2023-07-14T11:56:11.710689+0000","last_became_peered":"2023-07-14T11:56:11.710689+0000","last_unstale":"2023-07-14T11:56:12.944288+0000","last_undegraded":"2023-07-14T11:56:12.944288+0000","last_fullsized":"2023-07-14T11:56:12.944288+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:50:04.043370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.699375+0000","last_change":"2023-07-14T11:56:11.702354+0000","last_active":"2023-07-14T11:56:12.699375+0000","last_peered":"2023-07-14T11:56:12.699375+0000","last_clean":"2023-07-14T11:56:12.699375+0000","last_became_active":"2023-07-14T11:56:11.702110+0000","last_became_peered":"2023-07-14T11:56:11.702110+0000","last_unstale":"2023-07-14T11:56:12.699375+0000","last_undegraded":"2023-07-14T11:56:12.699375+0000","last_fullsized":"2023-07-14T11:56:12.699375+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:28:31.023450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943559+0000","last_change":"2023-07-14T11:56:11.721692+0000","last_active":"2023-07-14T11:56:12.943559+0000","last_peered":"2023-07-14T11:56:12.943559+0000","last_clean":"2023-07-14T11:56:12.943559+0000","last_became_active":"2023-07-14T11:56:11.720783+0000","last_became_peered":"2023-07-14T11:56:11.720783+0000","last_unstale":"2023-07-14T11:56:12.943559+0000","last_undegraded":"2023-07-14T11:56:12.943559+0000","last_fullsized":"2023-07-14T11:56:12.943559+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:33:01.427328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944780+0000","last_change":"2023-07-14T11:56:11.722761+0000","last_active":"2023-07-14T11:56:12.944780+0000","last_peered":"2023-07-14T11:56:12.944780+0000","last_clean":"2023-07-14T11:56:12.944780+0000","last_became_active":"2023-07-14T11:56:11.722563+0000","last_became_peered":"2023-07-14T11:56:11.722563+0000","last_unstale":"2023-07-14T11:56:12.944780+0000","last_undegraded":"2023-07-14T11:56:12.944780+0000","last_fullsized":"2023-07-14T11:56:12.944780+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:29:09.003223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.945051+0000","last_change":"2023-07-14T11:56:11.708257+0000","last_active":"2023-07-14T11:56:12.945051+0000","last_peered":"2023-07-14T11:56:12.945051+0000","last_clean":"2023-07-14T11:56:12.945051+0000","last_became_active":"2023-07-14T11:56:11.707913+0000","last_became_peered":"2023-07-14T11:56:11.707913+0000","last_unstale":"2023-07-14T11:56:12.945051+0000","last_undegraded":"2023-07-14T11:56:12.945051+0000","last_fullsized":"2023-07-14T11:56:12.945051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:21:51.299106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.704636+0000","last_change":"2023-07-14T11:56:11.717975+0000","last_active":"2023-07-14T11:56:12.704636+0000","last_peered":"2023-07-14T11:56:12.704636+0000","last_clean":"2023-07-14T11:56:12.704636+0000","last_became_active":"2023-07-14T11:56:11.713760+0000","last_became_peered":"2023-07-14T11:56:11.713760+0000","last_unstale":"2023-07-14T11:56:12.704636+0000","last_undegraded":"2023-07-14T11:56:12.704636+0000","last_fullsized":"2023-07-14T11:56:12.704636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:32:42.763504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943755+0000","last_change":"2023-07-14T11:56:11.721288+0000","last_active":"2023-07-14T11:56:12.943755+0000","last_peered":"2023-07-14T11:56:12.943755+0000","last_clean":"2023-07-14T11:56:12.943755+0000","last_became_active":"2023-07-14T11:56:11.720993+0000","last_became_peered":"2023-07-14T11:56:11.720993+0000","last_unstale":"2023-07-14T11:56:12.943755+0000","last_undegraded":"2023-07-14T11:56:12.943755+0000","last_fullsized":"2023-07-14T11:56:12.943755+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:59:26.712811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944050+0000","last_change":"2023-07-14T11:56:11.715957+0000","last_active":"2023-07-14T11:56:12.944050+0000","last_peered":"2023-07-14T11:56:12.944050+0000","last_clean":"2023-07-14T11:56:12.944050+0000","last_became_active":"2023-07-14T11:56:11.715411+0000","last_became_peered":"2023-07-14T11:56:11.715411+0000","last_unstale":"2023-07-14T11:56:12.944050+0000","last_undegraded":"2023-07-14T11:56:12.944050+0000","last_fullsized":"2023-07-14T11:56:12.944050+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:47:55.953746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944776+0000","last_change":"2023-07-14T11:56:11.720202+0000","last_active":"2023-07-14T11:56:12.944776+0000","last_peered":"2023-07-14T11:56:12.944776+0000","last_clean":"2023-07-14T11:56:12.944776+0000","last_became_active":"2023-07-14T11:56:11.719869+0000","last_became_peered":"2023-07-14T11:56:11.719869+0000","last_unstale":"2023-07-14T11:56:12.944776+0000","last_undegraded":"2023-07-14T11:56:12.944776+0000","last_fullsized":"2023-07-14T11:56:12.944776+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:21:37.230459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.945191+0000","last_change":"2023-07-14T11:56:11.708960+0000","last_active":"2023-07-14T11:56:12.945191+0000","last_peered":"2023-07-14T11:56:12.945191+0000","last_clean":"2023-07-14T11:56:12.945191+0000","last_became_active":"2023-07-14T11:56:11.708632+0000","last_became_peered":"2023-07-14T11:56:11.708632+0000","last_unstale":"2023-07-14T11:56:12.945191+0000","last_undegraded":"2023-07-14T11:56:12.945191+0000","last_fullsized":"2023-07-14T11:56:12.945191+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:06:39.668018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.699528+0000","last_change":"2023-07-14T11:56:11.708821+0000","last_active":"2023-07-14T11:56:12.699528+0000","last_peered":"2023-07-14T11:56:12.699528+0000","last_clean":"2023-07-14T11:56:12.699528+0000","last_became_active":"2023-07-14T11:56:11.708285+0000","last_became_peered":"2023-07-14T11:56:11.708285+0000","last_unstale":"2023-07-14T11:56:12.699528+0000","last_undegraded":"2023-07-14T11:56:12.699528+0000","last_fullsized":"2023-07-14T11:56:12.699528+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:59:52.732453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.942790+0000","last_change":"2023-07-14T11:56:11.716502+0000","last_active":"2023-07-14T11:56:12.942790+0000","last_peered":"2023-07-14T11:56:12.942790+0000","last_clean":"2023-07-14T11:56:12.942790+0000","last_became_active":"2023-07-14T11:56:11.716208+0000","last_became_peered":"2023-07-14T11:56:11.716208+0000","last_unstale":"2023-07-14T11:56:12.942790+0000","last_undegraded":"2023-07-14T11:56:12.942790+0000","last_fullsized":"2023-07-14T11:56:12.942790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:47:05.111158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.704140+0000","last_change":"2023-07-14T11:56:11.714449+0000","last_active":"2023-07-14T11:56:12.704140+0000","last_peered":"2023-07-14T11:56:12.704140+0000","last_clean":"2023-07-14T11:56:12.704140+0000","last_became_active":"2023-07-14T11:56:11.714175+0000","last_became_peered":"2023-07-14T11:56:11.714175+0000","last_unstale":"2023-07-14T11:56:12.704140+0000","last_undegraded":"2023-07-14T11:56:12.704140+0000","last_fullsized":"2023-07-14T11:56:12.704140+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:53:31.385981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.705007+0000","last_change":"2023-07-14T11:56:11.693322+0000","last_active":"2023-07-14T11:56:12.705007+0000","last_peered":"2023-07-14T11:56:12.705007+0000","last_clean":"2023-07-14T11:56:12.705007+0000","last_became_active":"2023-07-14T11:56:11.693028+0000","last_became_peered":"2023-07-14T11:56:11.693028+0000","last_unstale":"2023-07-14T11:56:12.705007+0000","last_undegraded":"2023-07-14T11:56:12.705007+0000","last_fullsized":"2023-07-14T11:56:12.705007+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:43:38.786676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943547+0000","last_change":"2023-07-14T11:56:11.701202+0000","last_active":"2023-07-14T11:56:12.943547+0000","last_peered":"2023-07-14T11:56:12.943547+0000","last_clean":"2023-07-14T11:56:12.943547+0000","last_became_active":"2023-07-14T11:56:11.700934+0000","last_became_peered":"2023-07-14T11:56:11.700934+0000","last_unstale":"2023-07-14T11:56:12.943547+0000","last_undegraded":"2023-07-14T11:56:12.943547+0000","last_fullsized":"2023-07-14T11:56:12.943547+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:23:01.590546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944186+0000","last_change":"2023-07-14T11:56:11.707434+0000","last_active":"2023-07-14T11:56:12.944186+0000","last_peered":"2023-07-14T11:56:12.944186+0000","last_clean":"2023-07-14T11:56:12.944186+0000","last_became_active":"2023-07-14T11:56:11.707144+0000","last_became_peered":"2023-07-14T11:56:11.707144+0000","last_unstale":"2023-07-14T11:56:12.944186+0000","last_undegraded":"2023-07-14T11:56:12.944186+0000","last_fullsized":"2023-07-14T11:56:12.944186+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:02:32.973079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943373+0000","last_change":"2023-07-14T11:56:11.700635+0000","last_active":"2023-07-14T11:56:12.943373+0000","last_peered":"2023-07-14T11:56:12.943373+0000","last_clean":"2023-07-14T11:56:12.943373+0000","last_became_active":"2023-07-14T11:56:11.700348+0000","last_became_peered":"2023-07-14T11:56:11.700348+0000","last_unstale":"2023-07-14T11:56:12.943373+0000","last_undegraded":"2023-07-14T11:56:12.943373+0000","last_fullsized":"2023-07-14T11:56:12.943373+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:49:25.320272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.942983+0000","last_change":"2023-07-14T11:56:11.710266+0000","last_active":"2023-07-14T11:56:12.942983+0000","last_peered":"2023-07-14T11:56:12.942983+0000","last_clean":"2023-07-14T11:56:12.942983+0000","last_became_active":"2023-07-14T11:56:11.709978+0000","last_became_peered":"2023-07-14T11:56:11.709978+0000","last_unstale":"2023-07-14T11:56:12.942983+0000","last_undegraded":"2023-07-14T11:56:12.942983+0000","last_fullsized":"2023-07-14T11:56:12.942983+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:12:15.794236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944171+0000","last_change":"2023-07-14T11:56:11.714523+0000","last_active":"2023-07-14T11:56:12.944171+0000","last_peered":"2023-07-14T11:56:12.944171+0000","last_clean":"2023-07-14T11:56:12.944171+0000","last_became_active":"2023-07-14T11:56:11.714260+0000","last_became_peered":"2023-07-14T11:56:11.714260+0000","last_unstale":"2023-07-14T11:56:12.944171+0000","last_undegraded":"2023-07-14T11:56:12.944171+0000","last_fullsized":"2023-07-14T11:56:12.944171+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:08:59.738123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944028+0000","last_change":"2023-07-14T11:56:11.709673+0000","last_active":"2023-07-14T11:56:12.944028+0000","last_peered":"2023-07-14T11:56:12.944028+0000","last_clean":"2023-07-14T11:56:12.944028+0000","last_became_active":"2023-07-14T11:56:11.709278+0000","last_became_peered":"2023-07-14T11:56:11.709278+0000","last_unstale":"2023-07-14T11:56:12.944028+0000","last_undegraded":"2023-07-14T11:56:12.944028+0000","last_fullsized":"2023-07-14T11:56:12.944028+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:29:34.626554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.945029+0000","last_change":"2023-07-14T11:56:11.716984+0000","last_active":"2023-07-14T11:56:12.945029+0000","last_peered":"2023-07-14T11:56:12.945029+0000","last_clean":"2023-07-14T11:56:12.945029+0000","last_became_active":"2023-07-14T11:56:11.716647+0000","last_became_peered":"2023-07-14T11:56:11.716647+0000","last_unstale":"2023-07-14T11:56:12.945029+0000","last_undegraded":"2023-07-14T11:56:12.945029+0000","last_fullsized":"2023-07-14T11:56:12.945029+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:23:01.590546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.704721+0000","last_change":"2023-07-14T11:56:11.718223+0000","last_active":"2023-07-14T11:56:12.704721+0000","last_peered":"2023-07-14T11:56:12.704721+0000","last_clean":"2023-07-14T11:56:12.704721+0000","last_became_active":"2023-07-14T11:56:11.717909+0000","last_became_peered":"2023-07-14T11:56:11.717909+0000","last_unstale":"2023-07-14T11:56:12.704721+0000","last_undegraded":"2023-07-14T11:56:12.704721+0000","last_fullsized":"2023-07-14T11:56:12.704721+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:52.137855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.942038+0000","last_change":"2023-07-14T11:56:11.706731+0000","last_active":"2023-07-14T11:56:12.942038+0000","last_peered":"2023-07-14T11:56:12.942038+0000","last_clean":"2023-07-14T11:56:12.942038+0000","last_became_active":"2023-07-14T11:56:11.706192+0000","last_became_peered":"2023-07-14T11:56:11.706192+0000","last_unstale":"2023-07-14T11:56:12.942038+0000","last_undegraded":"2023-07-14T11:56:12.942038+0000","last_fullsized":"2023-07-14T11:56:12.942038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:50:12.734874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944596+0000","last_change":"2023-07-14T11:56:11.713134+0000","last_active":"2023-07-14T11:56:12.944596+0000","last_peered":"2023-07-14T11:56:12.944596+0000","last_clean":"2023-07-14T11:56:12.944596+0000","last_became_active":"2023-07-14T11:56:11.707992+0000","last_became_peered":"2023-07-14T11:56:11.707992+0000","last_unstale":"2023-07-14T11:56:12.944596+0000","last_undegraded":"2023-07-14T11:56:12.944596+0000","last_fullsized":"2023-07-14T11:56:12.944596+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:02.516811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943548+0000","last_change":"2023-07-14T11:56:11.709728+0000","last_active":"2023-07-14T11:56:12.943548+0000","last_peered":"2023-07-14T11:56:12.943548+0000","last_clean":"2023-07-14T11:56:12.943548+0000","last_became_active":"2023-07-14T11:56:11.708719+0000","last_became_peered":"2023-07-14T11:56:11.708719+0000","last_unstale":"2023-07-14T11:56:12.943548+0000","last_undegraded":"2023-07-14T11:56:12.943548+0000","last_fullsized":"2023-07-14T11:56:12.943548+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:46:51.000848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.702723+0000","last_change":"2023-07-14T11:56:11.716599+0000","last_active":"2023-07-14T11:56:12.702723+0000","last_peered":"2023-07-14T11:56:12.702723+0000","last_clean":"2023-07-14T11:56:12.702723+0000","last_became_active":"2023-07-14T11:56:11.716289+0000","last_became_peered":"2023-07-14T11:56:11.716289+0000","last_unstale":"2023-07-14T11:56:12.702723+0000","last_undegraded":"2023-07-14T11:56:12.702723+0000","last_fullsized":"2023-07-14T11:56:12.702723+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:47:58.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943243+0000","last_change":"2023-07-14T11:56:11.716943+0000","last_active":"2023-07-14T11:56:12.943243+0000","last_peered":"2023-07-14T11:56:12.943243+0000","last_clean":"2023-07-14T11:56:12.943243+0000","last_became_active":"2023-07-14T11:56:11.716596+0000","last_became_peered":"2023-07-14T11:56:11.716596+0000","last_unstale":"2023-07-14T11:56:12.943243+0000","last_undegraded":"2023-07-14T11:56:12.943243+0000","last_fullsized":"2023-07-14T11:56:12.943243+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:09.475203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.704344+0000","last_change":"2023-07-14T11:56:11.710655+0000","last_active":"2023-07-14T11:56:12.704344+0000","last_peered":"2023-07-14T11:56:12.704344+0000","last_clean":"2023-07-14T11:56:12.704344+0000","last_became_active":"2023-07-14T11:56:11.710269+0000","last_became_peered":"2023-07-14T11:56:11.710269+0000","last_unstale":"2023-07-14T11:56:12.704344+0000","last_undegraded":"2023-07-14T11:56:12.704344+0000","last_fullsized":"2023-07-14T11:56:12.704344+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:52:10.596062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"58'4","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.942761+0000","last_change":"2023-07-14T11:56:10.696681+0000","last_active":"2023-07-14T11:56:12.942761+0000","last_peered":"2023-07-14T11:56:12.942761+0000","last_clean":"2023-07-14T11:56:12.942761+0000","last_became_active":"2023-07-14T11:56:10.696413+0000","last_became_peered":"2023-07-14T11:56:10.696413+0000","last_unstale":"2023-07-14T11:56:12.942761+0000","last_undegraded":"2023-07-14T11:56:12.942761+0000","last_fullsized":"2023-07-14T11:56:12.942761+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:58:12.249265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.704780+0000","last_change":"2023-07-14T11:56:11.706375+0000","last_active":"2023-07-14T11:56:12.704780+0000","last_peered":"2023-07-14T11:56:12.704780+0000","last_clean":"2023-07-14T11:56:12.704780+0000","last_became_active":"2023-07-14T11:56:11.706064+0000","last_became_peered":"2023-07-14T11:56:11.706064+0000","last_unstale":"2023-07-14T11:56:12.704780+0000","last_undegraded":"2023-07-14T11:56:12.704780+0000","last_fullsized":"2023-07-14T11:56:12.704780+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:54:39.223064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"58'1","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944861+0000","last_change":"2023-07-14T11:56:10.694408+0000","last_active":"2023-07-14T11:56:12.944861+0000","last_peered":"2023-07-14T11:56:12.944861+0000","last_clean":"2023-07-14T11:56:12.944861+0000","last_became_active":"2023-07-14T11:56:10.694097+0000","last_became_peered":"2023-07-14T11:56:10.694097+0000","last_unstale":"2023-07-14T11:56:12.944861+0000","last_undegraded":"2023-07-14T11:56:12.944861+0000","last_fullsized":"2023-07-14T11:56:12.944861+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:32:41.756270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944857+0000","last_change":"2023-07-14T11:56:11.716406+0000","last_active":"2023-07-14T11:56:12.944857+0000","last_peered":"2023-07-14T11:56:12.944857+0000","last_clean":"2023-07-14T11:56:12.944857+0000","last_became_active":"2023-07-14T11:56:11.716085+0000","last_became_peered":"2023-07-14T11:56:11.716085+0000","last_unstale":"2023-07-14T11:56:12.944857+0000","last_undegraded":"2023-07-14T11:56:12.944857+0000","last_fullsized":"2023-07-14T11:56:12.944857+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:52.137855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"58'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944585+0000","last_change":"2023-07-14T11:56:10.703709+0000","last_active":"2023-07-14T11:56:12.944585+0000","last_peered":"2023-07-14T11:56:12.944585+0000","last_clean":"2023-07-14T11:56:12.944585+0000","last_became_active":"2023-07-14T11:56:10.703444+0000","last_became_peered":"2023-07-14T11:56:10.703444+0000","last_unstale":"2023-07-14T11:56:12.944585+0000","last_undegraded":"2023-07-14T11:56:12.944585+0000","last_fullsized":"2023-07-14T11:56:12.944585+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:00:32.047458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944983+0000","last_change":"2023-07-14T11:56:11.721449+0000","last_active":"2023-07-14T11:56:12.944983+0000","last_peered":"2023-07-14T11:56:12.944983+0000","last_clean":"2023-07-14T11:56:12.944983+0000","last_became_active":"2023-07-14T11:56:11.721159+0000","last_became_peered":"2023-07-14T11:56:11.721159+0000","last_unstale":"2023-07-14T11:56:12.944983+0000","last_undegraded":"2023-07-14T11:56:12.944983+0000","last_fullsized":"2023-07-14T11:56:12.944983+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:29:46.926992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"58'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944674+0000","last_change":"2023-07-14T11:56:10.701563+0000","last_active":"2023-07-14T11:56:12.944674+0000","last_peered":"2023-07-14T11:56:12.944674+0000","last_clean":"2023-07-14T11:56:12.944674+0000","last_became_active":"2023-07-14T11:56:10.701262+0000","last_became_peered":"2023-07-14T11:56:10.701262+0000","last_unstale":"2023-07-14T11:56:12.944674+0000","last_undegraded":"2023-07-14T11:56:12.944674+0000","last_fullsized":"2023-07-14T11:56:12.944674+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:18:32.903293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.699687+0000","last_change":"2023-07-14T11:56:11.706248+0000","last_active":"2023-07-14T11:56:12.699687+0000","last_peered":"2023-07-14T11:56:12.699687+0000","last_clean":"2023-07-14T11:56:12.699687+0000","last_became_active":"2023-07-14T11:56:11.705979+0000","last_became_peered":"2023-07-14T11:56:11.705979+0000","last_unstale":"2023-07-14T11:56:12.699687+0000","last_undegraded":"2023-07-14T11:56:12.699687+0000","last_fullsized":"2023-07-14T11:56:12.699687+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:58:55.839807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"58'1","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943497+0000","last_change":"2023-07-14T11:56:10.694726+0000","last_active":"2023-07-14T11:56:12.943497+0000","last_peered":"2023-07-14T11:56:12.943497+0000","last_clean":"2023-07-14T11:56:12.943497+0000","last_became_active":"2023-07-14T11:56:10.694444+0000","last_became_peered":"2023-07-14T11:56:10.694444+0000","last_unstale":"2023-07-14T11:56:12.943497+0000","last_undegraded":"2023-07-14T11:56:12.943497+0000","last_fullsized":"2023-07-14T11:56:12.943497+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:49:29.030124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.941885+0000","last_change":"2023-07-14T11:56:11.716238+0000","last_active":"2023-07-14T11:56:12.941885+0000","last_peered":"2023-07-14T11:56:12.941885+0000","last_clean":"2023-07-14T11:56:12.941885+0000","last_became_active":"2023-07-14T11:56:11.715946+0000","last_became_peered":"2023-07-14T11:56:11.715946+0000","last_unstale":"2023-07-14T11:56:12.941885+0000","last_undegraded":"2023-07-14T11:56:12.941885+0000","last_fullsized":"2023-07-14T11:56:12.941885+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:52:14.767702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"58'1","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943309+0000","last_change":"2023-07-14T11:56:10.689607+0000","last_active":"2023-07-14T11:56:12.943309+0000","last_peered":"2023-07-14T11:56:12.943309+0000","last_clean":"2023-07-14T11:56:12.943309+0000","last_became_active":"2023-07-14T11:56:10.689345+0000","last_became_peered":"2023-07-14T11:56:10.689345+0000","last_unstale":"2023-07-14T11:56:12.943309+0000","last_undegraded":"2023-07-14T11:56:12.943309+0000","last_fullsized":"2023-07-14T11:56:12.943309+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:02:31.965846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.702693+0000","last_change":"2023-07-14T11:56:11.713450+0000","last_active":"2023-07-14T11:56:12.702693+0000","last_peered":"2023-07-14T11:56:12.702693+0000","last_clean":"2023-07-14T11:56:12.702693+0000","last_became_active":"2023-07-14T11:56:11.713005+0000","last_became_peered":"2023-07-14T11:56:11.713005+0000","last_unstale":"2023-07-14T11:56:12.702693+0000","last_undegraded":"2023-07-14T11:56:12.702693+0000","last_fullsized":"2023-07-14T11:56:12.702693+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:15:58.805650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943085+0000","last_change":"2023-07-14T11:56:10.691388+0000","last_active":"2023-07-14T11:56:12.943085+0000","last_peered":"2023-07-14T11:56:12.943085+0000","last_clean":"2023-07-14T11:56:12.943085+0000","last_became_active":"2023-07-14T11:56:10.690746+0000","last_became_peered":"2023-07-14T11:56:10.690746+0000","last_unstale":"2023-07-14T11:56:12.943085+0000","last_undegraded":"2023-07-14T11:56:12.943085+0000","last_fullsized":"2023-07-14T11:56:12.943085+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:47:54.946513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943438+0000","last_change":"2023-07-14T11:56:11.707804+0000","last_active":"2023-07-14T11:56:12.943438+0000","last_peered":"2023-07-14T11:56:12.943438+0000","last_clean":"2023-07-14T11:56:12.943438+0000","last_became_active":"2023-07-14T11:56:11.707534+0000","last_became_peered":"2023-07-14T11:56:11.707534+0000","last_unstale":"2023-07-14T11:56:12.943438+0000","last_undegraded":"2023-07-14T11:56:12.943438+0000","last_fullsized":"2023-07-14T11:56:12.943438+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:47:58.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.702829+0000","last_change":"2023-07-14T11:56:10.694837+0000","last_active":"2023-07-14T11:56:12.702829+0000","last_peered":"2023-07-14T11:56:12.702829+0000","last_clean":"2023-07-14T11:56:12.702829+0000","last_became_active":"2023-07-14T11:56:10.694071+0000","last_became_peered":"2023-07-14T11:56:10.694071+0000","last_unstale":"2023-07-14T11:56:12.702829+0000","last_undegraded":"2023-07-14T11:56:12.702829+0000","last_fullsized":"2023-07-14T11:56:12.702829+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:57:28.627548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.942298+0000","last_change":"2023-07-14T11:56:11.706522+0000","last_active":"2023-07-14T11:56:12.942298+0000","last_peered":"2023-07-14T11:56:12.942298+0000","last_clean":"2023-07-14T11:56:12.942298+0000","last_became_active":"2023-07-14T11:56:11.705871+0000","last_became_peered":"2023-07-14T11:56:11.705871+0000","last_unstale":"2023-07-14T11:56:12.942298+0000","last_undegraded":"2023-07-14T11:56:12.942298+0000","last_fullsized":"2023-07-14T11:56:12.942298+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:47:52.865464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.945285+0000","last_change":"2023-07-14T11:56:10.705201+0000","last_active":"2023-07-14T11:56:12.945285+0000","last_peered":"2023-07-14T11:56:12.945285+0000","last_clean":"2023-07-14T11:56:12.945285+0000","last_became_active":"2023-07-14T11:56:10.704922+0000","last_became_peered":"2023-07-14T11:56:10.704922+0000","last_unstale":"2023-07-14T11:56:12.945285+0000","last_undegraded":"2023-07-14T11:56:12.945285+0000","last_fullsized":"2023-07-14T11:56:12.945285+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:01.327056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943653+0000","last_change":"2023-07-14T11:56:11.710243+0000","last_active":"2023-07-14T11:56:12.943653+0000","last_peered":"2023-07-14T11:56:12.943653+0000","last_clean":"2023-07-14T11:56:12.943653+0000","last_became_active":"2023-07-14T11:56:11.709968+0000","last_became_peered":"2023-07-14T11:56:11.709968+0000","last_unstale":"2023-07-14T11:56:12.943653+0000","last_undegraded":"2023-07-14T11:56:12.943653+0000","last_fullsized":"2023-07-14T11:56:12.943653+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:50:04.043370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.704351+0000","last_change":"2023-07-14T11:56:10.692503+0000","last_active":"2023-07-14T11:56:12.704351+0000","last_peered":"2023-07-14T11:56:12.704351+0000","last_clean":"2023-07-14T11:56:12.704351+0000","last_became_active":"2023-07-14T11:56:10.692092+0000","last_became_peered":"2023-07-14T11:56:10.692092+0000","last_unstale":"2023-07-14T11:56:12.704351+0000","last_undegraded":"2023-07-14T11:56:12.704351+0000","last_fullsized":"2023-07-14T11:56:12.704351+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:18:32.903293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.704679+0000","last_change":"2023-07-14T11:56:11.711554+0000","last_active":"2023-07-14T11:56:12.704679+0000","last_peered":"2023-07-14T11:56:12.704679+0000","last_clean":"2023-07-14T11:56:12.704679+0000","last_became_active":"2023-07-14T11:56:11.711321+0000","last_became_peered":"2023-07-14T11:56:11.711321+0000","last_unstale":"2023-07-14T11:56:12.704679+0000","last_undegraded":"2023-07-14T11:56:12.704679+0000","last_fullsized":"2023-07-14T11:56:12.704679+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:16:04.455715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"58'3","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944861+0000","last_change":"2023-07-14T11:56:10.703832+0000","last_active":"2023-07-14T11:56:12.944861+0000","last_peered":"2023-07-14T11:56:12.944861+0000","last_clean":"2023-07-14T11:56:12.944861+0000","last_became_active":"2023-07-14T11:56:10.703548+0000","last_became_peered":"2023-07-14T11:56:10.703548+0000","last_unstale":"2023-07-14T11:56:12.944861+0000","last_undegraded":"2023-07-14T11:56:12.944861+0000","last_fullsized":"2023-07-14T11:56:12.944861+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:57:39.015990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.704867+0000","last_change":"2023-07-14T11:56:11.717390+0000","last_active":"2023-07-14T11:56:12.704867+0000","last_peered":"2023-07-14T11:56:12.704867+0000","last_clean":"2023-07-14T11:56:12.704867+0000","last_became_active":"2023-07-14T11:56:11.714980+0000","last_became_peered":"2023-07-14T11:56:11.714980+0000","last_unstale":"2023-07-14T11:56:12.704867+0000","last_undegraded":"2023-07-14T11:56:12.704867+0000","last_fullsized":"2023-07-14T11:56:12.704867+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:37:58.624385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"58'6","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.942578+0000","last_change":"2023-07-14T11:56:10.697093+0000","last_active":"2023-07-14T11:56:12.942578+0000","last_peered":"2023-07-14T11:56:12.942578+0000","last_clean":"2023-07-14T11:56:12.942578+0000","last_became_active":"2023-07-14T11:56:10.692072+0000","last_became_peered":"2023-07-14T11:56:10.692072+0000","last_unstale":"2023-07-14T11:56:12.942578+0000","last_undegraded":"2023-07-14T11:56:12.942578+0000","last_fullsized":"2023-07-14T11:56:12.942578+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:54:38.891893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.702737+0000","last_change":"2023-07-14T11:56:11.714620+0000","last_active":"2023-07-14T11:56:12.702737+0000","last_peered":"2023-07-14T11:56:12.702737+0000","last_clean":"2023-07-14T11:56:12.702737+0000","last_became_active":"2023-07-14T11:56:11.714349+0000","last_became_peered":"2023-07-14T11:56:11.714349+0000","last_unstale":"2023-07-14T11:56:12.702737+0000","last_undegraded":"2023-07-14T11:56:12.702737+0000","last_fullsized":"2023-07-14T11:56:12.702737+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:23:01.590546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"58'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944948+0000","last_change":"2023-07-14T11:56:10.701400+0000","last_active":"2023-07-14T11:56:12.944948+0000","last_peered":"2023-07-14T11:56:12.944948+0000","last_clean":"2023-07-14T11:56:12.944948+0000","last_became_active":"2023-07-14T11:56:10.701104+0000","last_became_peered":"2023-07-14T11:56:10.701104+0000","last_unstale":"2023-07-14T11:56:12.944948+0000","last_undegraded":"2023-07-14T11:56:12.944948+0000","last_fullsized":"2023-07-14T11:56:12.944948+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:41:34.762154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943483+0000","last_change":"2023-07-14T11:56:11.716175+0000","last_active":"2023-07-14T11:56:12.943483+0000","last_peered":"2023-07-14T11:56:12.943483+0000","last_clean":"2023-07-14T11:56:12.943483+0000","last_became_active":"2023-07-14T11:56:11.715905+0000","last_became_peered":"2023-07-14T11:56:11.715905+0000","last_unstale":"2023-07-14T11:56:12.943483+0000","last_undegraded":"2023-07-14T11:56:12.943483+0000","last_fullsized":"2023-07-14T11:56:12.943483+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:54:23.269580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943738+0000","last_change":"2023-07-14T11:56:10.702066+0000","last_active":"2023-07-14T11:56:12.943738+0000","last_peered":"2023-07-14T11:56:12.943738+0000","last_clean":"2023-07-14T11:56:12.943738+0000","last_became_active":"2023-07-14T11:56:10.701720+0000","last_became_peered":"2023-07-14T11:56:10.701720+0000","last_unstale":"2023-07-14T11:56:12.943738+0000","last_undegraded":"2023-07-14T11:56:12.943738+0000","last_fullsized":"2023-07-14T11:56:12.943738+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:24:41.759227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.702931+0000","last_change":"2023-07-14T11:56:11.693383+0000","last_active":"2023-07-14T11:56:12.702931+0000","last_peered":"2023-07-14T11:56:12.702931+0000","last_clean":"2023-07-14T11:56:12.702931+0000","last_became_active":"2023-07-14T11:56:11.693089+0000","last_became_peered":"2023-07-14T11:56:11.693089+0000","last_unstale":"2023-07-14T11:56:12.702931+0000","last_undegraded":"2023-07-14T11:56:12.702931+0000","last_fullsized":"2023-07-14T11:56:12.702931+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:11:49.281376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.945390+0000","last_change":"2023-07-14T11:56:10.699176+0000","last_active":"2023-07-14T11:56:12.945390+0000","last_peered":"2023-07-14T11:56:12.945390+0000","last_clean":"2023-07-14T11:56:12.945390+0000","last_became_active":"2023-07-14T11:56:10.698881+0000","last_became_peered":"2023-07-14T11:56:10.698881+0000","last_unstale":"2023-07-14T11:56:12.945390+0000","last_undegraded":"2023-07-14T11:56:12.945390+0000","last_fullsized":"2023-07-14T11:56:12.945390+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:17:28.732628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.942260+0000","last_change":"2023-07-14T11:56:11.712920+0000","last_active":"2023-07-14T11:56:12.942260+0000","last_peered":"2023-07-14T11:56:12.942260+0000","last_clean":"2023-07-14T11:56:12.942260+0000","last_became_active":"2023-07-14T11:56:11.712668+0000","last_became_peered":"2023-07-14T11:56:11.712668+0000","last_unstale":"2023-07-14T11:56:12.942260+0000","last_undegraded":"2023-07-14T11:56:12.942260+0000","last_fullsized":"2023-07-14T11:56:12.942260+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:16:08.125044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"58'8","reported_seq":21,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.703039+0000","last_change":"2023-07-14T11:56:10.695120+0000","last_active":"2023-07-14T11:56:12.703039+0000","last_peered":"2023-07-14T11:56:12.703039+0000","last_clean":"2023-07-14T11:56:12.703039+0000","last_became_active":"2023-07-14T11:56:10.693407+0000","last_became_peered":"2023-07-14T11:56:10.693407+0000","last_unstale":"2023-07-14T11:56:12.703039+0000","last_undegraded":"2023-07-14T11:56:12.703039+0000","last_fullsized":"2023-07-14T11:56:12.703039+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:49:29.030124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944523+0000","last_change":"2023-07-14T11:56:11.715932+0000","last_active":"2023-07-14T11:56:12.944523+0000","last_peered":"2023-07-14T11:56:12.944523+0000","last_clean":"2023-07-14T11:56:12.944523+0000","last_became_active":"2023-07-14T11:56:11.715650+0000","last_became_peered":"2023-07-14T11:56:11.715650+0000","last_unstale":"2023-07-14T11:56:12.944523+0000","last_undegraded":"2023-07-14T11:56:12.944523+0000","last_fullsized":"2023-07-14T11:56:12.944523+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:48:42.232955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"58'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944661+0000","last_change":"2023-07-14T11:56:10.700844+0000","last_active":"2023-07-14T11:56:12.944661+0000","last_peered":"2023-07-14T11:56:12.944661+0000","last_clean":"2023-07-14T11:56:12.944661+0000","last_became_active":"2023-07-14T11:56:10.700534+0000","last_became_peered":"2023-07-14T11:56:10.700534+0000","last_unstale":"2023-07-14T11:56:12.944661+0000","last_undegraded":"2023-07-14T11:56:12.944661+0000","last_fullsized":"2023-07-14T11:56:12.944661+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:47:57.078535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.699794+0000","last_change":"2023-07-14T11:56:11.710832+0000","last_active":"2023-07-14T11:56:12.699794+0000","last_peered":"2023-07-14T11:56:12.699794+0000","last_clean":"2023-07-14T11:56:12.699794+0000","last_became_active":"2023-07-14T11:56:11.710587+0000","last_became_peered":"2023-07-14T11:56:11.710587+0000","last_unstale":"2023-07-14T11:56:12.699794+0000","last_undegraded":"2023-07-14T11:56:12.699794+0000","last_fullsized":"2023-07-14T11:56:12.699794+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:29:56.901106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"59'1","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.945657+0000","last_change":"2023-07-14T11:56:10.695559+0000","last_active":"2023-07-14T11:56:12.945657+0000","last_peered":"2023-07-14T11:56:12.945657+0000","last_clean":"2023-07-14T11:56:12.945657+0000","last_became_active":"2023-07-14T11:56:10.695312+0000","last_became_peered":"2023-07-14T11:56:10.695312+0000","last_unstale":"2023-07-14T11:56:12.945657+0000","last_undegraded":"2023-07-14T11:56:12.945657+0000","last_fullsized":"2023-07-14T11:56:12.945657+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:50:03.036137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.945350+0000","last_change":"2023-07-14T11:56:11.716817+0000","last_active":"2023-07-14T11:56:12.945350+0000","last_peered":"2023-07-14T11:56:12.945350+0000","last_clean":"2023-07-14T11:56:12.945350+0000","last_became_active":"2023-07-14T11:56:11.716543+0000","last_became_peered":"2023-07-14T11:56:11.716543+0000","last_unstale":"2023-07-14T11:56:12.945350+0000","last_undegraded":"2023-07-14T11:56:12.945350+0000","last_fullsized":"2023-07-14T11:56:12.945350+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:49:30.037357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944586+0000","last_change":"2023-07-14T11:56:10.703244+0000","last_active":"2023-07-14T11:56:12.944586+0000","last_peered":"2023-07-14T11:56:12.944586+0000","last_clean":"2023-07-14T11:56:12.944586+0000","last_became_active":"2023-07-14T11:56:10.702957+0000","last_became_peered":"2023-07-14T11:56:10.702957+0000","last_unstale":"2023-07-14T11:56:12.944586+0000","last_undegraded":"2023-07-14T11:56:12.944586+0000","last_fullsized":"2023-07-14T11:56:12.944586+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:53:30.378748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.704905+0000","last_change":"2023-07-14T11:56:11.700912+0000","last_active":"2023-07-14T11:56:12.704905+0000","last_peered":"2023-07-14T11:56:12.704905+0000","last_clean":"2023-07-14T11:56:12.704905+0000","last_became_active":"2023-07-14T11:56:11.700629+0000","last_became_peered":"2023-07-14T11:56:11.700629+0000","last_unstale":"2023-07-14T11:56:12.704905+0000","last_undegraded":"2023-07-14T11:56:12.704905+0000","last_fullsized":"2023-07-14T11:56:12.704905+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:25:52.857851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.942813+0000","last_change":"2023-07-14T11:56:10.695223+0000","last_active":"2023-07-14T11:56:12.942813+0000","last_peered":"2023-07-14T11:56:12.942813+0000","last_clean":"2023-07-14T11:56:12.942813+0000","last_became_active":"2023-07-14T11:56:10.691638+0000","last_became_peered":"2023-07-14T11:56:10.691638+0000","last_unstale":"2023-07-14T11:56:12.942813+0000","last_undegraded":"2023-07-14T11:56:12.942813+0000","last_fullsized":"2023-07-14T11:56:12.942813+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:16:44.780397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.702532+0000","last_change":"2023-07-14T11:56:11.706479+0000","last_active":"2023-07-14T11:56:12.702532+0000","last_peered":"2023-07-14T11:56:12.702532+0000","last_clean":"2023-07-14T11:56:12.702532+0000","last_became_active":"2023-07-14T11:56:11.706200+0000","last_became_peered":"2023-07-14T11:56:11.706200+0000","last_unstale":"2023-07-14T11:56:12.702532+0000","last_undegraded":"2023-07-14T11:56:12.702532+0000","last_fullsized":"2023-07-14T11:56:12.702532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:06:06.500898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.945096+0000","last_change":"2023-07-14T11:56:10.699836+0000","last_active":"2023-07-14T11:56:12.945096+0000","last_peered":"2023-07-14T11:56:12.945096+0000","last_clean":"2023-07-14T11:56:12.945096+0000","last_became_active":"2023-07-14T11:56:10.699528+0000","last_became_peered":"2023-07-14T11:56:10.699528+0000","last_unstale":"2023-07-14T11:56:12.945096+0000","last_undegraded":"2023-07-14T11:56:12.945096+0000","last_fullsized":"2023-07-14T11:56:12.945096+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:58:15.607086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.703355+0000","last_change":"2023-07-14T11:56:11.710504+0000","last_active":"2023-07-14T11:56:12.703355+0000","last_peered":"2023-07-14T11:56:12.703355+0000","last_clean":"2023-07-14T11:56:12.703355+0000","last_became_active":"2023-07-14T11:56:11.710251+0000","last_became_peered":"2023-07-14T11:56:11.710251+0000","last_unstale":"2023-07-14T11:56:12.703355+0000","last_undegraded":"2023-07-14T11:56:12.703355+0000","last_fullsized":"2023-07-14T11:56:12.703355+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:50:04.043370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943423+0000","last_change":"2023-07-14T11:56:10.693503+0000","last_active":"2023-07-14T11:56:12.943423+0000","last_peered":"2023-07-14T11:56:12.943423+0000","last_clean":"2023-07-14T11:56:12.943423+0000","last_became_active":"2023-07-14T11:56:10.693197+0000","last_became_peered":"2023-07-14T11:56:10.693197+0000","last_unstale":"2023-07-14T11:56:12.943423+0000","last_undegraded":"2023-07-14T11:56:12.943423+0000","last_fullsized":"2023-07-14T11:56:12.943423+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:08.467969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.942440+0000","last_change":"2023-07-14T11:56:11.716455+0000","last_active":"2023-07-14T11:56:12.942440+0000","last_peered":"2023-07-14T11:56:12.942440+0000","last_clean":"2023-07-14T11:56:12.942440+0000","last_became_active":"2023-07-14T11:56:11.716038+0000","last_became_peered":"2023-07-14T11:56:11.716038+0000","last_unstale":"2023-07-14T11:56:12.942440+0000","last_undegraded":"2023-07-14T11:56:12.942440+0000","last_fullsized":"2023-07-14T11:56:12.942440+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:08:17.319747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943127+0000","last_change":"2023-07-14T11:56:10.691758+0000","last_active":"2023-07-14T11:56:12.943127+0000","last_peered":"2023-07-14T11:56:12.943127+0000","last_clean":"2023-07-14T11:56:12.943127+0000","last_became_active":"2023-07-14T11:56:10.691130+0000","last_became_peered":"2023-07-14T11:56:10.691130+0000","last_unstale":"2023-07-14T11:56:12.943127+0000","last_undegraded":"2023-07-14T11:56:12.943127+0000","last_fullsized":"2023-07-14T11:56:12.943127+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:07:44.412975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.699686+0000","last_change":"2023-07-14T11:56:11.708606+0000","last_active":"2023-07-14T11:56:12.699686+0000","last_peered":"2023-07-14T11:56:12.699686+0000","last_clean":"2023-07-14T11:56:12.699686+0000","last_became_active":"2023-07-14T11:56:11.708298+0000","last_became_peered":"2023-07-14T11:56:11.708298+0000","last_unstale":"2023-07-14T11:56:12.699686+0000","last_undegraded":"2023-07-14T11:56:12.699686+0000","last_fullsized":"2023-07-14T11:56:12.699686+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:35:10.225230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.941796+0000","last_change":"2023-07-14T11:56:10.692036+0000","last_active":"2023-07-14T11:56:12.941796+0000","last_peered":"2023-07-14T11:56:12.941796+0000","last_clean":"2023-07-14T11:56:12.941796+0000","last_became_active":"2023-07-14T11:56:10.691608+0000","last_became_peered":"2023-07-14T11:56:10.691608+0000","last_unstale":"2023-07-14T11:56:12.941796+0000","last_undegraded":"2023-07-14T11:56:12.941796+0000","last_fullsized":"2023-07-14T11:56:12.941796+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:40:29.929215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.705195+0000","last_change":"2023-07-14T11:56:11.700833+0000","last_active":"2023-07-14T11:56:12.705195+0000","last_peered":"2023-07-14T11:56:12.705195+0000","last_clean":"2023-07-14T11:56:12.705195+0000","last_became_active":"2023-07-14T11:56:11.700563+0000","last_became_peered":"2023-07-14T11:56:11.700563+0000","last_unstale":"2023-07-14T11:56:12.705195+0000","last_undegraded":"2023-07-14T11:56:12.705195+0000","last_fullsized":"2023-07-14T11:56:12.705195+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:06:55.466075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943718+0000","last_change":"2023-07-14T11:56:10.690467+0000","last_active":"2023-07-14T11:56:12.943718+0000","last_peered":"2023-07-14T11:56:12.943718+0000","last_clean":"2023-07-14T11:56:12.943718+0000","last_became_active":"2023-07-14T11:56:10.690201+0000","last_became_peered":"2023-07-14T11:56:10.690201+0000","last_unstale":"2023-07-14T11:56:12.943718+0000","last_undegraded":"2023-07-14T11:56:12.943718+0000","last_fullsized":"2023-07-14T11:56:12.943718+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:48:41.225721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":90,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.705097+0000","last_change":"2023-07-14T11:53:09.145201+0000","last_active":"2023-07-14T11:56:12.705097+0000","last_peered":"2023-07-14T11:56:12.705097+0000","last_clean":"2023-07-14T11:56:12.705097+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:56:12.705097+0000","last_undegraded":"2023-07-14T11:56:12.705097+0000","last_fullsized":"2023-07-14T11:56:12.705097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:27:53.079195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.704242+0000","last_change":"2023-07-14T11:56:11.714310+0000","last_active":"2023-07-14T11:56:12.704242+0000","last_peered":"2023-07-14T11:56:12.704242+0000","last_clean":"2023-07-14T11:56:12.704242+0000","last_became_active":"2023-07-14T11:56:11.713719+0000","last_became_peered":"2023-07-14T11:56:11.713719+0000","last_unstale":"2023-07-14T11:56:12.704242+0000","last_undegraded":"2023-07-14T11:56:12.704242+0000","last_fullsized":"2023-07-14T11:56:12.704242+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:00:33.054691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"58'1","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.945024+0000","last_change":"2023-07-14T11:56:10.704116+0000","last_active":"2023-07-14T11:56:12.945024+0000","last_peered":"2023-07-14T11:56:12.945024+0000","last_clean":"2023-07-14T11:56:12.945024+0000","last_became_active":"2023-07-14T11:56:10.703775+0000","last_became_peered":"2023-07-14T11:56:10.703775+0000","last_unstale":"2023-07-14T11:56:12.945024+0000","last_undegraded":"2023-07-14T11:56:12.945024+0000","last_fullsized":"2023-07-14T11:56:12.945024+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:23:30.211598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":78,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.942332+0000","last_change":"2023-07-14T11:53:09.146150+0000","last_active":"2023-07-14T11:56:12.942332+0000","last_peered":"2023-07-14T11:56:12.942332+0000","last_clean":"2023-07-14T11:56:12.942332+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:56:12.942332+0000","last_undegraded":"2023-07-14T11:56:12.942332+0000","last_fullsized":"2023-07-14T11:56:12.942332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:16:52.395406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.705042+0000","last_change":"2023-07-14T11:56:11.717611+0000","last_active":"2023-07-14T11:56:12.705042+0000","last_peered":"2023-07-14T11:56:12.705042+0000","last_clean":"2023-07-14T11:56:12.705042+0000","last_became_active":"2023-07-14T11:56:11.717346+0000","last_became_peered":"2023-07-14T11:56:11.717346+0000","last_unstale":"2023-07-14T11:56:12.705042+0000","last_undegraded":"2023-07-14T11:56:12.705042+0000","last_fullsized":"2023-07-14T11:56:12.705042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:48:19.851421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"58'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.945606+0000","last_change":"2023-07-14T11:56:10.700575+0000","last_active":"2023-07-14T11:56:12.945606+0000","last_peered":"2023-07-14T11:56:12.945606+0000","last_clean":"2023-07-14T11:56:12.945606+0000","last_became_active":"2023-07-14T11:56:10.700293+0000","last_became_peered":"2023-07-14T11:56:10.700293+0000","last_unstale":"2023-07-14T11:56:12.945606+0000","last_undegraded":"2023-07-14T11:56:12.945606+0000","last_fullsized":"2023-07-14T11:56:12.945606+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:01:25.510366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":76,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.945161+0000","last_change":"2023-07-14T11:53:09.435876+0000","last_active":"2023-07-14T11:56:12.945161+0000","last_peered":"2023-07-14T11:56:12.945161+0000","last_clean":"2023-07-14T11:56:12.945161+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:56:12.945161+0000","last_undegraded":"2023-07-14T11:56:12.945161+0000","last_fullsized":"2023-07-14T11:56:12.945161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:57:09.464720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944865+0000","last_change":"2023-07-14T11:56:11.713300+0000","last_active":"2023-07-14T11:56:12.944865+0000","last_peered":"2023-07-14T11:56:12.944865+0000","last_clean":"2023-07-14T11:56:12.944865+0000","last_became_active":"2023-07-14T11:56:11.711769+0000","last_became_peered":"2023-07-14T11:56:11.711769+0000","last_unstale":"2023-07-14T11:56:12.944865+0000","last_undegraded":"2023-07-14T11:56:12.944865+0000","last_fullsized":"2023-07-14T11:56:12.944865+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:10:34.711692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"58'1","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944376+0000","last_change":"2023-07-14T11:56:10.702843+0000","last_active":"2023-07-14T11:56:12.944376+0000","last_peered":"2023-07-14T11:56:12.944376+0000","last_clean":"2023-07-14T11:56:12.944376+0000","last_became_active":"2023-07-14T11:56:10.702583+0000","last_became_peered":"2023-07-14T11:56:10.702583+0000","last_unstale":"2023-07-14T11:56:12.944376+0000","last_undegraded":"2023-07-14T11:56:12.944376+0000","last_fullsized":"2023-07-14T11:56:12.944376+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:08.467969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":78,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943012+0000","last_change":"2023-07-14T11:53:09.434255+0000","last_active":"2023-07-14T11:56:12.943012+0000","last_peered":"2023-07-14T11:56:12.943012+0000","last_clean":"2023-07-14T11:56:12.943012+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:56:12.943012+0000","last_undegraded":"2023-07-14T11:56:12.943012+0000","last_fullsized":"2023-07-14T11:56:12.943012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:51:41.947637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.703785+0000","last_change":"2023-07-14T11:56:11.713251+0000","last_active":"2023-07-14T11:56:12.703785+0000","last_peered":"2023-07-14T11:56:12.703785+0000","last_clean":"2023-07-14T11:56:12.703785+0000","last_became_active":"2023-07-14T11:56:11.712665+0000","last_became_peered":"2023-07-14T11:56:11.712665+0000","last_unstale":"2023-07-14T11:56:12.703785+0000","last_undegraded":"2023-07-14T11:56:12.703785+0000","last_fullsized":"2023-07-14T11:56:12.703785+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:39:24.544557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944151+0000","last_change":"2023-07-14T11:56:10.697670+0000","last_active":"2023-07-14T11:56:12.944151+0000","last_peered":"2023-07-14T11:56:12.944151+0000","last_clean":"2023-07-14T11:56:12.944151+0000","last_became_active":"2023-07-14T11:56:10.697406+0000","last_became_peered":"2023-07-14T11:56:10.697406+0000","last_unstale":"2023-07-14T11:56:12.944151+0000","last_undegraded":"2023-07-14T11:56:12.944151+0000","last_fullsized":"2023-07-14T11:56:12.944151+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:23:00.583312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":88,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944063+0000","last_change":"2023-07-14T11:53:09.147040+0000","last_active":"2023-07-14T11:56:12.944063+0000","last_peered":"2023-07-14T11:56:12.944063+0000","last_clean":"2023-07-14T11:56:12.944063+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:56:12.944063+0000","last_undegraded":"2023-07-14T11:56:12.944063+0000","last_fullsized":"2023-07-14T11:56:12.944063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:04:42.879586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943792+0000","last_change":"2023-07-14T11:56:11.711855+0000","last_active":"2023-07-14T11:56:12.943792+0000","last_peered":"2023-07-14T11:56:12.943792+0000","last_clean":"2023-07-14T11:56:12.943792+0000","last_became_active":"2023-07-14T11:56:11.711585+0000","last_became_peered":"2023-07-14T11:56:11.711585+0000","last_unstale":"2023-07-14T11:56:12.943792+0000","last_undegraded":"2023-07-14T11:56:12.943792+0000","last_fullsized":"2023-07-14T11:56:12.943792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:22:29.379985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943849+0000","last_change":"2023-07-14T11:56:10.701923+0000","last_active":"2023-07-14T11:56:12.943849+0000","last_peered":"2023-07-14T11:56:12.943849+0000","last_clean":"2023-07-14T11:56:12.943849+0000","last_became_active":"2023-07-14T11:56:10.701351+0000","last_became_peered":"2023-07-14T11:56:10.701351+0000","last_unstale":"2023-07-14T11:56:12.943849+0000","last_undegraded":"2023-07-14T11:56:12.943849+0000","last_fullsized":"2023-07-14T11:56:12.943849+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:12:14.787003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":90,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.699582+0000","last_change":"2023-07-14T11:53:09.619294+0000","last_active":"2023-07-14T11:56:12.699582+0000","last_peered":"2023-07-14T11:56:12.699582+0000","last_clean":"2023-07-14T11:56:12.699582+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:56:12.699582+0000","last_undegraded":"2023-07-14T11:56:12.699582+0000","last_fullsized":"2023-07-14T11:56:12.699582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:02:06.466184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.943581+0000","last_change":"2023-07-14T11:56:11.716859+0000","last_active":"2023-07-14T11:56:12.943581+0000","last_peered":"2023-07-14T11:56:12.943581+0000","last_clean":"2023-07-14T11:56:12.943581+0000","last_became_active":"2023-07-14T11:56:11.716262+0000","last_became_peered":"2023-07-14T11:56:11.716262+0000","last_unstale":"2023-07-14T11:56:12.943581+0000","last_undegraded":"2023-07-14T11:56:12.943581+0000","last_fullsized":"2023-07-14T11:56:12.943581+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:47:58.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"58'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944294+0000","last_change":"2023-07-14T11:56:10.693854+0000","last_active":"2023-07-14T11:56:12.944294+0000","last_peered":"2023-07-14T11:56:12.944294+0000","last_clean":"2023-07-14T11:56:12.944294+0000","last_became_active":"2023-07-14T11:56:10.693588+0000","last_became_peered":"2023-07-14T11:56:10.693588+0000","last_unstale":"2023-07-14T11:56:12.944294+0000","last_undegraded":"2023-07-14T11:56:12.944294+0000","last_fullsized":"2023-07-14T11:56:12.944294+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:49:29.030124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":76,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.945477+0000","last_change":"2023-07-14T11:53:09.146156+0000","last_active":"2023-07-14T11:56:12.945477+0000","last_peered":"2023-07-14T11:56:12.945477+0000","last_clean":"2023-07-14T11:56:12.945477+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:56:12.945477+0000","last_undegraded":"2023-07-14T11:56:12.945477+0000","last_fullsized":"2023-07-14T11:56:12.945477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:29:13.428613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.704794+0000","last_change":"2023-07-14T11:56:11.714245+0000","last_active":"2023-07-14T11:56:12.704794+0000","last_peered":"2023-07-14T11:56:12.704794+0000","last_clean":"2023-07-14T11:56:12.704794+0000","last_became_active":"2023-07-14T11:56:11.713984+0000","last_became_peered":"2023-07-14T11:56:11.713984+0000","last_unstale":"2023-07-14T11:56:12.704794+0000","last_undegraded":"2023-07-14T11:56:12.704794+0000","last_fullsized":"2023-07-14T11:56:12.704794+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:10.656698+0000","last_clean_scrub_stamp":"2023-07-14T11:56:10.656698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:20:04.014040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.942964+0000","last_change":"2023-07-14T11:56:10.694832+0000","last_active":"2023-07-14T11:56:12.942964+0000","last_peered":"2023-07-14T11:56:12.942964+0000","last_clean":"2023-07-14T11:56:12.942964+0000","last_became_active":"2023-07-14T11:56:10.694539+0000","last_became_peered":"2023-07-14T11:56:10.694539+0000","last_unstale":"2023-07-14T11:56:12.942964+0000","last_undegraded":"2023-07-14T11:56:12.942964+0000","last_fullsized":"2023-07-14T11:56:12.942964+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:56:09.649464+0000","last_clean_scrub_stamp":"2023-07-14T11:56:09.649464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:29:33.619320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":88,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944324+0000","last_change":"2023-07-14T11:53:09.146556+0000","last_active":"2023-07-14T11:56:12.944324+0000","last_peered":"2023-07-14T11:56:12.944324+0000","last_clean":"2023-07-14T11:56:12.944324+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:56:12.944324+0000","last_undegraded":"2023-07-14T11:56:12.944324+0000","last_fullsized":"2023-07-14T11:56:12.944324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:46:22.779650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":232,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-14T11:56:12.944717+0000","last_change":"2023-07-14T11:53:03.159818+0000","last_active":"2023-07-14T11:56:12.944717+0000","last_peered":"2023-07-14T11:56:12.944717+0000","last_clean":"2023-07-14T11:56:12.944717+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:56:12.944717+0000","last_undegraded":"2023-07-14T11:56:12.944717+0000","last_fullsized":"2023-07-14T11:56:12.944717+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_clean_scrub_stamp":"2023-07-14T11:53:02.054348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:31.680340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1318912,"data_stored":923150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.093,"15min":1.111},"min":{"1min":0.754,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.474,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.399},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.103,"15min":1.127},"min":{"1min":0.65100000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2430000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.169}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1499999999999999,"15min":1.171},"min":{"1min":0.68300000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.298,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.22,"15min":1.262},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.4159999999999999,"5min":2.234,"15min":2.234},"last":1.0489999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.234,"15min":1.288},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.536,"5min":2.04,"15min":2.04},"last":0.753},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.25,"15min":1.2829999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8089999999999999,"5min":2.492,"15min":2.492},"last":0.97499999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.282,"15min":1.3360000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.665,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.498},{"interface":"front","average":{"1min":1.169,"5min":1.2430000000000001,"15min":1.274},"min":{"1min":0.81799999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5940000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.339}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.321,"15min":1.385},"min":{"1min":0.84399999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5660000000000001,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":1.276},{"interface":"front","average":{"1min":1.286,"5min":1.3400000000000001,"15min":1.3799999999999999},"min":{"1min":0.92500000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.76,"5min":2.399,"15min":2.399},"last":1.5509999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.284,"15min":1.3200000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6459999999999999,"5min":2.387,"15min":2.387},"last":1.448},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.456,"15min":1.538},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.464,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":1.472}]},{"osd":6,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.5309999999999999,"15min":1.609},"min":{"1min":1.0940000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5289999999999999,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.4550000000000001,"15min":1.5229999999999999},"min":{"1min":0.95199999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.53,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.611}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":712704,"data_stored":332040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0700000000000001,"15min":1.0620000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.397,"5min":1.954,"15min":1.954},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.165,"5min":1.1160000000000001,"15min":1.1000000000000001},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.0780000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1890000000000001,"15min":1.2090000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7110000000000001,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.272},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.214,"15min":1.23},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7769999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.3819999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.171,"15min":1.1659999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4690000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.339},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2709999999999999,"15min":1.2729999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6819999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.4259999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.3140000000000001,"15min":1.3440000000000001},"min":{"1min":0.94099999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.6579999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.206},{"interface":"front","average":{"1min":1.258,"5min":1.1899999999999999,"15min":1.1799999999999999},"min":{"1min":0.91700000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.482,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.585}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.236,"15min":1.2529999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.625,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.3129999999999999,"15min":1.3400000000000001},"min":{"1min":0.78000000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.512,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.542}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.238,"15min":1.2270000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.47},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4039999999999999,"15min":1.425},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.9139999999999999,"5min":1.984,"15min":1.984},"last":1.5069999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.4330000000000001,"15min":1.4650000000000001},"min":{"1min":0.99199999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.6519999999999999,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.365,"15min":1.379},"min":{"1min":0.92900000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.728,"5min":1.879,"15min":1.879},"last":1.675}]}]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":712704,"data_stored":332028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.1359999999999999,"15min":1.1819999999999999},"min":{"1min":0.68100000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4790000000000001,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.129,"15min":1.175},"min":{"1min":0.70299999999999996,"5min":0.622,"15min":0.622},"max":{"1min":1.474,"5min":1.581,"15min":1.581},"last":0.94999999999999996}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.167,"15min":1.1990000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.524,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.175,"15min":1.1890000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.498,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.198,"15min":1.226},"min":{"1min":0.86699999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4550000000000001,"5min":1.839,"15min":1.839},"last":1.331},{"interface":"front","average":{"1min":1.113,"5min":1.282,"15min":1.347},"min":{"1min":0.73799999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4990000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.006}]},{"osd":3,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.272,"15min":1.3220000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.5409999999999999,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.089},{"interface":"front","average":{"1min":1.242,"5min":1.3440000000000001,"15min":1.3919999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.5349999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.2370000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2090000000000001,"15min":1.2490000000000001},"min":{"1min":0.75900000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5900000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.3540000000000001,"15min":1.3740000000000001},"min":{"1min":1.024,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.5109999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.204}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3959999999999999,"15min":1.4179999999999999},"min":{"1min":1.1459999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.544,"5min":1.99,"15min":1.99},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.375,"15min":1.4159999999999999},"min":{"1min":0.86399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6779999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.0489999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4039999999999999,"15min":1.421},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.6220000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.5569999999999999,"15min":1.6160000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.5880000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.349}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":720896,"data_stored":333570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.149,"15min":1.171},"min":{"1min":0.67200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.5069999999999999,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.753},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.133,"15min":1.1599999999999999},"min":{"1min":0.71099999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.92700000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.159,"15min":1.1819999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.508,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.042},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1879999999999999,"15min":1.2090000000000001},"min":{"1min":0.75900000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.516,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.84599999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1359999999999999,"15min":1.143},"min":{"1min":0.83899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.546,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.077},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3400000000000001,"15min":1.3620000000000001},"min":{"1min":0.96599999999999997,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.581,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":0.99399999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2789999999999999,"15min":1.321},"min":{"1min":0.82199999999999995,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.681,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.25},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2270000000000001,"15min":1.2450000000000001},"min":{"1min":0.88100000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.4770000000000001,"5min":1.823,"15min":1.823},"last":1.2070000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2989999999999999,"15min":1.3420000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6299999999999999,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.89500000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.47,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.1259999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.264,"15min":1.2669999999999999},"min":{"1min":0.97799999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.522,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.325},{"interface":"front","average":{"1min":1.351,"5min":1.476,"15min":1.532},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6759999999999999,"5min":3.5289999999999999,"15min":3.5289999999999999},"last":1.288}]},{"osd":7,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.466,"15min":1.5049999999999999},"min":{"1min":1.0589999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.6899999999999999,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.47,"15min":1.5209999999999999},"min":{"1min":1.0640000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":1.605,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":1.389}]}]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":708608,"data_stored":332006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.84099999999999997,"15min":0.871},"min":{"1min":0.47699999999999998,"5min":0.247,"15min":0.247},"max":{"1min":1.234,"5min":1.325,"15min":1.325},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.875,"15min":0.90800000000000003},"min":{"1min":0.60599999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2490000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.84999999999999998}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92300000000000004,"15min":0.96299999999999997},"min":{"1min":0.52600000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6950000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.0600000000000001,"15min":1.111},"min":{"1min":0.54300000000000004,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.319,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":1.202}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.097,"15min":1.141},"min":{"1min":0.63500000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4259999999999999,"5min":1.627,"15min":1.627},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.0669999999999999,"15min":1.091},"min":{"1min":0.73399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.0900000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2410000000000001,"15min":1.2849999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.53,"5min":2.335,"15min":2.335},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.2470000000000001,"15min":1.2869999999999999},"min":{"1min":0.77700000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.1739999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.323,"15min":1.359},"min":{"1min":0.94999999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.3580000000000001,"15min":1.421},"min":{"1min":0.86899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.532,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.2909999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.329,"15min":1.387},"min":{"1min":0.85899999999999999,"5min":0.625,"15min":0.625},"max":{"1min":1.7010000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.4550000000000001,"15min":1.5009999999999999},"min":{"1min":1.083,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.0790000000000002,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.3100000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.419,"15min":1.4810000000000001},"min":{"1min":0.90500000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.712,"5min":3.395,"15min":3.395},"last":1.345},{"interface":"front","average":{"1min":1.306,"5min":1.367,"15min":1.431},"min":{"1min":1.0680000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.619,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.95099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":708608,"data_stored":332006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.874,"15min":0.90600000000000003},"min":{"1min":0.53800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.22,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.876,"15min":0.90200000000000002},"min":{"1min":0.48799999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1799999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.71399999999999997}]},{"osd":1,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.88100000000000001,"15min":0.89900000000000002},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.093,"5min":1.365,"15min":1.365},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0269999999999999,"15min":1.069},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.425,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.145}]},{"osd":3,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0649999999999999,"15min":1.0960000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5089999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.238},{"interface":"front","average":{"1min":1.018,"5min":1.026,"15min":1.052},"min":{"1min":0.65300000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.5900000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2709999999999999,"15min":1.2989999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.718,"5min":2.097,"15min":2.097},"last":1.175},{"interface":"front","average":{"1min":1.242,"5min":1.2769999999999999,"15min":1.3149999999999999},"min":{"1min":0.67800000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.861,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.0680000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.2889999999999999,"15min":1.319},"min":{"1min":0.95699999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.4710000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.357},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3180000000000001,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.645,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.401}]},{"osd":6,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.333,"15min":1.383},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7649999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.286},{"interface":"front","average":{"1min":1.361,"5min":1.3919999999999999,"15min":1.425},"min":{"1min":1.1040000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.726,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.4419999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.397,"15min":1.444},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.677,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4219999999999999,"15min":1.466},"min":{"1min":1.01,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6899999999999999,"5min":2.536,"15min":2.536},"last":1.514}]}]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":737280,"data_stored":334334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.80100000000000005,"15min":0.83299999999999996},"min":{"1min":0.48899999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.26,"5min":1.349,"15min":1.349},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83299999999999996,"15min":0.83399999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.67700000000000005}]},{"osd":2,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84799999999999998,"15min":0.85799999999999998},"min":{"1min":0.499,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.93500000000000005,"15min":0.93999999999999995},"min":{"1min":0.55900000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.99199999999999999,"15min":1.002},"min":{"1min":0.78500000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.399,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0169999999999999,"15min":1.0229999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.8169999999999999,"5min":1.857,"15min":1.857},"last":0.877}]},{"osd":4,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.2729999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6279999999999999,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.1830000000000001,"15min":1.2170000000000001},"min":{"1min":0.754,"5min":0.745,"15min":0.745},"max":{"1min":1.6279999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.0149999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.23,"15min":1.256},"min":{"1min":0.876,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7190000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3149999999999999,"15min":1.365},"min":{"1min":0.93100000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.778,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.0289999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3560000000000001,"15min":1.387},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.9119999999999999,"5min":2.633,"15min":2.633},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.335,"15min":1.359},"min":{"1min":1.091,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.873,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.109}]},{"osd":7,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3580000000000001,"15min":1.3979999999999999},"min":{"1min":1.004,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.992,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3300000000000001,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.754,"15min":0.754},"max":{"1min":2.056,"5min":2.375,"15min":2.375},"last":1.137}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1302528,"data_stored":922374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.879,"15min":0.89300000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3100000000000001,"5min":1.581,"15min":1.581},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0049999999999999,"15min":1},"min":{"1min":0.64800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.5880000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90800000000000003,"15min":0.95099999999999996},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.1639999999999999,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.40100000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.93600000000000005,"15min":0.94599999999999995},"min":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.228,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.099,"15min":1.1319999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3799999999999999,"5min":1.704,"15min":1.704},"last":0.871}]},{"osd":4,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2430000000000001,"15min":1.278},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5149999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.145},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3420000000000001,"15min":1.3720000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.837,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":0.96799999999999997}]},{"osd":5,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.228,"15min":1.284},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9870000000000001,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.276,"15min":1.3200000000000001},"min":{"1min":0.79800000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.7749999999999999,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.095}]},{"osd":6,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.0459999999999998,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.319,"15min":1.357},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.9299999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.1970000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.335,"15min":1.3819999999999999},"min":{"1min":0.83299999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.865,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.379,"15min":1.417},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.6870000000000001,"5min":2.109,"15min":2.109},"last":1.3400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:56:21.232 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-14T11:56:21.233 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-14T11:56:21.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.457+0000 7f50f04e7700 1 Processor -- start 2023-07-14T11:56:21.447 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.457+0000 7f50f04e7700 1 -- start start 2023-07-14T11:56:21.449 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.461+0000 7f50f04e7700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f50e80b4280 0x7f50e80b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.461+0000 7f50f04e7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f50e815e810 0x7f50e80b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.450 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.461+0000 7f50f04e7700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e806cdd0 con 0x7f50e806d720 2023-07-14T11:56:21.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.461+0000 7f50f04e7700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e80ce9e0 con 0x7f50e80b4280 2023-07-14T11:56:21.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.461+0000 7f50f04e7700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e815dd30 con 0x7f50e815e810 2023-07-14T11:56:21.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.461+0000 7f50ee283700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f50e80b4280 0x7f50e80b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.461+0000 7f50ee283700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f50e80b4280 0x7f50e80b05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45956/0 (socket says 172.21.15.104:45956) 2023-07-14T11:56:21.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.461+0000 7f50ee283700 1 -- 172.21.15.104:0/849082784 learned_addr learned my addr 172.21.15.104:0/849082784 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:21.451 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.461+0000 7f50eea84700 1 --2- 172.21.15.104:0/849082784 >> v2:172.21.15.121:3300/0 conn(0x7f50e815e810 0x7f50e80b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50ee283700 1 -- 172.21.15.104:0/849082784 >> v2:172.21.15.121:3300/0 conn(0x7f50e815e810 msgr2=0x7f50e80b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:21.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50ee283700 1 --2- 172.21.15.104:0/849082784 >> v2:172.21.15.121:3300/0 conn(0x7f50e815e810 0x7f50e80b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.452 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50ee283700 1 -- 172.21.15.104:0/849082784 >> v1:172.21.15.121:6789/0 conn(0x7f50e806d720 legacy=0x7f50e8070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50ee283700 1 -- 172.21.15.104:0/849082784 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e815e010 con 0x7f50e80b4280 2023-07-14T11:56:21.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50ee283700 1 --2- 172.21.15.104:0/849082784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50e80b4280 0x7f50e80b05a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f50e80c5b70 tx=0x7f50e4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cdb4b8cef1f81b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:21.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50ed281700 1 -- 172.21.15.104:0/849082784 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4011830 con 0x7f50e80b4280 2023-07-14T11:56:21.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50ed281700 1 -- 172.21.15.104:0/849082784 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e4011990 con 0x7f50e80b4280 2023-07-14T11:56:21.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50ed281700 1 -- 172.21.15.104:0/849082784 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4011d20 con 0x7f50e80b4280 2023-07-14T11:56:21.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50f04e7700 1 -- 172.21.15.104:0/849082784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50e80b4280 msgr2=0x7f50e80b05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50f04e7700 1 --2- 172.21.15.104:0/849082784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50e80b4280 0x7f50e80b05a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f50e80c5b70 tx=0x7f50e4009bf0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50f04e7700 1 -- 172.21.15.104:0/849082784 shutdown_connections 2023-07-14T11:56:21.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50f04e7700 1 --2- 172.21.15.104:0/849082784 >> v2:172.21.15.121:3300/0 conn(0x7f50e815e810 0x7f50e80b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50f04e7700 1 --2- 172.21.15.104:0/849082784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50e80b4280 0x7f50e80b05a0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50f04e7700 1 -- 172.21.15.104:0/849082784 >> 172.21.15.104:0/849082784 conn(0x7f50e80cb5a0 msgr2=0x7f50e80cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:21.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50f04e7700 1 -- 172.21.15.104:0/849082784 shutdown_connections 2023-07-14T11:56:21.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.465+0000 7f50f04e7700 1 -- 172.21.15.104:0/849082784 wait complete. 2023-07-14T11:56:21.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.469+0000 7f50f04e7700 1 Processor -- start 2023-07-14T11:56:21.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.469+0000 7f50f04e7700 1 -- start start 2023-07-14T11:56:21.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.469+0000 7f50f04e7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50e806d720 0x7f50e8151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.469+0000 7f50f04e7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e80b4280 0x7f50e80b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.469+0000 7f50f04e7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50e815e810 0x7f50e80b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.469+0000 7f50f04e7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80c76c0 con 0x7f50e815e810 2023-07-14T11:56:21.460 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.469+0000 7f50f04e7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80c6ab0 con 0x7f50e806d720 2023-07-14T11:56:21.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.469+0000 7f50f04e7700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80c6d90 con 0x7f50e80b4280 2023-07-14T11:56:21.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.469+0000 7f50eea84700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50e815e810 0x7f50e80b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.469+0000 7f50eea84700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50e815e810 0x7f50e80b2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45970/0 (socket says 172.21.15.104:45970) 2023-07-14T11:56:21.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.469+0000 7f50eea84700 1 -- 172.21.15.104:0/1710576368 learned_addr learned my addr 172.21.15.104:0/1710576368 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:21.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.469+0000 7f50ee283700 1 --2- 172.21.15.104:0/1710576368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50e806d720 0x7f50e8151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.469+0000 7f50eda82700 1 --2- 172.21.15.104:0/1710576368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e80b4280 0x7f50e80b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.473+0000 7f50ee283700 1 -- 172.21.15.104:0/1710576368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e80b4280 msgr2=0x7f50e80b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.461 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.473+0000 7f50ee283700 1 --2- 172.21.15.104:0/1710576368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e80b4280 0x7f50e80b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.462 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.473+0000 7f50ee283700 1 -- 172.21.15.104:0/1710576368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50e815e810 msgr2=0x7f50e80b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.462 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.473+0000 7f50ee283700 1 --2- 172.21.15.104:0/1710576368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50e815e810 0x7f50e80b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.462 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.473+0000 7f50ee283700 1 -- 172.21.15.104:0/1710576368 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e4009570 con 0x7f50e806d720 2023-07-14T11:56:21.462 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.473+0000 7f50eea84700 1 --2- 172.21.15.104:0/1710576368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50e815e810 0x7f50e80b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:21.462 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.473+0000 7f50ee283700 1 --2- 172.21.15.104:0/1710576368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50e806d720 0x7f50e8151ff0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f50e4016000 tx=0x7f50e4004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:21.462 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.473+0000 7f50deffd700 1 -- 172.21.15.104:0/1710576368 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4004450 con 0x7f50e806d720 2023-07-14T11:56:21.463 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.473+0000 7f50deffd700 1 -- 172.21.15.104:0/1710576368 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e401f070 con 0x7f50e806d720 2023-07-14T11:56:21.463 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.473+0000 7f50deffd700 1 -- 172.21.15.104:0/1710576368 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4003900 con 0x7f50e806d720 2023-07-14T11:56:21.463 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.473+0000 7f50f04e7700 1 -- 172.21.15.104:0/1710576368 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e80c7070 con 0x7f50e806d720 2023-07-14T11:56:21.463 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.473+0000 7f50f04e7700 1 -- 172.21.15.104:0/1710576368 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e8152530 con 0x7f50e806d720 2023-07-14T11:56:21.463 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.473+0000 7f50f04e7700 1 -- 172.21.15.104:0/1710576368 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e80c3d20 con 0x7f50e806d720 2023-07-14T11:56:21.464 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.477+0000 7f50deffd700 1 -- 172.21.15.104:0/1710576368 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f50e4003a60 con 0x7f50e806d720 2023-07-14T11:56:21.465 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.477+0000 7f50deffd700 1 --2- 172.21.15.104:0/1710576368 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f50cc0662f0 0x7f50cc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.465 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.477+0000 7f50deffd700 1 -- 172.21.15.104:0/1710576368 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f50e4021070 con 0x7f50e806d720 2023-07-14T11:56:21.465 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.477+0000 7f50eda82700 1 --2- 172.21.15.104:0/1710576368 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f50cc0662f0 0x7f50cc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.466 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.477+0000 7f50eda82700 1 --2- 172.21.15.104:0/1710576368 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f50cc0662f0 0x7f50cc0687a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f50e815ad20 tx=0x7f50d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:21.474 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.485+0000 7f50deffd700 1 -- 172.21.15.104:0/1710576368 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f50e4052650 con 0x7f50e806d720 2023-07-14T11:56:21.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.717+0000 7f50f04e7700 1 -- 172.21.15.104:0/1710576368 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f50e804dcb0 con 0x7f50e806d720 2023-07-14T11:56:21.720 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.729+0000 7f50deffd700 1 -- 172.21.15.104:0/1710576368 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f50e40045d0 con 0x7f50e806d720 2023-07-14T11:56:21.720 INFO:teuthology.orchestra.run.smithi104.stdout:[client.0] 2023-07-14T11:56:21.721 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDrNrFkbrzjERAA+9BBzuauwUdhALUcDtd80A== 2023-07-14T11:56:21.721 INFO:teuthology.orchestra.run.smithi104.stdout: caps mds = "allow" 2023-07-14T11:56:21.721 INFO:teuthology.orchestra.run.smithi104.stdout: caps mon = "allow r" 2023-07-14T11:56:21.722 INFO:teuthology.orchestra.run.smithi104.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-14T11:56:21.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.733+0000 7f50f04e7700 1 -- 172.21.15.104:0/1710576368 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f50cc0662f0 msgr2=0x7f50cc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.733+0000 7f50f04e7700 1 --2- 172.21.15.104:0/1710576368 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f50cc0662f0 0x7f50cc0687a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f50e815ad20 tx=0x7f50d8009040 comp rx=0 tx=0).stop 2023-07-14T11:56:21.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.733+0000 7f50f04e7700 1 -- 172.21.15.104:0/1710576368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50e806d720 msgr2=0x7f50e8151ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.733+0000 7f50f04e7700 1 --2- 172.21.15.104:0/1710576368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50e806d720 0x7f50e8151ff0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f50e4016000 tx=0x7f50e4004770 comp rx=0 tx=0).stop 2023-07-14T11:56:21.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.733+0000 7f50f04e7700 1 -- 172.21.15.104:0/1710576368 shutdown_connections 2023-07-14T11:56:21.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.733+0000 7f50f04e7700 1 --2- 172.21.15.104:0/1710576368 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f50cc0662f0 0x7f50cc0687a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.733+0000 7f50f04e7700 1 --2- 172.21.15.104:0/1710576368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50e815e810 0x7f50e80b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.724 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.733+0000 7f50f04e7700 1 --2- 172.21.15.104:0/1710576368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e80b4280 0x7f50e80b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.724 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.733+0000 7f50f04e7700 1 --2- 172.21.15.104:0/1710576368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50e806d720 0x7f50e8151ff0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.724 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.733+0000 7f50f04e7700 1 -- 172.21.15.104:0/1710576368 >> 172.21.15.104:0/1710576368 conn(0x7f50e80cb5a0 msgr2=0x7f50e80cdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:21.724 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.733+0000 7f50f04e7700 1 -- 172.21.15.104:0/1710576368 shutdown_connections 2023-07-14T11:56:21.724 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.733+0000 7f50f04e7700 1 -- 172.21.15.104:0/1710576368 wait complete. 2023-07-14T11:56:21.725 INFO:teuthology.orchestra.run.smithi104.stderr:updated caps for client.0 2023-07-14T11:56:21.746 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:56:21.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.933+0000 7fa900a78700 1 Processor -- start 2023-07-14T11:56:21.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.937+0000 7fa900a78700 1 -- start start 2023-07-14T11:56:21.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.937+0000 7fa900a78700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa8fc06be70 0x7fa8fc0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.937+0000 7fa900a78700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8fc0c1e00 0x7fa8fc0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.937+0000 7fa900a78700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8fc0cc1b0 con 0x7fa8fc06d860 2023-07-14T11:56:21.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.937+0000 7fa900a78700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8fc0cd690 con 0x7fa8fc06be70 2023-07-14T11:56:21.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.937+0000 7fa900a78700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc0cd970 con 0x7fa8fc0c1e00 2023-07-14T11:56:21.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.937+0000 7fa8f3fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa8fc06be70 0x7fa8fc0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.937+0000 7fa8f3fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa8fc06be70 0x7fa8fc0cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45980/0 (socket says 172.21.15.104:45980) 2023-07-14T11:56:21.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.937+0000 7fa8f3fff700 1 -- 172.21.15.104:0/2194083016 learned_addr learned my addr 172.21.15.104:0/2194083016 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:21.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.937+0000 7fa8fad9d700 1 --2- 172.21.15.104:0/2194083016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8fc0c1e00 0x7fa8fc0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.937+0000 7fa8f3fff700 1 -- 172.21.15.104:0/2194083016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8fc0c1e00 msgr2=0x7fa8fc0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:21.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.937+0000 7fa8f3fff700 1 --2- 172.21.15.104:0/2194083016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8fc0c1e00 0x7fa8fc0cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.937+0000 7fa8f3fff700 1 -- 172.21.15.104:0/2194083016 >> v1:172.21.15.121:6789/0 conn(0x7fa8fc06d860 legacy=0x7fa8fc06dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.937+0000 7fa8f3fff700 1 -- 172.21.15.104:0/2194083016 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc0cdc50 con 0x7fa8fc06be70 2023-07-14T11:56:21.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.941+0000 7fa8f3fff700 1 --2- 172.21.15.104:0/2194083016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8fc06be70 0x7fa8fc0cb840 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009910 tx=0x7fa8e4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f83bc4205853e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:21.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.941+0000 7fa8f9d9b700 1 -- 172.21.15.104:0/2194083016 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e400ba60 con 0x7fa8fc06be70 2023-07-14T11:56:21.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.941+0000 7fa8f9d9b700 1 -- 172.21.15.104:0/2194083016 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e400bbc0 con 0x7fa8fc06be70 2023-07-14T11:56:21.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.941+0000 7fa8f9d9b700 1 -- 172.21.15.104:0/2194083016 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4009e60 con 0x7fa8fc06be70 2023-07-14T11:56:21.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.941+0000 7fa900a78700 1 -- 172.21.15.104:0/2194083016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8fc06be70 msgr2=0x7fa8fc0cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.941+0000 7fa900a78700 1 --2- 172.21.15.104:0/2194083016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8fc06be70 0x7fa8fc0cb840 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009910 tx=0x7fa8e4009c20 comp rx=0 tx=0).stop 2023-07-14T11:56:21.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.941+0000 7fa900a78700 1 -- 172.21.15.104:0/2194083016 shutdown_connections 2023-07-14T11:56:21.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.941+0000 7fa900a78700 1 --2- 172.21.15.104:0/2194083016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8fc0c1e00 0x7fa8fc0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.941+0000 7fa900a78700 1 --2- 172.21.15.104:0/2194083016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8fc06be70 0x7fa8fc0cb840 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.941+0000 7fa900a78700 1 -- 172.21.15.104:0/2194083016 >> 172.21.15.104:0/2194083016 conn(0x7fa8fc0cad30 msgr2=0x7fa8fc0cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:21.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.941+0000 7fa900a78700 1 -- 172.21.15.104:0/2194083016 shutdown_connections 2023-07-14T11:56:21.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.941+0000 7fa900a78700 1 -- 172.21.15.104:0/2194083016 wait complete. 2023-07-14T11:56:21.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.945+0000 7fa900a78700 1 Processor -- start 2023-07-14T11:56:21.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.945+0000 7fa900a78700 1 -- start start 2023-07-14T11:56:21.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.945+0000 7fa900a78700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8fc06be70 0x7fa8fc153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.945+0000 7fa900a78700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8fc06d860 0x7fa8fc0c2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.945+0000 7fa900a78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8fc0c1e00 0x7fa8fc155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.945+0000 7fa900a78700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc153660 con 0x7fa8fc06be70 2023-07-14T11:56:21.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.945+0000 7fa900a78700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc15b9d0 con 0x7fa8fc0c1e00 2023-07-14T11:56:21.935 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.945+0000 7fa900a78700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc1f1290 con 0x7fa8fc06d860 2023-07-14T11:56:21.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.945+0000 7fa8fa59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8fc06be70 0x7fa8fc153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa8fa59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8fc06be70 0x7fa8fc153120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45982/0 (socket says 172.21.15.104:45982) 2023-07-14T11:56:21.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa8fa59c700 1 -- 172.21.15.104:0/4065724307 learned_addr learned my addr 172.21.15.104:0/4065724307 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:21.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa8fad9d700 1 --2- 172.21.15.104:0/4065724307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8fc0c1e00 0x7fa8fc155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa8f3fff700 1 --2- 172.21.15.104:0/4065724307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8fc06d860 0x7fa8fc0c2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.936 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa8fa59c700 1 -- 172.21.15.104:0/4065724307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8fc06d860 msgr2=0x7fa8fc0c2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa8fa59c700 1 --2- 172.21.15.104:0/4065724307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8fc06d860 0x7fa8fc0c2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa8fa59c700 1 -- 172.21.15.104:0/4065724307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8fc0c1e00 msgr2=0x7fa8fc155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:21.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa8fa59c700 1 --2- 172.21.15.104:0/4065724307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8fc0c1e00 0x7fa8fc155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:21.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa8fa59c700 1 -- 172.21.15.104:0/4065724307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e4009570 con 0x7fa8fc06be70 2023-07-14T11:56:21.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa8fa59c700 1 --2- 172.21.15.104:0/4065724307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8fc06be70 0x7fa8fc153120 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00c500 tx=0x7fa8ec00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:21.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa8f2ffd700 1 -- 172.21.15.104:0/4065724307 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec0052d0 con 0x7fa8fc06be70 2023-07-14T11:56:21.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa8f2ffd700 1 -- 172.21.15.104:0/4065724307 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ec005430 con 0x7fa8fc06be70 2023-07-14T11:56:21.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa900a78700 1 -- 172.21.15.104:0/4065724307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8fc155d50 con 0x7fa8fc06be70 2023-07-14T11:56:21.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa8f2ffd700 1 -- 172.21.15.104:0/4065724307 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec0055a0 con 0x7fa8fc06be70 2023-07-14T11:56:21.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa900a78700 1 -- 172.21.15.104:0/4065724307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8fc15b660 con 0x7fa8fc06be70 2023-07-14T11:56:21.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.949+0000 7fa900a78700 1 -- 172.21.15.104:0/4065724307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8fc04dcb0 con 0x7fa8fc06be70 2023-07-14T11:56:21.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.953+0000 7fa8f2ffd700 1 -- 172.21.15.104:0/4065724307 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa8ec0038e0 con 0x7fa8fc06be70 2023-07-14T11:56:21.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.953+0000 7fa8f2ffd700 1 --2- 172.21.15.104:0/4065724307 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa8dc0662f0 0x7fa8dc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:21.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.953+0000 7fa8f2ffd700 1 -- 172.21.15.104:0/4065724307 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa8ec0832a0 con 0x7fa8fc06be70 2023-07-14T11:56:21.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.957+0000 7fa8f3fff700 1 --2- 172.21.15.104:0/4065724307 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa8dc0662f0 0x7fa8dc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:21.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.957+0000 7fa8f3fff700 1 --2- 172.21.15.104:0/4065724307 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa8dc0662f0 0x7fa8dc0687a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4000c00 tx=0x7fa8e4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:21.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:21.957+0000 7fa8f2ffd700 1 -- 172.21.15.104:0/4065724307 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fa8ec050540 con 0x7fa8fc06be70 2023-07-14T11:56:22.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.161+0000 7fa900a78700 1 -- 172.21.15.104:0/4065724307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8fc0acea0 con 0x7fa8fc06be70 2023-07-14T11:56:22.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.161+0000 7fa8f2ffd700 1 -- 172.21.15.104:0/4065724307 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4722 (secure 0 0 0) 0x7fa8ec053b60 con 0x7fa8fc06be70 2023-07-14T11:56:22.149 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:22.149 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/4186544624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":4186544624},{"type":"v1","addr":"172.21.15.104:6837","nonce":4186544624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4234575863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4234575863},{"type":"v1","addr":"172.21.15.121:6835","nonce":4234575863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6220,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4127455794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4127455794},{"type":"v1","addr":"172.21.15.121:6833","nonce":4127455794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:56:11.123432+0000","modified":"2023-07-14T11:56:14.719624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/2705739882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2705739882},{"type":"v1","addr":"172.21.15.104:6836","nonce":2705739882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-14T11:56:22.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.165+0000 7fa900a78700 1 -- 172.21.15.104:0/4065724307 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa8dc0662f0 msgr2=0x7fa8dc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:22.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.165+0000 7fa900a78700 1 --2- 172.21.15.104:0/4065724307 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa8dc0662f0 0x7fa8dc0687a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4000c00 tx=0x7fa8e4008030 comp rx=0 tx=0).stop 2023-07-14T11:56:22.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.165+0000 7fa900a78700 1 -- 172.21.15.104:0/4065724307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8fc06be70 msgr2=0x7fa8fc153120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:22.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.165+0000 7fa900a78700 1 --2- 172.21.15.104:0/4065724307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8fc06be70 0x7fa8fc153120 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00c500 tx=0x7fa8ec00c810 comp rx=0 tx=0).stop 2023-07-14T11:56:22.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.165+0000 7fa900a78700 1 -- 172.21.15.104:0/4065724307 shutdown_connections 2023-07-14T11:56:22.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.165+0000 7fa900a78700 1 --2- 172.21.15.104:0/4065724307 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa8dc0662f0 0x7fa8dc0687a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.165+0000 7fa900a78700 1 --2- 172.21.15.104:0/4065724307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8fc0c1e00 0x7fa8fc155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.165+0000 7fa900a78700 1 --2- 172.21.15.104:0/4065724307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8fc06d860 0x7fa8fc0c2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.165+0000 7fa900a78700 1 --2- 172.21.15.104:0/4065724307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8fc06be70 0x7fa8fc153120 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.165+0000 7fa900a78700 1 -- 172.21.15.104:0/4065724307 >> 172.21.15.104:0/4065724307 conn(0x7fa8fc0cad30 msgr2=0x7fa8fc0ab8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:22.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.165+0000 7fa900a78700 1 -- 172.21.15.104:0/4065724307 shutdown_connections 2023-07-14T11:56:22.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.165+0000 7fa900a78700 1 -- 172.21.15.104:0/4065724307 wait complete. 2023-07-14T11:56:22.154 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 27 2023-07-14T11:56:22.575 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-14T11:56:11.123432+0000', 'modified': '2023-07-14T11:56:14.719624+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6312}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6312': {'gid': 6312, 'name': 'c', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6836/2705739882', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 2705739882}, {'type': 'v1', 'addr': '172.21.15.104:6836', 'nonce': 2705739882}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-14T11:56:22.575 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-14T11:56:22.576 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-14T11:56:22.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.773+0000 7f7990907700 1 Processor -- start 2023-07-14T11:56:22.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.773+0000 7f7990907700 1 -- start start 2023-07-14T11:56:22.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.773+0000 7f7990907700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f798c0cd240 0x7f798c0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.773+0000 7f7990907700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f798c0ceff0 0x7f798c0afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.773+0000 7f7990907700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f798c0af0a0 con 0x7f798c15bff0 2023-07-14T11:56:22.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.773+0000 7f7990907700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798c0cf720 con 0x7f798c0ceff0 2023-07-14T11:56:22.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.773+0000 7f7990907700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798c0af2f0 con 0x7f798c0cd240 2023-07-14T11:56:22.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.773+0000 7f798b59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f798c0ceff0 0x7f798c0afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.773+0000 7f798b59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f798c0ceff0 0x7f798c0afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45994/0 (socket says 172.21.15.104:45994) 2023-07-14T11:56:22.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.773+0000 7f798b59e700 1 -- 172.21.15.104:0/451335560 learned_addr learned my addr 172.21.15.104:0/451335560 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:22.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.773+0000 7f7983fff700 1 --2- 172.21.15.104:0/451335560 >> v2:172.21.15.121:3300/0 conn(0x7f798c0cd240 0x7f798c0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.773+0000 7f798b59e700 1 -- 172.21.15.104:0/451335560 >> v2:172.21.15.121:3300/0 conn(0x7f798c0cd240 msgr2=0x7f798c0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:22.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.773+0000 7f798b59e700 1 --2- 172.21.15.104:0/451335560 >> v2:172.21.15.121:3300/0 conn(0x7f798c0cd240 0x7f798c0ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.777+0000 7f798b59e700 1 -- 172.21.15.104:0/451335560 >> v1:172.21.15.121:6789/0 conn(0x7f798c15bff0 legacy=0x7f798c0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:22.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.777+0000 7f798b59e700 1 -- 172.21.15.104:0/451335560 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798c0af5d0 con 0x7f798c0ceff0 2023-07-14T11:56:22.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.777+0000 7f798b59e700 1 --2- 172.21.15.104:0/451335560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f798c0ceff0 0x7f798c0afdd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f798c06e6f0 tx=0x7f798400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0f183cabc4a674 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:22.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.777+0000 7f798a59c700 1 -- 172.21.15.104:0/451335560 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798400e070 con 0x7f798c0ceff0 2023-07-14T11:56:22.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.777+0000 7f798a59c700 1 -- 172.21.15.104:0/451335560 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7984003a40 con 0x7f798c0ceff0 2023-07-14T11:56:22.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.777+0000 7f798a59c700 1 -- 172.21.15.104:0/451335560 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984003df0 con 0x7f798c0ceff0 2023-07-14T11:56:22.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.777+0000 7f7990907700 1 -- 172.21.15.104:0/451335560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f798c0ceff0 msgr2=0x7f798c0afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:22.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.777+0000 7f7990907700 1 --2- 172.21.15.104:0/451335560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f798c0ceff0 0x7f798c0afdd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f798c06e6f0 tx=0x7f798400b330 comp rx=0 tx=0).stop 2023-07-14T11:56:22.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.777+0000 7f7990907700 1 -- 172.21.15.104:0/451335560 shutdown_connections 2023-07-14T11:56:22.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.777+0000 7f7990907700 1 --2- 172.21.15.104:0/451335560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f798c0ceff0 0x7f798c0afdd0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.777+0000 7f7990907700 1 --2- 172.21.15.104:0/451335560 >> v2:172.21.15.121:3300/0 conn(0x7f798c0cd240 0x7f798c0ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.777+0000 7f7990907700 1 -- 172.21.15.104:0/451335560 >> 172.21.15.104:0/451335560 conn(0x7f798c060c40 msgr2=0x7f798c09b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:22.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.777+0000 7f7990907700 1 -- 172.21.15.104:0/451335560 shutdown_connections 2023-07-14T11:56:22.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.777+0000 7f7990907700 1 -- 172.21.15.104:0/451335560 wait complete. 2023-07-14T11:56:22.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f7990907700 1 Processor -- start 2023-07-14T11:56:22.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f7990907700 1 -- start start 2023-07-14T11:56:22.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f7990907700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f798c0cd240 0x7f798c15b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f7990907700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f798c0ceff0 0x7f798c1561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f7990907700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f798c15bff0 0x7f798c15a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f7990907700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798c1566f0 con 0x7f798c15bff0 2023-07-14T11:56:22.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f7990907700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798c0b6020 con 0x7f798c0cd240 2023-07-14T11:56:22.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f7990907700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798c0cc0f0 con 0x7f798c0ceff0 2023-07-14T11:56:22.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f798b59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f798c15bff0 0x7f798c15a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f798b59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f798c15bff0 0x7f798c15a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46006/0 (socket says 172.21.15.104:46006) 2023-07-14T11:56:22.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f798b59e700 1 -- 172.21.15.104:0/3025818565 learned_addr learned my addr 172.21.15.104:0/3025818565 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:22.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f7983fff700 1 --2- 172.21.15.104:0/3025818565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f798c0ceff0 0x7f798c1561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f798ad9d700 1 --2- 172.21.15.104:0/3025818565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f798c0cd240 0x7f798c15b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f798b59e700 1 -- 172.21.15.104:0/3025818565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f798c0ceff0 msgr2=0x7f798c1561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:22.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f798b59e700 1 --2- 172.21.15.104:0/3025818565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f798c0ceff0 0x7f798c1561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f798b59e700 1 -- 172.21.15.104:0/3025818565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f798c0cd240 msgr2=0x7f798c15b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:22.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f798b59e700 1 --2- 172.21.15.104:0/3025818565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f798c0cd240 0x7f798c15b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f798b59e700 1 -- 172.21.15.104:0/3025818565 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7984009b30 con 0x7f798c15bff0 2023-07-14T11:56:22.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.781+0000 7f798b59e700 1 --2- 172.21.15.104:0/3025818565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f798c15bff0 0x7f798c15a5f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f798400b370 tx=0x7f7984004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:22.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.785+0000 7f79837fe700 1 -- 172.21.15.104:0/3025818565 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984004440 con 0x7f798c15bff0 2023-07-14T11:56:22.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.785+0000 7f79837fe700 1 -- 172.21.15.104:0/3025818565 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79840045a0 con 0x7f798c15bff0 2023-07-14T11:56:22.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.785+0000 7f79837fe700 1 -- 172.21.15.104:0/3025818565 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984004950 con 0x7f798c15bff0 2023-07-14T11:56:22.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.785+0000 7f7990907700 1 -- 172.21.15.104:0/3025818565 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f798c0cc3d0 con 0x7f798c15bff0 2023-07-14T11:56:22.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.785+0000 7f7990907700 1 -- 172.21.15.104:0/3025818565 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798c0b5c40 con 0x7f798c15bff0 2023-07-14T11:56:22.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.785+0000 7f79837fe700 1 -- 172.21.15.104:0/3025818565 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7984012430 con 0x7f798c15bff0 2023-07-14T11:56:22.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.789+0000 7f79837fe700 1 --2- 172.21.15.104:0/3025818565 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f796c0662f0 0x7f796c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.789+0000 7f79837fe700 1 -- 172.21.15.104:0/3025818565 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7984085330 con 0x7f798c15bff0 2023-07-14T11:56:22.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.789+0000 7f798ad9d700 1 --2- 172.21.15.104:0/3025818565 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f796c0662f0 0x7f796c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.789+0000 7f798ad9d700 1 --2- 172.21.15.104:0/3025818565 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f796c0662f0 0x7f796c0687a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f797c009890 tx=0x7f797c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:22.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.789+0000 7f799438c740 1 Processor -- start 2023-07-14T11:56:22.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.789+0000 7f799438c740 1 -- start start 2023-07-14T11:56:22.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f799438c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8eb0 0x22718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f799438c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8ae0 0x2271df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f799438c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8710 0x2226d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f799438c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x2145270 con 0x1fb8ae0 2023-07-14T11:56:22.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f799438c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x21983a0 con 0x1fb8710 2023-07-14T11:56:22.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f799438c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x2191600 con 0x1fb8eb0 2023-07-14T11:56:22.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f7983fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8ae0 0x2271df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f7983fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8ae0 0x2271df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46020/0 (socket says 172.21.15.104:46020) 2023-07-14T11:56:22.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f7983fff700 1 -- 172.21.15.104:0/787642231 learned_addr learned my addr 172.21.15.104:0/787642231 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:22.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f798b59e700 1 --2- 172.21.15.104:0/787642231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8710 0x2226d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f798ad9d700 1 --2- 172.21.15.104:0/787642231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8eb0 0x22718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f7983fff700 1 -- 172.21.15.104:0/787642231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8eb0 msgr2=0x22718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:22.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f7983fff700 1 --2- 172.21.15.104:0/787642231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8eb0 0x22718b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f7983fff700 1 -- 172.21.15.104:0/787642231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8710 msgr2=0x2226d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:22.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f7983fff700 1 --2- 172.21.15.104:0/787642231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8710 0x2226d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f7983fff700 1 -- 172.21.15.104:0/787642231 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20049e0 con 0x1fb8ae0 2023-07-14T11:56:22.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f7983fff700 1 --2- 172.21.15.104:0/787642231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8ae0 0x2271df0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f797400b880 tx=0x7f797400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f3819b41b6bef6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:22.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f7980ff9700 1 -- 172.21.15.104:0/787642231 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974014660 con 0x1fb8ae0 2023-07-14T11:56:22.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f7980ff9700 1 -- 172.21.15.104:0/787642231 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79740147c0 con 0x1fb8ae0 2023-07-14T11:56:22.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f7980ff9700 1 -- 172.21.15.104:0/787642231 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974014b30 con 0x1fb8ae0 2023-07-14T11:56:22.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f799438c740 1 -- 172.21.15.104:0/787642231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8ae0 msgr2=0x2271df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:22.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f799438c740 1 --2- 172.21.15.104:0/787642231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8ae0 0x2271df0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f797400b880 tx=0x7f797400bb90 comp rx=0 tx=0).stop 2023-07-14T11:56:22.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f799438c740 1 -- 172.21.15.104:0/787642231 shutdown_connections 2023-07-14T11:56:22.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f799438c740 1 --2- 172.21.15.104:0/787642231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8ae0 0x2271df0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f799438c740 1 --2- 172.21.15.104:0/787642231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8710 0x2226d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f799438c740 1 --2- 172.21.15.104:0/787642231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8eb0 0x22718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f799438c740 1 -- 172.21.15.104:0/787642231 >> 172.21.15.104:0/787642231 conn(0x2222b60 msgr2=0x222d160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:22.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f799438c740 1 -- 172.21.15.104:0/787642231 shutdown_connections 2023-07-14T11:56:22.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.793+0000 7f799438c740 1 -- 172.21.15.104:0/787642231 wait complete. 2023-07-14T11:56:22.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.797+0000 7f799438c740 1 -- start start 2023-07-14T11:56:22.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.797+0000 7f799438c740 1 Processor -- start 2023-07-14T11:56:22.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.797+0000 7f799438c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8710 0x225aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.797+0000 7f799438c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8ae0 0x227adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.797+0000 7f799438c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8eb0 0x227b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.797+0000 7f799438c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x2153b00 con 0x1fb8710 2023-07-14T11:56:22.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.797+0000 7f799438c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2152470 con 0x1fb8ae0 2023-07-14T11:56:22.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.797+0000 7f799438c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x203db20 con 0x1fb8eb0 2023-07-14T11:56:22.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.797+0000 7f799438c740 10 client.0 _ll_register_callbacks cb 0x21b7530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:56:22.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f798ad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8710 0x225aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f798ad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8710 0x225aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46022/0 (socket says 172.21.15.104:46022) 2023-07-14T11:56:22.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f798ad9d700 1 -- 172.21.15.104:0/468063710 learned_addr learned my addr 172.21.15.104:0/468063710 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:22.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f7983fff700 1 --2- 172.21.15.104:0/468063710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8ae0 0x227adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f798b59e700 1 --2- 172.21.15.104:0/468063710 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8eb0 0x227b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f798ad9d700 1 -- 172.21.15.104:0/468063710 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8eb0 msgr2=0x227b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:22.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f798ad9d700 1 --2- 172.21.15.104:0/468063710 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8eb0 0x227b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f798ad9d700 1 -- 172.21.15.104:0/468063710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8ae0 msgr2=0x227adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:22.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f798ad9d700 1 --2- 172.21.15.104:0/468063710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8ae0 0x227adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f798ad9d700 1 -- 172.21.15.104:0/468063710 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797400e040 con 0x1fb8710 2023-07-14T11:56:22.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f798ad9d700 1 --2- 172.21.15.104:0/468063710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8710 0x225aa80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f797c014d00 tx=0x7f797c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cde2a57084453 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:22.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f7969ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-14T11:56:22.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f7969ffb700 1 -- 172.21.15.104:0/468063710 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c012b50 con 0x1fb8710 2023-07-14T11:56:22.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f799438c740 1 -- 172.21.15.104:0/468063710 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x218a5e0 con 0x1fb8710 2023-07-14T11:56:22.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f7969ffb700 1 -- 172.21.15.104:0/468063710 <== mon.0 v2:172.21.15.104:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f797c012cb0 con 0x1fb8710 2023-07-14T11:56:22.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f7969ffb700 1 -- 172.21.15.104:0/468063710 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f797c012e70 con 0x1fb8710 2023-07-14T11:56:22.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f799438c740 10 client.6468 fetch_fsmap learned FSMap version 27 2023-07-14T11:56:22.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f799438c740 1 -- 172.21.15.104:0/468063710 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x2157250 con 0x1fb8710 2023-07-14T11:56:22.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f7969ffb700 1 -- 172.21.15.104:0/468063710 <== mon.0 v2:172.21.15.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x218a5e0 con 0x1fb8710 2023-07-14T11:56:22.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f7969ffb700 1 -- 172.21.15.104:0/468063710 <== mon.0 v2:172.21.15.104:3300/0 5 ==== fsmap(e 27) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f797c028360 con 0x1fb8710 2023-07-14T11:56:22.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f799438c740 10 client.6468 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T11:56:22.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f799438c740 10 client.6468 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-14T11:56:22.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f799438c740 20 client.6468 populate_metadata read hostname 'smithi104' 2023-07-14T11:56:22.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f799438c740 1 --2- 172.21.15.104:0/468063710 >> [v2:172.21.15.104:6834/2705739882,v1:172.21.15.104:6836/2705739882] conn(0x2003790 0x2257780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f799438c740 4 client.6468 mds_command: new command op to 6312 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:56:22.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f799438c740 1 -- 172.21.15.104:0/468063710 --> [v2:172.21.15.104:6834/2705739882,v1:172.21.15.104:6836/2705739882] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x218a300 con 0x2003790 2023-07-14T11:56:22.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.801+0000 7f7983fff700 1 --2- 172.21.15.104:0/468063710 >> [v2:172.21.15.104:6834/2705739882,v1:172.21.15.104:6836/2705739882] conn(0x2003790 0x2257780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f7983fff700 1 --2- 172.21.15.104:0/468063710 >> [v2:172.21.15.104:6834/2705739882,v1:172.21.15.104:6836/2705739882] conn(0x2003790 0x2257780 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccff6aacf56ebfc6 server_cookie=70c3bd7dc28dfc56 in_seq=0 out_seq=0 2023-07-14T11:56:22.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f7969ffb700 10 client.6468 ms_handle_connect on v2:172.21.15.104:6834/2705739882 2023-07-14T11:56:22.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f7969ffb700 1 -- 172.21.15.104:0/468063710 <== mds.0 v2:172.21.15.104:6834/2705739882 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x218a300 con 0x2003790 2023-07-14T11:56:22.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f7969ffb700 10 client.6468 handle_command_reply: tid=0 2023-07-14T11:56:22.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f799438c740 1 client.6468 shutdown 2023-07-14T11:56:22.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f799438c740 1 -- 172.21.15.104:0/468063710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8710 msgr2=0x225aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:22.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f799438c740 1 --2- 172.21.15.104:0/468063710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8710 0x225aa80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f797c014d00 tx=0x7f797c004730 comp rx=0 tx=0).stop 2023-07-14T11:56:22.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f799438c740 1 -- 172.21.15.104:0/468063710 shutdown_connections 2023-07-14T11:56:22.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f799438c740 1 --2- 172.21.15.104:0/468063710 >> [v2:172.21.15.104:6834/2705739882,v1:172.21.15.104:6836/2705739882] conn(0x2003790 0x2257780 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f799438c740 1 --2- 172.21.15.104:0/468063710 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8eb0 0x227b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f7969ffb700 0 client.6468 ms_handle_reset on v2:172.21.15.104:6834/2705739882 2023-07-14T11:56:22.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f799438c740 1 --2- 172.21.15.104:0/468063710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8ae0 0x227adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f799438c740 1 --2- 172.21.15.104:0/468063710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8710 0x225aa80 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f799438c740 1 -- 172.21.15.104:0/468063710 >> 172.21.15.104:0/468063710 conn(0x2222b60 msgr2=0x2226d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:22.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f799438c740 1 -- 172.21.15.104:0/468063710 shutdown_connections 2023-07-14T11:56:22.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f799438c740 1 -- 172.21.15.104:0/468063710 wait complete. 2023-07-14T11:56:22.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.805+0000 7f799438c740 20 client.6468 trim_cache size 0 max 16384 2023-07-14T11:56:22.814 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.825+0000 7f799438c740 1 Processor -- start 2023-07-14T11:56:22.815 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.825+0000 7f799438c740 1 -- start start 2023-07-14T11:56:22.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.825+0000 7f799438c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8ae0 0x22555c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.825+0000 7f799438c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8eb0 0x2255b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.825+0000 7f799438c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x22278c0 0x2227c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.816 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.825+0000 7f799438c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x218b900 con 0x22278c0 2023-07-14T11:56:22.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.825+0000 7f799438c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2137330 con 0x1fb8eb0 2023-07-14T11:56:22.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.825+0000 7f799438c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x2199630 con 0x1fb8ae0 2023-07-14T11:56:22.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f798b59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x22278c0 0x2227c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f798b59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x22278c0 0x2227c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46032/0 (socket says 172.21.15.104:46032) 2023-07-14T11:56:22.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f798b59e700 1 -- 172.21.15.104:0/1244431146 learned_addr learned my addr 172.21.15.104:0/1244431146 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:22.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f7983fff700 1 --2- 172.21.15.104:0/1244431146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8eb0 0x2255b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f798ad9d700 1 --2- 172.21.15.104:0/1244431146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8ae0 0x22555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f798b59e700 1 -- 172.21.15.104:0/1244431146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8ae0 msgr2=0x22555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:22.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f798b59e700 1 --2- 172.21.15.104:0/1244431146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8ae0 0x22555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f798b59e700 1 -- 172.21.15.104:0/1244431146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8eb0 msgr2=0x2255b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:22.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f798b59e700 1 --2- 172.21.15.104:0/1244431146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8eb0 0x2255b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f798b59e700 1 -- 172.21.15.104:0/1244431146 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798c15ae90 con 0x22278c0 2023-07-14T11:56:22.818 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f7983fff700 1 --2- 172.21.15.104:0/1244431146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8eb0 0x2255b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:22.819 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f798b59e700 1 --2- 172.21.15.104:0/1244431146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x22278c0 0x2227c90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7984056050 tx=0x7f798405c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73002a51d3b5efe9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:22.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f79817fa700 1 -- 172.21.15.104:0/1244431146 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984083630 con 0x22278c0 2023-07-14T11:56:22.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f79817fa700 1 -- 172.21.15.104:0/1244431146 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79840546f0 con 0x22278c0 2023-07-14T11:56:22.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f79817fa700 1 -- 172.21.15.104:0/1244431146 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984007be0 con 0x22278c0 2023-07-14T11:56:22.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f799438c740 1 -- 172.21.15.104:0/1244431146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x22278c0 msgr2=0x2227c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:22.820 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f799438c740 1 --2- 172.21.15.104:0/1244431146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x22278c0 0x2227c90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7984056050 tx=0x7f798405c040 comp rx=0 tx=0).stop 2023-07-14T11:56:22.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f799438c740 1 -- 172.21.15.104:0/1244431146 shutdown_connections 2023-07-14T11:56:22.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f799438c740 1 --2- 172.21.15.104:0/1244431146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x22278c0 0x2227c90 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f799438c740 1 --2- 172.21.15.104:0/1244431146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fb8eb0 0x2255b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f799438c740 1 --2- 172.21.15.104:0/1244431146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8ae0 0x22555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f799438c740 1 -- 172.21.15.104:0/1244431146 >> 172.21.15.104:0/1244431146 conn(0x2222b60 msgr2=0x2281480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:22.821 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f799438c740 1 -- 172.21.15.104:0/1244431146 shutdown_connections 2023-07-14T11:56:22.822 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.829+0000 7f799438c740 1 -- 172.21.15.104:0/1244431146 wait complete. 2023-07-14T11:56:22.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f799438c740 1 -- start start 2023-07-14T11:56:22.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f799438c740 1 Processor -- start 2023-07-14T11:56:22.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f799438c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8ae0 0x22762e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.823 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f799438c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8eb0 0x2275270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f799438c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22278c0 0x225a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f799438c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x2161510 con 0x1fb8eb0 2023-07-14T11:56:22.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f799438c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x218ea00 con 0x22278c0 2023-07-14T11:56:22.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f799438c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x218cf70 con 0x1fb8ae0 2023-07-14T11:56:22.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f799438c740 10 client.0 _ll_register_callbacks cb 0x2149800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:56:22.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f7983fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8eb0 0x2275270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.824 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f7983fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8eb0 0x2275270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46044/0 (socket says 172.21.15.104:46044) 2023-07-14T11:56:22.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f7983fff700 1 -- 172.21.15.104:0/2448654560 learned_addr learned my addr 172.21.15.104:0/2448654560 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:22.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f798ad9d700 1 --2- 172.21.15.104:0/2448654560 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8ae0 0x22762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f798b59e700 1 --2- 172.21.15.104:0/2448654560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22278c0 0x225a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f7983fff700 1 -- 172.21.15.104:0/2448654560 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8ae0 msgr2=0x22762e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:22.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f7983fff700 1 --2- 172.21.15.104:0/2448654560 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8ae0 0x22762e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f7983fff700 1 -- 172.21.15.104:0/2448654560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22278c0 msgr2=0x225a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:22.825 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f7983fff700 1 --2- 172.21.15.104:0/2448654560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22278c0 0x225a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f7983fff700 1 -- 172.21.15.104:0/2448654560 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x20049e0 con 0x1fb8eb0 2023-07-14T11:56:22.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.833+0000 7f798ad9d700 1 --2- 172.21.15.104:0/2448654560 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8ae0 0x22762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:22.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f7983fff700 1 --2- 172.21.15.104:0/2448654560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8eb0 0x2275270 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7974016000 tx=0x7f79740047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a6aed555a94a5f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:22.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f7969ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-14T11:56:22.826 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f7969ffb700 1 -- 172.21.15.104:0/2448654560 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974023070 con 0x1fb8eb0 2023-07-14T11:56:22.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 1 -- 172.21.15.104:0/2448654560 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2271330 con 0x1fb8eb0 2023-07-14T11:56:22.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f7969ffb700 1 -- 172.21.15.104:0/2448654560 <== mon.0 v2:172.21.15.104:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7974027570 con 0x1fb8eb0 2023-07-14T11:56:22.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f7969ffb700 1 -- 172.21.15.104:0/2448654560 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7974027780 con 0x1fb8eb0 2023-07-14T11:56:22.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 10 client.6474 fetch_fsmap learned FSMap version 27 2023-07-14T11:56:22.828 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 1 -- 172.21.15.104:0/2448654560 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x7f798c0b5c40 con 0x1fb8eb0 2023-07-14T11:56:22.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f7969ffb700 1 -- 172.21.15.104:0/2448654560 <== mon.0 v2:172.21.15.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2271330 con 0x1fb8eb0 2023-07-14T11:56:22.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f7969ffb700 1 -- 172.21.15.104:0/2448654560 <== mon.0 v2:172.21.15.104:3300/0 5 ==== fsmap(e 27) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f79740149b0 con 0x1fb8eb0 2023-07-14T11:56:22.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 10 client.6474 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T11:56:22.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 10 client.6474 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-14T11:56:22.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 20 client.6474 populate_metadata read hostname 'smithi104' 2023-07-14T11:56:22.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 1 --2- 172.21.15.104:0/2448654560 >> [v2:172.21.15.104:6834/2705739882,v1:172.21.15.104:6836/2705739882] conn(0x22743b0 0x2274780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:22.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 4 client.6474 mds_command: new command op to 6312 tid=0[{"prefix": "status"}] 2023-07-14T11:56:22.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 1 -- 172.21.15.104:0/2448654560 --> [v2:172.21.15.104:6834/2705739882,v1:172.21.15.104:6836/2705739882] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f79740147c0 con 0x22743b0 2023-07-14T11:56:22.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f798ad9d700 1 --2- 172.21.15.104:0/2448654560 >> [v2:172.21.15.104:6834/2705739882,v1:172.21.15.104:6836/2705739882] conn(0x22743b0 0x2274780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:22.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f798ad9d700 1 --2- 172.21.15.104:0/2448654560 >> [v2:172.21.15.104:6834/2705739882,v1:172.21.15.104:6836/2705739882] conn(0x22743b0 0x2274780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=411f54665b3fe0e1 server_cookie=154e162c1deb4439 in_seq=0 out_seq=0 2023-07-14T11:56:22.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f7969ffb700 10 client.6474 ms_handle_connect on v2:172.21.15.104:6834/2705739882 2023-07-14T11:56:22.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f7969ffb700 1 -- 172.21.15.104:0/2448654560 <== mds.0 v2:172.21.15.104:6834/2705739882 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x7f79740147c0 con 0x22743b0 2023-07-14T11:56:22.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f7969ffb700 10 client.6474 handle_command_reply: tid=0 2023-07-14T11:56:22.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 1 client.6474 shutdown 2023-07-14T11:56:22.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 1 -- 172.21.15.104:0/2448654560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8eb0 msgr2=0x2275270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:22.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 1 --2- 172.21.15.104:0/2448654560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8eb0 0x2275270 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7974016000 tx=0x7f79740047e0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 1 -- 172.21.15.104:0/2448654560 shutdown_connections 2023-07-14T11:56:22.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 1 --2- 172.21.15.104:0/2448654560 >> [v2:172.21.15.104:6834/2705739882,v1:172.21.15.104:6836/2705739882] conn(0x22743b0 0x2274780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 1 --2- 172.21.15.104:0/2448654560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fb8eb0 0x2275270 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 1 --2- 172.21.15.104:0/2448654560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22278c0 0x225a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f7969ffb700 0 client.6474 ms_handle_reset on v2:172.21.15.104:6834/2705739882 2023-07-14T11:56:22.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 1 --2- 172.21.15.104:0/2448654560 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x1fb8ae0 0x22762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.837+0000 7f799438c740 1 -- 172.21.15.104:0/2448654560 >> 172.21.15.104:0/2448654560 conn(0x2222b60 msgr2=0x2253e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:22.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.841+0000 7f799438c740 1 -- 172.21.15.104:0/2448654560 shutdown_connections 2023-07-14T11:56:22.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.841+0000 7f799438c740 1 -- 172.21.15.104:0/2448654560 wait complete. 2023-07-14T11:56:22.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.841+0000 7f799438c740 20 client.6474 trim_cache size 0 max 16384 2023-07-14T11:56:22.833 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:56:22.833 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_fsid": "c5576993-99e1-43b1-a253-659a3cee5212", 2023-07-14T11:56:22.833 INFO:teuthology.orchestra.run.smithi104.stdout: "whoami": 0, 2023-07-14T11:56:22.833 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 6312, 2023-07-14T11:56:22.833 INFO:teuthology.orchestra.run.smithi104.stdout: "want_state": "up:active", 2023-07-14T11:56:22.833 INFO:teuthology.orchestra.run.smithi104.stdout: "state": "up:active", 2023-07-14T11:56:22.834 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs", 2023-07-14T11:56:22.834 INFO:teuthology.orchestra.run.smithi104.stdout: "rank_uptime": 11.166963101, 2023-07-14T11:56:22.834 INFO:teuthology.orchestra.run.smithi104.stdout: "mdsmap_epoch": 27, 2023-07-14T11:56:22.834 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_epoch": 59, 2023-07-14T11:56:22.834 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_epoch_barrier": 59, 2023-07-14T11:56:22.834 INFO:teuthology.orchestra.run.smithi104.stdout: "uptime": 17.366484085 2023-07-14T11:56:22.834 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:56:22.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.841+0000 7f7990907700 1 -- 172.21.15.104:0/3025818565 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f796c0662f0 msgr2=0x7f796c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:22.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.841+0000 7f7990907700 1 --2- 172.21.15.104:0/3025818565 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f796c0662f0 0x7f796c0687a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f797c009890 tx=0x7f797c008040 comp rx=0 tx=0).stop 2023-07-14T11:56:22.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.841+0000 7f7990907700 1 -- 172.21.15.104:0/3025818565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f798c15bff0 msgr2=0x7f798c15a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:22.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.841+0000 7f7990907700 1 --2- 172.21.15.104:0/3025818565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f798c15bff0 0x7f798c15a5f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f798400b370 tx=0x7f7984004af0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.841+0000 7f7990907700 1 -- 172.21.15.104:0/3025818565 shutdown_connections 2023-07-14T11:56:22.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.841+0000 7f7990907700 1 --2- 172.21.15.104:0/3025818565 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f796c0662f0 0x7f796c0687a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.841+0000 7f7990907700 1 --2- 172.21.15.104:0/3025818565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f798c15bff0 0x7f798c15a5f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.841+0000 7f7990907700 1 --2- 172.21.15.104:0/3025818565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f798c0ceff0 0x7f798c1561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.841+0000 7f7990907700 1 --2- 172.21.15.104:0/3025818565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f798c0cd240 0x7f798c15b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:22.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.841+0000 7f7990907700 1 -- 172.21.15.104:0/3025818565 >> 172.21.15.104:0/3025818565 conn(0x7f798c060c40 msgr2=0x7f798c0ccb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:22.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.841+0000 7f7990907700 1 -- 172.21.15.104:0/3025818565 shutdown_connections 2023-07-14T11:56:22.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:22.841+0000 7f7990907700 1 -- 172.21.15.104:0/3025818565 wait complete. 2023-07-14T11:56:22.851 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-14T11:56:23.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.033+0000 7feeea15b700 1 Processor -- start 2023-07-14T11:56:23.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feeea15b700 1 -- start start 2023-07-14T11:56:23.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feeea15b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feee406cc10 0x7feee40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:23.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feeea15b700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee40cc9c0 con 0x7feee406e960 2023-07-14T11:56:23.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feeea15b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee406dab0 con 0x7feee406c7c0 2023-07-14T11:56:23.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feeea15b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee4145a20 con 0x7feee406cc10 2023-07-14T11:56:23.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee37fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7feee406e960 0x7feee406ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:39842/0 (socket says 172.21.15.104:39842) 2023-07-14T11:56:23.026 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee37fe700 1 -- 172.21.15.104:0/781809490 learned_addr learned my addr 172.21.15.104:0/781809490 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:23.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee3fff700 1 --2- 172.21.15.104:0/781809490 >> v2:172.21.15.121:3300/0 conn(0x7feee406cc10 0x7feee40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:23.028 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee2ffd700 1 -- 172.21.15.104:0/781809490 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629687397 0 0) 0x7feee40cc9c0 con 0x7feee406e960 2023-07-14T11:56:23.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee2ffd700 1 -- 172.21.15.104:0/781809490 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feed00035d0 con 0x7feee406e960 2023-07-14T11:56:23.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee2ffd700 1 -- 172.21.15.104:0/781809490 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 972862705 0 0) 0x7feed00035d0 con 0x7feee406e960 2023-07-14T11:56:23.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee2ffd700 1 -- 172.21.15.104:0/781809490 >> v2:172.21.15.121:3300/0 conn(0x7feee406cc10 msgr2=0x7feee40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:23.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee2ffd700 1 --2- 172.21.15.104:0/781809490 >> v2:172.21.15.121:3300/0 conn(0x7feee406cc10 0x7feee40cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:23.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee2ffd700 1 -- 172.21.15.104:0/781809490 >> v1:172.21.15.121:6789/0 conn(0x7feee406c7c0 legacy=0x7feee40cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:23.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee2ffd700 1 -- 172.21.15.104:0/781809490 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee406d210 con 0x7feee406e960 2023-07-14T11:56:23.029 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee2ffd700 1 -- 172.21.15.104:0/781809490 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7feed4001670 con 0x7feee406e960 2023-07-14T11:56:23.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee2ffd700 1 -- 172.21.15.104:0/781809490 >> v1:172.21.15.104:6789/0 conn(0x7feee406e960 legacy=0x7feee406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:23.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee2ffd700 1 --2- 172.21.15.104:0/781809490 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed0003a20 0x7feed0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:23.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee2ffd700 1 --2- 172.21.15.104:0/781809490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feed0004870 0x7feed0004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:23.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee2ffd700 1 --2- 172.21.15.104:0/781809490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feed0005220 0x7feed000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:23.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee2ffd700 1 -- 172.21.15.104:0/781809490 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee4145a20 con 0x7feed0004870 2023-07-14T11:56:23.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee2ffd700 1 -- 172.21.15.104:0/781809490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed000c0a0 con 0x7feed0005220 2023-07-14T11:56:23.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.037+0000 7feee2ffd700 1 -- 172.21.15.104:0/781809490 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed000c260 con 0x7feed0003a20 2023-07-14T11:56:23.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feedbfff700 1 --2- 172.21.15.104:0/781809490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feed0004870 0x7feed0004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:23.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feee3fff700 1 --2- 172.21.15.104:0/781809490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feed0005220 0x7feed000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:23.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feee37fe700 1 --2- 172.21.15.104:0/781809490 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed0003a20 0x7feed0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:23.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feedbfff700 1 -- 172.21.15.104:0/781809490 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed0003a20 msgr2=0x7feed0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:23.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feedbfff700 1 --2- 172.21.15.104:0/781809490 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed0003a20 0x7feed0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:23.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feedbfff700 1 -- 172.21.15.104:0/781809490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feed0005220 msgr2=0x7feed000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:23.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feedbfff700 1 --2- 172.21.15.104:0/781809490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feed0005220 0x7feed000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:23.031 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feedbfff700 1 -- 172.21.15.104:0/781809490 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed00035d0 con 0x7feed0004870 2023-07-14T11:56:23.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feee3fff700 1 -- 172.21.15.104:0/781809490 reap_dead start 2023-07-14T11:56:23.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feedbfff700 1 --2- 172.21.15.104:0/781809490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feed0004870 0x7feed0004ce0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feecc0017e0 tx=0x7feed00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5038fcda5dc828 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:23.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feee2ffd700 1 -- 172.21.15.104:0/781809490 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc0052d0 con 0x7feed0004870 2023-07-14T11:56:23.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feee2ffd700 1 -- 172.21.15.104:0/781809490 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc005450 con 0x7feed0004870 2023-07-14T11:56:23.032 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feee2ffd700 1 -- 172.21.15.104:0/781809490 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc0038e0 con 0x7feed0004870 2023-07-14T11:56:23.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feeea15b700 1 -- 172.21.15.104:0/781809490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feed0004870 msgr2=0x7feed0004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:23.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feeea15b700 1 --2- 172.21.15.104:0/781809490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feed0004870 0x7feed0004ce0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feecc0017e0 tx=0x7feed00058a0 comp rx=0 tx=0).stop 2023-07-14T11:56:23.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feeea15b700 1 -- 172.21.15.104:0/781809490 shutdown_connections 2023-07-14T11:56:23.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feeea15b700 1 --2- 172.21.15.104:0/781809490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feed0004870 0x7feed0004ce0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:23.033 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feeea15b700 1 -- 172.21.15.104:0/781809490 >> 172.21.15.104:0/781809490 conn(0x7feee4069800 msgr2=0x7feee4069bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:23.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feeea15b700 1 -- 172.21.15.104:0/781809490 shutdown_connections 2023-07-14T11:56:23.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.041+0000 7feeea15b700 1 -- 172.21.15.104:0/781809490 wait complete. 2023-07-14T11:56:23.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.045+0000 7feeea15b700 1 Processor -- start 2023-07-14T11:56:23.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.045+0000 7feeea15b700 1 -- start start 2023-07-14T11:56:23.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feeea15b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feed0004870 0x7feee415dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:23.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feeea15b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee40b13f0 0x7feee413d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:23.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feeea15b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feee41f4900 0x7feee41f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:23.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feeea15b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee41f52c0 con 0x7feed0004870 2023-07-14T11:56:23.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feeea15b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee413cd30 con 0x7feee40b13f0 2023-07-14T11:56:23.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feeea15b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee415ec80 con 0x7feee41f4900 2023-07-14T11:56:23.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feee37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feed0004870 0x7feee415dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:23.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feee37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feed0004870 0x7feee415dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46058/0 (socket says 172.21.15.104:46058) 2023-07-14T11:56:23.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feee37fe700 1 -- 172.21.15.104:0/1736365086 learned_addr learned my addr 172.21.15.104:0/1736365086 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:23.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feedbfff700 1 --2- 172.21.15.104:0/1736365086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee40b13f0 0x7feee413d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:23.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feee3fff700 1 --2- 172.21.15.104:0/1736365086 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feee41f4900 0x7feee41f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:23.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feee37fe700 1 -- 172.21.15.104:0/1736365086 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feee41f4900 msgr2=0x7feee41f4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:23.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feee37fe700 1 --2- 172.21.15.104:0/1736365086 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feee41f4900 0x7feee41f4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:23.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feee37fe700 1 -- 172.21.15.104:0/1736365086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee40b13f0 msgr2=0x7feee413d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:23.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feee37fe700 1 --2- 172.21.15.104:0/1736365086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee40b13f0 0x7feee413d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:23.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feee37fe700 1 -- 172.21.15.104:0/1736365086 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee413deb0 con 0x7feed0004870 2023-07-14T11:56:23.038 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feee37fe700 1 --2- 172.21.15.104:0/1736365086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feed0004870 0x7feee415dce0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7feed400eb50 tx=0x7feed400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:23.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feee0ff9700 1 -- 172.21.15.104:0/1736365086 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed400c570 con 0x7feed0004870 2023-07-14T11:56:23.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feee0ff9700 1 -- 172.21.15.104:0/1736365086 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed400c6d0 con 0x7feed0004870 2023-07-14T11:56:23.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feee0ff9700 1 -- 172.21.15.104:0/1736365086 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed400ca80 con 0x7feed0004870 2023-07-14T11:56:23.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feeea15b700 1 -- 172.21.15.104:0/1736365086 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee415e3e0 con 0x7feed0004870 2023-07-14T11:56:23.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feeea15b700 1 -- 172.21.15.104:0/1736365086 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee415e9a0 con 0x7feed0004870 2023-07-14T11:56:23.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.049+0000 7feeea15b700 1 -- 172.21.15.104:0/1736365086 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee404dcb0 con 0x7feed0004870 2023-07-14T11:56:23.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.053+0000 7feee0ff9700 1 -- 172.21.15.104:0/1736365086 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7feed400cbe0 con 0x7feed0004870 2023-07-14T11:56:23.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.053+0000 7feee0ff9700 1 --2- 172.21.15.104:0/1736365086 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feec4066320 0x7feec40687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:23.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.053+0000 7feee0ff9700 1 -- 172.21.15.104:0/1736365086 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7feed4083ae0 con 0x7feed0004870 2023-07-14T11:56:23.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.053+0000 7feedbfff700 1 --2- 172.21.15.104:0/1736365086 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feec4066320 0x7feec40687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:23.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.053+0000 7feedbfff700 1 --2- 172.21.15.104:0/1736365086 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feec4066320 0x7feec40687d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feecc005040 tx=0x7feecc009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:23.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.061+0000 7feee0ff9700 1 -- 172.21.15.104:0/1736365086 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7feed4050eb0 con 0x7feed0004870 2023-07-14T11:56:23.252 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.265+0000 7feeea15b700 1 -- 172.21.15.104:0/1736365086 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feee4142c60 con 0x7feed0004870 2023-07-14T11:56:23.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.265+0000 7feee0ff9700 1 -- 172.21.15.104:0/1736365086 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7feed40544d0 con 0x7feed0004870 2023-07-14T11:56:23.254 INFO:teuthology.orchestra.run.smithi104.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-14T11:56:23.257 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.269+0000 7feeea15b700 1 -- 172.21.15.104:0/1736365086 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feec4066320 msgr2=0x7feec40687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:23.257 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.269+0000 7feeea15b700 1 --2- 172.21.15.104:0/1736365086 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feec4066320 0x7feec40687d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feecc005040 tx=0x7feecc009030 comp rx=0 tx=0).stop 2023-07-14T11:56:23.257 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.269+0000 7feeea15b700 1 -- 172.21.15.104:0/1736365086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feed0004870 msgr2=0x7feee415dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:23.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.269+0000 7feeea15b700 1 --2- 172.21.15.104:0/1736365086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feed0004870 0x7feee415dce0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7feed400eb50 tx=0x7feed400ee60 comp rx=0 tx=0).stop 2023-07-14T11:56:23.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.269+0000 7feeea15b700 1 -- 172.21.15.104:0/1736365086 shutdown_connections 2023-07-14T11:56:23.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.269+0000 7feeea15b700 1 --2- 172.21.15.104:0/1736365086 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7feec4066320 0x7feec40687d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:23.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.269+0000 7feeea15b700 1 --2- 172.21.15.104:0/1736365086 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feee41f4900 0x7feee41f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:23.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.269+0000 7feeea15b700 1 --2- 172.21.15.104:0/1736365086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee40b13f0 0x7feee413d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:23.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.269+0000 7feeea15b700 1 --2- 172.21.15.104:0/1736365086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feed0004870 0x7feee415dce0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:23.259 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.269+0000 7feeea15b700 1 -- 172.21.15.104:0/1736365086 >> 172.21.15.104:0/1736365086 conn(0x7feee4069800 msgr2=0x7feee40b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:23.259 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.269+0000 7feeea15b700 1 -- 172.21.15.104:0/1736365086 shutdown_connections 2023-07-14T11:56:23.259 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:23.269+0000 7feeea15b700 1 -- 172.21.15.104:0/1736365086 wait complete. 2023-07-14T11:56:23.278 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-14T11:56:23.278 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T11:56:23.279 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2023-07-14T11:56:23.279 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-14T11:56:23.279 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:23.279 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-14T11:56:23.280 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T11:56:23.280 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T11:56:23.280 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:56:23.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:23.282 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2023-07-14T11:56:23.301 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:56:23.302 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:56:23.302 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:56:23.302 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:23.302 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2023-07-14T11:56:23.302 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:23.302 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:56:23.302 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:23.303 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2023-07-14T11:56:23.303 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:56:23.303 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:23.303 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2023-07-14T11:56:23.303 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:56:23.303 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:23.303 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T11:56:23.304 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:23.304 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-07-14T11:56:23.304 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42213sec preferred_lft 42213sec 2023-07-14T11:56:23.304 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2023-07-14T11:56:23.304 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:23.304 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-07-14T11:56:23.304 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:a2:a1:53:16 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:23.304 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-07-14T11:56:23.305 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:23.305 INFO:teuthology.orchestra.run.smithi104.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-14T11:56:23.305 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:b8:27:80:0d:ab brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:23.305 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-14T11:56:23.305 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:23.306 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5020:39ff:fe23:761d/64 scope link 2023-07-14T11:56:23.306 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:23.306 INFO:teuthology.orchestra.run.smithi104.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-14T11:56:23.306 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:b8:27:80:0d:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-14T11:56:23.306 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::e8b8:27ff:fe80:dab/64 scope link 2023-07-14T11:56:23.307 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:23.307 INFO:teuthology.orchestra.run.smithi104.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-14T11:56:23.307 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:e6:23:3b:72:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-14T11:56:23.307 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::e8e6:23ff:fe3b:7247/64 scope link 2023-07-14T11:56:23.307 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:23.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:23.308 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T11:56:23.385 INFO:teuthology.orchestra.run.smithi104.stdout:1 2023-07-14T11:56:23.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:23.389 DEBUG:teuthology.orchestra.run.smithi104:> route 2023-07-14T11:56:23.451 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2023-07-14T11:56:23.451 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T11:56:23.451 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-07-14T11:56:23.452 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T11:56:23.452 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-07-14T11:56:23.452 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-07-14T11:56:23.452 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-07-14T11:56:23.452 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T11:56:23.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:23.454 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:56:23.454 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:56:23.454 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T11:56:23.455 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T11:56:23.455 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T11:56:23.455 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:56:23.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:23.592 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2023-07-14T11:56:23.608 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-14T11:56:23.608 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-14T11:56:23.609 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:56:23.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:23.610 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2023-07-14T11:56:23.685 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-14T11:56:23.685 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:56:23.686 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:23.745 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:56:23.746 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:56:23.746 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:23.804 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:56:23.883 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:56:23.883 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:56:23.958 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:56:23.960 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:56:23.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:23.961 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:56:24.019 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-14T11:56:24.020 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-14T11:56:24.068 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:56:24.103 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:56:24.104 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:56:24.180 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:56:24.184 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:56:24.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:24.186 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:56:24.211 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:2023-07-14T11:56:24.221+0000 7fe9077a0640 -1 init, newargv = 0x5568ca0e5e00 newargc=15 2023-07-14T11:56:24.212 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[50071]: starting ceph client 2023-07-14T11:56:24.222 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[50071]: starting fuse 2023-07-14T11:56:24.225 INFO:teuthology.orchestra.run.smithi104.stdout:49 2023-07-14T11:56:24.225 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-07-14T11:56:24.226 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T11:56:24.227 DEBUG:teuthology.orchestra.run.smithi104:> import glob 2023-07-14T11:56:24.227 DEBUG:teuthology.orchestra.run.smithi104:> import re 2023-07-14T11:56:24.228 DEBUG:teuthology.orchestra.run.smithi104:> import os 2023-07-14T11:56:24.228 DEBUG:teuthology.orchestra.run.smithi104:> import subprocess 2023-07-14T11:56:24.229 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:56:24.229 DEBUG:teuthology.orchestra.run.smithi104:> def _find_admin_socket(client_name): 2023-07-14T11:56:24.230 DEBUG:teuthology.orchestra.run.smithi104:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-14T11:56:24.230 DEBUG:teuthology.orchestra.run.smithi104:> files = glob.glob(asok_path) 2023-07-14T11:56:24.231 DEBUG:teuthology.orchestra.run.smithi104:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-14T11:56:24.232 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:56:24.232 DEBUG:teuthology.orchestra.run.smithi104:> # Given a non-glob path, it better be there 2023-07-14T11:56:24.233 DEBUG:teuthology.orchestra.run.smithi104:> if "*" not in asok_path: 2023-07-14T11:56:24.233 DEBUG:teuthology.orchestra.run.smithi104:> assert(len(files) == 1) 2023-07-14T11:56:24.234 DEBUG:teuthology.orchestra.run.smithi104:> return files[0] 2023-07-14T11:56:24.234 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:56:24.235 DEBUG:teuthology.orchestra.run.smithi104:> for f in files: 2023-07-14T11:56:24.235 DEBUG:teuthology.orchestra.run.smithi104:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-14T11:56:24.236 DEBUG:teuthology.orchestra.run.smithi104:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-14T11:56:24.237 DEBUG:teuthology.orchestra.run.smithi104:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-14T11:56:24.237 DEBUG:teuthology.orchestra.run.smithi104:> contents = proc_f.read() 2023-07-14T11:56:24.238 DEBUG:teuthology.orchestra.run.smithi104:> if mountpoint in contents: 2023-07-14T11:56:24.238 DEBUG:teuthology.orchestra.run.smithi104:> return f 2023-07-14T11:56:24.239 DEBUG:teuthology.orchestra.run.smithi104:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-14T11:56:24.239 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:56:24.240 DEBUG:teuthology.orchestra.run.smithi104:> print(_find_admin_socket("client.0")) 2023-07-14T11:56:24.241 DEBUG:teuthology.orchestra.run.smithi104:> ' 2023-07-14T11:56:24.384 INFO:teuthology.orchestra.run.smithi104.stdout:/var/run/ceph/ceph-client.0.50071.asok 2023-07-14T11:56:24.566 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50071.asok 2023-07-14T11:56:24.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:24.567 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50071.asok status 2023-07-14T11:56:24.765 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:56:24.765 INFO:teuthology.orchestra.run.smithi104.stdout: "metadata": { 2023-07-14T11:56:24.765 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-14T11:56:24.766 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:56:24.766 INFO:teuthology.orchestra.run.smithi104.stdout: "entity_id": "0", 2023-07-14T11:56:24.766 INFO:teuthology.orchestra.run.smithi104.stdout: "hostname": "smithi104", 2023-07-14T11:56:24.766 INFO:teuthology.orchestra.run.smithi104.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-14T11:56:24.766 INFO:teuthology.orchestra.run.smithi104.stdout: "pid": "50071", 2023-07-14T11:56:24.766 INFO:teuthology.orchestra.run.smithi104.stdout: "root": "/" 2023-07-14T11:56:24.766 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:24.766 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_count": 0, 2023-07-14T11:56:24.767 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_pinned_count": 0, 2023-07-14T11:56:24.767 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 6486, 2023-07-14T11:56:24.767 INFO:teuthology.orchestra.run.smithi104.stdout: "inst": { 2023-07-14T11:56:24.767 INFO:teuthology.orchestra.run.smithi104.stdout: "name": { 2023-07-14T11:56:24.767 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "client", 2023-07-14T11:56:24.767 INFO:teuthology.orchestra.run.smithi104.stdout: "num": 6486 2023-07-14T11:56:24.767 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:24.767 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:56:24.768 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:56:24.768 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:56:24.768 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 3199381981 2023-07-14T11:56:24.769 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:24.769 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:24.769 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:56:24.769 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:56:24.769 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:56:24.769 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 3199381981 2023-07-14T11:56:24.769 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:24.770 INFO:teuthology.orchestra.run.smithi104.stdout: "inst_str": "client.6486 192.168.0.1:0/3199381981", 2023-07-14T11:56:24.770 INFO:teuthology.orchestra.run.smithi104.stdout: "addr_str": "192.168.0.1:0/3199381981", 2023-07-14T11:56:24.770 INFO:teuthology.orchestra.run.smithi104.stdout: "inode_count": 1, 2023-07-14T11:56:24.770 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_epoch": 27, 2023-07-14T11:56:24.770 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch": 59, 2023-07-14T11:56:24.770 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch_barrier": 0, 2023-07-14T11:56:24.770 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklisted": false, 2023-07-14T11:56:24.770 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs" 2023-07-14T11:56:24.771 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:56:24.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:24.779 DEBUG:teuthology.orchestra.run.smithi104:> stat --file-system '--printf=%T 2023-07-14T11:56:24.780 DEBUG:teuthology.orchestra.run.smithi104:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:24.792 INFO:teuthology.orchestra.run.smithi104.stdout:fuseblk 2023-07-14T11:56:24.793 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:24.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:24.793 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:24.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:24.869 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file) 2023-07-14T11:56:24.946 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow r pool=cephfs_data' 2023-07-14T11:56:25.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.189+0000 7f526d3af700 1 Processor -- start 2023-07-14T11:56:25.179 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.189+0000 7f526d3af700 1 -- start start 2023-07-14T11:56:25.179 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f526d3af700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f526815d050 0x7f52680cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:25.180 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f526d3af700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f526806f070 0x7f52680cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:25.180 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f526d3af700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f526815d420 con 0x7f52680cab70 2023-07-14T11:56:25.180 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f526d3af700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52680cc260 con 0x7f526815d050 2023-07-14T11:56:25.180 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f526d3af700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52680cb5c0 con 0x7f526806f070 2023-07-14T11:56:25.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f5267fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f526815d050 0x7f52680cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:25.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f5267fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f526815d050 0x7f52680cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46074/0 (socket says 172.21.15.104:46074) 2023-07-14T11:56:25.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f5267fff700 1 -- 172.21.15.104:0/2110837387 learned_addr learned my addr 172.21.15.104:0/2110837387 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:25.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f5267fff700 1 -- 172.21.15.104:0/2110837387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f526806f070 msgr2=0x7f52680cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:25.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f5267fff700 1 --2- 172.21.15.104:0/2110837387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f526806f070 0x7f52680cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f5267fff700 1 -- 172.21.15.104:0/2110837387 >> v1:172.21.15.121:6789/0 conn(0x7f52680cab70 legacy=0x7f526806e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:25.182 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f5267fff700 1 -- 172.21.15.104:0/2110837387 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52680cb8a0 con 0x7f526815d050 2023-07-14T11:56:25.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f5267fff700 1 --2- 172.21.15.104:0/2110837387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526815d050 0x7f52680cc6f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f526806f780 tx=0x7f5258009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9312fee13c15f89 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:25.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f5266ffd700 1 -- 172.21.15.104:0/2110837387 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525800b860 con 0x7f526815d050 2023-07-14T11:56:25.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f5266ffd700 1 -- 172.21.15.104:0/2110837387 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525800b9c0 con 0x7f526815d050 2023-07-14T11:56:25.183 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f5266ffd700 1 -- 172.21.15.104:0/2110837387 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525800bd50 con 0x7f526815d050 2023-07-14T11:56:25.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f526d3af700 1 -- 172.21.15.104:0/2110837387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526815d050 msgr2=0x7f52680cc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:25.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f526d3af700 1 --2- 172.21.15.104:0/2110837387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526815d050 0x7f52680cc6f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f526806f780 tx=0x7f5258009bf0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f526d3af700 1 -- 172.21.15.104:0/2110837387 shutdown_connections 2023-07-14T11:56:25.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f526d3af700 1 --2- 172.21.15.104:0/2110837387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f526806f070 0x7f52680cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f526d3af700 1 --2- 172.21.15.104:0/2110837387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526815d050 0x7f52680cc6f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.193+0000 7f526d3af700 1 -- 172.21.15.104:0/2110837387 >> 172.21.15.104:0/2110837387 conn(0x7f526806fe60 msgr2=0x7f5268070260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:25.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.197+0000 7f526d3af700 1 -- 172.21.15.104:0/2110837387 shutdown_connections 2023-07-14T11:56:25.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.197+0000 7f526d3af700 1 -- 172.21.15.104:0/2110837387 wait complete. 2023-07-14T11:56:25.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.197+0000 7f526d3af700 1 Processor -- start 2023-07-14T11:56:25.186 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.197+0000 7f526d3af700 1 -- start start 2023-07-14T11:56:25.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526d3af700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f526806f070 0x7f52680c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:25.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526d3af700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52680cab70 0x7f52681f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:25.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526d3af700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526815d050 0x7f52680c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:25.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526d3af700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52680c25e0 con 0x7f526815d050 2023-07-14T11:56:25.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526d3af700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52680b2b00 con 0x7f526806f070 2023-07-14T11:56:25.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526d3af700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52680b2de0 con 0x7f52680cab70 2023-07-14T11:56:25.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526cbae700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526815d050 0x7f52680c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:25.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526cbae700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526815d050 0x7f52680c3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46090/0 (socket says 172.21.15.104:46090) 2023-07-14T11:56:25.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526cbae700 1 -- 172.21.15.104:0/1371471270 learned_addr learned my addr 172.21.15.104:0/1371471270 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:25.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f52677fe700 1 --2- 172.21.15.104:0/1371471270 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52680cab70 0x7f52681f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:25.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f5267fff700 1 --2- 172.21.15.104:0/1371471270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f526806f070 0x7f52680c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:25.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526cbae700 1 -- 172.21.15.104:0/1371471270 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52680cab70 msgr2=0x7f52681f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:25.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526cbae700 1 --2- 172.21.15.104:0/1371471270 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52680cab70 0x7f52681f4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526cbae700 1 -- 172.21.15.104:0/1371471270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f526806f070 msgr2=0x7f52680c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:25.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526cbae700 1 --2- 172.21.15.104:0/1371471270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f526806f070 0x7f52680c20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526cbae700 1 -- 172.21.15.104:0/1371471270 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5258009570 con 0x7f526815d050 2023-07-14T11:56:25.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526cbae700 1 --2- 172.21.15.104:0/1371471270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526815d050 0x7f52680c3a20 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f526000b9e0 tx=0x7f526000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:25.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f5264ff9700 1 -- 172.21.15.104:0/1371471270 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000d8e0 con 0x7f526815d050 2023-07-14T11:56:25.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f5264ff9700 1 -- 172.21.15.104:0/1371471270 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526000da60 con 0x7f526815d050 2023-07-14T11:56:25.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526d3af700 1 -- 172.21.15.104:0/1371471270 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52680b30c0 con 0x7f526815d050 2023-07-14T11:56:25.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f5264ff9700 1 -- 172.21.15.104:0/1371471270 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000de10 con 0x7f526815d050 2023-07-14T11:56:25.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.201+0000 7f526d3af700 1 -- 172.21.15.104:0/1371471270 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52680b3680 con 0x7f526815d050 2023-07-14T11:56:25.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.205+0000 7f526d3af700 1 -- 172.21.15.104:0/1371471270 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f526804e500 con 0x7f526815d050 2023-07-14T11:56:25.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.205+0000 7f5264ff9700 1 -- 172.21.15.104:0/1371471270 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5260015420 con 0x7f526815d050 2023-07-14T11:56:25.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.209+0000 7f5264ff9700 1 --2- 172.21.15.104:0/1371471270 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52540662f0 0x7f52540687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:25.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.209+0000 7f5264ff9700 1 -- 172.21.15.104:0/1371471270 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5260011070 con 0x7f526815d050 2023-07-14T11:56:25.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.209+0000 7f5267fff700 1 --2- 172.21.15.104:0/1371471270 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52540662f0 0x7f52540687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:25.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.209+0000 7f5267fff700 1 --2- 172.21.15.104:0/1371471270 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52540662f0 0x7f52540687a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5258000c00 tx=0x7f5258008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:25.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.213+0000 7f5264ff9700 1 -- 172.21.15.104:0/1371471270 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f526004c720 con 0x7f526815d050 2023-07-14T11:56:25.431 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.441+0000 7f526d3af700 1 -- 172.21.15.104:0/1371471270 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]} v 0) v1 -- 0x7f526806d4a0 con 0x7f526815d050 2023-07-14T11:56:25.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.453+0000 7f5264ff9700 1 -- 172.21.15.104:0/1371471270 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]}]=0 updated caps for client.0 v18) v1 ==== 183+0+140 (secure 0 0 0) 0x7f5260050e80 con 0x7f526815d050 2023-07-14T11:56:25.440 INFO:teuthology.orchestra.run.smithi104.stdout:[client.0] 2023-07-14T11:56:25.440 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDrNrFkbrzjERAA+9BBzuauwUdhALUcDtd80A== 2023-07-14T11:56:25.440 INFO:teuthology.orchestra.run.smithi104.stdout: caps mds = "allow" 2023-07-14T11:56:25.440 INFO:teuthology.orchestra.run.smithi104.stdout: caps mon = "allow r" 2023-07-14T11:56:25.441 INFO:teuthology.orchestra.run.smithi104.stdout: caps osd = "allow r pool=cephfs_data" 2023-07-14T11:56:25.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.457+0000 7f526d3af700 1 -- 172.21.15.104:0/1371471270 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52540662f0 msgr2=0x7f52540687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:25.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.457+0000 7f526d3af700 1 --2- 172.21.15.104:0/1371471270 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52540662f0 0x7f52540687a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5258000c00 tx=0x7f5258008030 comp rx=0 tx=0).stop 2023-07-14T11:56:25.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.457+0000 7f526d3af700 1 -- 172.21.15.104:0/1371471270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526815d050 msgr2=0x7f52680c3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:25.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.457+0000 7f526d3af700 1 --2- 172.21.15.104:0/1371471270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526815d050 0x7f52680c3a20 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f526000b9e0 tx=0x7f526000bcf0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.457+0000 7f526d3af700 1 -- 172.21.15.104:0/1371471270 shutdown_connections 2023-07-14T11:56:25.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.457+0000 7f526d3af700 1 --2- 172.21.15.104:0/1371471270 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f52540662f0 0x7f52540687a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.457+0000 7f526d3af700 1 --2- 172.21.15.104:0/1371471270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526815d050 0x7f52680c3a20 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.457+0000 7f526d3af700 1 --2- 172.21.15.104:0/1371471270 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f52680cab70 0x7f52681f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.457+0000 7f526d3af700 1 --2- 172.21.15.104:0/1371471270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f526806f070 0x7f52680c20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.457+0000 7f526d3af700 1 -- 172.21.15.104:0/1371471270 >> 172.21.15.104:0/1371471270 conn(0x7f526806fe60 msgr2=0x7f52680c7030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:25.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.457+0000 7f526d3af700 1 -- 172.21.15.104:0/1371471270 shutdown_connections 2023-07-14T11:56:25.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.457+0000 7f526d3af700 1 -- 172.21.15.104:0/1371471270 wait complete. 2023-07-14T11:56:25.447 INFO:teuthology.orchestra.run.smithi104.stderr:updated caps for client.0 2023-07-14T11:56:25.466 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:56:25.466 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:56:25.481 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:56:25.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.697+0000 7f33c0850700 1 Processor -- start 2023-07-14T11:56:25.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.697+0000 7f33c0850700 1 -- start start 2023-07-14T11:56:25.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33c0850700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f33bc0ca510 0x7f33bc0713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:25.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33c0850700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33bc0b0620 0x7f33bc0cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:25.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33c0850700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33bc089de0 con 0x7f33bc0cbc30 2023-07-14T11:56:25.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33c0850700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33bc071920 con 0x7f33bc0ca510 2023-07-14T11:56:25.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33c0850700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc0cc0c0 con 0x7f33bc0b0620 2023-07-14T11:56:25.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33bad9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f33bc0ca510 0x7f33bc0713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:25.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33bad9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f33bc0ca510 0x7f33bc0713e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46102/0 (socket says 172.21.15.104:46102) 2023-07-14T11:56:25.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33bad9d700 1 -- 172.21.15.104:0/2472976591 learned_addr learned my addr 172.21.15.104:0/2472976591 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:25.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33bb59e700 1 --2- 172.21.15.104:0/2472976591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33bc0b0620 0x7f33bc0cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:25.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33b9d9b700 1 -- 172.21.15.104:0/2472976591 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2094668719 0 0) 0x7f33bc089de0 con 0x7f33bc0cbc30 2023-07-14T11:56:25.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33b9d9b700 1 -- 172.21.15.104:0/2472976591 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33a40035d0 con 0x7f33bc0cbc30 2023-07-14T11:56:25.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33bad9d700 1 -- 172.21.15.104:0/2472976591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33bc0b0620 msgr2=0x7f33bc0cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:25.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33bad9d700 1 --2- 172.21.15.104:0/2472976591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33bc0b0620 0x7f33bc0cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33bad9d700 1 -- 172.21.15.104:0/2472976591 >> v1:172.21.15.121:6789/0 conn(0x7f33bc0cbc30 legacy=0x7f33bc0aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:25.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33bad9d700 1 -- 172.21.15.104:0/2472976591 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33bc0cc3a0 con 0x7f33bc0ca510 2023-07-14T11:56:25.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33bad9d700 1 --2- 172.21.15.104:0/2472976591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33bc0ca510 0x7f33bc0713e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f33ac008990 tx=0x7f33ac008ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e7f484692e39e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:25.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.701+0000 7f33b9d9b700 1 -- 172.21.15.104:0/2472976591 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00fbd0 con 0x7f33bc0ca510 2023-07-14T11:56:25.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.705+0000 7f33b9d9b700 1 -- 172.21.15.104:0/2472976591 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac00fd30 con 0x7f33bc0ca510 2023-07-14T11:56:25.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.705+0000 7f33b9d9b700 1 -- 172.21.15.104:0/2472976591 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00fea0 con 0x7f33bc0ca510 2023-07-14T11:56:25.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.705+0000 7f33c0850700 1 -- 172.21.15.104:0/2472976591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33bc0ca510 msgr2=0x7f33bc0713e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:25.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.705+0000 7f33c0850700 1 --2- 172.21.15.104:0/2472976591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33bc0ca510 0x7f33bc0713e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f33ac008990 tx=0x7f33ac008ca0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.705+0000 7f33c0850700 1 -- 172.21.15.104:0/2472976591 shutdown_connections 2023-07-14T11:56:25.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.705+0000 7f33c0850700 1 --2- 172.21.15.104:0/2472976591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33bc0b0620 0x7f33bc0cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.705+0000 7f33c0850700 1 --2- 172.21.15.104:0/2472976591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33bc0ca510 0x7f33bc0713e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.705+0000 7f33c0850700 1 -- 172.21.15.104:0/2472976591 >> 172.21.15.104:0/2472976591 conn(0x7f33bc060320 msgr2=0x7f33bc0606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:25.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.705+0000 7f33c0850700 1 -- 172.21.15.104:0/2472976591 shutdown_connections 2023-07-14T11:56:25.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.705+0000 7f33c0850700 1 -- 172.21.15.104:0/2472976591 wait complete. 2023-07-14T11:56:25.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.705+0000 7f33c0850700 1 Processor -- start 2023-07-14T11:56:25.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33c0850700 1 -- start start 2023-07-14T11:56:25.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33c0850700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33bc0b0620 0x7f33bc1678a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:25.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33c0850700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33bc0ca510 0x7f33bc169960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:25.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33c0850700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33bc0cbc30 0x7f33bc0b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:25.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33c0850700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc16aad0 con 0x7f33bc0b0620 2023-07-14T11:56:25.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33c0850700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc16adb0 con 0x7f33bc0cbc30 2023-07-14T11:56:25.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33c0850700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc16b090 con 0x7f33bc0ca510 2023-07-14T11:56:25.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33bad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33bc0b0620 0x7f33bc1678a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:25.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33bad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33bc0b0620 0x7f33bc1678a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46106/0 (socket says 172.21.15.104:46106) 2023-07-14T11:56:25.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33bad9d700 1 -- 172.21.15.104:0/2795359949 learned_addr learned my addr 172.21.15.104:0/2795359949 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:25.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33bb59e700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33bc0cbc30 0x7f33bc0b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:25.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33ba59c700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33bc0ca510 0x7f33bc169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:25.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33bad9d700 1 -- 172.21.15.104:0/2795359949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33bc0ca510 msgr2=0x7f33bc169960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:25.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33bad9d700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33bc0ca510 0x7f33bc169960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33bad9d700 1 -- 172.21.15.104:0/2795359949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33bc0cbc30 msgr2=0x7f33bc0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:25.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33bad9d700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33bc0cbc30 0x7f33bc0b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33bad9d700 1 -- 172.21.15.104:0/2795359949 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac0085f0 con 0x7f33bc0b0620 2023-07-14T11:56:25.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33bb59e700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33bc0cbc30 0x7f33bc0b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:25.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33ba59c700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33bc0ca510 0x7f33bc169960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:25.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33bad9d700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33bc0b0620 0x7f33bc1678a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0cacc0 tx=0x7f33ac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:25.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33a37fe700 1 -- 172.21.15.104:0/2795359949 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00fea0 con 0x7f33bc0b0620 2023-07-14T11:56:25.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33a37fe700 1 -- 172.21.15.104:0/2795359949 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac006f40 con 0x7f33bc0b0620 2023-07-14T11:56:25.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33c0850700 1 -- 172.21.15.104:0/2795359949 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33bc0c1d30 con 0x7f33bc0b0620 2023-07-14T11:56:25.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33a37fe700 1 -- 172.21.15.104:0/2795359949 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00fbd0 con 0x7f33bc0b0620 2023-07-14T11:56:25.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.709+0000 7f33c0850700 1 -- 172.21.15.104:0/2795359949 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33bc169ea0 con 0x7f33bc0b0620 2023-07-14T11:56:25.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.713+0000 7f33c0850700 1 -- 172.21.15.104:0/2795359949 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33bc0c2250 con 0x7f33bc0b0620 2023-07-14T11:56:25.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.713+0000 7f33a37fe700 1 -- 172.21.15.104:0/2795359949 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f33ac01e030 con 0x7f33bc0b0620 2023-07-14T11:56:25.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.717+0000 7f33a37fe700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f33a8066320 0x7f33a80687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:25.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.717+0000 7f33a37fe700 1 -- 172.21.15.104:0/2795359949 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f33ac089360 con 0x7f33bc0b0620 2023-07-14T11:56:25.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.717+0000 7f33ba59c700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f33a8066320 0x7f33a80687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:25.706 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.717+0000 7f33ba59c700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f33a8066320 0x7f33a80687d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0ce490 tx=0x7f33b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:25.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.721+0000 7f33a37fe700 1 -- 172.21.15.104:0/2795359949 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f33bc0c2250 con 0x7f33bc0b0620 2023-07-14T11:56:25.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.893+0000 7f33c0850700 1 -- 172.21.15.104:0/2795359949 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33bc0c2250 con 0x7f33bc0b0620 2023-07-14T11:56:25.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.897+0000 7f33a37fe700 1 -- 172.21.15.104:0/2795359949 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+18534 (secure 0 0 0) 0x7f33bc0c2250 con 0x7f33bc0b0620 2023-07-14T11:56:25.884 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:25.884 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":59,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:56:12.692633+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:56:08.755551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-14T11:56:10.132688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:56:25.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.901+0000 7f33c0850700 1 -- 172.21.15.104:0/2795359949 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f33a8066320 msgr2=0x7f33a80687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:25.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.901+0000 7f33c0850700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f33a8066320 0x7f33a80687d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0ce490 tx=0x7f33b0009040 comp rx=0 tx=0).stop 2023-07-14T11:56:25.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.901+0000 7f33c0850700 1 -- 172.21.15.104:0/2795359949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33bc0b0620 msgr2=0x7f33bc1678a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:25.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.901+0000 7f33c0850700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33bc0b0620 0x7f33bc1678a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0cacc0 tx=0x7f33ac004770 comp rx=0 tx=0).stop 2023-07-14T11:56:25.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.901+0000 7f33c0850700 1 -- 172.21.15.104:0/2795359949 shutdown_connections 2023-07-14T11:56:25.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.901+0000 7f33c0850700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f33a8066320 0x7f33a80687d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.901+0000 7f33c0850700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33bc0cbc30 0x7f33bc0b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.901+0000 7f33c0850700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33bc0ca510 0x7f33bc169960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.901+0000 7f33c0850700 1 --2- 172.21.15.104:0/2795359949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33bc0b0620 0x7f33bc1678a0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:25.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.901+0000 7f33c0850700 1 -- 172.21.15.104:0/2795359949 >> 172.21.15.104:0/2795359949 conn(0x7f33bc060320 msgr2=0x7f33bc0cd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:25.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.901+0000 7f33c0850700 1 -- 172.21.15.104:0/2795359949 shutdown_connections 2023-07-14T11:56:25.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:25.901+0000 7f33c0850700 1 -- 172.21.15.104:0/2795359949 wait complete. 2023-07-14T11:56:25.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:56:08.755551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:56:10.132688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:56:25.910 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:56:26.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.097+0000 7f1e64308700 1 Processor -- start 2023-07-14T11:56:26.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.097+0000 7f1e64308700 1 -- start start 2023-07-14T11:56:26.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e64308700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1e5c0b4130 0x7f1e5c0d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e64308700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1e5c15c1a0 0x7f1e5c0b4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e64308700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c0b0ca0 con 0x7f1e5c0b1130 2023-07-14T11:56:26.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e64308700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5c0d2530 con 0x7f1e5c0b4130 2023-07-14T11:56:26.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e64308700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5c0b03b0 con 0x7f1e5c15c1a0 2023-07-14T11:56:26.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e620a4700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1e5c0b4130 0x7f1e5c0d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e620a4700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1e5c0b4130 0x7f1e5c0d1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46108/0 (socket says 172.21.15.104:46108) 2023-07-14T11:56:26.091 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e620a4700 1 -- 172.21.15.104:0/2277364255 learned_addr learned my addr 172.21.15.104:0/2277364255 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:26.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e618a3700 1 --2- 172.21.15.104:0/2277364255 >> v2:172.21.15.121:3300/0 conn(0x7f1e5c15c1a0 0x7f1e5c0b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e620a4700 1 -- 172.21.15.104:0/2277364255 >> v2:172.21.15.121:3300/0 conn(0x7f1e5c15c1a0 msgr2=0x7f1e5c0b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:26.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e620a4700 1 --2- 172.21.15.104:0/2277364255 >> v2:172.21.15.121:3300/0 conn(0x7f1e5c15c1a0 0x7f1e5c0b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e620a4700 1 -- 172.21.15.104:0/2277364255 >> v1:172.21.15.104:6789/0 conn(0x7f1e5c0b1130 legacy=0x7f1e5c0af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e620a4700 1 -- 172.21.15.104:0/2277364255 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c0b4a40 con 0x7f1e5c0b4130 2023-07-14T11:56:26.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e620a4700 1 --2- 172.21.15.104:0/2277364255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e5c0b4130 0x7f1e5c0d1ff0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c06f910 tx=0x7f1e58009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97376aa511caa44 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:26.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e610a2700 1 -- 172.21.15.104:0/2277364255 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e58005e50 con 0x7f1e5c0b4130 2023-07-14T11:56:26.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e610a2700 1 -- 172.21.15.104:0/2277364255 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e58006030 con 0x7f1e5c0b4130 2023-07-14T11:56:26.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e610a2700 1 -- 172.21.15.104:0/2277364255 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e580063c0 con 0x7f1e5c0b4130 2023-07-14T11:56:26.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e64308700 1 -- 172.21.15.104:0/2277364255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e5c0b4130 msgr2=0x7f1e5c0d1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.101+0000 7f1e64308700 1 --2- 172.21.15.104:0/2277364255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e5c0b4130 0x7f1e5c0d1ff0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c06f910 tx=0x7f1e58009bf0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.105+0000 7f1e64308700 1 -- 172.21.15.104:0/2277364255 shutdown_connections 2023-07-14T11:56:26.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.105+0000 7f1e64308700 1 --2- 172.21.15.104:0/2277364255 >> v2:172.21.15.121:3300/0 conn(0x7f1e5c15c1a0 0x7f1e5c0b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.105+0000 7f1e64308700 1 --2- 172.21.15.104:0/2277364255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e5c0b4130 0x7f1e5c0d1ff0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.105+0000 7f1e64308700 1 -- 172.21.15.104:0/2277364255 >> 172.21.15.104:0/2277364255 conn(0x7f1e5c09ba10 msgr2=0x7f1e5c0d26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:26.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.105+0000 7f1e64308700 1 -- 172.21.15.104:0/2277364255 shutdown_connections 2023-07-14T11:56:26.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.105+0000 7f1e64308700 1 -- 172.21.15.104:0/2277364255 wait complete. 2023-07-14T11:56:26.096 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.109+0000 7f1e64308700 1 Processor -- start 2023-07-14T11:56:26.097 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.109+0000 7f1e64308700 1 -- start start 2023-07-14T11:56:26.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.109+0000 7f1e64308700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e5c0b1130 0x7f1e5c1f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.109+0000 7f1e64308700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5c0b4130 0x7f1e5c0b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.109+0000 7f1e64308700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e5c15c1a0 0x7f1e5c0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.109+0000 7f1e64308700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0cc4e0 con 0x7f1e5c15c1a0 2023-07-14T11:56:26.098 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.109+0000 7f1e64308700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c1f5690 con 0x7f1e5c0b4130 2023-07-14T11:56:26.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.109+0000 7f1e64308700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0b5610 con 0x7f1e5c0b1130 2023-07-14T11:56:26.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.109+0000 7f1e628a5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e5c15c1a0 0x7f1e5c0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.109+0000 7f1e628a5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e5c15c1a0 0x7f1e5c0b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46120/0 (socket says 172.21.15.104:46120) 2023-07-14T11:56:26.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.109+0000 7f1e628a5700 1 -- 172.21.15.104:0/2403616555 learned_addr learned my addr 172.21.15.104:0/2403616555 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:26.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.109+0000 7f1e620a4700 1 --2- 172.21.15.104:0/2403616555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e5c0b1130 0x7f1e5c1f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.109+0000 7f1e618a3700 1 --2- 172.21.15.104:0/2403616555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5c0b4130 0x7f1e5c0b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.113+0000 7f1e628a5700 1 -- 172.21.15.104:0/2403616555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e5c0b1130 msgr2=0x7f1e5c1f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.113+0000 7f1e628a5700 1 --2- 172.21.15.104:0/2403616555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e5c0b1130 0x7f1e5c1f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.113+0000 7f1e628a5700 1 -- 172.21.15.104:0/2403616555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5c0b4130 msgr2=0x7f1e5c0b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.113+0000 7f1e628a5700 1 --2- 172.21.15.104:0/2403616555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5c0b4130 0x7f1e5c0b22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.113+0000 7f1e628a5700 1 -- 172.21.15.104:0/2403616555 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e58009520 con 0x7f1e5c15c1a0 2023-07-14T11:56:26.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.113+0000 7f1e628a5700 1 --2- 172.21.15.104:0/2403616555 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e5c15c1a0 0x7f1e5c0b2810 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400cb50 tx=0x7f1e5400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:26.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.113+0000 7f1e52ffd700 1 -- 172.21.15.104:0/2403616555 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54003c20 con 0x7f1e5c15c1a0 2023-07-14T11:56:26.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.113+0000 7f1e52ffd700 1 -- 172.21.15.104:0/2403616555 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e54003d80 con 0x7f1e5c15c1a0 2023-07-14T11:56:26.101 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.113+0000 7f1e64308700 1 -- 172.21.15.104:0/2403616555 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c0b58f0 con 0x7f1e5c15c1a0 2023-07-14T11:56:26.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.113+0000 7f1e52ffd700 1 -- 172.21.15.104:0/2403616555 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5400ab70 con 0x7f1e5c15c1a0 2023-07-14T11:56:26.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.113+0000 7f1e64308700 1 -- 172.21.15.104:0/2403616555 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c0b5eb0 con 0x7f1e5c15c1a0 2023-07-14T11:56:26.102 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.113+0000 7f1e64308700 1 -- 172.21.15.104:0/2403616555 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5c04dcb0 con 0x7f1e5c15c1a0 2023-07-14T11:56:26.103 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.113+0000 7f1e52ffd700 1 -- 172.21.15.104:0/2403616555 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1e5400acd0 con 0x7f1e5c15c1a0 2023-07-14T11:56:26.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.117+0000 7f1e52ffd700 1 --2- 172.21.15.104:0/2403616555 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1e400662f0 0x7f1e400687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.117+0000 7f1e52ffd700 1 -- 172.21.15.104:0/2403616555 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1e54083ed0 con 0x7f1e5c15c1a0 2023-07-14T11:56:26.105 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.117+0000 7f1e620a4700 1 --2- 172.21.15.104:0/2403616555 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1e400662f0 0x7f1e400687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.117+0000 7f1e620a4700 1 --2- 172.21.15.104:0/2403616555 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1e400662f0 0x7f1e400687a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009c30 tx=0x7f1e58008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:26.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.121+0000 7f1e52ffd700 1 -- 172.21.15.104:0/2403616555 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f1e54051200 con 0x7f1e5c15c1a0 2023-07-14T11:56:26.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.297+0000 7f1e64308700 1 -- 172.21.15.104:0/2403616555 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e5c15b950 con 0x7f1e5c15c1a0 2023-07-14T11:56:26.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.297+0000 7f1e52ffd700 1 -- 172.21.15.104:0/2403616555 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e54054820 con 0x7f1e5c15c1a0 2023-07-14T11:56:26.284 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:56:26.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.297+0000 7f1e64308700 1 -- 172.21.15.104:0/2403616555 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1e400662f0 msgr2=0x7f1e400687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.297+0000 7f1e64308700 1 --2- 172.21.15.104:0/2403616555 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1e400662f0 0x7f1e400687a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009c30 tx=0x7f1e58008070 comp rx=0 tx=0).stop 2023-07-14T11:56:26.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.301+0000 7f1e64308700 1 -- 172.21.15.104:0/2403616555 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e5c15c1a0 msgr2=0x7f1e5c0b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.301+0000 7f1e64308700 1 --2- 172.21.15.104:0/2403616555 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e5c15c1a0 0x7f1e5c0b2810 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400cb50 tx=0x7f1e5400cf10 comp rx=0 tx=0).stop 2023-07-14T11:56:26.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.301+0000 7f1e64308700 1 -- 172.21.15.104:0/2403616555 shutdown_connections 2023-07-14T11:56:26.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.301+0000 7f1e64308700 1 --2- 172.21.15.104:0/2403616555 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f1e400662f0 0x7f1e400687a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.301+0000 7f1e64308700 1 --2- 172.21.15.104:0/2403616555 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e5c15c1a0 0x7f1e5c0b2810 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.301+0000 7f1e64308700 1 --2- 172.21.15.104:0/2403616555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5c0b4130 0x7f1e5c0b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.301+0000 7f1e64308700 1 --2- 172.21.15.104:0/2403616555 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e5c0b1130 0x7f1e5c1f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.301+0000 7f1e64308700 1 -- 172.21.15.104:0/2403616555 >> 172.21.15.104:0/2403616555 conn(0x7f1e5c09ba10 msgr2=0x7f1e5c0afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:26.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.301+0000 7f1e64308700 1 -- 172.21.15.104:0/2403616555 shutdown_connections 2023-07-14T11:56:26.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.301+0000 7f1e64308700 1 -- 172.21.15.104:0/2403616555 wait complete. 2023-07-14T11:56:26.310 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:56:26.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.497+0000 7f7069c40700 1 Processor -- start 2023-07-14T11:56:26.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.497+0000 7f7069c40700 1 -- start start 2023-07-14T11:56:26.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f7069c40700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f70640b0ee0 0x7f70640ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f7069c40700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70640cd4e0 con 0x7f70640d07e0 2023-07-14T11:56:26.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f7069c40700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70640d1a60 con 0x7f70640cd090 2023-07-14T11:56:26.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f7069c40700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70640d0c60 con 0x7f70640b0ee0 2023-07-14T11:56:26.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f706943f700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f70640d07e0 0x7f70640af0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:39870/0 (socket says 172.21.15.104:39870) 2023-07-14T11:56:26.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f706943f700 1 -- 172.21.15.104:0/266893003 learned_addr learned my addr 172.21.15.104:0/266893003 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:26.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f7068c3e700 1 --2- 172.21.15.104:0/266893003 >> v2:172.21.15.121:3300/0 conn(0x7f70640b0ee0 0x7f70640ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f70637fe700 1 -- 172.21.15.104:0/266893003 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521818584 0 0) 0x7f705c001490 con 0x7f70640d07e0 2023-07-14T11:56:26.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f70637fe700 1 -- 172.21.15.104:0/266893003 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70540035d0 con 0x7f70640d07e0 2023-07-14T11:56:26.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f70637fe700 1 -- 172.21.15.104:0/266893003 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4006159041 0 0) 0x7f70540035d0 con 0x7f70640d07e0 2023-07-14T11:56:26.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f70637fe700 1 -- 172.21.15.104:0/266893003 >> v2:172.21.15.121:3300/0 conn(0x7f70640b0ee0 msgr2=0x7f70640ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:26.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f70637fe700 1 --2- 172.21.15.104:0/266893003 >> v2:172.21.15.121:3300/0 conn(0x7f70640b0ee0 0x7f70640ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f70637fe700 1 -- 172.21.15.104:0/266893003 >> v1:172.21.15.121:6789/0 conn(0x7f70640cd090 legacy=0x7f70640d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f70637fe700 1 -- 172.21.15.104:0/266893003 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70640d1ef0 con 0x7f70640d07e0 2023-07-14T11:56:26.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f70637fe700 1 -- 172.21.15.104:0/266893003 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f705c001b80 con 0x7f70640d07e0 2023-07-14T11:56:26.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f70637fe700 1 -- 172.21.15.104:0/266893003 >> v1:172.21.15.104:6789/0 conn(0x7f70640d07e0 legacy=0x7f70640af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f70637fe700 1 --2- 172.21.15.104:0/266893003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7054003a20 0x7f7054003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f70637fe700 1 --2- 172.21.15.104:0/266893003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7054004410 0x7f70540048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f70637fe700 1 --2- 172.21.15.104:0/266893003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70540050d0 0x7f705400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f70637fe700 1 -- 172.21.15.104:0/266893003 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705400c020 con 0x7f7054003a20 2023-07-14T11:56:26.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f70637fe700 1 -- 172.21.15.104:0/266893003 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705400c160 con 0x7f7054004410 2023-07-14T11:56:26.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f70637fe700 1 -- 172.21.15.104:0/266893003 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705400c320 con 0x7f70540050d0 2023-07-14T11:56:26.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f7068c3e700 1 --2- 172.21.15.104:0/266893003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7054003a20 0x7f7054003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f7063fff700 1 --2- 172.21.15.104:0/266893003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7054004410 0x7f70540048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f706943f700 1 --2- 172.21.15.104:0/266893003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70540050d0 0x7f705400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f7068c3e700 1 -- 172.21.15.104:0/266893003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70540050d0 msgr2=0x7f705400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:26.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f7068c3e700 1 --2- 172.21.15.104:0/266893003 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70540050d0 0x7f705400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f7068c3e700 1 -- 172.21.15.104:0/266893003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7054004410 msgr2=0x7f70540048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:26.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f7068c3e700 1 --2- 172.21.15.104:0/266893003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7054004410 0x7f70540048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f7068c3e700 1 -- 172.21.15.104:0/266893003 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70540035d0 con 0x7f7054003a20 2023-07-14T11:56:26.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f706943f700 1 -- 172.21.15.104:0/266893003 reap_dead start 2023-07-14T11:56:26.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.501+0000 7f7063fff700 1 --2- 172.21.15.104:0/266893003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7054004410 0x7f70540048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:26.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.505+0000 7f7068c3e700 1 --2- 172.21.15.104:0/266893003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7054003a20 0x7f7054003df0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7058009b70 tx=0x7f70540058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e888600b01d5f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:26.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.505+0000 7f70637fe700 1 -- 172.21.15.104:0/266893003 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705800e6c0 con 0x7f7054003a20 2023-07-14T11:56:26.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.505+0000 7f70637fe700 1 -- 172.21.15.104:0/266893003 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705800e820 con 0x7f7054003a20 2023-07-14T11:56:26.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.505+0000 7f70637fe700 1 -- 172.21.15.104:0/266893003 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705800eb90 con 0x7f7054003a20 2023-07-14T11:56:26.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.505+0000 7f7069c40700 1 -- 172.21.15.104:0/266893003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7054003a20 msgr2=0x7f7054003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.505+0000 7f7069c40700 1 --2- 172.21.15.104:0/266893003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7054003a20 0x7f7054003df0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7058009b70 tx=0x7f70540058a0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.505+0000 7f7069c40700 1 -- 172.21.15.104:0/266893003 shutdown_connections 2023-07-14T11:56:26.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.505+0000 7f7069c40700 1 --2- 172.21.15.104:0/266893003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7054003a20 0x7f7054003df0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.505+0000 7f7069c40700 1 -- 172.21.15.104:0/266893003 >> 172.21.15.104:0/266893003 conn(0x7f7064060c40 msgr2=0x7f706409b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:26.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.505+0000 7f7069c40700 1 -- 172.21.15.104:0/266893003 shutdown_connections 2023-07-14T11:56:26.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.505+0000 7f7069c40700 1 -- 172.21.15.104:0/266893003 wait complete. 2023-07-14T11:56:26.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7069c40700 1 Processor -- start 2023-07-14T11:56:26.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7069c40700 1 -- start start 2023-07-14T11:56:26.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7069c40700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7054003a20 0x7f70640cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7069c40700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70641f9420 0x7f70641f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7069c40700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f706415eb80 0x7f706415eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7069c40700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706415f530 con 0x7f7054003a20 2023-07-14T11:56:26.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7069c40700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70640cbee0 con 0x7f70641f9420 2023-07-14T11:56:26.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7069c40700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70640cb2d0 con 0x7f706415eb80 2023-07-14T11:56:26.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7068c3e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7054003a20 0x7f70640cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7068c3e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7054003a20 0x7f70640cac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46140/0 (socket says 172.21.15.104:46140) 2023-07-14T11:56:26.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7068c3e700 1 -- 172.21.15.104:0/3700043622 learned_addr learned my addr 172.21.15.104:0/3700043622 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:26.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7063fff700 1 --2- 172.21.15.104:0/3700043622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70641f9420 0x7f70641f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f706943f700 1 --2- 172.21.15.104:0/3700043622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f706415eb80 0x7f706415eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7068c3e700 1 -- 172.21.15.104:0/3700043622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f706415eb80 msgr2=0x7f706415eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7068c3e700 1 --2- 172.21.15.104:0/3700043622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f706415eb80 0x7f706415eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7068c3e700 1 -- 172.21.15.104:0/3700043622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70641f9420 msgr2=0x7f70641f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7068c3e700 1 --2- 172.21.15.104:0/3700043622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70641f9420 0x7f70641f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7068c3e700 1 -- 172.21.15.104:0/3700043622 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70640cb5b0 con 0x7f7054003a20 2023-07-14T11:56:26.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.509+0000 7f7063fff700 1 --2- 172.21.15.104:0/3700043622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70641f9420 0x7f70641f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:26.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.513+0000 7f7068c3e700 1 --2- 172.21.15.104:0/3700043622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7054003a20 0x7f70640cac40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7054005830 tx=0x7f70580047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:26.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.513+0000 7f70617fa700 1 -- 172.21.15.104:0/3700043622 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705800e6c0 con 0x7f7054003a20 2023-07-14T11:56:26.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.513+0000 7f70617fa700 1 -- 172.21.15.104:0/3700043622 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7058003af0 con 0x7f7054003a20 2023-07-14T11:56:26.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.513+0000 7f70617fa700 1 -- 172.21.15.104:0/3700043622 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058003ea0 con 0x7f7054003a20 2023-07-14T11:56:26.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.513+0000 7f7069c40700 1 -- 172.21.15.104:0/3700043622 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70640cb890 con 0x7f7054003a20 2023-07-14T11:56:26.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.513+0000 7f7069c40700 1 -- 172.21.15.104:0/3700043622 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70641f9890 con 0x7f7054003a20 2023-07-14T11:56:26.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.513+0000 7f7069c40700 1 -- 172.21.15.104:0/3700043622 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706404e500 con 0x7f7054003a20 2023-07-14T11:56:26.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.517+0000 7f70617fa700 1 -- 172.21.15.104:0/3700043622 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7058013470 con 0x7f7054003a20 2023-07-14T11:56:26.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.517+0000 7f70617fa700 1 --2- 172.21.15.104:0/3700043622 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f70480662f0 0x7f70480687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.517+0000 7f70617fa700 1 -- 172.21.15.104:0/3700043622 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7058085450 con 0x7f7054003a20 2023-07-14T11:56:26.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.517+0000 7f7063fff700 1 --2- 172.21.15.104:0/3700043622 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f70480662f0 0x7f70480687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.517+0000 7f7063fff700 1 --2- 172.21.15.104:0/3700043622 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f70480662f0 0x7f70480687a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f706406dbe0 tx=0x7f7050009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:26.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.525+0000 7f70617fa700 1 -- 172.21.15.104:0/3700043622 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f70580528a0 con 0x7f7054003a20 2023-07-14T11:56:26.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.701+0000 7f7069c40700 1 -- 172.21.15.104:0/3700043622 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f70640b18d0 con 0x7f7054003a20 2023-07-14T11:56:26.688 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.701+0000 7f70617fa700 1 -- 172.21.15.104:0/3700043622 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v59) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7058055ec0 con 0x7f7054003a20 2023-07-14T11:56:26.689 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:56:26.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.701+0000 7f7069c40700 1 -- 172.21.15.104:0/3700043622 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f70480662f0 msgr2=0x7f70480687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.701+0000 7f7069c40700 1 --2- 172.21.15.104:0/3700043622 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f70480662f0 0x7f70480687a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f706406dbe0 tx=0x7f7050009040 comp rx=0 tx=0).stop 2023-07-14T11:56:26.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.701+0000 7f7069c40700 1 -- 172.21.15.104:0/3700043622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7054003a20 msgr2=0x7f70640cac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.701+0000 7f7069c40700 1 --2- 172.21.15.104:0/3700043622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7054003a20 0x7f70640cac40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7054005830 tx=0x7f70580047e0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.701+0000 7f7069c40700 1 -- 172.21.15.104:0/3700043622 shutdown_connections 2023-07-14T11:56:26.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.701+0000 7f7069c40700 1 --2- 172.21.15.104:0/3700043622 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f70480662f0 0x7f70480687a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.701+0000 7f7069c40700 1 --2- 172.21.15.104:0/3700043622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f706415eb80 0x7f706415eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.701+0000 7f7069c40700 1 --2- 172.21.15.104:0/3700043622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70641f9420 0x7f70641f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.701+0000 7f7069c40700 1 --2- 172.21.15.104:0/3700043622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7054003a20 0x7f70640cac40 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.701+0000 7f7069c40700 1 -- 172.21.15.104:0/3700043622 >> 172.21.15.104:0/3700043622 conn(0x7f7064060c40 msgr2=0x7f70640b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:26.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.701+0000 7f7069c40700 1 -- 172.21.15.104:0/3700043622 shutdown_connections 2023-07-14T11:56:26.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.701+0000 7f7069c40700 1 -- 172.21.15.104:0/3700043622 wait complete. 2023-07-14T11:56:26.713 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:56:26.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.901+0000 7f86b9599700 1 Processor -- start 2023-07-14T11:56:26.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.901+0000 7f86b9599700 1 -- start start 2023-07-14T11:56:26.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.901+0000 7f86b9599700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f86b40d0210 0x7f86b40d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.901+0000 7f86b9599700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86b40cf000 0x7f86b40cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.901+0000 7f86b9599700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b4071d10 con 0x7f86b40b1c50 2023-07-14T11:56:26.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.901+0000 7f86b9599700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b40cf9c0 con 0x7f86b40d0210 2023-07-14T11:56:26.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.901+0000 7f86b9599700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b415c420 con 0x7f86b40cf000 2023-07-14T11:56:26.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.901+0000 7f86b2ffd700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f86b40b1c50 0x7f86b4071600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:39884/0 (socket says 172.21.15.104:39884) 2023-07-14T11:56:26.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.901+0000 7f86b2ffd700 1 -- 172.21.15.104:0/2317965613 learned_addr learned my addr 172.21.15.104:0/2317965613 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:26.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.901+0000 7f86b37fe700 1 --2- 172.21.15.104:0/2317965613 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86b40cf000 0x7f86b40cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b27fc700 1 --2- 172.21.15.104:0/2317965613 >> v2:172.21.15.121:3300/0 conn(0x7f86b40d0210 0x7f86b40d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b1ffb700 1 -- 172.21.15.104:0/2317965613 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247923405 0 0) 0x7f86a4001110 con 0x7f86b40b1c50 2023-07-14T11:56:26.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b1ffb700 1 -- 172.21.15.104:0/2317965613 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f869c0035d0 con 0x7f86b40b1c50 2023-07-14T11:56:26.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b1ffb700 1 -- 172.21.15.104:0/2317965613 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1047428942 0 0) 0x7f869c0035d0 con 0x7f86b40b1c50 2023-07-14T11:56:26.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b37fe700 1 -- 172.21.15.104:0/2317965613 >> v2:172.21.15.121:3300/0 conn(0x7f86b40d0210 msgr2=0x7f86b40d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:26.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b37fe700 1 --2- 172.21.15.104:0/2317965613 >> v2:172.21.15.121:3300/0 conn(0x7f86b40d0210 0x7f86b40d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b37fe700 1 -- 172.21.15.104:0/2317965613 >> v1:172.21.15.104:6789/0 conn(0x7f86b40b1c50 legacy=0x7f86b4071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b37fe700 1 -- 172.21.15.104:0/2317965613 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b4070840 con 0x7f86b40cf000 2023-07-14T11:56:26.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b37fe700 1 --2- 172.21.15.104:0/2317965613 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86b40cf000 0x7f86b40cf480 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009d90 tx=0x7f86ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96d2f47f8c1a519b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:26.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b1ffb700 1 -- 172.21.15.104:0/2317965613 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac00e070 con 0x7f86b40cf000 2023-07-14T11:56:26.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b1ffb700 1 -- 172.21.15.104:0/2317965613 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac003a40 con 0x7f86b40cf000 2023-07-14T11:56:26.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b1ffb700 1 -- 172.21.15.104:0/2317965613 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4001800 con 0x7f86b40cf000 2023-07-14T11:56:26.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b9599700 1 -- 172.21.15.104:0/2317965613 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86b40cf000 msgr2=0x7f86b40cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b9599700 1 --2- 172.21.15.104:0/2317965613 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86b40cf000 0x7f86b40cf480 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009d90 tx=0x7f86ac00b330 comp rx=0 tx=0).stop 2023-07-14T11:56:26.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b9599700 1 -- 172.21.15.104:0/2317965613 shutdown_connections 2023-07-14T11:56:26.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b9599700 1 --2- 172.21.15.104:0/2317965613 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86b40cf000 0x7f86b40cf480 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b9599700 1 --2- 172.21.15.104:0/2317965613 >> v2:172.21.15.121:3300/0 conn(0x7f86b40d0210 0x7f86b40d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b9599700 1 -- 172.21.15.104:0/2317965613 >> 172.21.15.104:0/2317965613 conn(0x7f86b409b9c0 msgr2=0x7f86b40d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:26.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b9599700 1 -- 172.21.15.104:0/2317965613 shutdown_connections 2023-07-14T11:56:26.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.905+0000 7f86b9599700 1 -- 172.21.15.104:0/2317965613 wait complete. 2023-07-14T11:56:26.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.909+0000 7f86b9599700 1 Processor -- start 2023-07-14T11:56:26.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.909+0000 7f86b9599700 1 -- start start 2023-07-14T11:56:26.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b9599700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86b40b1c50 0x7f86b40d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b9599700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86b40cf000 0x7f86b41e5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b9599700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86b40d0210 0x7f86b41e45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b9599700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b41e5fe0 con 0x7f86b40b1c50 2023-07-14T11:56:26.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b9599700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b40b4880 con 0x7f86b40cf000 2023-07-14T11:56:26.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b9599700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b4147b30 con 0x7f86b40d0210 2023-07-14T11:56:26.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86b40b1c50 0x7f86b40d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86b40b1c50 0x7f86b40d2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46146/0 (socket says 172.21.15.104:46146) 2023-07-14T11:56:26.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b2ffd700 1 -- 172.21.15.104:0/3395293483 learned_addr learned my addr 172.21.15.104:0/3395293483 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:26.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b27fc700 1 --2- 172.21.15.104:0/3395293483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86b40cf000 0x7f86b41e5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b37fe700 1 --2- 172.21.15.104:0/3395293483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86b40d0210 0x7f86b41e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b2ffd700 1 -- 172.21.15.104:0/3395293483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86b40d0210 msgr2=0x7f86b41e45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b2ffd700 1 --2- 172.21.15.104:0/3395293483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86b40d0210 0x7f86b41e45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b2ffd700 1 -- 172.21.15.104:0/3395293483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86b40cf000 msgr2=0x7f86b41e5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:26.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b2ffd700 1 --2- 172.21.15.104:0/3395293483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86b40cf000 0x7f86b41e5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:26.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b2ffd700 1 -- 172.21.15.104:0/3395293483 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ac009a70 con 0x7f86b40b1c50 2023-07-14T11:56:26.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b37fe700 1 --2- 172.21.15.104:0/3395293483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86b40d0210 0x7f86b41e45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:26.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b2ffd700 1 --2- 172.21.15.104:0/3395293483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86b40b1c50 0x7f86b40d2bc0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f86a400c700 tx=0x7f86a400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:26.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f869b7fe700 1 -- 172.21.15.104:0/3395293483 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4004ca0 con 0x7f86b40b1c50 2023-07-14T11:56:26.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f869b7fe700 1 -- 172.21.15.104:0/3395293483 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a4004e00 con 0x7f86b40b1c50 2023-07-14T11:56:26.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b9599700 1 -- 172.21.15.104:0/3395293483 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b40b4ab0 con 0x7f86b40b1c50 2023-07-14T11:56:26.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f869b7fe700 1 -- 172.21.15.104:0/3395293483 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a40072f0 con 0x7f86b40b1c50 2023-07-14T11:56:26.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.913+0000 7f86b9599700 1 -- 172.21.15.104:0/3395293483 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b40b4d50 con 0x7f86b40b1c50 2023-07-14T11:56:26.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.917+0000 7f869b7fe700 1 -- 172.21.15.104:0/3395293483 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f86a4007450 con 0x7f86b40b1c50 2023-07-14T11:56:26.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.917+0000 7f86b9599700 1 -- 172.21.15.104:0/3395293483 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b404dcb0 con 0x7f86b40b1c50 2023-07-14T11:56:26.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.917+0000 7f869b7fe700 1 --2- 172.21.15.104:0/3395293483 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86a0066320 0x7f86a00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:26.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.917+0000 7f869b7fe700 1 -- 172.21.15.104:0/3395293483 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f86a4083280 con 0x7f86b40b1c50 2023-07-14T11:56:26.911 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.917+0000 7f86b27fc700 1 --2- 172.21.15.104:0/3395293483 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86a0066320 0x7f86a00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:26.911 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.921+0000 7f86b27fc700 1 --2- 172.21.15.104:0/3395293483 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86a0066320 0x7f86a00687d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f86b40b6100 tx=0x7f86a8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:26.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:26.925+0000 7f869b7fe700 1 -- 172.21.15.104:0/3395293483 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f86a4050520 con 0x7f86b40b1c50 2023-07-14T11:56:27.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.101+0000 7f86b9599700 1 -- 172.21.15.104:0/3395293483 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f86b4071d10 con 0x7f86b40b1c50 2023-07-14T11:56:27.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.101+0000 7f869b7fe700 1 -- 172.21.15.104:0/3395293483 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7f86b4071d10 con 0x7f86b40b1c50 2023-07-14T11:56:27.089 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:56:27.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.105+0000 7f86b9599700 1 -- 172.21.15.104:0/3395293483 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86a0066320 msgr2=0x7f86a00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:27.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.105+0000 7f86b9599700 1 --2- 172.21.15.104:0/3395293483 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86a0066320 0x7f86a00687d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f86b40b6100 tx=0x7f86a8008040 comp rx=0 tx=0).stop 2023-07-14T11:56:27.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.105+0000 7f86b9599700 1 -- 172.21.15.104:0/3395293483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86b40b1c50 msgr2=0x7f86b40d2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:27.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.105+0000 7f86b9599700 1 --2- 172.21.15.104:0/3395293483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86b40b1c50 0x7f86b40d2bc0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f86a400c700 tx=0x7f86a400cac0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.105+0000 7f86b9599700 1 -- 172.21.15.104:0/3395293483 shutdown_connections 2023-07-14T11:56:27.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.105+0000 7f86b9599700 1 --2- 172.21.15.104:0/3395293483 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f86a0066320 0x7f86a00687d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.105+0000 7f86b9599700 1 --2- 172.21.15.104:0/3395293483 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86b40d0210 0x7f86b41e45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.105+0000 7f86b9599700 1 --2- 172.21.15.104:0/3395293483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86b40cf000 0x7f86b41e5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.105+0000 7f86b9599700 1 --2- 172.21.15.104:0/3395293483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86b40b1c50 0x7f86b40d2bc0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.105+0000 7f86b9599700 1 -- 172.21.15.104:0/3395293483 >> 172.21.15.104:0/3395293483 conn(0x7f86b409b9c0 msgr2=0x7f86b415b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:27.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.105+0000 7f86b9599700 1 -- 172.21.15.104:0/3395293483 shutdown_connections 2023-07-14T11:56:27.095 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.105+0000 7f86b9599700 1 -- 172.21.15.104:0/3395293483 wait complete. 2023-07-14T11:56:27.116 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:56:27.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.305+0000 7f144d986700 1 Processor -- start 2023-07-14T11:56:27.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.305+0000 7f144d986700 1 -- start start 2023-07-14T11:56:27.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f144d986700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f144815c200 0x7f14480cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:27.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f144d986700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f14480ad0f0 0x7f14480cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:27.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f144d986700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14480cbed0 con 0x7f14480afd80 2023-07-14T11:56:27.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f144d986700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1448071820 con 0x7f14480ad0f0 2023-07-14T11:56:27.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f144d986700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14480ad760 con 0x7f144815c200 2023-07-14T11:56:27.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f14477fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f14480ad0f0 0x7f14480cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:27.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f14477fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f14480ad0f0 0x7f14480cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46152/0 (socket says 172.21.15.104:46152) 2023-07-14T11:56:27.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f14477fe700 1 -- 172.21.15.104:0/4263804872 learned_addr learned my addr 172.21.15.104:0/4263804872 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:27.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f14467fc700 1 --2- 172.21.15.104:0/4263804872 >> v2:172.21.15.121:3300/0 conn(0x7f144815c200 0x7f14480cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:27.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f14477fe700 1 -- 172.21.15.104:0/4263804872 >> v2:172.21.15.121:3300/0 conn(0x7f144815c200 msgr2=0x7f14480cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:27.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f14477fe700 1 --2- 172.21.15.104:0/4263804872 >> v2:172.21.15.121:3300/0 conn(0x7f144815c200 0x7f14480cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f14477fe700 1 -- 172.21.15.104:0/4263804872 >> v1:172.21.15.121:6789/0 conn(0x7f14480afd80 legacy=0x7f14480b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:27.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f14477fe700 1 -- 172.21.15.104:0/4263804872 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14480ada40 con 0x7f14480ad0f0 2023-07-14T11:56:27.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f14477fe700 1 --2- 172.21.15.104:0/4263804872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14480ad0f0 0x7f14480cdae0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1440009e50 tx=0x7f144000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8423a52673720d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:27.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f1445ffb700 1 -- 172.21.15.104:0/4263804872 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144000e070 con 0x7f14480ad0f0 2023-07-14T11:56:27.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f1445ffb700 1 -- 172.21.15.104:0/4263804872 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144000b8d0 con 0x7f14480ad0f0 2023-07-14T11:56:27.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f1445ffb700 1 -- 172.21.15.104:0/4263804872 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144000bc80 con 0x7f14480ad0f0 2023-07-14T11:56:27.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f144d986700 1 -- 172.21.15.104:0/4263804872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14480ad0f0 msgr2=0x7f14480cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:27.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.309+0000 7f144d986700 1 --2- 172.21.15.104:0/4263804872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14480ad0f0 0x7f14480cdae0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1440009e50 tx=0x7f144000b330 comp rx=0 tx=0).stop 2023-07-14T11:56:27.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.313+0000 7f144d986700 1 -- 172.21.15.104:0/4263804872 shutdown_connections 2023-07-14T11:56:27.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.313+0000 7f144d986700 1 --2- 172.21.15.104:0/4263804872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14480ad0f0 0x7f14480cdae0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.313+0000 7f144d986700 1 --2- 172.21.15.104:0/4263804872 >> v2:172.21.15.121:3300/0 conn(0x7f144815c200 0x7f14480cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.313+0000 7f144d986700 1 -- 172.21.15.104:0/4263804872 >> 172.21.15.104:0/4263804872 conn(0x7f14480cad80 msgr2=0x7f14480cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:27.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.313+0000 7f144d986700 1 -- 172.21.15.104:0/4263804872 shutdown_connections 2023-07-14T11:56:27.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.313+0000 7f144d986700 1 -- 172.21.15.104:0/4263804872 wait complete. 2023-07-14T11:56:27.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.313+0000 7f144d986700 1 Processor -- start 2023-07-14T11:56:27.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.313+0000 7f144d986700 1 -- start start 2023-07-14T11:56:27.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f144d986700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14480ad0f0 0x7f14480c3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:27.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f144d986700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14480afd80 0x7f14480abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:27.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f144d986700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f144815c200 0x7f14480af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:27.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f144d986700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14480c25b0 con 0x7f14480ad0f0 2023-07-14T11:56:27.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f144d986700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14480c2890 con 0x7f14480afd80 2023-07-14T11:56:27.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f144d986700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14480c2b70 con 0x7f144815c200 2023-07-14T11:56:27.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f1446ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14480ad0f0 0x7f14480c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:27.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f1446ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14480ad0f0 0x7f14480c3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46154/0 (socket says 172.21.15.104:46154) 2023-07-14T11:56:27.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f1446ffd700 1 -- 172.21.15.104:0/4126789449 learned_addr learned my addr 172.21.15.104:0/4126789449 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:27.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f14467fc700 1 --2- 172.21.15.104:0/4126789449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14480afd80 0x7f14480abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:27.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f14477fe700 1 --2- 172.21.15.104:0/4126789449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f144815c200 0x7f14480af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:27.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f1446ffd700 1 -- 172.21.15.104:0/4126789449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f144815c200 msgr2=0x7f14480af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:27.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f1446ffd700 1 --2- 172.21.15.104:0/4126789449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f144815c200 0x7f14480af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f1446ffd700 1 -- 172.21.15.104:0/4126789449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14480afd80 msgr2=0x7f14480abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:27.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f1446ffd700 1 --2- 172.21.15.104:0/4126789449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14480afd80 0x7f14480abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f1446ffd700 1 -- 172.21.15.104:0/4126789449 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1440009b30 con 0x7f14480ad0f0 2023-07-14T11:56:27.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f14477fe700 1 --2- 172.21.15.104:0/4126789449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f144815c200 0x7f14480af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:27.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f1446ffd700 1 --2- 172.21.15.104:0/4126789449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14480ad0f0 0x7f14480c3650 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f143800c500 tx=0x7f143800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:27.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f142ffff700 1 -- 172.21.15.104:0/4126789449 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14380052d0 con 0x7f14480ad0f0 2023-07-14T11:56:27.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f142ffff700 1 -- 172.21.15.104:0/4126789449 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1438005430 con 0x7f14480ad0f0 2023-07-14T11:56:27.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f142ffff700 1 -- 172.21.15.104:0/4126789449 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14380055a0 con 0x7f14480ad0f0 2023-07-14T11:56:27.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f144d986700 1 -- 172.21.15.104:0/4126789449 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14480c2e50 con 0x7f14480ad0f0 2023-07-14T11:56:27.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f144d986700 1 -- 172.21.15.104:0/4126789449 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14480af600 con 0x7f14480ad0f0 2023-07-14T11:56:27.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.317+0000 7f144d986700 1 -- 172.21.15.104:0/4126789449 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14480c3370 con 0x7f14480ad0f0 2023-07-14T11:56:27.309 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.321+0000 7f142ffff700 1 -- 172.21.15.104:0/4126789449 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f14380038e0 con 0x7f14480ad0f0 2023-07-14T11:56:27.309 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.321+0000 7f142ffff700 1 --2- 172.21.15.104:0/4126789449 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f14340662f0 0x7f14340687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:27.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.321+0000 7f142ffff700 1 -- 172.21.15.104:0/4126789449 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1438083360 con 0x7f14480ad0f0 2023-07-14T11:56:27.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.321+0000 7f14467fc700 1 --2- 172.21.15.104:0/4126789449 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f14340662f0 0x7f14340687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:27.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.321+0000 7f14467fc700 1 --2- 172.21.15.104:0/4126789449 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f14340662f0 0x7f14340687a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f14480c9bd0 tx=0x7f143c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:27.316 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.329+0000 7f142ffff700 1 -- 172.21.15.104:0/4126789449 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f14480c3370 con 0x7f14480ad0f0 2023-07-14T11:56:27.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.497+0000 7f144d986700 1 -- 172.21.15.104:0/4126789449 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f14480c3370 con 0x7f14480ad0f0 2023-07-14T11:56:27.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.497+0000 7f142ffff700 1 -- 172.21.15.104:0/4126789449 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7f14480c3370 con 0x7f14480ad0f0 2023-07-14T11:56:27.487 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:56:27.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.501+0000 7f144d986700 1 -- 172.21.15.104:0/4126789449 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f14340662f0 msgr2=0x7f14340687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:27.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.501+0000 7f144d986700 1 --2- 172.21.15.104:0/4126789449 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f14340662f0 0x7f14340687a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f14480c9bd0 tx=0x7f143c008040 comp rx=0 tx=0).stop 2023-07-14T11:56:27.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.501+0000 7f144d986700 1 -- 172.21.15.104:0/4126789449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14480ad0f0 msgr2=0x7f14480c3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:27.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.501+0000 7f144d986700 1 --2- 172.21.15.104:0/4126789449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14480ad0f0 0x7f14480c3650 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f143800c500 tx=0x7f143800c8c0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.501+0000 7f144d986700 1 -- 172.21.15.104:0/4126789449 shutdown_connections 2023-07-14T11:56:27.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.501+0000 7f144d986700 1 --2- 172.21.15.104:0/4126789449 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f14340662f0 0x7f14340687a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.501+0000 7f144d986700 1 --2- 172.21.15.104:0/4126789449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f144815c200 0x7f14480af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.501+0000 7f144d986700 1 --2- 172.21.15.104:0/4126789449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14480afd80 0x7f14480abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.501+0000 7f144d986700 1 --2- 172.21.15.104:0/4126789449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14480ad0f0 0x7f14480c3650 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.501+0000 7f144d986700 1 -- 172.21.15.104:0/4126789449 >> 172.21.15.104:0/4126789449 conn(0x7f14480cad80 msgr2=0x7f14480cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:27.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.501+0000 7f144d986700 1 -- 172.21.15.104:0/4126789449 shutdown_connections 2023-07-14T11:56:27.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.501+0000 7f144d986700 1 -- 172.21.15.104:0/4126789449 wait complete. 2023-07-14T11:56:27.512 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:56:27.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.701+0000 7f2b86cf5700 1 Processor -- start 2023-07-14T11:56:27.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.701+0000 7f2b86cf5700 1 -- start start 2023-07-14T11:56:27.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.705+0000 7f2b86cf5700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2b800b1530 0x7f2b800d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:27.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.705+0000 7f2b86cf5700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2b800d1260 0x7f2b800b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:27.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.705+0000 7f2b86cf5700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8015c2f0 con 0x7f2b8015bf20 2023-07-14T11:56:27.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.705+0000 7f2b86cf5700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b800b5b30 con 0x7f2b800d1260 2023-07-14T11:56:27.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.705+0000 7f2b86cf5700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b80070830 con 0x7f2b800b1530 2023-07-14T11:56:27.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.705+0000 7f2b85292700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2b800d1260 0x7f2b800b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:27.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.705+0000 7f2b85292700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2b800d1260 0x7f2b800b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46160/0 (socket says 172.21.15.104:46160) 2023-07-14T11:56:27.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.705+0000 7f2b85292700 1 -- 172.21.15.104:0/3163674363 learned_addr learned my addr 172.21.15.104:0/3163674363 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:27.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.705+0000 7f2b7ffff700 1 --2- 172.21.15.104:0/3163674363 >> v2:172.21.15.121:3300/0 conn(0x7f2b800b1530 0x7f2b800d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:27.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.705+0000 7f2b85292700 1 -- 172.21.15.104:0/3163674363 >> v2:172.21.15.121:3300/0 conn(0x7f2b800b1530 msgr2=0x7f2b800d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:27.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.705+0000 7f2b85292700 1 --2- 172.21.15.104:0/3163674363 >> v2:172.21.15.121:3300/0 conn(0x7f2b800b1530 0x7f2b800d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.705+0000 7f2b85292700 1 -- 172.21.15.104:0/3163674363 >> v1:172.21.15.121:6789/0 conn(0x7f2b8015bf20 legacy=0x7f2b800b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:27.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.705+0000 7f2b85292700 1 -- 172.21.15.104:0/3163674363 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b80070b10 con 0x7f2b800d1260 2023-07-14T11:56:27.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.705+0000 7f2b85292700 1 --2- 172.21.15.104:0/3163674363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b800d1260 0x7f2b800b4050 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2b800ce350 tx=0x7f2b78009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4df242cf80a1b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:27.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.709+0000 7f2b7f7fe700 1 -- 172.21.15.104:0/3163674363 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78011830 con 0x7f2b800d1260 2023-07-14T11:56:27.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.709+0000 7f2b7f7fe700 1 -- 172.21.15.104:0/3163674363 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b78011990 con 0x7f2b800d1260 2023-07-14T11:56:27.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.709+0000 7f2b7f7fe700 1 -- 172.21.15.104:0/3163674363 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78011d20 con 0x7f2b800d1260 2023-07-14T11:56:27.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.709+0000 7f2b86cf5700 1 -- 172.21.15.104:0/3163674363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b800d1260 msgr2=0x7f2b800b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:27.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.709+0000 7f2b86cf5700 1 --2- 172.21.15.104:0/3163674363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b800d1260 0x7f2b800b4050 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2b800ce350 tx=0x7f2b78009bf0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.709+0000 7f2b86cf5700 1 -- 172.21.15.104:0/3163674363 shutdown_connections 2023-07-14T11:56:27.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.709+0000 7f2b86cf5700 1 --2- 172.21.15.104:0/3163674363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b800d1260 0x7f2b800b4050 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.709+0000 7f2b86cf5700 1 --2- 172.21.15.104:0/3163674363 >> v2:172.21.15.121:3300/0 conn(0x7f2b800b1530 0x7f2b800d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.704 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.709+0000 7f2b86cf5700 1 -- 172.21.15.104:0/3163674363 >> 172.21.15.104:0/3163674363 conn(0x7f2b8006fc80 msgr2=0x7f2b80070080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:27.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.709+0000 7f2b86cf5700 1 -- 172.21.15.104:0/3163674363 shutdown_connections 2023-07-14T11:56:27.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.709+0000 7f2b86cf5700 1 -- 172.21.15.104:0/3163674363 wait complete. 2023-07-14T11:56:27.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b86cf5700 1 Processor -- start 2023-07-14T11:56:27.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b86cf5700 1 -- start start 2023-07-14T11:56:27.706 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b86cf5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b800b1530 0x7f2b801f0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:27.706 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b86cf5700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b800d1260 0x7f2b800c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:27.706 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b86cf5700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b8015bf20 0x7f2b800b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:27.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b86cf5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8015f310 con 0x7f2b800b1530 2023-07-14T11:56:27.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b86cf5700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b800c7bb0 con 0x7f2b800d1260 2023-07-14T11:56:27.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b86cf5700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8015e440 con 0x7f2b8015bf20 2023-07-14T11:56:27.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b84a91700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b800b1530 0x7f2b801f0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:27.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b84a91700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b800b1530 0x7f2b801f0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46172/0 (socket says 172.21.15.104:46172) 2023-07-14T11:56:27.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b84a91700 1 -- 172.21.15.104:0/660389114 learned_addr learned my addr 172.21.15.104:0/660389114 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:27.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b7ffff700 1 --2- 172.21.15.104:0/660389114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b800d1260 0x7f2b800c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:27.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b85292700 1 --2- 172.21.15.104:0/660389114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b8015bf20 0x7f2b800b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:27.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b84a91700 1 -- 172.21.15.104:0/660389114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b8015bf20 msgr2=0x7f2b800b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:27.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b84a91700 1 --2- 172.21.15.104:0/660389114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b8015bf20 0x7f2b800b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b84a91700 1 -- 172.21.15.104:0/660389114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b800d1260 msgr2=0x7f2b800c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:27.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b84a91700 1 --2- 172.21.15.104:0/660389114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b800d1260 0x7f2b800c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b84a91700 1 -- 172.21.15.104:0/660389114 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b78009570 con 0x7f2b800b1530 2023-07-14T11:56:27.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.713+0000 7f2b84a91700 1 --2- 172.21.15.104:0/660389114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b800b1530 0x7f2b801f0de0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2b74001ed0 tx=0x7f2b740021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:27.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.717+0000 7f2b7d7fa700 1 -- 172.21.15.104:0/660389114 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74003c20 con 0x7f2b800b1530 2023-07-14T11:56:27.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.717+0000 7f2b7d7fa700 1 -- 172.21.15.104:0/660389114 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b74003d80 con 0x7f2b800b1530 2023-07-14T11:56:27.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.717+0000 7f2b7d7fa700 1 -- 172.21.15.104:0/660389114 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7400ab40 con 0x7f2b800b1530 2023-07-14T11:56:27.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.717+0000 7f2b86cf5700 1 -- 172.21.15.104:0/660389114 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8015e160 con 0x7f2b800b1530 2023-07-14T11:56:27.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.717+0000 7f2b86cf5700 1 -- 172.21.15.104:0/660389114 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b800c7840 con 0x7f2b800b1530 2023-07-14T11:56:27.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.717+0000 7f2b86cf5700 1 -- 172.21.15.104:0/660389114 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b800c79d0 con 0x7f2b800b1530 2023-07-14T11:56:27.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.717+0000 7f2b7d7fa700 1 -- 172.21.15.104:0/660389114 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2b7400aca0 con 0x7f2b800b1530 2023-07-14T11:56:27.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.717+0000 7f2b7d7fa700 1 --2- 172.21.15.104:0/660389114 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2b640662f0 0x7f2b640687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:27.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.721+0000 7f2b7d7fa700 1 -- 172.21.15.104:0/660389114 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2b74083f80 con 0x7f2b800b1530 2023-07-14T11:56:27.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.721+0000 7f2b7ffff700 1 --2- 172.21.15.104:0/660389114 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2b640662f0 0x7f2b640687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:27.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.721+0000 7f2b7ffff700 1 --2- 172.21.15.104:0/660389114 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2b640662f0 0x7f2b640687a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2b80156510 tx=0x7f2b6c009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:27.712 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.725+0000 7f2b7d7fa700 1 -- 172.21.15.104:0/660389114 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f2b740511a0 con 0x7f2b800b1530 2023-07-14T11:56:27.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.913+0000 7f2b86cf5700 1 -- 172.21.15.104:0/660389114 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2b8004dcb0 con 0x7f2b800b1530 2023-07-14T11:56:27.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.913+0000 7f2b7d7fa700 1 -- 172.21.15.104:0/660389114 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2b740547c0 con 0x7f2b800b1530 2023-07-14T11:56:27.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.917+0000 7f2b86cf5700 1 -- 172.21.15.104:0/660389114 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2b640662f0 msgr2=0x7f2b640687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:27.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.917+0000 7f2b86cf5700 1 --2- 172.21.15.104:0/660389114 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2b640662f0 0x7f2b640687a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2b80156510 tx=0x7f2b6c009240 comp rx=0 tx=0).stop 2023-07-14T11:56:27.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.917+0000 7f2b86cf5700 1 -- 172.21.15.104:0/660389114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b800b1530 msgr2=0x7f2b801f0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:27.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.917+0000 7f2b86cf5700 1 --2- 172.21.15.104:0/660389114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b800b1530 0x7f2b801f0de0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2b74001ed0 tx=0x7f2b740021e0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.917+0000 7f2b86cf5700 1 -- 172.21.15.104:0/660389114 shutdown_connections 2023-07-14T11:56:27.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.917+0000 7f2b86cf5700 1 --2- 172.21.15.104:0/660389114 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2b640662f0 0x7f2b640687a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.917+0000 7f2b86cf5700 1 --2- 172.21.15.104:0/660389114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b8015bf20 0x7f2b800b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.917+0000 7f2b86cf5700 1 --2- 172.21.15.104:0/660389114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b800d1260 0x7f2b800c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.917+0000 7f2b86cf5700 1 --2- 172.21.15.104:0/660389114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b800b1530 0x7f2b801f0de0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:27.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.917+0000 7f2b86cf5700 1 -- 172.21.15.104:0/660389114 >> 172.21.15.104:0/660389114 conn(0x7f2b8006fc80 msgr2=0x7f2b801591b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:27.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.917+0000 7f2b86cf5700 1 -- 172.21.15.104:0/660389114 shutdown_connections 2023-07-14T11:56:27.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:27.917+0000 7f2b86cf5700 1 -- 172.21.15.104:0/660389114 wait complete. 2023-07-14T11:56:27.906 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:56:27.923 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:56:27.923 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:56:27.939 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:56:28.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.169+0000 7ff22cc34700 1 Processor -- start 2023-07-14T11:56:28.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.169+0000 7ff22cc34700 1 -- start start 2023-07-14T11:56:28.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.173+0000 7ff22cc34700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2280b1530 0x7ff2280d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.173+0000 7ff22cc34700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff2280d1260 0x7ff2280b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.173+0000 7ff22cc34700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff22815c2f0 con 0x7ff22815bf20 2023-07-14T11:56:28.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.173+0000 7ff22cc34700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2280b5b30 con 0x7ff2280d1260 2023-07-14T11:56:28.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.173+0000 7ff22cc34700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff228070830 con 0x7ff2280b1530 2023-07-14T11:56:28.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.173+0000 7ff226d9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff2280d1260 0x7ff2280b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.173+0000 7ff226d9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff2280d1260 0x7ff2280b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46186/0 (socket says 172.21.15.104:46186) 2023-07-14T11:56:28.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.173+0000 7ff226d9d700 1 -- 172.21.15.104:0/1353198572 learned_addr learned my addr 172.21.15.104:0/1353198572 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:28.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.173+0000 7ff21ffff700 1 --2- 172.21.15.104:0/1353198572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2280b1530 0x7ff2280d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.173+0000 7ff226d9d700 1 -- 172.21.15.104:0/1353198572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2280b1530 msgr2=0x7ff2280d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:28.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.173+0000 7ff226d9d700 1 --2- 172.21.15.104:0/1353198572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2280b1530 0x7ff2280d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.173+0000 7ff226d9d700 1 -- 172.21.15.104:0/1353198572 >> v1:172.21.15.121:6789/0 conn(0x7ff22815bf20 legacy=0x7ff2280b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:28.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.173+0000 7ff226d9d700 1 -- 172.21.15.104:0/1353198572 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff228070b10 con 0x7ff2280d1260 2023-07-14T11:56:28.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.177+0000 7ff226d9d700 1 --2- 172.21.15.104:0/1353198572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2280d1260 0x7ff2280b4050 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff220009e50 tx=0x7ff22000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e913696cce92c013 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:28.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.177+0000 7ff225d9b700 1 -- 172.21.15.104:0/1353198572 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22000e070 con 0x7ff2280d1260 2023-07-14T11:56:28.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.177+0000 7ff225d9b700 1 -- 172.21.15.104:0/1353198572 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22000b9c0 con 0x7ff2280d1260 2023-07-14T11:56:28.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.177+0000 7ff225d9b700 1 -- 172.21.15.104:0/1353198572 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22000bd70 con 0x7ff2280d1260 2023-07-14T11:56:28.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.177+0000 7ff22cc34700 1 -- 172.21.15.104:0/1353198572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2280d1260 msgr2=0x7ff2280b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:28.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.177+0000 7ff22cc34700 1 --2- 172.21.15.104:0/1353198572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2280d1260 0x7ff2280b4050 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff220009e50 tx=0x7ff22000b330 comp rx=0 tx=0).stop 2023-07-14T11:56:28.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.177+0000 7ff22cc34700 1 -- 172.21.15.104:0/1353198572 shutdown_connections 2023-07-14T11:56:28.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.177+0000 7ff22cc34700 1 --2- 172.21.15.104:0/1353198572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2280d1260 0x7ff2280b4050 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.177+0000 7ff22cc34700 1 --2- 172.21.15.104:0/1353198572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2280b1530 0x7ff2280d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.177+0000 7ff22cc34700 1 -- 172.21.15.104:0/1353198572 >> 172.21.15.104:0/1353198572 conn(0x7ff22806fc80 msgr2=0x7ff228070080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:28.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.177+0000 7ff22cc34700 1 -- 172.21.15.104:0/1353198572 shutdown_connections 2023-07-14T11:56:28.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.177+0000 7ff22cc34700 1 -- 172.21.15.104:0/1353198572 wait complete. 2023-07-14T11:56:28.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.177+0000 7ff22cc34700 1 Processor -- start 2023-07-14T11:56:28.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.177+0000 7ff22cc34700 1 -- start start 2023-07-14T11:56:28.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22cc34700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2280b1530 0x7ff2280b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22cc34700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2280d1260 0x7ff2280c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22cc34700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22815bf20 0x7ff2280c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22cc34700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2280c7e70 con 0x7ff2280b1530 2023-07-14T11:56:28.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22cc34700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2280c71f0 con 0x7ff22815bf20 2023-07-14T11:56:28.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22cc34700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2280c74d0 con 0x7ff2280d1260 2023-07-14T11:56:28.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2280b1530 0x7ff2280b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2280b1530 0x7ff2280b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46188/0 (socket says 172.21.15.104:46188) 2023-07-14T11:56:28.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22659c700 1 -- 172.21.15.104:0/883570031 learned_addr learned my addr 172.21.15.104:0/883570031 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:28.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff21ffff700 1 --2- 172.21.15.104:0/883570031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2280d1260 0x7ff2280c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff226d9d700 1 --2- 172.21.15.104:0/883570031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22815bf20 0x7ff2280c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22659c700 1 -- 172.21.15.104:0/883570031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2280d1260 msgr2=0x7ff2280c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:28.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22659c700 1 --2- 172.21.15.104:0/883570031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2280d1260 0x7ff2280c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22659c700 1 -- 172.21.15.104:0/883570031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22815bf20 msgr2=0x7ff2280c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:28.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22659c700 1 --2- 172.21.15.104:0/883570031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22815bf20 0x7ff2280c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22659c700 1 -- 172.21.15.104:0/883570031 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff220009b30 con 0x7ff2280b1530 2023-07-14T11:56:28.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22659c700 1 --2- 172.21.15.104:0/883570031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2280b1530 0x7ff2280b21a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff21800c500 tx=0x7ff21800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:28.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff21effd700 1 -- 172.21.15.104:0/883570031 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2180052d0 con 0x7ff2280b1530 2023-07-14T11:56:28.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff21effd700 1 -- 172.21.15.104:0/883570031 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff218005430 con 0x7ff2280b1530 2023-07-14T11:56:28.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22cc34700 1 -- 172.21.15.104:0/883570031 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2280c77b0 con 0x7ff2280b1530 2023-07-14T11:56:28.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff21effd700 1 -- 172.21.15.104:0/883570031 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2180055a0 con 0x7ff2280b1530 2023-07-14T11:56:28.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.181+0000 7ff22cc34700 1 -- 172.21.15.104:0/883570031 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2280b26e0 con 0x7ff2280b1530 2023-07-14T11:56:28.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.185+0000 7ff22cc34700 1 -- 172.21.15.104:0/883570031 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2280b28a0 con 0x7ff2280b1530 2023-07-14T11:56:28.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.185+0000 7ff21effd700 1 -- 172.21.15.104:0/883570031 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff2180038e0 con 0x7ff2280b1530 2023-07-14T11:56:28.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.189+0000 7ff21effd700 1 --2- 172.21.15.104:0/883570031 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2080662f0 0x7ff2080687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.189+0000 7ff21effd700 1 -- 172.21.15.104:0/883570031 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff218083210 con 0x7ff2280b1530 2023-07-14T11:56:28.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.189+0000 7ff21ffff700 1 --2- 172.21.15.104:0/883570031 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2080662f0 0x7ff2080687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.176 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.189+0000 7ff21ffff700 1 --2- 172.21.15.104:0/883570031 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2080662f0 0x7ff2080687a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff2280c8ce0 tx=0x7ff210008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:28.181 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.193+0000 7ff21effd700 1 -- 172.21.15.104:0/883570031 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff218050540 con 0x7ff2280b1530 2023-07-14T11:56:28.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.369+0000 7ff22cc34700 1 -- 172.21.15.104:0/883570031 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff22804dcb0 con 0x7ff2280b1530 2023-07-14T11:56:28.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.373+0000 7ff21effd700 1 -- 172.21.15.104:0/883570031 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+18534 (secure 0 0 0) 0x7ff218053b60 con 0x7ff2280b1530 2023-07-14T11:56:28.362 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:28.363 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":59,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:56:12.692633+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:56:08.755551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-14T11:56:10.132688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:56:28.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.377+0000 7ff22cc34700 1 -- 172.21.15.104:0/883570031 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2080662f0 msgr2=0x7ff2080687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:28.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.377+0000 7ff22cc34700 1 --2- 172.21.15.104:0/883570031 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2080662f0 0x7ff2080687a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff2280c8ce0 tx=0x7ff210008040 comp rx=0 tx=0).stop 2023-07-14T11:56:28.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.377+0000 7ff22cc34700 1 -- 172.21.15.104:0/883570031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2280b1530 msgr2=0x7ff2280b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:28.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.377+0000 7ff22cc34700 1 --2- 172.21.15.104:0/883570031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2280b1530 0x7ff2280b21a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff21800c500 tx=0x7ff21800c810 comp rx=0 tx=0).stop 2023-07-14T11:56:28.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.377+0000 7ff22cc34700 1 -- 172.21.15.104:0/883570031 shutdown_connections 2023-07-14T11:56:28.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.377+0000 7ff22cc34700 1 --2- 172.21.15.104:0/883570031 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2080662f0 0x7ff2080687a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.377+0000 7ff22cc34700 1 --2- 172.21.15.104:0/883570031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22815bf20 0x7ff2280c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.377+0000 7ff22cc34700 1 --2- 172.21.15.104:0/883570031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2280d1260 0x7ff2280c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.377+0000 7ff22cc34700 1 --2- 172.21.15.104:0/883570031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2280b1530 0x7ff2280b21a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.377+0000 7ff22cc34700 1 -- 172.21.15.104:0/883570031 >> 172.21.15.104:0/883570031 conn(0x7ff22806fc80 msgr2=0x7ff228070050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:28.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.377+0000 7ff22cc34700 1 -- 172.21.15.104:0/883570031 shutdown_connections 2023-07-14T11:56:28.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.377+0000 7ff22cc34700 1 -- 172.21.15.104:0/883570031 wait complete. 2023-07-14T11:56:28.385 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:56:08.755551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:56:10.132688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:56:28.386 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:56:28.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.577+0000 7f66a1aa2700 1 Processor -- start 2023-07-14T11:56:28.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.577+0000 7f66a1aa2700 1 -- start start 2023-07-14T11:56:28.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.577+0000 7f66a1aa2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f669c0afd80 0x7f669c15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.577+0000 7f66a1aa2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f669c0c8950 0x7f669c0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.577+0000 7f66a1aa2700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669c0c9290 con 0x7f669c15df10 2023-07-14T11:56:28.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.577+0000 7f66a1aa2700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f669c15e830 con 0x7f669c0afd80 2023-07-14T11:56:28.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.577+0000 7f66a1aa2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669c070a30 con 0x7f669c0c8950 2023-07-14T11:56:28.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.581+0000 7f669b7fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f669c15df10 0x7f669c0711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:39896/0 (socket says 172.21.15.104:39896) 2023-07-14T11:56:28.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.581+0000 7f669b7fe700 1 -- 172.21.15.104:0/3022615154 learned_addr learned my addr 172.21.15.104:0/3022615154 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:28.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.581+0000 7f669affd700 1 --2- 172.21.15.104:0/3022615154 >> v2:172.21.15.121:3300/0 conn(0x7f669c0afd80 0x7f669c15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.581+0000 7f669bfff700 1 --2- 172.21.15.104:0/3022615154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f669c0c8950 0x7f669c0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.581+0000 7f669a7fc700 1 -- 172.21.15.104:0/3022615154 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934128689 0 0) 0x7f668c001110 con 0x7f669c15df10 2023-07-14T11:56:28.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.581+0000 7f669a7fc700 1 -- 172.21.15.104:0/3022615154 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66840035d0 con 0x7f669c15df10 2023-07-14T11:56:28.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.581+0000 7f669affd700 1 -- 172.21.15.104:0/3022615154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f669c0c8950 msgr2=0x7f669c0c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:28.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.581+0000 7f669affd700 1 --2- 172.21.15.104:0/3022615154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f669c0c8950 0x7f669c0c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.581+0000 7f669affd700 1 -- 172.21.15.104:0/3022615154 >> v1:172.21.15.104:6789/0 conn(0x7f669c15df10 legacy=0x7f669c0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:28.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.581+0000 7f669affd700 1 -- 172.21.15.104:0/3022615154 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c0b0210 con 0x7f669c0afd80 2023-07-14T11:56:28.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.581+0000 7f669bfff700 1 --2- 172.21.15.104:0/3022615154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f669c0c8950 0x7f669c0c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:28.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.581+0000 7f669affd700 1 --2- 172.21.15.104:0/3022615154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f669c0afd80 0x7f669c15e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6690009850 tx=0x7f6690009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0d122b1013852e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:28.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.581+0000 7f669a7fc700 1 -- 172.21.15.104:0/3022615154 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690011830 con 0x7f669c0afd80 2023-07-14T11:56:28.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.581+0000 7f669a7fc700 1 -- 172.21.15.104:0/3022615154 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6690011990 con 0x7f669c0afd80 2023-07-14T11:56:28.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.581+0000 7f669a7fc700 1 -- 172.21.15.104:0/3022615154 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690011d40 con 0x7f669c0afd80 2023-07-14T11:56:28.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.585+0000 7f66a1aa2700 1 -- 172.21.15.104:0/3022615154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f669c0afd80 msgr2=0x7f669c15e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:28.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.585+0000 7f66a1aa2700 1 --2- 172.21.15.104:0/3022615154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f669c0afd80 0x7f669c15e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6690009850 tx=0x7f6690009b60 comp rx=0 tx=0).stop 2023-07-14T11:56:28.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.585+0000 7f66a1aa2700 1 -- 172.21.15.104:0/3022615154 shutdown_connections 2023-07-14T11:56:28.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.585+0000 7f66a1aa2700 1 --2- 172.21.15.104:0/3022615154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f669c0c8950 0x7f669c0c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.585+0000 7f66a1aa2700 1 --2- 172.21.15.104:0/3022615154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f669c0afd80 0x7f669c15e2f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.585+0000 7f66a1aa2700 1 -- 172.21.15.104:0/3022615154 >> 172.21.15.104:0/3022615154 conn(0x7f669c0cad80 msgr2=0x7f669c0cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:28.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.585+0000 7f66a1aa2700 1 -- 172.21.15.104:0/3022615154 shutdown_connections 2023-07-14T11:56:28.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.585+0000 7f66a1aa2700 1 -- 172.21.15.104:0/3022615154 wait complete. 2023-07-14T11:56:28.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.585+0000 7f66a1aa2700 1 Processor -- start 2023-07-14T11:56:28.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f66a1aa2700 1 -- start start 2023-07-14T11:56:28.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f66a1aa2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f669c0afd80 0x7f669c0acb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f66a1aa2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f669c0c8950 0x7f669c1f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f66a1aa2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f669c1f6080 0x7f669c06d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f66a1aa2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c0cbf70 con 0x7f669c0c8950 2023-07-14T11:56:28.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f66a1aa2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c0cc210 con 0x7f669c1f6080 2023-07-14T11:56:28.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f66a1aa2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669c0cc4f0 con 0x7f669c0afd80 2023-07-14T11:56:28.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f669affd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f669c0c8950 0x7f669c1f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f669affd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f669c0c8950 0x7f669c1f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46200/0 (socket says 172.21.15.104:46200) 2023-07-14T11:56:28.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f669affd700 1 -- 172.21.15.104:0/2668240088 learned_addr learned my addr 172.21.15.104:0/2668240088 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:28.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f669b7fe700 1 --2- 172.21.15.104:0/2668240088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f669c0afd80 0x7f669c0acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f669bfff700 1 --2- 172.21.15.104:0/2668240088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f669c1f6080 0x7f669c06d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f669affd700 1 -- 172.21.15.104:0/2668240088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f669c0afd80 msgr2=0x7f669c0acb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:28.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f669affd700 1 --2- 172.21.15.104:0/2668240088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f669c0afd80 0x7f669c0acb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f669affd700 1 -- 172.21.15.104:0/2668240088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f669c1f6080 msgr2=0x7f669c06d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:28.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f669affd700 1 --2- 172.21.15.104:0/2668240088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f669c1f6080 0x7f669c06d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.589+0000 7f669affd700 1 -- 172.21.15.104:0/2668240088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6694009a70 con 0x7f669c0c8950 2023-07-14T11:56:28.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.593+0000 7f669affd700 1 --2- 172.21.15.104:0/2668240088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f669c0c8950 0x7f669c1f5b40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6690014040 tx=0x7f6690004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:28.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.593+0000 7f6683fff700 1 -- 172.21.15.104:0/2668240088 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66900044d0 con 0x7f669c0c8950 2023-07-14T11:56:28.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.593+0000 7f6683fff700 1 -- 172.21.15.104:0/2668240088 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669001e070 con 0x7f669c0c8950 2023-07-14T11:56:28.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.593+0000 7f6683fff700 1 -- 172.21.15.104:0/2668240088 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690003900 con 0x7f669c0c8950 2023-07-14T11:56:28.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.593+0000 7f66a1aa2700 1 -- 172.21.15.104:0/2668240088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66900094b0 con 0x7f669c0c8950 2023-07-14T11:56:28.581 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.593+0000 7f66a1aa2700 1 -- 172.21.15.104:0/2668240088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669c0c73f0 con 0x7f669c0c8950 2023-07-14T11:56:28.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.593+0000 7f66a1aa2700 1 -- 172.21.15.104:0/2668240088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669c0cca10 con 0x7f669c0c8950 2023-07-14T11:56:28.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.593+0000 7f6683fff700 1 -- 172.21.15.104:0/2668240088 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6690003a60 con 0x7f669c0c8950 2023-07-14T11:56:28.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.597+0000 7f6683fff700 1 --2- 172.21.15.104:0/2668240088 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f66880662f0 0x7f66880687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.597+0000 7f6683fff700 1 -- 172.21.15.104:0/2668240088 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6690020070 con 0x7f669c0c8950 2023-07-14T11:56:28.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.597+0000 7f669b7fe700 1 --2- 172.21.15.104:0/2668240088 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f66880662f0 0x7f66880687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.597+0000 7f669b7fe700 1 --2- 172.21.15.104:0/2668240088 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f66880662f0 0x7f66880687a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f668c007fd0 tx=0x7f668c007c60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:28.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.601+0000 7f6683fff700 1 -- 172.21.15.104:0/2668240088 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f669c0cca10 con 0x7f669c0c8950 2023-07-14T11:56:28.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.773+0000 7f66a1aa2700 1 -- 172.21.15.104:0/2668240088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f669c0cca10 con 0x7f669c0c8950 2023-07-14T11:56:28.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.773+0000 7f6683fff700 1 -- 172.21.15.104:0/2668240088 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f669c0cca10 con 0x7f669c0c8950 2023-07-14T11:56:28.761 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:56:28.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.777+0000 7f66a1aa2700 1 -- 172.21.15.104:0/2668240088 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f66880662f0 msgr2=0x7f66880687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:28.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.777+0000 7f66a1aa2700 1 --2- 172.21.15.104:0/2668240088 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f66880662f0 0x7f66880687a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f668c007fd0 tx=0x7f668c007c60 comp rx=0 tx=0).stop 2023-07-14T11:56:28.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.777+0000 7f66a1aa2700 1 -- 172.21.15.104:0/2668240088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f669c0c8950 msgr2=0x7f669c1f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:28.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.777+0000 7f66a1aa2700 1 --2- 172.21.15.104:0/2668240088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f669c0c8950 0x7f669c1f5b40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6690014040 tx=0x7f6690004770 comp rx=0 tx=0).stop 2023-07-14T11:56:28.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.777+0000 7f66a1aa2700 1 -- 172.21.15.104:0/2668240088 shutdown_connections 2023-07-14T11:56:28.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.777+0000 7f66a1aa2700 1 --2- 172.21.15.104:0/2668240088 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f66880662f0 0x7f66880687a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.777+0000 7f66a1aa2700 1 --2- 172.21.15.104:0/2668240088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f669c0c8950 0x7f669c1f5b40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.777+0000 7f66a1aa2700 1 --2- 172.21.15.104:0/2668240088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f669c1f6080 0x7f669c06d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.777+0000 7f66a1aa2700 1 --2- 172.21.15.104:0/2668240088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f669c0afd80 0x7f669c0acb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.777+0000 7f66a1aa2700 1 -- 172.21.15.104:0/2668240088 >> 172.21.15.104:0/2668240088 conn(0x7f669c0cad80 msgr2=0x7f669c0cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:28.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.777+0000 7f66a1aa2700 1 -- 172.21.15.104:0/2668240088 shutdown_connections 2023-07-14T11:56:28.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.777+0000 7f66a1aa2700 1 -- 172.21.15.104:0/2668240088 wait complete. 2023-07-14T11:56:28.786 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:56:28.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096f59e700 1 Processor -- start 2023-07-14T11:56:28.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096f59e700 1 -- start start 2023-07-14T11:56:28.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096f59e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f09701597a0 0x7f09700c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096f59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0970159b70 0x7f09700705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096f59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f09700cada0 0x7f09700cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096f59e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0970071560 con 0x7f09701597a0 2023-07-14T11:56:28.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096f59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0970070cc0 con 0x7f09700cada0 2023-07-14T11:56:28.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096f59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0970070fa0 con 0x7f0970159b70 2023-07-14T11:56:28.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096e59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f09701597a0 0x7f09700c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096e59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f09701597a0 0x7f09700c87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46202/0 (socket says 172.21.15.104:46202) 2023-07-14T11:56:28.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096e59c700 1 -- 172.21.15.104:0/3169702286 learned_addr learned my addr 172.21.15.104:0/3169702286 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:28.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096dd9b700 1 --2- 172.21.15.104:0/3169702286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0970159b70 0x7f09700705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096ed9d700 1 --2- 172.21.15.104:0/3169702286 >> v2:172.21.15.121:3300/0 conn(0x7f09700cada0 0x7f09700cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096e59c700 1 -- 172.21.15.104:0/3169702286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0970159b70 msgr2=0x7f09700705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:28.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096e59c700 1 --2- 172.21.15.104:0/3169702286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0970159b70 0x7f09700705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096e59c700 1 -- 172.21.15.104:0/3169702286 >> v2:172.21.15.121:3300/0 conn(0x7f09700cada0 msgr2=0x7f09700cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:28.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096e59c700 1 --2- 172.21.15.104:0/3169702286 >> v2:172.21.15.121:3300/0 conn(0x7f09700cada0 0x7f09700cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.977+0000 7f096e59c700 1 -- 172.21.15.104:0/3169702286 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0970071280 con 0x7f09701597a0 2023-07-14T11:56:28.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.981+0000 7f096e59c700 1 --2- 172.21.15.104:0/3169702286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09701597a0 0x7f09700c87a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f09700cf220 tx=0x7f0960009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7376b1eb9d5caf41 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:28.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.981+0000 7f096d59a700 1 -- 172.21.15.104:0/3169702286 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960011830 con 0x7f09701597a0 2023-07-14T11:56:28.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.981+0000 7f096d59a700 1 -- 172.21.15.104:0/3169702286 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0960011990 con 0x7f09701597a0 2023-07-14T11:56:28.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.981+0000 7f096d59a700 1 -- 172.21.15.104:0/3169702286 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960011d20 con 0x7f09701597a0 2023-07-14T11:56:28.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.981+0000 7f096f59e700 1 -- 172.21.15.104:0/3169702286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09701597a0 msgr2=0x7f09700c87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:28.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.981+0000 7f096f59e700 1 --2- 172.21.15.104:0/3169702286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09701597a0 0x7f09700c87a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f09700cf220 tx=0x7f0960009bf0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.981+0000 7f096f59e700 1 -- 172.21.15.104:0/3169702286 shutdown_connections 2023-07-14T11:56:28.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.981+0000 7f096f59e700 1 --2- 172.21.15.104:0/3169702286 >> v2:172.21.15.121:3300/0 conn(0x7f09700cada0 0x7f09700cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.981+0000 7f096f59e700 1 --2- 172.21.15.104:0/3169702286 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0970159b70 0x7f09700705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.981+0000 7f096f59e700 1 --2- 172.21.15.104:0/3169702286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09701597a0 0x7f09700c87a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.981+0000 7f096f59e700 1 -- 172.21.15.104:0/3169702286 >> 172.21.15.104:0/3169702286 conn(0x7f09700afc60 msgr2=0x7f09700b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:28.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.981+0000 7f096f59e700 1 -- 172.21.15.104:0/3169702286 shutdown_connections 2023-07-14T11:56:28.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.981+0000 7f096f59e700 1 -- 172.21.15.104:0/3169702286 wait complete. 2023-07-14T11:56:28.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.985+0000 7f096f59e700 1 Processor -- start 2023-07-14T11:56:28.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.985+0000 7f096f59e700 1 -- start start 2023-07-14T11:56:28.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.985+0000 7f096f59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09700cada0 0x7f09701f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.985+0000 7f096f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09701597a0 0x7f0970156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.985+0000 7f096f59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0970159b70 0x7f09700b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.985+0000 7f096f59e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0970157090 con 0x7f09700cada0 2023-07-14T11:56:28.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.985+0000 7f096f59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09701f2500 con 0x7f09701597a0 2023-07-14T11:56:28.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.985+0000 7f096f59e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09701f1c60 con 0x7f0970159b70 2023-07-14T11:56:28.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.985+0000 7f096e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09700cada0 0x7f09701f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.985+0000 7f096e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09700cada0 0x7f09701f1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46204/0 (socket says 172.21.15.104:46204) 2023-07-14T11:56:28.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.985+0000 7f096e59c700 1 -- 172.21.15.104:0/1736342303 learned_addr learned my addr 172.21.15.104:0/1736342303 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:28.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.989+0000 7f096dd9b700 1 --2- 172.21.15.104:0/1736342303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09701597a0 0x7f0970156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.989+0000 7f096ed9d700 1 --2- 172.21.15.104:0/1736342303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0970159b70 0x7f09700b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.989+0000 7f096e59c700 1 -- 172.21.15.104:0/1736342303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0970159b70 msgr2=0x7f09700b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:28.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.989+0000 7f096e59c700 1 --2- 172.21.15.104:0/1736342303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0970159b70 0x7f09700b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.989+0000 7f096e59c700 1 -- 172.21.15.104:0/1736342303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09701597a0 msgr2=0x7f0970156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:28.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.989+0000 7f096e59c700 1 --2- 172.21.15.104:0/1736342303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09701597a0 0x7f0970156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:28.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.989+0000 7f096e59c700 1 -- 172.21.15.104:0/1736342303 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0960009570 con 0x7f09700cada0 2023-07-14T11:56:28.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.989+0000 7f096dd9b700 1 --2- 172.21.15.104:0/1736342303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09701597a0 0x7f0970156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:28.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.989+0000 7f096e59c700 1 --2- 172.21.15.104:0/1736342303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09700cada0 0x7f09701f1580 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f097006b5d0 tx=0x7f0960004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:28.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.989+0000 7f095effd700 1 -- 172.21.15.104:0/1736342303 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960004240 con 0x7f09700cada0 2023-07-14T11:56:28.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.989+0000 7f095effd700 1 -- 172.21.15.104:0/1736342303 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096001f070 con 0x7f09700cada0 2023-07-14T11:56:28.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.989+0000 7f096f59e700 1 -- 172.21.15.104:0/1736342303 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09701f1f40 con 0x7f09700cada0 2023-07-14T11:56:28.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.989+0000 7f095effd700 1 -- 172.21.15.104:0/1736342303 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09600045e0 con 0x7f09700cada0 2023-07-14T11:56:28.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.989+0000 7f096f59e700 1 -- 172.21.15.104:0/1736342303 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09700b1d50 con 0x7f09700cada0 2023-07-14T11:56:28.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.989+0000 7f096f59e700 1 -- 172.21.15.104:0/1736342303 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09700b1bb0 con 0x7f09700cada0 2023-07-14T11:56:28.979 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.993+0000 7f095effd700 1 -- 172.21.15.104:0/1736342303 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f0960021030 con 0x7f09700cada0 2023-07-14T11:56:28.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.993+0000 7f095effd700 1 --2- 172.21.15.104:0/1736342303 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f09500662f0 0x7f09500687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:28.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.993+0000 7f095effd700 1 -- 172.21.15.104:0/1736342303 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0960084dd0 con 0x7f09700cada0 2023-07-14T11:56:28.982 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.993+0000 7f096dd9b700 1 --2- 172.21.15.104:0/1736342303 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f09500662f0 0x7f09500687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:28.982 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.993+0000 7f096dd9b700 1 --2- 172.21.15.104:0/1736342303 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f09500662f0 0x7f09500687a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f097006d880 tx=0x7f0964008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:28.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:28.997+0000 7f095effd700 1 -- 172.21.15.104:0/1736342303 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f09700b1bb0 con 0x7f09700cada0 2023-07-14T11:56:29.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.165+0000 7f096f59e700 1 -- 172.21.15.104:0/1736342303 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f09700b1bb0 con 0x7f09700cada0 2023-07-14T11:56:29.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.165+0000 7f095effd700 1 -- 172.21.15.104:0/1736342303 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v59) v1 ==== 92+0+10 (secure 0 0 0) 0x7f09700b1bb0 con 0x7f09700cada0 2023-07-14T11:56:29.153 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:56:29.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.165+0000 7f096f59e700 1 -- 172.21.15.104:0/1736342303 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f09500662f0 msgr2=0x7f09500687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:29.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.165+0000 7f096f59e700 1 --2- 172.21.15.104:0/1736342303 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f09500662f0 0x7f09500687a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f097006d880 tx=0x7f0964008040 comp rx=0 tx=0).stop 2023-07-14T11:56:29.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.165+0000 7f096f59e700 1 -- 172.21.15.104:0/1736342303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09700cada0 msgr2=0x7f09701f1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:29.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.165+0000 7f096f59e700 1 --2- 172.21.15.104:0/1736342303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09700cada0 0x7f09701f1580 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f097006b5d0 tx=0x7f0960004770 comp rx=0 tx=0).stop 2023-07-14T11:56:29.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.169+0000 7f096f59e700 1 -- 172.21.15.104:0/1736342303 shutdown_connections 2023-07-14T11:56:29.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.169+0000 7f096f59e700 1 --2- 172.21.15.104:0/1736342303 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f09500662f0 0x7f09500687a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.169+0000 7f096f59e700 1 --2- 172.21.15.104:0/1736342303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0970159b70 0x7f09700b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.169+0000 7f096f59e700 1 --2- 172.21.15.104:0/1736342303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09701597a0 0x7f0970156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.169+0000 7f096f59e700 1 --2- 172.21.15.104:0/1736342303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09700cada0 0x7f09701f1580 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.169+0000 7f096f59e700 1 -- 172.21.15.104:0/1736342303 >> 172.21.15.104:0/1736342303 conn(0x7f09700afc60 msgr2=0x7f0970158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:29.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.169+0000 7f096f59e700 1 -- 172.21.15.104:0/1736342303 shutdown_connections 2023-07-14T11:56:29.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.169+0000 7f096f59e700 1 -- 172.21.15.104:0/1736342303 wait complete. 2023-07-14T11:56:29.176 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:56:29.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.373+0000 7fe6ff565700 1 Processor -- start 2023-07-14T11:56:29.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.373+0000 7fe6ff565700 1 -- start start 2023-07-14T11:56:29.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.373+0000 7fe6ff565700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe6f80b0490 0x7fe6f80cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:29.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.373+0000 7fe6ff565700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe6f8160610 0x7fe6f80ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:29.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.373+0000 7fe6ff565700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f8163fe0 con 0x7fe6f8160240 2023-07-14T11:56:29.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.373+0000 7fe6ff565700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f80c8c30 con 0x7fe6f8160610 2023-07-14T11:56:29.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.373+0000 7fe6ff565700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f80ca9e0 con 0x7fe6f80b0490 2023-07-14T11:56:29.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.373+0000 7fe6fdb02700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe6f8160610 0x7fe6f80ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:29.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6fcb00700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe6f80b0490 0x7fe6f80cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:29.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6fdb02700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe6f8160610 0x7fe6f80ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46208/0 (socket says 172.21.15.104:46208) 2023-07-14T11:56:29.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6fdb02700 1 -- 172.21.15.104:0/101589261 learned_addr learned my addr 172.21.15.104:0/101589261 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:29.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6fcb00700 1 -- 172.21.15.104:0/101589261 >> v2:172.21.15.104:3300/0 conn(0x7fe6f8160610 msgr2=0x7fe6f80ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:29.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6fcb00700 1 --2- 172.21.15.104:0/101589261 >> v2:172.21.15.104:3300/0 conn(0x7fe6f8160610 0x7fe6f80ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6fcb00700 1 -- 172.21.15.104:0/101589261 >> v1:172.21.15.121:6789/0 conn(0x7fe6f8160240 legacy=0x7fe6f80afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:29.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6fcb00700 1 -- 172.21.15.104:0/101589261 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f8164230 con 0x7fe6f80b0490 2023-07-14T11:56:29.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6fcb00700 1 --2- 172.21.15.104:0/101589261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6f80b0490 0x7fe6f80cdc20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80b09b0 tx=0x7fe6e8009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8620ec959ece19a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:29.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6effff700 1 -- 172.21.15.104:0/101589261 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8011830 con 0x7fe6f80b0490 2023-07-14T11:56:29.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6effff700 1 -- 172.21.15.104:0/101589261 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e8011990 con 0x7fe6f80b0490 2023-07-14T11:56:29.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6effff700 1 -- 172.21.15.104:0/101589261 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8011d20 con 0x7fe6f80b0490 2023-07-14T11:56:29.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6ff565700 1 -- 172.21.15.104:0/101589261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6f80b0490 msgr2=0x7fe6f80cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:29.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6ff565700 1 --2- 172.21.15.104:0/101589261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6f80b0490 0x7fe6f80cdc20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80b09b0 tx=0x7fe6e8009b30 comp rx=0 tx=0).stop 2023-07-14T11:56:29.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6ff565700 1 -- 172.21.15.104:0/101589261 shutdown_connections 2023-07-14T11:56:29.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6ff565700 1 --2- 172.21.15.104:0/101589261 >> v2:172.21.15.104:3300/0 conn(0x7fe6f8160610 0x7fe6f80ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6ff565700 1 --2- 172.21.15.104:0/101589261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6f80b0490 0x7fe6f80cdc20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.377+0000 7fe6ff565700 1 -- 172.21.15.104:0/101589261 >> 172.21.15.104:0/101589261 conn(0x7fe6f80cad80 msgr2=0x7fe6f80cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:29.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.381+0000 7fe6ff565700 1 -- 172.21.15.104:0/101589261 shutdown_connections 2023-07-14T11:56:29.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.381+0000 7fe6ff565700 1 -- 172.21.15.104:0/101589261 wait complete. 2023-07-14T11:56:29.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.381+0000 7fe6ff565700 1 Processor -- start 2023-07-14T11:56:29.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.381+0000 7fe6ff565700 1 -- start start 2023-07-14T11:56:29.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6ff565700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6f80b0490 0x7fe6f80c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:29.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6ff565700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6f8160240 0x7fe6f815c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:29.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6ff565700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6f8160610 0x7fe6f80b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:29.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6ff565700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80b2730 con 0x7fe6f80b0490 2023-07-14T11:56:29.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6ff565700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80c7200 con 0x7fe6f8160610 2023-07-14T11:56:29.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6ff565700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80b1b20 con 0x7fe6f8160240 2023-07-14T11:56:29.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6fd301700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6f80b0490 0x7fe6f80c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:29.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6fd301700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6f80b0490 0x7fe6f80c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46210/0 (socket says 172.21.15.104:46210) 2023-07-14T11:56:29.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6fd301700 1 -- 172.21.15.104:0/1098295540 learned_addr learned my addr 172.21.15.104:0/1098295540 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:29.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6fcb00700 1 --2- 172.21.15.104:0/1098295540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6f8160240 0x7fe6f815c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:29.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6fdb02700 1 --2- 172.21.15.104:0/1098295540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6f8160610 0x7fe6f80b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:29.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6fd301700 1 -- 172.21.15.104:0/1098295540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6f8160240 msgr2=0x7fe6f815c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:29.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6fd301700 1 --2- 172.21.15.104:0/1098295540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6f8160240 0x7fe6f815c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6fd301700 1 -- 172.21.15.104:0/1098295540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6f8160610 msgr2=0x7fe6f80b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:29.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6fd301700 1 --2- 172.21.15.104:0/1098295540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6f8160610 0x7fe6f80b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6fd301700 1 -- 172.21.15.104:0/1098295540 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e80094b0 con 0x7fe6f80b0490 2023-07-14T11:56:29.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6fcb00700 1 --2- 172.21.15.104:0/1098295540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6f8160240 0x7fe6f815c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:29.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6fd301700 1 --2- 172.21.15.104:0/1098295540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6f80b0490 0x7fe6f80c6cc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400c410 tx=0x7fe6f400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:29.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6edffb700 1 -- 172.21.15.104:0/1098295540 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40052d0 con 0x7fe6f80b0490 2023-07-14T11:56:29.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6edffb700 1 -- 172.21.15.104:0/1098295540 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f4005430 con 0x7fe6f80b0490 2023-07-14T11:56:29.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6ff565700 1 -- 172.21.15.104:0/1098295540 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f80b1e00 con 0x7fe6f80b0490 2023-07-14T11:56:29.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6edffb700 1 -- 172.21.15.104:0/1098295540 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40055a0 con 0x7fe6f80b0490 2023-07-14T11:56:29.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.385+0000 7fe6ff565700 1 -- 172.21.15.104:0/1098295540 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f80b23c0 con 0x7fe6f80b0490 2023-07-14T11:56:29.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.389+0000 7fe6ff565700 1 -- 172.21.15.104:0/1098295540 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f80b2550 con 0x7fe6f80b0490 2023-07-14T11:56:29.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.389+0000 7fe6edffb700 1 -- 172.21.15.104:0/1098295540 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe6f40036a0 con 0x7fe6f80b0490 2023-07-14T11:56:29.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.393+0000 7fe6edffb700 1 --2- 172.21.15.104:0/1098295540 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6dc0662f0 0x7fe6dc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:29.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.393+0000 7fe6edffb700 1 -- 172.21.15.104:0/1098295540 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe6f4083190 con 0x7fe6f80b0490 2023-07-14T11:56:29.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.393+0000 7fe6fcb00700 1 --2- 172.21.15.104:0/1098295540 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6dc0662f0 0x7fe6dc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:29.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.393+0000 7fe6fcb00700 1 --2- 172.21.15.104:0/1098295540 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6dc0662f0 0x7fe6dc0687a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80b09b0 tx=0x7fe6e801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:29.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.397+0000 7fe6edffb700 1 -- 172.21.15.104:0/1098295540 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fe6f4050430 con 0x7fe6f80b0490 2023-07-14T11:56:29.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.573+0000 7fe6ff565700 1 -- 172.21.15.104:0/1098295540 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe6f81647c0 con 0x7fe6f80b0490 2023-07-14T11:56:29.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.573+0000 7fe6edffb700 1 -- 172.21.15.104:0/1098295540 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe6f4053a50 con 0x7fe6f80b0490 2023-07-14T11:56:29.561 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:56:29.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.577+0000 7fe6ff565700 1 -- 172.21.15.104:0/1098295540 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6dc0662f0 msgr2=0x7fe6dc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:29.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.577+0000 7fe6ff565700 1 --2- 172.21.15.104:0/1098295540 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6dc0662f0 0x7fe6dc0687a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80b09b0 tx=0x7fe6e801c040 comp rx=0 tx=0).stop 2023-07-14T11:56:29.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.577+0000 7fe6ff565700 1 -- 172.21.15.104:0/1098295540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6f80b0490 msgr2=0x7fe6f80c6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:29.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.577+0000 7fe6ff565700 1 --2- 172.21.15.104:0/1098295540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6f80b0490 0x7fe6f80c6cc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400c410 tx=0x7fe6f400c7d0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.577+0000 7fe6ff565700 1 -- 172.21.15.104:0/1098295540 shutdown_connections 2023-07-14T11:56:29.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.577+0000 7fe6ff565700 1 --2- 172.21.15.104:0/1098295540 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fe6dc0662f0 0x7fe6dc0687a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.577+0000 7fe6ff565700 1 --2- 172.21.15.104:0/1098295540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6f8160610 0x7fe6f80b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.577+0000 7fe6ff565700 1 --2- 172.21.15.104:0/1098295540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6f8160240 0x7fe6f815c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.577+0000 7fe6ff565700 1 --2- 172.21.15.104:0/1098295540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6f80b0490 0x7fe6f80c6cc0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.577+0000 7fe6ff565700 1 -- 172.21.15.104:0/1098295540 >> 172.21.15.104:0/1098295540 conn(0x7fe6f80cad80 msgr2=0x7fe6f80afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:29.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.577+0000 7fe6ff565700 1 -- 172.21.15.104:0/1098295540 shutdown_connections 2023-07-14T11:56:29.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.577+0000 7fe6ff565700 1 -- 172.21.15.104:0/1098295540 wait complete. 2023-07-14T11:56:29.588 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:56:29.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.781+0000 7fd8784ee700 1 Processor -- start 2023-07-14T11:56:29.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.781+0000 7fd8784ee700 1 -- start start 2023-07-14T11:56:29.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd8784ee700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8700b1c50 0x7fd870071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:29.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd8784ee700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd8700cf710 0x7fd8700d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:29.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd8784ee700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd87015c420 con 0x7fd8700d0210 2023-07-14T11:56:29.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd8784ee700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd870070840 con 0x7fd8700cf710 2023-07-14T11:56:29.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd8784ee700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd870070b20 con 0x7fd8700b1c50 2023-07-14T11:56:29.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd875a89700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fd8700d0210 0x7fd8700cf000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:39900/0 (socket says 172.21.15.104:39900) 2023-07-14T11:56:29.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd875a89700 1 -- 172.21.15.104:0/866780102 learned_addr learned my addr 172.21.15.104:0/866780102 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:29.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd87628a700 1 --2- 172.21.15.104:0/866780102 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8700b1c50 0x7fd870071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:29.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd876a8b700 1 --2- 172.21.15.104:0/866780102 >> v2:172.21.15.121:3300/0 conn(0x7fd8700cf710 0x7fd8700d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:29.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd875288700 1 -- 172.21.15.104:0/866780102 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561465928 0 0) 0x7fd87015c420 con 0x7fd8700d0210 2023-07-14T11:56:29.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd875288700 1 -- 172.21.15.104:0/866780102 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd85c0035d0 con 0x7fd8700d0210 2023-07-14T11:56:29.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd876a8b700 1 -- 172.21.15.104:0/866780102 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8700b1c50 msgr2=0x7fd870071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:29.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd876a8b700 1 --2- 172.21.15.104:0/866780102 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8700b1c50 0x7fd870071600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd876a8b700 1 -- 172.21.15.104:0/866780102 >> v1:172.21.15.104:6789/0 conn(0x7fd8700d0210 legacy=0x7fd8700cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:29.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd876a8b700 1 -- 172.21.15.104:0/866780102 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd870070e00 con 0x7fd8700cf710 2023-07-14T11:56:29.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd876a8b700 1 --2- 172.21.15.104:0/866780102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8700cf710 0x7fd8700d0640 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd868009d90 tx=0x7fd86800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0c5eafb2077c431 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:29.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.785+0000 7fd875288700 1 -- 172.21.15.104:0/866780102 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86800e070 con 0x7fd8700cf710 2023-07-14T11:56:29.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.789+0000 7fd875288700 1 -- 172.21.15.104:0/866780102 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86800bb20 con 0x7fd8700cf710 2023-07-14T11:56:29.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.789+0000 7fd875288700 1 -- 172.21.15.104:0/866780102 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868012430 con 0x7fd8700cf710 2023-07-14T11:56:29.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.789+0000 7fd8784ee700 1 -- 172.21.15.104:0/866780102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8700cf710 msgr2=0x7fd8700d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:29.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.789+0000 7fd8784ee700 1 --2- 172.21.15.104:0/866780102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8700cf710 0x7fd8700d0640 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd868009d90 tx=0x7fd86800b330 comp rx=0 tx=0).stop 2023-07-14T11:56:29.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.789+0000 7fd8784ee700 1 -- 172.21.15.104:0/866780102 shutdown_connections 2023-07-14T11:56:29.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.789+0000 7fd8784ee700 1 --2- 172.21.15.104:0/866780102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8700cf710 0x7fd8700d0640 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.789+0000 7fd8784ee700 1 --2- 172.21.15.104:0/866780102 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8700b1c50 0x7fd870071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.789+0000 7fd8784ee700 1 -- 172.21.15.104:0/866780102 >> 172.21.15.104:0/866780102 conn(0x7fd87009b9c0 msgr2=0x7fd8700d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:29.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.789+0000 7fd8784ee700 1 -- 172.21.15.104:0/866780102 shutdown_connections 2023-07-14T11:56:29.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.789+0000 7fd8784ee700 1 -- 172.21.15.104:0/866780102 wait complete. 2023-07-14T11:56:29.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.789+0000 7fd8784ee700 1 Processor -- start 2023-07-14T11:56:29.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.789+0000 7fd8784ee700 1 -- start start 2023-07-14T11:56:29.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.789+0000 7fd8784ee700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8700b1c50 0x7fd8701e35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:29.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd8784ee700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8700cf710 0x7fd8701e5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:29.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd8784ee700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8700d0210 0x7fd8701e59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:29.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd8784ee700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8700b24a0 con 0x7fd8700b1c50 2023-07-14T11:56:29.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd8784ee700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8701e6230 con 0x7fd8700cf710 2023-07-14T11:56:29.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd8784ee700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8701e6510 con 0x7fd8700d0210 2023-07-14T11:56:29.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd87628a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8700b1c50 0x7fd8701e35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:29.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd87628a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8700b1c50 0x7fd8701e35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46224/0 (socket says 172.21.15.104:46224) 2023-07-14T11:56:29.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd87628a700 1 -- 172.21.15.104:0/3099905255 learned_addr learned my addr 172.21.15.104:0/3099905255 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:29.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd876a8b700 1 --2- 172.21.15.104:0/3099905255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8700d0210 0x7fd8701e59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:29.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd875a89700 1 --2- 172.21.15.104:0/3099905255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8700cf710 0x7fd8701e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:29.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd87628a700 1 -- 172.21.15.104:0/3099905255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8700d0210 msgr2=0x7fd8701e59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:29.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd87628a700 1 --2- 172.21.15.104:0/3099905255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8700d0210 0x7fd8701e59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd87628a700 1 -- 172.21.15.104:0/3099905255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8700cf710 msgr2=0x7fd8701e5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:29.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd87628a700 1 --2- 172.21.15.104:0/3099905255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8700cf710 0x7fd8701e5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd87628a700 1 -- 172.21.15.104:0/3099905255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd868009a70 con 0x7fd8700b1c50 2023-07-14T11:56:29.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd87628a700 1 --2- 172.21.15.104:0/3099905255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8700b1c50 0x7fd8701e35c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00e9d0 tx=0x7fd86c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:29.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd866ffd700 1 -- 172.21.15.104:0/3099905255 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c009720 con 0x7fd8700b1c50 2023-07-14T11:56:29.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd866ffd700 1 -- 172.21.15.104:0/3099905255 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86c00c850 con 0x7fd8700b1c50 2023-07-14T11:56:29.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd866ffd700 1 -- 172.21.15.104:0/3099905255 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c00cbc0 con 0x7fd8700b1c50 2023-07-14T11:56:29.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd8784ee700 1 -- 172.21.15.104:0/3099905255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8700b2740 con 0x7fd8700b1c50 2023-07-14T11:56:29.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.793+0000 7fd8784ee700 1 -- 172.21.15.104:0/3099905255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8701e3b00 con 0x7fd8700b1c50 2023-07-14T11:56:29.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.797+0000 7fd8784ee700 1 -- 172.21.15.104:0/3099905255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8700b28d0 con 0x7fd8700b1c50 2023-07-14T11:56:29.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.797+0000 7fd866ffd700 1 -- 172.21.15.104:0/3099905255 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd86c00cd20 con 0x7fd8700b1c50 2023-07-14T11:56:29.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.797+0000 7fd866ffd700 1 --2- 172.21.15.104:0/3099905255 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8540662f0 0x7fd8540687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:29.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.797+0000 7fd866ffd700 1 -- 172.21.15.104:0/3099905255 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd86c083fc0 con 0x7fd8700b1c50 2023-07-14T11:56:29.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.797+0000 7fd875a89700 1 --2- 172.21.15.104:0/3099905255 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8540662f0 0x7fd8540687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:29.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.801+0000 7fd875a89700 1 --2- 172.21.15.104:0/3099905255 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8540662f0 0x7fd8540687a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd87015a240 tx=0x7fd860009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:29.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.805+0000 7fd866ffd700 1 -- 172.21.15.104:0/3099905255 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fd86c0b4720 con 0x7fd8700b1c50 2023-07-14T11:56:29.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.977+0000 7fd8784ee700 1 -- 172.21.15.104:0/3099905255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd87004dcb0 con 0x7fd8700b1c50 2023-07-14T11:56:29.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.977+0000 7fd866ffd700 1 -- 172.21.15.104:0/3099905255 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd86c0210a0 con 0x7fd8700b1c50 2023-07-14T11:56:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:56:29.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.981+0000 7fd8784ee700 1 -- 172.21.15.104:0/3099905255 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8540662f0 msgr2=0x7fd8540687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:29.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.981+0000 7fd8784ee700 1 --2- 172.21.15.104:0/3099905255 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8540662f0 0x7fd8540687a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd87015a240 tx=0x7fd860009040 comp rx=0 tx=0).stop 2023-07-14T11:56:29.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.981+0000 7fd8784ee700 1 -- 172.21.15.104:0/3099905255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8700b1c50 msgr2=0x7fd8701e35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:29.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.981+0000 7fd8784ee700 1 --2- 172.21.15.104:0/3099905255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8700b1c50 0x7fd8701e35c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00e9d0 tx=0x7fd86c00ece0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.981+0000 7fd8784ee700 1 -- 172.21.15.104:0/3099905255 shutdown_connections 2023-07-14T11:56:29.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.981+0000 7fd8784ee700 1 --2- 172.21.15.104:0/3099905255 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd8540662f0 0x7fd8540687a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.981+0000 7fd8784ee700 1 --2- 172.21.15.104:0/3099905255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8700d0210 0x7fd8701e59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.981+0000 7fd8784ee700 1 --2- 172.21.15.104:0/3099905255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8700cf710 0x7fd8701e5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.981+0000 7fd8784ee700 1 --2- 172.21.15.104:0/3099905255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8700b1c50 0x7fd8701e35c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:29.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.981+0000 7fd8784ee700 1 -- 172.21.15.104:0/3099905255 >> 172.21.15.104:0/3099905255 conn(0x7fd87009b9c0 msgr2=0x7fd8700b7310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:29.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.981+0000 7fd8784ee700 1 -- 172.21.15.104:0/3099905255 shutdown_connections 2023-07-14T11:56:29.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:29.981+0000 7fd8784ee700 1 -- 172.21.15.104:0/3099905255 wait complete. 2023-07-14T11:56:29.991 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:56:30.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.169+0000 7ff141d24700 1 Processor -- start 2023-07-14T11:56:30.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.169+0000 7ff141d24700 1 -- start start 2023-07-14T11:56:30.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff141d24700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff13c0c9d90 0x7ff13c0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:30.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff141d24700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff13c15d7f0 0x7ff13c15dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:30.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff141d24700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13c0b0ef0 con 0x7ff13c159f20 2023-07-14T11:56:30.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff141d24700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13c0ce9c0 con 0x7ff13c0c9d90 2023-07-14T11:56:30.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff141d24700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13c0cee50 con 0x7ff13c15d7f0 2023-07-14T11:56:30.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff140d22700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff13c0c9d90 0x7ff13c0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:30.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff140d22700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff13c0c9d90 0x7ff13c0ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46234/0 (socket says 172.21.15.104:46234) 2023-07-14T11:56:30.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff140d22700 1 -- 172.21.15.104:0/1072445980 learned_addr learned my addr 172.21.15.104:0/1072445980 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:30.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff141523700 1 --2- 172.21.15.104:0/1072445980 >> v2:172.21.15.121:3300/0 conn(0x7ff13c15d7f0 0x7ff13c15dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:30.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff140d22700 1 -- 172.21.15.104:0/1072445980 >> v2:172.21.15.121:3300/0 conn(0x7ff13c15d7f0 msgr2=0x7ff13c15dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:30.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff140d22700 1 --2- 172.21.15.104:0/1072445980 >> v2:172.21.15.121:3300/0 conn(0x7ff13c15d7f0 0x7ff13c15dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:30.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff140d22700 1 -- 172.21.15.104:0/1072445980 >> v1:172.21.15.121:6789/0 conn(0x7ff13c159f20 legacy=0x7ff13c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:30.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff140d22700 1 -- 172.21.15.104:0/1072445980 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c0ca6a0 con 0x7ff13c0c9d90 2023-07-14T11:56:30.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff140d22700 1 --2- 172.21.15.104:0/1072445980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff13c0c9d90 0x7ff13c0ca160 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0ca950 tx=0x7ff138009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f5877fd15274e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:30.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff1337fe700 1 -- 172.21.15.104:0/1072445980 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13800b8a0 con 0x7ff13c0c9d90 2023-07-14T11:56:30.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff1337fe700 1 -- 172.21.15.104:0/1072445980 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13800ba00 con 0x7ff13c0c9d90 2023-07-14T11:56:30.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff1337fe700 1 -- 172.21.15.104:0/1072445980 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13800bd90 con 0x7ff13c0c9d90 2023-07-14T11:56:30.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff141d24700 1 -- 172.21.15.104:0/1072445980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff13c0c9d90 msgr2=0x7ff13c0ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:30.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff141d24700 1 --2- 172.21.15.104:0/1072445980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff13c0c9d90 0x7ff13c0ca160 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0ca950 tx=0x7ff138009bf0 comp rx=0 tx=0).stop 2023-07-14T11:56:30.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff141d24700 1 -- 172.21.15.104:0/1072445980 shutdown_connections 2023-07-14T11:56:30.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff141d24700 1 --2- 172.21.15.104:0/1072445980 >> v2:172.21.15.121:3300/0 conn(0x7ff13c15d7f0 0x7ff13c15dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:30.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff141d24700 1 --2- 172.21.15.104:0/1072445980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff13c0c9d90 0x7ff13c0ca160 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:30.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff141d24700 1 -- 172.21.15.104:0/1072445980 >> 172.21.15.104:0/1072445980 conn(0x7ff13c06e960 msgr2=0x7ff13c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:30.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff141d24700 1 -- 172.21.15.104:0/1072445980 shutdown_connections 2023-07-14T11:56:30.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.173+0000 7ff141d24700 1 -- 172.21.15.104:0/1072445980 wait complete. 2023-07-14T11:56:30.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.177+0000 7ff141d24700 1 Processor -- start 2023-07-14T11:56:30.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.177+0000 7ff141d24700 1 -- start start 2023-07-14T11:56:30.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.177+0000 7ff141d24700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff13c0c9d90 0x7ff13c152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:30.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.177+0000 7ff141d24700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff13c159f20 0x7ff13c0cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:30.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.177+0000 7ff141d24700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff13c15d7f0 0x7ff13c1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:30.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.177+0000 7ff141d24700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c152c60 con 0x7ff13c159f20 2023-07-14T11:56:30.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.177+0000 7ff141d24700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c0b1080 con 0x7ff13c15d7f0 2023-07-14T11:56:30.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.177+0000 7ff141d24700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c0cbc30 con 0x7ff13c0c9d90 2023-07-14T11:56:30.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.177+0000 7ff133fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff13c159f20 0x7ff13c0cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:30.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.177+0000 7ff133fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff13c159f20 0x7ff13c0cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46236/0 (socket says 172.21.15.104:46236) 2023-07-14T11:56:30.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.177+0000 7ff133fff700 1 -- 172.21.15.104:0/2121396443 learned_addr learned my addr 172.21.15.104:0/2121396443 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:30.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.181+0000 7ff141523700 1 --2- 172.21.15.104:0/2121396443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff13c15d7f0 0x7ff13c1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:30.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.181+0000 7ff140d22700 1 --2- 172.21.15.104:0/2121396443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff13c0c9d90 0x7ff13c152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:30.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.181+0000 7ff133fff700 1 -- 172.21.15.104:0/2121396443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff13c0c9d90 msgr2=0x7ff13c152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:30.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.181+0000 7ff133fff700 1 --2- 172.21.15.104:0/2121396443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff13c0c9d90 0x7ff13c152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:30.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.181+0000 7ff133fff700 1 -- 172.21.15.104:0/2121396443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff13c15d7f0 msgr2=0x7ff13c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:30.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.181+0000 7ff133fff700 1 --2- 172.21.15.104:0/2121396443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff13c15d7f0 0x7ff13c1536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:30.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.181+0000 7ff133fff700 1 -- 172.21.15.104:0/2121396443 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff138009570 con 0x7ff13c159f20 2023-07-14T11:56:30.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.181+0000 7ff141523700 1 --2- 172.21.15.104:0/2121396443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff13c15d7f0 0x7ff13c1536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:30.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.181+0000 7ff133fff700 1 --2- 172.21.15.104:0/2121396443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff13c159f20 0x7ff13c0cd3d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00c500 tx=0x7ff12c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:30.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.181+0000 7ff1317fa700 1 -- 172.21.15.104:0/2121396443 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0052d0 con 0x7ff13c159f20 2023-07-14T11:56:30.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.181+0000 7ff1317fa700 1 -- 172.21.15.104:0/2121396443 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c005430 con 0x7ff13c159f20 2023-07-14T11:56:30.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.181+0000 7ff1317fa700 1 -- 172.21.15.104:0/2121396443 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0055a0 con 0x7ff13c159f20 2023-07-14T11:56:30.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.181+0000 7ff141d24700 1 -- 172.21.15.104:0/2121396443 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13c0cbf10 con 0x7ff13c159f20 2023-07-14T11:56:30.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.181+0000 7ff141d24700 1 -- 172.21.15.104:0/2121396443 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13c0cc4d0 con 0x7ff13c159f20 2023-07-14T11:56:30.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.181+0000 7ff141d24700 1 -- 172.21.15.104:0/2121396443 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13c1e2900 con 0x7ff13c159f20 2023-07-14T11:56:30.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.185+0000 7ff1317fa700 1 -- 172.21.15.104:0/2121396443 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff12c0038e0 con 0x7ff13c159f20 2023-07-14T11:56:30.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.185+0000 7ff1317fa700 1 --2- 172.21.15.104:0/2121396443 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff1280662f0 0x7ff1280687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:30.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.185+0000 7ff1317fa700 1 -- 172.21.15.104:0/2121396443 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff12c083340 con 0x7ff13c159f20 2023-07-14T11:56:30.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.185+0000 7ff140d22700 1 --2- 172.21.15.104:0/2121396443 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff1280662f0 0x7ff1280687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:30.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.185+0000 7ff140d22700 1 --2- 172.21.15.104:0/2121396443 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff1280662f0 0x7ff1280687a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff138000c00 tx=0x7ff138008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:30.178 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.189+0000 7ff1317fa700 1 -- 172.21.15.104:0/2121396443 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff13c1e2900 con 0x7ff13c159f20 2023-07-14T11:56:30.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.373+0000 7ff141d24700 1 -- 172.21.15.104:0/2121396443 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff13c1e2900 con 0x7ff13c159f20 2023-07-14T11:56:30.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.373+0000 7ff1317fa700 1 -- 172.21.15.104:0/2121396443 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff13c1e2900 con 0x7ff13c159f20 2023-07-14T11:56:30.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.377+0000 7ff141d24700 1 -- 172.21.15.104:0/2121396443 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff1280662f0 msgr2=0x7ff1280687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:30.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.377+0000 7ff141d24700 1 --2- 172.21.15.104:0/2121396443 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff1280662f0 0x7ff1280687a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff138000c00 tx=0x7ff138008030 comp rx=0 tx=0).stop 2023-07-14T11:56:30.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.377+0000 7ff141d24700 1 -- 172.21.15.104:0/2121396443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff13c159f20 msgr2=0x7ff13c0cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:30.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.377+0000 7ff141d24700 1 --2- 172.21.15.104:0/2121396443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff13c159f20 0x7ff13c0cd3d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00c500 tx=0x7ff12c00c810 comp rx=0 tx=0).stop 2023-07-14T11:56:30.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.377+0000 7ff141d24700 1 -- 172.21.15.104:0/2121396443 shutdown_connections 2023-07-14T11:56:30.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.377+0000 7ff141d24700 1 --2- 172.21.15.104:0/2121396443 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff1280662f0 0x7ff1280687a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:30.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.377+0000 7ff141d24700 1 --2- 172.21.15.104:0/2121396443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff13c159f20 0x7ff13c0cd3d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:30.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.377+0000 7ff141d24700 1 --2- 172.21.15.104:0/2121396443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff13c15d7f0 0x7ff13c1536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:30.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.377+0000 7ff141d24700 1 --2- 172.21.15.104:0/2121396443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff13c0c9d90 0x7ff13c152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:30.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.377+0000 7ff141d24700 1 -- 172.21.15.104:0/2121396443 >> 172.21.15.104:0/2121396443 conn(0x7ff13c06e960 msgr2=0x7ff13c0afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:30.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.377+0000 7ff141d24700 1 -- 172.21.15.104:0/2121396443 shutdown_connections 2023-07-14T11:56:30.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:30.377+0000 7ff141d24700 1 -- 172.21.15.104:0/2121396443 wait complete. 2023-07-14T11:56:30.367 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:56:30.386 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi104.front.sepia.ceph.com... 2023-07-14T11:56:30.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:30.387 DEBUG:teuthology.orchestra.run.smithi104:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:30.415 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[50071]: fuse finished with error 0 and tester_r 0 2023-07-14T11:56:30.417 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:56:36.520 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:56:36.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:36.521 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-14T11:56:36.544 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-14T11:56:36.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.784+0000 7fa73e2a2700 1 Processor -- start 2023-07-14T11:56:36.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.784+0000 7fa73e2a2700 1 -- start start 2023-07-14T11:56:36.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.784+0000 7fa73e2a2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa7380afd30 0x7fa7380c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:36.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.784+0000 7fa73e2a2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7380cc1b0 0x7fa7380cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:36.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.784+0000 7fa73e2a2700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7380c97f0 con 0x7fa7380b0100 2023-07-14T11:56:36.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.784+0000 7fa73e2a2700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7380c8900 con 0x7fa7380afd30 2023-07-14T11:56:36.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.784+0000 7fa73e2a2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7380c8be0 con 0x7fa7380cc1b0 2023-07-14T11:56:36.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa72f7fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fa7380b0100 0x7fa73806d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:52964/0 (socket says 172.21.15.104:52964) 2023-07-14T11:56:36.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa72f7fe700 1 -- 172.21.15.104:0/430748698 learned_addr learned my addr 172.21.15.104:0/430748698 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:36.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa73c83f700 1 --2- 172.21.15.104:0/430748698 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7380cc1b0 0x7fa7380cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:36.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa737fff700 1 --2- 172.21.15.104:0/430748698 >> v2:172.21.15.121:3300/0 conn(0x7fa7380afd30 0x7fa7380c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:36.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa7377fe700 1 -- 172.21.15.104:0/430748698 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158375758 0 0) 0x7fa7380c97f0 con 0x7fa7380b0100 2023-07-14T11:56:36.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa7377fe700 1 -- 172.21.15.104:0/430748698 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7240035d0 con 0x7fa7380b0100 2023-07-14T11:56:36.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa73c83f700 1 -- 172.21.15.104:0/430748698 >> v2:172.21.15.121:3300/0 conn(0x7fa7380afd30 msgr2=0x7fa7380c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:36.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa73c83f700 1 --2- 172.21.15.104:0/430748698 >> v2:172.21.15.121:3300/0 conn(0x7fa7380afd30 0x7fa7380c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:36.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa73c83f700 1 -- 172.21.15.104:0/430748698 >> v1:172.21.15.104:6789/0 conn(0x7fa7380b0100 legacy=0x7fa73806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:36.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa73c83f700 1 -- 172.21.15.104:0/430748698 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7380c8ec0 con 0x7fa7380cc1b0 2023-07-14T11:56:36.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa73c83f700 1 --2- 172.21.15.104:0/430748698 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7380cc1b0 0x7fa7380cc630 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa730009d90 tx=0x7fa73000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db986542f5e2001b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:36.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa7377fe700 1 -- 172.21.15.104:0/430748698 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73000e070 con 0x7fa7380cc1b0 2023-07-14T11:56:36.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa7377fe700 1 -- 172.21.15.104:0/430748698 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73000bb20 con 0x7fa7380cc1b0 2023-07-14T11:56:36.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa7377fe700 1 -- 172.21.15.104:0/430748698 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa730012430 con 0x7fa7380cc1b0 2023-07-14T11:56:36.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa73e2a2700 1 -- 172.21.15.104:0/430748698 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7380cc1b0 msgr2=0x7fa7380cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:36.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa73e2a2700 1 --2- 172.21.15.104:0/430748698 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7380cc1b0 0x7fa7380cc630 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa730009d90 tx=0x7fa73000b330 comp rx=0 tx=0).stop 2023-07-14T11:56:36.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa73e2a2700 1 -- 172.21.15.104:0/430748698 shutdown_connections 2023-07-14T11:56:36.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa73e2a2700 1 --2- 172.21.15.104:0/430748698 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7380cc1b0 0x7fa7380cc630 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:36.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa73e2a2700 1 --2- 172.21.15.104:0/430748698 >> v2:172.21.15.121:3300/0 conn(0x7fa7380afd30 0x7fa7380c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:36.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa73e2a2700 1 -- 172.21.15.104:0/430748698 >> 172.21.15.104:0/430748698 conn(0x7fa7380cad30 msgr2=0x7fa7380cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:36.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa73e2a2700 1 -- 172.21.15.104:0/430748698 shutdown_connections 2023-07-14T11:56:36.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.788+0000 7fa73e2a2700 1 -- 172.21.15.104:0/430748698 wait complete. 2023-07-14T11:56:36.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.792+0000 7fa73e2a2700 1 Processor -- start 2023-07-14T11:56:36.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.792+0000 7fa73e2a2700 1 -- start start 2023-07-14T11:56:36.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa73e2a2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7380afd30 0x7fa738159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:36.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa73e2a2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7380b0100 0x7fa73815b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:36.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa73e2a2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7380cc1b0 0x7fa7381552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:36.782 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa73e2a2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7380c7c10 con 0x7fa7380b0100 2023-07-14T11:56:36.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa73e2a2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7380b1d60 con 0x7fa7380afd30 2023-07-14T11:56:36.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa73e2a2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7380b2040 con 0x7fa7380cc1b0 2023-07-14T11:56:36.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa72f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7380b0100 0x7fa73815b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:36.783 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa72f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7380b0100 0x7fa73815b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45378/0 (socket says 172.21.15.104:45378) 2023-07-14T11:56:36.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa72f7fe700 1 -- 172.21.15.104:0/437547937 learned_addr learned my addr 172.21.15.104:0/437547937 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:36.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa73c83f700 1 --2- 172.21.15.104:0/437547937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7380cc1b0 0x7fa7381552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:36.784 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa737fff700 1 --2- 172.21.15.104:0/437547937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7380afd30 0x7fa738159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:36.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa72f7fe700 1 -- 172.21.15.104:0/437547937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7380cc1b0 msgr2=0x7fa7381552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:36.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa72f7fe700 1 --2- 172.21.15.104:0/437547937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7380cc1b0 0x7fa7381552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:36.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa72f7fe700 1 -- 172.21.15.104:0/437547937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7380afd30 msgr2=0x7fa738159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:36.785 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa72f7fe700 1 --2- 172.21.15.104:0/437547937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7380afd30 0x7fa738159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:36.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa72f7fe700 1 -- 172.21.15.104:0/437547937 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa730009a70 con 0x7fa7380b0100 2023-07-14T11:56:36.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa72f7fe700 1 --2- 172.21.15.104:0/437547937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7380b0100 0x7fa73815b400 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa72000c700 tx=0x7fa72000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:36.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa7357fa700 1 -- 172.21.15.104:0/437547937 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720005d90 con 0x7fa7380b0100 2023-07-14T11:56:36.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa7357fa700 1 -- 172.21.15.104:0/437547937 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7200070d0 con 0x7fa7380b0100 2023-07-14T11:56:36.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa7357fa700 1 -- 172.21.15.104:0/437547937 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720007480 con 0x7fa7380b0100 2023-07-14T11:56:36.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa73e2a2700 1 -- 172.21.15.104:0/437547937 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7380b2320 con 0x7fa7380b0100 2023-07-14T11:56:36.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.796+0000 7fa73e2a2700 1 -- 172.21.15.104:0/437547937 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73815b940 con 0x7fa7380b0100 2023-07-14T11:56:36.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.800+0000 7fa73e2a2700 1 -- 172.21.15.104:0/437547937 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7380ce0b0 con 0x7fa7380b0100 2023-07-14T11:56:36.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.800+0000 7fa7357fa700 1 -- 172.21.15.104:0/437547937 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa7200075e0 con 0x7fa7380b0100 2023-07-14T11:56:36.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.800+0000 7fa7357fa700 1 --2- 172.21.15.104:0/437547937 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7180662f0 0x7fa7180687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:36.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.800+0000 7fa7357fa700 1 -- 172.21.15.104:0/437547937 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa7200832b0 con 0x7fa7380b0100 2023-07-14T11:56:36.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.804+0000 7fa737fff700 1 --2- 172.21.15.104:0/437547937 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7180662f0 0x7fa7180687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:36.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.804+0000 7fa737fff700 1 --2- 172.21.15.104:0/437547937 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7180662f0 0x7fa7180687a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa728009b50 tx=0x7fa728009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:36.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:36.808+0000 7fa7357fa700 1 -- 172.21.15.104:0/437547937 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fa7200505f0 con 0x7fa7380b0100 2023-07-14T11:56:37.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:37.024+0000 7fa73e2a2700 1 -- 172.21.15.104:0/437547937 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa7380b2560 con 0x7fa7380b0100 2023-07-14T11:56:37.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:37.024+0000 7fa7357fa700 1 -- 172.21.15.104:0/437547937 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa7380b2560 con 0x7fa7380b0100 2023-07-14T11:56:37.009 INFO:teuthology.orchestra.run.smithi104.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-14T11:56:37.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:37.024+0000 7fa73e2a2700 1 -- 172.21.15.104:0/437547937 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7180662f0 msgr2=0x7fa7180687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:37.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:37.024+0000 7fa73e2a2700 1 --2- 172.21.15.104:0/437547937 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7180662f0 0x7fa7180687a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa728009b50 tx=0x7fa728009240 comp rx=0 tx=0).stop 2023-07-14T11:56:37.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:37.024+0000 7fa73e2a2700 1 -- 172.21.15.104:0/437547937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7380b0100 msgr2=0x7fa73815b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:37.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:37.024+0000 7fa73e2a2700 1 --2- 172.21.15.104:0/437547937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7380b0100 0x7fa73815b400 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa72000c700 tx=0x7fa72000ca10 comp rx=0 tx=0).stop 2023-07-14T11:56:37.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:37.028+0000 7fa73e2a2700 1 -- 172.21.15.104:0/437547937 shutdown_connections 2023-07-14T11:56:37.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:37.028+0000 7fa73e2a2700 1 --2- 172.21.15.104:0/437547937 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa7180662f0 0x7fa7180687a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:37.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:37.028+0000 7fa73e2a2700 1 --2- 172.21.15.104:0/437547937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7380b0100 0x7fa73815b400 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:37.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:37.028+0000 7fa73e2a2700 1 --2- 172.21.15.104:0/437547937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7380cc1b0 0x7fa7381552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:37.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:37.028+0000 7fa73e2a2700 1 --2- 172.21.15.104:0/437547937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7380afd30 0x7fa738159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:37.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:37.028+0000 7fa73e2a2700 1 -- 172.21.15.104:0/437547937 >> 172.21.15.104:0/437547937 conn(0x7fa7380cad30 msgr2=0x7fa73806c680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:37.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:37.028+0000 7fa73e2a2700 1 -- 172.21.15.104:0/437547937 shutdown_connections 2023-07-14T11:56:37.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:37.028+0000 7fa73e2a2700 1 -- 172.21.15.104:0/437547937 wait complete. 2023-07-14T11:56:37.033 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-14T11:56:37.034 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T11:56:37.034 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2023-07-14T11:56:37.034 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-14T11:56:37.034 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:37.035 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-14T11:56:37.035 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T11:56:37.035 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T11:56:37.035 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:56:37.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:37.036 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2023-07-14T11:56:37.057 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:56:37.057 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:56:37.057 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:56:37.057 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:37.058 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2023-07-14T11:56:37.058 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:37.058 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:56:37.058 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:37.058 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2023-07-14T11:56:37.058 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:56:37.059 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:37.059 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2023-07-14T11:56:37.059 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:56:37.059 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:37.059 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T11:56:37.060 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:37.060 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-07-14T11:56:37.060 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42199sec preferred_lft 42199sec 2023-07-14T11:56:37.060 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2023-07-14T11:56:37.060 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:37.061 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-07-14T11:56:37.061 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:a2:a1:53:16 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:37.061 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-07-14T11:56:37.061 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:37.062 INFO:teuthology.orchestra.run.smithi104.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-14T11:56:37.062 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:b8:27:80:0d:ab brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:37.062 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-14T11:56:37.062 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:37.063 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5020:39ff:fe23:761d/64 scope link 2023-07-14T11:56:37.063 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:37.063 INFO:teuthology.orchestra.run.smithi104.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-14T11:56:37.063 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:b8:27:80:0d:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-14T11:56:37.063 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::e8b8:27ff:fe80:dab/64 scope link 2023-07-14T11:56:37.063 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:37.063 INFO:teuthology.orchestra.run.smithi104.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-14T11:56:37.064 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:e6:23:3b:72:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-14T11:56:37.064 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::e8e6:23ff:fe3b:7247/64 scope link 2023-07-14T11:56:37.064 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:37.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:37.065 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T11:56:37.137 INFO:teuthology.orchestra.run.smithi104.stdout:1 2023-07-14T11:56:37.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:37.140 DEBUG:teuthology.orchestra.run.smithi104:> route 2023-07-14T11:56:37.205 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2023-07-14T11:56:37.206 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T11:56:37.206 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-07-14T11:56:37.206 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T11:56:37.207 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-07-14T11:56:37.207 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-07-14T11:56:37.207 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-07-14T11:56:37.207 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T11:56:37.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:37.208 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:56:37.209 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:56:37.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T11:56:37.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T11:56:37.211 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T11:56:37.212 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:56:37.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:37.343 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2023-07-14T11:56:37.361 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-14T11:56:37.361 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-14T11:56:37.362 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:56:37.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:37.362 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2023-07-14T11:56:37.438 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-14T11:56:37.439 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:56:37.439 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:37.491 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:56:37.492 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:56:37.493 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:37.552 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:56:37.632 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:56:37.677 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:56:37.713 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:56:37.716 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:56:37.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:37.717 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:56:37.776 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-14T11:56:37.776 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-14T11:56:37.826 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:56:37.857 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:56:37.857 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:56:37.936 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:56:37.937 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:56:37.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:37.938 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:56:37.966 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:2023-07-14T11:56:37.980+0000 7f33e980a640 -1 init, newargv = 0x5581949aadb0 newargc=15 2023-07-14T11:56:37.967 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[50536]: starting ceph client 2023-07-14T11:56:37.979 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[50536]: starting fuse 2023-07-14T11:56:37.984 INFO:teuthology.orchestra.run.smithi104.stdout:49 2023-07-14T11:56:37.984 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-07-14T11:56:37.985 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T11:56:37.986 DEBUG:teuthology.orchestra.run.smithi104:> import glob 2023-07-14T11:56:37.987 DEBUG:teuthology.orchestra.run.smithi104:> import re 2023-07-14T11:56:37.987 DEBUG:teuthology.orchestra.run.smithi104:> import os 2023-07-14T11:56:37.987 DEBUG:teuthology.orchestra.run.smithi104:> import subprocess 2023-07-14T11:56:37.988 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:56:37.988 DEBUG:teuthology.orchestra.run.smithi104:> def _find_admin_socket(client_name): 2023-07-14T11:56:37.988 DEBUG:teuthology.orchestra.run.smithi104:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-14T11:56:37.989 DEBUG:teuthology.orchestra.run.smithi104:> files = glob.glob(asok_path) 2023-07-14T11:56:37.989 DEBUG:teuthology.orchestra.run.smithi104:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-14T11:56:37.989 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:56:37.989 DEBUG:teuthology.orchestra.run.smithi104:> # Given a non-glob path, it better be there 2023-07-14T11:56:37.990 DEBUG:teuthology.orchestra.run.smithi104:> if "*" not in asok_path: 2023-07-14T11:56:37.990 DEBUG:teuthology.orchestra.run.smithi104:> assert(len(files) == 1) 2023-07-14T11:56:37.990 DEBUG:teuthology.orchestra.run.smithi104:> return files[0] 2023-07-14T11:56:37.991 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:56:37.991 DEBUG:teuthology.orchestra.run.smithi104:> for f in files: 2023-07-14T11:56:37.991 DEBUG:teuthology.orchestra.run.smithi104:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-14T11:56:37.991 DEBUG:teuthology.orchestra.run.smithi104:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-14T11:56:37.992 DEBUG:teuthology.orchestra.run.smithi104:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-14T11:56:37.992 DEBUG:teuthology.orchestra.run.smithi104:> contents = proc_f.read() 2023-07-14T11:56:37.993 DEBUG:teuthology.orchestra.run.smithi104:> if mountpoint in contents: 2023-07-14T11:56:37.993 DEBUG:teuthology.orchestra.run.smithi104:> return f 2023-07-14T11:56:37.993 DEBUG:teuthology.orchestra.run.smithi104:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-14T11:56:37.993 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:56:37.994 DEBUG:teuthology.orchestra.run.smithi104:> print(_find_admin_socket("client.0")) 2023-07-14T11:56:37.994 DEBUG:teuthology.orchestra.run.smithi104:> ' 2023-07-14T11:56:38.135 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:56:38.143 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:56:38.143 INFO:teuthology.orchestra.run.smithi104.stdout:/var/run/ceph/ceph-client.0.50536.asok 2023-07-14T11:56:38.330 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50536.asok 2023-07-14T11:56:38.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:38.331 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50536.asok status 2023-07-14T11:56:38.481 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:56:38.481 INFO:teuthology.orchestra.run.smithi104.stdout: "metadata": { 2023-07-14T11:56:38.481 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-14T11:56:38.482 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:56:38.482 INFO:teuthology.orchestra.run.smithi104.stdout: "entity_id": "0", 2023-07-14T11:56:38.482 INFO:teuthology.orchestra.run.smithi104.stdout: "hostname": "smithi104", 2023-07-14T11:56:38.482 INFO:teuthology.orchestra.run.smithi104.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-14T11:56:38.482 INFO:teuthology.orchestra.run.smithi104.stdout: "pid": "50536", 2023-07-14T11:56:38.482 INFO:teuthology.orchestra.run.smithi104.stdout: "root": "/" 2023-07-14T11:56:38.482 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:38.483 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_count": 0, 2023-07-14T11:56:38.483 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_pinned_count": 0, 2023-07-14T11:56:38.483 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 5843, 2023-07-14T11:56:38.483 INFO:teuthology.orchestra.run.smithi104.stdout: "inst": { 2023-07-14T11:56:38.483 INFO:teuthology.orchestra.run.smithi104.stdout: "name": { 2023-07-14T11:56:38.484 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "client", 2023-07-14T11:56:38.484 INFO:teuthology.orchestra.run.smithi104.stdout: "num": 5843 2023-07-14T11:56:38.484 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:38.484 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:56:38.484 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:56:38.484 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:0", 2023-07-14T11:56:38.485 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 3319479192 2023-07-14T11:56:38.485 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:38.485 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:38.485 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:56:38.486 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:56:38.486 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:0", 2023-07-14T11:56:38.486 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 3319479192 2023-07-14T11:56:38.486 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:38.486 INFO:teuthology.orchestra.run.smithi104.stdout: "inst_str": "client.5843 172.21.15.104:0/3319479192", 2023-07-14T11:56:38.486 INFO:teuthology.orchestra.run.smithi104.stdout: "addr_str": "172.21.15.104:0/3319479192", 2023-07-14T11:56:38.487 INFO:teuthology.orchestra.run.smithi104.stdout: "inode_count": 1, 2023-07-14T11:56:38.487 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_epoch": 27, 2023-07-14T11:56:38.487 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch": 59, 2023-07-14T11:56:38.487 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch_barrier": 0, 2023-07-14T11:56:38.488 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklisted": false, 2023-07-14T11:56:38.488 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs" 2023-07-14T11:56:38.488 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:56:38.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:38.496 DEBUG:teuthology.orchestra.run.smithi104:> stat --file-system '--printf=%T 2023-07-14T11:56:38.496 DEBUG:teuthology.orchestra.run.smithi104:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:38.514 INFO:teuthology.orchestra.run.smithi104.stdout:fuseblk 2023-07-14T11:56:38.515 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:38.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:38.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:38.592 DEBUG:teuthology.orchestra.run.smithi104:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T11:56:38.593 DEBUG:teuthology.orchestra.run.smithi104:> import os 2023-07-14T11:56:38.594 DEBUG:teuthology.orchestra.run.smithi104:> import errno 2023-07-14T11:56:38.595 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:56:38.595 DEBUG:teuthology.orchestra.run.smithi104:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2023-07-14T11:56:38.596 DEBUG:teuthology.orchestra.run.smithi104:> try: 2023-07-14T11:56:38.597 DEBUG:teuthology.orchestra.run.smithi104:> if False: 2023-07-14T11:56:38.597 DEBUG:teuthology.orchestra.run.smithi104:> ret = os.read(fd, 1024) 2023-07-14T11:56:38.598 DEBUG:teuthology.orchestra.run.smithi104:> else: 2023-07-14T11:56:38.598 DEBUG:teuthology.orchestra.run.smithi104:> os.write(fd, b'"'"'content'"'"') 2023-07-14T11:56:38.599 DEBUG:teuthology.orchestra.run.smithi104:> except OSError as e: 2023-07-14T11:56:38.599 DEBUG:teuthology.orchestra.run.smithi104:> if e.errno != errno.EPERM: 2023-07-14T11:56:38.600 DEBUG:teuthology.orchestra.run.smithi104:> raise 2023-07-14T11:56:38.601 DEBUG:teuthology.orchestra.run.smithi104:> else: 2023-07-14T11:56:38.601 DEBUG:teuthology.orchestra.run.smithi104:> raise RuntimeError("client does not check permission of data pool") 2023-07-14T11:56:38.601 DEBUG:teuthology.orchestra.run.smithi104:> ' 2023-07-14T11:56:38.923 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow w pool=cephfs_data' 2023-07-14T11:56:39.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.172+0000 7fbb65eaa700 1 Processor -- start 2023-07-14T11:56:39.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.172+0000 7fbb65eaa700 1 -- start start 2023-07-14T11:56:39.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.172+0000 7fbb65eaa700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbb6015df10 0x7fbb6015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:39.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.172+0000 7fbb65eaa700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb60070a30 con 0x7fbb600afd80 2023-07-14T11:56:39.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.172+0000 7fbb65eaa700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb600b0210 con 0x7fbb600c8950 2023-07-14T11:56:39.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.172+0000 7fbb65eaa700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6015e830 con 0x7fbb6015df10 2023-07-14T11:56:39.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.172+0000 7fbb5effd700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fbb600afd80 0x7fbb600711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:52968/0 (socket says 172.21.15.104:52968) 2023-07-14T11:56:39.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.172+0000 7fbb5effd700 1 -- 172.21.15.104:0/2379483954 learned_addr learned my addr 172.21.15.104:0/2379483954 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:39.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.172+0000 7fbb5f7fe700 1 --2- 172.21.15.104:0/2379483954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbb6015df10 0x7fbb6015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:39.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.172+0000 7fbb5e7fc700 1 -- 172.21.15.104:0/2379483954 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495699992 0 0) 0x7fbb60070a30 con 0x7fbb600afd80 2023-07-14T11:56:39.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb5e7fc700 1 -- 172.21.15.104:0/2379483954 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb480035d0 con 0x7fbb600afd80 2023-07-14T11:56:39.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb5e7fc700 1 -- 172.21.15.104:0/2379483954 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253161261 0 0) 0x7fbb600b0210 con 0x7fbb600c8950 2023-07-14T11:56:39.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb5f7fe700 1 -- 172.21.15.104:0/2379483954 >> v1:172.21.15.121:6789/0 conn(0x7fbb600c8950 legacy=0x7fbb6006d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:39.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb5f7fe700 1 -- 172.21.15.104:0/2379483954 >> v1:172.21.15.104:6789/0 conn(0x7fbb600afd80 legacy=0x7fbb600711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:39.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb5f7fe700 1 -- 172.21.15.104:0/2379483954 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb600b04f0 con 0x7fbb6015df10 2023-07-14T11:56:39.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb5f7fe700 1 --2- 172.21.15.104:0/2379483954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbb6015df10 0x7fbb6015e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009880 tx=0x7fbb50009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5ee23e0d334f7c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:39.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb5e7fc700 1 -- 172.21.15.104:0/2379483954 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50011830 con 0x7fbb6015df10 2023-07-14T11:56:39.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb5e7fc700 1 -- 172.21.15.104:0/2379483954 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb50011990 con 0x7fbb6015df10 2023-07-14T11:56:39.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb5e7fc700 1 -- 172.21.15.104:0/2379483954 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50011d20 con 0x7fbb6015df10 2023-07-14T11:56:39.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb65eaa700 1 -- 172.21.15.104:0/2379483954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbb6015df10 msgr2=0x7fbb6015e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:39.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb65eaa700 1 --2- 172.21.15.104:0/2379483954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbb6015df10 0x7fbb6015e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009880 tx=0x7fbb50009b90 comp rx=0 tx=0).stop 2023-07-14T11:56:39.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb65eaa700 1 -- 172.21.15.104:0/2379483954 shutdown_connections 2023-07-14T11:56:39.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb65eaa700 1 --2- 172.21.15.104:0/2379483954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbb6015df10 0x7fbb6015e2f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb65eaa700 1 -- 172.21.15.104:0/2379483954 >> 172.21.15.104:0/2379483954 conn(0x7fbb600cad80 msgr2=0x7fbb600cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:39.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb65eaa700 1 -- 172.21.15.104:0/2379483954 shutdown_connections 2023-07-14T11:56:39.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.176+0000 7fbb65eaa700 1 -- 172.21.15.104:0/2379483954 wait complete. 2023-07-14T11:56:39.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.180+0000 7fbb65eaa700 1 Processor -- start 2023-07-14T11:56:39.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.180+0000 7fbb65eaa700 1 -- start start 2023-07-14T11:56:39.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.180+0000 7fbb65eaa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb600afd80 0x7fbb600ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:39.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.180+0000 7fbb65eaa700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb600c8950 0x7fbb6014c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:39.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.180+0000 7fbb65eaa700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbb600ad020 0x7fbb600cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:39.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.180+0000 7fbb65eaa700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb600acb90 con 0x7fbb600c8950 2023-07-14T11:56:39.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.180+0000 7fbb65eaa700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb60157f50 con 0x7fbb600afd80 2023-07-14T11:56:39.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.180+0000 7fbb65eaa700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb600abf80 con 0x7fbb600ad020 2023-07-14T11:56:39.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.180+0000 7fbb5effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb600c8950 0x7fbb6014c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:39.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.180+0000 7fbb5effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb600c8950 0x7fbb6014c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45388/0 (socket says 172.21.15.104:45388) 2023-07-14T11:56:39.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.180+0000 7fbb5f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb600afd80 0x7fbb600ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:39.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.180+0000 7fbb5effd700 1 -- 172.21.15.104:0/1353766120 learned_addr learned my addr 172.21.15.104:0/1353766120 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:39.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.184+0000 7fbb5ffff700 1 --2- 172.21.15.104:0/1353766120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbb600ad020 0x7fbb600cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:39.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.184+0000 7fbb5effd700 1 -- 172.21.15.104:0/1353766120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbb600ad020 msgr2=0x7fbb600cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:39.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.184+0000 7fbb5effd700 1 --2- 172.21.15.104:0/1353766120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbb600ad020 0x7fbb600cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.184+0000 7fbb5effd700 1 -- 172.21.15.104:0/1353766120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb600afd80 msgr2=0x7fbb600ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:39.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.184+0000 7fbb5effd700 1 --2- 172.21.15.104:0/1353766120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb600afd80 0x7fbb600ab8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.184+0000 7fbb5effd700 1 -- 172.21.15.104:0/1353766120 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb500094b0 con 0x7fbb600c8950 2023-07-14T11:56:39.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.184+0000 7fbb5effd700 1 --2- 172.21.15.104:0/1353766120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb600c8950 0x7fbb6014c710 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400c4f0 tx=0x7fbb5400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:39.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.184+0000 7fbb47fff700 1 -- 172.21.15.104:0/1353766120 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54005d90 con 0x7fbb600c8950 2023-07-14T11:56:39.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.184+0000 7fbb65eaa700 1 -- 172.21.15.104:0/1353766120 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb600ac260 con 0x7fbb600c8950 2023-07-14T11:56:39.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.184+0000 7fbb47fff700 1 -- 172.21.15.104:0/1353766120 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb540070d0 con 0x7fbb600c8950 2023-07-14T11:56:39.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.184+0000 7fbb47fff700 1 -- 172.21.15.104:0/1353766120 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54007440 con 0x7fbb600c8950 2023-07-14T11:56:39.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.184+0000 7fbb65eaa700 1 -- 172.21.15.104:0/1353766120 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb600ac820 con 0x7fbb600c8950 2023-07-14T11:56:39.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.184+0000 7fbb65eaa700 1 -- 172.21.15.104:0/1353766120 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb600acd40 con 0x7fbb600c8950 2023-07-14T11:56:39.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.188+0000 7fbb47fff700 1 -- 172.21.15.104:0/1353766120 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fbb540075a0 con 0x7fbb600c8950 2023-07-14T11:56:39.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.188+0000 7fbb47fff700 1 --2- 172.21.15.104:0/1353766120 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbb4c0662f0 0x7fbb4c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:39.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.188+0000 7fbb47fff700 1 -- 172.21.15.104:0/1353766120 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbb54083210 con 0x7fbb600c8950 2023-07-14T11:56:39.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.188+0000 7fbb5f7fe700 1 --2- 172.21.15.104:0/1353766120 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbb4c0662f0 0x7fbb4c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:39.174 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.188+0000 7fbb5f7fe700 1 --2- 172.21.15.104:0/1353766120 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbb4c0662f0 0x7fbb4c0687a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbb500090b0 tx=0x7fbb5001c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:39.179 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.196+0000 7fbb47fff700 1 -- 172.21.15.104:0/1353766120 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fbb600acd40 con 0x7fbb600c8950 2023-07-14T11:56:39.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.424+0000 7fbb65eaa700 1 -- 172.21.15.104:0/1353766120 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]} v 0) v1 -- 0x7fbb6004dcb0 con 0x7fbb600c8950 2023-07-14T11:56:39.416 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.432+0000 7fbb47fff700 1 -- 172.21.15.104:0/1353766120 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]}]=0 updated caps for client.0 v19) v1 ==== 183+0+140 (secure 0 0 0) 0x7fbb540504e0 con 0x7fbb600c8950 2023-07-14T11:56:39.417 INFO:teuthology.orchestra.run.smithi104.stdout:[client.0] 2023-07-14T11:56:39.417 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDrNrFkbrzjERAA+9BBzuauwUdhALUcDtd80A== 2023-07-14T11:56:39.417 INFO:teuthology.orchestra.run.smithi104.stdout: caps mds = "allow" 2023-07-14T11:56:39.417 INFO:teuthology.orchestra.run.smithi104.stdout: caps mon = "allow r" 2023-07-14T11:56:39.417 INFO:teuthology.orchestra.run.smithi104.stdout: caps osd = "allow w pool=cephfs_data" 2023-07-14T11:56:39.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.436+0000 7fbb65eaa700 1 -- 172.21.15.104:0/1353766120 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbb4c0662f0 msgr2=0x7fbb4c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:39.419 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.436+0000 7fbb65eaa700 1 --2- 172.21.15.104:0/1353766120 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbb4c0662f0 0x7fbb4c0687a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbb500090b0 tx=0x7fbb5001c040 comp rx=0 tx=0).stop 2023-07-14T11:56:39.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.436+0000 7fbb65eaa700 1 -- 172.21.15.104:0/1353766120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb600c8950 msgr2=0x7fbb6014c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:39.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.436+0000 7fbb65eaa700 1 --2- 172.21.15.104:0/1353766120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb600c8950 0x7fbb6014c710 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400c4f0 tx=0x7fbb5400c800 comp rx=0 tx=0).stop 2023-07-14T11:56:39.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.436+0000 7fbb65eaa700 1 -- 172.21.15.104:0/1353766120 shutdown_connections 2023-07-14T11:56:39.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.436+0000 7fbb65eaa700 1 --2- 172.21.15.104:0/1353766120 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbb4c0662f0 0x7fbb4c0687a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.420 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.436+0000 7fbb65eaa700 1 --2- 172.21.15.104:0/1353766120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb600c8950 0x7fbb6014c710 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.436+0000 7fbb65eaa700 1 --2- 172.21.15.104:0/1353766120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbb600ad020 0x7fbb600cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.436+0000 7fbb65eaa700 1 --2- 172.21.15.104:0/1353766120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb600afd80 0x7fbb600ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.436+0000 7fbb65eaa700 1 -- 172.21.15.104:0/1353766120 >> 172.21.15.104:0/1353766120 conn(0x7fbb600cad80 msgr2=0x7fbb600c8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:39.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.436+0000 7fbb65eaa700 1 -- 172.21.15.104:0/1353766120 shutdown_connections 2023-07-14T11:56:39.421 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.436+0000 7fbb65eaa700 1 -- 172.21.15.104:0/1353766120 wait complete. 2023-07-14T11:56:39.422 INFO:teuthology.orchestra.run.smithi104.stderr:updated caps for client.0 2023-07-14T11:56:39.441 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:56:39.442 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:56:39.460 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:56:39.676 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.692+0000 7f01458dd700 1 Processor -- start 2023-07-14T11:56:39.676 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.692+0000 7f01458dd700 1 -- start start 2023-07-14T11:56:39.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.692+0000 7f01458dd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014015bf20 0x7f01400b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:39.679 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.692+0000 7f01458dd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f01400d1430 0x7f01400b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:39.679 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.692+0000 7f01458dd700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014015c2f0 con 0x7f01400b1530 2023-07-14T11:56:39.679 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.692+0000 7f01458dd700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0140070830 con 0x7f01400d1430 2023-07-14T11:56:39.679 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.692+0000 7f01458dd700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0140070b10 con 0x7f014015bf20 2023-07-14T11:56:39.680 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f013f7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f01400d1430 0x7f01400b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:39.680 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f013f7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f01400d1430 0x7f01400b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45400/0 (socket says 172.21.15.104:45400) 2023-07-14T11:56:39.680 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f013f7fe700 1 -- 172.21.15.104:0/2878982743 learned_addr learned my addr 172.21.15.104:0/2878982743 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:39.680 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f013effd700 1 --2- 172.21.15.104:0/2878982743 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014015bf20 0x7f01400b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:39.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f013f7fe700 1 -- 172.21.15.104:0/2878982743 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014015bf20 msgr2=0x7f01400b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:39.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f013f7fe700 1 --2- 172.21.15.104:0/2878982743 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014015bf20 0x7f01400b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f013f7fe700 1 -- 172.21.15.104:0/2878982743 >> v1:172.21.15.121:6789/0 conn(0x7f01400b1530 legacy=0x7f01400d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:39.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f013f7fe700 1 -- 172.21.15.104:0/2878982743 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01400cccc0 con 0x7f01400d1430 2023-07-14T11:56:39.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f013f7fe700 1 --2- 172.21.15.104:0/2878982743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01400d1430 0x7f01400b4050 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0138009e50 tx=0x7f013800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11b7b36a20133b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:39.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f013dffb700 1 -- 172.21.15.104:0/2878982743 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013800e070 con 0x7f01400d1430 2023-07-14T11:56:39.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f013dffb700 1 -- 172.21.15.104:0/2878982743 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0138003a40 con 0x7f01400d1430 2023-07-14T11:56:39.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f013dffb700 1 -- 172.21.15.104:0/2878982743 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0138003df0 con 0x7f01400d1430 2023-07-14T11:56:39.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f01458dd700 1 -- 172.21.15.104:0/2878982743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01400d1430 msgr2=0x7f01400b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:39.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f01458dd700 1 --2- 172.21.15.104:0/2878982743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01400d1430 0x7f01400b4050 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0138009e50 tx=0x7f013800b330 comp rx=0 tx=0).stop 2023-07-14T11:56:39.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f01458dd700 1 -- 172.21.15.104:0/2878982743 shutdown_connections 2023-07-14T11:56:39.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f01458dd700 1 --2- 172.21.15.104:0/2878982743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01400d1430 0x7f01400b4050 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f01458dd700 1 --2- 172.21.15.104:0/2878982743 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f014015bf20 0x7f01400b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f01458dd700 1 -- 172.21.15.104:0/2878982743 >> 172.21.15.104:0/2878982743 conn(0x7f014006fc80 msgr2=0x7f0140070080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:39.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f01458dd700 1 -- 172.21.15.104:0/2878982743 shutdown_connections 2023-07-14T11:56:39.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.696+0000 7f01458dd700 1 -- 172.21.15.104:0/2878982743 wait complete. 2023-07-14T11:56:39.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.700+0000 7f01458dd700 1 Processor -- start 2023-07-14T11:56:39.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.700+0000 7f01458dd700 1 -- start start 2023-07-14T11:56:39.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f01458dd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01400b1530 0x7f01401f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:39.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f01458dd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01400d1430 0x7f01400c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:39.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f01458dd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014015bf20 0x7f01400b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:39.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f01458dd700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01400c6170 con 0x7f01400d1430 2023-07-14T11:56:39.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f01458dd700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01400b3880 con 0x7f014015bf20 2023-07-14T11:56:39.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f01458dd700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01400b2e20 con 0x7f01400b1530 2023-07-14T11:56:39.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f013e7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01400d1430 0x7f01400c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:39.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f013e7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01400d1430 0x7f01400c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45412/0 (socket says 172.21.15.104:45412) 2023-07-14T11:56:39.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f013e7fc700 1 -- 172.21.15.104:0/1159600440 learned_addr learned my addr 172.21.15.104:0/1159600440 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:39.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f013effd700 1 --2- 172.21.15.104:0/1159600440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01400b1530 0x7f01401f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:39.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f013f7fe700 1 --2- 172.21.15.104:0/1159600440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014015bf20 0x7f01400b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:39.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f013e7fc700 1 -- 172.21.15.104:0/1159600440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01400b1530 msgr2=0x7f01401f0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:39.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f013e7fc700 1 --2- 172.21.15.104:0/1159600440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01400b1530 0x7f01401f0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f013e7fc700 1 -- 172.21.15.104:0/1159600440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014015bf20 msgr2=0x7f01400b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:39.691 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f013e7fc700 1 --2- 172.21.15.104:0/1159600440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014015bf20 0x7f01400b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f013e7fc700 1 -- 172.21.15.104:0/1159600440 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0138009b30 con 0x7f01400d1430 2023-07-14T11:56:39.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f013f7fe700 1 --2- 172.21.15.104:0/1159600440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014015bf20 0x7f01400b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:39.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f013e7fc700 1 --2- 172.21.15.104:0/1159600440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01400d1430 0x7f01400c96a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f013400b4a0 tx=0x7f013400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:39.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f0127fff700 1 -- 172.21.15.104:0/1159600440 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134011830 con 0x7f01400d1430 2023-07-14T11:56:39.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f0127fff700 1 -- 172.21.15.104:0/1159600440 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0134011990 con 0x7f01400d1430 2023-07-14T11:56:39.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f01458dd700 1 -- 172.21.15.104:0/1159600440 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01400b3ad0 con 0x7f01400d1430 2023-07-14T11:56:39.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f0127fff700 1 -- 172.21.15.104:0/1159600440 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134011d40 con 0x7f01400d1430 2023-07-14T11:56:39.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f01458dd700 1 -- 172.21.15.104:0/1159600440 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01400b4090 con 0x7f01400d1430 2023-07-14T11:56:39.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.704+0000 7f01458dd700 1 -- 172.21.15.104:0/1159600440 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01400b42b0 con 0x7f01400d1430 2023-07-14T11:56:39.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.708+0000 7f0127fff700 1 -- 172.21.15.104:0/1159600440 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f013401a400 con 0x7f01400d1430 2023-07-14T11:56:39.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.708+0000 7f0127fff700 1 --2- 172.21.15.104:0/1159600440 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f012c066320 0x7f012c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:39.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.708+0000 7f0127fff700 1 -- 172.21.15.104:0/1159600440 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0134083b10 con 0x7f01400d1430 2023-07-14T11:56:39.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.708+0000 7f013effd700 1 --2- 172.21.15.104:0/1159600440 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f012c066320 0x7f012c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:39.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.712+0000 7f013effd700 1 --2- 172.21.15.104:0/1159600440 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f012c066320 0x7f012c0687d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f014015ec80 tx=0x7f0130008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:39.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.716+0000 7f0127fff700 1 -- 172.21.15.104:0/1159600440 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f0134050dc0 con 0x7f01400d1430 2023-07-14T11:56:39.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.884+0000 7f01458dd700 1 -- 172.21.15.104:0/1159600440 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01400d1f40 con 0x7f01400d1430 2023-07-14T11:56:39.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.884+0000 7f0127fff700 1 -- 172.21.15.104:0/1159600440 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+18534 (secure 0 0 0) 0x7f0134015020 con 0x7f01400d1430 2023-07-14T11:56:39.870 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:39.870 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":59,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:56:12.692633+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:56:08.755551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-14T11:56:10.132688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:56:39.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.888+0000 7f01458dd700 1 -- 172.21.15.104:0/1159600440 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f012c066320 msgr2=0x7f012c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:39.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.888+0000 7f01458dd700 1 --2- 172.21.15.104:0/1159600440 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f012c066320 0x7f012c0687d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f014015ec80 tx=0x7f0130008040 comp rx=0 tx=0).stop 2023-07-14T11:56:39.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.888+0000 7f01458dd700 1 -- 172.21.15.104:0/1159600440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01400d1430 msgr2=0x7f01400c96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:39.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.888+0000 7f01458dd700 1 --2- 172.21.15.104:0/1159600440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01400d1430 0x7f01400c96a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f013400b4a0 tx=0x7f013400b7b0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.888+0000 7f01458dd700 1 -- 172.21.15.104:0/1159600440 shutdown_connections 2023-07-14T11:56:39.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.888+0000 7f01458dd700 1 --2- 172.21.15.104:0/1159600440 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f012c066320 0x7f012c0687d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.888+0000 7f01458dd700 1 --2- 172.21.15.104:0/1159600440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01400d1430 0x7f01400c96a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.888+0000 7f01458dd700 1 --2- 172.21.15.104:0/1159600440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014015bf20 0x7f01400b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.888+0000 7f01458dd700 1 --2- 172.21.15.104:0/1159600440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01400b1530 0x7f01401f0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:39.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.888+0000 7f01458dd700 1 -- 172.21.15.104:0/1159600440 >> 172.21.15.104:0/1159600440 conn(0x7f014006fc80 msgr2=0x7f01400cf150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:39.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.888+0000 7f01458dd700 1 -- 172.21.15.104:0/1159600440 shutdown_connections 2023-07-14T11:56:39.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:39.888+0000 7f01458dd700 1 -- 172.21.15.104:0/1159600440 wait complete. 2023-07-14T11:56:39.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:56:08.755551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:56:10.132688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:56:39.892 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:56:40.072 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.088+0000 7f7244491700 1 Processor -- start 2023-07-14T11:56:40.072 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.088+0000 7f7244491700 1 -- start start 2023-07-14T11:56:40.075 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.088+0000 7f7244491700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f723c15bff0 0x7f723c06bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.075 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.088+0000 7f7244491700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f723c06c550 0x7f723c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.075 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.088+0000 7f7244491700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f723c0cd3a0 0x7f723c0b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.088+0000 7f7244491700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723c15fa60 con 0x7f723c06c550 2023-07-14T11:56:40.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.088+0000 7f7244491700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723c06c2f0 con 0x7f723c0cd3a0 2023-07-14T11:56:40.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.088+0000 7f7244491700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723c0cd810 con 0x7f723c15bff0 2023-07-14T11:56:40.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.088+0000 7f7241a2c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f723c06c550 0x7f723c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.088+0000 7f7241a2c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f723c06c550 0x7f723c0cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45422/0 (socket says 172.21.15.104:45422) 2023-07-14T11:56:40.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.088+0000 7f7241a2c700 1 -- 172.21.15.104:0/2231069356 learned_addr learned my addr 172.21.15.104:0/2231069356 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:40.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7242a2e700 1 --2- 172.21.15.104:0/2231069356 >> v2:172.21.15.121:3300/0 conn(0x7f723c0cd3a0 0x7f723c0b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f724222d700 1 --2- 172.21.15.104:0/2231069356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f723c15bff0 0x7f723c06bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7241a2c700 1 -- 172.21.15.104:0/2231069356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f723c15bff0 msgr2=0x7f723c06bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:40.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7241a2c700 1 --2- 172.21.15.104:0/2231069356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f723c15bff0 0x7f723c06bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7241a2c700 1 -- 172.21.15.104:0/2231069356 >> v2:172.21.15.121:3300/0 conn(0x7f723c0cd3a0 msgr2=0x7f723c0b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:40.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7241a2c700 1 --2- 172.21.15.104:0/2231069356 >> v2:172.21.15.121:3300/0 conn(0x7f723c0cd3a0 0x7f723c0b1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7241a2c700 1 -- 172.21.15.104:0/2231069356 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c15c3c0 con 0x7f723c06c550 2023-07-14T11:56:40.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7241a2c700 1 --2- 172.21.15.104:0/2231069356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f723c06c550 0x7f723c0cce60 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f723c15f9c0 tx=0x7f722c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39cb93a7e192f6d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:40.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f724122b700 1 -- 172.21.15.104:0/2231069356 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c011830 con 0x7f723c06c550 2023-07-14T11:56:40.078 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f724122b700 1 -- 172.21.15.104:0/2231069356 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722c011990 con 0x7f723c06c550 2023-07-14T11:56:40.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f724122b700 1 -- 172.21.15.104:0/2231069356 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c011d40 con 0x7f723c06c550 2023-07-14T11:56:40.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7244491700 1 -- 172.21.15.104:0/2231069356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f723c06c550 msgr2=0x7f723c0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.079 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7244491700 1 --2- 172.21.15.104:0/2231069356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f723c06c550 0x7f723c0cce60 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f723c15f9c0 tx=0x7f722c009c20 comp rx=0 tx=0).stop 2023-07-14T11:56:40.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7244491700 1 -- 172.21.15.104:0/2231069356 shutdown_connections 2023-07-14T11:56:40.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7244491700 1 --2- 172.21.15.104:0/2231069356 >> v2:172.21.15.121:3300/0 conn(0x7f723c0cd3a0 0x7f723c0b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7244491700 1 --2- 172.21.15.104:0/2231069356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f723c06c550 0x7f723c0cce60 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7244491700 1 --2- 172.21.15.104:0/2231069356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f723c15bff0 0x7f723c06bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7244491700 1 -- 172.21.15.104:0/2231069356 >> 172.21.15.104:0/2231069356 conn(0x7f723c060f70 msgr2=0x7f723c06a190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:40.080 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7244491700 1 -- 172.21.15.104:0/2231069356 shutdown_connections 2023-07-14T11:56:40.081 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.092+0000 7f7244491700 1 -- 172.21.15.104:0/2231069356 wait complete. 2023-07-14T11:56:40.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.096+0000 7f7244491700 1 Processor -- start 2023-07-14T11:56:40.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.096+0000 7f7244491700 1 -- start start 2023-07-14T11:56:40.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7244491700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f723c06c550 0x7f723c0c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.085 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7244491700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f723c0cd3a0 0x7f723c0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7244491700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f723c15bff0 0x7f723c15e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7244491700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723c0c7590 con 0x7f723c15bff0 2023-07-14T11:56:40.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7244491700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723c071910 con 0x7f723c06c550 2023-07-14T11:56:40.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7244491700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723c159f50 con 0x7f723c0cd3a0 2023-07-14T11:56:40.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7242a2e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f723c15bff0 0x7f723c15e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7242a2e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f723c15bff0 0x7f723c15e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45426/0 (socket says 172.21.15.104:45426) 2023-07-14T11:56:40.086 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7242a2e700 1 -- 172.21.15.104:0/2246726075 learned_addr learned my addr 172.21.15.104:0/2246726075 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:40.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f724222d700 1 --2- 172.21.15.104:0/2246726075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f723c06c550 0x7f723c0c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.087 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7241a2c700 1 --2- 172.21.15.104:0/2246726075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f723c0cd3a0 0x7f723c0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7242a2e700 1 -- 172.21.15.104:0/2246726075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f723c0cd3a0 msgr2=0x7f723c0c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7242a2e700 1 --2- 172.21.15.104:0/2246726075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f723c0cd3a0 0x7f723c0c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7242a2e700 1 -- 172.21.15.104:0/2246726075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f723c06c550 msgr2=0x7f723c0c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7242a2e700 1 --2- 172.21.15.104:0/2246726075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f723c06c550 0x7f723c0c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.088 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7242a2e700 1 -- 172.21.15.104:0/2246726075 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c009570 con 0x7f723c15bff0 2023-07-14T11:56:40.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.100+0000 7f7242a2e700 1 --2- 172.21.15.104:0/2246726075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f723c15bff0 0x7f723c15e730 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f723400eb60 tx=0x7f723400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:40.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.104+0000 7f7232ffd700 1 -- 172.21.15.104:0/2246726075 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234009740 con 0x7f723c15bff0 2023-07-14T11:56:40.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.104+0000 7f7232ffd700 1 -- 172.21.15.104:0/2246726075 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7234009e00 con 0x7f723c15bff0 2023-07-14T11:56:40.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.104+0000 7f7244491700 1 -- 172.21.15.104:0/2246726075 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f723c15a230 con 0x7f723c15bff0 2023-07-14T11:56:40.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.104+0000 7f7232ffd700 1 -- 172.21.15.104:0/2246726075 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723400ca70 con 0x7f723c15bff0 2023-07-14T11:56:40.089 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.104+0000 7f7244491700 1 -- 172.21.15.104:0/2246726075 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f723c15a7f0 con 0x7f723c15bff0 2023-07-14T11:56:40.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.104+0000 7f7244491700 1 -- 172.21.15.104:0/2246726075 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f723c071c00 con 0x7f723c15bff0 2023-07-14T11:56:40.090 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.104+0000 7f7232ffd700 1 -- 172.21.15.104:0/2246726075 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f723400cbd0 con 0x7f723c15bff0 2023-07-14T11:56:40.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.108+0000 7f7232ffd700 1 --2- 172.21.15.104:0/2246726075 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7220066320 0x7f72200687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.108+0000 7f7232ffd700 1 -- 172.21.15.104:0/2246726075 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7234083e70 con 0x7f723c15bff0 2023-07-14T11:56:40.092 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.108+0000 7f724222d700 1 --2- 172.21.15.104:0/2246726075 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7220066320 0x7f72200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.093 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.108+0000 7f724222d700 1 --2- 172.21.15.104:0/2246726075 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7220066320 0x7f72200687d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7238011340 tx=0x7f723800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:40.099 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.116+0000 7f7232ffd700 1 -- 172.21.15.104:0/2246726075 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f7234054ba0 con 0x7f723c15bff0 2023-07-14T11:56:40.273 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.288+0000 7f7244491700 1 -- 172.21.15.104:0/2246726075 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f723c06f5b0 con 0x7f723c15bff0 2023-07-14T11:56:40.274 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.288+0000 7f7232ffd700 1 -- 172.21.15.104:0/2246726075 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f723c06f5b0 con 0x7f723c15bff0 2023-07-14T11:56:40.275 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:56:40.278 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.292+0000 7f7244491700 1 -- 172.21.15.104:0/2246726075 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7220066320 msgr2=0x7f72200687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.292+0000 7f7244491700 1 --2- 172.21.15.104:0/2246726075 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7220066320 0x7f72200687d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7238011340 tx=0x7f723800b040 comp rx=0 tx=0).stop 2023-07-14T11:56:40.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.292+0000 7f7244491700 1 -- 172.21.15.104:0/2246726075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f723c15bff0 msgr2=0x7f723c15e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.279 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.292+0000 7f7244491700 1 --2- 172.21.15.104:0/2246726075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f723c15bff0 0x7f723c15e730 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f723400eb60 tx=0x7f723400ee70 comp rx=0 tx=0).stop 2023-07-14T11:56:40.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.292+0000 7f7244491700 1 -- 172.21.15.104:0/2246726075 shutdown_connections 2023-07-14T11:56:40.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.292+0000 7f7244491700 1 --2- 172.21.15.104:0/2246726075 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7220066320 0x7f72200687d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.280 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.292+0000 7f7244491700 1 --2- 172.21.15.104:0/2246726075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f723c15bff0 0x7f723c15e730 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.292+0000 7f7244491700 1 --2- 172.21.15.104:0/2246726075 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f723c0cd3a0 0x7f723c0c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.292+0000 7f7244491700 1 --2- 172.21.15.104:0/2246726075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f723c06c550 0x7f723c0c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.292+0000 7f7244491700 1 -- 172.21.15.104:0/2246726075 >> 172.21.15.104:0/2246726075 conn(0x7f723c060f70 msgr2=0x7f723c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:40.281 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.292+0000 7f7244491700 1 -- 172.21.15.104:0/2246726075 shutdown_connections 2023-07-14T11:56:40.282 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.292+0000 7f7244491700 1 -- 172.21.15.104:0/2246726075 wait complete. 2023-07-14T11:56:40.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:56:40.472 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.488+0000 7f86756a2700 1 Processor -- start 2023-07-14T11:56:40.472 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.488+0000 7f86756a2700 1 -- start start 2023-07-14T11:56:40.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.488+0000 7f86756a2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86700cb210 0x7f867006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.488+0000 7f86756a2700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86700714f0 con 0x7f8670159c60 2023-07-14T11:56:40.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.488+0000 7f86756a2700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867006da00 con 0x7f86700b0770 2023-07-14T11:56:40.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.488+0000 7f86756a2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867006dce0 con 0x7f86700cb210 2023-07-14T11:56:40.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.488+0000 7f866effd700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f8670159c60 0x7f8670070de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:50192/0 (socket says 172.21.15.104:50192) 2023-07-14T11:56:40.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.488+0000 7f866effd700 1 -- 172.21.15.104:0/700907058 learned_addr learned my addr 172.21.15.104:0/700907058 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:40.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866f7fe700 1 --2- 172.21.15.104:0/700907058 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86700cb210 0x7f867006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790382490 0 0) 0x7f86700714f0 con 0x7f8670159c60 2023-07-14T11:56:40.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86580035d0 con 0x7f8670159c60 2023-07-14T11:56:40.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076043701 0 0) 0x7f867006da00 con 0x7f86700b0770 2023-07-14T11:56:40.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86700714f0 con 0x7f86700b0770 2023-07-14T11:56:40.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2565076754 0 0) 0x7f86580035d0 con 0x7f8670159c60 2023-07-14T11:56:40.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86700cb210 msgr2=0x7f867006d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:40.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 --2- 172.21.15.104:0/700907058 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86700cb210 0x7f867006d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 >> v1:172.21.15.121:6789/0 conn(0x7f86700b0770 legacy=0x7f86700cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867006dfc0 con 0x7f8670159c60 2023-07-14T11:56:40.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f8660001670 con 0x7f8670159c60 2023-07-14T11:56:40.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 >> v1:172.21.15.104:6789/0 conn(0x7f8670159c60 legacy=0x7f8670070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 --2- 172.21.15.104:0/700907058 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86700cb210 0x7f8658003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 --2- 172.21.15.104:0/700907058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658004140 0x7f8658004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.483 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 --2- 172.21.15.104:0/700907058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8658004db0 0x7f8658005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867006dce0 con 0x7f8658004db0 2023-07-14T11:56:40.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865800bae0 con 0x7f8658004140 2023-07-14T11:56:40.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865800bca0 con 0x7f86700cb210 2023-07-14T11:56:40.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866f7fe700 1 --2- 172.21.15.104:0/700907058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8658004db0 0x7f8658005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866effd700 1 --2- 172.21.15.104:0/700907058 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86700cb210 0x7f8658003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866e7fc700 1 --2- 172.21.15.104:0/700907058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658004140 0x7f8658004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866f7fe700 1 -- 172.21.15.104:0/700907058 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86700cb210 msgr2=0x7f8658003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:40.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866f7fe700 1 --2- 172.21.15.104:0/700907058 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86700cb210 0x7f8658003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866f7fe700 1 -- 172.21.15.104:0/700907058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658004140 msgr2=0x7f8658004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:40.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866f7fe700 1 --2- 172.21.15.104:0/700907058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658004140 0x7f8658004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866f7fe700 1 -- 172.21.15.104:0/700907058 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86580035d0 con 0x7f8658004db0 2023-07-14T11:56:40.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866f7fe700 1 --2- 172.21.15.104:0/700907058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8658004db0 0x7f8658005220 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8668009f90 tx=0x7f86580058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1237885bb1508a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:40.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866800ec30 con 0x7f8658004db0 2023-07-14T11:56:40.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866800f040 con 0x7f8658004db0 2023-07-14T11:56:40.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.492+0000 7f866dffb700 1 -- 172.21.15.104:0/700907058 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668014480 con 0x7f8658004db0 2023-07-14T11:56:40.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.496+0000 7f86756a2700 1 -- 172.21.15.104:0/700907058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8658004db0 msgr2=0x7f8658005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.496+0000 7f86756a2700 1 --2- 172.21.15.104:0/700907058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8658004db0 0x7f8658005220 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8668009f90 tx=0x7f86580058a0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.496+0000 7f866f7fe700 1 -- 172.21.15.104:0/700907058 reap_dead start 2023-07-14T11:56:40.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.496+0000 7f86756a2700 1 -- 172.21.15.104:0/700907058 shutdown_connections 2023-07-14T11:56:40.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.496+0000 7f86756a2700 1 -- 172.21.15.104:0/700907058 >> 172.21.15.104:0/700907058 conn(0x7f86700afc60 msgr2=0x7f86700b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:40.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.496+0000 7f86756a2700 1 -- 172.21.15.104:0/700907058 shutdown_connections 2023-07-14T11:56:40.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.496+0000 7f86756a2700 1 -- 172.21.15.104:0/700907058 wait complete. 2023-07-14T11:56:40.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f86756a2700 1 Processor -- start 2023-07-14T11:56:40.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f86756a2700 1 -- start start 2023-07-14T11:56:40.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f86756a2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658004db0 0x7f86700b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f86756a2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86700b26e0 0x7f86701f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f86756a2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86700b2b50 0x7f86701f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f86756a2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8670156160 con 0x7f86700b26e0 2023-07-14T11:56:40.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f86756a2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86700c1c00 con 0x7f8658004db0 2023-07-14T11:56:40.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f86756a2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86700c4000 con 0x7f86700b2b50 2023-07-14T11:56:40.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f866e7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86700b26e0 0x7f86701f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f866e7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86700b26e0 0x7f86701f2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57814/0 (socket says 172.21.15.104:57814) 2023-07-14T11:56:40.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f866e7fc700 1 -- 172.21.15.104:0/165794979 learned_addr learned my addr 172.21.15.104:0/165794979 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:40.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f866effd700 1 --2- 172.21.15.104:0/165794979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658004db0 0x7f86700b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f866f7fe700 1 --2- 172.21.15.104:0/165794979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86700b2b50 0x7f86701f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f866e7fc700 1 -- 172.21.15.104:0/165794979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86700b2b50 msgr2=0x7f86701f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f866e7fc700 1 --2- 172.21.15.104:0/165794979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86700b2b50 0x7f86701f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f866e7fc700 1 -- 172.21.15.104:0/165794979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658004db0 msgr2=0x7f86700b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f866e7fc700 1 --2- 172.21.15.104:0/165794979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658004db0 0x7f86700b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f866e7fc700 1 -- 172.21.15.104:0/165794979 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86700c42e0 con 0x7f86700b26e0 2023-07-14T11:56:40.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f866effd700 1 --2- 172.21.15.104:0/165794979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658004db0 0x7f86700b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:40.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f866e7fc700 1 --2- 172.21.15.104:0/165794979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86700b26e0 0x7f86701f2920 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f866400e7b0 tx=0x7f866400eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:40.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.500+0000 7f86577fe700 1 -- 172.21.15.104:0/165794979 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86640052e0 con 0x7f86700b26e0 2023-07-14T11:56:40.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.504+0000 7f86577fe700 1 -- 172.21.15.104:0/165794979 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8664005440 con 0x7f86700b26e0 2023-07-14T11:56:40.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.504+0000 7f86756a2700 1 -- 172.21.15.104:0/165794979 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86700c45c0 con 0x7f86700b26e0 2023-07-14T11:56:40.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.504+0000 7f86577fe700 1 -- 172.21.15.104:0/165794979 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86640038c0 con 0x7f86700b26e0 2023-07-14T11:56:40.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.504+0000 7f86756a2700 1 -- 172.21.15.104:0/165794979 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86700c3c90 con 0x7f86700b26e0 2023-07-14T11:56:40.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.504+0000 7f86756a2700 1 -- 172.21.15.104:0/165794979 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86700c4a80 con 0x7f86700b26e0 2023-07-14T11:56:40.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.504+0000 7f86577fe700 1 -- 172.21.15.104:0/165794979 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8664003a20 con 0x7f86700b26e0 2023-07-14T11:56:40.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.508+0000 7f86577fe700 1 --2- 172.21.15.104:0/165794979 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f865c0662f0 0x7f865c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.508+0000 7f86577fe700 1 -- 172.21.15.104:0/165794979 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8664083cb0 con 0x7f86700b26e0 2023-07-14T11:56:40.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.508+0000 7f866effd700 1 --2- 172.21.15.104:0/165794979 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f865c0662f0 0x7f865c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.508+0000 7f866effd700 1 --2- 172.21.15.104:0/165794979 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f865c0662f0 0x7f865c0687a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8660001eb0 tx=0x7f8660009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:40.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.512+0000 7f86577fe700 1 -- 172.21.15.104:0/165794979 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f8664050ff0 con 0x7f86700b26e0 2023-07-14T11:56:40.668 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.684+0000 7f86756a2700 1 -- 172.21.15.104:0/165794979 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f867004e500 con 0x7f86700b26e0 2023-07-14T11:56:40.675 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.692+0000 7f86577fe700 1 -- 172.21.15.104:0/165794979 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v59) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8664054610 con 0x7f86700b26e0 2023-07-14T11:56:40.675 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:56:40.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.692+0000 7f86756a2700 1 -- 172.21.15.104:0/165794979 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f865c0662f0 msgr2=0x7f865c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.692+0000 7f86756a2700 1 --2- 172.21.15.104:0/165794979 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f865c0662f0 0x7f865c0687a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8660001eb0 tx=0x7f8660009040 comp rx=0 tx=0).stop 2023-07-14T11:56:40.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.692+0000 7f86756a2700 1 -- 172.21.15.104:0/165794979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86700b26e0 msgr2=0x7f86701f2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.692+0000 7f86756a2700 1 --2- 172.21.15.104:0/165794979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86700b26e0 0x7f86701f2920 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f866400e7b0 tx=0x7f866400eb70 comp rx=0 tx=0).stop 2023-07-14T11:56:40.679 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.692+0000 7f86756a2700 1 -- 172.21.15.104:0/165794979 shutdown_connections 2023-07-14T11:56:40.679 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.692+0000 7f86756a2700 1 --2- 172.21.15.104:0/165794979 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f865c0662f0 0x7f865c0687a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.679 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.692+0000 7f86756a2700 1 --2- 172.21.15.104:0/165794979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86700b26e0 0x7f86701f2920 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.679 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.692+0000 7f86756a2700 1 --2- 172.21.15.104:0/165794979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86700b2b50 0x7f86701f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.679 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.692+0000 7f86756a2700 1 --2- 172.21.15.104:0/165794979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658004db0 0x7f86700b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.680 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.692+0000 7f86756a2700 1 -- 172.21.15.104:0/165794979 >> 172.21.15.104:0/165794979 conn(0x7f86700afc60 msgr2=0x7f86700ae7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:40.680 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.692+0000 7f86756a2700 1 -- 172.21.15.104:0/165794979 shutdown_connections 2023-07-14T11:56:40.680 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.692+0000 7f86756a2700 1 -- 172.21.15.104:0/165794979 wait complete. 2023-07-14T11:56:40.699 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:56:40.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.892+0000 7f71864aa700 1 Processor -- start 2023-07-14T11:56:40.879 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.892+0000 7f71864aa700 1 -- start start 2023-07-14T11:56:40.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f71864aa700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71800b0490 0x7f71800c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f71864aa700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71800c8d90 0x7f71800cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f71864aa700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718006d860 con 0x7f718015c410 2023-07-14T11:56:40.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f71864aa700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71800ca710 con 0x7f71800c8d90 2023-07-14T11:56:40.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f71864aa700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718006dcf0 con 0x7f71800b0490 2023-07-14T11:56:40.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f717ffff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f718015c410 0x7f71800afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:50198/0 (socket says 172.21.15.104:50198) 2023-07-14T11:56:40.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f717ffff700 1 -- 172.21.15.104:0/2860307094 learned_addr learned my addr 172.21.15.104:0/2860307094 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:40.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f7184a47700 1 --2- 172.21.15.104:0/2860307094 >> v2:172.21.15.121:3300/0 conn(0x7f71800c8d90 0x7f71800cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f71777fe700 1 --2- 172.21.15.104:0/2860307094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71800b0490 0x7f71800c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f717f7fe700 1 -- 172.21.15.104:0/2860307094 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483025886 0 0) 0x7f718006d860 con 0x7f718015c410 2023-07-14T11:56:40.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f717f7fe700 1 -- 172.21.15.104:0/2860307094 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f716c0035d0 con 0x7f718015c410 2023-07-14T11:56:40.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f7184a47700 1 -- 172.21.15.104:0/2860307094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71800b0490 msgr2=0x7f71800c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:40.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f7184a47700 1 --2- 172.21.15.104:0/2860307094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71800b0490 0x7f71800c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f7184a47700 1 -- 172.21.15.104:0/2860307094 >> v1:172.21.15.104:6789/0 conn(0x7f718015c410 legacy=0x7f71800afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f7184a47700 1 -- 172.21.15.104:0/2860307094 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718006cc50 con 0x7f71800c8d90 2023-07-14T11:56:40.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.896+0000 7f71777fe700 1 --2- 172.21.15.104:0/2860307094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71800b0490 0x7f71800c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:40.886 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.900+0000 7f7184a47700 1 --2- 172.21.15.104:0/2860307094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71800c8d90 0x7f71800cbed0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7178009d90 tx=0x7f717800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b66949711b06d5f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:40.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.900+0000 7f717f7fe700 1 -- 172.21.15.104:0/2860307094 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717800e070 con 0x7f71800c8d90 2023-07-14T11:56:40.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.900+0000 7f717f7fe700 1 -- 172.21.15.104:0/2860307094 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717800bb20 con 0x7f71800c8d90 2023-07-14T11:56:40.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.900+0000 7f717f7fe700 1 -- 172.21.15.104:0/2860307094 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178012430 con 0x7f71800c8d90 2023-07-14T11:56:40.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.900+0000 7f71864aa700 1 -- 172.21.15.104:0/2860307094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71800c8d90 msgr2=0x7f71800cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.900+0000 7f71864aa700 1 --2- 172.21.15.104:0/2860307094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71800c8d90 0x7f71800cbed0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7178009d90 tx=0x7f717800b330 comp rx=0 tx=0).stop 2023-07-14T11:56:40.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.900+0000 7f71864aa700 1 -- 172.21.15.104:0/2860307094 shutdown_connections 2023-07-14T11:56:40.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.900+0000 7f71864aa700 1 --2- 172.21.15.104:0/2860307094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71800c8d90 0x7f71800cbed0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.900+0000 7f71864aa700 1 --2- 172.21.15.104:0/2860307094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71800b0490 0x7f71800c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.900+0000 7f71864aa700 1 -- 172.21.15.104:0/2860307094 >> 172.21.15.104:0/2860307094 conn(0x7f71800cad80 msgr2=0x7f71800cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:40.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.900+0000 7f71864aa700 1 -- 172.21.15.104:0/2860307094 shutdown_connections 2023-07-14T11:56:40.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.900+0000 7f71864aa700 1 -- 172.21.15.104:0/2860307094 wait complete. 2023-07-14T11:56:40.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f71864aa700 1 Processor -- start 2023-07-14T11:56:40.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f71864aa700 1 -- start start 2023-07-14T11:56:40.893 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f71864aa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71800b0490 0x7f7180159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f71864aa700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71800c8d90 0x7f71800c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f71864aa700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f718015c410 0x7f71800a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f71864aa700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800a88c0 con 0x7f718015c410 2023-07-14T11:56:40.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f71864aa700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800ab4d0 con 0x7f71800b0490 2023-07-14T11:56:40.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f71864aa700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718015a250 con 0x7f71800c8d90 2023-07-14T11:56:40.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f7184a47700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f718015c410 0x7f71800a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f7184a47700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f718015c410 0x7f71800a92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57824/0 (socket says 172.21.15.104:57824) 2023-07-14T11:56:40.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f7184a47700 1 -- 172.21.15.104:0/1776824851 learned_addr learned my addr 172.21.15.104:0/1776824851 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:40.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f71777fe700 1 --2- 172.21.15.104:0/1776824851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71800c8d90 0x7f71800c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f717ffff700 1 --2- 172.21.15.104:0/1776824851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71800b0490 0x7f7180159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f7184a47700 1 -- 172.21.15.104:0/1776824851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71800c8d90 msgr2=0x7f71800c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f7184a47700 1 --2- 172.21.15.104:0/1776824851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71800c8d90 0x7f71800c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f7184a47700 1 -- 172.21.15.104:0/1776824851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71800b0490 msgr2=0x7f7180159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:40.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f7184a47700 1 --2- 172.21.15.104:0/1776824851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71800b0490 0x7f7180159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:40.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f7184a47700 1 -- 172.21.15.104:0/1776824851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71680094b0 con 0x7f718015c410 2023-07-14T11:56:40.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.904+0000 7f717ffff700 1 --2- 172.21.15.104:0/1776824851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71800b0490 0x7f7180159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:40.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.908+0000 7f7184a47700 1 --2- 172.21.15.104:0/1776824851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f718015c410 0x7f71800a92f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7178014040 tx=0x7f7178004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:40.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.908+0000 7f717d7fa700 1 -- 172.21.15.104:0/1776824851 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71780048e0 con 0x7f718015c410 2023-07-14T11:56:40.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.908+0000 7f717d7fa700 1 -- 172.21.15.104:0/1776824851 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7178020070 con 0x7f718015c410 2023-07-14T11:56:40.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.908+0000 7f717d7fa700 1 -- 172.21.15.104:0/1776824851 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71780276c0 con 0x7f718015c410 2023-07-14T11:56:40.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.908+0000 7f71864aa700 1 -- 172.21.15.104:0/1776824851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7178009a70 con 0x7f718015c410 2023-07-14T11:56:40.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.908+0000 7f71864aa700 1 -- 172.21.15.104:0/1776824851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71800a8300 con 0x7f718015c410 2023-07-14T11:56:40.899 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.908+0000 7f71864aa700 1 -- 172.21.15.104:0/1776824851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718004dcb0 con 0x7f718015c410 2023-07-14T11:56:40.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.912+0000 7f717d7fa700 1 -- 172.21.15.104:0/1776824851 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7178012430 con 0x7f718015c410 2023-07-14T11:56:40.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.912+0000 7f717d7fa700 1 --2- 172.21.15.104:0/1776824851 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f71600662f0 0x7f71600687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:40.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.912+0000 7f717d7fa700 1 -- 172.21.15.104:0/1776824851 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7178022070 con 0x7f718015c410 2023-07-14T11:56:40.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.912+0000 7f717ffff700 1 --2- 172.21.15.104:0/1776824851 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f71600662f0 0x7f71600687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:40.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.912+0000 7f717ffff700 1 --2- 172.21.15.104:0/1776824851 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f71600662f0 0x7f71600687a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7170005930 tx=0x7f7170009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:40.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:40.916+0000 7f717d7fa700 1 -- 172.21.15.104:0/1776824851 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f7178052e20 con 0x7f718015c410 2023-07-14T11:56:41.076 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.092+0000 7f71864aa700 1 -- 172.21.15.104:0/1776824851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f71800b0c30 con 0x7f718015c410 2023-07-14T11:56:41.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.092+0000 7f717d7fa700 1 -- 172.21.15.104:0/1776824851 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7f717802c020 con 0x7f718015c410 2023-07-14T11:56:41.077 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:56:41.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.096+0000 7f71864aa700 1 -- 172.21.15.104:0/1776824851 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f71600662f0 msgr2=0x7f71600687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:41.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.096+0000 7f71864aa700 1 --2- 172.21.15.104:0/1776824851 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f71600662f0 0x7f71600687a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7170005930 tx=0x7f7170009040 comp rx=0 tx=0).stop 2023-07-14T11:56:41.082 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.096+0000 7f71864aa700 1 -- 172.21.15.104:0/1776824851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f718015c410 msgr2=0x7f71800a92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:41.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.096+0000 7f71864aa700 1 --2- 172.21.15.104:0/1776824851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f718015c410 0x7f71800a92f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7178014040 tx=0x7f7178004af0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.100+0000 7f71864aa700 1 -- 172.21.15.104:0/1776824851 shutdown_connections 2023-07-14T11:56:41.083 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.100+0000 7f71864aa700 1 --2- 172.21.15.104:0/1776824851 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f71600662f0 0x7f71600687a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.100+0000 7f71864aa700 1 --2- 172.21.15.104:0/1776824851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f718015c410 0x7f71800a92f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.100+0000 7f71864aa700 1 --2- 172.21.15.104:0/1776824851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71800c8d90 0x7f71800c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.100+0000 7f71864aa700 1 --2- 172.21.15.104:0/1776824851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71800b0490 0x7f7180159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.100+0000 7f71864aa700 1 -- 172.21.15.104:0/1776824851 >> 172.21.15.104:0/1776824851 conn(0x7f71800cad80 msgr2=0x7f71800af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:41.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.100+0000 7f71864aa700 1 -- 172.21.15.104:0/1776824851 shutdown_connections 2023-07-14T11:56:41.084 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.100+0000 7f71864aa700 1 -- 172.21.15.104:0/1776824851 wait complete. 2023-07-14T11:56:41.106 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:56:41.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.300+0000 7f763498b700 1 Processor -- start 2023-07-14T11:56:41.283 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.300+0000 7f763498b700 1 -- start start 2023-07-14T11:56:41.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.300+0000 7f763498b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f76300b0770 0x7f763006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:41.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.300+0000 7f763498b700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76300714f0 con 0x7f7630159c60 2023-07-14T11:56:41.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.300+0000 7f763498b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763006da00 con 0x7f76300cab00 2023-07-14T11:56:41.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.300+0000 7f763498b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763006dce0 con 0x7f76300b0770 2023-07-14T11:56:41.286 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.300+0000 7f762f7fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f7630159c60 0x7f7630070de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:50200/0 (socket says 172.21.15.104:50200) 2023-07-14T11:56:41.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.300+0000 7f762f7fe700 1 -- 172.21.15.104:0/2828239885 learned_addr learned my addr 172.21.15.104:0/2828239885 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:41.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.300+0000 7f762effd700 1 --2- 172.21.15.104:0/2828239885 >> v2:172.21.15.121:3300/0 conn(0x7f76300b0770 0x7f763006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:41.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.300+0000 7f762e7fc700 1 -- 172.21.15.104:0/2828239885 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130758194 0 0) 0x7f76300714f0 con 0x7f7630159c60 2023-07-14T11:56:41.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.300+0000 7f762e7fc700 1 -- 172.21.15.104:0/2828239885 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76180035d0 con 0x7f7630159c60 2023-07-14T11:56:41.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.304+0000 7f762effd700 1 -- 172.21.15.104:0/2828239885 >> v1:172.21.15.121:6789/0 conn(0x7f76300cab00 legacy=0x7f76300c1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:41.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.304+0000 7f762effd700 1 -- 172.21.15.104:0/2828239885 >> v1:172.21.15.104:6789/0 conn(0x7f7630159c60 legacy=0x7f7630070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:41.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.304+0000 7f762effd700 1 -- 172.21.15.104:0/2828239885 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763006dfc0 con 0x7f76300b0770 2023-07-14T11:56:41.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.304+0000 7f762effd700 1 --2- 172.21.15.104:0/2828239885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76300b0770 0x7f763006d3b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7624009850 tx=0x7f7624009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8fb4baed061dca5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:41.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.304+0000 7f762e7fc700 1 -- 172.21.15.104:0/2828239885 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624011830 con 0x7f76300b0770 2023-07-14T11:56:41.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.304+0000 7f762e7fc700 1 -- 172.21.15.104:0/2828239885 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7624011990 con 0x7f76300b0770 2023-07-14T11:56:41.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.304+0000 7f762e7fc700 1 -- 172.21.15.104:0/2828239885 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624011d00 con 0x7f76300b0770 2023-07-14T11:56:41.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.304+0000 7f763498b700 1 -- 172.21.15.104:0/2828239885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76300b0770 msgr2=0x7f763006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:41.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.304+0000 7f763498b700 1 --2- 172.21.15.104:0/2828239885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76300b0770 0x7f763006d3b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7624009850 tx=0x7f7624009b60 comp rx=0 tx=0).stop 2023-07-14T11:56:41.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.304+0000 7f763498b700 1 -- 172.21.15.104:0/2828239885 shutdown_connections 2023-07-14T11:56:41.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.304+0000 7f763498b700 1 --2- 172.21.15.104:0/2828239885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76300b0770 0x7f763006d3b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.291 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.304+0000 7f763498b700 1 -- 172.21.15.104:0/2828239885 >> 172.21.15.104:0/2828239885 conn(0x7f76300afc60 msgr2=0x7f76300b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:41.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.304+0000 7f763498b700 1 -- 172.21.15.104:0/2828239885 shutdown_connections 2023-07-14T11:56:41.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.304+0000 7f763498b700 1 -- 172.21.15.104:0/2828239885 wait complete. 2023-07-14T11:56:41.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.308+0000 7f763498b700 1 Processor -- start 2023-07-14T11:56:41.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.308+0000 7f763498b700 1 -- start start 2023-07-14T11:56:41.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f763498b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76300b0770 0x7f76301555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:41.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f763498b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76300cab00 0x7f76301efb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:41.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f763498b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7630159c60 0x7f76300b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:41.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f763498b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7630155b30 con 0x7f7630159c60 2023-07-14T11:56:41.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f763498b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76301f0090 con 0x7f76300b0770 2023-07-14T11:56:41.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f763498b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76300b12b0 con 0x7f76300cab00 2023-07-14T11:56:41.298 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f762ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7630159c60 0x7f76300b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:41.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f762ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7630159c60 0x7f76300b2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57830/0 (socket says 172.21.15.104:57830) 2023-07-14T11:56:41.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f762ffff700 1 -- 172.21.15.104:0/3232878038 learned_addr learned my addr 172.21.15.104:0/3232878038 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:41.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f762f7fe700 1 --2- 172.21.15.104:0/3232878038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76300b0770 0x7f76301555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:41.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f762ffff700 1 -- 172.21.15.104:0/3232878038 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76300cab00 msgr2=0x7f76301efb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:41.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f762ffff700 1 --2- 172.21.15.104:0/3232878038 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76300cab00 0x7f76301efb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f762ffff700 1 -- 172.21.15.104:0/3232878038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76300b0770 msgr2=0x7f76301555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:41.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f762ffff700 1 --2- 172.21.15.104:0/3232878038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76300b0770 0x7f76301555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f762ffff700 1 -- 172.21.15.104:0/3232878038 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76240094b0 con 0x7f7630159c60 2023-07-14T11:56:41.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f762f7fe700 1 --2- 172.21.15.104:0/3232878038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76300b0770 0x7f76301555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:41.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f762ffff700 1 --2- 172.21.15.104:0/3232878038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7630159c60 0x7f76300b2350 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f762800cc50 tx=0x7f762800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:41.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f7617fff700 1 -- 172.21.15.104:0/3232878038 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762800ad80 con 0x7f7630159c60 2023-07-14T11:56:41.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f7617fff700 1 -- 172.21.15.104:0/3232878038 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762800ce50 con 0x7f7630159c60 2023-07-14T11:56:41.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f7617fff700 1 -- 172.21.15.104:0/3232878038 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628003c60 con 0x7f7630159c60 2023-07-14T11:56:41.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f763498b700 1 -- 172.21.15.104:0/3232878038 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76300b1590 con 0x7f7630159c60 2023-07-14T11:56:41.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f763498b700 1 -- 172.21.15.104:0/3232878038 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76300b1b50 con 0x7f7630159c60 2023-07-14T11:56:41.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.312+0000 7f763498b700 1 -- 172.21.15.104:0/3232878038 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763004e500 con 0x7f7630159c60 2023-07-14T11:56:41.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.316+0000 7f7617fff700 1 -- 172.21.15.104:0/3232878038 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f76280050d0 con 0x7f7630159c60 2023-07-14T11:56:41.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.316+0000 7f7617fff700 1 --2- 172.21.15.104:0/3232878038 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f761c0662f0 0x7f761c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:41.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.316+0000 7f7617fff700 1 -- 172.21.15.104:0/3232878038 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7628084070 con 0x7f7630159c60 2023-07-14T11:56:41.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.316+0000 7f762f7fe700 1 --2- 172.21.15.104:0/3232878038 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f761c0662f0 0x7f761c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:41.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.316+0000 7f762f7fe700 1 --2- 172.21.15.104:0/3232878038 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f761c0662f0 0x7f761c0687a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f76200011e0 tx=0x7f7620009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:41.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.324+0000 7f7617fff700 1 -- 172.21.15.104:0/3232878038 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f76280b4720 con 0x7f7630159c60 2023-07-14T11:56:41.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.496+0000 7f763498b700 1 -- 172.21.15.104:0/3232878038 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f763006c6f0 con 0x7f7630159c60 2023-07-14T11:56:41.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.496+0000 7f7617fff700 1 -- 172.21.15.104:0/3232878038 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7f762805a020 con 0x7f7630159c60 2023-07-14T11:56:41.481 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:56:41.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.500+0000 7f763498b700 1 -- 172.21.15.104:0/3232878038 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f761c0662f0 msgr2=0x7f761c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:41.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.500+0000 7f763498b700 1 --2- 172.21.15.104:0/3232878038 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f761c0662f0 0x7f761c0687a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f76200011e0 tx=0x7f7620009040 comp rx=0 tx=0).stop 2023-07-14T11:56:41.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.500+0000 7f763498b700 1 -- 172.21.15.104:0/3232878038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7630159c60 msgr2=0x7f76300b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:41.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.500+0000 7f763498b700 1 --2- 172.21.15.104:0/3232878038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7630159c60 0x7f76300b2350 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f762800cc50 tx=0x7f762800a340 comp rx=0 tx=0).stop 2023-07-14T11:56:41.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.500+0000 7f763498b700 1 -- 172.21.15.104:0/3232878038 shutdown_connections 2023-07-14T11:56:41.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.500+0000 7f763498b700 1 --2- 172.21.15.104:0/3232878038 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f761c0662f0 0x7f761c0687a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.500+0000 7f763498b700 1 --2- 172.21.15.104:0/3232878038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7630159c60 0x7f76300b2350 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.500+0000 7f763498b700 1 --2- 172.21.15.104:0/3232878038 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f76300cab00 0x7f76301efb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.500+0000 7f763498b700 1 --2- 172.21.15.104:0/3232878038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76300b0770 0x7f76301555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.500+0000 7f763498b700 1 -- 172.21.15.104:0/3232878038 >> 172.21.15.104:0/3232878038 conn(0x7f76300afc60 msgr2=0x7f76300ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:41.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.500+0000 7f763498b700 1 -- 172.21.15.104:0/3232878038 shutdown_connections 2023-07-14T11:56:41.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.500+0000 7f763498b700 1 -- 172.21.15.104:0/3232878038 wait complete. 2023-07-14T11:56:41.506 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:56:41.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.696+0000 7f8ceb2d1700 1 Processor -- start 2023-07-14T11:56:41.681 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.696+0000 7f8ceb2d1700 1 -- start start 2023-07-14T11:56:41.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.696+0000 7f8ceb2d1700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ce415bff0 0x7f8ce40ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:41.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.696+0000 7f8ceb2d1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8ce40cd070 0x7f8ce40ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:41.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.696+0000 7f8ceb2d1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce40cd4c0 con 0x7f8ce40ceff0 2023-07-14T11:56:41.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.696+0000 7f8ceb2d1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce40af0a0 con 0x7f8ce40cd070 2023-07-14T11:56:41.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.696+0000 7f8ceb2d1700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce40cf470 con 0x7f8ce415bff0 2023-07-14T11:56:41.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.696+0000 7f8cea2cf700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ce415bff0 0x7f8ce40ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:41.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ce9ace700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8ce40cd070 0x7f8ce40ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:41.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ce9ace700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8ce40cd070 0x7f8ce40ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:55844/0 (socket says 172.21.15.104:55844) 2023-07-14T11:56:41.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8cea2cf700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ce415bff0 0x7f8ce40ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:50990/0 (socket says 172.21.15.104:50990) 2023-07-14T11:56:41.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ce9ace700 1 -- 172.21.15.104:0/1312989147 learned_addr learned my addr 172.21.15.104:0/1312989147 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:41.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ce9ace700 1 -- 172.21.15.104:0/1312989147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ce415bff0 msgr2=0x7f8ce40ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:41.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ce9ace700 1 --2- 172.21.15.104:0/1312989147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ce415bff0 0x7f8ce40ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ce9ace700 1 -- 172.21.15.104:0/1312989147 >> v1:172.21.15.121:6789/0 conn(0x7f8ce40ceff0 legacy=0x7f8ce40d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:41.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ce9ace700 1 -- 172.21.15.104:0/1312989147 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce40cf720 con 0x7f8ce40cd070 2023-07-14T11:56:41.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ce9ace700 1 --2- 172.21.15.104:0/1312989147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce40cd070 0x7f8ce40ceab0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009910 tx=0x7f8cd4009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61a6b0fc86c28e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:41.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ce92cd700 1 -- 172.21.15.104:0/1312989147 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4011830 con 0x7f8ce40cd070 2023-07-14T11:56:41.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ce92cd700 1 -- 172.21.15.104:0/1312989147 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd4011990 con 0x7f8ce40cd070 2023-07-14T11:56:41.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ce92cd700 1 -- 172.21.15.104:0/1312989147 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4011d40 con 0x7f8ce40cd070 2023-07-14T11:56:41.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ceb2d1700 1 -- 172.21.15.104:0/1312989147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce40cd070 msgr2=0x7f8ce40ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:41.686 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ceb2d1700 1 --2- 172.21.15.104:0/1312989147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce40cd070 0x7f8ce40ceab0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009910 tx=0x7f8cd4009c20 comp rx=0 tx=0).stop 2023-07-14T11:56:41.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ceb2d1700 1 -- 172.21.15.104:0/1312989147 shutdown_connections 2023-07-14T11:56:41.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ceb2d1700 1 --2- 172.21.15.104:0/1312989147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce40cd070 0x7f8ce40ceab0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ceb2d1700 1 --2- 172.21.15.104:0/1312989147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ce415bff0 0x7f8ce40ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ceb2d1700 1 -- 172.21.15.104:0/1312989147 >> 172.21.15.104:0/1312989147 conn(0x7f8ce4060c40 msgr2=0x7f8ce409b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:41.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ceb2d1700 1 -- 172.21.15.104:0/1312989147 shutdown_connections 2023-07-14T11:56:41.687 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.700+0000 7f8ceb2d1700 1 -- 172.21.15.104:0/1312989147 wait complete. 2023-07-14T11:56:41.689 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.704+0000 7f8ceb2d1700 1 Processor -- start 2023-07-14T11:56:41.690 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.704+0000 7f8ceb2d1700 1 -- start start 2023-07-14T11:56:41.692 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ceb2d1700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ce40cd070 0x7f8ce4159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:41.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ceb2d1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ce40ceff0 0x7f8ce4159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:41.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ceb2d1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce415bff0 0x7f8ce415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:41.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ceb2d1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce41f58f0 con 0x7f8ce40ceff0 2023-07-14T11:56:41.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ceb2d1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce41f6430 con 0x7f8ce415bff0 2023-07-14T11:56:41.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ceb2d1700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce41f5b00 con 0x7f8ce40cd070 2023-07-14T11:56:41.693 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ce9ace700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ce40ceff0 0x7f8ce4159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:41.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ce9ace700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ce40ceff0 0x7f8ce4159f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57844/0 (socket says 172.21.15.104:57844) 2023-07-14T11:56:41.694 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ce9ace700 1 -- 172.21.15.104:0/2412562626 learned_addr learned my addr 172.21.15.104:0/2412562626 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:41.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8cea2cf700 1 --2- 172.21.15.104:0/2412562626 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ce40cd070 0x7f8ce4159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:41.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ceaad0700 1 --2- 172.21.15.104:0/2412562626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce415bff0 0x7f8ce415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:41.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ce9ace700 1 -- 172.21.15.104:0/2412562626 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ce40cd070 msgr2=0x7f8ce4159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:41.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ce9ace700 1 --2- 172.21.15.104:0/2412562626 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ce40cd070 0x7f8ce4159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.695 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ce9ace700 1 -- 172.21.15.104:0/2412562626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce415bff0 msgr2=0x7f8ce415cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:41.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ce9ace700 1 --2- 172.21.15.104:0/2412562626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce415bff0 0x7f8ce415cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ce9ace700 1 -- 172.21.15.104:0/2412562626 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd4009570 con 0x7f8ce40ceff0 2023-07-14T11:56:41.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ceaad0700 1 --2- 172.21.15.104:0/2412562626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce415bff0 0x7f8ce415cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:41.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ce9ace700 1 --2- 172.21.15.104:0/2412562626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ce40ceff0 0x7f8ce4159f60 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4000c00 tx=0x7f8cd4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:41.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8cdaffd700 1 -- 172.21.15.104:0/2412562626 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4004510 con 0x7f8ce40ceff0 2023-07-14T11:56:41.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8cdaffd700 1 -- 172.21.15.104:0/2412562626 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd401f070 con 0x7f8ce40ceff0 2023-07-14T11:56:41.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ceb2d1700 1 -- 172.21.15.104:0/2412562626 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce41f5de0 con 0x7f8ce40ceff0 2023-07-14T11:56:41.696 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8cdaffd700 1 -- 172.21.15.104:0/2412562626 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd40038e0 con 0x7f8ce40ceff0 2023-07-14T11:56:41.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.708+0000 7f8ceb2d1700 1 -- 172.21.15.104:0/2412562626 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce41f3040 con 0x7f8ce40ceff0 2023-07-14T11:56:41.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.712+0000 7f8ceb2d1700 1 -- 172.21.15.104:0/2412562626 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce40af2d0 con 0x7f8ce40ceff0 2023-07-14T11:56:41.697 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.712+0000 7f8cdaffd700 1 -- 172.21.15.104:0/2412562626 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8cd4003a40 con 0x7f8ce40ceff0 2023-07-14T11:56:41.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.716+0000 7f8cdaffd700 1 --2- 172.21.15.104:0/2412562626 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cc80662f0 0x7f8cc80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:41.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.716+0000 7f8cdaffd700 1 -- 172.21.15.104:0/2412562626 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8cd4021070 con 0x7f8ce40ceff0 2023-07-14T11:56:41.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.716+0000 7f8cea2cf700 1 --2- 172.21.15.104:0/2412562626 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cc80662f0 0x7f8cc80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:41.700 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.716+0000 7f8cea2cf700 1 --2- 172.21.15.104:0/2412562626 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cc80662f0 0x7f8cc80687a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009b50 tx=0x7f8ce0009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:41.705 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.720+0000 7f8cdaffd700 1 -- 172.21.15.104:0/2412562626 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f8ce40af2d0 con 0x7f8ce40ceff0 2023-07-14T11:56:41.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.896+0000 7f8ceb2d1700 1 -- 172.21.15.104:0/2412562626 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8ce41f5c40 con 0x7f8ce40ceff0 2023-07-14T11:56:41.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.900+0000 7f8cdaffd700 1 -- 172.21.15.104:0/2412562626 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8ce41f5c40 con 0x7f8ce40ceff0 2023-07-14T11:56:41.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.904+0000 7f8ceb2d1700 1 -- 172.21.15.104:0/2412562626 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cc80662f0 msgr2=0x7f8cc80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:41.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.904+0000 7f8ceb2d1700 1 --2- 172.21.15.104:0/2412562626 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cc80662f0 0x7f8cc80687a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009b50 tx=0x7f8ce0009240 comp rx=0 tx=0).stop 2023-07-14T11:56:41.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.904+0000 7f8ceb2d1700 1 -- 172.21.15.104:0/2412562626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ce40ceff0 msgr2=0x7f8ce4159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:41.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.904+0000 7f8ceb2d1700 1 --2- 172.21.15.104:0/2412562626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ce40ceff0 0x7f8ce4159f60 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4000c00 tx=0x7f8cd4004770 comp rx=0 tx=0).stop 2023-07-14T11:56:41.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.904+0000 7f8ceb2d1700 1 -- 172.21.15.104:0/2412562626 shutdown_connections 2023-07-14T11:56:41.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.904+0000 7f8ceb2d1700 1 --2- 172.21.15.104:0/2412562626 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8cc80662f0 0x7f8cc80687a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.904+0000 7f8ceb2d1700 1 --2- 172.21.15.104:0/2412562626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ce40ceff0 0x7f8ce4159f60 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.904+0000 7f8ceb2d1700 1 --2- 172.21.15.104:0/2412562626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce415bff0 0x7f8ce415cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.904+0000 7f8ceb2d1700 1 --2- 172.21.15.104:0/2412562626 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ce40cd070 0x7f8ce4159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:41.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.904+0000 7f8ceb2d1700 1 -- 172.21.15.104:0/2412562626 >> 172.21.15.104:0/2412562626 conn(0x7f8ce4060c40 msgr2=0x7f8ce40d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:41.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.904+0000 7f8ceb2d1700 1 -- 172.21.15.104:0/2412562626 shutdown_connections 2023-07-14T11:56:41.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:41.904+0000 7f8ceb2d1700 1 -- 172.21.15.104:0/2412562626 wait complete. 2023-07-14T11:56:41.890 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:56:41.909 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:56:41.910 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:56:41.928 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:56:42.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.164+0000 7f7b950dc700 1 Processor -- start 2023-07-14T11:56:42.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.164+0000 7f7b950dc700 1 -- start start 2023-07-14T11:56:42.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.168+0000 7f7b950dc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7b9015ffa0 0x7f7b9006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.168+0000 7f7b950dc700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b900ca8a0 con 0x7f7b900cb180 2023-07-14T11:56:42.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.168+0000 7f7b950dc700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b900cbcd0 con 0x7f7b9006dc50 2023-07-14T11:56:42.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.168+0000 7f7b950dc700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b900cbf70 con 0x7f7b9015ffa0 2023-07-14T11:56:42.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.168+0000 7f7b8ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7b9015ffa0 0x7f7b9006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.168+0000 7f7b948db700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f7b900cb180 0x7f7b900ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:50210/0 (socket says 172.21.15.104:50210) 2023-07-14T11:56:42.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.168+0000 7f7b948db700 1 -- 172.21.15.104:0/1821985646 learned_addr learned my addr 172.21.15.104:0/1821985646 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:42.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.168+0000 7f7b8ffff700 1 -- 172.21.15.104:0/1821985646 >> v1:172.21.15.121:6789/0 conn(0x7f7b9006dc50 legacy=0x7f7b900caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.168+0000 7f7b8ffff700 1 -- 172.21.15.104:0/1821985646 >> v1:172.21.15.104:6789/0 conn(0x7f7b900cb180 legacy=0x7f7b900ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.168+0000 7f7b8ffff700 1 -- 172.21.15.104:0/1821985646 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900cc250 con 0x7f7b9015ffa0 2023-07-14T11:56:42.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.168+0000 7f7b8f7fe700 1 -- 172.21.15.104:0/1821985646 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1991141149 0 0) 0x7f7b88001490 con 0x7f7b900cb180 2023-07-14T11:56:42.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.168+0000 7f7b8ffff700 1 --2- 172.21.15.104:0/1821985646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9015ffa0 0x7f7b9006d710 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009850 tx=0x7f7b80009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920472a47bf5b25f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:42.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.168+0000 7f7b8f7fe700 1 -- 172.21.15.104:0/1821985646 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80011830 con 0x7f7b9015ffa0 2023-07-14T11:56:42.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.168+0000 7f7b8f7fe700 1 -- 172.21.15.104:0/1821985646 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b80011990 con 0x7f7b9015ffa0 2023-07-14T11:56:42.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.168+0000 7f7b8f7fe700 1 -- 172.21.15.104:0/1821985646 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80011d20 con 0x7f7b9015ffa0 2023-07-14T11:56:42.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.172+0000 7f7b950dc700 1 -- 172.21.15.104:0/1821985646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9015ffa0 msgr2=0x7f7b9006d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.172+0000 7f7b950dc700 1 --2- 172.21.15.104:0/1821985646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9015ffa0 0x7f7b9006d710 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009850 tx=0x7f7b80009b60 comp rx=0 tx=0).stop 2023-07-14T11:56:42.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.172+0000 7f7b950dc700 1 -- 172.21.15.104:0/1821985646 shutdown_connections 2023-07-14T11:56:42.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.172+0000 7f7b950dc700 1 --2- 172.21.15.104:0/1821985646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9015ffa0 0x7f7b9006d710 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.172+0000 7f7b950dc700 1 -- 172.21.15.104:0/1821985646 >> 172.21.15.104:0/1821985646 conn(0x7f7b9006e960 msgr2=0x7f7b9006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:42.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.172+0000 7f7b950dc700 1 -- 172.21.15.104:0/1821985646 shutdown_connections 2023-07-14T11:56:42.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.172+0000 7f7b950dc700 1 -- 172.21.15.104:0/1821985646 wait complete. 2023-07-14T11:56:42.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.172+0000 7f7b950dc700 1 Processor -- start 2023-07-14T11:56:42.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.172+0000 7f7b950dc700 1 -- start start 2023-07-14T11:56:42.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b950dc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9006dc50 0x7f7b901f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b950dc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b900cb180 0x7f7b900c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b950dc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b9015ffa0 0x7f7b900c19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b950dc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900c8470 con 0x7f7b9006dc50 2023-07-14T11:56:42.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b950dc700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b901612f0 con 0x7f7b900cb180 2023-07-14T11:56:42.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b950dc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b90160650 con 0x7f7b9015ffa0 2023-07-14T11:56:42.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b8ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9006dc50 0x7f7b901f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b8ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9006dc50 0x7f7b901f5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57868/0 (socket says 172.21.15.104:57868) 2023-07-14T11:56:42.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b8ffff700 1 -- 172.21.15.104:0/4281924381 learned_addr learned my addr 172.21.15.104:0/4281924381 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:42.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b87fff700 1 --2- 172.21.15.104:0/4281924381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b900cb180 0x7f7b900c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b948db700 1 --2- 172.21.15.104:0/4281924381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b9015ffa0 0x7f7b900c19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b8ffff700 1 -- 172.21.15.104:0/4281924381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b9015ffa0 msgr2=0x7f7b900c19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b8ffff700 1 --2- 172.21.15.104:0/4281924381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b9015ffa0 0x7f7b900c19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b8ffff700 1 -- 172.21.15.104:0/4281924381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b900cb180 msgr2=0x7f7b900c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b8ffff700 1 --2- 172.21.15.104:0/4281924381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b900cb180 0x7f7b900c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b8ffff700 1 -- 172.21.15.104:0/4281924381 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b800094b0 con 0x7f7b9006dc50 2023-07-14T11:56:42.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b87fff700 1 --2- 172.21.15.104:0/4281924381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b900cb180 0x7f7b900c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:42.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b8ffff700 1 --2- 172.21.15.104:0/4281924381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9006dc50 0x7f7b901f5340 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009fd0 tx=0x7f7b80004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:42.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b8d7fa700 1 -- 172.21.15.104:0/4281924381 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8000bc10 con 0x7f7b9006dc50 2023-07-14T11:56:42.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b950dc700 1 -- 172.21.15.104:0/4281924381 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b90161010 con 0x7f7b9006dc50 2023-07-14T11:56:42.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b8d7fa700 1 -- 172.21.15.104:0/4281924381 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8001e070 con 0x7f7b9006dc50 2023-07-14T11:56:42.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b8d7fa700 1 -- 172.21.15.104:0/4281924381 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8001a420 con 0x7f7b9006dc50 2023-07-14T11:56:42.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.176+0000 7f7b950dc700 1 -- 172.21.15.104:0/4281924381 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b900c65c0 con 0x7f7b9006dc50 2023-07-14T11:56:42.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.180+0000 7f7b950dc700 1 -- 172.21.15.104:0/4281924381 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b90071620 con 0x7f7b9006dc50 2023-07-14T11:56:42.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.180+0000 7f7b8d7fa700 1 -- 172.21.15.104:0/4281924381 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7b8001a580 con 0x7f7b9006dc50 2023-07-14T11:56:42.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.180+0000 7f7b8d7fa700 1 --2- 172.21.15.104:0/4281924381 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7b700662f0 0x7f7b700687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.180+0000 7f7b8d7fa700 1 -- 172.21.15.104:0/4281924381 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7b80085340 con 0x7f7b9006dc50 2023-07-14T11:56:42.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.184+0000 7f7b87fff700 1 --2- 172.21.15.104:0/4281924381 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7b700662f0 0x7f7b700687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.184+0000 7f7b87fff700 1 --2- 172.21.15.104:0/4281924381 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7b700662f0 0x7f7b700687a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7b9015fce0 tx=0x7f7b78009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:42.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.188+0000 7f7b8d7fa700 1 -- 172.21.15.104:0/4281924381 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f7b800525f0 con 0x7f7b9006dc50 2023-07-14T11:56:42.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.360+0000 7f7b950dc700 1 -- 172.21.15.104:0/4281924381 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b9004e500 con 0x7f7b9006dc50 2023-07-14T11:56:42.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.364+0000 7f7b8d7fa700 1 -- 172.21.15.104:0/4281924381 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+18534 (secure 0 0 0) 0x7f7b80055c10 con 0x7f7b9006dc50 2023-07-14T11:56:42.350 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:42.351 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":59,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:56:12.692633+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:56:08.755551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-14T11:56:10.132688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:56:42.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.368+0000 7f7b950dc700 1 -- 172.21.15.104:0/4281924381 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7b700662f0 msgr2=0x7f7b700687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.368+0000 7f7b950dc700 1 --2- 172.21.15.104:0/4281924381 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7b700662f0 0x7f7b700687a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7b9015fce0 tx=0x7f7b78009040 comp rx=0 tx=0).stop 2023-07-14T11:56:42.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.368+0000 7f7b950dc700 1 -- 172.21.15.104:0/4281924381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9006dc50 msgr2=0x7f7b901f5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.368+0000 7f7b950dc700 1 --2- 172.21.15.104:0/4281924381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9006dc50 0x7f7b901f5340 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009fd0 tx=0x7f7b80004770 comp rx=0 tx=0).stop 2023-07-14T11:56:42.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.368+0000 7f7b950dc700 1 -- 172.21.15.104:0/4281924381 shutdown_connections 2023-07-14T11:56:42.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.368+0000 7f7b950dc700 1 --2- 172.21.15.104:0/4281924381 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7b700662f0 0x7f7b700687a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.368+0000 7f7b950dc700 1 --2- 172.21.15.104:0/4281924381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b9015ffa0 0x7f7b900c19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.368+0000 7f7b950dc700 1 --2- 172.21.15.104:0/4281924381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b900cb180 0x7f7b900c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.368+0000 7f7b950dc700 1 --2- 172.21.15.104:0/4281924381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9006dc50 0x7f7b901f5340 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.368+0000 7f7b950dc700 1 -- 172.21.15.104:0/4281924381 >> 172.21.15.104:0/4281924381 conn(0x7f7b9006e960 msgr2=0x7f7b9006bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:42.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.368+0000 7f7b950dc700 1 -- 172.21.15.104:0/4281924381 shutdown_connections 2023-07-14T11:56:42.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.368+0000 7f7b950dc700 1 -- 172.21.15.104:0/4281924381 wait complete. 2023-07-14T11:56:42.375 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:56:08.755551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:56:10.132688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:56:42.376 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:56:42.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.568+0000 7f32f29d2700 1 Processor -- start 2023-07-14T11:56:42.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.568+0000 7f32f29d2700 1 -- start start 2023-07-14T11:56:42.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f29d2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f32ec0c39e0 0x7f32ec0af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f29d2700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32ec0d10a0 con 0x7f32ec160450 2023-07-14T11:56:42.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f29d2700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32ec070f40 con 0x7f32ec0af270 2023-07-14T11:56:42.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f29d2700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ec0711e0 con 0x7f32ec0c39e0 2023-07-14T11:56:42.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f19d0700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f32ec160450 0x7f32ec0c32d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:50220/0 (socket says 172.21.15.104:50220) 2023-07-14T11:56:42.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f19d0700 1 -- 172.21.15.104:0/922234217 learned_addr learned my addr 172.21.15.104:0/922234217 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:42.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f11cf700 1 --2- 172.21.15.104:0/922234217 >> v2:172.21.15.121:3300/0 conn(0x7f32ec0c39e0 0x7f32ec0af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f09ce700 1 -- 172.21.15.104:0/922234217 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2242831348 0 0) 0x7f32ec0d10a0 con 0x7f32ec160450 2023-07-14T11:56:42.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f09ce700 1 -- 172.21.15.104:0/922234217 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32d80035d0 con 0x7f32ec160450 2023-07-14T11:56:42.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f11cf700 1 -- 172.21.15.104:0/922234217 >> v1:172.21.15.121:6789/0 conn(0x7f32ec0af270 legacy=0x7f32ec0ad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f11cf700 1 -- 172.21.15.104:0/922234217 >> v1:172.21.15.104:6789/0 conn(0x7f32ec160450 legacy=0x7f32ec0c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f11cf700 1 -- 172.21.15.104:0/922234217 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec071480 con 0x7f32ec0c39e0 2023-07-14T11:56:42.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f11cf700 1 --2- 172.21.15.104:0/922234217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32ec0c39e0 0x7f32ec0af7c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009850 tx=0x7f32dc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ad0fc51d18859de server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:42.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f09ce700 1 -- 172.21.15.104:0/922234217 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc011830 con 0x7f32ec0c39e0 2023-07-14T11:56:42.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f09ce700 1 -- 172.21.15.104:0/922234217 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc011990 con 0x7f32ec0c39e0 2023-07-14T11:56:42.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.572+0000 7f32f09ce700 1 -- 172.21.15.104:0/922234217 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc011d00 con 0x7f32ec0c39e0 2023-07-14T11:56:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.576+0000 7f32f29d2700 1 -- 172.21.15.104:0/922234217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32ec0c39e0 msgr2=0x7f32ec0af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.576+0000 7f32f29d2700 1 --2- 172.21.15.104:0/922234217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32ec0c39e0 0x7f32ec0af7c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009850 tx=0x7f32dc009b60 comp rx=0 tx=0).stop 2023-07-14T11:56:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.576+0000 7f32f29d2700 1 -- 172.21.15.104:0/922234217 shutdown_connections 2023-07-14T11:56:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.576+0000 7f32f29d2700 1 --2- 172.21.15.104:0/922234217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32ec0c39e0 0x7f32ec0af7c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.576+0000 7f32f29d2700 1 -- 172.21.15.104:0/922234217 >> 172.21.15.104:0/922234217 conn(0x7f32ec0d0300 msgr2=0x7f32ec0d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.576+0000 7f32f29d2700 1 -- 172.21.15.104:0/922234217 shutdown_connections 2023-07-14T11:56:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.576+0000 7f32f29d2700 1 -- 172.21.15.104:0/922234217 wait complete. 2023-07-14T11:56:42.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f29d2700 1 Processor -- start 2023-07-14T11:56:42.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f29d2700 1 -- start start 2023-07-14T11:56:42.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f29d2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32ec0af270 0x7f32ec1583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f29d2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32ec0c39e0 0x7f32ec15b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f29d2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32ec160450 0x7f32ec0b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f29d2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec15cea0 con 0x7f32ec0af270 2023-07-14T11:56:42.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f29d2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec15c220 con 0x7f32ec0c39e0 2023-07-14T11:56:42.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f29d2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec15c500 con 0x7f32ec160450 2023-07-14T11:56:42.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f19d0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32ec0af270 0x7f32ec1583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f19d0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32ec0af270 0x7f32ec1583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57880/0 (socket says 172.21.15.104:57880) 2023-07-14T11:56:42.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f19d0700 1 -- 172.21.15.104:0/2355992884 learned_addr learned my addr 172.21.15.104:0/2355992884 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:42.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f21d1700 1 --2- 172.21.15.104:0/2355992884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32ec160450 0x7f32ec0b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f11cf700 1 --2- 172.21.15.104:0/2355992884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32ec0c39e0 0x7f32ec15b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f19d0700 1 -- 172.21.15.104:0/2355992884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32ec160450 msgr2=0x7f32ec0b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f19d0700 1 --2- 172.21.15.104:0/2355992884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32ec160450 0x7f32ec0b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f19d0700 1 -- 172.21.15.104:0/2355992884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32ec0c39e0 msgr2=0x7f32ec15b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f19d0700 1 --2- 172.21.15.104:0/2355992884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32ec0c39e0 0x7f32ec15b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f19d0700 1 -- 172.21.15.104:0/2355992884 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc0094b0 con 0x7f32ec0af270 2023-07-14T11:56:42.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.580+0000 7f32f11cf700 1 --2- 172.21.15.104:0/2355992884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32ec0c39e0 0x7f32ec15b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:42.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.584+0000 7f32f19d0700 1 --2- 172.21.15.104:0/2355992884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32ec0af270 0x7f32ec1583b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f32e8001260 tx=0x7f32e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:42.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.584+0000 7f32e27fc700 1 -- 172.21.15.104:0/2355992884 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e80074e0 con 0x7f32ec0af270 2023-07-14T11:56:42.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.584+0000 7f32e27fc700 1 -- 172.21.15.104:0/2355992884 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e8007640 con 0x7f32ec0af270 2023-07-14T11:56:42.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.584+0000 7f32e27fc700 1 -- 172.21.15.104:0/2355992884 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8005a50 con 0x7f32ec0af270 2023-07-14T11:56:42.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.584+0000 7f32f29d2700 1 -- 172.21.15.104:0/2355992884 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ec15c7e0 con 0x7f32ec0af270 2023-07-14T11:56:42.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.584+0000 7f32f29d2700 1 -- 172.21.15.104:0/2355992884 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ec0b4550 con 0x7f32ec0af270 2023-07-14T11:56:42.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.584+0000 7f32f29d2700 1 -- 172.21.15.104:0/2355992884 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ec04e500 con 0x7f32ec0af270 2023-07-14T11:56:42.572 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.588+0000 7f32e27fc700 1 -- 172.21.15.104:0/2355992884 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f32e8005bb0 con 0x7f32ec0af270 2023-07-14T11:56:42.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.588+0000 7f32e27fc700 1 --2- 172.21.15.104:0/2355992884 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f32d00662f0 0x7f32d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.588+0000 7f32e27fc700 1 -- 172.21.15.104:0/2355992884 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f32e80832d0 con 0x7f32ec0af270 2023-07-14T11:56:42.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.588+0000 7f32f11cf700 1 --2- 172.21.15.104:0/2355992884 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f32d00662f0 0x7f32d00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.588+0000 7f32f11cf700 1 --2- 172.21.15.104:0/2355992884 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f32d00662f0 0x7f32d00687a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0c3dd0 tx=0x7f32dc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:42.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.596+0000 7f32e27fc700 1 -- 172.21.15.104:0/2355992884 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f32e8050620 con 0x7f32ec0af270 2023-07-14T11:56:42.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.768+0000 7f32f29d2700 1 -- 172.21.15.104:0/2355992884 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32ec06d530 con 0x7f32ec0af270 2023-07-14T11:56:42.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.772+0000 7f32e27fc700 1 -- 172.21.15.104:0/2355992884 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32e8005e60 con 0x7f32ec0af270 2023-07-14T11:56:42.756 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:56:42.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.776+0000 7f32f29d2700 1 -- 172.21.15.104:0/2355992884 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f32d00662f0 msgr2=0x7f32d00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.776+0000 7f32f29d2700 1 --2- 172.21.15.104:0/2355992884 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f32d00662f0 0x7f32d00687a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0c3dd0 tx=0x7f32dc008030 comp rx=0 tx=0).stop 2023-07-14T11:56:42.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.776+0000 7f32f29d2700 1 -- 172.21.15.104:0/2355992884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32ec0af270 msgr2=0x7f32ec1583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.776+0000 7f32f29d2700 1 --2- 172.21.15.104:0/2355992884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32ec0af270 0x7f32ec1583b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f32e8001260 tx=0x7f32e800c880 comp rx=0 tx=0).stop 2023-07-14T11:56:42.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.776+0000 7f32f29d2700 1 -- 172.21.15.104:0/2355992884 shutdown_connections 2023-07-14T11:56:42.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.776+0000 7f32f29d2700 1 --2- 172.21.15.104:0/2355992884 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f32d00662f0 0x7f32d00687a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.776+0000 7f32f29d2700 1 --2- 172.21.15.104:0/2355992884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32ec160450 0x7f32ec0b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.776+0000 7f32f29d2700 1 --2- 172.21.15.104:0/2355992884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32ec0c39e0 0x7f32ec15b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.776+0000 7f32f29d2700 1 --2- 172.21.15.104:0/2355992884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32ec0af270 0x7f32ec1583b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.776+0000 7f32f29d2700 1 -- 172.21.15.104:0/2355992884 >> 172.21.15.104:0/2355992884 conn(0x7f32ec0d0300 msgr2=0x7f32ec0d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:42.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.776+0000 7f32f29d2700 1 -- 172.21.15.104:0/2355992884 shutdown_connections 2023-07-14T11:56:42.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.776+0000 7f32f29d2700 1 -- 172.21.15.104:0/2355992884 wait complete. 2023-07-14T11:56:42.780 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:56:42.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.968+0000 7f16b8fae700 1 Processor -- start 2023-07-14T11:56:42.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.968+0000 7f16b8fae700 1 -- start start 2023-07-14T11:56:42.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.968+0000 7f16b8fae700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b40c9060 0x7f16b415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.968+0000 7f16b8fae700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b4070a30 con 0x7f16b40afd80 2023-07-14T11:56:42.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.968+0000 7f16b8fae700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b40b0210 con 0x7f16b415df10 2023-07-14T11:56:42.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.968+0000 7f16b8fae700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b415e830 con 0x7f16b40c9060 2023-07-14T11:56:42.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.968+0000 7f16b1d9b700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f16b40afd80 0x7f16b40c8950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:50232/0 (socket says 172.21.15.104:50232) 2023-07-14T11:56:42.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.968+0000 7f16b1d9b700 1 -- 172.21.15.104:0/859302118 learned_addr learned my addr 172.21.15.104:0/859302118 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:42.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b2d9d700 1 --2- 172.21.15.104:0/859302118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b40c9060 0x7f16b415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 -- 172.21.15.104:0/859302118 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422632915 0 0) 0x7f16b4070a30 con 0x7f16b40afd80 2023-07-14T11:56:42.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 -- 172.21.15.104:0/859302118 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f169c0035d0 con 0x7f16b40afd80 2023-07-14T11:56:42.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 -- 172.21.15.104:0/859302118 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1805102391 0 0) 0x7f169c0035d0 con 0x7f16b40afd80 2023-07-14T11:56:42.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 -- 172.21.15.104:0/859302118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b40c9060 msgr2=0x7f16b415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:42.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 --2- 172.21.15.104:0/859302118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b40c9060 0x7f16b415e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 -- 172.21.15.104:0/859302118 >> v1:172.21.15.121:6789/0 conn(0x7f16b415df10 legacy=0x7f16b40711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 -- 172.21.15.104:0/859302118 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b40b04f0 con 0x7f16b40afd80 2023-07-14T11:56:42.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 -- 172.21.15.104:0/859302118 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f16a8001670 con 0x7f16b40afd80 2023-07-14T11:56:42.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 -- 172.21.15.104:0/859302118 >> v1:172.21.15.104:6789/0 conn(0x7f16b40afd80 legacy=0x7f16b40c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 --2- 172.21.15.104:0/859302118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b40c9060 0x7f169c003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 --2- 172.21.15.104:0/859302118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f169c003f90 0x7f169c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 --2- 172.21.15.104:0/859302118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169c004db0 0x7f169c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 -- 172.21.15.104:0/859302118 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c00bae0 con 0x7f169c003f90 2023-07-14T11:56:42.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 -- 172.21.15.104:0/859302118 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c00bc50 con 0x7f169c004db0 2023-07-14T11:56:42.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 -- 172.21.15.104:0/859302118 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169c00be10 con 0x7f16b40c9060 2023-07-14T11:56:42.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b1d9b700 1 --2- 172.21.15.104:0/859302118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f169c003f90 0x7f169c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b259c700 1 --2- 172.21.15.104:0/859302118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b40c9060 0x7f169c003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b2d9d700 1 --2- 172.21.15.104:0/859302118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169c004db0 0x7f169c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b1d9b700 1 -- 172.21.15.104:0/859302118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b40c9060 msgr2=0x7f169c003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:42.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b1d9b700 1 --2- 172.21.15.104:0/859302118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b40c9060 0x7f169c003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b1d9b700 1 -- 172.21.15.104:0/859302118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169c004db0 msgr2=0x7f169c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:42.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b1d9b700 1 --2- 172.21.15.104:0/859302118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169c004db0 0x7f169c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b1d9b700 1 -- 172.21.15.104:0/859302118 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c0035d0 con 0x7f169c003f90 2023-07-14T11:56:42.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b259c700 1 --2- 172.21.15.104:0/859302118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b40c9060 0x7f169c003a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:42.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b1d9b700 1 --2- 172.21.15.104:0/859302118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f169c003f90 0x7f169c004870 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f16a8001eb0 tx=0x7f169c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34b58bbe233a60a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:42.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 -- 172.21.15.104:0/859302118 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a800ce00 con 0x7f169c003f90 2023-07-14T11:56:42.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 -- 172.21.15.104:0/859302118 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a8005980 con 0x7f169c003f90 2023-07-14T11:56:42.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b159a700 1 -- 172.21.15.104:0/859302118 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8005d30 con 0x7f169c003f90 2023-07-14T11:56:42.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b8fae700 1 -- 172.21.15.104:0/859302118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f169c003f90 msgr2=0x7f169c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b8fae700 1 --2- 172.21.15.104:0/859302118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f169c003f90 0x7f169c004870 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f16a8001eb0 tx=0x7f169c0058a0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.972+0000 7f16b2d9d700 1 -- 172.21.15.104:0/859302118 reap_dead start 2023-07-14T11:56:42.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.976+0000 7f16b8fae700 1 -- 172.21.15.104:0/859302118 shutdown_connections 2023-07-14T11:56:42.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.976+0000 7f16b8fae700 1 -- 172.21.15.104:0/859302118 >> 172.21.15.104:0/859302118 conn(0x7f16b40cad80 msgr2=0x7f16b40cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:42.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.976+0000 7f16b8fae700 1 -- 172.21.15.104:0/859302118 shutdown_connections 2023-07-14T11:56:42.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.976+0000 7f16b8fae700 1 -- 172.21.15.104:0/859302118 wait complete. 2023-07-14T11:56:42.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.976+0000 7f16b8fae700 1 Processor -- start 2023-07-14T11:56:42.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.976+0000 7f16b8fae700 1 -- start start 2023-07-14T11:56:42.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.976+0000 7f16b8fae700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f169c003f90 0x7f16b4157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.976+0000 7f16b8fae700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b40ae530 0x7f16b40ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.976+0000 7f16b8fae700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16b415c310 0x7f16b415c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.976+0000 7f16b8fae700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b40c81d0 con 0x7f169c003f90 2023-07-14T11:56:42.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b8fae700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b415ccc0 con 0x7f16b415c310 2023-07-14T11:56:42.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b8fae700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b40c5010 con 0x7f16b40ae530 2023-07-14T11:56:42.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f169c003f90 0x7f16b4157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f169c003f90 0x7f16b4157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57912/0 (socket says 172.21.15.104:57912) 2023-07-14T11:56:42.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b259c700 1 -- 172.21.15.104:0/2530189383 learned_addr learned my addr 172.21.15.104:0/2530189383 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:42.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b2d9d700 1 --2- 172.21.15.104:0/2530189383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16b415c310 0x7f16b415c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b1d9b700 1 --2- 172.21.15.104:0/2530189383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b40ae530 0x7f16b40ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b259c700 1 -- 172.21.15.104:0/2530189383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b40ae530 msgr2=0x7f16b40ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b259c700 1 --2- 172.21.15.104:0/2530189383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b40ae530 0x7f16b40ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b259c700 1 -- 172.21.15.104:0/2530189383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16b415c310 msgr2=0x7f16b415c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:42.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b259c700 1 --2- 172.21.15.104:0/2530189383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16b415c310 0x7f16b415c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:42.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b259c700 1 -- 172.21.15.104:0/2530189383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b40c7300 con 0x7f169c003f90 2023-07-14T11:56:42.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b259c700 1 --2- 172.21.15.104:0/2530189383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f169c003f90 0x7f16b4157c50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f16a400e860 tx=0x7f16a400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:42.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16a2ffd700 1 -- 172.21.15.104:0/2530189383 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a40050d0 con 0x7f169c003f90 2023-07-14T11:56:42.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16a2ffd700 1 -- 172.21.15.104:0/2530189383 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a4005230 con 0x7f169c003f90 2023-07-14T11:56:42.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b8fae700 1 -- 172.21.15.104:0/2530189383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b40c7550 con 0x7f169c003f90 2023-07-14T11:56:42.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16a2ffd700 1 -- 172.21.15.104:0/2530189383 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a40055a0 con 0x7f169c003f90 2023-07-14T11:56:42.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b8fae700 1 -- 172.21.15.104:0/2530189383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b40c7b10 con 0x7f169c003f90 2023-07-14T11:56:42.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.980+0000 7f16b8fae700 1 -- 172.21.15.104:0/2530189383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b404dcb0 con 0x7f169c003f90 2023-07-14T11:56:42.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.984+0000 7f16a2ffd700 1 -- 172.21.15.104:0/2530189383 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f16a40036a0 con 0x7f169c003f90 2023-07-14T11:56:42.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.984+0000 7f16a2ffd700 1 --2- 172.21.15.104:0/2530189383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16940662f0 0x7f16940687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:42.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.984+0000 7f16a2ffd700 1 -- 172.21.15.104:0/2530189383 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f16a4083b50 con 0x7f169c003f90 2023-07-14T11:56:42.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.984+0000 7f16b1d9b700 1 --2- 172.21.15.104:0/2530189383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16940662f0 0x7f16940687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:42.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.988+0000 7f16b1d9b700 1 --2- 172.21.15.104:0/2530189383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16940662f0 0x7f16940687a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f16b41f41c0 tx=0x7f16a8007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:42.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:42.988+0000 7f16a2ffd700 1 -- 172.21.15.104:0/2530189383 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f16a4050eb0 con 0x7f169c003f90 2023-07-14T11:56:43.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.156+0000 7f16b8fae700 1 -- 172.21.15.104:0/2530189383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f16b415d6f0 con 0x7f169c003f90 2023-07-14T11:56:43.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.156+0000 7f16a2ffd700 1 -- 172.21.15.104:0/2530189383 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v59) v1 ==== 92+0+10 (secure 0 0 0) 0x7f16a40544d0 con 0x7f169c003f90 2023-07-14T11:56:43.142 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:56:43.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.160+0000 7f16b8fae700 1 -- 172.21.15.104:0/2530189383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16940662f0 msgr2=0x7f16940687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:43.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.160+0000 7f16b8fae700 1 --2- 172.21.15.104:0/2530189383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16940662f0 0x7f16940687a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f16b41f41c0 tx=0x7f16a8007000 comp rx=0 tx=0).stop 2023-07-14T11:56:43.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.160+0000 7f16b8fae700 1 -- 172.21.15.104:0/2530189383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f169c003f90 msgr2=0x7f16b4157c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:43.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.160+0000 7f16b8fae700 1 --2- 172.21.15.104:0/2530189383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f169c003f90 0x7f16b4157c50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f16a400e860 tx=0x7f16a400ec20 comp rx=0 tx=0).stop 2023-07-14T11:56:43.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.160+0000 7f16b8fae700 1 -- 172.21.15.104:0/2530189383 shutdown_connections 2023-07-14T11:56:43.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.160+0000 7f16b8fae700 1 --2- 172.21.15.104:0/2530189383 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16940662f0 0x7f16940687a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.160+0000 7f16b8fae700 1 --2- 172.21.15.104:0/2530189383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16b415c310 0x7f16b415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.160+0000 7f16b8fae700 1 --2- 172.21.15.104:0/2530189383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b40ae530 0x7f16b40ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.160+0000 7f16b8fae700 1 --2- 172.21.15.104:0/2530189383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f169c003f90 0x7f16b4157c50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.160+0000 7f16b8fae700 1 -- 172.21.15.104:0/2530189383 >> 172.21.15.104:0/2530189383 conn(0x7f16b40cad80 msgr2=0x7f16b40c8930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:43.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.160+0000 7f16b8fae700 1 -- 172.21.15.104:0/2530189383 shutdown_connections 2023-07-14T11:56:43.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.160+0000 7f16b8fae700 1 -- 172.21.15.104:0/2530189383 wait complete. 2023-07-14T11:56:43.166 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:56:43.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.360+0000 7f534b946700 1 Processor -- start 2023-07-14T11:56:43.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.360+0000 7f534b946700 1 -- start start 2023-07-14T11:56:43.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f534b946700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f53440d0210 0x7f53440d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:43.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f534b946700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53440cf000 0x7f53440cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:43.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f534b946700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5344071d10 con 0x7f53440b1c50 2023-07-14T11:56:43.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f534b946700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53440cf9c0 con 0x7f53440d0210 2023-07-14T11:56:43.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f534b946700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534415c420 con 0x7f53440cf000 2023-07-14T11:56:43.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f5348ee1700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f53440d0210 0x7f53440d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:43.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f5348ee1700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f53440d0210 0x7f53440d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57914/0 (socket says 172.21.15.104:57914) 2023-07-14T11:56:43.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f5348ee1700 1 -- 172.21.15.104:0/2881040614 learned_addr learned my addr 172.21.15.104:0/2881040614 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:43.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f5349ee3700 1 --2- 172.21.15.104:0/2881040614 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53440cf000 0x7f53440cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:43.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f5348ee1700 1 -- 172.21.15.104:0/2881040614 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53440cf000 msgr2=0x7f53440cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:43.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f5348ee1700 1 --2- 172.21.15.104:0/2881040614 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53440cf000 0x7f53440cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f5348ee1700 1 -- 172.21.15.104:0/2881040614 >> v1:172.21.15.104:6789/0 conn(0x7f53440b1c50 legacy=0x7f5344071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:43.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f5348ee1700 1 -- 172.21.15.104:0/2881040614 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5344070840 con 0x7f53440d0210 2023-07-14T11:56:43.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f5348ee1700 1 --2- 172.21.15.104:0/2881040614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53440d0210 0x7f53440d0640 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5334009910 tx=0x7f5334009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5bb7947575d848 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:43.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f533bfff700 1 -- 172.21.15.104:0/2881040614 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533400ba60 con 0x7f53440d0210 2023-07-14T11:56:43.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f533bfff700 1 -- 172.21.15.104:0/2881040614 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533400bbc0 con 0x7f53440d0210 2023-07-14T11:56:43.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f533bfff700 1 -- 172.21.15.104:0/2881040614 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5334009e60 con 0x7f53440d0210 2023-07-14T11:56:43.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f534b946700 1 -- 172.21.15.104:0/2881040614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53440d0210 msgr2=0x7f53440d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:43.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f534b946700 1 --2- 172.21.15.104:0/2881040614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53440d0210 0x7f53440d0640 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5334009910 tx=0x7f5334009c20 comp rx=0 tx=0).stop 2023-07-14T11:56:43.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f534b946700 1 -- 172.21.15.104:0/2881040614 shutdown_connections 2023-07-14T11:56:43.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f534b946700 1 --2- 172.21.15.104:0/2881040614 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53440cf000 0x7f53440cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f534b946700 1 --2- 172.21.15.104:0/2881040614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53440d0210 0x7f53440d0640 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.364+0000 7f534b946700 1 -- 172.21.15.104:0/2881040614 >> 172.21.15.104:0/2881040614 conn(0x7f534409b9c0 msgr2=0x7f53440d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:43.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.368+0000 7f534b946700 1 -- 172.21.15.104:0/2881040614 shutdown_connections 2023-07-14T11:56:43.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.368+0000 7f534b946700 1 -- 172.21.15.104:0/2881040614 wait complete. 2023-07-14T11:56:43.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.368+0000 7f534b946700 1 Processor -- start 2023-07-14T11:56:43.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.368+0000 7f534b946700 1 -- start start 2023-07-14T11:56:43.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f534b946700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53440b1c50 0x7f53441e5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:43.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f534b946700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53440cf000 0x7f5344150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:43.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f534b946700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53440d0210 0x7f53441e4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:43.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f534b946700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53441e6a20 con 0x7f53440b1c50 2023-07-14T11:56:43.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f534b946700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53441e6180 con 0x7f53440cf000 2023-07-14T11:56:43.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f534b946700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53441e6460 con 0x7f53440d0210 2023-07-14T11:56:43.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f53496e2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53440b1c50 0x7f53441e5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:43.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f53496e2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53440b1c50 0x7f53441e5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57926/0 (socket says 172.21.15.104:57926) 2023-07-14T11:56:43.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f53496e2700 1 -- 172.21.15.104:0/2333859061 learned_addr learned my addr 172.21.15.104:0/2333859061 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:43.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f5349ee3700 1 --2- 172.21.15.104:0/2333859061 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53440d0210 0x7f53441e4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:43.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f5348ee1700 1 --2- 172.21.15.104:0/2333859061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53440cf000 0x7f5344150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:43.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f53496e2700 1 -- 172.21.15.104:0/2333859061 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53440d0210 msgr2=0x7f53441e4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:43.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f53496e2700 1 --2- 172.21.15.104:0/2333859061 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53440d0210 0x7f53441e4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f53496e2700 1 -- 172.21.15.104:0/2333859061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53440cf000 msgr2=0x7f5344150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:43.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f53496e2700 1 --2- 172.21.15.104:0/2333859061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53440cf000 0x7f5344150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f53496e2700 1 -- 172.21.15.104:0/2333859061 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5334009570 con 0x7f53440b1c50 2023-07-14T11:56:43.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f53496e2700 1 --2- 172.21.15.104:0/2333859061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53440b1c50 0x7f53441e5b30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f534000c500 tx=0x7f534000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:43.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f5339ffb700 1 -- 172.21.15.104:0/2333859061 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53400052d0 con 0x7f53440b1c50 2023-07-14T11:56:43.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f5339ffb700 1 -- 172.21.15.104:0/2333859061 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5340005430 con 0x7f53440b1c50 2023-07-14T11:56:43.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f5339ffb700 1 -- 172.21.15.104:0/2333859061 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53400055a0 con 0x7f53440b1c50 2023-07-14T11:56:43.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f534b946700 1 -- 172.21.15.104:0/2333859061 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53441e6740 con 0x7f53440b1c50 2023-07-14T11:56:43.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.372+0000 7f534b946700 1 -- 172.21.15.104:0/2333859061 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53441e4560 con 0x7f53440b1c50 2023-07-14T11:56:43.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.376+0000 7f534b946700 1 -- 172.21.15.104:0/2333859061 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53440b2cd0 con 0x7f53440b1c50 2023-07-14T11:56:43.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.376+0000 7f5339ffb700 1 -- 172.21.15.104:0/2333859061 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f53400038e0 con 0x7f53440b1c50 2023-07-14T11:56:43.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.376+0000 7f5339ffb700 1 --2- 172.21.15.104:0/2333859061 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5328066320 0x7f53280687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:43.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.376+0000 7f5339ffb700 1 -- 172.21.15.104:0/2333859061 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5340083220 con 0x7f53440b1c50 2023-07-14T11:56:43.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.376+0000 7f5348ee1700 1 --2- 172.21.15.104:0/2333859061 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5328066320 0x7f53280687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:43.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.376+0000 7f5348ee1700 1 --2- 172.21.15.104:0/2333859061 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5328066320 0x7f53280687d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5334000c00 tx=0x7f5334008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:43.369 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.384+0000 7f5339ffb700 1 -- 172.21.15.104:0/2333859061 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f53440b2cd0 con 0x7f53440b1c50 2023-07-14T11:56:43.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.552+0000 7f534b946700 1 -- 172.21.15.104:0/2333859061 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5344071d10 con 0x7f53440b1c50 2023-07-14T11:56:43.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.556+0000 7f5339ffb700 1 -- 172.21.15.104:0/2333859061 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5344071d10 con 0x7f53440b1c50 2023-07-14T11:56:43.540 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:56:43.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.556+0000 7f534b946700 1 -- 172.21.15.104:0/2333859061 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5328066320 msgr2=0x7f53280687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:43.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.556+0000 7f534b946700 1 --2- 172.21.15.104:0/2333859061 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5328066320 0x7f53280687d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5334000c00 tx=0x7f5334008030 comp rx=0 tx=0).stop 2023-07-14T11:56:43.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.556+0000 7f534b946700 1 -- 172.21.15.104:0/2333859061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53440b1c50 msgr2=0x7f53441e5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:43.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.556+0000 7f534b946700 1 --2- 172.21.15.104:0/2333859061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53440b1c50 0x7f53441e5b30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f534000c500 tx=0x7f534000c8c0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.556+0000 7f534b946700 1 -- 172.21.15.104:0/2333859061 shutdown_connections 2023-07-14T11:56:43.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.556+0000 7f534b946700 1 --2- 172.21.15.104:0/2333859061 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5328066320 0x7f53280687d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.556+0000 7f534b946700 1 --2- 172.21.15.104:0/2333859061 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53440d0210 0x7f53441e4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.556+0000 7f534b946700 1 --2- 172.21.15.104:0/2333859061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53440cf000 0x7f5344150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.556+0000 7f534b946700 1 --2- 172.21.15.104:0/2333859061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53440b1c50 0x7f53441e5b30 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.556+0000 7f534b946700 1 -- 172.21.15.104:0/2333859061 >> 172.21.15.104:0/2333859061 conn(0x7f534409b9c0 msgr2=0x7f53440b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:43.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.560+0000 7f534b946700 1 -- 172.21.15.104:0/2333859061 shutdown_connections 2023-07-14T11:56:43.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.560+0000 7f534b946700 1 -- 172.21.15.104:0/2333859061 wait complete. 2023-07-14T11:56:43.564 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:56:43.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.756+0000 7f2a4a9e8700 1 Processor -- start 2023-07-14T11:56:43.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.756+0000 7f2a4a9e8700 1 -- start start 2023-07-14T11:56:43.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a4a9e8700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a440b0ee0 0x7f2a440ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:43.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a4a9e8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2a440cd090 0x7f2a440d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:43.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a4a9e8700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2a440d0610 0x7f2a4415f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:43.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a4a9e8700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a440cd4e0 con 0x7f2a440d0610 2023-07-14T11:56:43.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a4a9e8700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a440d1a60 con 0x7f2a440cd090 2023-07-14T11:56:43.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a4a9e8700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440d1ef0 con 0x7f2a440b0ee0 2023-07-14T11:56:43.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a4a1e7700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2a440d0610 0x7f2a4415f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:43.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a499e6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a440b0ee0 0x7f2a440ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:43.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a4a1e7700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f2a440d0610 0x7f2a4415f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57932/0 (socket says 172.21.15.104:57932) 2023-07-14T11:56:43.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a4a1e7700 1 -- 172.21.15.104:0/4174695617 learned_addr learned my addr 172.21.15.104:0/4174695617 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:43.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a491e5700 1 --2- 172.21.15.104:0/4174695617 >> v2:172.21.15.121:3300/0 conn(0x7f2a440cd090 0x7f2a440d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:43.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a4a1e7700 1 -- 172.21.15.104:0/4174695617 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a440b0ee0 msgr2=0x7f2a440ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:43.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a4a1e7700 1 --2- 172.21.15.104:0/4174695617 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a440b0ee0 0x7f2a440ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a4a1e7700 1 -- 172.21.15.104:0/4174695617 >> v2:172.21.15.121:3300/0 conn(0x7f2a440cd090 msgr2=0x7f2a440d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:43.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a4a1e7700 1 --2- 172.21.15.104:0/4174695617 >> v2:172.21.15.121:3300/0 conn(0x7f2a440cd090 0x7f2a440d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.760+0000 7f2a4a1e7700 1 -- 172.21.15.104:0/4174695617 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a440d0e50 con 0x7f2a440d0610 2023-07-14T11:56:43.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.764+0000 7f2a4a1e7700 1 --2- 172.21.15.104:0/4174695617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a440d0610 0x7f2a4415f850 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009e50 tx=0x7f2a3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901f37f6a2abb021 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:43.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.764+0000 7f2a489e4700 1 -- 172.21.15.104:0/4174695617 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c00e070 con 0x7f2a440d0610 2023-07-14T11:56:43.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.764+0000 7f2a489e4700 1 -- 172.21.15.104:0/4174695617 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3c00b8d0 con 0x7f2a440d0610 2023-07-14T11:56:43.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.764+0000 7f2a489e4700 1 -- 172.21.15.104:0/4174695617 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c00bc80 con 0x7f2a440d0610 2023-07-14T11:56:43.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.764+0000 7f2a4a9e8700 1 -- 172.21.15.104:0/4174695617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a440d0610 msgr2=0x7f2a4415f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:43.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.764+0000 7f2a4a9e8700 1 --2- 172.21.15.104:0/4174695617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a440d0610 0x7f2a4415f850 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009e50 tx=0x7f2a3c00b330 comp rx=0 tx=0).stop 2023-07-14T11:56:43.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.764+0000 7f2a4a9e8700 1 -- 172.21.15.104:0/4174695617 shutdown_connections 2023-07-14T11:56:43.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.764+0000 7f2a4a9e8700 1 --2- 172.21.15.104:0/4174695617 >> v2:172.21.15.121:3300/0 conn(0x7f2a440cd090 0x7f2a440d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.764+0000 7f2a4a9e8700 1 --2- 172.21.15.104:0/4174695617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a440d0610 0x7f2a4415f850 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.764+0000 7f2a4a9e8700 1 --2- 172.21.15.104:0/4174695617 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a440b0ee0 0x7f2a440ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.764+0000 7f2a4a9e8700 1 -- 172.21.15.104:0/4174695617 >> 172.21.15.104:0/4174695617 conn(0x7f2a44060c40 msgr2=0x7f2a4409b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:43.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.764+0000 7f2a4a9e8700 1 -- 172.21.15.104:0/4174695617 shutdown_connections 2023-07-14T11:56:43.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.764+0000 7f2a4a9e8700 1 -- 172.21.15.104:0/4174695617 wait complete. 2023-07-14T11:56:43.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.768+0000 7f2a4a9e8700 1 Processor -- start 2023-07-14T11:56:43.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.768+0000 7f2a4a9e8700 1 -- start start 2023-07-14T11:56:43.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.768+0000 7f2a4a9e8700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a440b0ee0 0x7f2a440b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:43.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.768+0000 7f2a4a9e8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a440cd090 0x7f2a440cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:43.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.768+0000 7f2a4a9e8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a440d0610 0x7f2a4415fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:43.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.768+0000 7f2a4a9e8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440cb5d0 con 0x7f2a440d0610 2023-07-14T11:56:43.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.768+0000 7f2a4a9e8700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440b2e80 con 0x7f2a440cd090 2023-07-14T11:56:43.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.768+0000 7f2a4a9e8700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440b6ee0 con 0x7f2a440b0ee0 2023-07-14T11:56:43.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.768+0000 7f2a4a1e7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a440d0610 0x7f2a4415fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:43.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.768+0000 7f2a4a1e7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a440d0610 0x7f2a4415fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57934/0 (socket says 172.21.15.104:57934) 2023-07-14T11:56:43.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.768+0000 7f2a4a1e7700 1 -- 172.21.15.104:0/1561342470 learned_addr learned my addr 172.21.15.104:0/1561342470 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:43.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.768+0000 7f2a499e6700 1 --2- 172.21.15.104:0/1561342470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a440b0ee0 0x7f2a440b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:43.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.768+0000 7f2a491e5700 1 --2- 172.21.15.104:0/1561342470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a440cd090 0x7f2a440cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:43.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.772+0000 7f2a4a1e7700 1 -- 172.21.15.104:0/1561342470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a440b0ee0 msgr2=0x7f2a440b79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:43.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.772+0000 7f2a4a1e7700 1 --2- 172.21.15.104:0/1561342470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a440b0ee0 0x7f2a440b79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.772+0000 7f2a4a1e7700 1 -- 172.21.15.104:0/1561342470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a440cd090 msgr2=0x7f2a440cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:43.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.772+0000 7f2a4a1e7700 1 --2- 172.21.15.104:0/1561342470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a440cd090 0x7f2a440cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.772+0000 7f2a4a1e7700 1 -- 172.21.15.104:0/1561342470 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c009b30 con 0x7f2a440d0610 2023-07-14T11:56:43.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.772+0000 7f2a4a1e7700 1 --2- 172.21.15.104:0/1561342470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a440d0610 0x7f2a4415fd10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2a4408adc0 tx=0x7f2a3c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:43.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.772+0000 7f2a3a7fc700 1 -- 172.21.15.104:0/1561342470 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c004450 con 0x7f2a440d0610 2023-07-14T11:56:43.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.772+0000 7f2a3a7fc700 1 -- 172.21.15.104:0/1561342470 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3c0045b0 con 0x7f2a440d0610 2023-07-14T11:56:43.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.772+0000 7f2a3a7fc700 1 -- 172.21.15.104:0/1561342470 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c004960 con 0x7f2a440d0610 2023-07-14T11:56:43.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.772+0000 7f2a4a9e8700 1 -- 172.21.15.104:0/1561342470 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a440ca950 con 0x7f2a440d0610 2023-07-14T11:56:43.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.772+0000 7f2a4a9e8700 1 -- 172.21.15.104:0/1561342470 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a440caf10 con 0x7f2a440d0610 2023-07-14T11:56:43.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.772+0000 7f2a4a9e8700 1 -- 172.21.15.104:0/1561342470 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a440d13f0 con 0x7f2a440d0610 2023-07-14T11:56:43.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.776+0000 7f2a3a7fc700 1 -- 172.21.15.104:0/1561342470 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2a3c01c410 con 0x7f2a440d0610 2023-07-14T11:56:43.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.776+0000 7f2a3a7fc700 1 --2- 172.21.15.104:0/1561342470 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2a28066320 0x7f2a280687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:43.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.776+0000 7f2a3a7fc700 1 -- 172.21.15.104:0/1561342470 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2a3c085830 con 0x7f2a440d0610 2023-07-14T11:56:43.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.776+0000 7f2a499e6700 1 --2- 172.21.15.104:0/1561342470 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2a28066320 0x7f2a280687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:43.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.776+0000 7f2a499e6700 1 --2- 172.21.15.104:0/1561342470 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2a28066320 0x7f2a280687d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009ad0 tx=0x7f2a40009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:43.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.780+0000 7f2a3a7fc700 1 -- 172.21.15.104:0/1561342470 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f2a440d13f0 con 0x7f2a440d0610 2023-07-14T11:56:43.937 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.952+0000 7f2a4a9e8700 1 -- 172.21.15.104:0/1561342470 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a440d13c0 con 0x7f2a440d0610 2023-07-14T11:56:43.938 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.952+0000 7f2a3a7fc700 1 -- 172.21.15.104:0/1561342470 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2a3c052b60 con 0x7f2a440d0610 2023-07-14T11:56:43.939 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:56:43.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.960+0000 7f2a4a9e8700 1 -- 172.21.15.104:0/1561342470 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2a28066320 msgr2=0x7f2a280687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:43.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.960+0000 7f2a4a9e8700 1 --2- 172.21.15.104:0/1561342470 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2a28066320 0x7f2a280687d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009ad0 tx=0x7f2a40009240 comp rx=0 tx=0).stop 2023-07-14T11:56:43.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.960+0000 7f2a4a9e8700 1 -- 172.21.15.104:0/1561342470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a440d0610 msgr2=0x7f2a4415fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:43.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.960+0000 7f2a4a9e8700 1 --2- 172.21.15.104:0/1561342470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a440d0610 0x7f2a4415fd10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2a4408adc0 tx=0x7f2a3c004af0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.960+0000 7f2a4a9e8700 1 -- 172.21.15.104:0/1561342470 shutdown_connections 2023-07-14T11:56:43.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.960+0000 7f2a4a9e8700 1 --2- 172.21.15.104:0/1561342470 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2a28066320 0x7f2a280687d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.960+0000 7f2a4a9e8700 1 --2- 172.21.15.104:0/1561342470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a440d0610 0x7f2a4415fd10 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.960+0000 7f2a4a9e8700 1 --2- 172.21.15.104:0/1561342470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a440cd090 0x7f2a440cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.960+0000 7f2a4a9e8700 1 --2- 172.21.15.104:0/1561342470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2a440b0ee0 0x7f2a440b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:43.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.960+0000 7f2a4a9e8700 1 -- 172.21.15.104:0/1561342470 >> 172.21.15.104:0/1561342470 conn(0x7f2a44060c40 msgr2=0x7f2a440cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:43.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.960+0000 7f2a4a9e8700 1 -- 172.21.15.104:0/1561342470 shutdown_connections 2023-07-14T11:56:43.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:43.960+0000 7f2a4a9e8700 1 -- 172.21.15.104:0/1561342470 wait complete. 2023-07-14T11:56:43.965 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:56:44.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.148+0000 7fbba8900700 1 Processor -- start 2023-07-14T11:56:44.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.148+0000 7fbba8900700 1 -- start start 2023-07-14T11:56:44.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba8900700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbba40cd240 0x7fbba40ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:44.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba8900700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbba40ceff0 0x7fbba40afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:44.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba8900700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba40af0a0 con 0x7fbba415bff0 2023-07-14T11:56:44.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba8900700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba40cf470 con 0x7fbba40ceff0 2023-07-14T11:56:44.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba8900700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba40cf720 con 0x7fbba40cd240 2023-07-14T11:56:44.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba359e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbba40ceff0 0x7fbba40afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:44.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba2d9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fbba415bff0 0x7fbba40ccb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:50244/0 (socket says 172.21.15.104:50244) 2023-07-14T11:56:44.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba2d9d700 1 -- 172.21.15.104:0/3357043674 learned_addr learned my addr 172.21.15.104:0/3357043674 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:44.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbb9bfff700 1 --2- 172.21.15.104:0/3357043674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbba40cd240 0x7fbba40ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:44.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba359e700 1 -- 172.21.15.104:0/3357043674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbba40cd240 msgr2=0x7fbba40ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:44.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba359e700 1 --2- 172.21.15.104:0/3357043674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbba40cd240 0x7fbba40ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:44.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba359e700 1 -- 172.21.15.104:0/3357043674 >> v1:172.21.15.104:6789/0 conn(0x7fbba415bff0 legacy=0x7fbba40ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:44.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba259c700 1 -- 172.21.15.104:0/3357043674 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123186322 0 0) 0x7fbba40af0a0 con 0x7fbba415bff0 2023-07-14T11:56:44.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba359e700 1 -- 172.21.15.104:0/3357043674 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba40af2f0 con 0x7fbba40ceff0 2023-07-14T11:56:44.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba359e700 1 --2- 172.21.15.104:0/3357043674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbba40ceff0 0x7fbba40afdd0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009d90 tx=0x7fbb9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff685f983d21b1e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:44.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba259c700 1 -- 172.21.15.104:0/3357043674 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c00e070 con 0x7fbba40ceff0 2023-07-14T11:56:44.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba259c700 1 -- 172.21.15.104:0/3357043674 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9c00bbd0 con 0x7fbba40ceff0 2023-07-14T11:56:44.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba259c700 1 -- 172.21.15.104:0/3357043674 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c012430 con 0x7fbba40ceff0 2023-07-14T11:56:44.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba8900700 1 -- 172.21.15.104:0/3357043674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbba40ceff0 msgr2=0x7fbba40afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:44.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.152+0000 7fbba8900700 1 --2- 172.21.15.104:0/3357043674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbba40ceff0 0x7fbba40afdd0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009d90 tx=0x7fbb9c00b330 comp rx=0 tx=0).stop 2023-07-14T11:56:44.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.156+0000 7fbba8900700 1 -- 172.21.15.104:0/3357043674 shutdown_connections 2023-07-14T11:56:44.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.156+0000 7fbba8900700 1 --2- 172.21.15.104:0/3357043674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbba40ceff0 0x7fbba40afdd0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:44.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.156+0000 7fbba8900700 1 --2- 172.21.15.104:0/3357043674 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbba40cd240 0x7fbba40ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:44.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.156+0000 7fbba8900700 1 -- 172.21.15.104:0/3357043674 >> 172.21.15.104:0/3357043674 conn(0x7fbba4060c40 msgr2=0x7fbba409b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:44.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.156+0000 7fbba8900700 1 -- 172.21.15.104:0/3357043674 shutdown_connections 2023-07-14T11:56:44.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.156+0000 7fbba8900700 1 -- 172.21.15.104:0/3357043674 wait complete. 2023-07-14T11:56:44.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba8900700 1 Processor -- start 2023-07-14T11:56:44.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba8900700 1 -- start start 2023-07-14T11:56:44.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba8900700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbba40cd240 0x7fbba40c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:44.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba8900700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbba40ceff0 0x7fbba41f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:44.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba8900700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbba415bff0 0x7fbba406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:44.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba8900700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba40c9370 con 0x7fbba40cd240 2023-07-14T11:56:44.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba8900700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba41f26e0 con 0x7fbba40ceff0 2023-07-14T11:56:44.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba8900700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba415c9f0 con 0x7fbba415bff0 2023-07-14T11:56:44.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba2d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbba40cd240 0x7fbba40c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:44.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba2d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbba40cd240 0x7fbba40c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57954/0 (socket says 172.21.15.104:57954) 2023-07-14T11:56:44.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba2d9d700 1 -- 172.21.15.104:0/2967685304 learned_addr learned my addr 172.21.15.104:0/2967685304 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:44.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbb9bfff700 1 --2- 172.21.15.104:0/2967685304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbba40ceff0 0x7fbba41f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:44.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba359e700 1 --2- 172.21.15.104:0/2967685304 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbba415bff0 0x7fbba406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:44.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba2d9d700 1 -- 172.21.15.104:0/2967685304 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbba415bff0 msgr2=0x7fbba406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:44.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba2d9d700 1 --2- 172.21.15.104:0/2967685304 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbba415bff0 0x7fbba406fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:44.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba2d9d700 1 -- 172.21.15.104:0/2967685304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbba40ceff0 msgr2=0x7fbba41f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:44.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba2d9d700 1 --2- 172.21.15.104:0/2967685304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbba40ceff0 0x7fbba41f21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:44.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba2d9d700 1 -- 172.21.15.104:0/2967685304 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c009a70 con 0x7fbba40cd240 2023-07-14T11:56:44.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.160+0000 7fbba359e700 1 --2- 172.21.15.104:0/2967685304 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbba415bff0 0x7fbba406fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:44.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.164+0000 7fbba2d9d700 1 --2- 172.21.15.104:0/2967685304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbba40cd240 0x7fbba40c8e30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400c4a0 tx=0x7fbb9400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:44.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.164+0000 7fbb9b7fe700 1 -- 172.21.15.104:0/2967685304 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb94004ca0 con 0x7fbba40cd240 2023-07-14T11:56:44.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.164+0000 7fbb9b7fe700 1 -- 172.21.15.104:0/2967685304 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb94004e00 con 0x7fbba40cd240 2023-07-14T11:56:44.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.164+0000 7fbb9b7fe700 1 -- 172.21.15.104:0/2967685304 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb94007310 con 0x7fbba40cd240 2023-07-14T11:56:44.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.164+0000 7fbba8900700 1 -- 172.21.15.104:0/2967685304 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba415ccd0 con 0x7fbba40cd240 2023-07-14T11:56:44.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.164+0000 7fbba8900700 1 -- 172.21.15.104:0/2967685304 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba415d290 con 0x7fbba40cd240 2023-07-14T11:56:44.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.164+0000 7fbba8900700 1 -- 172.21.15.104:0/2967685304 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba404e500 con 0x7fbba40cd240 2023-07-14T11:56:44.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.168+0000 7fbb9b7fe700 1 -- 172.21.15.104:0/2967685304 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fbb94007470 con 0x7fbba40cd240 2023-07-14T11:56:44.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.168+0000 7fbb9b7fe700 1 --2- 172.21.15.104:0/2967685304 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbb840662f0 0x7fbb840687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:44.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.168+0000 7fbb9b7fe700 1 -- 172.21.15.104:0/2967685304 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbb94083b10 con 0x7fbba40cd240 2023-07-14T11:56:44.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.168+0000 7fbb9bfff700 1 --2- 172.21.15.104:0/2967685304 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbb840662f0 0x7fbb840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:44.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.168+0000 7fbb9bfff700 1 --2- 172.21.15.104:0/2967685304 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbb840662f0 0x7fbb840687a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbba415df40 tx=0x7fbb8c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:44.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.176+0000 7fbb9b7fe700 1 -- 172.21.15.104:0/2967685304 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fbb94050dc0 con 0x7fbba40cd240 2023-07-14T11:56:44.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.360+0000 7fbba8900700 1 -- 172.21.15.104:0/2967685304 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbba415a330 con 0x7fbba40cd240 2023-07-14T11:56:44.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.360+0000 7fbb9b7fe700 1 -- 172.21.15.104:0/2967685304 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbb94007720 con 0x7fbba40cd240 2023-07-14T11:56:44.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.364+0000 7fbba8900700 1 -- 172.21.15.104:0/2967685304 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbb840662f0 msgr2=0x7fbb840687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:44.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.364+0000 7fbba8900700 1 --2- 172.21.15.104:0/2967685304 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbb840662f0 0x7fbb840687a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbba415df40 tx=0x7fbb8c008040 comp rx=0 tx=0).stop 2023-07-14T11:56:44.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.364+0000 7fbba8900700 1 -- 172.21.15.104:0/2967685304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbba40cd240 msgr2=0x7fbba40c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:44.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.364+0000 7fbba8900700 1 --2- 172.21.15.104:0/2967685304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbba40cd240 0x7fbba40c8e30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400c4a0 tx=0x7fbb9400c7b0 comp rx=0 tx=0).stop 2023-07-14T11:56:44.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.364+0000 7fbba8900700 1 -- 172.21.15.104:0/2967685304 shutdown_connections 2023-07-14T11:56:44.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.364+0000 7fbba8900700 1 --2- 172.21.15.104:0/2967685304 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbb840662f0 0x7fbb840687a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:44.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.364+0000 7fbba8900700 1 --2- 172.21.15.104:0/2967685304 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbba415bff0 0x7fbba406fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:44.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.364+0000 7fbba8900700 1 --2- 172.21.15.104:0/2967685304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbba40ceff0 0x7fbba41f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:44.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.364+0000 7fbba8900700 1 --2- 172.21.15.104:0/2967685304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbba40cd240 0x7fbba40c8e30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:44.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.364+0000 7fbba8900700 1 -- 172.21.15.104:0/2967685304 >> 172.21.15.104:0/2967685304 conn(0x7fbba4060c40 msgr2=0x7fbba40d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:44.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.364+0000 7fbba8900700 1 -- 172.21.15.104:0/2967685304 shutdown_connections 2023-07-14T11:56:44.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:44.364+0000 7fbba8900700 1 -- 172.21.15.104:0/2967685304 wait complete. 2023-07-14T11:56:44.356 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:56:44.367 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi104.front.sepia.ceph.com... 2023-07-14T11:56:44.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:44.368 DEBUG:teuthology.orchestra.run.smithi104:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:44.396 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[50536]: fuse finished with error 0 and tester_r 0 2023-07-14T11:56:44.400 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:56:50.501 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:56:50.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:50.502 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-14T11:56:50.522 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-14T11:56:50.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.752+0000 7f9418976700 1 Processor -- start 2023-07-14T11:56:50.737 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.752+0000 7f9418976700 1 -- start start 2023-07-14T11:56:50.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9418976700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f94140af4b0 0x7f94140cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:50.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9418976700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f941408a400 0x7f94140ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:50.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9418976700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94140cbbe0 con 0x7f941406fa70 2023-07-14T11:56:50.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9418976700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94140cbec0 con 0x7f941408a400 2023-07-14T11:56:50.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9418976700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94140cc1a0 con 0x7f94140af4b0 2023-07-14T11:56:50.741 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9412d9d700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f941406fa70 0x7f941406fe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:40888/0 (socket says 172.21.15.104:40888) 2023-07-14T11:56:50.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9412d9d700 1 -- 172.21.15.104:0/270253710 learned_addr learned my addr 172.21.15.104:0/270253710 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:50.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f941359e700 1 --2- 172.21.15.104:0/270253710 >> v2:172.21.15.104:3300/0 conn(0x7f941408a400 0x7f94140ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:50.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f941259c700 1 --2- 172.21.15.104:0/270253710 >> v2:172.21.15.121:3300/0 conn(0x7f94140af4b0 0x7f94140cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:50.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f941359e700 1 -- 172.21.15.104:0/270253710 >> v2:172.21.15.121:3300/0 conn(0x7f94140af4b0 msgr2=0x7f94140cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:50.742 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f941359e700 1 --2- 172.21.15.104:0/270253710 >> v2:172.21.15.121:3300/0 conn(0x7f94140af4b0 0x7f94140cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:50.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f941359e700 1 -- 172.21.15.104:0/270253710 >> v1:172.21.15.104:6789/0 conn(0x7f941406fa70 legacy=0x7f941406fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:50.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9411d9b700 1 -- 172.21.15.104:0/270253710 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692134682 0 0) 0x7f94140cbbe0 con 0x7f941406fa70 2023-07-14T11:56:50.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f941359e700 1 -- 172.21.15.104:0/270253710 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94140cc480 con 0x7f941408a400 2023-07-14T11:56:50.743 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f941359e700 1 --2- 172.21.15.104:0/270253710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f941408a400 0x7f94140ce440 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f941406ccf0 tx=0x7f940c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87aae36129c2b3be server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:50.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9411d9b700 1 -- 172.21.15.104:0/270253710 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c00e070 con 0x7f941408a400 2023-07-14T11:56:50.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9411d9b700 1 -- 172.21.15.104:0/270253710 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c00bb20 con 0x7f941408a400 2023-07-14T11:56:50.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9411d9b700 1 -- 172.21.15.104:0/270253710 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c012430 con 0x7f941408a400 2023-07-14T11:56:50.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9418976700 1 -- 172.21.15.104:0/270253710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f941408a400 msgr2=0x7f94140ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:50.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9418976700 1 --2- 172.21.15.104:0/270253710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f941408a400 0x7f94140ce440 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f941406ccf0 tx=0x7f940c00b330 comp rx=0 tx=0).stop 2023-07-14T11:56:50.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9418976700 1 -- 172.21.15.104:0/270253710 shutdown_connections 2023-07-14T11:56:50.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9418976700 1 --2- 172.21.15.104:0/270253710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f941408a400 0x7f94140ce440 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:50.747 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9418976700 1 --2- 172.21.15.104:0/270253710 >> v2:172.21.15.121:3300/0 conn(0x7f94140af4b0 0x7f94140cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:50.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9418976700 1 -- 172.21.15.104:0/270253710 >> 172.21.15.104:0/270253710 conn(0x7f941408a000 msgr2=0x7f94140d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:50.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9418976700 1 -- 172.21.15.104:0/270253710 shutdown_connections 2023-07-14T11:56:50.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.756+0000 7f9418976700 1 -- 172.21.15.104:0/270253710 wait complete. 2023-07-14T11:56:50.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.760+0000 7f9418976700 1 Processor -- start 2023-07-14T11:56:50.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.760+0000 7f9418976700 1 -- start start 2023-07-14T11:56:50.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f9418976700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f941406fa70 0x7f941415e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:50.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f9418976700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f941408a400 0x7f9414150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:50.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f9418976700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94140af4b0 0x7f9414150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:50.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f9418976700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941415dda0 con 0x7f941408a400 2023-07-14T11:56:50.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f9418976700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94140c7b80 con 0x7f94140af4b0 2023-07-14T11:56:50.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f9418976700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941415d060 con 0x7f941406fa70 2023-07-14T11:56:50.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f941259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f941408a400 0x7f9414150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:50.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f941259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f941408a400 0x7f9414150640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59668/0 (socket says 172.21.15.104:59668) 2023-07-14T11:56:50.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f941259c700 1 -- 172.21.15.104:0/183225758 learned_addr learned my addr 172.21.15.104:0/183225758 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:50.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f9412d9d700 1 --2- 172.21.15.104:0/183225758 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f941406fa70 0x7f941415e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:50.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f941359e700 1 --2- 172.21.15.104:0/183225758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94140af4b0 0x7f9414150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:50.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f941259c700 1 -- 172.21.15.104:0/183225758 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f941406fa70 msgr2=0x7f941415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:50.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f941259c700 1 --2- 172.21.15.104:0/183225758 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f941406fa70 0x7f941415e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:50.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f941259c700 1 -- 172.21.15.104:0/183225758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94140af4b0 msgr2=0x7f9414150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:50.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f941259c700 1 --2- 172.21.15.104:0/183225758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94140af4b0 0x7f9414150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:50.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f941259c700 1 -- 172.21.15.104:0/183225758 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c009a70 con 0x7f941408a400 2023-07-14T11:56:50.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f9412d9d700 1 --2- 172.21.15.104:0/183225758 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f941406fa70 0x7f941415e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:50.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f941259c700 1 --2- 172.21.15.104:0/183225758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f941408a400 0x7f9414150640 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f940800e9a0 tx=0x7f940800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:50.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f93fb7fe700 1 -- 172.21.15.104:0/183225758 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408009720 con 0x7f941408a400 2023-07-14T11:56:50.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f93fb7fe700 1 -- 172.21.15.104:0/183225758 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940800c690 con 0x7f941408a400 2023-07-14T11:56:50.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f9418976700 1 -- 172.21.15.104:0/183225758 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941415d2c0 con 0x7f941408a400 2023-07-14T11:56:50.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f93fb7fe700 1 -- 172.21.15.104:0/183225758 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940800ca40 con 0x7f941408a400 2023-07-14T11:56:50.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.764+0000 7f9418976700 1 -- 172.21.15.104:0/183225758 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941415d880 con 0x7f941408a400 2023-07-14T11:56:50.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.768+0000 7f9418976700 1 -- 172.21.15.104:0/183225758 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941404e500 con 0x7f941408a400 2023-07-14T11:56:50.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.768+0000 7f93fb7fe700 1 -- 172.21.15.104:0/183225758 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f940800cba0 con 0x7f941408a400 2023-07-14T11:56:50.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.772+0000 7f93fb7fe700 1 --2- 172.21.15.104:0/183225758 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94000662f0 0x7f94000687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:50.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.772+0000 7f93fb7fe700 1 -- 172.21.15.104:0/183225758 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9408083e00 con 0x7f941408a400 2023-07-14T11:56:50.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.772+0000 7f9412d9d700 1 --2- 172.21.15.104:0/183225758 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94000662f0 0x7f94000687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:50.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.772+0000 7f9412d9d700 1 --2- 172.21.15.104:0/183225758 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94000662f0 0x7f94000687a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9404000f60 tx=0x7f9404007e90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:50.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.776+0000 7f93fb7fe700 1 -- 172.21.15.104:0/183225758 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f9408054bc0 con 0x7f941408a400 2023-07-14T11:56:50.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.984+0000 7f9418976700 1 -- 172.21.15.104:0/183225758 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f94140ca4c0 con 0x7f941408a400 2023-07-14T11:56:50.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.984+0000 7f93fb7fe700 1 -- 172.21.15.104:0/183225758 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9408017070 con 0x7f941408a400 2023-07-14T11:56:50.969 INFO:teuthology.orchestra.run.smithi104.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-14T11:56:50.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.988+0000 7f9418976700 1 -- 172.21.15.104:0/183225758 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94000662f0 msgr2=0x7f94000687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:50.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.988+0000 7f9418976700 1 --2- 172.21.15.104:0/183225758 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94000662f0 0x7f94000687a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9404000f60 tx=0x7f9404007e90 comp rx=0 tx=0).stop 2023-07-14T11:56:50.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.988+0000 7f9418976700 1 -- 172.21.15.104:0/183225758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f941408a400 msgr2=0x7f9414150640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:50.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.988+0000 7f9418976700 1 --2- 172.21.15.104:0/183225758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f941408a400 0x7f9414150640 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f940800e9a0 tx=0x7f940800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:56:50.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.988+0000 7f9418976700 1 -- 172.21.15.104:0/183225758 shutdown_connections 2023-07-14T11:56:50.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.988+0000 7f9418976700 1 --2- 172.21.15.104:0/183225758 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f94000662f0 0x7f94000687a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:50.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.988+0000 7f9418976700 1 --2- 172.21.15.104:0/183225758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f941408a400 0x7f9414150640 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:50.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.988+0000 7f9418976700 1 --2- 172.21.15.104:0/183225758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94140af4b0 0x7f9414150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:50.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.988+0000 7f9418976700 1 --2- 172.21.15.104:0/183225758 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f941406fa70 0x7f941415e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:50.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.988+0000 7f9418976700 1 -- 172.21.15.104:0/183225758 >> 172.21.15.104:0/183225758 conn(0x7f941408a000 msgr2=0x7f94140cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:50.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.988+0000 7f9418976700 1 -- 172.21.15.104:0/183225758 shutdown_connections 2023-07-14T11:56:50.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:50.988+0000 7f9418976700 1 -- 172.21.15.104:0/183225758 wait complete. 2023-07-14T11:56:50.995 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-14T11:56:50.996 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T11:56:50.996 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2023-07-14T11:56:50.996 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-14T11:56:50.997 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:50.997 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-14T11:56:50.997 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T11:56:50.997 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T11:56:50.998 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:56:50.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:50.998 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2023-07-14T11:56:51.015 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:56:51.015 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:56:51.015 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:56:51.015 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:51.015 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2023-07-14T11:56:51.016 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:51.016 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:56:51.016 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:51.016 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2023-07-14T11:56:51.016 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:56:51.016 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:51.016 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2023-07-14T11:56:51.017 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-07-14T11:56:51.017 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:51.017 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T11:56:51.017 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:51.017 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-07-14T11:56:51.017 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42185sec preferred_lft 42185sec 2023-07-14T11:56:51.017 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2023-07-14T11:56:51.018 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:51.018 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-07-14T11:56:51.018 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:a2:a1:53:16 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:51.018 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-07-14T11:56:51.018 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:51.018 INFO:teuthology.orchestra.run.smithi104.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-14T11:56:51.019 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:b8:27:80:0d:ab brd ff:ff:ff:ff:ff:ff 2023-07-14T11:56:51.019 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-14T11:56:51.019 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:51.019 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5020:39ff:fe23:761d/64 scope link 2023-07-14T11:56:51.019 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:51.019 INFO:teuthology.orchestra.run.smithi104.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-14T11:56:51.019 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:b8:27:80:0d:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-14T11:56:51.019 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::e8b8:27ff:fe80:dab/64 scope link 2023-07-14T11:56:51.020 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:51.020 INFO:teuthology.orchestra.run.smithi104.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-14T11:56:51.020 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether ea:e6:23:3b:72:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-14T11:56:51.020 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::e8e6:23ff:fe3b:7247/64 scope link 2023-07-14T11:56:51.020 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:56:51.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:51.021 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T11:56:51.093 INFO:teuthology.orchestra.run.smithi104.stdout:1 2023-07-14T11:56:51.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:51.097 DEBUG:teuthology.orchestra.run.smithi104:> route 2023-07-14T11:56:51.158 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2023-07-14T11:56:51.159 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T11:56:51.159 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-07-14T11:56:51.159 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T11:56:51.159 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-07-14T11:56:51.159 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-07-14T11:56:51.159 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-07-14T11:56:51.160 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T11:56:51.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:51.161 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec bash -c ' 2023-07-14T11:56:51.161 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2023-07-14T11:56:51.161 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T11:56:51.161 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T11:56:51.162 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T11:56:51.162 DEBUG:teuthology.orchestra.run.smithi104:> ') 2023-07-14T11:56:51.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:51.298 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2023-07-14T11:56:51.316 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-14T11:56:51.317 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-14T11:56:51.317 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:56:51.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:51.319 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2023-07-14T11:56:51.393 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-14T11:56:51.393 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:56:51.394 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:51.451 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-14T11:56:51.452 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:56:51.452 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:51.512 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:56:51.591 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:56:51.592 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:56:51.667 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:56:51.669 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:56:51.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:51.671 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:56:51.733 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-14T11:56:51.733 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-14T11:56:51.782 DEBUG:teuthology.orchestra.run.smithi104:> sudo modprobe fuse 2023-07-14T11:56:51.812 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:56:51.813 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:56:51.890 INFO:teuthology.orchestra.run.smithi104.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:56:51.893 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:56:51.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:51.895 DEBUG:teuthology.orchestra.run.smithi104:> ls /sys/fs/fuse/connections 2023-07-14T11:56:51.922 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:2023-07-14T11:56:51.936+0000 7fbf33fe6640 -1 init, newargv = 0x5639fd7aac60 newargc=15 2023-07-14T11:56:51.923 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[51003]: starting ceph client 2023-07-14T11:56:51.935 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[51003]: starting fuse 2023-07-14T11:56:51.937 INFO:teuthology.orchestra.run.smithi104.stdout:49 2023-07-14T11:56:51.938 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-07-14T11:56:51.938 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T11:56:51.939 DEBUG:teuthology.orchestra.run.smithi104:> import glob 2023-07-14T11:56:51.940 DEBUG:teuthology.orchestra.run.smithi104:> import re 2023-07-14T11:56:51.941 DEBUG:teuthology.orchestra.run.smithi104:> import os 2023-07-14T11:56:51.941 DEBUG:teuthology.orchestra.run.smithi104:> import subprocess 2023-07-14T11:56:51.941 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:56:51.942 DEBUG:teuthology.orchestra.run.smithi104:> def _find_admin_socket(client_name): 2023-07-14T11:56:51.942 DEBUG:teuthology.orchestra.run.smithi104:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-14T11:56:51.942 DEBUG:teuthology.orchestra.run.smithi104:> files = glob.glob(asok_path) 2023-07-14T11:56:51.943 DEBUG:teuthology.orchestra.run.smithi104:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-14T11:56:51.943 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:56:51.943 DEBUG:teuthology.orchestra.run.smithi104:> # Given a non-glob path, it better be there 2023-07-14T11:56:51.943 DEBUG:teuthology.orchestra.run.smithi104:> if "*" not in asok_path: 2023-07-14T11:56:51.944 DEBUG:teuthology.orchestra.run.smithi104:> assert(len(files) == 1) 2023-07-14T11:56:51.944 DEBUG:teuthology.orchestra.run.smithi104:> return files[0] 2023-07-14T11:56:51.945 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:56:51.945 DEBUG:teuthology.orchestra.run.smithi104:> for f in files: 2023-07-14T11:56:51.945 DEBUG:teuthology.orchestra.run.smithi104:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-14T11:56:51.946 DEBUG:teuthology.orchestra.run.smithi104:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-14T11:56:51.946 DEBUG:teuthology.orchestra.run.smithi104:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-14T11:56:51.946 DEBUG:teuthology.orchestra.run.smithi104:> contents = proc_f.read() 2023-07-14T11:56:51.947 DEBUG:teuthology.orchestra.run.smithi104:> if mountpoint in contents: 2023-07-14T11:56:51.947 DEBUG:teuthology.orchestra.run.smithi104:> return f 2023-07-14T11:56:51.947 DEBUG:teuthology.orchestra.run.smithi104:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-14T11:56:51.948 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:56:51.948 DEBUG:teuthology.orchestra.run.smithi104:> print(_find_admin_socket("client.0")) 2023-07-14T11:56:51.948 DEBUG:teuthology.orchestra.run.smithi104:> ' 2023-07-14T11:56:52.095 INFO:teuthology.orchestra.run.smithi104.stdout:/var/run/ceph/ceph-client.0.51003.asok 2023-07-14T11:56:52.281 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51003.asok 2023-07-14T11:56:52.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:52.282 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51003.asok status 2023-07-14T11:56:52.473 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:56:52.474 INFO:teuthology.orchestra.run.smithi104.stdout: "metadata": { 2023-07-14T11:56:52.474 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-14T11:56:52.474 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:56:52.474 INFO:teuthology.orchestra.run.smithi104.stdout: "entity_id": "0", 2023-07-14T11:56:52.474 INFO:teuthology.orchestra.run.smithi104.stdout: "hostname": "smithi104", 2023-07-14T11:56:52.475 INFO:teuthology.orchestra.run.smithi104.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-14T11:56:52.475 INFO:teuthology.orchestra.run.smithi104.stdout: "pid": "51003", 2023-07-14T11:56:52.475 INFO:teuthology.orchestra.run.smithi104.stdout: "root": "/" 2023-07-14T11:56:52.475 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:52.475 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_count": 0, 2023-07-14T11:56:52.475 INFO:teuthology.orchestra.run.smithi104.stdout: "dentry_pinned_count": 0, 2023-07-14T11:56:52.475 INFO:teuthology.orchestra.run.smithi104.stdout: "id": 6678, 2023-07-14T11:56:52.475 INFO:teuthology.orchestra.run.smithi104.stdout: "inst": { 2023-07-14T11:56:52.476 INFO:teuthology.orchestra.run.smithi104.stdout: "name": { 2023-07-14T11:56:52.476 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "client", 2023-07-14T11:56:52.476 INFO:teuthology.orchestra.run.smithi104.stdout: "num": 6678 2023-07-14T11:56:52.476 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:52.476 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:56:52.476 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:56:52.476 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:56:52.477 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 3324392248 2023-07-14T11:56:52.477 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-14T11:56:52.477 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:52.477 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": { 2023-07-14T11:56:52.477 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2023-07-14T11:56:52.477 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:56:52.477 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 3324392248 2023-07-14T11:56:52.477 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-14T11:56:52.478 INFO:teuthology.orchestra.run.smithi104.stdout: "inst_str": "client.6678 192.168.0.1:0/3324392248", 2023-07-14T11:56:52.478 INFO:teuthology.orchestra.run.smithi104.stdout: "addr_str": "192.168.0.1:0/3324392248", 2023-07-14T11:56:52.478 INFO:teuthology.orchestra.run.smithi104.stdout: "inode_count": 1, 2023-07-14T11:56:52.478 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_epoch": 27, 2023-07-14T11:56:52.478 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch": 59, 2023-07-14T11:56:52.478 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_epoch_barrier": 0, 2023-07-14T11:56:52.478 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklisted": false, 2023-07-14T11:56:52.478 INFO:teuthology.orchestra.run.smithi104.stdout: "fs_name": "cephfs" 2023-07-14T11:56:52.479 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:56:52.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:52.488 DEBUG:teuthology.orchestra.run.smithi104:> stat --file-system '--printf=%T 2023-07-14T11:56:52.488 DEBUG:teuthology.orchestra.run.smithi104:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:52.504 INFO:teuthology.orchestra.run.smithi104.stdout:fuseblk 2023-07-14T11:56:52.505 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:52.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:52.506 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:52.580 DEBUG:teuthology.orchestra.run.smithi104:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T11:56:52.580 DEBUG:teuthology.orchestra.run.smithi104:> import os 2023-07-14T11:56:52.580 DEBUG:teuthology.orchestra.run.smithi104:> import errno 2023-07-14T11:56:52.581 DEBUG:teuthology.orchestra.run.smithi104:> 2023-07-14T11:56:52.581 DEBUG:teuthology.orchestra.run.smithi104:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2023-07-14T11:56:52.581 DEBUG:teuthology.orchestra.run.smithi104:> try: 2023-07-14T11:56:52.581 DEBUG:teuthology.orchestra.run.smithi104:> if True: 2023-07-14T11:56:52.582 DEBUG:teuthology.orchestra.run.smithi104:> ret = os.read(fd, 1024) 2023-07-14T11:56:52.582 DEBUG:teuthology.orchestra.run.smithi104:> else: 2023-07-14T11:56:52.582 DEBUG:teuthology.orchestra.run.smithi104:> os.write(fd, b'"'"'content'"'"') 2023-07-14T11:56:52.582 DEBUG:teuthology.orchestra.run.smithi104:> except OSError as e: 2023-07-14T11:56:52.583 DEBUG:teuthology.orchestra.run.smithi104:> if e.errno != errno.EPERM: 2023-07-14T11:56:52.583 DEBUG:teuthology.orchestra.run.smithi104:> raise 2023-07-14T11:56:52.583 DEBUG:teuthology.orchestra.run.smithi104:> else: 2023-07-14T11:56:52.584 DEBUG:teuthology.orchestra.run.smithi104:> raise RuntimeError("client does not check permission of data pool") 2023-07-14T11:56:52.584 DEBUG:teuthology.orchestra.run.smithi104:> ' 2023-07-14T11:56:52.721 INFO:teuthology.orchestra.run.smithi104.stderr:Traceback (most recent call last): 2023-07-14T11:56:52.721 INFO:teuthology.orchestra.run.smithi104.stderr: File "", line 15, in 2023-07-14T11:56:52.721 INFO:teuthology.orchestra.run.smithi104.stderr:RuntimeError: client does not check permission of data pool 2023-07-14T11:56:52.893 INFO:teuthology.orchestra.run.smithi104.stderr:daemon-helper: command failed with exit status 1 2023-07-14T11:56:52.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:56:52.908 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2023-07-14T11:56:53.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.152+0000 7ffa6a093700 1 Processor -- start 2023-07-14T11:56:53.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.156+0000 7ffa6a093700 1 -- start start 2023-07-14T11:56:53.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.156+0000 7ffa6a093700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa641459c0 0x7ffa640ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:53.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.156+0000 7ffa6a093700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ffa640d0860 0x7ffa640cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:53.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.156+0000 7ffa6a093700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa64145d90 con 0x7ffa640ccbd0 2023-07-14T11:56:53.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.156+0000 7ffa6a093700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa640cd650 con 0x7ffa640d0860 2023-07-14T11:56:53.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.156+0000 7ffa6a093700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa640cd930 con 0x7ffa641459c0 2023-07-14T11:56:53.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.156+0000 7ffa69091700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa641459c0 0x7ffa640ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:53.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.156+0000 7ffa68890700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ffa640ccbd0 0x7ffa640d0150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:37490/0 (socket says 172.21.15.104:37490) 2023-07-14T11:56:53.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.156+0000 7ffa68890700 1 -- 172.21.15.104:0/2981340722 learned_addr learned my addr 172.21.15.104:0/2981340722 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:53.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.156+0000 7ffa69892700 1 --2- 172.21.15.104:0/2981340722 >> v2:172.21.15.104:3300/0 conn(0x7ffa640d0860 0x7ffa640cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:53.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa5bfff700 1 -- 172.21.15.104:0/2981340722 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845496077 0 0) 0x7ffa64145d90 con 0x7ffa640ccbd0 2023-07-14T11:56:53.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa69091700 1 -- 172.21.15.104:0/2981340722 >> v2:172.21.15.104:3300/0 conn(0x7ffa640d0860 msgr2=0x7ffa640cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:53.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa69091700 1 --2- 172.21.15.104:0/2981340722 >> v2:172.21.15.104:3300/0 conn(0x7ffa640d0860 0x7ffa640cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa69091700 1 -- 172.21.15.104:0/2981340722 >> v1:172.21.15.121:6789/0 conn(0x7ffa640ccbd0 legacy=0x7ffa640d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:53.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa69091700 1 -- 172.21.15.104:0/2981340722 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa640cdc10 con 0x7ffa641459c0 2023-07-14T11:56:53.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa69091700 1 --2- 172.21.15.104:0/2981340722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa641459c0 0x7ffa640ce6f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009840 tx=0x7ffa60009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1957bf56ba472fcc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:53.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa5bfff700 1 -- 172.21.15.104:0/2981340722 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60005e50 con 0x7ffa641459c0 2023-07-14T11:56:53.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa5bfff700 1 -- 172.21.15.104:0/2981340722 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa60006030 con 0x7ffa641459c0 2023-07-14T11:56:53.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa5bfff700 1 -- 172.21.15.104:0/2981340722 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa600063c0 con 0x7ffa641459c0 2023-07-14T11:56:53.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa6a093700 1 -- 172.21.15.104:0/2981340722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa641459c0 msgr2=0x7ffa640ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:53.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa6a093700 1 --2- 172.21.15.104:0/2981340722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa641459c0 0x7ffa640ce6f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009840 tx=0x7ffa60009b50 comp rx=0 tx=0).stop 2023-07-14T11:56:53.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa6a093700 1 -- 172.21.15.104:0/2981340722 shutdown_connections 2023-07-14T11:56:53.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa6a093700 1 --2- 172.21.15.104:0/2981340722 >> v2:172.21.15.104:3300/0 conn(0x7ffa640d0860 0x7ffa640cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa6a093700 1 --2- 172.21.15.104:0/2981340722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa641459c0 0x7ffa640ce6f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa6a093700 1 -- 172.21.15.104:0/2981340722 >> 172.21.15.104:0/2981340722 conn(0x7ffa64060c40 msgr2=0x7ffa6409b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:53.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa6a093700 1 -- 172.21.15.104:0/2981340722 shutdown_connections 2023-07-14T11:56:53.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.160+0000 7ffa6a093700 1 -- 172.21.15.104:0/2981340722 wait complete. 2023-07-14T11:56:53.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa6a093700 1 Processor -- start 2023-07-14T11:56:53.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa6a093700 1 -- start start 2023-07-14T11:56:53.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa6a093700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa640ccbd0 0x7ffa641598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:53.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa6a093700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa640d0860 0x7ffa641e36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:53.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa6a093700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa641459c0 0x7ffa640b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:53.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa6a093700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa640c73f0 con 0x7ffa640ccbd0 2023-07-14T11:56:53.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa6a093700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa640c76d0 con 0x7ffa640d0860 2023-07-14T11:56:53.153 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa6a093700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa640c79b0 con 0x7ffa641459c0 2023-07-14T11:56:53.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa69091700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa640ccbd0 0x7ffa641598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:53.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa69091700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa640ccbd0 0x7ffa641598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59696/0 (socket says 172.21.15.104:59696) 2023-07-14T11:56:53.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa69091700 1 -- 172.21.15.104:0/3860234365 learned_addr learned my addr 172.21.15.104:0/3860234365 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:53.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa69892700 1 --2- 172.21.15.104:0/3860234365 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa641459c0 0x7ffa640b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:53.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa68890700 1 --2- 172.21.15.104:0/3860234365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa640d0860 0x7ffa641e36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:53.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa69091700 1 -- 172.21.15.104:0/3860234365 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa641459c0 msgr2=0x7ffa640b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:53.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa69091700 1 --2- 172.21.15.104:0/3860234365 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa641459c0 0x7ffa640b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa69091700 1 -- 172.21.15.104:0/3860234365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa640d0860 msgr2=0x7ffa641e36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:53.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa69091700 1 --2- 172.21.15.104:0/3860234365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa640d0860 0x7ffa641e36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.164+0000 7ffa69091700 1 -- 172.21.15.104:0/3860234365 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa60009450 con 0x7ffa640ccbd0 2023-07-14T11:56:53.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.168+0000 7ffa69091700 1 --2- 172.21.15.104:0/3860234365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa640ccbd0 0x7ffa641598b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ffa60000c00 tx=0x7ffa60004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:53.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.168+0000 7ffa59ffb700 1 -- 172.21.15.104:0/3860234365 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa600036c0 con 0x7ffa640ccbd0 2023-07-14T11:56:53.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.168+0000 7ffa59ffb700 1 -- 172.21.15.104:0/3860234365 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa60020070 con 0x7ffa640ccbd0 2023-07-14T11:56:53.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.168+0000 7ffa6a093700 1 -- 172.21.15.104:0/3860234365 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa640c7c90 con 0x7ffa640ccbd0 2023-07-14T11:56:53.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.168+0000 7ffa59ffb700 1 -- 172.21.15.104:0/3860234365 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60003a60 con 0x7ffa640ccbd0 2023-07-14T11:56:53.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.168+0000 7ffa6a093700 1 -- 172.21.15.104:0/3860234365 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa64159df0 con 0x7ffa640ccbd0 2023-07-14T11:56:53.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.168+0000 7ffa6a093700 1 -- 172.21.15.104:0/3860234365 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6404e500 con 0x7ffa640ccbd0 2023-07-14T11:56:53.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.172+0000 7ffa59ffb700 1 -- 172.21.15.104:0/3860234365 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ffa60022030 con 0x7ffa640ccbd0 2023-07-14T11:56:53.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.172+0000 7ffa59ffb700 1 --2- 172.21.15.104:0/3860234365 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa480662f0 0x7ffa480687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:53.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.172+0000 7ffa59ffb700 1 -- 172.21.15.104:0/3860234365 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ffa60085430 con 0x7ffa640ccbd0 2023-07-14T11:56:53.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.172+0000 7ffa68890700 1 --2- 172.21.15.104:0/3860234365 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa480662f0 0x7ffa480687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:53.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.172+0000 7ffa68890700 1 --2- 172.21.15.104:0/3860234365 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa480662f0 0x7ffa480687a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffa640b3cb0 tx=0x7ffa54007e00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:53.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.180+0000 7ffa59ffb700 1 -- 172.21.15.104:0/3860234365 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ffa60052880 con 0x7ffa640ccbd0 2023-07-14T11:56:53.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.412+0000 7ffa6a093700 1 -- 172.21.15.104:0/3860234365 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ffa64072a10 con 0x7ffa640ccbd0 2023-07-14T11:56:53.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.420+0000 7ffa59ffb700 1 -- 172.21.15.104:0/3860234365 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v20) v1 ==== 184+0+141 (secure 0 0 0) 0x7ffa60055ea0 con 0x7ffa640ccbd0 2023-07-14T11:56:53.405 INFO:teuthology.orchestra.run.smithi104.stdout:[client.0] 2023-07-14T11:56:53.405 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDrNrFkbrzjERAA+9BBzuauwUdhALUcDtd80A== 2023-07-14T11:56:53.405 INFO:teuthology.orchestra.run.smithi104.stdout: caps mds = "allow" 2023-07-14T11:56:53.405 INFO:teuthology.orchestra.run.smithi104.stdout: caps mon = "allow r" 2023-07-14T11:56:53.406 INFO:teuthology.orchestra.run.smithi104.stdout: caps osd = "allow rw pool=cephfs_data" 2023-07-14T11:56:53.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.424+0000 7ffa6a093700 1 -- 172.21.15.104:0/3860234365 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa480662f0 msgr2=0x7ffa480687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:53.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.424+0000 7ffa6a093700 1 --2- 172.21.15.104:0/3860234365 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa480662f0 0x7ffa480687a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffa640b3cb0 tx=0x7ffa54007e00 comp rx=0 tx=0).stop 2023-07-14T11:56:53.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.424+0000 7ffa6a093700 1 -- 172.21.15.104:0/3860234365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa640ccbd0 msgr2=0x7ffa641598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:53.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.424+0000 7ffa6a093700 1 --2- 172.21.15.104:0/3860234365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa640ccbd0 0x7ffa641598b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ffa60000c00 tx=0x7ffa60004770 comp rx=0 tx=0).stop 2023-07-14T11:56:53.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.424+0000 7ffa6a093700 1 -- 172.21.15.104:0/3860234365 shutdown_connections 2023-07-14T11:56:53.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.424+0000 7ffa6a093700 1 --2- 172.21.15.104:0/3860234365 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ffa480662f0 0x7ffa480687a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.424+0000 7ffa6a093700 1 --2- 172.21.15.104:0/3860234365 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa641459c0 0x7ffa640b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.424+0000 7ffa6a093700 1 --2- 172.21.15.104:0/3860234365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa640d0860 0x7ffa641e36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.424+0000 7ffa6a093700 1 --2- 172.21.15.104:0/3860234365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa640ccbd0 0x7ffa641598b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.424+0000 7ffa6a093700 1 -- 172.21.15.104:0/3860234365 >> 172.21.15.104:0/3860234365 conn(0x7ffa64060c40 msgr2=0x7ffa64072300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:53.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.424+0000 7ffa6a093700 1 -- 172.21.15.104:0/3860234365 shutdown_connections 2023-07-14T11:56:53.410 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.424+0000 7ffa6a093700 1 -- 172.21.15.104:0/3860234365 wait complete. 2023-07-14T11:56:53.410 INFO:teuthology.orchestra.run.smithi104.stderr:updated caps for client.0 2023-07-14T11:56:53.429 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-14T11:56:53.430 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-14T11:56:53.473 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-14T11:56:53.519 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-14T11:56:53.520 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:56:53.520 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:56:53.540 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:56:53.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.772+0000 7ff67359e700 1 Processor -- start 2023-07-14T11:56:53.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.772+0000 7ff67359e700 1 -- start start 2023-07-14T11:56:53.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff67359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6740c9d90 0x7ff6740ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:53.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff67359e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff674159f20 0x7ff6740caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:53.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff67359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6740b0ef0 con 0x7ff6740caf70 2023-07-14T11:56:53.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff67359e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6740ce9c0 con 0x7ff674159f20 2023-07-14T11:56:53.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff67359e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740cee50 con 0x7ff6740c9d90 2023-07-14T11:56:53.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff669d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff674159f20 0x7ff6740caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:53.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff669d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff674159f20 0x7ff6740caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59702/0 (socket says 172.21.15.104:59702) 2023-07-14T11:56:53.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff669d9b700 1 -- 172.21.15.104:0/1294973327 learned_addr learned my addr 172.21.15.104:0/1294973327 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:53.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff67259c700 1 --2- 172.21.15.104:0/1294973327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6740c9d90 0x7ff6740ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:53.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff669d9b700 1 -- 172.21.15.104:0/1294973327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6740c9d90 msgr2=0x7ff6740ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:53.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff669d9b700 1 --2- 172.21.15.104:0/1294973327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6740c9d90 0x7ff6740ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff669d9b700 1 -- 172.21.15.104:0/1294973327 >> v1:172.21.15.121:6789/0 conn(0x7ff6740caf70 legacy=0x7ff6740c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:53.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff669d9b700 1 -- 172.21.15.104:0/1294973327 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740ca6a0 con 0x7ff674159f20 2023-07-14T11:56:53.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff669d9b700 1 --2- 172.21.15.104:0/1294973327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff674159f20 0x7ff6740caa30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009910 tx=0x7ff65c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc42e3e8c5f93cb7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:53.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff671d9b700 1 -- 172.21.15.104:0/1294973327 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c00ba60 con 0x7ff674159f20 2023-07-14T11:56:53.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff671d9b700 1 -- 172.21.15.104:0/1294973327 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65c00bbc0 con 0x7ff674159f20 2023-07-14T11:56:53.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff671d9b700 1 -- 172.21.15.104:0/1294973327 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c009e60 con 0x7ff674159f20 2023-07-14T11:56:53.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff67359e700 1 -- 172.21.15.104:0/1294973327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff674159f20 msgr2=0x7ff6740caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:53.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.776+0000 7ff67359e700 1 --2- 172.21.15.104:0/1294973327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff674159f20 0x7ff6740caa30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009910 tx=0x7ff65c009c20 comp rx=0 tx=0).stop 2023-07-14T11:56:53.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.780+0000 7ff67359e700 1 -- 172.21.15.104:0/1294973327 shutdown_connections 2023-07-14T11:56:53.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.780+0000 7ff67359e700 1 --2- 172.21.15.104:0/1294973327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff674159f20 0x7ff6740caa30 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.780+0000 7ff67359e700 1 --2- 172.21.15.104:0/1294973327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6740c9d90 0x7ff6740ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.780+0000 7ff67359e700 1 -- 172.21.15.104:0/1294973327 >> 172.21.15.104:0/1294973327 conn(0x7ff67406e960 msgr2=0x7ff67406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:53.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.780+0000 7ff67359e700 1 -- 172.21.15.104:0/1294973327 shutdown_connections 2023-07-14T11:56:53.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.780+0000 7ff67359e700 1 -- 172.21.15.104:0/1294973327 wait complete. 2023-07-14T11:56:53.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff67359e700 1 Processor -- start 2023-07-14T11:56:53.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff67359e700 1 -- start start 2023-07-14T11:56:53.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff67359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6740c9d90 0x7ff67414f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:53.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff67359e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6740caf70 0x7ff6740c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:53.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff67359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff674159f20 0x7ff674157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:53.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff67359e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740c3370 con 0x7ff6740caf70 2023-07-14T11:56:53.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff67359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740c3650 con 0x7ff674159f20 2023-07-14T11:56:53.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff67359e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740c3930 con 0x7ff6740c9d90 2023-07-14T11:56:53.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff669d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6740caf70 0x7ff6740c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:53.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff669d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6740caf70 0x7ff6740c4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59710/0 (socket says 172.21.15.104:59710) 2023-07-14T11:56:53.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff669d9b700 1 -- 172.21.15.104:0/437619613 learned_addr learned my addr 172.21.15.104:0/437619613 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:53.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff67259c700 1 --2- 172.21.15.104:0/437619613 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6740c9d90 0x7ff67414f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:53.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff672d9d700 1 --2- 172.21.15.104:0/437619613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff674159f20 0x7ff674157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:53.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff669d9b700 1 -- 172.21.15.104:0/437619613 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6740c9d90 msgr2=0x7ff67414f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:53.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff669d9b700 1 --2- 172.21.15.104:0/437619613 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6740c9d90 0x7ff67414f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff669d9b700 1 -- 172.21.15.104:0/437619613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff674159f20 msgr2=0x7ff674157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:53.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff669d9b700 1 --2- 172.21.15.104:0/437619613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff674159f20 0x7ff674157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.784+0000 7ff669d9b700 1 -- 172.21.15.104:0/437619613 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c009570 con 0x7ff6740caf70 2023-07-14T11:56:53.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.788+0000 7ff669d9b700 1 --2- 172.21.15.104:0/437619613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6740caf70 0x7ff6740c4410 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff67406a6f0 tx=0x7ff65c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:53.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.788+0000 7ff66b7fe700 1 -- 172.21.15.104:0/437619613 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c004610 con 0x7ff6740caf70 2023-07-14T11:56:53.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.788+0000 7ff66b7fe700 1 -- 172.21.15.104:0/437619613 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65c01f070 con 0x7ff6740caf70 2023-07-14T11:56:53.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.788+0000 7ff66b7fe700 1 -- 172.21.15.104:0/437619613 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c003900 con 0x7ff6740caf70 2023-07-14T11:56:53.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.788+0000 7ff67359e700 1 -- 172.21.15.104:0/437619613 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6740c3c10 con 0x7ff6740caf70 2023-07-14T11:56:53.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.788+0000 7ff67359e700 1 -- 172.21.15.104:0/437619613 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67414f600 con 0x7ff6740caf70 2023-07-14T11:56:53.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.788+0000 7ff67359e700 1 -- 172.21.15.104:0/437619613 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6740c4130 con 0x7ff6740caf70 2023-07-14T11:56:53.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.792+0000 7ff66b7fe700 1 -- 172.21.15.104:0/437619613 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff65c003a60 con 0x7ff6740caf70 2023-07-14T11:56:53.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.792+0000 7ff66b7fe700 1 --2- 172.21.15.104:0/437619613 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6540662f0 0x7ff6540687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:53.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.792+0000 7ff66b7fe700 1 -- 172.21.15.104:0/437619613 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff65c021070 con 0x7ff6740caf70 2023-07-14T11:56:53.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.792+0000 7ff67259c700 1 --2- 172.21.15.104:0/437619613 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6540662f0 0x7ff6540687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:53.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.792+0000 7ff67259c700 1 --2- 172.21.15.104:0/437619613 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6540662f0 0x7ff6540687a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff66400b9d0 tx=0x7ff664008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:53.781 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.796+0000 7ff66b7fe700 1 -- 172.21.15.104:0/437619613 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff6740c4130 con 0x7ff6740caf70 2023-07-14T11:56:53.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.964+0000 7ff67359e700 1 -- 172.21.15.104:0/437619613 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6740c4130 con 0x7ff6740caf70 2023-07-14T11:56:53.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.968+0000 7ff66b7fe700 1 -- 172.21.15.104:0/437619613 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+18534 (secure 0 0 0) 0x7ff6740c4130 con 0x7ff6740caf70 2023-07-14T11:56:53.954 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:53.955 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":59,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:56:12.692633+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:56:08.755551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-14T11:56:10.132688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:56:53.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.972+0000 7ff67359e700 1 -- 172.21.15.104:0/437619613 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6540662f0 msgr2=0x7ff6540687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:53.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.972+0000 7ff67359e700 1 --2- 172.21.15.104:0/437619613 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6540662f0 0x7ff6540687a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff66400b9d0 tx=0x7ff664008040 comp rx=0 tx=0).stop 2023-07-14T11:56:53.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.972+0000 7ff67359e700 1 -- 172.21.15.104:0/437619613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6740caf70 msgr2=0x7ff6740c4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:53.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.972+0000 7ff67359e700 1 --2- 172.21.15.104:0/437619613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6740caf70 0x7ff6740c4410 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff67406a6f0 tx=0x7ff65c004770 comp rx=0 tx=0).stop 2023-07-14T11:56:53.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.972+0000 7ff67359e700 1 -- 172.21.15.104:0/437619613 shutdown_connections 2023-07-14T11:56:53.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.972+0000 7ff67359e700 1 --2- 172.21.15.104:0/437619613 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff6540662f0 0x7ff6540687a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.972+0000 7ff67359e700 1 --2- 172.21.15.104:0/437619613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6740caf70 0x7ff6740c4410 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.972+0000 7ff67359e700 1 --2- 172.21.15.104:0/437619613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff674159f20 0x7ff674157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.972+0000 7ff67359e700 1 --2- 172.21.15.104:0/437619613 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6740c9d90 0x7ff67414f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:53.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.972+0000 7ff67359e700 1 -- 172.21.15.104:0/437619613 >> 172.21.15.104:0/437619613 conn(0x7ff67406e960 msgr2=0x7ff67415b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:53.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.972+0000 7ff67359e700 1 -- 172.21.15.104:0/437619613 shutdown_connections 2023-07-14T11:56:53.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:53.972+0000 7ff67359e700 1 -- 172.21.15.104:0/437619613 wait complete. 2023-07-14T11:56:53.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:56:08.755551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:56:10.132688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:56:53.978 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:56:54.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.168+0000 7f16fd0b7700 1 Processor -- start 2023-07-14T11:56:54.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.168+0000 7f16fd0b7700 1 -- start start 2023-07-14T11:56:54.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.168+0000 7f16fd0b7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16f8160000 0x7f16f806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.168+0000 7f16fd0b7700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f16f80cd7f0 0x7f16f80c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.168+0000 7f16fd0b7700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f80cdfb0 con 0x7f16f81603d0 2023-07-14T11:56:54.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.168+0000 7f16fd0b7700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f8070e40 con 0x7f16f80cd7f0 2023-07-14T11:56:54.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.168+0000 7f16fd0b7700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f80cdcd0 con 0x7f16f8160000 2023-07-14T11:56:54.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.168+0000 7f16ef7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f16f81603d0 0x7f16f80c9e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:37510/0 (socket says 172.21.15.104:37510) 2023-07-14T11:56:54.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.168+0000 7f16ef7fe700 1 -- 172.21.15.104:0/1215231351 learned_addr learned my addr 172.21.15.104:0/1215231351 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:54.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.168+0000 7f16fc8b6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f16f80cd7f0 0x7f16f80c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.168+0000 7f16f7fff700 1 --2- 172.21.15.104:0/1215231351 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16f8160000 0x7f16f806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16f77fe700 1 -- 172.21.15.104:0/1215231351 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594230570 0 0) 0x7f16e0001110 con 0x7f16f81603d0 2023-07-14T11:56:54.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16f77fe700 1 -- 172.21.15.104:0/1215231351 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16e40035d0 con 0x7f16f81603d0 2023-07-14T11:56:54.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16fc8b6700 1 -- 172.21.15.104:0/1215231351 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16f8160000 msgr2=0x7f16f806dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:54.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16fc8b6700 1 --2- 172.21.15.104:0/1215231351 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16f8160000 0x7f16f806dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16fc8b6700 1 -- 172.21.15.104:0/1215231351 >> v1:172.21.15.121:6789/0 conn(0x7f16f81603d0 legacy=0x7f16f80c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16fc8b6700 1 -- 172.21.15.104:0/1215231351 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f8070780 con 0x7f16f80cd7f0 2023-07-14T11:56:54.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16f7fff700 1 --2- 172.21.15.104:0/1215231351 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16f8160000 0x7f16f806dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:54.157 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16fc8b6700 1 --2- 172.21.15.104:0/1215231351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f80cd7f0 0x7f16f80c0d50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009d90 tx=0x7f16f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c18978354a5cda3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:54.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16f77fe700 1 -- 172.21.15.104:0/1215231351 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f000e070 con 0x7f16f80cd7f0 2023-07-14T11:56:54.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16f77fe700 1 -- 172.21.15.104:0/1215231351 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f000bb20 con 0x7f16f80cd7f0 2023-07-14T11:56:54.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16f77fe700 1 -- 172.21.15.104:0/1215231351 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0012430 con 0x7f16f80cd7f0 2023-07-14T11:56:54.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16fd0b7700 1 -- 172.21.15.104:0/1215231351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f80cd7f0 msgr2=0x7f16f80c0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16fd0b7700 1 --2- 172.21.15.104:0/1215231351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f80cd7f0 0x7f16f80c0d50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009d90 tx=0x7f16f000b330 comp rx=0 tx=0).stop 2023-07-14T11:56:54.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16fd0b7700 1 -- 172.21.15.104:0/1215231351 shutdown_connections 2023-07-14T11:56:54.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16fd0b7700 1 --2- 172.21.15.104:0/1215231351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f80cd7f0 0x7f16f80c0d50 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16fd0b7700 1 --2- 172.21.15.104:0/1215231351 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16f8160000 0x7f16f806dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16fd0b7700 1 -- 172.21.15.104:0/1215231351 >> 172.21.15.104:0/1215231351 conn(0x7f16f80afc60 msgr2=0x7f16f80b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:54.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16fd0b7700 1 -- 172.21.15.104:0/1215231351 shutdown_connections 2023-07-14T11:56:54.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.172+0000 7f16fd0b7700 1 -- 172.21.15.104:0/1215231351 wait complete. 2023-07-14T11:56:54.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.176+0000 7f16fd0b7700 1 Processor -- start 2023-07-14T11:56:54.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.176+0000 7f16fd0b7700 1 -- start start 2023-07-14T11:56:54.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.176+0000 7f16fd0b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f80cd7f0 0x7f16f81543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.176+0000 7f16fd0b7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f8160000 0x7f16f80b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.176+0000 7f16fd0b7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16f81603d0 0x7f16f80b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.176+0000 7f16fd0b7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f80b1020 con 0x7f16f8160000 2023-07-14T11:56:54.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.176+0000 7f16fd0b7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f81548f0 con 0x7f16f80cd7f0 2023-07-14T11:56:54.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.176+0000 7f16fd0b7700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f8162d20 con 0x7f16f81603d0 2023-07-14T11:56:54.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.176+0000 7f16ef7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f8160000 0x7f16f80b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.180+0000 7f16ef7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f8160000 0x7f16f80b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59732/0 (socket says 172.21.15.104:59732) 2023-07-14T11:56:54.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.180+0000 7f16ef7fe700 1 -- 172.21.15.104:0/1497697168 learned_addr learned my addr 172.21.15.104:0/1497697168 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:54.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.180+0000 7f16f7fff700 1 --2- 172.21.15.104:0/1497697168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f80cd7f0 0x7f16f81543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.180+0000 7f16fc8b6700 1 --2- 172.21.15.104:0/1497697168 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16f81603d0 0x7f16f80b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.180+0000 7f16ef7fe700 1 -- 172.21.15.104:0/1497697168 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16f81603d0 msgr2=0x7f16f80b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.180+0000 7f16ef7fe700 1 --2- 172.21.15.104:0/1497697168 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16f81603d0 0x7f16f80b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.180+0000 7f16ef7fe700 1 -- 172.21.15.104:0/1497697168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f80cd7f0 msgr2=0x7f16f81543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.180+0000 7f16ef7fe700 1 --2- 172.21.15.104:0/1497697168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f80cd7f0 0x7f16f81543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.180+0000 7f16ef7fe700 1 -- 172.21.15.104:0/1497697168 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e80094b0 con 0x7f16f8160000 2023-07-14T11:56:54.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.180+0000 7f16ef7fe700 1 --2- 172.21.15.104:0/1497697168 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f8160000 0x7f16f80b3490 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f16e0001270 tx=0x7f16e000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:54.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.180+0000 7f16f57fa700 1 -- 172.21.15.104:0/1497697168 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e0005d90 con 0x7f16f8160000 2023-07-14T11:56:54.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.180+0000 7f16f57fa700 1 -- 172.21.15.104:0/1497697168 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e00070d0 con 0x7f16f8160000 2023-07-14T11:56:54.166 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.180+0000 7f16fd0b7700 1 -- 172.21.15.104:0/1497697168 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f0009a70 con 0x7f16f8160000 2023-07-14T11:56:54.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.180+0000 7f16f57fa700 1 -- 172.21.15.104:0/1497697168 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e0007480 con 0x7f16f8160000 2023-07-14T11:56:54.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.180+0000 7f16fd0b7700 1 -- 172.21.15.104:0/1497697168 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f80b15a0 con 0x7f16f8160000 2023-07-14T11:56:54.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.184+0000 7f16fd0b7700 1 -- 172.21.15.104:0/1497697168 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f804e500 con 0x7f16f8160000 2023-07-14T11:56:54.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.184+0000 7f16f57fa700 1 -- 172.21.15.104:0/1497697168 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f16e00075e0 con 0x7f16f8160000 2023-07-14T11:56:54.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.184+0000 7f16f57fa700 1 --2- 172.21.15.104:0/1497697168 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16d80662f0 0x7f16d80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.184+0000 7f16f57fa700 1 -- 172.21.15.104:0/1497697168 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f16e00830d0 con 0x7f16f8160000 2023-07-14T11:56:54.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.188+0000 7f16f7fff700 1 --2- 172.21.15.104:0/1497697168 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16d80662f0 0x7f16d80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.188+0000 7f16f7fff700 1 --2- 172.21.15.104:0/1497697168 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16d80662f0 0x7f16d80687a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f16f815b860 tx=0x7f16e8009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:54.175 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.192+0000 7f16f57fa700 1 -- 172.21.15.104:0/1497697168 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f16e0050400 con 0x7f16f8160000 2023-07-14T11:56:54.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.360+0000 7f16fd0b7700 1 -- 172.21.15.104:0/1497697168 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16f80cdfb0 con 0x7f16f8160000 2023-07-14T11:56:54.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.360+0000 7f16f57fa700 1 -- 172.21.15.104:0/1497697168 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16f80cdfb0 con 0x7f16f8160000 2023-07-14T11:56:54.346 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:56:54.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.364+0000 7f16fd0b7700 1 -- 172.21.15.104:0/1497697168 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16d80662f0 msgr2=0x7f16d80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.364+0000 7f16fd0b7700 1 --2- 172.21.15.104:0/1497697168 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16d80662f0 0x7f16d80687a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f16f815b860 tx=0x7f16e8009240 comp rx=0 tx=0).stop 2023-07-14T11:56:54.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.364+0000 7f16fd0b7700 1 -- 172.21.15.104:0/1497697168 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f8160000 msgr2=0x7f16f80b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.364+0000 7f16fd0b7700 1 --2- 172.21.15.104:0/1497697168 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f8160000 0x7f16f80b3490 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f16e0001270 tx=0x7f16e000c7d0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.364+0000 7f16fd0b7700 1 -- 172.21.15.104:0/1497697168 shutdown_connections 2023-07-14T11:56:54.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.364+0000 7f16fd0b7700 1 --2- 172.21.15.104:0/1497697168 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f16d80662f0 0x7f16d80687a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.364+0000 7f16fd0b7700 1 --2- 172.21.15.104:0/1497697168 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f8160000 0x7f16f80b3490 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.364+0000 7f16fd0b7700 1 --2- 172.21.15.104:0/1497697168 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16f81603d0 0x7f16f80b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.364+0000 7f16fd0b7700 1 --2- 172.21.15.104:0/1497697168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f80cd7f0 0x7f16f81543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.364+0000 7f16fd0b7700 1 -- 172.21.15.104:0/1497697168 >> 172.21.15.104:0/1497697168 conn(0x7f16f80afc60 msgr2=0x7f16f8164500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:54.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.364+0000 7f16fd0b7700 1 -- 172.21.15.104:0/1497697168 shutdown_connections 2023-07-14T11:56:54.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.364+0000 7f16fd0b7700 1 -- 172.21.15.104:0/1497697168 wait complete. 2023-07-14T11:56:54.371 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:56:54.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.556+0000 7fcb46ddc700 1 Processor -- start 2023-07-14T11:56:54.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.556+0000 7fcb46ddc700 1 -- start start 2023-07-14T11:56:54.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb46ddc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcb40160000 0x7fcb4006dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb46ddc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400cd7f0 0x7fcb400c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb46ddc700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb400cdfb0 con 0x7fcb401603d0 2023-07-14T11:56:54.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb46ddc700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb40070e40 con 0x7fcb40160000 2023-07-14T11:56:54.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb46ddc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb400cdcd0 con 0x7fcb400cd7f0 2023-07-14T11:56:54.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb465db700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400cd7f0 0x7fcb400c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb465db700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400cd7f0 0x7fcb400c0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:57510/0 (socket says 172.21.15.104:57510) 2023-07-14T11:56:54.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb465db700 1 -- 172.21.15.104:0/3550827497 learned_addr learned my addr 172.21.15.104:0/3550827497 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:54.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb45dda700 1 --2- 172.21.15.104:0/3550827497 >> v2:172.21.15.121:3300/0 conn(0x7fcb40160000 0x7fcb4006dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb465db700 1 -- 172.21.15.104:0/3550827497 >> v2:172.21.15.121:3300/0 conn(0x7fcb40160000 msgr2=0x7fcb4006dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:54.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb465db700 1 --2- 172.21.15.104:0/3550827497 >> v2:172.21.15.121:3300/0 conn(0x7fcb40160000 0x7fcb4006dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb465db700 1 -- 172.21.15.104:0/3550827497 >> v1:172.21.15.121:6789/0 conn(0x7fcb401603d0 legacy=0x7fcb400c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb465db700 1 -- 172.21.15.104:0/3550827497 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb40070780 con 0x7fcb400cd7f0 2023-07-14T11:56:54.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb465db700 1 --2- 172.21.15.104:0/3550827497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400cd7f0 0x7fcb400c0d50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009e50 tx=0x7fcb3800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d8d21b4430d88d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:54.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb44dd8700 1 -- 172.21.15.104:0/3550827497 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3800e070 con 0x7fcb400cd7f0 2023-07-14T11:56:54.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb44dd8700 1 -- 172.21.15.104:0/3550827497 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3800b910 con 0x7fcb400cd7f0 2023-07-14T11:56:54.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.560+0000 7fcb44dd8700 1 -- 172.21.15.104:0/3550827497 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3800bcc0 con 0x7fcb400cd7f0 2023-07-14T11:56:54.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.564+0000 7fcb46ddc700 1 -- 172.21.15.104:0/3550827497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400cd7f0 msgr2=0x7fcb400c0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.564+0000 7fcb46ddc700 1 --2- 172.21.15.104:0/3550827497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400cd7f0 0x7fcb400c0d50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009e50 tx=0x7fcb3800b330 comp rx=0 tx=0).stop 2023-07-14T11:56:54.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.564+0000 7fcb46ddc700 1 -- 172.21.15.104:0/3550827497 shutdown_connections 2023-07-14T11:56:54.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.564+0000 7fcb46ddc700 1 --2- 172.21.15.104:0/3550827497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400cd7f0 0x7fcb400c0d50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.564+0000 7fcb46ddc700 1 --2- 172.21.15.104:0/3550827497 >> v2:172.21.15.121:3300/0 conn(0x7fcb40160000 0x7fcb4006dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.564+0000 7fcb46ddc700 1 -- 172.21.15.104:0/3550827497 >> 172.21.15.104:0/3550827497 conn(0x7fcb400afc60 msgr2=0x7fcb400b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:54.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.564+0000 7fcb46ddc700 1 -- 172.21.15.104:0/3550827497 shutdown_connections 2023-07-14T11:56:54.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.564+0000 7fcb46ddc700 1 -- 172.21.15.104:0/3550827497 wait complete. 2023-07-14T11:56:54.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb46ddc700 1 Processor -- start 2023-07-14T11:56:54.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb46ddc700 1 -- start start 2023-07-14T11:56:54.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb46ddc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400cd7f0 0x7fcb401544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb46ddc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb40160000 0x7fcb400b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb46ddc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb401603d0 0x7fcb400b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb46ddc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb400b0f80 con 0x7fcb40160000 2023-07-14T11:56:54.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb46ddc700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb401549f0 con 0x7fcb401603d0 2023-07-14T11:56:54.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb46ddc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb400c7650 con 0x7fcb400cd7f0 2023-07-14T11:56:54.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb455d9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb40160000 0x7fcb400b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb455d9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb40160000 0x7fcb400b3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59738/0 (socket says 172.21.15.104:59738) 2023-07-14T11:56:54.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb45dda700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400cd7f0 0x7fcb401544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb455d9700 1 -- 172.21.15.104:0/2670219788 learned_addr learned my addr 172.21.15.104:0/2670219788 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:54.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb465db700 1 --2- 172.21.15.104:0/2670219788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb401603d0 0x7fcb400b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb45dda700 1 -- 172.21.15.104:0/2670219788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb401603d0 msgr2=0x7fcb400b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb45dda700 1 --2- 172.21.15.104:0/2670219788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb401603d0 0x7fcb400b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb45dda700 1 -- 172.21.15.104:0/2670219788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb40160000 msgr2=0x7fcb400b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb45dda700 1 --2- 172.21.15.104:0/2670219788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb40160000 0x7fcb400b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb45dda700 1 -- 172.21.15.104:0/2670219788 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb38009b30 con 0x7fcb400cd7f0 2023-07-14T11:56:54.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb45dda700 1 --2- 172.21.15.104:0/2670219788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400cd7f0 0x7fcb401544b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00e930 tx=0x7fcb3c00ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:54.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb367fc700 1 -- 172.21.15.104:0/2670219788 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c013680 con 0x7fcb400cd7f0 2023-07-14T11:56:54.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb367fc700 1 -- 172.21.15.104:0/2670219788 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3c0137e0 con 0x7fcb400cd7f0 2023-07-14T11:56:54.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb46ddc700 1 -- 172.21.15.104:0/2670219788 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb400c7930 con 0x7fcb400cd7f0 2023-07-14T11:56:54.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb367fc700 1 -- 172.21.15.104:0/2670219788 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c013b50 con 0x7fcb400cd7f0 2023-07-14T11:56:54.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.568+0000 7fcb46ddc700 1 -- 172.21.15.104:0/2670219788 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb400b14b0 con 0x7fcb400cd7f0 2023-07-14T11:56:54.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.572+0000 7fcb46ddc700 1 -- 172.21.15.104:0/2670219788 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb400c7b70 con 0x7fcb400cd7f0 2023-07-14T11:56:54.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.572+0000 7fcb367fc700 1 -- 172.21.15.104:0/2670219788 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fcb3c013cb0 con 0x7fcb400cd7f0 2023-07-14T11:56:54.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.576+0000 7fcb367fc700 1 --2- 172.21.15.104:0/2670219788 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcb240662f0 0x7fcb240687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.576+0000 7fcb367fc700 1 -- 172.21.15.104:0/2670219788 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcb3c083e60 con 0x7fcb400cd7f0 2023-07-14T11:56:54.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.576+0000 7fcb455d9700 1 --2- 172.21.15.104:0/2670219788 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcb240662f0 0x7fcb240687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.576+0000 7fcb455d9700 1 --2- 172.21.15.104:0/2670219788 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcb240662f0 0x7fcb240687a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcb400b2be0 tx=0x7fcb30009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:54.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.580+0000 7fcb367fc700 1 -- 172.21.15.104:0/2670219788 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fcb3c051100 con 0x7fcb400cd7f0 2023-07-14T11:56:54.738 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.752+0000 7fcb46ddc700 1 -- 172.21.15.104:0/2670219788 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcb4004e500 con 0x7fcb400cd7f0 2023-07-14T11:56:54.740 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.756+0000 7fcb367fc700 1 -- 172.21.15.104:0/2670219788 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v59) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcb3c0170a0 con 0x7fcb400cd7f0 2023-07-14T11:56:54.741 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:56:54.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.760+0000 7fcb46ddc700 1 -- 172.21.15.104:0/2670219788 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcb240662f0 msgr2=0x7fcb240687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.760+0000 7fcb46ddc700 1 --2- 172.21.15.104:0/2670219788 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcb240662f0 0x7fcb240687a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcb400b2be0 tx=0x7fcb30009040 comp rx=0 tx=0).stop 2023-07-14T11:56:54.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.760+0000 7fcb46ddc700 1 -- 172.21.15.104:0/2670219788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400cd7f0 msgr2=0x7fcb401544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.760+0000 7fcb46ddc700 1 --2- 172.21.15.104:0/2670219788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400cd7f0 0x7fcb401544b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00e930 tx=0x7fcb3c00ec40 comp rx=0 tx=0).stop 2023-07-14T11:56:54.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.760+0000 7fcb46ddc700 1 -- 172.21.15.104:0/2670219788 shutdown_connections 2023-07-14T11:56:54.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.760+0000 7fcb46ddc700 1 --2- 172.21.15.104:0/2670219788 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fcb240662f0 0x7fcb240687a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.760+0000 7fcb46ddc700 1 --2- 172.21.15.104:0/2670219788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb40160000 0x7fcb400b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.760+0000 7fcb46ddc700 1 --2- 172.21.15.104:0/2670219788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb401603d0 0x7fcb400b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.760+0000 7fcb46ddc700 1 --2- 172.21.15.104:0/2670219788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcb400cd7f0 0x7fcb401544b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.760+0000 7fcb46ddc700 1 -- 172.21.15.104:0/2670219788 >> 172.21.15.104:0/2670219788 conn(0x7fcb400afc60 msgr2=0x7fcb400ce150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:54.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.760+0000 7fcb46ddc700 1 -- 172.21.15.104:0/2670219788 shutdown_connections 2023-07-14T11:56:54.746 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.760+0000 7fcb46ddc700 1 -- 172.21.15.104:0/2670219788 wait complete. 2023-07-14T11:56:54.766 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:56:54.939 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.956+0000 7f2123999700 1 Processor -- start 2023-07-14T11:56:54.940 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.956+0000 7f2123999700 1 -- start start 2023-07-14T11:56:54.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.956+0000 7f2123999700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f211c15df10 0x7f211c15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.956+0000 7f2123999700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f211c0c8950 0x7f211c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.956+0000 7f2123999700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f211c070a30 con 0x7f211c0afd80 2023-07-14T11:56:54.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.956+0000 7f2123999700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f211c0c92f0 con 0x7f211c0c8950 2023-07-14T11:56:54.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.956+0000 7f2123999700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c15e830 con 0x7f211c15df10 2023-07-14T11:56:54.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.956+0000 7f2120f34700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f211c0afd80 0x7f211c0711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:40894/0 (socket says 172.21.15.104:40894) 2023-07-14T11:56:54.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.956+0000 7f2120f34700 1 -- 172.21.15.104:0/3061047703 learned_addr learned my addr 172.21.15.104:0/3061047703 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:54.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.956+0000 7f2121f36700 1 --2- 172.21.15.104:0/3061047703 >> v2:172.21.15.121:3300/0 conn(0x7f211c0c8950 0x7f211c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.956+0000 7f2121735700 1 --2- 172.21.15.104:0/3061047703 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f211c15df10 0x7f211c15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2121f36700 1 -- 172.21.15.104:0/3061047703 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f211c15df10 msgr2=0x7f211c15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:54.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2121f36700 1 --2- 172.21.15.104:0/3061047703 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f211c15df10 0x7f211c15e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2121f36700 1 -- 172.21.15.104:0/3061047703 >> v1:172.21.15.104:6789/0 conn(0x7f211c0afd80 legacy=0x7f211c0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2121f36700 1 -- 172.21.15.104:0/3061047703 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c0b0210 con 0x7f211c0c8950 2023-07-14T11:56:54.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2113fff700 1 -- 172.21.15.104:0/3061047703 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268008047 0 0) 0x7f211c070a30 con 0x7f211c0afd80 2023-07-14T11:56:54.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2121f36700 1 --2- 172.21.15.104:0/3061047703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f211c0c8950 0x7f211c0c8db0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2114009d90 tx=0x7f211400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb0dcb58410e3035 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:54.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2113fff700 1 -- 172.21.15.104:0/3061047703 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211400e070 con 0x7f211c0c8950 2023-07-14T11:56:54.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2113fff700 1 -- 172.21.15.104:0/3061047703 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211400bb20 con 0x7f211c0c8950 2023-07-14T11:56:54.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2113fff700 1 -- 172.21.15.104:0/3061047703 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114012430 con 0x7f211c0c8950 2023-07-14T11:56:54.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2123999700 1 -- 172.21.15.104:0/3061047703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f211c0c8950 msgr2=0x7f211c0c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2123999700 1 --2- 172.21.15.104:0/3061047703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f211c0c8950 0x7f211c0c8db0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2114009d90 tx=0x7f211400b330 comp rx=0 tx=0).stop 2023-07-14T11:56:54.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2123999700 1 -- 172.21.15.104:0/3061047703 shutdown_connections 2023-07-14T11:56:54.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2123999700 1 --2- 172.21.15.104:0/3061047703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f211c0c8950 0x7f211c0c8db0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2123999700 1 --2- 172.21.15.104:0/3061047703 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f211c15df10 0x7f211c15e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2123999700 1 -- 172.21.15.104:0/3061047703 >> 172.21.15.104:0/3061047703 conn(0x7f211c0cad80 msgr2=0x7f211c0cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:54.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2123999700 1 -- 172.21.15.104:0/3061047703 shutdown_connections 2023-07-14T11:56:54.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.960+0000 7f2123999700 1 -- 172.21.15.104:0/3061047703 wait complete. 2023-07-14T11:56:54.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.964+0000 7f2123999700 1 Processor -- start 2023-07-14T11:56:54.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.964+0000 7f2123999700 1 -- start start 2023-07-14T11:56:54.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.964+0000 7f2123999700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f211c0afd80 0x7f211c0c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.964+0000 7f2123999700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f211c0c8950 0x7f211c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.964+0000 7f2123999700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f211c1f0c90 0x7f211c0c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.964+0000 7f2123999700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c0c65e0 con 0x7f211c0c8950 2023-07-14T11:56:54.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.964+0000 7f2123999700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c0b2080 con 0x7f211c0afd80 2023-07-14T11:56:54.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.964+0000 7f2123999700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c0cbed0 con 0x7f211c1f0c90 2023-07-14T11:56:54.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.964+0000 7f2120f34700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f211c0c8950 0x7f211c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.964+0000 7f2120f34700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f211c0c8950 0x7f211c0b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59748/0 (socket says 172.21.15.104:59748) 2023-07-14T11:56:54.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.964+0000 7f2120f34700 1 -- 172.21.15.104:0/2794320479 learned_addr learned my addr 172.21.15.104:0/2794320479 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:54.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.964+0000 7f2121f36700 1 --2- 172.21.15.104:0/2794320479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f211c1f0c90 0x7f211c0c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.964+0000 7f2121735700 1 --2- 172.21.15.104:0/2794320479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f211c0afd80 0x7f211c0c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.968+0000 7f2120f34700 1 -- 172.21.15.104:0/2794320479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f211c1f0c90 msgr2=0x7f211c0c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.968+0000 7f2120f34700 1 --2- 172.21.15.104:0/2794320479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f211c1f0c90 0x7f211c0c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.968+0000 7f2120f34700 1 -- 172.21.15.104:0/2794320479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f211c0afd80 msgr2=0x7f211c0c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:54.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.968+0000 7f2120f34700 1 --2- 172.21.15.104:0/2794320479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f211c0afd80 0x7f211c0c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:54.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.968+0000 7f2120f34700 1 -- 172.21.15.104:0/2794320479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2114009a70 con 0x7f211c0c8950 2023-07-14T11:56:54.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.968+0000 7f2121f36700 1 --2- 172.21.15.104:0/2794320479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f211c1f0c90 0x7f211c0c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:54.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.968+0000 7f2120f34700 1 --2- 172.21.15.104:0/2794320479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f211c0c8950 0x7f211c0b1b40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f210c00c5b0 tx=0x7f210c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:54.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.968+0000 7f2111ffb700 1 -- 172.21.15.104:0/2794320479 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c004ca0 con 0x7f211c0c8950 2023-07-14T11:56:54.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.968+0000 7f2111ffb700 1 -- 172.21.15.104:0/2794320479 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210c004e20 con 0x7f211c0c8950 2023-07-14T11:56:54.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.968+0000 7f2111ffb700 1 -- 172.21.15.104:0/2794320479 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c007310 con 0x7f211c0c8950 2023-07-14T11:56:54.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.968+0000 7f2123999700 1 -- 172.21.15.104:0/2794320479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c0b22b0 con 0x7f211c0c8950 2023-07-14T11:56:54.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.968+0000 7f2123999700 1 -- 172.21.15.104:0/2794320479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c0b2870 con 0x7f211c0c8950 2023-07-14T11:56:54.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.968+0000 7f2123999700 1 -- 172.21.15.104:0/2794320479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211c04dcb0 con 0x7f211c0c8950 2023-07-14T11:56:54.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.972+0000 7f2111ffb700 1 -- 172.21.15.104:0/2794320479 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f210c007470 con 0x7f211c0c8950 2023-07-14T11:56:54.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.972+0000 7f2111ffb700 1 --2- 172.21.15.104:0/2794320479 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2100066320 0x7f21000687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:54.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.972+0000 7f2111ffb700 1 -- 172.21.15.104:0/2794320479 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f210c083b10 con 0x7f211c0c8950 2023-07-14T11:56:54.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.972+0000 7f2121735700 1 --2- 172.21.15.104:0/2794320479 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2100066320 0x7f21000687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:54.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.972+0000 7f2121735700 1 --2- 172.21.15.104:0/2794320479 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2100066320 0x7f21000687d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2118009b50 tx=0x7f2118009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:54.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:54.976+0000 7f2111ffb700 1 -- 172.21.15.104:0/2794320479 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f210c050d20 con 0x7f211c0c8950 2023-07-14T11:56:55.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.152+0000 7f2123999700 1 -- 172.21.15.104:0/2794320479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f211c0cc010 con 0x7f211c0c8950 2023-07-14T11:56:55.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.152+0000 7f2111ffb700 1 -- 172.21.15.104:0/2794320479 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) v1 ==== 104+0+11 (secure 0 0 0) 0x7f210c007720 con 0x7f211c0c8950 2023-07-14T11:56:55.138 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:56:55.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.156+0000 7f2123999700 1 -- 172.21.15.104:0/2794320479 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2100066320 msgr2=0x7f21000687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:55.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.156+0000 7f2123999700 1 --2- 172.21.15.104:0/2794320479 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2100066320 0x7f21000687d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2118009b50 tx=0x7f2118009240 comp rx=0 tx=0).stop 2023-07-14T11:56:55.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.156+0000 7f2123999700 1 -- 172.21.15.104:0/2794320479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f211c0c8950 msgr2=0x7f211c0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:55.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.156+0000 7f2123999700 1 --2- 172.21.15.104:0/2794320479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f211c0c8950 0x7f211c0b1b40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f210c00c5b0 tx=0x7f210c00c8c0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.156+0000 7f2123999700 1 -- 172.21.15.104:0/2794320479 shutdown_connections 2023-07-14T11:56:55.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.156+0000 7f2123999700 1 --2- 172.21.15.104:0/2794320479 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f2100066320 0x7f21000687d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.156+0000 7f2123999700 1 --2- 172.21.15.104:0/2794320479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f211c0c8950 0x7f211c0b1b40 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.156+0000 7f2123999700 1 --2- 172.21.15.104:0/2794320479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f211c1f0c90 0x7f211c0c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.156+0000 7f2123999700 1 --2- 172.21.15.104:0/2794320479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f211c0afd80 0x7f211c0c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.156+0000 7f2123999700 1 -- 172.21.15.104:0/2794320479 >> 172.21.15.104:0/2794320479 conn(0x7f211c0cad80 msgr2=0x7f211c0cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:55.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.156+0000 7f2123999700 1 -- 172.21.15.104:0/2794320479 shutdown_connections 2023-07-14T11:56:55.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.156+0000 7f2123999700 1 -- 172.21.15.104:0/2794320479 wait complete. 2023-07-14T11:56:55.163 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:56:55.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.364+0000 7f3f065c1700 1 Processor -- start 2023-07-14T11:56:55.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3f065c1700 1 -- start start 2023-07-14T11:56:55.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3f065c1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3f00158420 0x7f3f000cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:55.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3f065c1700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3f0006d770 0x7f3f0006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:55.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3f065c1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f001587f0 con 0x7f3f000ccef0 2023-07-14T11:56:55.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3f065c1700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0015cc50 con 0x7f3f0006d770 2023-07-14T11:56:55.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3f065c1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0015bfd0 con 0x7f3f00158420 2023-07-14T11:56:55.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3f04b5e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3f0006d770 0x7f3f0006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:55.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3f04b5e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3f0006d770 0x7f3f0006e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59756/0 (socket says 172.21.15.104:59756) 2023-07-14T11:56:55.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3f04b5e700 1 -- 172.21.15.104:0/4205589971 learned_addr learned my addr 172.21.15.104:0/4205589971 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:55.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3efffff700 1 --2- 172.21.15.104:0/4205589971 >> v2:172.21.15.121:3300/0 conn(0x7f3f00158420 0x7f3f000cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:55.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3f04b5e700 1 -- 172.21.15.104:0/4205589971 >> v2:172.21.15.121:3300/0 conn(0x7f3f00158420 msgr2=0x7f3f000cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:55.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3f04b5e700 1 --2- 172.21.15.104:0/4205589971 >> v2:172.21.15.121:3300/0 conn(0x7f3f00158420 0x7f3f000cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3f04b5e700 1 -- 172.21.15.104:0/4205589971 >> v1:172.21.15.121:6789/0 conn(0x7f3f000ccef0 legacy=0x7f3f0006d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:55.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3f04b5e700 1 -- 172.21.15.104:0/4205589971 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0015c2b0 con 0x7f3f0006d770 2023-07-14T11:56:55.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3efffff700 1 --2- 172.21.15.104:0/4205589971 >> v2:172.21.15.121:3300/0 conn(0x7f3f00158420 0x7f3f000cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:55.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3f04b5e700 1 --2- 172.21.15.104:0/4205589971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f0006d770 0x7f3f0006e9f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3f000709b0 tx=0x7f3ef800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cf4c91961e5679 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:55.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.368+0000 7f3eff7fe700 1 -- 172.21.15.104:0/4205589971 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8010070 con 0x7f3f0006d770 2023-07-14T11:56:55.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.372+0000 7f3eff7fe700 1 -- 172.21.15.104:0/4205589971 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef800db40 con 0x7f3f0006d770 2023-07-14T11:56:55.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.372+0000 7f3eff7fe700 1 -- 172.21.15.104:0/4205589971 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8003a40 con 0x7f3f0006d770 2023-07-14T11:56:55.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.372+0000 7f3f065c1700 1 -- 172.21.15.104:0/4205589971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f0006d770 msgr2=0x7f3f0006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:55.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.372+0000 7f3f065c1700 1 --2- 172.21.15.104:0/4205589971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f0006d770 0x7f3f0006e9f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3f000709b0 tx=0x7f3ef800d430 comp rx=0 tx=0).stop 2023-07-14T11:56:55.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.372+0000 7f3f065c1700 1 -- 172.21.15.104:0/4205589971 shutdown_connections 2023-07-14T11:56:55.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.372+0000 7f3f065c1700 1 --2- 172.21.15.104:0/4205589971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f0006d770 0x7f3f0006e9f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.372+0000 7f3f065c1700 1 --2- 172.21.15.104:0/4205589971 >> v2:172.21.15.121:3300/0 conn(0x7f3f00158420 0x7f3f000cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.372+0000 7f3f065c1700 1 -- 172.21.15.104:0/4205589971 >> 172.21.15.104:0/4205589971 conn(0x7f3f00069800 msgr2=0x7f3f00069bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:55.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.372+0000 7f3f065c1700 1 -- 172.21.15.104:0/4205589971 shutdown_connections 2023-07-14T11:56:55.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.372+0000 7f3f065c1700 1 -- 172.21.15.104:0/4205589971 wait complete. 2023-07-14T11:56:55.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.372+0000 7f3f065c1700 1 Processor -- start 2023-07-14T11:56:55.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.372+0000 7f3f065c1700 1 -- start start 2023-07-14T11:56:55.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f065c1700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f0006d770 0x7f3f0014c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:55.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f065c1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f000ccef0 0x7f3f00145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:55.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f065c1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f00158420 0x7f3f00158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:55.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f065c1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f00150130 con 0x7f3f00158420 2023-07-14T11:56:55.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f065c1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0014f920 con 0x7f3f000ccef0 2023-07-14T11:56:55.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f065c1700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f00151ba0 con 0x7f3f0006d770 2023-07-14T11:56:55.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f04b5e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f00158420 0x7f3f00158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:55.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f04b5e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f00158420 0x7f3f00158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59764/0 (socket says 172.21.15.104:59764) 2023-07-14T11:56:55.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f04b5e700 1 -- 172.21.15.104:0/27515435 learned_addr learned my addr 172.21.15.104:0/27515435 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:55.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3ef77fe700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f000ccef0 0x7f3f00145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:55.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f04b5e700 1 -- 172.21.15.104:0/27515435 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f0006d770 msgr2=0x7f3f0014c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:55.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3efffff700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f0006d770 0x7f3f0014c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:55.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f04b5e700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f0006d770 0x7f3f0014c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f04b5e700 1 -- 172.21.15.104:0/27515435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f000ccef0 msgr2=0x7f3f00145380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:55.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f04b5e700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f000ccef0 0x7f3f00145380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f04b5e700 1 -- 172.21.15.104:0/27515435 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef800bd40 con 0x7f3f00158420 2023-07-14T11:56:55.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3ef77fe700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f000ccef0 0x7f3f00145380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:55.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3efffff700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f0006d770 0x7f3f0014c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:55.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f04b5e700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f00158420 0x7f3f00158980 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800d470 tx=0x7f3ef8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:55.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3efd7fa700 1 -- 172.21.15.104:0/27515435 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8025070 con 0x7f3f00158420 2023-07-14T11:56:55.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3efd7fa700 1 -- 172.21.15.104:0/27515435 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef800b020 con 0x7f3f00158420 2023-07-14T11:56:55.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3efd7fa700 1 -- 172.21.15.104:0/27515435 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef800b410 con 0x7f3f00158420 2023-07-14T11:56:55.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f065c1700 1 -- 172.21.15.104:0/27515435 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0014fb70 con 0x7f3f00158420 2023-07-14T11:56:55.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.376+0000 7f3f065c1700 1 -- 172.21.15.104:0/27515435 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f00151830 con 0x7f3f00158420 2023-07-14T11:56:55.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.380+0000 7f3f065c1700 1 -- 172.21.15.104:0/27515435 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f000cacd0 con 0x7f3f00158420 2023-07-14T11:56:55.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.380+0000 7f3efd7fa700 1 -- 172.21.15.104:0/27515435 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3ef801d070 con 0x7f3f00158420 2023-07-14T11:56:55.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.380+0000 7f3efd7fa700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3ee0066320 0x7f3ee00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:55.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.380+0000 7f3efd7fa700 1 -- 172.21.15.104:0/27515435 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f3ef8087630 con 0x7f3f00158420 2023-07-14T11:56:55.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.384+0000 7f3efffff700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3ee0066320 0x7f3ee00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:55.368 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.384+0000 7f3efffff700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3ee0066320 0x7f3ee00687d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0008930 tx=0x7f3ef0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:55.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.388+0000 7f3efd7fa700 1 -- 172.21.15.104:0/27515435 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f3ef80548d0 con 0x7f3f00158420 2023-07-14T11:56:55.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.560+0000 7f3f065c1700 1 -- 172.21.15.104:0/27515435 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f0004dcb0 con 0x7f3f00158420 2023-07-14T11:56:55.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.560+0000 7f3efd7fa700 1 -- 172.21.15.104:0/27515435 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ef8057ef0 con 0x7f3f00158420 2023-07-14T11:56:55.544 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:56:55.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.560+0000 7f3f065c1700 1 -- 172.21.15.104:0/27515435 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3ee0066320 msgr2=0x7f3ee00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:55.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.560+0000 7f3f065c1700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3ee0066320 0x7f3ee00687d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0008930 tx=0x7f3ef0008040 comp rx=0 tx=0).stop 2023-07-14T11:56:55.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.560+0000 7f3f065c1700 1 -- 172.21.15.104:0/27515435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f00158420 msgr2=0x7f3f00158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:55.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.560+0000 7f3f065c1700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f00158420 0x7f3f00158980 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800d470 tx=0x7f3ef8004af0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.560+0000 7f3f065c1700 1 -- 172.21.15.104:0/27515435 shutdown_connections 2023-07-14T11:56:55.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.560+0000 7f3f065c1700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3ee0066320 0x7f3ee00687d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.560+0000 7f3f065c1700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f00158420 0x7f3f00158980 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.560+0000 7f3f065c1700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f000ccef0 0x7f3f00145380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.560+0000 7f3f065c1700 1 --2- 172.21.15.104:0/27515435 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f0006d770 0x7f3f0014c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.560+0000 7f3f065c1700 1 -- 172.21.15.104:0/27515435 >> 172.21.15.104:0/27515435 conn(0x7f3f00069800 msgr2=0x7f3f00146820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:55.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.560+0000 7f3f065c1700 1 -- 172.21.15.104:0/27515435 shutdown_connections 2023-07-14T11:56:55.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.560+0000 7f3f065c1700 1 -- 172.21.15.104:0/27515435 wait complete. 2023-07-14T11:56:55.568 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T11:56:55.744 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.760+0000 7f402cac4700 1 Processor -- start 2023-07-14T11:56:55.745 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.760+0000 7f402cac4700 1 -- start start 2023-07-14T11:56:55.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.764+0000 7f402cac4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4028158370 0x7f402815bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:55.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.764+0000 7f402cac4700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f40280cb100 0x7f402806bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:55.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.764+0000 7f402cac4700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40280cbf80 con 0x7f40280c86b0 2023-07-14T11:56:55.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.764+0000 7f402cac4700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40280cbc30 con 0x7f40280cb100 2023-07-14T11:56:55.748 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.764+0000 7f402cac4700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402815c4d0 con 0x7f4028158370 2023-07-14T11:56:55.749 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.764+0000 7f40277fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f40280c86b0 0x7f40280c8a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:40910/0 (socket says 172.21.15.104:40910) 2023-07-14T11:56:55.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.764+0000 7f40277fe700 1 -- 172.21.15.104:0/3350297451 learned_addr learned my addr 172.21.15.104:0/3350297451 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:55.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.764+0000 7f4027fff700 1 --2- 172.21.15.104:0/3350297451 >> v2:172.21.15.104:3300/0 conn(0x7f40280cb100 0x7f402806bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:55.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.764+0000 7f4026ffd700 1 --2- 172.21.15.104:0/3350297451 >> v2:172.21.15.121:3300/0 conn(0x7f4028158370 0x7f402815bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:55.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.764+0000 7f4027fff700 1 -- 172.21.15.104:0/3350297451 >> v2:172.21.15.121:3300/0 conn(0x7f4028158370 msgr2=0x7f402815bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:55.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.764+0000 7f4027fff700 1 --2- 172.21.15.104:0/3350297451 >> v2:172.21.15.121:3300/0 conn(0x7f4028158370 0x7f402815bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.764+0000 7f4027fff700 1 -- 172.21.15.104:0/3350297451 >> v1:172.21.15.104:6789/0 conn(0x7f40280c86b0 legacy=0x7f40280c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:55.750 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.764+0000 7f4027fff700 1 -- 172.21.15.104:0/3350297451 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402815c7b0 con 0x7f40280cb100 2023-07-14T11:56:55.751 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.764+0000 7f4027fff700 1 --2- 172.21.15.104:0/3350297451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40280cb100 0x7f402806bc60 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4020009d90 tx=0x7f402000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9ce2da2372825c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:55.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.768+0000 7f40267fc700 1 -- 172.21.15.104:0/3350297451 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402000bce0 con 0x7f40280cb100 2023-07-14T11:56:55.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.768+0000 7f40267fc700 1 -- 172.21.15.104:0/3350297451 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402000be40 con 0x7f40280cb100 2023-07-14T11:56:55.754 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.768+0000 7f40267fc700 1 -- 172.21.15.104:0/3350297451 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40200106c0 con 0x7f40280cb100 2023-07-14T11:56:55.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.772+0000 7f402cac4700 1 -- 172.21.15.104:0/3350297451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40280cb100 msgr2=0x7f402806bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:55.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.772+0000 7f402cac4700 1 --2- 172.21.15.104:0/3350297451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40280cb100 0x7f402806bc60 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4020009d90 tx=0x7f402000b330 comp rx=0 tx=0).stop 2023-07-14T11:56:55.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.772+0000 7f402cac4700 1 -- 172.21.15.104:0/3350297451 shutdown_connections 2023-07-14T11:56:55.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.772+0000 7f402cac4700 1 --2- 172.21.15.104:0/3350297451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40280cb100 0x7f402806bc60 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.772+0000 7f402cac4700 1 --2- 172.21.15.104:0/3350297451 >> v2:172.21.15.121:3300/0 conn(0x7f4028158370 0x7f402815bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.772+0000 7f402cac4700 1 -- 172.21.15.104:0/3350297451 >> 172.21.15.104:0/3350297451 conn(0x7f402806e960 msgr2=0x7f402806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:55.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.772+0000 7f402cac4700 1 -- 172.21.15.104:0/3350297451 shutdown_connections 2023-07-14T11:56:55.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.772+0000 7f402cac4700 1 -- 172.21.15.104:0/3350297451 wait complete. 2023-07-14T11:56:55.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.776+0000 7f402cac4700 1 Processor -- start 2023-07-14T11:56:55.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.776+0000 7f402cac4700 1 -- start start 2023-07-14T11:56:55.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f402cac4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40280c86b0 0x7f40281f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:55.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f402cac4700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40280cb100 0x7f40280c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:55.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f402cac4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4028158370 0x7f4028154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:55.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f402cac4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40280b3c00 con 0x7f4028158370 2023-07-14T11:56:55.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f402cac4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40280c49a0 con 0x7f40280c86b0 2023-07-14T11:56:55.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f402cac4700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40280b2fc0 con 0x7f40280cb100 2023-07-14T11:56:55.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f40277fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40280c86b0 0x7f40281f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:55.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f40277fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40280c86b0 0x7f40281f5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:53778/0 (socket says 172.21.15.104:53778) 2023-07-14T11:56:55.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f40277fe700 1 -- 172.21.15.104:0/3484971452 learned_addr learned my addr 172.21.15.104:0/3484971452 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:55.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f4026ffd700 1 --2- 172.21.15.104:0/3484971452 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40280cb100 0x7f40280c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:55.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f4027fff700 1 --2- 172.21.15.104:0/3484971452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4028158370 0x7f4028154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:55.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f4027fff700 1 -- 172.21.15.104:0/3484971452 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40280cb100 msgr2=0x7f40280c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:55.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f4027fff700 1 --2- 172.21.15.104:0/3484971452 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40280cb100 0x7f40280c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f4027fff700 1 -- 172.21.15.104:0/3484971452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40280c86b0 msgr2=0x7f40281f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:55.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f4027fff700 1 --2- 172.21.15.104:0/3484971452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40280c86b0 0x7f40281f5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f4027fff700 1 -- 172.21.15.104:0/3484971452 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4020009a70 con 0x7f4028158370 2023-07-14T11:56:55.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f4027fff700 1 --2- 172.21.15.104:0/3484971452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4028158370 0x7f4028154420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4020018000 tx=0x7f4020004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:55.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f400ffff700 1 -- 172.21.15.104:0/3484971452 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020003c40 con 0x7f4028158370 2023-07-14T11:56:55.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f400ffff700 1 -- 172.21.15.104:0/3484971452 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4020003da0 con 0x7f4028158370 2023-07-14T11:56:55.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f402cac4700 1 -- 172.21.15.104:0/3484971452 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40280b3260 con 0x7f4028158370 2023-07-14T11:56:55.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f400ffff700 1 -- 172.21.15.104:0/3484971452 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020004290 con 0x7f4028158370 2023-07-14T11:56:55.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.780+0000 7f402cac4700 1 -- 172.21.15.104:0/3484971452 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40280b3820 con 0x7f4028158370 2023-07-14T11:56:55.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.784+0000 7f402cac4700 1 -- 172.21.15.104:0/3484971452 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40280b3d40 con 0x7f4028158370 2023-07-14T11:56:55.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.784+0000 7f400ffff700 1 -- 172.21.15.104:0/3484971452 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f40200043f0 con 0x7f4028158370 2023-07-14T11:56:55.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.784+0000 7f400ffff700 1 --2- 172.21.15.104:0/3484971452 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4014066320 0x7f40140687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:55.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.784+0000 7f400ffff700 1 -- 172.21.15.104:0/3484971452 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f402000e030 con 0x7f4028158370 2023-07-14T11:56:55.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.784+0000 7f40277fe700 1 --2- 172.21.15.104:0/3484971452 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4014066320 0x7f40140687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:55.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.784+0000 7f40277fe700 1 --2- 172.21.15.104:0/3484971452 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4014066320 0x7f40140687d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4018005930 tx=0x7f4018009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:55.773 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.788+0000 7f400ffff700 1 -- 172.21.15.104:0/3484971452 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f4020052b10 con 0x7f4028158370 2023-07-14T11:56:55.951 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.968+0000 7f402cac4700 1 -- 172.21.15.104:0/3484971452 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f402804e500 con 0x7f4028158370 2023-07-14T11:56:55.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.968+0000 7f400ffff700 1 -- 172.21.15.104:0/3484971452 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v61) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4020056130 con 0x7f4028158370 2023-07-14T11:56:55.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.968+0000 7f402cac4700 1 -- 172.21.15.104:0/3484971452 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4014066320 msgr2=0x7f40140687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:55.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.968+0000 7f402cac4700 1 --2- 172.21.15.104:0/3484971452 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4014066320 0x7f40140687d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4018005930 tx=0x7f4018009040 comp rx=0 tx=0).stop 2023-07-14T11:56:55.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.968+0000 7f402cac4700 1 -- 172.21.15.104:0/3484971452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4028158370 msgr2=0x7f4028154420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:55.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.968+0000 7f402cac4700 1 --2- 172.21.15.104:0/3484971452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4028158370 0x7f4028154420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4020018000 tx=0x7f4020004af0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.968+0000 7f402cac4700 1 -- 172.21.15.104:0/3484971452 shutdown_connections 2023-07-14T11:56:55.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.968+0000 7f402cac4700 1 --2- 172.21.15.104:0/3484971452 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f4014066320 0x7f40140687d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.968+0000 7f402cac4700 1 --2- 172.21.15.104:0/3484971452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4028158370 0x7f4028154420 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.968+0000 7f402cac4700 1 --2- 172.21.15.104:0/3484971452 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40280cb100 0x7f40280c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.968+0000 7f402cac4700 1 --2- 172.21.15.104:0/3484971452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40280c86b0 0x7f40281f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:55.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.968+0000 7f402cac4700 1 -- 172.21.15.104:0/3484971452 >> 172.21.15.104:0/3484971452 conn(0x7f402806e960 msgr2=0x7f40280cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:55.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.972+0000 7f402cac4700 1 -- 172.21.15.104:0/3484971452 shutdown_connections 2023-07-14T11:56:55.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:55.972+0000 7f402cac4700 1 -- 172.21.15.104:0/3484971452 wait complete. 2023-07-14T11:56:55.957 INFO:teuthology.orchestra.run.smithi104.stderr:listed 0 entries 2023-07-14T11:56:55.976 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi104.front.sepia.ceph.com... 2023-07-14T11:56:55.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:55.977 DEBUG:teuthology.orchestra.run.smithi104:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-14T11:56:56.004 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi104.stderr:ceph-fuse[51003]: fuse finished with error 0 and tester_r 0 2023-07-14T11:56:56.008 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:56:56.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:56.009 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-14T11:56:56.109 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:56:56.110 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:56:56.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352176k,nr_inodes=4088044,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18484 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275352k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-07-14T11:56:56.130 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:56:56.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:56.130 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-14T11:56:56.195 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-07-14T11:56:56.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:56:56.197 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:56:56.198 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:56:56.216 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352340k,nr_inodes=4088085,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275392k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275388k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275392k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-07-14T11:56:56.217 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:56:56.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:56.218 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-14T11:56:56.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:56:56.280 INFO:teuthology.orchestra.run.smithi104.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-14T11:56:56.281 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:56:56.281 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:56:56.299 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352176k,nr_inodes=4088044,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18484 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275352k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-07-14T11:56:56.300 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:56:56.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:56:56.301 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-14T11:56:56.362 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-14T11:56:56.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:56:56.365 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:56:56.532 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.548+0000 7f1978562700 1 Processor -- start 2023-07-14T11:56:56.533 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.548+0000 7f1978562700 1 -- start start 2023-07-14T11:56:56.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.548+0000 7f1978562700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197006d860 0x7f19700cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:56.534 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.548+0000 7f1978562700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19700cc1b0 con 0x7f197006be70 2023-07-14T11:56:56.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.548+0000 7f1978562700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19700cd690 con 0x7f19700c1e00 2023-07-14T11:56:56.535 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.548+0000 7f1978562700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19700cd970 con 0x7f197006d860 2023-07-14T11:56:56.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.548+0000 7f1975afd700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f197006be70 0x7f197006dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:40918/0 (socket says 172.21.15.104:40918) 2023-07-14T11:56:56.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.548+0000 7f1975afd700 1 -- 172.21.15.104:0/2677993741 learned_addr learned my addr 172.21.15.104:0/2677993741 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:56.536 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19762fe700 1 --2- 172.21.15.104:0/2677993741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197006d860 0x7f19700cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:56.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 -- 172.21.15.104:0/2677993741 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3416420725 0 0) 0x7f19700cc1b0 con 0x7f197006be70 2023-07-14T11:56:56.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 -- 172.21.15.104:0/2677993741 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f195c0035d0 con 0x7f197006be70 2023-07-14T11:56:56.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 -- 172.21.15.104:0/2677993741 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 572254288 0 0) 0x7f195c0035d0 con 0x7f197006be70 2023-07-14T11:56:56.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 -- 172.21.15.104:0/2677993741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197006d860 msgr2=0x7f19700cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:56.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 --2- 172.21.15.104:0/2677993741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197006d860 0x7f19700cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 -- 172.21.15.104:0/2677993741 >> v1:172.21.15.121:6789/0 conn(0x7f19700c1e00 legacy=0x7f19700cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:56.537 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 -- 172.21.15.104:0/2677993741 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19700cdc50 con 0x7f197006be70 2023-07-14T11:56:56.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 -- 172.21.15.104:0/2677993741 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f1960001670 con 0x7f197006be70 2023-07-14T11:56:56.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19762fe700 1 --2- 172.21.15.104:0/2677993741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197006d860 0x7f19700cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:56.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 -- 172.21.15.104:0/2677993741 >> v1:172.21.15.104:6789/0 conn(0x7f197006be70 legacy=0x7f197006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:56.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 --2- 172.21.15.104:0/2677993741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f195c003a50 0x7f195c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:56.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 --2- 172.21.15.104:0/2677993741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f195c004870 0x7f195c004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:56.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 --2- 172.21.15.104:0/2677993741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197006d860 0x7f195c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:56.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 -- 172.21.15.104:0/2677993741 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19700cd970 con 0x7f195c003a50 2023-07-14T11:56:56.538 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 -- 172.21.15.104:0/2677993741 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c00bb10 con 0x7f195c004870 2023-07-14T11:56:56.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 -- 172.21.15.104:0/2677993741 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195c00bcd0 con 0x7f197006d860 2023-07-14T11:56:56.539 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19762fe700 1 --2- 172.21.15.104:0/2677993741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f195c003a50 0x7f195c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:56.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f1976aff700 1 --2- 172.21.15.104:0/2677993741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197006d860 0x7f195c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:56.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f1975afd700 1 --2- 172.21.15.104:0/2677993741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f195c004870 0x7f195c004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:56.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19762fe700 1 -- 172.21.15.104:0/2677993741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197006d860 msgr2=0x7f195c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:56.540 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19762fe700 1 --2- 172.21.15.104:0/2677993741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f197006d860 0x7f195c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19762fe700 1 -- 172.21.15.104:0/2677993741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f195c004870 msgr2=0x7f195c004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:56.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19762fe700 1 --2- 172.21.15.104:0/2677993741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f195c004870 0x7f195c004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19762fe700 1 -- 172.21.15.104:0/2677993741 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c0035d0 con 0x7f195c003a50 2023-07-14T11:56:56.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19762fe700 1 --2- 172.21.15.104:0/2677993741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f195c003a50 0x7f195c003e60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f196c009970 tx=0x7f195c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5019285c06ee175a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:56.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 -- 172.21.15.104:0/2677993741 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c00c070 con 0x7f195c003a50 2023-07-14T11:56:56.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 -- 172.21.15.104:0/2677993741 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c004020 con 0x7f195c003a50 2023-07-14T11:56:56.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.552+0000 7f19752fc700 1 -- 172.21.15.104:0/2677993741 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c0043d0 con 0x7f195c003a50 2023-07-14T11:56:56.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.556+0000 7f1978562700 1 -- 172.21.15.104:0/2677993741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f195c003a50 msgr2=0x7f195c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:56.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.556+0000 7f1978562700 1 --2- 172.21.15.104:0/2677993741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f195c003a50 0x7f195c003e60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f196c009970 tx=0x7f195c0058a0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.556+0000 7f1976aff700 1 -- 172.21.15.104:0/2677993741 reap_dead start 2023-07-14T11:56:56.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.556+0000 7f1978562700 1 -- 172.21.15.104:0/2677993741 shutdown_connections 2023-07-14T11:56:56.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.556+0000 7f1978562700 1 -- 172.21.15.104:0/2677993741 >> 172.21.15.104:0/2677993741 conn(0x7f19700cad30 msgr2=0x7f19700cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:56.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.556+0000 7f1978562700 1 -- 172.21.15.104:0/2677993741 shutdown_connections 2023-07-14T11:56:56.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.556+0000 7f1978562700 1 -- 172.21.15.104:0/2677993741 wait complete. 2023-07-14T11:56:56.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.556+0000 7f1978562700 1 Processor -- start 2023-07-14T11:56:56.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.556+0000 7f1978562700 1 -- start start 2023-07-14T11:56:56.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1978562700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f195c003a50 0x7f1970146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:56.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1978562700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19701f3580 0x7f19701582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:56.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1978562700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1970158820 0x7f19701488a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:56.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1978562700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1970158c90 con 0x7f19701f3580 2023-07-14T11:56:56.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1978562700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1970146570 con 0x7f195c003a50 2023-07-14T11:56:56.544 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1978562700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19700c7280 con 0x7f1970158820 2023-07-14T11:56:56.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1975afd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19701f3580 0x7f19701582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:56.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1975afd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19701f3580 0x7f19701582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59798/0 (socket says 172.21.15.104:59798) 2023-07-14T11:56:56.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1975afd700 1 -- 172.21.15.104:0/2791073337 learned_addr learned my addr 172.21.15.104:0/2791073337 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:56.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f19762fe700 1 --2- 172.21.15.104:0/2791073337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f195c003a50 0x7f1970146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:56.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1976aff700 1 --2- 172.21.15.104:0/2791073337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1970158820 0x7f19701488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:56.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1975afd700 1 -- 172.21.15.104:0/2791073337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1970158820 msgr2=0x7f19701488a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:56.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1975afd700 1 --2- 172.21.15.104:0/2791073337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1970158820 0x7f19701488a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1975afd700 1 -- 172.21.15.104:0/2791073337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f195c003a50 msgr2=0x7f1970146030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:56.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1975afd700 1 --2- 172.21.15.104:0/2791073337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f195c003a50 0x7f1970146030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1975afd700 1 -- 172.21.15.104:0/2791073337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19700c63d0 con 0x7f19701f3580 2023-07-14T11:56:56.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1975afd700 1 --2- 172.21.15.104:0/2791073337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19701f3580 0x7f19701582e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f19600072c0 tx=0x7f19600075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:56.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1966ffd700 1 -- 172.21.15.104:0/2791073337 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960007cc0 con 0x7f19701f3580 2023-07-14T11:56:56.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1966ffd700 1 -- 172.21.15.104:0/2791073337 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196000f660 con 0x7f19701f3580 2023-07-14T11:56:56.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1978562700 1 -- 172.21.15.104:0/2791073337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19700c6670 con 0x7f19701f3580 2023-07-14T11:56:56.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1966ffd700 1 -- 172.21.15.104:0/2791073337 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000fa10 con 0x7f19701f3580 2023-07-14T11:56:56.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1978562700 1 -- 172.21.15.104:0/2791073337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19700c6c30 con 0x7f19701f3580 2023-07-14T11:56:56.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.560+0000 7f1978562700 1 -- 172.21.15.104:0/2791073337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19700c73c0 con 0x7f19701f3580 2023-07-14T11:56:56.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.564+0000 7f1966ffd700 1 -- 172.21.15.104:0/2791073337 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f196000fb70 con 0x7f19701f3580 2023-07-14T11:56:56.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.564+0000 7f1966ffd700 1 --2- 172.21.15.104:0/2791073337 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19540662f0 0x7f19540687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:56.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.564+0000 7f1966ffd700 1 -- 172.21.15.104:0/2791073337 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f1960083950 con 0x7f19701f3580 2023-07-14T11:56:56.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.564+0000 7f19762fe700 1 --2- 172.21.15.104:0/2791073337 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19540662f0 0x7f19540687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:56.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.568+0000 7f19762fe700 1 --2- 172.21.15.104:0/2791073337 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19540662f0 0x7f19540687a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f196c016040 tx=0x7f196c013000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:56.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.572+0000 7f1966ffd700 1 -- 172.21.15.104:0/2791073337 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f19700c73c0 con 0x7f19701f3580 2023-07-14T11:56:56.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.772+0000 7f1978562700 1 -- 172.21.15.104:0/2791073337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19700c73c0 con 0x7f19701f3580 2023-07-14T11:56:56.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.772+0000 7f1966ffd700 1 -- 172.21.15.104:0/2791073337 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4722 (secure 0 0 0) 0x7f19700c73c0 con 0x7f19701f3580 2023-07-14T11:56:56.758 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:56.758 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/4186544624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":4186544624},{"type":"v1","addr":"172.21.15.104:6837","nonce":4186544624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4234575863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4234575863},{"type":"v1","addr":"172.21.15.121:6835","nonce":4234575863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6220,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4127455794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4127455794},{"type":"v1","addr":"172.21.15.121:6833","nonce":4127455794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:56:11.123432+0000","modified":"2023-07-14T11:56:14.719624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/2705739882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2705739882},{"type":"v1","addr":"172.21.15.104:6836","nonce":2705739882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-14T11:56:56.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.776+0000 7f1978562700 1 -- 172.21.15.104:0/2791073337 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19540662f0 msgr2=0x7f19540687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:56.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.776+0000 7f1978562700 1 --2- 172.21.15.104:0/2791073337 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19540662f0 0x7f19540687a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f196c016040 tx=0x7f196c013000 comp rx=0 tx=0).stop 2023-07-14T11:56:56.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.776+0000 7f1978562700 1 -- 172.21.15.104:0/2791073337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19701f3580 msgr2=0x7f19701582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:56.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.776+0000 7f1978562700 1 --2- 172.21.15.104:0/2791073337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19701f3580 0x7f19701582e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f19600072c0 tx=0x7f19600075d0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.776+0000 7f1978562700 1 -- 172.21.15.104:0/2791073337 shutdown_connections 2023-07-14T11:56:56.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.776+0000 7f1978562700 1 --2- 172.21.15.104:0/2791073337 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f19540662f0 0x7f19540687a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.776+0000 7f1978562700 1 --2- 172.21.15.104:0/2791073337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19701f3580 0x7f19701582e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.776+0000 7f1978562700 1 --2- 172.21.15.104:0/2791073337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1970158820 0x7f19701488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.776+0000 7f1978562700 1 --2- 172.21.15.104:0/2791073337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f195c003a50 0x7f1970146030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.776+0000 7f1978562700 1 -- 172.21.15.104:0/2791073337 >> 172.21.15.104:0/2791073337 conn(0x7f19700cad30 msgr2=0x7f1970159510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:56.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.776+0000 7f1978562700 1 -- 172.21.15.104:0/2791073337 shutdown_connections 2023-07-14T11:56:56.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.776+0000 7f1978562700 1 -- 172.21.15.104:0/2791073337 wait complete. 2023-07-14T11:56:56.763 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 27 2023-07-14T11:56:56.785 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:56:56.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.968+0000 7f7db51ef700 1 Processor -- start 2023-07-14T11:56:56.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.968+0000 7f7db51ef700 1 -- start start 2023-07-14T11:56:56.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.972+0000 7f7db51ef700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0160370 0x7f7db00caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:56.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.972+0000 7f7db51ef700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7db00cafb0 0x7f7db00ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:56.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.972+0000 7f7db51ef700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db00ca8a0 con 0x7f7db015ffa0 2023-07-14T11:56:56.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.972+0000 7f7db51ef700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db00cb3f0 con 0x7f7db00cafb0 2023-07-14T11:56:56.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.972+0000 7f7db51ef700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00cbcd0 con 0x7f7db0160370 2023-07-14T11:56:56.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.972+0000 7f7da7fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0160370 0x7f7db00caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:56.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.972+0000 7f7db49ee700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7db00cafb0 0x7f7db00ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:56.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.972+0000 7f7da7fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0160370 0x7f7db00caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:57588/0 (socket says 172.21.15.104:57588) 2023-07-14T11:56:56.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.972+0000 7f7da7fff700 1 -- 172.21.15.104:0/2880332865 learned_addr learned my addr 172.21.15.104:0/2880332865 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:56.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.972+0000 7f7db49ee700 1 -- 172.21.15.104:0/2880332865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0160370 msgr2=0x7f7db00caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:56.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.972+0000 7f7db49ee700 1 --2- 172.21.15.104:0/2880332865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0160370 0x7f7db00caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.972+0000 7f7db49ee700 1 -- 172.21.15.104:0/2880332865 >> v1:172.21.15.121:6789/0 conn(0x7f7db015ffa0 legacy=0x7f7db006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:56.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.972+0000 7f7db49ee700 1 -- 172.21.15.104:0/2880332865 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db00cbf70 con 0x7f7db00cafb0 2023-07-14T11:56:56.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.972+0000 7f7da7fff700 1 --2- 172.21.15.104:0/2880332865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0160370 0x7f7db00caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:56.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.976+0000 7f7db49ee700 1 --2- 172.21.15.104:0/2880332865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00cafb0 0x7f7db00ce340 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009d90 tx=0x7f7da800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f17fe65e41e6f17 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:56.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.976+0000 7f7daf7fe700 1 -- 172.21.15.104:0/2880332865 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da800e070 con 0x7f7db00cafb0 2023-07-14T11:56:56.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.976+0000 7f7daf7fe700 1 -- 172.21.15.104:0/2880332865 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da800bb20 con 0x7f7db00cafb0 2023-07-14T11:56:56.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.976+0000 7f7daf7fe700 1 -- 172.21.15.104:0/2880332865 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8012430 con 0x7f7db00cafb0 2023-07-14T11:56:56.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.976+0000 7f7db51ef700 1 -- 172.21.15.104:0/2880332865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00cafb0 msgr2=0x7f7db00ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:56.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.976+0000 7f7db51ef700 1 --2- 172.21.15.104:0/2880332865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00cafb0 0x7f7db00ce340 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009d90 tx=0x7f7da800b330 comp rx=0 tx=0).stop 2023-07-14T11:56:56.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.976+0000 7f7db51ef700 1 -- 172.21.15.104:0/2880332865 shutdown_connections 2023-07-14T11:56:56.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.976+0000 7f7db51ef700 1 --2- 172.21.15.104:0/2880332865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00cafb0 0x7f7db00ce340 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.976+0000 7f7db51ef700 1 --2- 172.21.15.104:0/2880332865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db0160370 0x7f7db00caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.976+0000 7f7db51ef700 1 -- 172.21.15.104:0/2880332865 >> 172.21.15.104:0/2880332865 conn(0x7f7db006e960 msgr2=0x7f7db006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:56.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.976+0000 7f7db51ef700 1 -- 172.21.15.104:0/2880332865 shutdown_connections 2023-07-14T11:56:56.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.976+0000 7f7db51ef700 1 -- 172.21.15.104:0/2880332865 wait complete. 2023-07-14T11:56:56.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7db51ef700 1 Processor -- start 2023-07-14T11:56:56.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7db51ef700 1 -- start start 2023-07-14T11:56:56.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7db51ef700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db00cafb0 0x7f7db015df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:56.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7db51ef700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7db015ffa0 0x7f7db00ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:56.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7db51ef700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db0160370 0x7f7db0162720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:56.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7db51ef700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00aeee0 con 0x7f7db015ffa0 2023-07-14T11:56:56.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7db51ef700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00af1c0 con 0x7f7db0160370 2023-07-14T11:56:56.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7db51ef700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00af4a0 con 0x7f7db00cafb0 2023-07-14T11:56:56.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7da7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7db015ffa0 0x7f7db00ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:56.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7da7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7db015ffa0 0x7f7db00ae520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59810/0 (socket says 172.21.15.104:59810) 2023-07-14T11:56:56.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7da7fff700 1 -- 172.21.15.104:0/853802981 learned_addr learned my addr 172.21.15.104:0/853802981 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:56.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7daffff700 1 --2- 172.21.15.104:0/853802981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db00cafb0 0x7f7db015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:56.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7db49ee700 1 --2- 172.21.15.104:0/853802981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db0160370 0x7f7db0162720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:56.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7da7fff700 1 -- 172.21.15.104:0/853802981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db00cafb0 msgr2=0x7f7db015df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:56.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7da7fff700 1 --2- 172.21.15.104:0/853802981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db00cafb0 0x7f7db015df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7da7fff700 1 -- 172.21.15.104:0/853802981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db0160370 msgr2=0x7f7db0162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:56.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7da7fff700 1 --2- 172.21.15.104:0/853802981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db0160370 0x7f7db0162720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:56.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7da7fff700 1 -- 172.21.15.104:0/853802981 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d98009570 con 0x7f7db015ffa0 2023-07-14T11:56:56.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7da7fff700 1 --2- 172.21.15.104:0/853802981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7db015ffa0 0x7f7db00ae520 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800e9d0 tx=0x7f7d9800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:56.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7dad7fa700 1 -- 172.21.15.104:0/853802981 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9800c850 con 0x7f7db015ffa0 2023-07-14T11:56:56.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7dad7fa700 1 -- 172.21.15.104:0/853802981 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9800c9b0 con 0x7f7db015ffa0 2023-07-14T11:56:56.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.980+0000 7f7db51ef700 1 -- 172.21.15.104:0/853802981 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da8009a70 con 0x7f7db015ffa0 2023-07-14T11:56:56.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.984+0000 7f7dad7fa700 1 -- 172.21.15.104:0/853802981 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9800cd20 con 0x7f7db015ffa0 2023-07-14T11:56:56.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.984+0000 7f7db51ef700 1 -- 172.21.15.104:0/853802981 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db00c5bb0 con 0x7f7db015ffa0 2023-07-14T11:56:56.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.984+0000 7f7db51ef700 1 -- 172.21.15.104:0/853802981 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db004e500 con 0x7f7db015ffa0 2023-07-14T11:56:56.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.984+0000 7f7dad7fa700 1 -- 172.21.15.104:0/853802981 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7d98013470 con 0x7f7db015ffa0 2023-07-14T11:56:56.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.988+0000 7f7dad7fa700 1 --2- 172.21.15.104:0/853802981 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7d900662f0 0x7f7d900687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:56.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.988+0000 7f7dad7fa700 1 -- 172.21.15.104:0/853802981 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f7d98083f90 con 0x7f7db015ffa0 2023-07-14T11:56:56.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.988+0000 7f7daffff700 1 --2- 172.21.15.104:0/853802981 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7d900662f0 0x7f7d900687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:56.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.988+0000 7f7daffff700 1 --2- 172.21.15.104:0/853802981 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7d900662f0 0x7f7d900687a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7db00cac50 tx=0x7f7da0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:56.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:56.992+0000 7f7dad7fa700 1 -- 172.21.15.104:0/853802981 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f7d98051230 con 0x7f7db015ffa0 2023-07-14T11:56:57.154 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.168+0000 7f7db51ef700 1 -- 172.21.15.104:0/853802981 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7db0070de0 con 0x7f7db015ffa0 2023-07-14T11:56:57.155 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.172+0000 7f7dad7fa700 1 -- 172.21.15.104:0/853802981 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18774 (secure 0 0 0) 0x7f7d98054850 con 0x7f7db015ffa0 2023-07-14T11:56:57.156 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:57.156 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":61,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:56:55.760804+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:56:08.755551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-14T11:56:10.132688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.9","mappings":[{"from":7,"to":2}]},{"pgid":"10.1d","mappings":[{"from":4,"to":3}]},{"pgid":"10.2f","mappings":[{"from":4,"to":5}]},{"pgid":"10.32","mappings":[{"from":7,"to":3}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:56:57.158 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.172+0000 7f7db51ef700 1 -- 172.21.15.104:0/853802981 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7d900662f0 msgr2=0x7f7d900687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:57.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.172+0000 7f7db51ef700 1 --2- 172.21.15.104:0/853802981 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7d900662f0 0x7f7d900687a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7db00cac50 tx=0x7f7da0009040 comp rx=0 tx=0).stop 2023-07-14T11:56:57.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.172+0000 7f7db51ef700 1 -- 172.21.15.104:0/853802981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7db015ffa0 msgr2=0x7f7db00ae520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:57.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.172+0000 7f7db51ef700 1 --2- 172.21.15.104:0/853802981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7db015ffa0 0x7f7db00ae520 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800e9d0 tx=0x7f7d9800ece0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.172+0000 7f7db51ef700 1 -- 172.21.15.104:0/853802981 shutdown_connections 2023-07-14T11:56:57.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.172+0000 7f7db51ef700 1 --2- 172.21.15.104:0/853802981 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7d900662f0 0x7f7d900687a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.172+0000 7f7db51ef700 1 --2- 172.21.15.104:0/853802981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7db015ffa0 0x7f7db00ae520 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.172+0000 7f7db51ef700 1 --2- 172.21.15.104:0/853802981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db0160370 0x7f7db0162720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.172+0000 7f7db51ef700 1 --2- 172.21.15.104:0/853802981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7db00cafb0 0x7f7db015df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.172+0000 7f7db51ef700 1 -- 172.21.15.104:0/853802981 >> 172.21.15.104:0/853802981 conn(0x7f7db006e960 msgr2=0x7f7db00b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:57.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.172+0000 7f7db51ef700 1 -- 172.21.15.104:0/853802981 shutdown_connections 2023-07-14T11:56:57.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.172+0000 7f7db51ef700 1 -- 172.21.15.104:0/853802981 wait complete. 2023-07-14T11:56:57.180 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:53:01.976900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-14T11:53:07.267421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:56:08.755551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:56:10.132688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:56:57.181 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:56:57.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.372+0000 7ff96adf6700 1 Processor -- start 2023-07-14T11:56:57.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.372+0000 7ff96adf6700 1 -- start start 2023-07-14T11:56:57.358 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.372+0000 7ff96adf6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff964160000 0x7ff96406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:57.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff96adf6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9640cd7f0 0x7ff9640c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:57.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff96adf6700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9640cdfb0 con 0x7ff9641603d0 2023-07-14T11:56:57.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff96adf6700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff964070e40 con 0x7ff964160000 2023-07-14T11:56:57.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff96adf6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9640cdcd0 con 0x7ff9640cd7f0 2023-07-14T11:56:57.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff969df4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff964160000 0x7ff96406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:57.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff96a5f5700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9640cd7f0 0x7ff9640c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:57.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff9695f3700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff9641603d0 0x7ff9640c9e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:37562/0 (socket says 172.21.15.104:37562) 2023-07-14T11:56:57.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff9695f3700 1 -- 172.21.15.104:0/1000861405 learned_addr learned my addr 172.21.15.104:0/1000861405 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:57.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff968df2700 1 -- 172.21.15.104:0/1000861405 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734908992 0 0) 0x7ff954001110 con 0x7ff9641603d0 2023-07-14T11:56:57.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff968df2700 1 -- 172.21.15.104:0/1000861405 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9500035d0 con 0x7ff9641603d0 2023-07-14T11:56:57.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff969df4700 1 -- 172.21.15.104:0/1000861405 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9640cd7f0 msgr2=0x7ff9640c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:57.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff969df4700 1 --2- 172.21.15.104:0/1000861405 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9640cd7f0 0x7ff9640c0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff969df4700 1 -- 172.21.15.104:0/1000861405 >> v1:172.21.15.121:6789/0 conn(0x7ff9641603d0 legacy=0x7ff9640c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:57.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff969df4700 1 -- 172.21.15.104:0/1000861405 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff964070780 con 0x7ff964160000 2023-07-14T11:56:57.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff96a5f5700 1 --2- 172.21.15.104:0/1000861405 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9640cd7f0 0x7ff9640c0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:57.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff969df4700 1 --2- 172.21.15.104:0/1000861405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff964160000 0x7ff96406dd10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff960009850 tx=0x7ff960009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3090f24735e112a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:57.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff968df2700 1 -- 172.21.15.104:0/1000861405 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960011830 con 0x7ff964160000 2023-07-14T11:56:57.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff968df2700 1 -- 172.21.15.104:0/1000861405 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff960011990 con 0x7ff964160000 2023-07-14T11:56:57.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.376+0000 7ff968df2700 1 -- 172.21.15.104:0/1000861405 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960011d40 con 0x7ff964160000 2023-07-14T11:56:57.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.380+0000 7ff96adf6700 1 -- 172.21.15.104:0/1000861405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff964160000 msgr2=0x7ff96406dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:57.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.380+0000 7ff96adf6700 1 --2- 172.21.15.104:0/1000861405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff964160000 0x7ff96406dd10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff960009850 tx=0x7ff960009b60 comp rx=0 tx=0).stop 2023-07-14T11:56:57.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.380+0000 7ff96adf6700 1 -- 172.21.15.104:0/1000861405 shutdown_connections 2023-07-14T11:56:57.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.380+0000 7ff96adf6700 1 --2- 172.21.15.104:0/1000861405 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9640cd7f0 0x7ff9640c0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.380+0000 7ff96adf6700 1 --2- 172.21.15.104:0/1000861405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff964160000 0x7ff96406dd10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.380+0000 7ff96adf6700 1 -- 172.21.15.104:0/1000861405 >> 172.21.15.104:0/1000861405 conn(0x7ff9640afc60 msgr2=0x7ff9640b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:57.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.380+0000 7ff96adf6700 1 -- 172.21.15.104:0/1000861405 shutdown_connections 2023-07-14T11:56:57.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.380+0000 7ff96adf6700 1 -- 172.21.15.104:0/1000861405 wait complete. 2023-07-14T11:56:57.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.380+0000 7ff96adf6700 1 Processor -- start 2023-07-14T11:56:57.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.380+0000 7ff96adf6700 1 -- start start 2023-07-14T11:56:57.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff96adf6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9640cd7f0 0x7ff9641545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:57.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff96adf6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff964160000 0x7ff9640b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:57.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff96adf6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9641603d0 0x7ff9640b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:57.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff96adf6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff964161600 con 0x7ff9640cd7f0 2023-07-14T11:56:57.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff96adf6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9640b2910 con 0x7ff9641603d0 2023-07-14T11:56:57.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff96adf6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9640b2070 con 0x7ff964160000 2023-07-14T11:56:57.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff969df4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9640cd7f0 0x7ff9641545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:57.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff9695f3700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff964160000 0x7ff9640b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:57.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff969df4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9640cd7f0 0x7ff9641545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59824/0 (socket says 172.21.15.104:59824) 2023-07-14T11:56:57.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff969df4700 1 -- 172.21.15.104:0/2384729196 learned_addr learned my addr 172.21.15.104:0/2384729196 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:57.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff96a5f5700 1 --2- 172.21.15.104:0/2384729196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9641603d0 0x7ff9640b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:57.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff9695f3700 1 -- 172.21.15.104:0/2384729196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9641603d0 msgr2=0x7ff9640b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:57.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff9695f3700 1 --2- 172.21.15.104:0/2384729196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9641603d0 0x7ff9640b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.374 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff9695f3700 1 -- 172.21.15.104:0/2384729196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9640cd7f0 msgr2=0x7ff9641545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:57.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff9695f3700 1 --2- 172.21.15.104:0/2384729196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9640cd7f0 0x7ff9641545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff9695f3700 1 -- 172.21.15.104:0/2384729196 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c009a70 con 0x7ff964160000 2023-07-14T11:56:57.375 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff9695f3700 1 --2- 172.21.15.104:0/2384729196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff964160000 0x7ff9640b3470 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff95400c410 tx=0x7ff95400c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:57.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff95a7fc700 1 -- 172.21.15.104:0/2384729196 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff954005d90 con 0x7ff964160000 2023-07-14T11:56:57.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff95a7fc700 1 -- 172.21.15.104:0/2384729196 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95400ce10 con 0x7ff964160000 2023-07-14T11:56:57.376 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff95a7fc700 1 -- 172.21.15.104:0/2384729196 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9540072f0 con 0x7ff964160000 2023-07-14T11:56:57.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.384+0000 7ff96adf6700 1 -- 172.21.15.104:0/2384729196 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9600094b0 con 0x7ff964160000 2023-07-14T11:56:57.377 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.388+0000 7ff96adf6700 1 -- 172.21.15.104:0/2384729196 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9640b2630 con 0x7ff964160000 2023-07-14T11:56:57.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.388+0000 7ff96adf6700 1 -- 172.21.15.104:0/2384729196 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff964071240 con 0x7ff964160000 2023-07-14T11:56:57.378 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.388+0000 7ff95a7fc700 1 -- 172.21.15.104:0/2384729196 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff954007450 con 0x7ff964160000 2023-07-14T11:56:57.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.392+0000 7ff95a7fc700 1 --2- 172.21.15.104:0/2384729196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9480662f0 0x7ff9480687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:57.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.392+0000 7ff95a7fc700 1 -- 172.21.15.104:0/2384729196 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6527+0+0 (secure 0 0 0) 0x7ff954082db0 con 0x7ff964160000 2023-07-14T11:56:57.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.392+0000 7ff969df4700 1 --2- 172.21.15.104:0/2384729196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9480662f0 0x7ff9480687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:57.379 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.392+0000 7ff969df4700 1 --2- 172.21.15.104:0/2384729196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9480662f0 0x7ff9480687a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff960000c00 tx=0x7ff960008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:57.382 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.396+0000 7ff95a7fc700 1 -- 172.21.15.104:0/2384729196 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff964071240 con 0x7ff964160000 2023-07-14T11:56:57.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.568+0000 7ff96adf6700 1 -- 172.21.15.104:0/2384729196 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9640cdfb0 con 0x7ff964160000 2023-07-14T11:56:57.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.568+0000 7ff95a7fc700 1 -- 172.21.15.104:0/2384729196 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9640cdfb0 con 0x7ff964160000 2023-07-14T11:56:57.555 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2023-07-14T11:56:57.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.572+0000 7ff96adf6700 1 -- 172.21.15.104:0/2384729196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9480662f0 msgr2=0x7ff9480687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:57.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.572+0000 7ff96adf6700 1 --2- 172.21.15.104:0/2384729196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9480662f0 0x7ff9480687a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff960000c00 tx=0x7ff960008030 comp rx=0 tx=0).stop 2023-07-14T11:56:57.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.572+0000 7ff96adf6700 1 -- 172.21.15.104:0/2384729196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff964160000 msgr2=0x7ff9640b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:57.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.572+0000 7ff96adf6700 1 --2- 172.21.15.104:0/2384729196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff964160000 0x7ff9640b3470 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff95400c410 tx=0x7ff95400c720 comp rx=0 tx=0).stop 2023-07-14T11:56:57.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.572+0000 7ff96adf6700 1 -- 172.21.15.104:0/2384729196 shutdown_connections 2023-07-14T11:56:57.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.572+0000 7ff96adf6700 1 --2- 172.21.15.104:0/2384729196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9480662f0 0x7ff9480687a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.572+0000 7ff96adf6700 1 --2- 172.21.15.104:0/2384729196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9641603d0 0x7ff9640b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.572+0000 7ff96adf6700 1 --2- 172.21.15.104:0/2384729196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff964160000 0x7ff9640b3470 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.572+0000 7ff96adf6700 1 --2- 172.21.15.104:0/2384729196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9640cd7f0 0x7ff9641545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.572+0000 7ff96adf6700 1 -- 172.21.15.104:0/2384729196 >> 172.21.15.104:0/2384729196 conn(0x7ff9640afc60 msgr2=0x7ff9640ca780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:57.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.572+0000 7ff96adf6700 1 -- 172.21.15.104:0/2384729196 shutdown_connections 2023-07-14T11:56:57.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.572+0000 7ff96adf6700 1 -- 172.21.15.104:0/2384729196 wait complete. 2023-07-14T11:56:57.580 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-14T11:56:57.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.772+0000 7fc47e1fe700 1 Processor -- start 2023-07-14T11:56:57.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.772+0000 7fc47e1fe700 1 -- start start 2023-07-14T11:56:57.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.772+0000 7fc47e1fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc4781603d0 0x7fc47806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:57.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.772+0000 7fc47e1fe700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4780cdfb0 con 0x7fc478160000 2023-07-14T11:56:57.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.772+0000 7fc47e1fe700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4780cdcd0 con 0x7fc4780cd7f0 2023-07-14T11:56:57.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.772+0000 7fc47e1fe700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc478070e40 con 0x7fc4781603d0 2023-07-14T11:56:57.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc47c9fb700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc4781603d0 0x7fc47806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:57.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc47c9fb700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc4781603d0 0x7fc47806dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59840/0 (socket says 172.21.15.104:59840) 2023-07-14T11:56:57.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc47c9fb700 1 -- 172.21.15.104:0/1027294573 learned_addr learned my addr 172.21.15.104:0/1027294573 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:57.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc47d1fc700 1 --1- 172.21.15.104:0/1027294573 >> v1:172.21.15.104:6789/0 conn(0x7fc478160000 0x7fc4780c9e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:40932/0 (socket says 172.21.15.104:40932) 2023-07-14T11:56:57.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc47c9fb700 1 -- 172.21.15.104:0/1027294573 >> v1:172.21.15.121:6789/0 conn(0x7fc4780cd7f0 legacy=0x7fc4780c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:57.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc47c9fb700 1 -- 172.21.15.104:0/1027294573 >> v1:172.21.15.104:6789/0 conn(0x7fc478160000 legacy=0x7fc4780c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:57.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc47c9fb700 1 -- 172.21.15.104:0/1027294573 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc478070780 con 0x7fc4781603d0 2023-07-14T11:56:57.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc47c9fb700 1 --2- 172.21.15.104:0/1027294573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4781603d0 0x7fc47806dd10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc468009910 tx=0x7fc468009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e4200018529264 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:57.761 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc46ffff700 1 -- 172.21.15.104:0/1027294573 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468004230 con 0x7fc4781603d0 2023-07-14T11:56:57.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc46ffff700 1 -- 172.21.15.104:0/1027294573 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc468004390 con 0x7fc4781603d0 2023-07-14T11:56:57.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc46ffff700 1 -- 172.21.15.104:0/1027294573 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468009e60 con 0x7fc4781603d0 2023-07-14T11:56:57.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc47e1fe700 1 -- 172.21.15.104:0/1027294573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4781603d0 msgr2=0x7fc47806dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:57.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc47e1fe700 1 --2- 172.21.15.104:0/1027294573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4781603d0 0x7fc47806dd10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc468009910 tx=0x7fc468009c20 comp rx=0 tx=0).stop 2023-07-14T11:56:57.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc47e1fe700 1 -- 172.21.15.104:0/1027294573 shutdown_connections 2023-07-14T11:56:57.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc47e1fe700 1 --2- 172.21.15.104:0/1027294573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4781603d0 0x7fc47806dd10 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.762 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc47e1fe700 1 -- 172.21.15.104:0/1027294573 >> 172.21.15.104:0/1027294573 conn(0x7fc4780afc60 msgr2=0x7fc4780b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:57.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc47e1fe700 1 -- 172.21.15.104:0/1027294573 shutdown_connections 2023-07-14T11:56:57.763 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.776+0000 7fc47e1fe700 1 -- 172.21.15.104:0/1027294573 wait complete. 2023-07-14T11:56:57.764 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.780+0000 7fc47e1fe700 1 Processor -- start 2023-07-14T11:56:57.765 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.780+0000 7fc47e1fe700 1 -- start start 2023-07-14T11:56:57.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.780+0000 7fc47e1fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4780cd7f0 0x7fc478154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:57.766 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.780+0000 7fc47e1fe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc478160000 0x7fc4780b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:57.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.780+0000 7fc47e1fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4781603d0 0x7fc4780aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:57.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.780+0000 7fc47e1fe700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc478154940 con 0x7fc4781603d0 2023-07-14T11:56:57.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.780+0000 7fc47e1fe700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4781f5810 con 0x7fc4780cd7f0 2023-07-14T11:56:57.767 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.780+0000 7fc47e1fe700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4781f5af0 con 0x7fc478160000 2023-07-14T11:56:57.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc47d9fd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4781603d0 0x7fc4780aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:57.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc47d9fd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4781603d0 0x7fc4780aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59846/0 (socket says 172.21.15.104:59846) 2023-07-14T11:56:57.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc47d9fd700 1 -- 172.21.15.104:0/3337314405 learned_addr learned my addr 172.21.15.104:0/3337314405 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:57.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc47d1fc700 1 --2- 172.21.15.104:0/3337314405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4780cd7f0 0x7fc478154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:57.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc47c9fb700 1 --2- 172.21.15.104:0/3337314405 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc478160000 0x7fc4780b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:57.768 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc47d9fd700 1 -- 172.21.15.104:0/3337314405 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc478160000 msgr2=0x7fc4780b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:57.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc47d9fd700 1 --2- 172.21.15.104:0/3337314405 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc478160000 0x7fc4780b3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc47d9fd700 1 -- 172.21.15.104:0/3337314405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4780cd7f0 msgr2=0x7fc478154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:57.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc47d9fd700 1 --2- 172.21.15.104:0/3337314405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4780cd7f0 0x7fc478154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc47d9fd700 1 -- 172.21.15.104:0/3337314405 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc468009570 con 0x7fc4781603d0 2023-07-14T11:56:57.769 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc47d9fd700 1 --2- 172.21.15.104:0/3337314405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4781603d0 0x7fc4780aeb60 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc47000cc10 tx=0x7fc47000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:57.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc46dffb700 1 -- 172.21.15.104:0/3337314405 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47000aa60 con 0x7fc4781603d0 2023-07-14T11:56:57.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc46dffb700 1 -- 172.21.15.104:0/3337314405 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47000abc0 con 0x7fc4781603d0 2023-07-14T11:56:57.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc47e1fe700 1 -- 172.21.15.104:0/3337314405 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4781f5dd0 con 0x7fc4781603d0 2023-07-14T11:56:57.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc46dffb700 1 -- 172.21.15.104:0/3337314405 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470012420 con 0x7fc4781603d0 2023-07-14T11:56:57.770 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.784+0000 7fc47e1fe700 1 -- 172.21.15.104:0/3337314405 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4781f6390 con 0x7fc4781603d0 2023-07-14T11:56:57.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.788+0000 7fc47e1fe700 1 -- 172.21.15.104:0/3337314405 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47804e500 con 0x7fc4781603d0 2023-07-14T11:56:57.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.788+0000 7fc46dffb700 1 -- 172.21.15.104:0/3337314405 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc470012580 con 0x7fc4781603d0 2023-07-14T11:56:57.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.788+0000 7fc46dffb700 1 --2- 172.21.15.104:0/3337314405 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc45c0662f0 0x7fc45c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:57.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.788+0000 7fc46dffb700 1 -- 172.21.15.104:0/3337314405 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6527+0+0 (secure 0 0 0) 0x7fc470084270 con 0x7fc4781603d0 2023-07-14T11:56:57.774 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.788+0000 7fc47d1fc700 1 --2- 172.21.15.104:0/3337314405 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc45c0662f0 0x7fc45c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:57.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.792+0000 7fc47d1fc700 1 --2- 172.21.15.104:0/3337314405 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc45c0662f0 0x7fc45c0687a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc47806cac0 tx=0x7fc474007f30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:57.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.792+0000 7fc46dffb700 1 -- 172.21.15.104:0/3337314405 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fc470051480 con 0x7fc4781603d0 2023-07-14T11:56:57.954 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.968+0000 7fc47e1fe700 1 -- 172.21.15.104:0/3337314405 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc4780b40d0 con 0x7fc4781603d0 2023-07-14T11:56:57.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.968+0000 7fc46dffb700 1 -- 172.21.15.104:0/3337314405 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v61) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc470054aa0 con 0x7fc4781603d0 2023-07-14T11:56:57.955 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 8 2023-07-14T11:56:57.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.976+0000 7fc47e1fe700 1 -- 172.21.15.104:0/3337314405 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc45c0662f0 msgr2=0x7fc45c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:57.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.976+0000 7fc47e1fe700 1 --2- 172.21.15.104:0/3337314405 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc45c0662f0 0x7fc45c0687a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc47806cac0 tx=0x7fc474007f30 comp rx=0 tx=0).stop 2023-07-14T11:56:57.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.976+0000 7fc47e1fe700 1 -- 172.21.15.104:0/3337314405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4781603d0 msgr2=0x7fc4780aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:57.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.976+0000 7fc47e1fe700 1 --2- 172.21.15.104:0/3337314405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4781603d0 0x7fc4780aeb60 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc47000cc10 tx=0x7fc47000cf20 comp rx=0 tx=0).stop 2023-07-14T11:56:57.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.976+0000 7fc47e1fe700 1 -- 172.21.15.104:0/3337314405 shutdown_connections 2023-07-14T11:56:57.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.976+0000 7fc47e1fe700 1 --2- 172.21.15.104:0/3337314405 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc45c0662f0 0x7fc45c0687a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.976+0000 7fc47e1fe700 1 --2- 172.21.15.104:0/3337314405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4781603d0 0x7fc4780aeb60 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.976+0000 7fc47e1fe700 1 --2- 172.21.15.104:0/3337314405 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc478160000 0x7fc4780b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.976+0000 7fc47e1fe700 1 --2- 172.21.15.104:0/3337314405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4780cd7f0 0x7fc478154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:57.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.976+0000 7fc47e1fe700 1 -- 172.21.15.104:0/3337314405 >> 172.21.15.104:0/3337314405 conn(0x7fc4780afc60 msgr2=0x7fc478072510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:57.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.976+0000 7fc47e1fe700 1 -- 172.21.15.104:0/3337314405 shutdown_connections 2023-07-14T11:56:57.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:57.976+0000 7fc47e1fe700 1 -- 172.21.15.104:0/3337314405 wait complete. 2023-07-14T11:56:57.981 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T11:56:58.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.172+0000 7f8125b46700 1 Processor -- start 2023-07-14T11:56:58.156 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.172+0000 7f8125b46700 1 -- start start 2023-07-14T11:56:58.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.172+0000 7f8125b46700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f81200b0490 0x7f81200cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.160 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.172+0000 7f8125b46700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8120160610 0x7f81200ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.172+0000 7f8125b46700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8120163fe0 con 0x7f8120160240 2023-07-14T11:56:58.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.172+0000 7f8125b46700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81200c8c30 con 0x7f8120160610 2023-07-14T11:56:58.161 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.172+0000 7f8125b46700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81200ca9e0 con 0x7f81200b0490 2023-07-14T11:56:58.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.172+0000 7f811ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8120160610 0x7f81200ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.172+0000 7f811ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8120160610 0x7f81200ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59852/0 (socket says 172.21.15.104:59852) 2023-07-14T11:56:58.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.172+0000 7f811ffff700 1 -- 172.21.15.104:0/2476887473 learned_addr learned my addr 172.21.15.104:0/2476887473 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:58.162 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.172+0000 7f811effd700 1 --2- 172.21.15.104:0/2476887473 >> v2:172.21.15.121:3300/0 conn(0x7f81200b0490 0x7f81200cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.172+0000 7f811ffff700 1 -- 172.21.15.104:0/2476887473 >> v2:172.21.15.121:3300/0 conn(0x7f81200b0490 msgr2=0x7f81200cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:58.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.172+0000 7f811ffff700 1 --2- 172.21.15.104:0/2476887473 >> v2:172.21.15.121:3300/0 conn(0x7f81200b0490 0x7f81200cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.172+0000 7f811ffff700 1 -- 172.21.15.104:0/2476887473 >> v1:172.21.15.121:6789/0 conn(0x7f8120160240 legacy=0x7f81200afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:58.163 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.176+0000 7f811ffff700 1 -- 172.21.15.104:0/2476887473 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8120164230 con 0x7f8120160610 2023-07-14T11:56:58.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.176+0000 7f811effd700 1 --2- 172.21.15.104:0/2476887473 >> v2:172.21.15.121:3300/0 conn(0x7f81200b0490 0x7f81200cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:58.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.176+0000 7f811ffff700 1 --2- 172.21.15.104:0/2476887473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8120160610 0x7f81200ca320 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f81200b09b0 tx=0x7f811800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f2e8d2935c4d91 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:58.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.176+0000 7f811e7fc700 1 -- 172.21.15.104:0/2476887473 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811800e070 con 0x7f8120160610 2023-07-14T11:56:58.164 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.176+0000 7f811e7fc700 1 -- 172.21.15.104:0/2476887473 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8118003a40 con 0x7f8120160610 2023-07-14T11:56:58.165 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.176+0000 7f811e7fc700 1 -- 172.21.15.104:0/2476887473 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118003df0 con 0x7f8120160610 2023-07-14T11:56:58.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.176+0000 7f8125b46700 1 -- 172.21.15.104:0/2476887473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8120160610 msgr2=0x7f81200ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:58.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.176+0000 7f8125b46700 1 --2- 172.21.15.104:0/2476887473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8120160610 0x7f81200ca320 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f81200b09b0 tx=0x7f811800b330 comp rx=0 tx=0).stop 2023-07-14T11:56:58.167 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.176+0000 7f8125b46700 1 -- 172.21.15.104:0/2476887473 shutdown_connections 2023-07-14T11:56:58.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.176+0000 7f8125b46700 1 --2- 172.21.15.104:0/2476887473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8120160610 0x7f81200ca320 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.176+0000 7f8125b46700 1 --2- 172.21.15.104:0/2476887473 >> v2:172.21.15.121:3300/0 conn(0x7f81200b0490 0x7f81200cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.176+0000 7f8125b46700 1 -- 172.21.15.104:0/2476887473 >> 172.21.15.104:0/2476887473 conn(0x7f81200cad80 msgr2=0x7f81200cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:58.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.176+0000 7f8125b46700 1 -- 172.21.15.104:0/2476887473 shutdown_connections 2023-07-14T11:56:58.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.176+0000 7f8125b46700 1 -- 172.21.15.104:0/2476887473 wait complete. 2023-07-14T11:56:58.168 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f8125b46700 1 Processor -- start 2023-07-14T11:56:58.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f8125b46700 1 -- start start 2023-07-14T11:56:58.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f8125b46700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81200b0490 0x7f81200c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f8125b46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8120160240 0x7f81200c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f8125b46700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8120160610 0x7f81200c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f8125b46700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81201f5b50 con 0x7f8120160610 2023-07-14T11:56:58.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f8125b46700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81200b12d0 con 0x7f8120160240 2023-07-14T11:56:58.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f8125b46700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81200c55e0 con 0x7f81200b0490 2023-07-14T11:56:58.169 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f811ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8120160610 0x7f81200c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f811ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8120160610 0x7f81200c27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59856/0 (socket says 172.21.15.104:59856) 2023-07-14T11:56:58.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f811ffff700 1 -- 172.21.15.104:0/1215037713 learned_addr learned my addr 172.21.15.104:0/1215037713 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:58.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f811effd700 1 --2- 172.21.15.104:0/1215037713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8120160240 0x7f81200c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f811f7fe700 1 --2- 172.21.15.104:0/1215037713 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81200b0490 0x7f81200c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f811ffff700 1 -- 172.21.15.104:0/1215037713 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81200b0490 msgr2=0x7f81200c1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:58.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f811ffff700 1 --2- 172.21.15.104:0/1215037713 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81200b0490 0x7f81200c1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f811ffff700 1 -- 172.21.15.104:0/1215037713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8120160240 msgr2=0x7f81200c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:58.170 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f811ffff700 1 --2- 172.21.15.104:0/1215037713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8120160240 0x7f81200c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.180+0000 7f811ffff700 1 -- 172.21.15.104:0/1215037713 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8118009b30 con 0x7f8120160610 2023-07-14T11:56:58.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.184+0000 7f811ffff700 1 --2- 172.21.15.104:0/1215037713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8120160610 0x7f81200c27e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8118015040 tx=0x7f8118004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:58.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.184+0000 7f8107fff700 1 -- 172.21.15.104:0/1215037713 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118004440 con 0x7f8120160610 2023-07-14T11:56:58.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.184+0000 7f8107fff700 1 -- 172.21.15.104:0/1215037713 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81180092e0 con 0x7f8120160610 2023-07-14T11:56:58.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.184+0000 7f8107fff700 1 -- 172.21.15.104:0/1215037713 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81180047e0 con 0x7f8120160610 2023-07-14T11:56:58.171 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.184+0000 7f8125b46700 1 -- 172.21.15.104:0/1215037713 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81200b1570 con 0x7f8120160610 2023-07-14T11:56:58.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.184+0000 7f8125b46700 1 -- 172.21.15.104:0/1215037713 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81200b1b30 con 0x7f8120160610 2023-07-14T11:56:58.172 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.184+0000 7f8125b46700 1 -- 172.21.15.104:0/1215037713 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812004dcb0 con 0x7f8120160610 2023-07-14T11:56:58.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.184+0000 7f8107fff700 1 -- 172.21.15.104:0/1215037713 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8118012430 con 0x7f8120160610 2023-07-14T11:56:58.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.188+0000 7f8107fff700 1 --2- 172.21.15.104:0/1215037713 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f810c0662f0 0x7f810c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.188+0000 7f8107fff700 1 -- 172.21.15.104:0/1215037713 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f8118085260 con 0x7f8120160610 2023-07-14T11:56:58.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.188+0000 7f811f7fe700 1 --2- 172.21.15.104:0/1215037713 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f810c0662f0 0x7f810c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.173 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.188+0000 7f811f7fe700 1 --2- 172.21.15.104:0/1215037713 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f810c0662f0 0x7f810c0687a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8120163ef0 tx=0x7f81100090e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:58.177 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.192+0000 7f8107fff700 1 -- 172.21.15.104:0/1215037713 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f8118055f90 con 0x7f8120160610 2023-07-14T11:56:58.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.364+0000 7f8125b46700 1 -- 172.21.15.104:0/1215037713 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f81200b0bc0 con 0x7f8120160610 2023-07-14T11:56:58.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.368+0000 7f8107fff700 1 -- 172.21.15.104:0/1215037713 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8118055b20 con 0x7f8120160610 2023-07-14T11:56:58.351 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 32 2023-07-14T11:56:58.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.372+0000 7f8125b46700 1 -- 172.21.15.104:0/1215037713 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f810c0662f0 msgr2=0x7f810c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:58.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.372+0000 7f8125b46700 1 --2- 172.21.15.104:0/1215037713 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f810c0662f0 0x7f810c0687a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8120163ef0 tx=0x7f81100090e0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.372+0000 7f8125b46700 1 -- 172.21.15.104:0/1215037713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8120160610 msgr2=0x7f81200c27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:58.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.372+0000 7f8125b46700 1 --2- 172.21.15.104:0/1215037713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8120160610 0x7f81200c27e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8118015040 tx=0x7f8118004af0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.372+0000 7f8125b46700 1 -- 172.21.15.104:0/1215037713 shutdown_connections 2023-07-14T11:56:58.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.372+0000 7f8125b46700 1 --2- 172.21.15.104:0/1215037713 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f810c0662f0 0x7f810c0687a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.372+0000 7f8125b46700 1 --2- 172.21.15.104:0/1215037713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8120160610 0x7f81200c27e0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.372+0000 7f8125b46700 1 --2- 172.21.15.104:0/1215037713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8120160240 0x7f81200c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.372+0000 7f8125b46700 1 --2- 172.21.15.104:0/1215037713 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81200b0490 0x7f81200c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.372+0000 7f8125b46700 1 -- 172.21.15.104:0/1215037713 >> 172.21.15.104:0/1215037713 conn(0x7f81200cad80 msgr2=0x7f812006d420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:58.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.372+0000 7f8125b46700 1 -- 172.21.15.104:0/1215037713 shutdown_connections 2023-07-14T11:56:58.357 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.372+0000 7f8125b46700 1 -- 172.21.15.104:0/1215037713 wait complete. 2023-07-14T11:56:58.377 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T11:56:58.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.568+0000 7f542e1a0700 1 Processor -- start 2023-07-14T11:56:58.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.568+0000 7f542e1a0700 1 -- start start 2023-07-14T11:56:58.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.568+0000 7f542e1a0700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f542815df10 0x7f542815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.568+0000 7f542e1a0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54280afd80 0x7f54280711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.568+0000 7f542e1a0700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5428070a30 con 0x7f54280716e0 2023-07-14T11:56:58.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.568+0000 7f542e1a0700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542815e830 con 0x7f542815df10 2023-07-14T11:56:58.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.568+0000 7f542e1a0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54280b0210 con 0x7f54280afd80 2023-07-14T11:56:58.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f5426ffd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54280afd80 0x7f54280711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f54277fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f542815df10 0x7f542815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f5427fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f54280716e0 0x7f54280c8950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:37586/0 (socket says 172.21.15.104:37586) 2023-07-14T11:56:58.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f54277fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f542815df10 0x7f542815e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:53876/0 (socket says 172.21.15.104:53876) 2023-07-14T11:56:58.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f54277fe700 1 -- 172.21.15.104:0/2393649864 learned_addr learned my addr 172.21.15.104:0/2393649864 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:58.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f5426ffd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54280afd80 0x7f54280711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:57644/0 (socket says 172.21.15.104:57644) 2023-07-14T11:56:58.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f54267fc700 1 -- 172.21.15.104:0/2393649864 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887251178 0 0) 0x7f5428070a30 con 0x7f54280716e0 2023-07-14T11:56:58.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f54267fc700 1 -- 172.21.15.104:0/2393649864 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54100035d0 con 0x7f54280716e0 2023-07-14T11:56:58.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f54277fe700 1 -- 172.21.15.104:0/2393649864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54280afd80 msgr2=0x7f54280711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:58.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f54277fe700 1 --2- 172.21.15.104:0/2393649864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54280afd80 0x7f54280711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f54277fe700 1 -- 172.21.15.104:0/2393649864 >> v1:172.21.15.121:6789/0 conn(0x7f54280716e0 legacy=0x7f54280c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:58.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f54277fe700 1 -- 172.21.15.104:0/2393649864 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54280b04f0 con 0x7f542815df10 2023-07-14T11:56:58.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f5426ffd700 1 --2- 172.21.15.104:0/2393649864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54280afd80 0x7f54280711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:58.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f54277fe700 1 --2- 172.21.15.104:0/2393649864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f542815df10 0x7f542815e2f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5418009910 tx=0x7f5418009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13ffde2a36a64df4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:58.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f54267fc700 1 -- 172.21.15.104:0/2393649864 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418011830 con 0x7f542815df10 2023-07-14T11:56:58.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f54267fc700 1 -- 172.21.15.104:0/2393649864 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5418011990 con 0x7f542815df10 2023-07-14T11:56:58.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f54267fc700 1 -- 172.21.15.104:0/2393649864 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418011d40 con 0x7f542815df10 2023-07-14T11:56:58.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f542e1a0700 1 -- 172.21.15.104:0/2393649864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f542815df10 msgr2=0x7f542815e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:58.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f542e1a0700 1 --2- 172.21.15.104:0/2393649864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f542815df10 0x7f542815e2f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5418009910 tx=0x7f5418009c20 comp rx=0 tx=0).stop 2023-07-14T11:56:58.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f542e1a0700 1 -- 172.21.15.104:0/2393649864 shutdown_connections 2023-07-14T11:56:58.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f542e1a0700 1 --2- 172.21.15.104:0/2393649864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54280afd80 0x7f54280711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f542e1a0700 1 --2- 172.21.15.104:0/2393649864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f542815df10 0x7f542815e2f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.572+0000 7f542e1a0700 1 -- 172.21.15.104:0/2393649864 >> 172.21.15.104:0/2393649864 conn(0x7f54280cad80 msgr2=0x7f54280cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:58.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.576+0000 7f542e1a0700 1 -- 172.21.15.104:0/2393649864 shutdown_connections 2023-07-14T11:56:58.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.576+0000 7f542e1a0700 1 -- 172.21.15.104:0/2393649864 wait complete. 2023-07-14T11:56:58.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.576+0000 7f542e1a0700 1 Processor -- start 2023-07-14T11:56:58.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f542e1a0700 1 -- start start 2023-07-14T11:56:58.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f542e1a0700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54280716e0 0x7f54280c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f542e1a0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54280afd80 0x7f54281576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f542e1a0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5428157bf0 0x7f5428143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f542e1a0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5428156c40 con 0x7f54280afd80 2023-07-14T11:56:58.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f542e1a0700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54280c82a0 con 0x7f54280716e0 2023-07-14T11:56:58.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f542e1a0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54280c8580 con 0x7f5428157bf0 2023-07-14T11:56:58.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f5426ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54280afd80 0x7f54281576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f5426ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54280afd80 0x7f54281576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59872/0 (socket says 172.21.15.104:59872) 2023-07-14T11:56:58.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f5426ffd700 1 -- 172.21.15.104:0/3816063110 learned_addr learned my addr 172.21.15.104:0/3816063110 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:58.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f5427fff700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5428157bf0 0x7f5428143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f54277fe700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54280716e0 0x7f54280c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f5426ffd700 1 -- 172.21.15.104:0/3816063110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5428157bf0 msgr2=0x7f5428143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:58.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f5426ffd700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5428157bf0 0x7f5428143610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f5426ffd700 1 -- 172.21.15.104:0/3816063110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54280716e0 msgr2=0x7f54280c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:58.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f5426ffd700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54280716e0 0x7f54280c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f5426ffd700 1 -- 172.21.15.104:0/3816063110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c00d950 con 0x7f54280afd80 2023-07-14T11:56:58.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f54277fe700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54280716e0 0x7f54280c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:58.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f5427fff700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5428157bf0 0x7f5428143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:58.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.580+0000 7f5426ffd700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54280afd80 0x7f54281576b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f541c012fc0 tx=0x7f541c0103a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:58.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.584+0000 7f540ffff700 1 -- 172.21.15.104:0/3816063110 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c010bb0 con 0x7f54280afd80 2023-07-14T11:56:58.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.584+0000 7f540ffff700 1 -- 172.21.15.104:0/3816063110 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c013040 con 0x7f54280afd80 2023-07-14T11:56:58.569 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.584+0000 7f542e1a0700 1 -- 172.21.15.104:0/3816063110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5418009570 con 0x7f54280afd80 2023-07-14T11:56:58.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.584+0000 7f540ffff700 1 -- 172.21.15.104:0/3816063110 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c00d2e0 con 0x7f54280afd80 2023-07-14T11:56:58.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.584+0000 7f542e1a0700 1 -- 172.21.15.104:0/3816063110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5428157190 con 0x7f54280afd80 2023-07-14T11:56:58.570 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.584+0000 7f542e1a0700 1 -- 172.21.15.104:0/3816063110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5428157350 con 0x7f54280afd80 2023-07-14T11:56:58.571 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.588+0000 7f540ffff700 1 -- 172.21.15.104:0/3816063110 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f541c018480 con 0x7f54280afd80 2023-07-14T11:56:58.573 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.588+0000 7f540ffff700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5414066320 0x7f54140687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.588+0000 7f540ffff700 1 -- 172.21.15.104:0/3816063110 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f541c0880d0 con 0x7f54280afd80 2023-07-14T11:56:58.574 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.588+0000 7f54277fe700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5414066320 0x7f54140687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.575 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.588+0000 7f54277fe700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5414066320 0x7f54140687d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5428158150 tx=0x7f5418008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:58.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.596+0000 7f540ffff700 1 -- 172.21.15.104:0/3816063110 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f541c0552e0 con 0x7f54280afd80 2023-07-14T11:56:58.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.768+0000 7f542e1a0700 1 -- 172.21.15.104:0/3816063110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f542804dcb0 con 0x7f54280afd80 2023-07-14T11:56:58.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.772+0000 7f540ffff700 1 -- 172.21.15.104:0/3816063110 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7f541c058900 con 0x7f54280afd80 2023-07-14T11:56:58.756 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 64 2023-07-14T11:56:58.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.772+0000 7f542e1a0700 1 -- 172.21.15.104:0/3816063110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5414066320 msgr2=0x7f54140687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:58.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.772+0000 7f542e1a0700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5414066320 0x7f54140687d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5428158150 tx=0x7f5418008030 comp rx=0 tx=0).stop 2023-07-14T11:56:58.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.772+0000 7f542e1a0700 1 -- 172.21.15.104:0/3816063110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54280afd80 msgr2=0x7f54281576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:58.758 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.772+0000 7f542e1a0700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54280afd80 0x7f54281576b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f541c012fc0 tx=0x7f541c0103a0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.772+0000 7f542e1a0700 1 -- 172.21.15.104:0/3816063110 shutdown_connections 2023-07-14T11:56:58.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.772+0000 7f542e1a0700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f5414066320 0x7f54140687d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.759 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.772+0000 7f542e1a0700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54280afd80 0x7f54281576b0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.772+0000 7f542e1a0700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5428157bf0 0x7f5428143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.772+0000 7f542e1a0700 1 --2- 172.21.15.104:0/3816063110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54280716e0 0x7f54280c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.772+0000 7f542e1a0700 1 -- 172.21.15.104:0/3816063110 >> 172.21.15.104:0/3816063110 conn(0x7f54280cad80 msgr2=0x7f5428159740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:58.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.776+0000 7f542e1a0700 1 -- 172.21.15.104:0/3816063110 shutdown_connections 2023-07-14T11:56:58.760 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.776+0000 7f542e1a0700 1 -- 172.21.15.104:0/3816063110 wait complete. 2023-07-14T11:56:58.781 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:56:58.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.968+0000 7fb241d1a700 1 Processor -- start 2023-07-14T11:56:58.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.968+0000 7fb241d1a700 1 -- start start 2023-07-14T11:56:58.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.972+0000 7fb241d1a700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb23c0afd30 0x7fb23c0b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.972+0000 7fb241d1a700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23c0703a0 0x7fb23c070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.972+0000 7fb241d1a700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23c0cd530 con 0x7fb23c15c070 2023-07-14T11:56:58.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.972+0000 7fb241d1a700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23c0b06e0 con 0x7fb23c0afd30 2023-07-14T11:56:58.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.972+0000 7fb241d1a700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c15c440 con 0x7fb23c0703a0 2023-07-14T11:56:58.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.972+0000 7fb23affd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb23c0afd30 0x7fb23c0b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.972+0000 7fb23affd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb23c0afd30 0x7fb23c0b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59886/0 (socket says 172.21.15.104:59886) 2023-07-14T11:56:58.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.972+0000 7fb23affd700 1 -- 172.21.15.104:0/4268987647 learned_addr learned my addr 172.21.15.104:0/4268987647 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:58.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.972+0000 7fb23bfff700 1 --2- 172.21.15.104:0/4268987647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23c0703a0 0x7fb23c070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.972+0000 7fb23affd700 1 -- 172.21.15.104:0/4268987647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23c0703a0 msgr2=0x7fb23c070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:58.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.972+0000 7fb23affd700 1 --2- 172.21.15.104:0/4268987647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23c0703a0 0x7fb23c070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.972+0000 7fb23affd700 1 -- 172.21.15.104:0/4268987647 >> v1:172.21.15.121:6789/0 conn(0x7fb23c15c070 legacy=0x7fb23c0ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:58.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.972+0000 7fb23affd700 1 -- 172.21.15.104:0/4268987647 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c0cc1b0 con 0x7fb23c0afd30 2023-07-14T11:56:58.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.972+0000 7fb23bfff700 1 --2- 172.21.15.104:0/4268987647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23c0703a0 0x7fb23c070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:58.960 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.976+0000 7fb23affd700 1 --2- 172.21.15.104:0/4268987647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb23c0afd30 0x7fb23c0b01a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb230009910 tx=0x7fb230009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121a1f4f1992ce0e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:58.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.976+0000 7fb23a7fc700 1 -- 172.21.15.104:0/4268987647 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23000ba60 con 0x7fb23c0afd30 2023-07-14T11:56:58.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.976+0000 7fb23a7fc700 1 -- 172.21.15.104:0/4268987647 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23000bbc0 con 0x7fb23c0afd30 2023-07-14T11:56:58.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.976+0000 7fb23a7fc700 1 -- 172.21.15.104:0/4268987647 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230009e60 con 0x7fb23c0afd30 2023-07-14T11:56:58.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.976+0000 7fb241d1a700 1 -- 172.21.15.104:0/4268987647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb23c0afd30 msgr2=0x7fb23c0b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:58.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.976+0000 7fb241d1a700 1 --2- 172.21.15.104:0/4268987647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb23c0afd30 0x7fb23c0b01a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb230009910 tx=0x7fb230009c20 comp rx=0 tx=0).stop 2023-07-14T11:56:58.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.976+0000 7fb241d1a700 1 -- 172.21.15.104:0/4268987647 shutdown_connections 2023-07-14T11:56:58.961 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.976+0000 7fb241d1a700 1 --2- 172.21.15.104:0/4268987647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23c0703a0 0x7fb23c070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.976+0000 7fb241d1a700 1 --2- 172.21.15.104:0/4268987647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb23c0afd30 0x7fb23c0b01a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.976+0000 7fb241d1a700 1 -- 172.21.15.104:0/4268987647 >> 172.21.15.104:0/4268987647 conn(0x7fb23c0cad30 msgr2=0x7fb23c0cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:58.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.976+0000 7fb241d1a700 1 -- 172.21.15.104:0/4268987647 shutdown_connections 2023-07-14T11:56:58.962 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.976+0000 7fb241d1a700 1 -- 172.21.15.104:0/4268987647 wait complete. 2023-07-14T11:56:58.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.976+0000 7fb241d1a700 1 Processor -- start 2023-07-14T11:56:58.963 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.976+0000 7fb241d1a700 1 -- start start 2023-07-14T11:56:58.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb241d1a700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23c0703a0 0x7fb23c151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb241d1a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb23c0afd30 0x7fb23c0c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb241d1a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb23c15c070 0x7fb23c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb241d1a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c151f50 con 0x7fb23c0afd30 2023-07-14T11:56:58.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb241d1a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c0c3210 con 0x7fb23c15c070 2023-07-14T11:56:58.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb241d1a700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c0c34f0 con 0x7fb23c0703a0 2023-07-14T11:56:58.965 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb23affd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb23c0afd30 0x7fb23c0c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb23affd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb23c0afd30 0x7fb23c0c5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59900/0 (socket says 172.21.15.104:59900) 2023-07-14T11:56:58.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb23affd700 1 -- 172.21.15.104:0/2986694285 learned_addr learned my addr 172.21.15.104:0/2986694285 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:58.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb23bfff700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb23c15c070 0x7fb23c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb23b7fe700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23c0703a0 0x7fb23c151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb23affd700 1 -- 172.21.15.104:0/2986694285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23c0703a0 msgr2=0x7fb23c151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:58.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb23affd700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23c0703a0 0x7fb23c151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.966 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb23affd700 1 -- 172.21.15.104:0/2986694285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb23c15c070 msgr2=0x7fb23c06d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:58.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb23affd700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb23c15c070 0x7fb23c06d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:58.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb23affd700 1 -- 172.21.15.104:0/2986694285 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb230009570 con 0x7fb23c0afd30 2023-07-14T11:56:58.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb23bfff700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb23c15c070 0x7fb23c06d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:58.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb23b7fe700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23c0703a0 0x7fb23c151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:58.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb23affd700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb23c0afd30 0x7fb23c0c5380 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fb23c070270 tx=0x7fb230004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:58.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb223fff700 1 -- 172.21.15.104:0/2986694285 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230004610 con 0x7fb23c0afd30 2023-07-14T11:56:58.967 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb223fff700 1 -- 172.21.15.104:0/2986694285 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23001f070 con 0x7fb23c0afd30 2023-07-14T11:56:58.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb241d1a700 1 -- 172.21.15.104:0/2986694285 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb23c0c37d0 con 0x7fb23c0afd30 2023-07-14T11:56:58.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb223fff700 1 -- 172.21.15.104:0/2986694285 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230003900 con 0x7fb23c0afd30 2023-07-14T11:56:58.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.980+0000 7fb241d1a700 1 -- 172.21.15.104:0/2986694285 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23c0c3d90 con 0x7fb23c0afd30 2023-07-14T11:56:58.968 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.984+0000 7fb241d1a700 1 -- 172.21.15.104:0/2986694285 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb23c04dcb0 con 0x7fb23c0afd30 2023-07-14T11:56:58.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.984+0000 7fb223fff700 1 -- 172.21.15.104:0/2986694285 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb230003a60 con 0x7fb23c0afd30 2023-07-14T11:56:58.970 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.984+0000 7fb223fff700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb228066320 0x7fb2280687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:58.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.984+0000 7fb223fff700 1 -- 172.21.15.104:0/2986694285 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6527+0+0 (secure 0 0 0) 0x7fb230021070 con 0x7fb23c0afd30 2023-07-14T11:56:58.971 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.988+0000 7fb23b7fe700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb228066320 0x7fb2280687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:58.972 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.988+0000 7fb23b7fe700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb228066320 0x7fb2280687d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0012e0 tx=0x7fb22c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:58.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:58.992+0000 7fb223fff700 1 -- 172.21.15.104:0/2986694285 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb230052e90 con 0x7fb23c0afd30 2023-07-14T11:56:59.184 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.200+0000 7fb241d1a700 1 -- 172.21.15.104:0/2986694285 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb23c0abc10 con 0x7fb23c0afd30 2023-07-14T11:56:59.185 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.200+0000 7fb223fff700 1 -- 172.21.15.104:0/2986694285 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4722 (secure 0 0 0) 0x7fb2300564b0 con 0x7fb23c0afd30 2023-07-14T11:56:59.186 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:59.186 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/4186544624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":4186544624},{"type":"v1","addr":"172.21.15.104:6837","nonce":4186544624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4234575863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4234575863},{"type":"v1","addr":"172.21.15.121:6835","nonce":4234575863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6220,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4127455794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4127455794},{"type":"v1","addr":"172.21.15.121:6833","nonce":4127455794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:56:11.123432+0000","modified":"2023-07-14T11:56:14.719624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/2705739882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2705739882},{"type":"v1","addr":"172.21.15.104:6836","nonce":2705739882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-14T11:56:59.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.204+0000 7fb241d1a700 1 -- 172.21.15.104:0/2986694285 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb228066320 msgr2=0x7fb2280687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:59.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.204+0000 7fb241d1a700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb228066320 0x7fb2280687d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0012e0 tx=0x7fb22c009040 comp rx=0 tx=0).stop 2023-07-14T11:56:59.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.204+0000 7fb241d1a700 1 -- 172.21.15.104:0/2986694285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb23c0afd30 msgr2=0x7fb23c0c5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:59.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.204+0000 7fb241d1a700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb23c0afd30 0x7fb23c0c5380 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fb23c070270 tx=0x7fb230004770 comp rx=0 tx=0).stop 2023-07-14T11:56:59.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.204+0000 7fb241d1a700 1 -- 172.21.15.104:0/2986694285 shutdown_connections 2023-07-14T11:56:59.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.204+0000 7fb241d1a700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb228066320 0x7fb2280687d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.204+0000 7fb241d1a700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb23c0afd30 0x7fb23c0c5380 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.204+0000 7fb241d1a700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb23c15c070 0x7fb23c06d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.204+0000 7fb241d1a700 1 --2- 172.21.15.104:0/2986694285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb23c0703a0 0x7fb23c151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.204+0000 7fb241d1a700 1 -- 172.21.15.104:0/2986694285 >> 172.21.15.104:0/2986694285 conn(0x7fb23c0cad30 msgr2=0x7fb23c0c97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:59.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.204+0000 7fb241d1a700 1 -- 172.21.15.104:0/2986694285 shutdown_connections 2023-07-14T11:56:59.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.204+0000 7fb241d1a700 1 -- 172.21.15.104:0/2986694285 wait complete. 2023-07-14T11:56:59.192 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 27 2023-07-14T11:56:59.207 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:56:59.383 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff956d4c700 1 Processor -- start 2023-07-14T11:56:59.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff956d4c700 1 -- start start 2023-07-14T11:56:59.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff956d4c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff9500cd240 0x7ff9500ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:59.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff956d4c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff9500ceff0 0x7ff9500afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:59.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff956d4c700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9500af0a0 con 0x7ff95015bff0 2023-07-14T11:56:59.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff956d4c700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9500cf720 con 0x7ff9500cd240 2023-07-14T11:56:59.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff956d4c700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9500af2f0 con 0x7ff9500ceff0 2023-07-14T11:56:59.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff955d4a700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ff95015bff0 0x7ff9500ccb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:40946/0 (socket says 172.21.15.104:40946) 2023-07-14T11:56:59.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff955d4a700 1 -- 172.21.15.104:0/1273669026 learned_addr learned my addr 172.21.15.104:0/1273669026 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:59.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff955549700 1 --2- 172.21.15.104:0/1273669026 >> v2:172.21.15.104:3300/0 conn(0x7ff9500cd240 0x7ff9500ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:59.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff954d48700 1 -- 172.21.15.104:0/1273669026 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270399073 0 0) 0x7ff9500af0a0 con 0x7ff95015bff0 2023-07-14T11:56:59.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff954d48700 1 -- 172.21.15.104:0/1273669026 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff93c0035d0 con 0x7ff95015bff0 2023-07-14T11:56:59.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff95654b700 1 --2- 172.21.15.104:0/1273669026 >> v2:172.21.15.121:3300/0 conn(0x7ff9500ceff0 0x7ff9500afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:59.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff955549700 1 -- 172.21.15.104:0/1273669026 >> v2:172.21.15.121:3300/0 conn(0x7ff9500ceff0 msgr2=0x7ff9500afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:59.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff955549700 1 --2- 172.21.15.104:0/1273669026 >> v2:172.21.15.121:3300/0 conn(0x7ff9500ceff0 0x7ff9500afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff955549700 1 -- 172.21.15.104:0/1273669026 >> v1:172.21.15.104:6789/0 conn(0x7ff95015bff0 legacy=0x7ff9500ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:59.387 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.400+0000 7ff955549700 1 -- 172.21.15.104:0/1273669026 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500af5d0 con 0x7ff9500cd240 2023-07-14T11:56:59.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.404+0000 7ff955549700 1 --2- 172.21.15.104:0/1273669026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9500cd240 0x7ff9500ceab0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff95008b700 tx=0x7ff940009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511eb57aa9bf7eb7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:59.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.404+0000 7ff954d48700 1 -- 172.21.15.104:0/1273669026 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940011830 con 0x7ff9500cd240 2023-07-14T11:56:59.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.404+0000 7ff954d48700 1 -- 172.21.15.104:0/1273669026 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff940011990 con 0x7ff9500cd240 2023-07-14T11:56:59.388 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.404+0000 7ff954d48700 1 -- 172.21.15.104:0/1273669026 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940011d20 con 0x7ff9500cd240 2023-07-14T11:56:59.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.404+0000 7ff956d4c700 1 -- 172.21.15.104:0/1273669026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9500cd240 msgr2=0x7ff9500ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:59.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.404+0000 7ff956d4c700 1 --2- 172.21.15.104:0/1273669026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9500cd240 0x7ff9500ceab0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff95008b700 tx=0x7ff940009b30 comp rx=0 tx=0).stop 2023-07-14T11:56:59.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.404+0000 7ff956d4c700 1 -- 172.21.15.104:0/1273669026 shutdown_connections 2023-07-14T11:56:59.389 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.404+0000 7ff956d4c700 1 --2- 172.21.15.104:0/1273669026 >> v2:172.21.15.121:3300/0 conn(0x7ff9500ceff0 0x7ff9500afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.404+0000 7ff956d4c700 1 --2- 172.21.15.104:0/1273669026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9500cd240 0x7ff9500ceab0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.404+0000 7ff956d4c700 1 -- 172.21.15.104:0/1273669026 >> 172.21.15.104:0/1273669026 conn(0x7ff950060c40 msgr2=0x7ff95009b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:59.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.404+0000 7ff956d4c700 1 -- 172.21.15.104:0/1273669026 shutdown_connections 2023-07-14T11:56:59.390 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.404+0000 7ff956d4c700 1 -- 172.21.15.104:0/1273669026 wait complete. 2023-07-14T11:56:59.391 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff956d4c700 1 Processor -- start 2023-07-14T11:56:59.392 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff956d4c700 1 -- start start 2023-07-14T11:56:59.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff956d4c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9500cd240 0x7ff9500b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:59.393 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff956d4c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9500ceff0 0x7ff9500b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:59.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff956d4c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff95015bff0 0x7ff9500b7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:59.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff956d4c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9500b7d50 con 0x7ff9500ceff0 2023-07-14T11:56:59.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff956d4c700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff950158200 con 0x7ff9500cd240 2023-07-14T11:56:59.394 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff956d4c700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9500b1bf0 con 0x7ff95015bff0 2023-07-14T11:56:59.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff955549700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9500ceff0 0x7ff9500b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:59.395 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff955549700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9500ceff0 0x7ff9500b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59924/0 (socket says 172.21.15.104:59924) 2023-07-14T11:56:59.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff955549700 1 -- 172.21.15.104:0/4257466953 learned_addr learned my addr 172.21.15.104:0/4257466953 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:59.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff955d4a700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9500cd240 0x7ff9500b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:59.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff95654b700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff95015bff0 0x7ff9500b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:59.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff955549700 1 -- 172.21.15.104:0/4257466953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff95015bff0 msgr2=0x7ff9500b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:59.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff955549700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff95015bff0 0x7ff9500b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff955549700 1 -- 172.21.15.104:0/4257466953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9500cd240 msgr2=0x7ff9500b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:59.396 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff955549700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9500cd240 0x7ff9500b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff955549700 1 -- 172.21.15.104:0/4257466953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9400094b0 con 0x7ff9500ceff0 2023-07-14T11:56:59.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff95654b700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff95015bff0 0x7ff9500b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:59.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.408+0000 7ff955d4a700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9500cd240 0x7ff9500b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:59.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.412+0000 7ff955549700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9500ceff0 0x7ff9500b1560 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff9400090b0 tx=0x7ff940004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:59.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.412+0000 7ff9467fc700 1 -- 172.21.15.104:0/4257466953 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9400036c0 con 0x7ff9500ceff0 2023-07-14T11:56:59.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.412+0000 7ff9467fc700 1 -- 172.21.15.104:0/4257466953 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94001e070 con 0x7ff9500ceff0 2023-07-14T11:56:59.397 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.412+0000 7ff956d4c700 1 -- 172.21.15.104:0/4257466953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9500b1ed0 con 0x7ff9500ceff0 2023-07-14T11:56:59.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.412+0000 7ff9467fc700 1 -- 172.21.15.104:0/4257466953 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940003a40 con 0x7ff9500ceff0 2023-07-14T11:56:59.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.412+0000 7ff956d4c700 1 -- 172.21.15.104:0/4257466953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9500b2490 con 0x7ff9500ceff0 2023-07-14T11:56:59.398 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.412+0000 7ff956d4c700 1 -- 172.21.15.104:0/4257466953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95004e500 con 0x7ff9500ceff0 2023-07-14T11:56:59.399 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.416+0000 7ff9467fc700 1 -- 172.21.15.104:0/4257466953 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff940003ba0 con 0x7ff9500ceff0 2023-07-14T11:56:59.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.416+0000 7ff9467fc700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9340662f0 0x7ff9340687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:59.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.416+0000 7ff9467fc700 1 -- 172.21.15.104:0/4257466953 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6527+0+0 (secure 0 0 0) 0x7ff940084db0 con 0x7ff9500ceff0 2023-07-14T11:56:59.400 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.416+0000 7ff955d4a700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9340662f0 0x7ff9340687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:59.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.416+0000 7ff955d4a700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9340662f0 0x7ff9340687a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0011e0 tx=0x7ff94c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:59.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.420+0000 7ff9467fc700 1 -- 172.21.15.104:0/4257466953 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff9400520f0 con 0x7ff9500ceff0 2023-07-14T11:56:59.579 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.592+0000 7ff956d4c700 1 -- 172.21.15.104:0/4257466953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9500cdb10 con 0x7ff9500ceff0 2023-07-14T11:56:59.582 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.596+0000 7ff9467fc700 1 -- 172.21.15.104:0/4257466953 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18774 (secure 0 0 0) 0x7ff940020090 con 0x7ff9500ceff0 2023-07-14T11:56:59.583 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:56:59.583 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":61,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:56:55.760804+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:56:08.755551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-14T11:56:10.132688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.9","mappings":[{"from":7,"to":2}]},{"pgid":"10.1d","mappings":[{"from":4,"to":3}]},{"pgid":"10.2f","mappings":[{"from":4,"to":5}]},{"pgid":"10.32","mappings":[{"from":7,"to":3}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:56:59.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.600+0000 7ff956d4c700 1 -- 172.21.15.104:0/4257466953 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9340662f0 msgr2=0x7ff9340687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:59.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.600+0000 7ff956d4c700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9340662f0 0x7ff9340687a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0011e0 tx=0x7ff94c009040 comp rx=0 tx=0).stop 2023-07-14T11:56:59.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.600+0000 7ff956d4c700 1 -- 172.21.15.104:0/4257466953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9500ceff0 msgr2=0x7ff9500b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:59.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.600+0000 7ff956d4c700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9500ceff0 0x7ff9500b1560 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff9400090b0 tx=0x7ff940004770 comp rx=0 tx=0).stop 2023-07-14T11:56:59.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.600+0000 7ff956d4c700 1 -- 172.21.15.104:0/4257466953 shutdown_connections 2023-07-14T11:56:59.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.600+0000 7ff956d4c700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff9340662f0 0x7ff9340687a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.600+0000 7ff956d4c700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9500ceff0 0x7ff9500b1560 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.600+0000 7ff956d4c700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff95015bff0 0x7ff9500b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.600+0000 7ff956d4c700 1 --2- 172.21.15.104:0/4257466953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9500cd240 0x7ff9500b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.600+0000 7ff956d4c700 1 -- 172.21.15.104:0/4257466953 >> 172.21.15.104:0/4257466953 conn(0x7ff950060c40 msgr2=0x7ff950070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:59.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.600+0000 7ff956d4c700 1 -- 172.21.15.104:0/4257466953 shutdown_connections 2023-07-14T11:56:59.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.600+0000 7ff956d4c700 1 -- 172.21.15.104:0/4257466953 wait complete. 2023-07-14T11:56:59.608 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-14T11:56:59.608 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:56:59.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.800+0000 7f12b95dc700 1 Processor -- start 2023-07-14T11:56:59.786 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.800+0000 7f12b95dc700 1 -- start start 2023-07-14T11:56:59.787 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12b95dc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b40c9d90 0x7f12b40ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:59.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12b95dc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f12b4159f20 0x7f12b40caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:59.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12b95dc700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b40b0ef0 con 0x7f12b40caf70 2023-07-14T11:56:59.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12b95dc700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b40ce9c0 con 0x7f12b4159f20 2023-07-14T11:56:59.788 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12b95dc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40cee50 con 0x7f12b40c9d90 2023-07-14T11:56:59.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12b8ddb700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f12b40caf70 0x7f12b40c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:40956/0 (socket says 172.21.15.104:40956) 2023-07-14T11:56:59.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12b8ddb700 1 -- 172.21.15.104:0/1830330474 learned_addr learned my addr 172.21.15.104:0/1830330474 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:56:59.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12b3fff700 1 --2- 172.21.15.104:0/1830330474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b40c9d90 0x7f12b40ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:59.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12ab7fe700 1 --2- 172.21.15.104:0/1830330474 >> v2:172.21.15.121:3300/0 conn(0x7f12b4159f20 0x7f12b40caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:59.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12b37fe700 1 -- 172.21.15.104:0/1830330474 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772537087 0 0) 0x7f12b40b0ef0 con 0x7f12b40caf70 2023-07-14T11:56:59.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12b37fe700 1 -- 172.21.15.104:0/1830330474 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12a00035d0 con 0x7f12b40caf70 2023-07-14T11:56:59.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12ab7fe700 1 -- 172.21.15.104:0/1830330474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b40c9d90 msgr2=0x7f12b40ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:56:59.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12ab7fe700 1 --2- 172.21.15.104:0/1830330474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b40c9d90 0x7f12b40ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12ab7fe700 1 -- 172.21.15.104:0/1830330474 >> v1:172.21.15.104:6789/0 conn(0x7f12b40caf70 legacy=0x7f12b40c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:59.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12ab7fe700 1 -- 172.21.15.104:0/1830330474 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b40ca6a0 con 0x7f12b4159f20 2023-07-14T11:56:59.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12b3fff700 1 --2- 172.21.15.104:0/1830330474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b40c9d90 0x7f12b40ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:59.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12ab7fe700 1 --2- 172.21.15.104:0/1830330474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12b4159f20 0x7f12b40caa30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f129c009850 tx=0x7f129c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d038c7d93ec2a5f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:59.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12b37fe700 1 -- 172.21.15.104:0/1830330474 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c011830 con 0x7f12b4159f20 2023-07-14T11:56:59.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12b37fe700 1 -- 172.21.15.104:0/1830330474 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c011990 con 0x7f12b4159f20 2023-07-14T11:56:59.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.804+0000 7f12b37fe700 1 -- 172.21.15.104:0/1830330474 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c011d40 con 0x7f12b4159f20 2023-07-14T11:56:59.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.808+0000 7f12b95dc700 1 -- 172.21.15.104:0/1830330474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12b4159f20 msgr2=0x7f12b40caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:59.795 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.808+0000 7f12b95dc700 1 --2- 172.21.15.104:0/1830330474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12b4159f20 0x7f12b40caa30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f129c009850 tx=0x7f129c009b60 comp rx=0 tx=0).stop 2023-07-14T11:56:59.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.808+0000 7f12b95dc700 1 -- 172.21.15.104:0/1830330474 shutdown_connections 2023-07-14T11:56:59.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.808+0000 7f12b95dc700 1 --2- 172.21.15.104:0/1830330474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12b4159f20 0x7f12b40caa30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.808+0000 7f12b95dc700 1 --2- 172.21.15.104:0/1830330474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b40c9d90 0x7f12b40ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.808+0000 7f12b95dc700 1 -- 172.21.15.104:0/1830330474 >> 172.21.15.104:0/1830330474 conn(0x7f12b406e960 msgr2=0x7f12b406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:59.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.808+0000 7f12b95dc700 1 -- 172.21.15.104:0/1830330474 shutdown_connections 2023-07-14T11:56:59.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.808+0000 7f12b95dc700 1 -- 172.21.15.104:0/1830330474 wait complete. 2023-07-14T11:56:59.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.808+0000 7f12b95dc700 1 Processor -- start 2023-07-14T11:56:59.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.808+0000 7f12b95dc700 1 -- start start 2023-07-14T11:56:59.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12b95dc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12b40c9d90 0x7f12b40c3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:59.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12b95dc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b40caf70 0x7f12b4152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:59.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12b95dc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b4159f20 0x7f12b40b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:59.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12b95dc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b4152da0 con 0x7f12b40caf70 2023-07-14T11:56:59.799 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12b95dc700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40b31c0 con 0x7f12b40c9d90 2023-07-14T11:56:59.799 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12b95dc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40c37c0 con 0x7f12b4159f20 2023-07-14T11:56:59.799 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12ab7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b40caf70 0x7f12b4152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:59.799 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12ab7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b40caf70 0x7f12b4152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59938/0 (socket says 172.21.15.104:59938) 2023-07-14T11:56:59.799 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12ab7fe700 1 -- 172.21.15.104:0/517373119 learned_addr learned my addr 172.21.15.104:0/517373119 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:56:59.799 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12b3fff700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12b40c9d90 0x7f12b40c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:59.799 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12b8ddb700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b4159f20 0x7f12b40b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:59.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12ab7fe700 1 -- 172.21.15.104:0/517373119 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b4159f20 msgr2=0x7f12b40b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:59.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12ab7fe700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b4159f20 0x7f12b40b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12ab7fe700 1 -- 172.21.15.104:0/517373119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12b40c9d90 msgr2=0x7f12b40c3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:59.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12ab7fe700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12b40c9d90 0x7f12b40c3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12ab7fe700 1 -- 172.21.15.104:0/517373119 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a40094b0 con 0x7f12b40caf70 2023-07-14T11:56:59.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12b3fff700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12b40c9d90 0x7f12b40c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:59.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12b8ddb700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b4159f20 0x7f12b40b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:56:59.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12ab7fe700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b40caf70 0x7f12b4152860 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f129c0090b0 tx=0x7f129c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:59.801 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12b17fa700 1 -- 172.21.15.104:0/517373119 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c004480 con 0x7f12b40caf70 2023-07-14T11:56:59.801 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12b17fa700 1 -- 172.21.15.104:0/517373119 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c01f070 con 0x7f12b40caf70 2023-07-14T11:56:59.801 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12b95dc700 1 -- 172.21.15.104:0/517373119 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129c0094b0 con 0x7f12b40caf70 2023-07-14T11:56:59.801 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12b17fa700 1 -- 172.21.15.104:0/517373119 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c003900 con 0x7f12b40caf70 2023-07-14T11:56:59.801 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.812+0000 7f12b95dc700 1 -- 172.21.15.104:0/517373119 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b40c3d80 con 0x7f12b40caf70 2023-07-14T11:56:59.802 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.816+0000 7f12b95dc700 1 -- 172.21.15.104:0/517373119 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b404e500 con 0x7f12b40caf70 2023-07-14T11:56:59.802 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.816+0000 7f12b17fa700 1 -- 172.21.15.104:0/517373119 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f129c021030 con 0x7f12b40caf70 2023-07-14T11:56:59.803 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.820+0000 7f12b17fa700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12940662f0 0x7f12940687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:59.803 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.820+0000 7f12b17fa700 1 -- 172.21.15.104:0/517373119 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f129c084690 con 0x7f12b40caf70 2023-07-14T11:56:59.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.820+0000 7f12b3fff700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12940662f0 0x7f12940687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:59.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.820+0000 7f12b3fff700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12940662f0 0x7f12940687a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009c60 tx=0x7f12a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:59.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:56:59.824+0000 7f12b17fa700 1 -- 172.21.15.104:0/517373119 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f129c051940 con 0x7f12b40caf70 2023-07-14T11:57:00.018 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.032+0000 7f12b95dc700 1 -- 172.21.15.104:0/517373119 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12b4070c90 con 0x7f12b40caf70 2023-07-14T11:57:00.019 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.036+0000 7f12b17fa700 1 -- 172.21.15.104:0/517373119 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4722 (secure 0 0 0) 0x7f129c054f60 con 0x7f12b40caf70 2023-07-14T11:57:00.020 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:57:00.020 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/4186544624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":4186544624},{"type":"v1","addr":"172.21.15.104:6837","nonce":4186544624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4234575863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4234575863},{"type":"v1","addr":"172.21.15.121:6835","nonce":4234575863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6220,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4127455794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4127455794},{"type":"v1","addr":"172.21.15.121:6833","nonce":4127455794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:56:11.123432+0000","modified":"2023-07-14T11:56:14.719624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/2705739882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2705739882},{"type":"v1","addr":"172.21.15.104:6836","nonce":2705739882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-14T11:57:00.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.040+0000 7f12b95dc700 1 -- 172.21.15.104:0/517373119 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12940662f0 msgr2=0x7f12940687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:00.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.040+0000 7f12b95dc700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12940662f0 0x7f12940687a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009c60 tx=0x7f12a4008040 comp rx=0 tx=0).stop 2023-07-14T11:57:00.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.040+0000 7f12b95dc700 1 -- 172.21.15.104:0/517373119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b40caf70 msgr2=0x7f12b4152860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:00.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.040+0000 7f12b95dc700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b40caf70 0x7f12b4152860 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f129c0090b0 tx=0x7f129c004770 comp rx=0 tx=0).stop 2023-07-14T11:57:00.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.040+0000 7f12b95dc700 1 -- 172.21.15.104:0/517373119 shutdown_connections 2023-07-14T11:57:00.024 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.040+0000 7f12b95dc700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f12940662f0 0x7f12940687a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.040+0000 7f12b95dc700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12b40caf70 0x7f12b4152860 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.040+0000 7f12b95dc700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12b4159f20 0x7f12b40b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.040+0000 7f12b95dc700 1 --2- 172.21.15.104:0/517373119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12b40c9d90 0x7f12b40c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.040+0000 7f12b95dc700 1 -- 172.21.15.104:0/517373119 >> 172.21.15.104:0/517373119 conn(0x7f12b406e960 msgr2=0x7f12b415a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:00.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.040+0000 7f12b95dc700 1 -- 172.21.15.104:0/517373119 shutdown_connections 2023-07-14T11:57:00.025 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.040+0000 7f12b95dc700 1 -- 172.21.15.104:0/517373119 wait complete. 2023-07-14T11:57:00.026 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 27 2023-07-14T11:57:00.045 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:57:00.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.232+0000 7fdbb2c9f700 1 Processor -- start 2023-07-14T11:57:00.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.232+0000 7fdbb2c9f700 1 -- start start 2023-07-14T11:57:00.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.236+0000 7fdbb2c9f700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdbac0cd240 0x7fdbac0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:00.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.236+0000 7fdbb2c9f700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbac0af0a0 con 0x7fdbac0ceff0 2023-07-14T11:57:00.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.236+0000 7fdbb2c9f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbac0cf720 con 0x7fdbac15bff0 2023-07-14T11:57:00.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.236+0000 7fdbb2c9f700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbac0af2f0 con 0x7fdbac0cd240 2023-07-14T11:57:00.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.236+0000 7fdbb249e700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fdbac0ceff0 0x7fdbac0afdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:40964/0 (socket says 172.21.15.104:40964) 2023-07-14T11:57:00.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.236+0000 7fdbb249e700 1 -- 172.21.15.104:0/1416977752 learned_addr learned my addr 172.21.15.104:0/1416977752 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:00.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.236+0000 7fdbb149c700 1 --2- 172.21.15.104:0/1416977752 >> v2:172.21.15.104:3300/0 conn(0x7fdbac0cd240 0x7fdbac0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:00.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.236+0000 7fdbb149c700 1 -- 172.21.15.104:0/1416977752 >> v1:172.21.15.121:6789/0 conn(0x7fdbac15bff0 legacy=0x7fdbac0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:00.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.236+0000 7fdbb149c700 1 -- 172.21.15.104:0/1416977752 >> v1:172.21.15.104:6789/0 conn(0x7fdbac0ceff0 legacy=0x7fdbac0afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:00.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.236+0000 7fdbb0c9b700 1 -- 172.21.15.104:0/1416977752 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251735584 0 0) 0x7fdbac0af0a0 con 0x7fdbac0ceff0 2023-07-14T11:57:00.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.236+0000 7fdbb149c700 1 -- 172.21.15.104:0/1416977752 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac0af5d0 con 0x7fdbac0cd240 2023-07-14T11:57:00.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.236+0000 7fdbb149c700 1 --2- 172.21.15.104:0/1416977752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0cd240 0x7fdbac0ceab0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009850 tx=0x7fdb9c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66565cd3c6ab040 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:00.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.236+0000 7fdbb0c9b700 1 -- 172.21.15.104:0/1416977752 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c011830 con 0x7fdbac0cd240 2023-07-14T11:57:00.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.236+0000 7fdbb0c9b700 1 -- 172.21.15.104:0/1416977752 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9c011990 con 0x7fdbac0cd240 2023-07-14T11:57:00.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.236+0000 7fdbb0c9b700 1 -- 172.21.15.104:0/1416977752 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c011d20 con 0x7fdbac0cd240 2023-07-14T11:57:00.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.240+0000 7fdbb2c9f700 1 -- 172.21.15.104:0/1416977752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0cd240 msgr2=0x7fdbac0ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:00.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.240+0000 7fdbb2c9f700 1 --2- 172.21.15.104:0/1416977752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0cd240 0x7fdbac0ceab0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009850 tx=0x7fdb9c009b60 comp rx=0 tx=0).stop 2023-07-14T11:57:00.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.240+0000 7fdbb2c9f700 1 -- 172.21.15.104:0/1416977752 shutdown_connections 2023-07-14T11:57:00.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.240+0000 7fdbb2c9f700 1 --2- 172.21.15.104:0/1416977752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0cd240 0x7fdbac0ceab0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.240+0000 7fdbb2c9f700 1 -- 172.21.15.104:0/1416977752 >> 172.21.15.104:0/1416977752 conn(0x7fdbac060c40 msgr2=0x7fdbac09b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:00.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.240+0000 7fdbb2c9f700 1 -- 172.21.15.104:0/1416977752 shutdown_connections 2023-07-14T11:57:00.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.240+0000 7fdbb2c9f700 1 -- 172.21.15.104:0/1416977752 wait complete. 2023-07-14T11:57:00.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb2c9f700 1 Processor -- start 2023-07-14T11:57:00.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb2c9f700 1 -- start start 2023-07-14T11:57:00.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb2c9f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbac0cd240 0x7fdbac15c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:00.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb2c9f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0ceff0 0x7fdbac1580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:00.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb2c9f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdbac15bff0 0x7fdbac15e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:00.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb2c9f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0b5310 con 0x7fdbac0ceff0 2023-07-14T11:57:00.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb2c9f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0b55f0 con 0x7fdbac0cd240 2023-07-14T11:57:00.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb2c9f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac1585f0 con 0x7fdbac15bff0 2023-07-14T11:57:00.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb149c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0ceff0 0x7fdbac1580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:00.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb149c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0ceff0 0x7fdbac1580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59952/0 (socket says 172.21.15.104:59952) 2023-07-14T11:57:00.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb149c700 1 -- 172.21.15.104:0/2812176994 learned_addr learned my addr 172.21.15.104:0/2812176994 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:00.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb1c9d700 1 --2- 172.21.15.104:0/2812176994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbac0cd240 0x7fdbac15c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:00.231 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb249e700 1 --2- 172.21.15.104:0/2812176994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdbac15bff0 0x7fdbac15e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:00.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb149c700 1 -- 172.21.15.104:0/2812176994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdbac15bff0 msgr2=0x7fdbac15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:00.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb149c700 1 --2- 172.21.15.104:0/2812176994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdbac15bff0 0x7fdbac15e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb149c700 1 -- 172.21.15.104:0/2812176994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbac0cd240 msgr2=0x7fdbac15c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:00.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb149c700 1 --2- 172.21.15.104:0/2812176994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbac0cd240 0x7fdbac15c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.244+0000 7fdbb149c700 1 -- 172.21.15.104:0/2812176994 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c0094b0 con 0x7fdbac0ceff0 2023-07-14T11:57:00.232 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.248+0000 7fdbb149c700 1 --2- 172.21.15.104:0/2812176994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0ceff0 0x7fdbac1580b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c000c00 tx=0x7fdb9c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:00.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.248+0000 7fdba27fc700 1 -- 172.21.15.104:0/2812176994 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c0045d0 con 0x7fdbac0ceff0 2023-07-14T11:57:00.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.248+0000 7fdba27fc700 1 -- 172.21.15.104:0/2812176994 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9c01e070 con 0x7fdbac0ceff0 2023-07-14T11:57:00.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.248+0000 7fdbb2c9f700 1 -- 172.21.15.104:0/2812176994 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbac15eb90 con 0x7fdbac0ceff0 2023-07-14T11:57:00.233 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.248+0000 7fdba27fc700 1 -- 172.21.15.104:0/2812176994 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c003920 con 0x7fdbac0ceff0 2023-07-14T11:57:00.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.248+0000 7fdbb2c9f700 1 -- 172.21.15.104:0/2812176994 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbac15f150 con 0x7fdbac0ceff0 2023-07-14T11:57:00.234 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.248+0000 7fdbb2c9f700 1 -- 172.21.15.104:0/2812176994 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbac0b5760 con 0x7fdbac0ceff0 2023-07-14T11:57:00.235 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.252+0000 7fdba27fc700 1 -- 172.21.15.104:0/2812176994 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fdb9c003a80 con 0x7fdbac0ceff0 2023-07-14T11:57:00.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.252+0000 7fdba27fc700 1 --2- 172.21.15.104:0/2812176994 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdb90066320 0x7fdb900687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:00.237 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.252+0000 7fdba27fc700 1 -- 172.21.15.104:0/2812176994 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6527+0+0 (secure 0 0 0) 0x7fdb9c0850c0 con 0x7fdbac0ceff0 2023-07-14T11:57:00.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.252+0000 7fdbb1c9d700 1 --2- 172.21.15.104:0/2812176994 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdb90066320 0x7fdb900687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:00.238 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.252+0000 7fdbb1c9d700 1 --2- 172.21.15.104:0/2812176994 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdb90066320 0x7fdb900687d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdbac1f9920 tx=0x7fdba8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:00.242 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.256+0000 7fdba27fc700 1 -- 172.21.15.104:0/2812176994 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fdb9c052360 con 0x7fdbac0ceff0 2023-07-14T11:57:00.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.464+0000 7fdbb2c9f700 1 -- 172.21.15.104:0/2812176994 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbac04e500 con 0x7fdbac0ceff0 2023-07-14T11:57:00.449 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.464+0000 7fdba27fc700 1 -- 172.21.15.104:0/2812176994 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4722 (secure 0 0 0) 0x7fdb9c055980 con 0x7fdbac0ceff0 2023-07-14T11:57:00.449 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:57:00.450 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/4186544624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":4186544624},{"type":"v1","addr":"172.21.15.104:6837","nonce":4186544624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4234575863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4234575863},{"type":"v1","addr":"172.21.15.121:6835","nonce":4234575863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6220,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4127455794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4127455794},{"type":"v1","addr":"172.21.15.121:6833","nonce":4127455794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:56:11.123432+0000","modified":"2023-07-14T11:56:14.719624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/2705739882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2705739882},{"type":"v1","addr":"172.21.15.104:6836","nonce":2705739882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-14T11:57:00.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.468+0000 7fdbb2c9f700 1 -- 172.21.15.104:0/2812176994 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdb90066320 msgr2=0x7fdb900687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:00.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.468+0000 7fdbb2c9f700 1 --2- 172.21.15.104:0/2812176994 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdb90066320 0x7fdb900687d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdbac1f9920 tx=0x7fdba8009040 comp rx=0 tx=0).stop 2023-07-14T11:57:00.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.468+0000 7fdbb2c9f700 1 -- 172.21.15.104:0/2812176994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0ceff0 msgr2=0x7fdbac1580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:00.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.468+0000 7fdbb2c9f700 1 --2- 172.21.15.104:0/2812176994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0ceff0 0x7fdbac1580b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c000c00 tx=0x7fdb9c004770 comp rx=0 tx=0).stop 2023-07-14T11:57:00.454 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.468+0000 7fdbb2c9f700 1 -- 172.21.15.104:0/2812176994 shutdown_connections 2023-07-14T11:57:00.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.468+0000 7fdbb2c9f700 1 --2- 172.21.15.104:0/2812176994 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdb90066320 0x7fdb900687d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.468+0000 7fdbb2c9f700 1 --2- 172.21.15.104:0/2812176994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac0ceff0 0x7fdbac1580b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.468+0000 7fdbb2c9f700 1 --2- 172.21.15.104:0/2812176994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdbac15bff0 0x7fdbac15e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.468+0000 7fdbb2c9f700 1 --2- 172.21.15.104:0/2812176994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbac0cd240 0x7fdbac15c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.468+0000 7fdbb2c9f700 1 -- 172.21.15.104:0/2812176994 >> 172.21.15.104:0/2812176994 conn(0x7fdbac060c40 msgr2=0x7fdbac0b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:00.455 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.468+0000 7fdbb2c9f700 1 -- 172.21.15.104:0/2812176994 shutdown_connections 2023-07-14T11:57:00.456 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.468+0000 7fdbb2c9f700 1 -- 172.21.15.104:0/2812176994 wait complete. 2023-07-14T11:57:00.456 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 27 2023-07-14T11:57:00.475 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:57:00.640 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.656+0000 7fd207565700 1 Processor -- start 2023-07-14T11:57:00.641 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.656+0000 7fd207565700 1 -- start start 2023-07-14T11:57:00.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd207565700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd2000cc630 0x7fd20013eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:00.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd207565700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd200141b00 0x7fd20006f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:00.643 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd207565700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2000ccaa0 con 0x7fd20013e580 2023-07-14T11:57:00.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd207565700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2000cd060 con 0x7fd2000cc630 2023-07-14T11:57:00.644 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd207565700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20006e960 con 0x7fd200141b00 2023-07-14T11:57:00.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd205301700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fd20013e580 0x7fd2000ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53638/0 (socket says 172.21.15.104:53638) 2023-07-14T11:57:00.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd205301700 1 -- 172.21.15.104:0/4059158606 learned_addr learned my addr 172.21.15.104:0/4059158606 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:00.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd205b02700 1 --2- 172.21.15.104:0/4059158606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd200141b00 0x7fd20006f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:00.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd204b00700 1 --2- 172.21.15.104:0/4059158606 >> v2:172.21.15.121:3300/0 conn(0x7fd2000cc630 0x7fd20013eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:00.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd1f7fff700 1 -- 172.21.15.104:0/4059158606 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478630957 0 0) 0x7fd2000ccaa0 con 0x7fd20013e580 2023-07-14T11:57:00.645 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd1f7fff700 1 -- 172.21.15.104:0/4059158606 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1ec0057e0 con 0x7fd20013e580 2023-07-14T11:57:00.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd205b02700 1 -- 172.21.15.104:0/4059158606 >> v2:172.21.15.121:3300/0 conn(0x7fd2000cc630 msgr2=0x7fd20013eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:00.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd205b02700 1 --2- 172.21.15.104:0/4059158606 >> v2:172.21.15.121:3300/0 conn(0x7fd2000cc630 0x7fd20013eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd205b02700 1 -- 172.21.15.104:0/4059158606 >> v1:172.21.15.104:6789/0 conn(0x7fd20013e580 legacy=0x7fd2000ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:00.646 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd205b02700 1 -- 172.21.15.104:0/4059158606 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2000ccd80 con 0x7fd200141b00 2023-07-14T11:57:00.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd205b02700 1 --2- 172.21.15.104:0/4059158606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd200141b00 0x7fd20006f370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800aea0 tx=0x7fd1f800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=518b724dedce4eca server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:00.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd1f7fff700 1 -- 172.21.15.104:0/4059158606 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f800a2d0 con 0x7fd200141b00 2023-07-14T11:57:00.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd1f7fff700 1 -- 172.21.15.104:0/4059158606 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f800c910 con 0x7fd200141b00 2023-07-14T11:57:00.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd1f7fff700 1 -- 172.21.15.104:0/4059158606 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f800ccc0 con 0x7fd200141b00 2023-07-14T11:57:00.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd207565700 1 -- 172.21.15.104:0/4059158606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd200141b00 msgr2=0x7fd20006f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:00.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.660+0000 7fd207565700 1 --2- 172.21.15.104:0/4059158606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd200141b00 0x7fd20006f370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800aea0 tx=0x7fd1f800c330 comp rx=0 tx=0).stop 2023-07-14T11:57:00.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.664+0000 7fd207565700 1 -- 172.21.15.104:0/4059158606 shutdown_connections 2023-07-14T11:57:00.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.664+0000 7fd207565700 1 --2- 172.21.15.104:0/4059158606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd200141b00 0x7fd20006f370 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.664+0000 7fd207565700 1 --2- 172.21.15.104:0/4059158606 >> v2:172.21.15.121:3300/0 conn(0x7fd2000cc630 0x7fd20013eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.648 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.664+0000 7fd207565700 1 -- 172.21.15.104:0/4059158606 >> 172.21.15.104:0/4059158606 conn(0x7fd200069800 msgr2=0x7fd200069bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:00.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.664+0000 7fd207565700 1 -- 172.21.15.104:0/4059158606 shutdown_connections 2023-07-14T11:57:00.649 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.664+0000 7fd207565700 1 -- 172.21.15.104:0/4059158606 wait complete. 2023-07-14T11:57:00.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.664+0000 7fd207565700 1 Processor -- start 2023-07-14T11:57:00.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd207565700 1 -- start start 2023-07-14T11:57:00.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd207565700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2000cc630 0x7fd20014e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:00.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd207565700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd20013e580 0x7fd200157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:00.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd207565700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd20014fb40 0x7fd20014ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:00.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd207565700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd200150500 con 0x7fd2000cc630 2023-07-14T11:57:00.653 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd207565700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2001e2820 con 0x7fd20014fb40 2023-07-14T11:57:00.654 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd207565700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2001e2b00 con 0x7fd20013e580 2023-07-14T11:57:00.655 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd205301700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2000cc630 0x7fd20014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:00.655 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd205301700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2000cc630 0x7fd20014e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46402/0 (socket says 172.21.15.104:46402) 2023-07-14T11:57:00.655 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd205301700 1 -- 172.21.15.104:0/2754210824 learned_addr learned my addr 172.21.15.104:0/2754210824 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:00.655 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd204b00700 1 --2- 172.21.15.104:0/2754210824 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd20013e580 0x7fd200157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:00.655 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd205b02700 1 --2- 172.21.15.104:0/2754210824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd20014fb40 0x7fd20014ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:00.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd205301700 1 -- 172.21.15.104:0/2754210824 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd20013e580 msgr2=0x7fd200157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:00.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd205301700 1 --2- 172.21.15.104:0/2754210824 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd20013e580 0x7fd200157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd205301700 1 -- 172.21.15.104:0/2754210824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd20014fb40 msgr2=0x7fd20014ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:00.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd205301700 1 --2- 172.21.15.104:0/2754210824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd20014fb40 0x7fd20014ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd205301700 1 -- 172.21.15.104:0/2754210824 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f800ab80 con 0x7fd2000cc630 2023-07-14T11:57:00.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd205301700 1 --2- 172.21.15.104:0/2754210824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2000cc630 0x7fd20014e060 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00c4f0 tx=0x7fd1fc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:00.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.668+0000 7fd1f5ffb700 1 -- 172.21.15.104:0/2754210824 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc005d90 con 0x7fd2000cc630 2023-07-14T11:57:00.656 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.672+0000 7fd1f5ffb700 1 -- 172.21.15.104:0/2754210824 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1fc0070d0 con 0x7fd2000cc630 2023-07-14T11:57:00.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.672+0000 7fd207565700 1 -- 172.21.15.104:0/2754210824 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2000c9130 con 0x7fd2000cc630 2023-07-14T11:57:00.657 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.672+0000 7fd1f5ffb700 1 -- 172.21.15.104:0/2754210824 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc007440 con 0x7fd2000cc630 2023-07-14T11:57:00.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.672+0000 7fd207565700 1 -- 172.21.15.104:0/2754210824 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2000c96f0 con 0x7fd2000cc630 2023-07-14T11:57:00.658 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.672+0000 7fd207565700 1 -- 172.21.15.104:0/2754210824 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd20004dcb0 con 0x7fd2000cc630 2023-07-14T11:57:00.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.676+0000 7fd1f5ffb700 1 -- 172.21.15.104:0/2754210824 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd1fc0075a0 con 0x7fd2000cc630 2023-07-14T11:57:00.660 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.676+0000 7fd1f5ffb700 1 --2- 172.21.15.104:0/2754210824 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd1e4066320 0x7fd1e40687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:00.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.676+0000 7fd1f5ffb700 1 -- 172.21.15.104:0/2754210824 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6527+0+0 (secure 0 0 0) 0x7fd1fc0832a0 con 0x7fd2000cc630 2023-07-14T11:57:00.661 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.676+0000 7fd204b00700 1 --2- 172.21.15.104:0/2754210824 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd1e4066320 0x7fd1e40687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:00.662 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.676+0000 7fd204b00700 1 --2- 172.21.15.104:0/2754210824 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd1e4066320 0x7fd1e40687d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd2000cabf0 tx=0x7fd1f0009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:00.667 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.684+0000 7fd1f5ffb700 1 -- 172.21.15.104:0/2754210824 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fd1fc050580 con 0x7fd2000cc630 2023-07-14T11:57:00.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.856+0000 7fd207565700 1 -- 172.21.15.104:0/2754210824 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd20006cd80 con 0x7fd2000cc630 2023-07-14T11:57:00.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.860+0000 7fd1f5ffb700 1 -- 172.21.15.104:0/2754210824 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18774 (secure 0 0 0) 0x7fd1fc053ba0 con 0x7fd2000cc630 2023-07-14T11:57:00.846 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:57:00.846 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":61,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:56:55.760804+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:56:08.755551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-14T11:56:10.132688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.9","mappings":[{"from":7,"to":2}]},{"pgid":"10.1d","mappings":[{"from":4,"to":3}]},{"pgid":"10.2f","mappings":[{"from":4,"to":5}]},{"pgid":"10.32","mappings":[{"from":7,"to":3}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:57:00.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.864+0000 7fd207565700 1 -- 172.21.15.104:0/2754210824 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd1e4066320 msgr2=0x7fd1e40687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:00.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.864+0000 7fd207565700 1 --2- 172.21.15.104:0/2754210824 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd1e4066320 0x7fd1e40687d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd2000cabf0 tx=0x7fd1f0009240 comp rx=0 tx=0).stop 2023-07-14T11:57:00.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.864+0000 7fd207565700 1 -- 172.21.15.104:0/2754210824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2000cc630 msgr2=0x7fd20014e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:00.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.864+0000 7fd207565700 1 --2- 172.21.15.104:0/2754210824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2000cc630 0x7fd20014e060 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00c4f0 tx=0x7fd1fc00c800 comp rx=0 tx=0).stop 2023-07-14T11:57:00.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.864+0000 7fd207565700 1 -- 172.21.15.104:0/2754210824 shutdown_connections 2023-07-14T11:57:00.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.864+0000 7fd207565700 1 --2- 172.21.15.104:0/2754210824 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd1e4066320 0x7fd1e40687d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.864+0000 7fd207565700 1 --2- 172.21.15.104:0/2754210824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd20014fb40 0x7fd20014ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.864+0000 7fd207565700 1 --2- 172.21.15.104:0/2754210824 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd20013e580 0x7fd200157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.864+0000 7fd207565700 1 --2- 172.21.15.104:0/2754210824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2000cc630 0x7fd20014e060 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:00.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.864+0000 7fd207565700 1 -- 172.21.15.104:0/2754210824 >> 172.21.15.104:0/2754210824 conn(0x7fd200069800 msgr2=0x7fd200156780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:00.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.864+0000 7fd207565700 1 -- 172.21.15.104:0/2754210824 shutdown_connections 2023-07-14T11:57:00.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:00.864+0000 7fd207565700 1 -- 172.21.15.104:0/2754210824 wait complete. 2023-07-14T11:57:00.871 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-14T11:57:01.048 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.064+0000 7fb1d441d700 1 Processor -- start 2023-07-14T11:57:01.049 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.064+0000 7fb1d441d700 1 -- start start 2023-07-14T11:57:01.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d441d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb1cc0b1530 0x7fb1cc0d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:01.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d441d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc0d1260 0x7fb1cc0b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:01.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d441d700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1cc15c2f0 con 0x7fb1cc15bf20 2023-07-14T11:57:01.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d441d700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1cc0b5b30 con 0x7fb1cc0b1530 2023-07-14T11:57:01.053 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d441d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc070830 con 0x7fb1cc0d1260 2023-07-14T11:57:01.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d21b9700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb1cc15bf20 0x7fb1cc0b5420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53640/0 (socket says 172.21.15.104:53640) 2023-07-14T11:57:01.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d21b9700 1 -- 172.21.15.104:0/966774366 learned_addr learned my addr 172.21.15.104:0/966774366 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:01.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d19b8700 1 --2- 172.21.15.104:0/966774366 >> v2:172.21.15.121:3300/0 conn(0x7fb1cc0b1530 0x7fb1cc0d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:01.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d29ba700 1 --2- 172.21.15.104:0/966774366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc0d1260 0x7fb1cc0b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:01.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d11b7700 1 -- 172.21.15.104:0/966774366 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293853897 0 0) 0x7fb1cc15c2f0 con 0x7fb1cc15bf20 2023-07-14T11:57:01.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d11b7700 1 -- 172.21.15.104:0/966774366 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1b80035d0 con 0x7fb1cc15bf20 2023-07-14T11:57:01.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d11b7700 1 -- 172.21.15.104:0/966774366 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1226735274 0 0) 0x7fb1b80035d0 con 0x7fb1cc15bf20 2023-07-14T11:57:01.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d11b7700 1 -- 172.21.15.104:0/966774366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc0d1260 msgr2=0x7fb1cc0b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:01.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d11b7700 1 --2- 172.21.15.104:0/966774366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc0d1260 0x7fb1cc0b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:01.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d11b7700 1 -- 172.21.15.104:0/966774366 >> v2:172.21.15.121:3300/0 conn(0x7fb1cc0b1530 msgr2=0x7fb1cc0d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:01.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d11b7700 1 --2- 172.21.15.104:0/966774366 >> v2:172.21.15.121:3300/0 conn(0x7fb1cc0b1530 0x7fb1cc0d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:01.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d11b7700 1 -- 172.21.15.104:0/966774366 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc070b10 con 0x7fb1cc15bf20 2023-07-14T11:57:01.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d11b7700 1 -- 172.21.15.104:0/966774366 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7fb1c8001670 con 0x7fb1cc15bf20 2023-07-14T11:57:01.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d11b7700 1 -- 172.21.15.104:0/966774366 >> v1:172.21.15.104:6789/0 conn(0x7fb1cc15bf20 legacy=0x7fb1cc0b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:01.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d11b7700 1 --2- 172.21.15.104:0/966774366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1b8003a20 0x7fb1b8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:01.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d11b7700 1 --2- 172.21.15.104:0/966774366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc0d1260 0x7fb1b8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:01.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d11b7700 1 --2- 172.21.15.104:0/966774366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1b8004db0 0x7fb1b8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:01.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.068+0000 7fb1d11b7700 1 -- 172.21.15.104:0/966774366 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc0b5b30 con 0x7fb1b8003a20 2023-07-14T11:57:01.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d11b7700 1 -- 172.21.15.104:0/966774366 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc070830 con 0x7fb1b8004db0 2023-07-14T11:57:01.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d11b7700 1 -- 172.21.15.104:0/966774366 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b800ba30 con 0x7fb1cc0d1260 2023-07-14T11:57:01.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d21b9700 1 --2- 172.21.15.104:0/966774366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1b8003a20 0x7fb1b8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:01.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d29ba700 1 --2- 172.21.15.104:0/966774366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1b8004db0 0x7fb1b8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:01.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d21b9700 1 -- 172.21.15.104:0/966774366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc0d1260 msgr2=0x7fb1b8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:01.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d21b9700 1 --2- 172.21.15.104:0/966774366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc0d1260 0x7fb1b8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:01.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d21b9700 1 -- 172.21.15.104:0/966774366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1b8004db0 msgr2=0x7fb1b8005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:01.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d21b9700 1 --2- 172.21.15.104:0/966774366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1b8004db0 0x7fb1b8005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:01.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d21b9700 1 -- 172.21.15.104:0/966774366 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b80035d0 con 0x7fb1b8003a20 2023-07-14T11:57:01.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d29ba700 1 --2- 172.21.15.104:0/966774366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1b8004db0 0x7fb1b8005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:57:01.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d21b9700 1 --2- 172.21.15.104:0/966774366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1b8003a20 0x7fb1b8003df0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8001eb0 tx=0x7fb1b80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7054a4be48c6d609 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:01.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d11b7700 1 -- 172.21.15.104:0/966774366 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8004c90 con 0x7fb1b8003a20 2023-07-14T11:57:01.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d11b7700 1 -- 172.21.15.104:0/966774366 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c8005b80 con 0x7fb1b8003a20 2023-07-14T11:57:01.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d11b7700 1 -- 172.21.15.104:0/966774366 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80080d0 con 0x7fb1b8003a20 2023-07-14T11:57:01.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d441d700 1 -- 172.21.15.104:0/966774366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1b8003a20 msgr2=0x7fb1b8003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:01.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d441d700 1 --2- 172.21.15.104:0/966774366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1b8003a20 0x7fb1b8003df0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8001eb0 tx=0x7fb1b80058a0 comp rx=0 tx=0).stop 2023-07-14T11:57:01.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d29ba700 1 -- 172.21.15.104:0/966774366 reap_dead start 2023-07-14T11:57:01.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d441d700 1 -- 172.21.15.104:0/966774366 shutdown_connections 2023-07-14T11:57:01.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d441d700 1 -- 172.21.15.104:0/966774366 >> 172.21.15.104:0/966774366 conn(0x7fb1cc06fc80 msgr2=0x7fb1cc070080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:01.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d441d700 1 -- 172.21.15.104:0/966774366 shutdown_connections 2023-07-14T11:57:01.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.072+0000 7fb1d441d700 1 -- 172.21.15.104:0/966774366 wait complete. 2023-07-14T11:57:01.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d441d700 1 Processor -- start 2023-07-14T11:57:01.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d441d700 1 -- start start 2023-07-14T11:57:01.063 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d441d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1b8003a20 0x7fb1cc0b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:01.063 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d441d700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc14dff0 0x7fb1cc0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:01.063 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d441d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1cc0cb830 0x7fb1cc0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:01.063 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d441d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc0cc210 con 0x7fb1b8003a20 2023-07-14T11:57:01.065 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d441d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc0b32e0 con 0x7fb1cc0cb830 2023-07-14T11:57:01.065 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d441d700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc0b2760 con 0x7fb1cc14dff0 2023-07-14T11:57:01.065 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d21b9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1b8003a20 0x7fb1cc0b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:01.065 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d21b9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1b8003a20 0x7fb1cc0b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46434/0 (socket says 172.21.15.104:46434) 2023-07-14T11:57:01.065 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d21b9700 1 -- 172.21.15.104:0/1819023895 learned_addr learned my addr 172.21.15.104:0/1819023895 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:01.065 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d29ba700 1 --2- 172.21.15.104:0/1819023895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1cc0cb830 0x7fb1cc0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:01.066 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d19b8700 1 --2- 172.21.15.104:0/1819023895 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc14dff0 0x7fb1cc0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:01.066 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d21b9700 1 -- 172.21.15.104:0/1819023895 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc14dff0 msgr2=0x7fb1cc0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:01.066 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d21b9700 1 --2- 172.21.15.104:0/1819023895 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc14dff0 0x7fb1cc0b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:01.066 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d21b9700 1 -- 172.21.15.104:0/1819023895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1cc0cb830 msgr2=0x7fb1cc0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:01.066 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d21b9700 1 --2- 172.21.15.104:0/1819023895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1cc0cb830 0x7fb1cc0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:01.066 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d21b9700 1 -- 172.21.15.104:0/1819023895 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc0b2a40 con 0x7fb1b8003a20 2023-07-14T11:57:01.066 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d21b9700 1 --2- 172.21.15.104:0/1819023895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1b8003a20 0x7fb1cc0b1f30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80057b0 tx=0x7fb1c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:01.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1c2ffd700 1 -- 172.21.15.104:0/1819023895 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8025070 con 0x7fb1b8003a20 2023-07-14T11:57:01.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1c2ffd700 1 -- 172.21.15.104:0/1819023895 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c8005e70 con 0x7fb1b8003a20 2023-07-14T11:57:01.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1c2ffd700 1 -- 172.21.15.104:0/1819023895 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8001ae0 con 0x7fb1b8003a20 2023-07-14T11:57:01.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.076+0000 7fb1d441d700 1 -- 172.21.15.104:0/1819023895 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1cc0b2d20 con 0x7fb1b8003a20 2023-07-14T11:57:01.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.080+0000 7fb1d441d700 1 -- 172.21.15.104:0/1819023895 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1cc0b2470 con 0x7fb1b8003a20 2023-07-14T11:57:01.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.080+0000 7fb1d441d700 1 -- 172.21.15.104:0/1819023895 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1cc15d5a0 con 0x7fb1b8003a20 2023-07-14T11:57:01.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.080+0000 7fb1c2ffd700 1 -- 172.21.15.104:0/1819023895 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb1c80080d0 con 0x7fb1b8003a20 2023-07-14T11:57:01.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.084+0000 7fb1c2ffd700 1 --2- 172.21.15.104:0/1819023895 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1b00662f0 0x7fb1b00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:01.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.084+0000 7fb1c2ffd700 1 -- 172.21.15.104:0/1819023895 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6527+0+0 (secure 0 0 0) 0x7fb1c8085220 con 0x7fb1b8003a20 2023-07-14T11:57:01.069 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.084+0000 7fb1d19b8700 1 --2- 172.21.15.104:0/1819023895 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1b00662f0 0x7fb1b00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:01.069 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.084+0000 7fb1d19b8700 1 --2- 172.21.15.104:0/1819023895 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1b00662f0 0x7fb1b00687a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc14e660 tx=0x7fb1bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:01.073 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.088+0000 7fb1c2ffd700 1 -- 172.21.15.104:0/1819023895 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fb1cc15d5a0 con 0x7fb1b8003a20 2023-07-14T11:57:01.275 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.288+0000 7fb1d441d700 1 -- 172.21.15.104:0/1819023895 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb1cc15d5a0 con 0x7fb1b8003a20 2023-07-14T11:57:01.276 INFO:tasks.ceph.mon.a.smithi104.stderr:2023-07-14T11:57:01.292+0000 7fbde97f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T11:57:01.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.304+0000 7fb1c2ffd700 1 -- 172.21.15.104:0/1819023895 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb1cc15d5a0 con 0x7fb1b8003a20 2023-07-14T11:57:01.292 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.308+0000 7fb1d441d700 1 -- 172.21.15.104:0/1819023895 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1b00662f0 msgr2=0x7fb1b00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:01.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.308+0000 7fb1d441d700 1 --2- 172.21.15.104:0/1819023895 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1b00662f0 0x7fb1b00687a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc14e660 tx=0x7fb1bc008040 comp rx=0 tx=0).stop 2023-07-14T11:57:01.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.308+0000 7fb1d441d700 1 -- 172.21.15.104:0/1819023895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1b8003a20 msgr2=0x7fb1cc0b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:01.293 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.308+0000 7fb1d441d700 1 --2- 172.21.15.104:0/1819023895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1b8003a20 0x7fb1cc0b1f30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80057b0 tx=0x7fb1c8004ab0 comp rx=0 tx=0).stop 2023-07-14T11:57:01.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.308+0000 7fb1d441d700 1 -- 172.21.15.104:0/1819023895 shutdown_connections 2023-07-14T11:57:01.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.308+0000 7fb1d441d700 1 --2- 172.21.15.104:0/1819023895 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fb1b00662f0 0x7fb1b00687a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:01.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.308+0000 7fb1d441d700 1 --2- 172.21.15.104:0/1819023895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1cc0cb830 0x7fb1cc0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:01.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.308+0000 7fb1d441d700 1 --2- 172.21.15.104:0/1819023895 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc14dff0 0x7fb1cc0b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:01.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.308+0000 7fb1d441d700 1 --2- 172.21.15.104:0/1819023895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1b8003a20 0x7fb1cc0b1f30 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:01.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.308+0000 7fb1d441d700 1 -- 172.21.15.104:0/1819023895 >> 172.21.15.104:0/1819023895 conn(0x7fb1cc06fc80 msgr2=0x7fb1cc0cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:01.294 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.308+0000 7fb1d441d700 1 -- 172.21.15.104:0/1819023895 shutdown_connections 2023-07-14T11:57:01.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.308+0000 7fb1d441d700 1 -- 172.21.15.104:0/1819023895 wait complete. 2023-07-14T11:57:01.295 INFO:teuthology.orchestra.run.smithi104.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-14T11:57:01.296 INFO:tasks.ceph.mds.c.smithi104.stderr: -3142> 2023-07-14T11:56:16.209+0000 7f11ba383700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-14T11:57:01.312 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-14T11:57:01.357 INFO:tasks.ceph.mds.c.smithi104.stdout:starting mds.c at 2023-07-14T11:57:01.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.500+0000 7f7dce715700 1 Processor -- start 2023-07-14T11:57:01.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.500+0000 7f7dce715700 1 -- start start 2023-07-14T11:57:01.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dce715700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7dc80d07e0 0x7f7dc81669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:01.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dce715700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc80d1a60 con 0x7f7dc80b0ee0 2023-07-14T11:57:01.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dce715700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc80d1ef0 con 0x7f7dc80cd260 2023-07-14T11:57:01.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dce715700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc80d0c60 con 0x7f7dc80d07e0 2023-07-14T11:57:01.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dcd713700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f7dc80b0ee0 0x7f7dc80ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53664/0 (socket says 172.21.15.104:53664) 2023-07-14T11:57:01.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dcd713700 1 -- 172.21.15.104:0/439149689 learned_addr learned my addr 172.21.15.104:0/439149689 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:01.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dcdf14700 1 --2- 172.21.15.104:0/439149689 >> v2:172.21.15.121:3300/0 conn(0x7f7dc80d07e0 0x7f7dc81669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:01.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dbffff700 1 -- 172.21.15.104:0/439149689 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497021327 0 0) 0x7f7dc80d1a60 con 0x7f7dc80b0ee0 2023-07-14T11:57:01.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dbffff700 1 -- 172.21.15.104:0/439149689 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7db40035d0 con 0x7f7dc80b0ee0 2023-07-14T11:57:01.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dcdf14700 1 -- 172.21.15.104:0/439149689 >> v1:172.21.15.121:6789/0 conn(0x7f7dc80cd260 legacy=0x7f7dc80d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:01.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dbffff700 1 -- 172.21.15.104:0/439149689 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719713420 0 0) 0x7f7dc80d1ef0 con 0x7f7dc80cd260 2023-07-14T11:57:01.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dcdf14700 1 -- 172.21.15.104:0/439149689 >> v1:172.21.15.104:6789/0 conn(0x7f7dc80b0ee0 legacy=0x7f7dc80ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:01.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dcdf14700 1 -- 172.21.15.104:0/439149689 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc80d0e50 con 0x7f7dc80d07e0 2023-07-14T11:57:01.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dcdf14700 1 --2- 172.21.15.104:0/439149689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc80d07e0 0x7f7dc81669b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009d90 tx=0x7f7dc000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bee301ffd7a764fe server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:01.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dbffff700 1 -- 172.21.15.104:0/439149689 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc000e070 con 0x7f7dc80d07e0 2023-07-14T11:57:01.491 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dbffff700 1 -- 172.21.15.104:0/439149689 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc000bb20 con 0x7f7dc80d07e0 2023-07-14T11:57:01.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.504+0000 7f7dbffff700 1 -- 172.21.15.104:0/439149689 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc000bed0 con 0x7f7dc80d07e0 2023-07-14T11:57:01.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.508+0000 7f7dce715700 1 -- 172.21.15.104:0/439149689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc80d07e0 msgr2=0x7f7dc81669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:01.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.508+0000 7f7dce715700 1 --2- 172.21.15.104:0/439149689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc80d07e0 0x7f7dc81669b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009d90 tx=0x7f7dc000b330 comp rx=0 tx=0).stop 2023-07-14T11:57:01.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.508+0000 7f7dce715700 1 -- 172.21.15.104:0/439149689 shutdown_connections 2023-07-14T11:57:01.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.508+0000 7f7dce715700 1 --2- 172.21.15.104:0/439149689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc80d07e0 0x7f7dc81669b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:01.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.508+0000 7f7dce715700 1 -- 172.21.15.104:0/439149689 >> 172.21.15.104:0/439149689 conn(0x7f7dc8060c40 msgr2=0x7f7dc809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:01.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.508+0000 7f7dce715700 1 -- 172.21.15.104:0/439149689 shutdown_connections 2023-07-14T11:57:01.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.508+0000 7f7dce715700 1 -- 172.21.15.104:0/439149689 wait complete. 2023-07-14T11:57:01.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.508+0000 7f7dce715700 1 Processor -- start 2023-07-14T11:57:01.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.508+0000 7f7dce715700 1 -- start start 2023-07-14T11:57:01.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dce715700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dc80b0ee0 0x7f7dc815b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:01.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dce715700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7dc80cd260 0x7f7dc81f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:01.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dce715700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc80d07e0 0x7f7dc80cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:01.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dce715700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc815baf0 con 0x7f7dc80b0ee0 2023-07-14T11:57:01.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dce715700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc80ca750 con 0x7f7dc80d07e0 2023-07-14T11:57:01.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dce715700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc81f2040 con 0x7f7dc80cd260 2023-07-14T11:57:01.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dcd713700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dc80b0ee0 0x7f7dc815b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:01.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dcdf14700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc80d07e0 0x7f7dc80cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:01.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dcd713700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dc80b0ee0 0x7f7dc815b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46450/0 (socket says 172.21.15.104:46450) 2023-07-14T11:57:01.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dcd713700 1 -- 172.21.15.104:0/4256049896 learned_addr learned my addr 172.21.15.104:0/4256049896 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:01.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dccf12700 1 --2- 172.21.15.104:0/4256049896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7dc80cd260 0x7f7dc81f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:01.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dcdf14700 1 -- 172.21.15.104:0/4256049896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7dc80cd260 msgr2=0x7f7dc81f1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:01.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dcdf14700 1 --2- 172.21.15.104:0/4256049896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7dc80cd260 0x7f7dc81f1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:01.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dcdf14700 1 -- 172.21.15.104:0/4256049896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dc80b0ee0 msgr2=0x7f7dc815b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:01.500 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dcdf14700 1 --2- 172.21.15.104:0/4256049896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dc80b0ee0 0x7f7dc815b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:01.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dcdf14700 1 -- 172.21.15.104:0/4256049896 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc0009a70 con 0x7f7dc80d07e0 2023-07-14T11:57:01.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dccf12700 1 --2- 172.21.15.104:0/4256049896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7dc80cd260 0x7f7dc81f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:57:01.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dcd713700 1 --2- 172.21.15.104:0/4256049896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dc80b0ee0 0x7f7dc815b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:57:01.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dcdf14700 1 --2- 172.21.15.104:0/4256049896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc80d07e0 0x7f7dc80cb6f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7dc808b380 tx=0x7f7dc0004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:01.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dbdffb700 1 -- 172.21.15.104:0/4256049896 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0004930 con 0x7f7dc80d07e0 2023-07-14T11:57:01.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dbdffb700 1 -- 172.21.15.104:0/4256049896 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc0020070 con 0x7f7dc80d07e0 2023-07-14T11:57:01.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dbdffb700 1 -- 172.21.15.104:0/4256049896 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00276c0 con 0x7f7dc80d07e0 2023-07-14T11:57:01.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dce715700 1 -- 172.21.15.104:0/4256049896 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc80c9b40 con 0x7f7dc80d07e0 2023-07-14T11:57:01.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.512+0000 7f7dce715700 1 -- 172.21.15.104:0/4256049896 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc80ca100 con 0x7f7dc80d07e0 2023-07-14T11:57:01.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.516+0000 7f7dce715700 1 -- 172.21.15.104:0/4256049896 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc80ca900 con 0x7f7dc80d07e0 2023-07-14T11:57:01.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.516+0000 7f7dbdffb700 1 -- 172.21.15.104:0/4256049896 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7dc0027820 con 0x7f7dc80d07e0 2023-07-14T11:57:01.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.520+0000 7f7dbdffb700 1 --2- 172.21.15.104:0/4256049896 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7dac0662f0 0x7f7dac0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:01.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.520+0000 7f7dbdffb700 1 -- 172.21.15.104:0/4256049896 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6613+0+0 (secure 0 0 0) 0x7f7dc0022070 con 0x7f7dc80d07e0 2023-07-14T11:57:01.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.520+0000 7f7dcd713700 1 --2- 172.21.15.104:0/4256049896 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7dac0662f0 0x7f7dac0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:01.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.520+0000 7f7dcd713700 1 --2- 172.21.15.104:0/4256049896 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7dac0662f0 0x7f7dac0687a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4005930 tx=0x7f7dc4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:01.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.524+0000 7f7dbdffb700 1 -- 172.21.15.104:0/4256049896 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f7dc0053500 con 0x7f7dc80d07e0 2023-07-14T11:57:01.717 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:01.732+0000 7f7dce715700 1 -- 172.21.15.104:0/4256049896 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7dc80ca900 con 0x7f7dc80d07e0 2023-07-14T11:57:02.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.304+0000 7f7dbdffb700 1 -- 172.21.15.104:0/4256049896 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7dc80ca900 con 0x7f7dc80d07e0 2023-07-14T11:57:02.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.308+0000 7f7dce715700 1 -- 172.21.15.104:0/4256049896 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7dac0662f0 msgr2=0x7f7dac0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:02.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.308+0000 7f7dce715700 1 --2- 172.21.15.104:0/4256049896 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7dac0662f0 0x7f7dac0687a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4005930 tx=0x7f7dc4009040 comp rx=0 tx=0).stop 2023-07-14T11:57:02.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.308+0000 7f7dce715700 1 -- 172.21.15.104:0/4256049896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc80d07e0 msgr2=0x7f7dc80cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:02.295 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.308+0000 7f7dce715700 1 --2- 172.21.15.104:0/4256049896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc80d07e0 0x7f7dc80cb6f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7dc808b380 tx=0x7f7dc0004af0 comp rx=0 tx=0).stop 2023-07-14T11:57:02.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.308+0000 7f7dce715700 1 -- 172.21.15.104:0/4256049896 shutdown_connections 2023-07-14T11:57:02.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.308+0000 7f7dce715700 1 --2- 172.21.15.104:0/4256049896 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f7dac0662f0 0x7f7dac0687a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:02.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.308+0000 7f7dce715700 1 --2- 172.21.15.104:0/4256049896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc80d07e0 0x7f7dc80cb6f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:02.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.308+0000 7f7dce715700 1 --2- 172.21.15.104:0/4256049896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7dc80cd260 0x7f7dc81f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:02.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.308+0000 7f7dce715700 1 --2- 172.21.15.104:0/4256049896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dc80b0ee0 0x7f7dc815b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:02.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.308+0000 7f7dce715700 1 -- 172.21.15.104:0/4256049896 >> 172.21.15.104:0/4256049896 conn(0x7f7dc8060c40 msgr2=0x7f7dc8071470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:02.296 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.312+0000 7f7dce715700 1 -- 172.21.15.104:0/4256049896 shutdown_connections 2023-07-14T11:57:02.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.312+0000 7f7dce715700 1 -- 172.21.15.104:0/4256049896 wait complete. 2023-07-14T11:57:02.317 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-14T11:57:02.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.556+0000 7ff7309d5700 1 Processor -- start 2023-07-14T11:57:02.543 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.560+0000 7ff7309d5700 1 -- start start 2023-07-14T11:57:02.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.560+0000 7ff7309d5700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff72c0c8c60 0x7ff72c15d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:02.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.560+0000 7ff7309d5700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72c0c9100 con 0x7ff72c0cac60 2023-07-14T11:57:02.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.560+0000 7ff7309d5700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72c0c9340 con 0x7ff72c06cac0 2023-07-14T11:57:02.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.560+0000 7ff7309d5700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72c06c670 con 0x7ff72c0c8c60 2023-07-14T11:57:02.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff729d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff72c0c8c60 0x7ff72c15d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:02.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff729d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff72c0c8c60 0x7ff72c15d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46462/0 (socket says 172.21.15.104:46462) 2023-07-14T11:57:02.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff729d9b700 1 -- 172.21.15.104:0/3141587445 learned_addr learned my addr 172.21.15.104:0/3141587445 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:02.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff729d9b700 1 -- 172.21.15.104:0/3141587445 >> v1:172.21.15.121:6789/0 conn(0x7ff72c06cac0 legacy=0x7ff72c06cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:02.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff729d9b700 1 -- 172.21.15.104:0/3141587445 >> v1:172.21.15.104:6789/0 conn(0x7ff72c0cac60 legacy=0x7ff72c0cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:02.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff729d9b700 1 -- 172.21.15.104:0/3141587445 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c0c9620 con 0x7ff72c0c8c60 2023-07-14T11:57:02.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff729d9b700 1 --2- 172.21.15.104:0/3141587445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff72c0c8c60 0x7ff72c15d150 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff72000bbb0 tx=0x7ff72000bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeba6dabe8057823 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:02.552 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff72959a700 1 -- 172.21.15.104:0/3141587445 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72000dd50 con 0x7ff72c0c8c60 2023-07-14T11:57:02.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff72959a700 1 -- 172.21.15.104:0/3141587445 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff720004030 con 0x7ff72c0c8c60 2023-07-14T11:57:02.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff72959a700 1 -- 172.21.15.104:0/3141587445 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7200043e0 con 0x7ff72c0c8c60 2023-07-14T11:57:02.553 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff7309d5700 1 -- 172.21.15.104:0/3141587445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff72c0c8c60 msgr2=0x7ff72c15d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:02.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff7309d5700 1 --2- 172.21.15.104:0/3141587445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff72c0c8c60 0x7ff72c15d150 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff72000bbb0 tx=0x7ff72000bec0 comp rx=0 tx=0).stop 2023-07-14T11:57:02.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff7309d5700 1 -- 172.21.15.104:0/3141587445 shutdown_connections 2023-07-14T11:57:02.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff7309d5700 1 --2- 172.21.15.104:0/3141587445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff72c0c8c60 0x7ff72c15d150 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:02.554 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff7309d5700 1 -- 172.21.15.104:0/3141587445 >> 172.21.15.104:0/3141587445 conn(0x7ff72c0afd60 msgr2=0x7ff72c0b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:02.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff7309d5700 1 -- 172.21.15.104:0/3141587445 shutdown_connections 2023-07-14T11:57:02.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.564+0000 7ff7309d5700 1 -- 172.21.15.104:0/3141587445 wait complete. 2023-07-14T11:57:02.555 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.568+0000 7ff7309d5700 1 Processor -- start 2023-07-14T11:57:02.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.568+0000 7ff7309d5700 1 -- start start 2023-07-14T11:57:02.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.568+0000 7ff7309d5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff72c06cac0 0x7ff72c0c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:02.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.568+0000 7ff7309d5700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff72c0c8c60 0x7ff72c1509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:02.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.568+0000 7ff7309d5700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c0cac60 0x7ff72c1534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:02.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.568+0000 7ff7309d5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c150f00 con 0x7ff72c06cac0 2023-07-14T11:57:02.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.568+0000 7ff7309d5700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c1f59c0 con 0x7ff72c0cac60 2023-07-14T11:57:02.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.568+0000 7ff7309d5700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c0c3cd0 con 0x7ff72c0c8c60 2023-07-14T11:57:02.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.568+0000 7ff72a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff72c06cac0 0x7ff72c0c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:02.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.568+0000 7ff72a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff72c06cac0 0x7ff72c0c47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46466/0 (socket says 172.21.15.104:46466) 2023-07-14T11:57:02.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.568+0000 7ff72a59c700 1 -- 172.21.15.104:0/534758646 learned_addr learned my addr 172.21.15.104:0/534758646 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:02.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.568+0000 7ff72ad9d700 1 --2- 172.21.15.104:0/534758646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c0cac60 0x7ff72c1534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:02.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.568+0000 7ff729d9b700 1 --2- 172.21.15.104:0/534758646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff72c0c8c60 0x7ff72c1509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:02.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.572+0000 7ff72a59c700 1 -- 172.21.15.104:0/534758646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff72c0c8c60 msgr2=0x7ff72c1509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:02.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.572+0000 7ff72a59c700 1 --2- 172.21.15.104:0/534758646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff72c0c8c60 0x7ff72c1509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:02.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.572+0000 7ff72a59c700 1 -- 172.21.15.104:0/534758646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c0cac60 msgr2=0x7ff72c1534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:02.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.572+0000 7ff72a59c700 1 --2- 172.21.15.104:0/534758646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c0cac60 0x7ff72c1534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:02.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.572+0000 7ff72a59c700 1 -- 172.21.15.104:0/534758646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72000b810 con 0x7ff72c06cac0 2023-07-14T11:57:02.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.572+0000 7ff72ad9d700 1 --2- 172.21.15.104:0/534758646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c0cac60 0x7ff72c1534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:57:02.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.572+0000 7ff72a59c700 1 --2- 172.21.15.104:0/534758646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff72c06cac0 0x7ff72c0c47b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00c500 tx=0x7ff71c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:02.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.572+0000 7ff71affd700 1 -- 172.21.15.104:0/534758646 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0052d0 con 0x7ff72c06cac0 2023-07-14T11:57:02.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.572+0000 7ff71affd700 1 -- 172.21.15.104:0/534758646 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71c005430 con 0x7ff72c06cac0 2023-07-14T11:57:02.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.572+0000 7ff7309d5700 1 -- 172.21.15.104:0/534758646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72c0c3fb0 con 0x7ff72c06cac0 2023-07-14T11:57:02.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.572+0000 7ff71affd700 1 -- 172.21.15.104:0/534758646 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0055a0 con 0x7ff72c06cac0 2023-07-14T11:57:02.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.572+0000 7ff7309d5700 1 -- 172.21.15.104:0/534758646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72c0c3980 con 0x7ff72c06cac0 2023-07-14T11:57:02.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.572+0000 7ff7309d5700 1 -- 172.21.15.104:0/534758646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72c0c3b10 con 0x7ff72c06cac0 2023-07-14T11:57:02.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.576+0000 7ff71affd700 1 -- 172.21.15.104:0/534758646 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff71c0036a0 con 0x7ff72c06cac0 2023-07-14T11:57:02.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.576+0000 7ff71affd700 1 --2- 172.21.15.104:0/534758646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff70c0662f0 0x7ff70c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:02.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.576+0000 7ff71affd700 1 -- 172.21.15.104:0/534758646 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6613+0+0 (secure 0 0 0) 0x7ff71c082fe0 con 0x7ff72c06cac0 2023-07-14T11:57:02.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.576+0000 7ff729d9b700 1 --2- 172.21.15.104:0/534758646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff70c0662f0 0x7ff70c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:02.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.576+0000 7ff729d9b700 1 --2- 172.21.15.104:0/534758646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff70c0662f0 0x7ff70c0687a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff720000c00 tx=0x7ff72001c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:02.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.580+0000 7ff71affd700 1 -- 172.21.15.104:0/534758646 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff71c050200 con 0x7ff72c06cac0 2023-07-14T11:57:02.739 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:02.756+0000 7ff7309d5700 1 -- 172.21.15.104:0/534758646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff72c0c9100 con 0x7ff72c06cac0 2023-07-14T11:57:03.659 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.676+0000 7ff71affd700 1 -- 172.21.15.104:0/534758646 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v63) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff72c0c9100 con 0x7ff72c06cac0 2023-07-14T11:57:03.663 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.680+0000 7ff7309d5700 1 -- 172.21.15.104:0/534758646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff70c0662f0 msgr2=0x7ff70c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:03.664 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.680+0000 7ff7309d5700 1 --2- 172.21.15.104:0/534758646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff70c0662f0 0x7ff70c0687a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff720000c00 tx=0x7ff72001c040 comp rx=0 tx=0).stop 2023-07-14T11:57:03.664 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.680+0000 7ff7309d5700 1 -- 172.21.15.104:0/534758646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff72c06cac0 msgr2=0x7ff72c0c47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:03.664 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.680+0000 7ff7309d5700 1 --2- 172.21.15.104:0/534758646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff72c06cac0 0x7ff72c0c47b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00c500 tx=0x7ff71c00c810 comp rx=0 tx=0).stop 2023-07-14T11:57:03.665 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.680+0000 7ff7309d5700 1 -- 172.21.15.104:0/534758646 shutdown_connections 2023-07-14T11:57:03.665 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.680+0000 7ff7309d5700 1 --2- 172.21.15.104:0/534758646 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff70c0662f0 0x7ff70c0687a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:03.665 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.680+0000 7ff7309d5700 1 --2- 172.21.15.104:0/534758646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c0cac60 0x7ff72c1534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:03.665 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.680+0000 7ff7309d5700 1 --2- 172.21.15.104:0/534758646 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff72c0c8c60 0x7ff72c1509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:03.665 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.680+0000 7ff7309d5700 1 --2- 172.21.15.104:0/534758646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff72c06cac0 0x7ff72c0c47b0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:03.665 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.680+0000 7ff7309d5700 1 -- 172.21.15.104:0/534758646 >> 172.21.15.104:0/534758646 conn(0x7ff72c0afd60 msgr2=0x7ff72c06d740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:03.666 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.680+0000 7ff7309d5700 1 -- 172.21.15.104:0/534758646 shutdown_connections 2023-07-14T11:57:03.666 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.680+0000 7ff7309d5700 1 -- 172.21.15.104:0/534758646 wait complete. 2023-07-14T11:57:03.667 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'cephfs_metadata' removed 2023-07-14T11:57:03.687 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-14T11:57:03.868 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.884+0000 7efd34ffe700 1 Processor -- start 2023-07-14T11:57:03.869 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.884+0000 7efd34ffe700 1 -- start start 2023-07-14T11:57:03.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.884+0000 7efd34ffe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7efd300b0ee0 0x7efd300ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:03.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.884+0000 7efd34ffe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efd300d07e0 0x7efd300af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:03.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.884+0000 7efd34ffe700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd300cd4e0 con 0x7efd300cd090 2023-07-14T11:57:03.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.884+0000 7efd34ffe700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd300d0c60 con 0x7efd300b0ee0 2023-07-14T11:57:03.870 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.884+0000 7efd34ffe700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd300d1a60 con 0x7efd300d07e0 2023-07-14T11:57:03.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.884+0000 7efd2f7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7efd300b0ee0 0x7efd300ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:03.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.884+0000 7efd2effd700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7efd300cd090 0x7efd300d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53686/0 (socket says 172.21.15.104:53686) 2023-07-14T11:57:03.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.884+0000 7efd2effd700 1 -- 172.21.15.104:0/4290821729 learned_addr learned my addr 172.21.15.104:0/4290821729 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:03.871 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.884+0000 7efd2ffff700 1 --2- 172.21.15.104:0/4290821729 >> v2:172.21.15.121:3300/0 conn(0x7efd300d07e0 0x7efd300af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:03.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd2f7fe700 1 -- 172.21.15.104:0/4290821729 >> v2:172.21.15.121:3300/0 conn(0x7efd300d07e0 msgr2=0x7efd300af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:03.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd2f7fe700 1 --2- 172.21.15.104:0/4290821729 >> v2:172.21.15.121:3300/0 conn(0x7efd300d07e0 0x7efd300af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:03.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd2f7fe700 1 -- 172.21.15.104:0/4290821729 >> v1:172.21.15.104:6789/0 conn(0x7efd300cd090 legacy=0x7efd300d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:03.872 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd2e7fc700 1 -- 172.21.15.104:0/4290821729 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962570656 0 0) 0x7efd24001110 con 0x7efd300cd090 2023-07-14T11:57:03.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd2f7fe700 1 -- 172.21.15.104:0/4290821729 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd300d1ef0 con 0x7efd300b0ee0 2023-07-14T11:57:03.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd2f7fe700 1 --2- 172.21.15.104:0/4290821729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd300b0ee0 0x7efd300ccb50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7efd300b4fb0 tx=0x7efd20009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6ec179a5093056 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:03.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd2e7fc700 1 -- 172.21.15.104:0/4290821729 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20011830 con 0x7efd300b0ee0 2023-07-14T11:57:03.873 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd2e7fc700 1 -- 172.21.15.104:0/4290821729 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd20011990 con 0x7efd300b0ee0 2023-07-14T11:57:03.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd2e7fc700 1 -- 172.21.15.104:0/4290821729 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20011d20 con 0x7efd300b0ee0 2023-07-14T11:57:03.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd34ffe700 1 -- 172.21.15.104:0/4290821729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd300b0ee0 msgr2=0x7efd300ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:03.874 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd34ffe700 1 --2- 172.21.15.104:0/4290821729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd300b0ee0 0x7efd300ccb50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7efd300b4fb0 tx=0x7efd20009b30 comp rx=0 tx=0).stop 2023-07-14T11:57:03.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd34ffe700 1 -- 172.21.15.104:0/4290821729 shutdown_connections 2023-07-14T11:57:03.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd34ffe700 1 --2- 172.21.15.104:0/4290821729 >> v2:172.21.15.121:3300/0 conn(0x7efd300d07e0 0x7efd300af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:03.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd34ffe700 1 --2- 172.21.15.104:0/4290821729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd300b0ee0 0x7efd300ccb50 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:03.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd34ffe700 1 -- 172.21.15.104:0/4290821729 >> 172.21.15.104:0/4290821729 conn(0x7efd30060c40 msgr2=0x7efd3009b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:03.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd34ffe700 1 -- 172.21.15.104:0/4290821729 shutdown_connections 2023-07-14T11:57:03.875 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.888+0000 7efd34ffe700 1 -- 172.21.15.104:0/4290821729 wait complete. 2023-07-14T11:57:03.878 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.892+0000 7efd34ffe700 1 Processor -- start 2023-07-14T11:57:03.879 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd34ffe700 1 -- start start 2023-07-14T11:57:03.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd34ffe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd300b0ee0 0x7efd301f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:03.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd34ffe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd300cd090 0x7efd300b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:03.880 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd34ffe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd300d07e0 0x7efd3009c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:03.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd34ffe700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd300b39c0 con 0x7efd300cd090 2023-07-14T11:57:03.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd34ffe700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd301f69e0 con 0x7efd300b0ee0 2023-07-14T11:57:03.881 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd34ffe700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd300b3c00 con 0x7efd300d07e0 2023-07-14T11:57:03.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd2effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd300cd090 0x7efd300b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:03.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd2effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd300cd090 0x7efd300b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46482/0 (socket says 172.21.15.104:46482) 2023-07-14T11:57:03.882 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd2effd700 1 -- 172.21.15.104:0/4246975280 learned_addr learned my addr 172.21.15.104:0/4246975280 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:03.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd2ffff700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd300d07e0 0x7efd3009c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:03.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd2f7fe700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd300b0ee0 0x7efd301f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:03.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd2effd700 1 -- 172.21.15.104:0/4246975280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd300d07e0 msgr2=0x7efd3009c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:03.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd2effd700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd300d07e0 0x7efd3009c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:03.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd2effd700 1 -- 172.21.15.104:0/4246975280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd300b0ee0 msgr2=0x7efd301f87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:03.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd2effd700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd300b0ee0 0x7efd301f87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:03.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd2effd700 1 -- 172.21.15.104:0/4246975280 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd200094b0 con 0x7efd300cd090 2023-07-14T11:57:03.883 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd2ffff700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd300d07e0 0x7efd3009c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:57:03.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd2f7fe700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd300b0ee0 0x7efd301f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:57:03.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd2effd700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd300cd090 0x7efd300b3480 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7efd2400c750 tx=0x7efd2400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:03.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd17fff700 1 -- 172.21.15.104:0/4246975280 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd24004ca0 con 0x7efd300cd090 2023-07-14T11:57:03.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd17fff700 1 -- 172.21.15.104:0/4246975280 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd24004e20 con 0x7efd300cd090 2023-07-14T11:57:03.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd34ffe700 1 -- 172.21.15.104:0/4246975280 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd300b3ee0 con 0x7efd300cd090 2023-07-14T11:57:03.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd17fff700 1 -- 172.21.15.104:0/4246975280 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd24007310 con 0x7efd300cd090 2023-07-14T11:57:03.884 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.896+0000 7efd34ffe700 1 -- 172.21.15.104:0/4246975280 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd300b44a0 con 0x7efd300cd090 2023-07-14T11:57:03.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.900+0000 7efd34ffe700 1 -- 172.21.15.104:0/4246975280 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3004e500 con 0x7efd300cd090 2023-07-14T11:57:03.885 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.900+0000 7efd17fff700 1 -- 172.21.15.104:0/4246975280 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7efd24007470 con 0x7efd300cd090 2023-07-14T11:57:03.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.904+0000 7efd17fff700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efd1c0662f0 0x7efd1c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:03.887 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.904+0000 7efd17fff700 1 -- 172.21.15.104:0/4246975280 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6146+0+0 (secure 0 0 0) 0x7efd24083990 con 0x7efd300cd090 2023-07-14T11:57:03.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.904+0000 7efd2f7fe700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efd1c0662f0 0x7efd1c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:03.888 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.904+0000 7efd2f7fe700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efd1c0662f0 0x7efd1c0687a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7efd200090b0 tx=0x7efd20008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:03.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:03.908+0000 7efd17fff700 1 -- 172.21.15.104:0/4246975280 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7efd24050da0 con 0x7efd300cd090 2023-07-14T11:57:04.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:04.084+0000 7efd34ffe700 1 -- 172.21.15.104:0/4246975280 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efd300cd4e0 con 0x7efd300cd090 2023-07-14T11:57:04.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:04.860+0000 7efd17fff700 1 -- 172.21.15.104:0/4246975280 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v64) v1 ==== 169+0+0 (secure 0 0 0) 0x7efd300cd4e0 con 0x7efd300cd090 2023-07-14T11:57:04.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:04.864+0000 7efd34ffe700 1 -- 172.21.15.104:0/4246975280 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efd1c0662f0 msgr2=0x7efd1c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:04.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:04.864+0000 7efd34ffe700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efd1c0662f0 0x7efd1c0687a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7efd200090b0 tx=0x7efd20008030 comp rx=0 tx=0).stop 2023-07-14T11:57:04.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:04.864+0000 7efd34ffe700 1 -- 172.21.15.104:0/4246975280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd300cd090 msgr2=0x7efd300b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:04.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:04.864+0000 7efd34ffe700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd300cd090 0x7efd300b3480 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7efd2400c750 tx=0x7efd2400cb10 comp rx=0 tx=0).stop 2023-07-14T11:57:04.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:04.864+0000 7efd34ffe700 1 -- 172.21.15.104:0/4246975280 shutdown_connections 2023-07-14T11:57:04.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:04.864+0000 7efd34ffe700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7efd1c0662f0 0x7efd1c0687a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:04.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:04.864+0000 7efd34ffe700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd300cd090 0x7efd300b3480 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:04.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:04.864+0000 7efd34ffe700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd300d07e0 0x7efd3009c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:04.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:04.864+0000 7efd34ffe700 1 --2- 172.21.15.104:0/4246975280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd300b0ee0 0x7efd301f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:04.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:04.864+0000 7efd34ffe700 1 -- 172.21.15.104:0/4246975280 >> 172.21.15.104:0/4246975280 conn(0x7efd30060c40 msgr2=0x7efd3015a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:04.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:04.864+0000 7efd34ffe700 1 -- 172.21.15.104:0/4246975280 shutdown_connections 2023-07-14T11:57:04.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:04.864+0000 7efd34ffe700 1 -- 172.21.15.104:0/4246975280 wait complete. 2023-07-14T11:57:04.850 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'cephfs_data' removed 2023-07-14T11:57:04.869 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-14T11:57:05.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.068+0000 7f55c2bfe700 1 Processor -- start 2023-07-14T11:57:05.052 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.068+0000 7f55c2bfe700 1 -- start start 2023-07-14T11:57:05.054 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.068+0000 7f55c2bfe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55bc1597a0 0x7f55bc0c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:05.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.068+0000 7f55c2bfe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55bc159b70 0x7f55bc0705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:05.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.068+0000 7f55c2bfe700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc071560 con 0x7f55bc0cada0 2023-07-14T11:57:05.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.068+0000 7f55c2bfe700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55bc070cc0 con 0x7f55bc159b70 2023-07-14T11:57:05.055 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.068+0000 7f55c2bfe700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc070fa0 con 0x7f55bc1597a0 2023-07-14T11:57:05.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c23fd700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f55bc0cada0 0x7f55bc0cb1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53694/0 (socket says 172.21.15.104:53694) 2023-07-14T11:57:05.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c23fd700 1 -- 172.21.15.104:0/917293068 learned_addr learned my addr 172.21.15.104:0/917293068 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:05.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c13fb700 1 --2- 172.21.15.104:0/917293068 >> v2:172.21.15.121:3300/0 conn(0x7f55bc159b70 0x7f55bc0705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:05.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c1bfc700 1 --2- 172.21.15.104:0/917293068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55bc1597a0 0x7f55bc0c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:05.056 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 -- 172.21.15.104:0/917293068 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366604223 0 0) 0x7f55bc071560 con 0x7f55bc0cada0 2023-07-14T11:57:05.057 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 -- 172.21.15.104:0/917293068 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55a80035d0 con 0x7f55bc0cada0 2023-07-14T11:57:05.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 -- 172.21.15.104:0/917293068 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1645510557 0 0) 0x7f55a80035d0 con 0x7f55bc0cada0 2023-07-14T11:57:05.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 -- 172.21.15.104:0/917293068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55bc1597a0 msgr2=0x7f55bc0c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:05.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 --2- 172.21.15.104:0/917293068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55bc1597a0 0x7f55bc0c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 -- 172.21.15.104:0/917293068 >> v2:172.21.15.121:3300/0 conn(0x7f55bc159b70 msgr2=0x7f55bc0705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:05.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 --2- 172.21.15.104:0/917293068 >> v2:172.21.15.121:3300/0 conn(0x7f55bc159b70 0x7f55bc0705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.058 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 -- 172.21.15.104:0/917293068 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc071280 con 0x7f55bc0cada0 2023-07-14T11:57:05.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 -- 172.21.15.104:0/917293068 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f55b4001670 con 0x7f55bc0cada0 2023-07-14T11:57:05.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 -- 172.21.15.104:0/917293068 >> v1:172.21.15.104:6789/0 conn(0x7f55bc0cada0 legacy=0x7f55bc0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:05.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 --2- 172.21.15.104:0/917293068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55a8003a20 0x7f55a8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:05.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 --2- 172.21.15.104:0/917293068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55bc1597a0 0x7f55a8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:05.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 --2- 172.21.15.104:0/917293068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a8004db0 0x7f55a8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:05.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 -- 172.21.15.104:0/917293068 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc070cc0 con 0x7f55a8003a20 2023-07-14T11:57:05.059 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 -- 172.21.15.104:0/917293068 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc070fa0 con 0x7f55a8004db0 2023-07-14T11:57:05.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 -- 172.21.15.104:0/917293068 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a800ba30 con 0x7f55bc1597a0 2023-07-14T11:57:05.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c1bfc700 1 --2- 172.21.15.104:0/917293068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55a8003a20 0x7f55a8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:05.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c13fb700 1 --2- 172.21.15.104:0/917293068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55bc1597a0 0x7f55a8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:05.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c23fd700 1 --2- 172.21.15.104:0/917293068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a8004db0 0x7f55a8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:05.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c1bfc700 1 -- 172.21.15.104:0/917293068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55bc1597a0 msgr2=0x7f55a8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:05.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c1bfc700 1 --2- 172.21.15.104:0/917293068 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55bc1597a0 0x7f55a8004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c1bfc700 1 -- 172.21.15.104:0/917293068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a8004db0 msgr2=0x7f55a8005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:05.060 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c1bfc700 1 --2- 172.21.15.104:0/917293068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a8004db0 0x7f55a8005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c1bfc700 1 -- 172.21.15.104:0/917293068 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a80035d0 con 0x7f55a8003a20 2023-07-14T11:57:05.061 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c1bfc700 1 --2- 172.21.15.104:0/917293068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55a8003a20 0x7f55a8003e30 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009fc0 tx=0x7f55a80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17579f6595e55d6f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:05.062 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 -- 172.21.15.104:0/917293068 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b800ec30 con 0x7f55a8003a20 2023-07-14T11:57:05.062 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 -- 172.21.15.104:0/917293068 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b800f040 con 0x7f55a8003a20 2023-07-14T11:57:05.062 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.072+0000 7f55c0bfa700 1 -- 172.21.15.104:0/917293068 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b8014480 con 0x7f55a8003a20 2023-07-14T11:57:05.062 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.076+0000 7f55c2bfe700 1 -- 172.21.15.104:0/917293068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55a8003a20 msgr2=0x7f55a8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:05.062 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.076+0000 7f55c2bfe700 1 --2- 172.21.15.104:0/917293068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55a8003a20 0x7f55a8003e30 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009fc0 tx=0x7f55a80058a0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.062 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.076+0000 7f55c23fd700 1 -- 172.21.15.104:0/917293068 reap_dead start 2023-07-14T11:57:05.062 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.076+0000 7f55c2bfe700 1 -- 172.21.15.104:0/917293068 shutdown_connections 2023-07-14T11:57:05.063 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.076+0000 7f55c2bfe700 1 -- 172.21.15.104:0/917293068 >> 172.21.15.104:0/917293068 conn(0x7f55bc0afc60 msgr2=0x7f55bc0b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:05.063 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.076+0000 7f55c2bfe700 1 -- 172.21.15.104:0/917293068 shutdown_connections 2023-07-14T11:57:05.063 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.076+0000 7f55c2bfe700 1 -- 172.21.15.104:0/917293068 wait complete. 2023-07-14T11:57:05.063 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.076+0000 7f55c2bfe700 1 Processor -- start 2023-07-14T11:57:05.063 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c2bfe700 1 -- start start 2023-07-14T11:57:05.064 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c2bfe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55a8003a20 0x7f55bc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:05.064 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c2bfe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55bc0b2c80 0x7f55bc1556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:05.064 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c2bfe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55bc0ac940 0x7f55bc0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:05.064 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c2bfe700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc06dca0 con 0x7f55bc0ac940 2023-07-14T11:57:05.065 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c2bfe700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0ad300 con 0x7f55bc0b2c80 2023-07-14T11:57:05.065 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c2bfe700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc156010 con 0x7f55a8003a20 2023-07-14T11:57:05.066 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c23fd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55bc0ac940 0x7f55bc0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:05.066 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c23fd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55bc0ac940 0x7f55bc0acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46496/0 (socket says 172.21.15.104:46496) 2023-07-14T11:57:05.066 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c23fd700 1 -- 172.21.15.104:0/1989846664 learned_addr learned my addr 172.21.15.104:0/1989846664 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:05.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c13fb700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55bc0b2c80 0x7f55bc1556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:05.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c23fd700 1 -- 172.21.15.104:0/1989846664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55a8003a20 msgr2=0x7f55bc06d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:05.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c1bfc700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55a8003a20 0x7f55bc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:05.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c23fd700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55a8003a20 0x7f55bc06d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c23fd700 1 -- 172.21.15.104:0/1989846664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55bc0b2c80 msgr2=0x7f55bc1556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:05.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c23fd700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55bc0b2c80 0x7f55bc1556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c23fd700 1 -- 172.21.15.104:0/1989846664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc1565d0 con 0x7f55bc0ac940 2023-07-14T11:57:05.067 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c13fb700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55bc0b2c80 0x7f55bc1556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:57:05.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c1bfc700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55a8003a20 0x7f55bc06d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:57:05.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c23fd700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55bc0ac940 0x7f55bc0acdc0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f55b4000c00 tx=0x7f55b400ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:05.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55b27fc700 1 -- 172.21.15.104:0/1989846664 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4014480 con 0x7f55bc0ac940 2023-07-14T11:57:05.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55b27fc700 1 -- 172.21.15.104:0/1989846664 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b40145e0 con 0x7f55bc0ac940 2023-07-14T11:57:05.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c2bfe700 1 -- 172.21.15.104:0/1989846664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc0b33d0 con 0x7f55bc0ac940 2023-07-14T11:57:05.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55b27fc700 1 -- 172.21.15.104:0/1989846664 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4014990 con 0x7f55bc0ac940 2023-07-14T11:57:05.068 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.080+0000 7f55c2bfe700 1 -- 172.21.15.104:0/1989846664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc156260 con 0x7f55bc0ac940 2023-07-14T11:57:05.069 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.084+0000 7f55c2bfe700 1 -- 172.21.15.104:0/1989846664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc156760 con 0x7f55bc0ac940 2023-07-14T11:57:05.069 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.084+0000 7f55b27fc700 1 -- 172.21.15.104:0/1989846664 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f55b40114f0 con 0x7f55bc0ac940 2023-07-14T11:57:05.069 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.084+0000 7f55b27fc700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f55a00662f0 0x7f55a00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:05.070 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.084+0000 7f55b27fc700 1 -- 172.21.15.104:0/1989846664 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f55b4083140 con 0x7f55bc0ac940 2023-07-14T11:57:05.071 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.084+0000 7f55c1bfc700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f55a00662f0 0x7f55a00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:05.071 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.088+0000 7f55c1bfc700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f55a00662f0 0x7f55a00687a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f55b8019000 tx=0x7f55b8013000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:05.077 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.092+0000 7f55b27fc700 1 -- 172.21.15.104:0/1989846664 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f55b4054220 con 0x7f55bc0ac940 2023-07-14T11:57:05.252 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.268+0000 7f55c2bfe700 1 -- 172.21.15.104:0/1989846664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f55bc0668e0 con 0x7f55bc0ac940 2023-07-14T11:57:05.263 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.280+0000 7f55b27fc700 1 -- 172.21.15.104:0/1989846664 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f55b4053db0 con 0x7f55bc0ac940 2023-07-14T11:57:05.268 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.284+0000 7f55c2bfe700 1 -- 172.21.15.104:0/1989846664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f55a00662f0 msgr2=0x7f55a00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:05.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.284+0000 7f55c2bfe700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f55a00662f0 0x7f55a00687a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f55b8019000 tx=0x7f55b8013000 comp rx=0 tx=0).stop 2023-07-14T11:57:05.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.284+0000 7f55c2bfe700 1 -- 172.21.15.104:0/1989846664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55bc0ac940 msgr2=0x7f55bc0acdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:05.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.284+0000 7f55c2bfe700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55bc0ac940 0x7f55bc0acdc0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f55b4000c00 tx=0x7f55b400ea90 comp rx=0 tx=0).stop 2023-07-14T11:57:05.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.284+0000 7f55c2bfe700 1 -- 172.21.15.104:0/1989846664 shutdown_connections 2023-07-14T11:57:05.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.284+0000 7f55c2bfe700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f55a00662f0 0x7f55a00687a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.284+0000 7f55c2bfe700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55bc0ac940 0x7f55bc0acdc0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.284+0000 7f55c2bfe700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55bc0b2c80 0x7f55bc1556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.284+0000 7f55c2bfe700 1 --2- 172.21.15.104:0/1989846664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55a8003a20 0x7f55bc06d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.284+0000 7f55c2bfe700 1 -- 172.21.15.104:0/1989846664 >> 172.21.15.104:0/1989846664 conn(0x7f55bc0afc60 msgr2=0x7f55bc15c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:05.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.284+0000 7f55c2bfe700 1 -- 172.21.15.104:0/1989846664 shutdown_connections 2023-07-14T11:57:05.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.284+0000 7f55c2bfe700 1 -- 172.21.15.104:0/1989846664 wait complete. 2023-07-14T11:57:05.290 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2023-07-14T11:57:05.465 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.480+0000 7f435359e700 1 Processor -- start 2023-07-14T11:57:05.465 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.480+0000 7f435359e700 1 -- start start 2023-07-14T11:57:05.466 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.480+0000 7f435359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4354159f20 0x7f43540caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.480+0000 7f435359e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f43540caf70 0x7f435415d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.480+0000 7f435359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43540b0ef0 con 0x7f43540c9d90 2023-07-14T11:57:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.480+0000 7f435359e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43540ce9c0 con 0x7f43540caf70 2023-07-14T11:57:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.480+0000 7f435359e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43540cee50 con 0x7f4354159f20 2023-07-14T11:57:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f4351d9b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4354159f20 0x7f43540caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:05.468 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f4351d9b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4354159f20 0x7f43540caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.104:52342/0 (socket says 172.21.15.104:52342) 2023-07-14T11:57:05.468 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f4351d9b700 1 -- 172.21.15.104:0/1354541060 learned_addr learned my addr 172.21.15.104:0/1354541060 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:05.468 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f4352d9d700 1 --2- 172.21.15.104:0/1354541060 >> v2:172.21.15.121:3300/0 conn(0x7f43540caf70 0x7f435415d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:05.469 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f4351d9b700 1 -- 172.21.15.104:0/1354541060 >> v2:172.21.15.121:3300/0 conn(0x7f43540caf70 msgr2=0x7f435415d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:05.469 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f4351d9b700 1 --2- 172.21.15.104:0/1354541060 >> v2:172.21.15.121:3300/0 conn(0x7f43540caf70 0x7f435415d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.469 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f4351d9b700 1 -- 172.21.15.104:0/1354541060 >> v1:172.21.15.121:6789/0 conn(0x7f43540c9d90 legacy=0x7f43540ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:05.469 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f4351d9b700 1 -- 172.21.15.104:0/1354541060 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43540cb4f0 con 0x7f4354159f20 2023-07-14T11:57:05.469 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f4351d9b700 1 --2- 172.21.15.104:0/1354541060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4354159f20 0x7f43540caa30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4348009910 tx=0x7f4348009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52b313310a96fa97 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:05.470 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f435159a700 1 -- 172.21.15.104:0/1354541060 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348011830 con 0x7f4354159f20 2023-07-14T11:57:05.470 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f435159a700 1 -- 172.21.15.104:0/1354541060 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4348011990 con 0x7f4354159f20 2023-07-14T11:57:05.470 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f435159a700 1 -- 172.21.15.104:0/1354541060 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348011d40 con 0x7f4354159f20 2023-07-14T11:57:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f435359e700 1 -- 172.21.15.104:0/1354541060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4354159f20 msgr2=0x7f43540caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f435359e700 1 --2- 172.21.15.104:0/1354541060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4354159f20 0x7f43540caa30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4348009910 tx=0x7f4348009c20 comp rx=0 tx=0).stop 2023-07-14T11:57:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f435359e700 1 -- 172.21.15.104:0/1354541060 shutdown_connections 2023-07-14T11:57:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f435359e700 1 --2- 172.21.15.104:0/1354541060 >> v2:172.21.15.121:3300/0 conn(0x7f43540caf70 0x7f435415d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f435359e700 1 --2- 172.21.15.104:0/1354541060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4354159f20 0x7f43540caa30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.472 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f435359e700 1 -- 172.21.15.104:0/1354541060 >> 172.21.15.104:0/1354541060 conn(0x7f435406e960 msgr2=0x7f435406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:05.472 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f435359e700 1 -- 172.21.15.104:0/1354541060 shutdown_connections 2023-07-14T11:57:05.472 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.484+0000 7f435359e700 1 -- 172.21.15.104:0/1354541060 wait complete. 2023-07-14T11:57:05.473 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.488+0000 7f435359e700 1 Processor -- start 2023-07-14T11:57:05.474 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.488+0000 7f435359e700 1 -- start start 2023-07-14T11:57:05.475 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435359e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43540c9d90 0x7f435415c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:05.475 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43540caf70 0x7f4354156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:05.475 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435359e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4354159f20 0x7f43540c0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:05.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435359e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43541566b0 con 0x7f43540c9d90 2023-07-14T11:57:05.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4354156930 con 0x7f43540caf70 2023-07-14T11:57:05.476 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435359e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4354156c10 con 0x7f4354159f20 2023-07-14T11:57:05.477 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43540c9d90 0x7f435415c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:05.477 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43540c9d90 0x7f435415c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46508/0 (socket says 172.21.15.104:46508) 2023-07-14T11:57:05.477 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435259c700 1 -- 172.21.15.104:0/670222166 learned_addr learned my addr 172.21.15.104:0/670222166 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:05.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f4351d9b700 1 --2- 172.21.15.104:0/670222166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43540caf70 0x7f4354156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:05.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f4352d9d700 1 --2- 172.21.15.104:0/670222166 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4354159f20 0x7f43540c0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:05.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435259c700 1 -- 172.21.15.104:0/670222166 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4354159f20 msgr2=0x7f43540c0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:05.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435259c700 1 --2- 172.21.15.104:0/670222166 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4354159f20 0x7f43540c0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435259c700 1 -- 172.21.15.104:0/670222166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43540caf70 msgr2=0x7f4354156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:05.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435259c700 1 --2- 172.21.15.104:0/670222166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43540caf70 0x7f4354156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.478 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435259c700 1 -- 172.21.15.104:0/670222166 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4348009570 con 0x7f43540c9d90 2023-07-14T11:57:05.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435259c700 1 --2- 172.21.15.104:0/670222166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43540c9d90 0x7f435415c4c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f434400c500 tx=0x7f434400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:05.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f4342ffd700 1 -- 172.21.15.104:0/670222166 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43440052d0 con 0x7f43540c9d90 2023-07-14T11:57:05.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f4342ffd700 1 -- 172.21.15.104:0/670222166 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4344005430 con 0x7f43540c9d90 2023-07-14T11:57:05.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435359e700 1 -- 172.21.15.104:0/670222166 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4354156ef0 con 0x7f43540c9d90 2023-07-14T11:57:05.479 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f4342ffd700 1 -- 172.21.15.104:0/670222166 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43440055a0 con 0x7f43540c9d90 2023-07-14T11:57:05.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.492+0000 7f435359e700 1 -- 172.21.15.104:0/670222166 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43541574b0 con 0x7f43540c9d90 2023-07-14T11:57:05.480 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.496+0000 7f435359e700 1 -- 172.21.15.104:0/670222166 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43540c2330 con 0x7f43540c9d90 2023-07-14T11:57:05.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.496+0000 7f4342ffd700 1 -- 172.21.15.104:0/670222166 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f43440038e0 con 0x7f43540c9d90 2023-07-14T11:57:05.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.500+0000 7f4342ffd700 1 --2- 172.21.15.104:0/670222166 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f43340662f0 0x7f43340687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:05.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.500+0000 7f4342ffd700 1 -- 172.21.15.104:0/670222166 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f4344082ed0 con 0x7f43540c9d90 2023-07-14T11:57:05.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.500+0000 7f4351d9b700 1 --2- 172.21.15.104:0/670222166 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f43340662f0 0x7f43340687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:05.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.500+0000 7f4351d9b700 1 --2- 172.21.15.104:0/670222166 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f43340662f0 0x7f43340687a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4348000c00 tx=0x7f4348008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:05.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.504+0000 7f4342ffd700 1 -- 172.21.15.104:0/670222166 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f4344050490 con 0x7f43540c9d90 2023-07-14T11:57:05.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.736+0000 7f435359e700 1 -- 172.21.15.104:0/670222166 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7f435404e500 con 0x7f43540c9d90 2023-07-14T11:57:05.847 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.864+0000 7f4342ffd700 1 -- 172.21.15.104:0/670222166 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v226) v1 ==== 134+0+0 (secure 0 0 0) 0x7f4344053ab0 con 0x7f43540c9d90 2023-07-14T11:57:05.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.868+0000 7f435359e700 1 -- 172.21.15.104:0/670222166 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f43340662f0 msgr2=0x7f43340687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:05.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.868+0000 7f435359e700 1 --2- 172.21.15.104:0/670222166 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f43340662f0 0x7f43340687a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4348000c00 tx=0x7f4348008030 comp rx=0 tx=0).stop 2023-07-14T11:57:05.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.868+0000 7f435359e700 1 -- 172.21.15.104:0/670222166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43540c9d90 msgr2=0x7f435415c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:05.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.868+0000 7f435359e700 1 --2- 172.21.15.104:0/670222166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43540c9d90 0x7f435415c4c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f434400c500 tx=0x7f434400c810 comp rx=0 tx=0).stop 2023-07-14T11:57:05.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.868+0000 7f435359e700 1 -- 172.21.15.104:0/670222166 shutdown_connections 2023-07-14T11:57:05.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.868+0000 7f435359e700 1 --2- 172.21.15.104:0/670222166 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f43340662f0 0x7f43340687a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.868+0000 7f435359e700 1 --2- 172.21.15.104:0/670222166 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4354159f20 0x7f43540c0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.868+0000 7f435359e700 1 --2- 172.21.15.104:0/670222166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43540caf70 0x7f4354156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.868+0000 7f435359e700 1 --2- 172.21.15.104:0/670222166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43540c9d90 0x7f435415c4c0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:05.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.868+0000 7f435359e700 1 -- 172.21.15.104:0/670222166 >> 172.21.15.104:0/670222166 conn(0x7f435406e960 msgr2=0x7f43540cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:05.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.868+0000 7f435359e700 1 -- 172.21.15.104:0/670222166 shutdown_connections 2023-07-14T11:57:05.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:05.868+0000 7f435359e700 1 -- 172.21.15.104:0/670222166 wait complete. 2023-07-14T11:57:05.874 INFO:tasks.cephfs_test_runner:test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ERROR 2023-07-14T11:57:05.875 INFO:tasks.cephfs_test_runner: 2023-07-14T11:57:05.875 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-14T11:57:05.875 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-07-14T11:57:05.875 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-14T11:57:05.876 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-14T11:57:05.876 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2023-07-14T11:57:05.876 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2023-07-14T11:57:05.876 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/cephfs/mount.py", line 771, in run_python 2023-07-14T11:57:05.876 INFO:tasks.cephfs_test_runner: p.wait() 2023-07-14T11:57:05.877 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/orchestra/run.py", line 161, in wait 2023-07-14T11:57:05.877 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-07-14T11:57:05.877 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-07-14T11:57:05.877 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-07-14T11:57:05.877 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi104 with status 1: 'adjust-ulimits daemon-helper kill python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2023-07-14T11:57:05.878 INFO:tasks.cephfs_test_runner: 2023-07-14T11:57:05.878 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-14T11:57:05.878 INFO:tasks.cephfs_test_runner:Ran 2 tests in 169.833s 2023-07-14T11:57:05.878 INFO:tasks.cephfs_test_runner: 2023-07-14T11:57:05.879 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-07-14T11:57:05.880 INFO:tasks.cephfs_test_runner: 2023-07-14T11:57:05.880 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-14T11:57:05.880 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-07-14T11:57:05.880 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-14T11:57:05.881 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-14T11:57:05.881 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2023-07-14T11:57:05.881 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2023-07-14T11:57:05.881 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/cephfs/mount.py", line 771, in run_python 2023-07-14T11:57:05.881 INFO:tasks.cephfs_test_runner: p.wait() 2023-07-14T11:57:05.881 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/orchestra/run.py", line 161, in wait 2023-07-14T11:57:05.882 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-07-14T11:57:05.882 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-07-14T11:57:05.882 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-07-14T11:57:05.882 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi104 with status 1: 'adjust-ulimits daemon-helper kill python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2023-07-14T11:57:05.882 INFO:tasks.cephfs_test_runner: 2023-07-14T11:57:05.883 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-07-14T11:57:05.948 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5dc8b464a56c4996ab01c933dc6cde10 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-07-14T11:57:05.950 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-07-14T11:57:05.960 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-07-14T11:57:05.993 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-07-14T11:57:05.993 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:57:05.994 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:57:06.014 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352340k,nr_inodes=4088085,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275392k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275388k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275392k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-07-14T11:57:06.014 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-14T11:57:06.014 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:57:06.070 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352340k,nr_inodes=4088085,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275392k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275388k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275392k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-07-14T11:57:06.070 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:57:06.071 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:57:06.089 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352176k,nr_inodes=4088044,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18484 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275352k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-07-14T11:57:06.089 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-14T11:57:06.089 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:57:06.148 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352176k,nr_inodes=4088044,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18484 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275352k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275356k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-07-14T11:57:06.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:57:06.148 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-07-14T11:57:06.208 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-14T11:57:06.209 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-14T11:57:06.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:57:06.210 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-14T11:57:06.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:57:06.289 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-14T11:57:06.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:57:06.373 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2023-07-14T11:57:06.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:57:06.599 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2023-07-14T11:57:06.616 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-14T11:57:06.617 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-14T11:57:06.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:57:06.618 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-14T11:57:06.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:57:06.694 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-14T11:57:06.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:57:06.787 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link delete ceph-brx 2023-07-14T11:57:06.929 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-07-14T11:57:06.939 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-14T11:57:06.940 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:57:07.113 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.128+0000 7f22060dc700 1 Processor -- start 2023-07-14T11:57:07.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.128+0000 7f22060dc700 1 -- start start 2023-07-14T11:57:07.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.128+0000 7f22060dc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22000c9d90 0x7f22000ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.128+0000 7f22060dc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f220015d7f0 0x7f220015dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.128+0000 7f22060dc700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22000b0ef0 con 0x7f2200159f20 2023-07-14T11:57:07.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22060dc700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22000ce9c0 con 0x7f220015d7f0 2023-07-14T11:57:07.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22060dc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22000cee50 con 0x7f22000c9d90 2023-07-14T11:57:07.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22058db700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f220015d7f0 0x7f220015dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22058db700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f220015d7f0 0x7f220015dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46514/0 (socket says 172.21.15.104:46514) 2023-07-14T11:57:07.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22058db700 1 -- 172.21.15.104:0/3650992020 learned_addr learned my addr 172.21.15.104:0/3650992020 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:07.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22050da700 1 --2- 172.21.15.104:0/3650992020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22000c9d90 0x7f22000ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22058db700 1 -- 172.21.15.104:0/3650992020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22000c9d90 msgr2=0x7f22000ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:07.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22058db700 1 --2- 172.21.15.104:0/3650992020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22000c9d90 0x7f22000ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22058db700 1 -- 172.21.15.104:0/3650992020 >> v1:172.21.15.121:6789/0 conn(0x7f2200159f20 legacy=0x7f22000caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22058db700 1 -- 172.21.15.104:0/3650992020 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22000ca6a0 con 0x7f220015d7f0 2023-07-14T11:57:07.120 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22058db700 1 --2- 172.21.15.104:0/3650992020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f220015d7f0 0x7f220015dc50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009e50 tx=0x7f21f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5935d0d853e025 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:07.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f21f7fff700 1 -- 172.21.15.104:0/3650992020 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f800e070 con 0x7f220015d7f0 2023-07-14T11:57:07.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f21f7fff700 1 -- 172.21.15.104:0/3650992020 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f800b9c0 con 0x7f220015d7f0 2023-07-14T11:57:07.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f21f7fff700 1 -- 172.21.15.104:0/3650992020 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f800bd70 con 0x7f220015d7f0 2023-07-14T11:57:07.121 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22060dc700 1 -- 172.21.15.104:0/3650992020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f220015d7f0 msgr2=0x7f220015dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22060dc700 1 --2- 172.21.15.104:0/3650992020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f220015d7f0 0x7f220015dc50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009e50 tx=0x7f21f800b330 comp rx=0 tx=0).stop 2023-07-14T11:57:07.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22060dc700 1 -- 172.21.15.104:0/3650992020 shutdown_connections 2023-07-14T11:57:07.122 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22060dc700 1 --2- 172.21.15.104:0/3650992020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f220015d7f0 0x7f220015dc50 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22060dc700 1 --2- 172.21.15.104:0/3650992020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22000c9d90 0x7f22000ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22060dc700 1 -- 172.21.15.104:0/3650992020 >> 172.21.15.104:0/3650992020 conn(0x7f220006e960 msgr2=0x7f220006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:07.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22060dc700 1 -- 172.21.15.104:0/3650992020 shutdown_connections 2023-07-14T11:57:07.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.132+0000 7f22060dc700 1 -- 172.21.15.104:0/3650992020 wait complete. 2023-07-14T11:57:07.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.136+0000 7f22060dc700 1 Processor -- start 2023-07-14T11:57:07.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.136+0000 7f22060dc700 1 -- start start 2023-07-14T11:57:07.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.136+0000 7f22060dc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22000c9d90 0x7f220015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.136+0000 7f22060dc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2200159f20 0x7f220015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.136+0000 7f22060dc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f220015d7f0 0x7f220015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.136+0000 7f22060dc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22000c4530 con 0x7f220015d7f0 2023-07-14T11:57:07.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.136+0000 7f22060dc700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22000aec90 con 0x7f22000c9d90 2023-07-14T11:57:07.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.136+0000 7f22060dc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22000ae9b0 con 0x7f2200159f20 2023-07-14T11:57:07.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f22050da700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22000c9d90 0x7f220015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f22058db700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f220015d7f0 0x7f220015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f22050da700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22000c9d90 0x7f220015a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:40418/0 (socket says 172.21.15.104:40418) 2023-07-14T11:57:07.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f22050da700 1 -- 172.21.15.104:0/2870711541 learned_addr learned my addr 172.21.15.104:0/2870711541 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:07.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f22048d9700 1 --2- 172.21.15.104:0/2870711541 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2200159f20 0x7f220015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f22058db700 1 -- 172.21.15.104:0/2870711541 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2200159f20 msgr2=0x7f220015aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f22058db700 1 --2- 172.21.15.104:0/2870711541 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2200159f20 0x7f220015aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f22058db700 1 -- 172.21.15.104:0/2870711541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22000c9d90 msgr2=0x7f220015a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f22058db700 1 --2- 172.21.15.104:0/2870711541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22000c9d90 0x7f220015a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f22058db700 1 -- 172.21.15.104:0/2870711541 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f8009b30 con 0x7f220015d7f0 2023-07-14T11:57:07.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f22058db700 1 --2- 172.21.15.104:0/2870711541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f220015d7f0 0x7f220015b420 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f21f8015000 tx=0x7f21f8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:07.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f21f5ffb700 1 -- 172.21.15.104:0/2870711541 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8004800 con 0x7f220015d7f0 2023-07-14T11:57:07.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f21f5ffb700 1 -- 172.21.15.104:0/2870711541 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f80099b0 con 0x7f220015d7f0 2023-07-14T11:57:07.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f22060dc700 1 -- 172.21.15.104:0/2870711541 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22000afaa0 con 0x7f220015d7f0 2023-07-14T11:57:07.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f21f5ffb700 1 -- 172.21.15.104:0/2870711541 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f801c680 con 0x7f220015d7f0 2023-07-14T11:57:07.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f22060dc700 1 -- 172.21.15.104:0/2870711541 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22000b0060 con 0x7f220015d7f0 2023-07-14T11:57:07.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.140+0000 7f22060dc700 1 -- 172.21.15.104:0/2870711541 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f220004e500 con 0x7f220015d7f0 2023-07-14T11:57:07.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.144+0000 7f21f5ffb700 1 -- 172.21.15.104:0/2870711541 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f21f801c7e0 con 0x7f220015d7f0 2023-07-14T11:57:07.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.144+0000 7f21f5ffb700 1 --2- 172.21.15.104:0/2870711541 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21e4066320 0x7f21e40687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.144+0000 7f21f5ffb700 1 -- 172.21.15.104:0/2870711541 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f21f80853a0 con 0x7f220015d7f0 2023-07-14T11:57:07.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.144+0000 7f22050da700 1 --2- 172.21.15.104:0/2870711541 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21e4066320 0x7f21e40687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.144+0000 7f22050da700 1 --2- 172.21.15.104:0/2870711541 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21e4066320 0x7f21e40687d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f21fc007b10 tx=0x7f21fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:07.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.148+0000 7f21f5ffb700 1 -- 172.21.15.104:0/2870711541 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f21f80529f0 con 0x7f220015d7f0 2023-07-14T11:57:07.297 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.312+0000 7f22060dc700 1 -- 172.21.15.104:0/2870711541 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22000ce090 con 0x7f21e4066320 2023-07-14T11:57:07.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.320+0000 7f21f5ffb700 1 -- 172.21.15.104:0/2870711541 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70833 (secure 0 0 0) 0x7f21fc010720 con 0x7f21e4066320 2023-07-14T11:57:07.306 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:57:07.309 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.324+0000 7f22060dc700 1 -- 172.21.15.104:0/2870711541 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21e4066320 msgr2=0x7f21e40687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.324+0000 7f22060dc700 1 --2- 172.21.15.104:0/2870711541 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21e4066320 0x7f21e40687d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f21fc007b10 tx=0x7f21fc008040 comp rx=0 tx=0).stop 2023-07-14T11:57:07.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.324+0000 7f22060dc700 1 -- 172.21.15.104:0/2870711541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f220015d7f0 msgr2=0x7f220015b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.324+0000 7f22060dc700 1 --2- 172.21.15.104:0/2870711541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f220015d7f0 0x7f220015b420 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f21f8015000 tx=0x7f21f8004af0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.324+0000 7f22060dc700 1 -- 172.21.15.104:0/2870711541 shutdown_connections 2023-07-14T11:57:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.324+0000 7f22060dc700 1 --2- 172.21.15.104:0/2870711541 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f21e4066320 0x7f21e40687d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.324+0000 7f22060dc700 1 --2- 172.21.15.104:0/2870711541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f220015d7f0 0x7f220015b420 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.324+0000 7f22060dc700 1 --2- 172.21.15.104:0/2870711541 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2200159f20 0x7f220015aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.324+0000 7f22060dc700 1 --2- 172.21.15.104:0/2870711541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22000c9d90 0x7f220015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.324+0000 7f22060dc700 1 -- 172.21.15.104:0/2870711541 >> 172.21.15.104:0/2870711541 conn(0x7f220006e960 msgr2=0x7f22001524c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:07.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.324+0000 7f22060dc700 1 -- 172.21.15.104:0/2870711541 shutdown_connections 2023-07-14T11:57:07.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.324+0000 7f22060dc700 1 -- 172.21.15.104:0/2870711541 wait complete. 2023-07-14T11:57:07.315 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:57:07.332 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2023-07-14T11:57:05.978898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230784,"kb_used_data":6976,"kb_used_omap":0,"kb_used_meta":223744,"kb_avail":754743936,"statfs":{"total":773094113280,"available":772857790464,"internally_reserved":0,"allocated":7143424,"data_stored":3979974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":229113856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14219,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-68,"num_write_kb":-36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001582"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":94,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:55.775584+0000","last_change":"2023-07-14T11:53:09.145201+0000","last_active":"2023-07-14T11:56:55.775584+0000","last_peered":"2023-07-14T11:56:55.775584+0000","last_clean":"2023-07-14T11:56:55.775584+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:56:55.775584+0000","last_undegraded":"2023-07-14T11:56:55.775584+0000","last_fullsized":"2023-07-14T11:56:55.775584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:07:32.171071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:55.779236+0000","last_change":"2023-07-14T11:53:09.146150+0000","last_active":"2023-07-14T11:56:55.779236+0000","last_peered":"2023-07-14T11:56:55.779236+0000","last_clean":"2023-07-14T11:56:55.779236+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:56:55.779236+0000","last_undegraded":"2023-07-14T11:56:55.779236+0000","last_fullsized":"2023-07-14T11:56:55.779236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:04:42.879586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":80,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:56.043674+0000","last_change":"2023-07-14T11:53:09.435876+0000","last_active":"2023-07-14T11:56:56.043674+0000","last_peered":"2023-07-14T11:56:56.043674+0000","last_clean":"2023-07-14T11:56:56.043674+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:56:56.043674+0000","last_undegraded":"2023-07-14T11:56:56.043674+0000","last_fullsized":"2023-07-14T11:56:56.043674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:20:52.617276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:55.778954+0000","last_change":"2023-07-14T11:53:09.434255+0000","last_active":"2023-07-14T11:56:55.778954+0000","last_peered":"2023-07-14T11:56:55.778954+0000","last_clean":"2023-07-14T11:56:55.778954+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:56:55.778954+0000","last_undegraded":"2023-07-14T11:56:55.778954+0000","last_fullsized":"2023-07-14T11:56:55.778954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:26:32.085932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":92,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:56.323019+0000","last_change":"2023-07-14T11:53:09.147040+0000","last_active":"2023-07-14T11:56:56.323019+0000","last_peered":"2023-07-14T11:56:56.323019+0000","last_clean":"2023-07-14T11:56:56.323019+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:56:56.323019+0000","last_undegraded":"2023-07-14T11:56:56.323019+0000","last_fullsized":"2023-07-14T11:56:56.323019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:10.775563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:55.769790+0000","last_change":"2023-07-14T11:53:09.619294+0000","last_active":"2023-07-14T11:56:55.769790+0000","last_peered":"2023-07-14T11:56:55.769790+0000","last_clean":"2023-07-14T11:56:55.769790+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:56:55.769790+0000","last_undegraded":"2023-07-14T11:56:55.769790+0000","last_fullsized":"2023-07-14T11:56:55.769790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:13:01.915093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":80,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:56.044051+0000","last_change":"2023-07-14T11:53:09.146156+0000","last_active":"2023-07-14T11:56:56.044051+0000","last_peered":"2023-07-14T11:56:56.044051+0000","last_clean":"2023-07-14T11:56:56.044051+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:56:56.044051+0000","last_undegraded":"2023-07-14T11:56:56.044051+0000","last_fullsized":"2023-07-14T11:56:56.044051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:49:56.714534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":92,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:56.323076+0000","last_change":"2023-07-14T11:53:09.146556+0000","last_active":"2023-07-14T11:56:56.323076+0000","last_peered":"2023-07-14T11:56:56.323076+0000","last_clean":"2023-07-14T11:56:56.323076+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:56:56.323076+0000","last_undegraded":"2023-07-14T11:56:56.323076+0000","last_fullsized":"2023-07-14T11:56:56.323076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:27:14.150354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":236,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:56.044116+0000","last_change":"2023-07-14T11:53:03.159818+0000","last_active":"2023-07-14T11:56:56.044116+0000","last_peered":"2023-07-14T11:56:56.044116+0000","last_clean":"2023-07-14T11:56:56.044116+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:56:56.044116+0000","last_undegraded":"2023-07-14T11:56:56.044116+0000","last_fullsized":"2023-07-14T11:56:56.044116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_clean_scrub_stamp":"2023-07-14T11:53:02.054348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:45:44.242761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28264,"kb_used_data":1312,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343576,"statfs":{"total":96636764160,"available":96607821824,"internally_reserved":0,"allocated":1343488,"data_stored":936918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.093,"15min":1.111},"min":{"1min":0.754,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.474,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.014},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.103,"15min":1.127},"min":{"1min":0.65100000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2430000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.0429999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1499999999999999,"15min":1.171},"min":{"1min":0.68300000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.298,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.073},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.22,"15min":1.262},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.4159999999999999,"5min":2.234,"15min":2.234},"last":1.232}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.234,"15min":1.288},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.536,"5min":2.04,"15min":2.04},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.25,"15min":1.2829999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8089999999999999,"5min":2.492,"15min":2.492},"last":0.90800000000000003}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.282,"15min":1.3360000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.665,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.2430000000000001,"15min":1.274},"min":{"1min":0.81799999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5940000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.95499999999999996}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.321,"15min":1.385},"min":{"1min":0.84399999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5660000000000001,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":1.149},{"interface":"front","average":{"1min":1.286,"5min":1.3400000000000001,"15min":1.3799999999999999},"min":{"1min":0.92500000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.76,"5min":2.399,"15min":2.399},"last":0.873}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.284,"15min":1.3200000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6459999999999999,"5min":2.387,"15min":2.387},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.456,"15min":1.538},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.464,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":1.2969999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.5309999999999999,"15min":1.609},"min":{"1min":1.0940000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5289999999999999,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.4550000000000001,"15min":1.5229999999999999},"min":{"1min":0.95199999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.53,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.196}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":737280,"data_stored":345808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0700000000000001,"15min":1.0620000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.397,"5min":1.954,"15min":1.954},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.1160000000000001,"15min":1.1000000000000001},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.90600000000000003}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1890000000000001,"15min":1.2090000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7110000000000001,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.214,"15min":1.23},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7769999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.171,"15min":1.1659999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4690000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2709999999999999,"15min":1.2729999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6819999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.159}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.3140000000000001,"15min":1.3440000000000001},"min":{"1min":0.94099999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.6579999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.258,"5min":1.1899999999999999,"15min":1.1799999999999999},"min":{"1min":0.91700000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.482,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.2609999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.236,"15min":1.2529999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.625,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.228},{"interface":"front","average":{"1min":1.194,"5min":1.3129999999999999,"15min":1.3400000000000001},"min":{"1min":0.78000000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.512,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.99399999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.238,"15min":1.2270000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4039999999999999,"15min":1.425},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.9139999999999999,"5min":1.984,"15min":1.984},"last":1.306}]},{"osd":7,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.4330000000000001,"15min":1.4650000000000001},"min":{"1min":0.99199999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.6519999999999999,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.121},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.365,"15min":1.379},"min":{"1min":0.92900000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.728,"5min":1.879,"15min":1.879},"last":1.3919999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":737280,"data_stored":345796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.1359999999999999,"15min":1.1819999999999999},"min":{"1min":0.68100000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4790000000000001,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.129,"15min":1.175},"min":{"1min":0.70299999999999996,"5min":0.622,"15min":0.622},"max":{"1min":1.474,"5min":1.581,"15min":1.581},"last":1.1519999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.167,"15min":1.1990000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.524,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.175,"15min":1.1890000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.498,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.92800000000000005}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.198,"15min":1.226},"min":{"1min":0.86699999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4550000000000001,"5min":1.839,"15min":1.839},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.282,"15min":1.347},"min":{"1min":0.73799999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4990000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.2829999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.272,"15min":1.3220000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.5409999999999999,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.3440000000000001,"15min":1.3919999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.5349999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.2110000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2090000000000001,"15min":1.2490000000000001},"min":{"1min":0.75900000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5900000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.3540000000000001,"15min":1.3740000000000001},"min":{"1min":1.024,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.5109999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.337}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3959999999999999,"15min":1.4179999999999999},"min":{"1min":1.1459999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.544,"5min":1.99,"15min":1.99},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.375,"15min":1.4159999999999999},"min":{"1min":0.86399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6779999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.1240000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4039999999999999,"15min":1.421},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.6220000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.387,"5min":1.5569999999999999,"15min":1.6160000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.5880000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.446}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":757760,"data_stored":361499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.149,"15min":1.171},"min":{"1min":0.67200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.5069999999999999,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.133,"15min":1.1599999999999999},"min":{"1min":0.71099999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.99399999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.159,"15min":1.1819999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.508,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1879999999999999,"15min":1.2090000000000001},"min":{"1min":0.75900000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.516,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.246}]},{"osd":2,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1359999999999999,"15min":1.143},"min":{"1min":0.83899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.546,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3400000000000001,"15min":1.3620000000000001},"min":{"1min":0.96599999999999997,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.581,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":1.514}]},{"osd":3,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2789999999999999,"15min":1.321},"min":{"1min":0.82199999999999995,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.681,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.194},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2270000000000001,"15min":1.2450000000000001},"min":{"1min":0.88100000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.4770000000000001,"5min":1.823,"15min":1.823},"last":1.4099999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2989999999999999,"15min":1.3420000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6299999999999999,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.89500000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.47,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.3819999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.264,"15min":1.2669999999999999},"min":{"1min":0.97799999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.522,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.349},{"interface":"front","average":{"1min":1.351,"5min":1.476,"15min":1.532},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6759999999999999,"5min":3.5289999999999999,"15min":3.5289999999999999},"last":1.5760000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.466,"15min":1.5049999999999999},"min":{"1min":1.0589999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.6899999999999999,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.47,"15min":1.5209999999999999},"min":{"1min":1.0640000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":1.605,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":1.4790000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":733184,"data_stored":345774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.84099999999999997,"15min":0.871},"min":{"1min":0.47699999999999998,"5min":0.247,"15min":0.247},"max":{"1min":1.234,"5min":1.325,"15min":1.325},"last":0.437},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.875,"15min":0.90800000000000003},"min":{"1min":0.60599999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2490000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.53900000000000003}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92300000000000004,"15min":0.96299999999999997},"min":{"1min":0.52600000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6950000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.0600000000000001,"15min":1.111},"min":{"1min":0.54300000000000004,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.319,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.97399999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.097,"15min":1.141},"min":{"1min":0.63500000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4259999999999999,"5min":1.627,"15min":1.627},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.079,"5min":1.0669999999999999,"15min":1.091},"min":{"1min":0.73399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.81299999999999994}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2410000000000001,"15min":1.2849999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.53,"5min":2.335,"15min":2.335},"last":1.022},{"interface":"front","average":{"1min":1.246,"5min":1.2470000000000001,"15min":1.2869999999999999},"min":{"1min":0.77700000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.2669999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.323,"15min":1.359},"min":{"1min":0.94999999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.357},{"interface":"front","average":{"1min":1.212,"5min":1.3580000000000001,"15min":1.421},"min":{"1min":0.86899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.532,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.0589999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.329,"15min":1.387},"min":{"1min":0.85899999999999999,"5min":0.625,"15min":0.625},"max":{"1min":1.7010000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.4550000000000001,"15min":1.5009999999999999},"min":{"1min":1.083,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.0790000000000002,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.4419999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.419,"15min":1.4810000000000001},"min":{"1min":0.90500000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.712,"5min":3.395,"15min":3.395},"last":1.21},{"interface":"front","average":{"1min":1.306,"5min":1.367,"15min":1.431},"min":{"1min":1.0680000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.619,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.95199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":733184,"data_stored":345774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.874,"15min":0.90600000000000003},"min":{"1min":0.53800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.22,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.876,"15min":0.90200000000000002},"min":{"1min":0.48799999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1799999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.53}]},{"osd":1,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.88100000000000001,"15min":0.89900000000000002},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.093,"5min":1.365,"15min":1.365},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0269999999999999,"15min":1.069},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.425,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.6319999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0649999999999999,"15min":1.0960000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5089999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.018,"5min":1.026,"15min":1.052},"min":{"1min":0.65300000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.5900000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.90200000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2709999999999999,"15min":1.2989999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.718,"5min":2.097,"15min":2.097},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.2769999999999999,"15min":1.3149999999999999},"min":{"1min":0.67800000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.861,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.9359999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.2889999999999999,"15min":1.319},"min":{"1min":0.95699999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.4710000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3180000000000001,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.645,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.98499999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.333,"15min":1.383},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7649999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.8280000000000001},{"interface":"front","average":{"1min":1.361,"5min":1.3919999999999999,"15min":1.425},"min":{"1min":1.1040000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.726,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.008}]},{"osd":7,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.397,"15min":1.444},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.677,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.8839999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4219999999999999,"15min":1.466},"min":{"1min":1.01,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6899999999999999,"5min":2.536,"15min":2.536},"last":0.94299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":774144,"data_stored":362263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.80100000000000005,"15min":0.83299999999999996},"min":{"1min":0.48899999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.26,"5min":1.349,"15min":1.349},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83299999999999996,"15min":0.83399999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84799999999999998,"15min":0.85799999999999998},"min":{"1min":0.499,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.93500000000000005,"15min":0.93999999999999995},"min":{"1min":0.55900000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.97499999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.99199999999999999,"15min":1.002},"min":{"1min":0.78500000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.399,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0169999999999999,"15min":1.0229999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.8169999999999999,"5min":1.857,"15min":1.857},"last":1.1579999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.2729999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6279999999999999,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.1830000000000001,"15min":1.2170000000000001},"min":{"1min":0.754,"5min":0.745,"15min":0.745},"max":{"1min":1.6279999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.88400000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.23,"15min":1.256},"min":{"1min":0.876,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7190000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3149999999999999,"15min":1.365},"min":{"1min":0.93100000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.778,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.107}]},{"osd":6,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3560000000000001,"15min":1.387},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.9119999999999999,"5min":2.633,"15min":2.633},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.335,"15min":1.359},"min":{"1min":1.091,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.873,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.135}]},{"osd":7,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3580000000000001,"15min":1.3979999999999999},"min":{"1min":1.004,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.992,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3300000000000001,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.754,"15min":0.754},"max":{"1min":2.056,"5min":2.375,"15min":2.375},"last":1.2070000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1327104,"data_stored":936142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.879,"15min":0.89300000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3100000000000001,"5min":1.581,"15min":1.581},"last":0.83699999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0049999999999999,"15min":1},"min":{"1min":0.64800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.5880000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90800000000000003,"15min":0.95099999999999996},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.1639999999999999,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.2470000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.93600000000000005,"15min":0.94599999999999995},"min":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.228,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.099,"15min":1.1319999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3799999999999999,"5min":1.704,"15min":1.704},"last":1.1910000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2430000000000001,"15min":1.278},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5149999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3420000000000001,"15min":1.3720000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.837,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.3460000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.228,"15min":1.284},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9870000000000001,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.161,"5min":1.276,"15min":1.3200000000000001},"min":{"1min":0.79800000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.7749999999999999,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.3149999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.0459999999999998,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.319,"15min":1.357},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.9299999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.1619999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.335,"15min":1.3819999999999999},"min":{"1min":0.83299999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.865,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.379,"15min":1.417},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.6870000000000001,"5min":2.109,"15min":2.109},"last":1.1279999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:57:07.335 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:57:07.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.516+0000 7f8be53e0700 1 Processor -- start 2023-07-14T11:57:07.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.516+0000 7f8be53e0700 1 -- start start 2023-07-14T11:57:07.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8be53e0700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8be015ffa0 0x7f8be006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8be53e0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8be006dc50 0x7f8be00caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8be53e0700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8be0160370 0x7f8be00cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8be53e0700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be00ca8a0 con 0x7f8be0160370 2023-07-14T11:57:07.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8be53e0700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be00cbcd0 con 0x7f8be015ffa0 2023-07-14T11:57:07.505 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8be53e0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be00cbf70 con 0x7f8be006dc50 2023-07-14T11:57:07.506 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bdf7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8be0160370 0x7f8be00cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bdf7fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8be0160370 0x7f8be00cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46536/0 (socket says 172.21.15.104:46536) 2023-07-14T11:57:07.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bdf7fe700 1 -- 172.21.15.104:0/770209570 learned_addr learned my addr 172.21.15.104:0/770209570 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:07.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bd67fc700 1 --2- 172.21.15.104:0/770209570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8be006dc50 0x7f8be00caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bdeffd700 1 --2- 172.21.15.104:0/770209570 >> v2:172.21.15.121:3300/0 conn(0x7f8be015ffa0 0x7f8be006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bdf7fe700 1 -- 172.21.15.104:0/770209570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8be006dc50 msgr2=0x7f8be00caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:07.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bdf7fe700 1 --2- 172.21.15.104:0/770209570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8be006dc50 0x7f8be00caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bdf7fe700 1 -- 172.21.15.104:0/770209570 >> v2:172.21.15.121:3300/0 conn(0x7f8be015ffa0 msgr2=0x7f8be006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:07.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bdf7fe700 1 --2- 172.21.15.104:0/770209570 >> v2:172.21.15.121:3300/0 conn(0x7f8be015ffa0 0x7f8be006d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bdf7fe700 1 -- 172.21.15.104:0/770209570 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be00cc250 con 0x7f8be0160370 2023-07-14T11:57:07.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bdeffd700 1 --2- 172.21.15.104:0/770209570 >> v2:172.21.15.121:3300/0 conn(0x7f8be015ffa0 0x7f8be006d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:57:07.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bd67fc700 1 --2- 172.21.15.104:0/770209570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8be006dc50 0x7f8be00caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:57:07.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bdf7fe700 1 --2- 172.21.15.104:0/770209570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be0160370 0x7f8be00cafb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8be00caa40 tx=0x7f8bd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d98e56a4bcd459 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:07.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bde7fc700 1 -- 172.21.15.104:0/770209570 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd800e070 con 0x7f8be0160370 2023-07-14T11:57:07.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bde7fc700 1 -- 172.21.15.104:0/770209570 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd800b9c0 con 0x7f8be0160370 2023-07-14T11:57:07.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.520+0000 7f8bde7fc700 1 -- 172.21.15.104:0/770209570 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd800bd70 con 0x7f8be0160370 2023-07-14T11:57:07.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.524+0000 7f8be53e0700 1 -- 172.21.15.104:0/770209570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be0160370 msgr2=0x7f8be00cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.524+0000 7f8be53e0700 1 --2- 172.21.15.104:0/770209570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be0160370 0x7f8be00cafb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8be00caa40 tx=0x7f8bd800b330 comp rx=0 tx=0).stop 2023-07-14T11:57:07.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.524+0000 7f8be53e0700 1 -- 172.21.15.104:0/770209570 shutdown_connections 2023-07-14T11:57:07.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.524+0000 7f8be53e0700 1 --2- 172.21.15.104:0/770209570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be0160370 0x7f8be00cafb0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.512 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.524+0000 7f8be53e0700 1 --2- 172.21.15.104:0/770209570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8be006dc50 0x7f8be00caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.524+0000 7f8be53e0700 1 --2- 172.21.15.104:0/770209570 >> v2:172.21.15.121:3300/0 conn(0x7f8be015ffa0 0x7f8be006d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.524+0000 7f8be53e0700 1 -- 172.21.15.104:0/770209570 >> 172.21.15.104:0/770209570 conn(0x7f8be006e960 msgr2=0x7f8be006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:07.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.524+0000 7f8be53e0700 1 -- 172.21.15.104:0/770209570 shutdown_connections 2023-07-14T11:57:07.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.524+0000 7f8be53e0700 1 -- 172.21.15.104:0/770209570 wait complete. 2023-07-14T11:57:07.513 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8be53e0700 1 Processor -- start 2023-07-14T11:57:07.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8be53e0700 1 -- start start 2023-07-14T11:57:07.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8be53e0700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8be006dc50 0x7f8be015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.515 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8be53e0700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8be015ffa0 0x7f8be00c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8be53e0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be0160370 0x7f8be00c19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8be53e0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be015dbf0 con 0x7f8be0160370 2023-07-14T11:57:07.516 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8be53e0700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be0160e90 con 0x7f8be006dc50 2023-07-14T11:57:07.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8be53e0700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be015d6e0 con 0x7f8be015ffa0 2023-07-14T11:57:07.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8bdf7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be0160370 0x7f8be00c19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.517 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8bdf7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be0160370 0x7f8be00c19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46546/0 (socket says 172.21.15.104:46546) 2023-07-14T11:57:07.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8bdf7fe700 1 -- 172.21.15.104:0/4216515784 learned_addr learned my addr 172.21.15.104:0/4216515784 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:07.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8bd67fc700 1 --2- 172.21.15.104:0/4216515784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8be015ffa0 0x7f8be00c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8bdeffd700 1 --2- 172.21.15.104:0/4216515784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8be006dc50 0x7f8be015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8bdf7fe700 1 -- 172.21.15.104:0/4216515784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8be015ffa0 msgr2=0x7f8be00c3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8bdf7fe700 1 --2- 172.21.15.104:0/4216515784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8be015ffa0 0x7f8be00c3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8bdf7fe700 1 -- 172.21.15.104:0/4216515784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8be006dc50 msgr2=0x7f8be015d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8bdf7fe700 1 --2- 172.21.15.104:0/4216515784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8be006dc50 0x7f8be015d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.528+0000 7f8bdf7fe700 1 -- 172.21.15.104:0/4216515784 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd8009b30 con 0x7f8be0160370 2023-07-14T11:57:07.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.532+0000 7f8bdf7fe700 1 --2- 172.21.15.104:0/4216515784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be0160370 0x7f8be00c19b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800b730 tx=0x7f8bd8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:07.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.532+0000 7f8bd7fff700 1 -- 172.21.15.104:0/4216515784 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8004250 con 0x7f8be0160370 2023-07-14T11:57:07.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.532+0000 7f8bd7fff700 1 -- 172.21.15.104:0/4216515784 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd80043b0 con 0x7f8be0160370 2023-07-14T11:57:07.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.532+0000 7f8be53e0700 1 -- 172.21.15.104:0/4216515784 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be0163260 con 0x7f8be0160370 2023-07-14T11:57:07.521 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.532+0000 7f8bd7fff700 1 -- 172.21.15.104:0/4216515784 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8004760 con 0x7f8be0160370 2023-07-14T11:57:07.521 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.532+0000 7f8be53e0700 1 -- 172.21.15.104:0/4216515784 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be0160840 con 0x7f8be0160370 2023-07-14T11:57:07.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.532+0000 7f8be53e0700 1 -- 172.21.15.104:0/4216515784 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be0161020 con 0x7f8be0160370 2023-07-14T11:57:07.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.532+0000 7f8bd7fff700 1 -- 172.21.15.104:0/4216515784 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8bd801b430 con 0x7f8be0160370 2023-07-14T11:57:07.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.532+0000 7f8bd7fff700 1 --2- 172.21.15.104:0/4216515784 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8bc40662f0 0x7f8bc40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.532+0000 7f8bd7fff700 1 -- 172.21.15.104:0/4216515784 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f8bd80850b0 con 0x7f8be0160370 2023-07-14T11:57:07.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.536+0000 7f8bdeffd700 1 --2- 172.21.15.104:0/4216515784 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8bc40662f0 0x7f8bc40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.523 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.536+0000 7f8bdeffd700 1 --2- 172.21.15.104:0/4216515784 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8bc40662f0 0x7f8bc40687a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0003710 tx=0x7f8bd0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:07.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.540+0000 7f8bd7fff700 1 -- 172.21.15.104:0/4216515784 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f8bd8052680 con 0x7f8be0160370 2023-07-14T11:57:07.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.712+0000 7f8be53e0700 1 -- 172.21.15.104:0/4216515784 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8be00cc810 con 0x7f8bc40662f0 2023-07-14T11:57:07.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.720+0000 7f8bd7fff700 1 -- 172.21.15.104:0/4216515784 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70833 (secure 0 0 0) 0x7f8bd0014460 con 0x7f8bc40662f0 2023-07-14T11:57:07.704 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:57:07.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.724+0000 7f8be53e0700 1 -- 172.21.15.104:0/4216515784 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8bc40662f0 msgr2=0x7f8bc40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.724+0000 7f8be53e0700 1 --2- 172.21.15.104:0/4216515784 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8bc40662f0 0x7f8bc40687a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0003710 tx=0x7f8bd0008040 comp rx=0 tx=0).stop 2023-07-14T11:57:07.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.724+0000 7f8be53e0700 1 -- 172.21.15.104:0/4216515784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be0160370 msgr2=0x7f8be00c19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.724+0000 7f8be53e0700 1 --2- 172.21.15.104:0/4216515784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be0160370 0x7f8be00c19b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800b730 tx=0x7f8bd8004af0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.724+0000 7f8be53e0700 1 -- 172.21.15.104:0/4216515784 shutdown_connections 2023-07-14T11:57:07.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.724+0000 7f8be53e0700 1 --2- 172.21.15.104:0/4216515784 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8bc40662f0 0x7f8bc40687a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.724+0000 7f8be53e0700 1 --2- 172.21.15.104:0/4216515784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be0160370 0x7f8be00c19b0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.724+0000 7f8be53e0700 1 --2- 172.21.15.104:0/4216515784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8be015ffa0 0x7f8be00c3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.724+0000 7f8be53e0700 1 --2- 172.21.15.104:0/4216515784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8be006dc50 0x7f8be015d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.724+0000 7f8be53e0700 1 -- 172.21.15.104:0/4216515784 >> 172.21.15.104:0/4216515784 conn(0x7f8be006e960 msgr2=0x7f8be006ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:07.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.724+0000 7f8be53e0700 1 -- 172.21.15.104:0/4216515784 shutdown_connections 2023-07-14T11:57:07.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.724+0000 7f8be53e0700 1 -- 172.21.15.104:0/4216515784 wait complete. 2023-07-14T11:57:07.710 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:57:07.730 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2023-07-14T11:57:05.978898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230784,"kb_used_data":6976,"kb_used_omap":0,"kb_used_meta":223744,"kb_avail":754743936,"statfs":{"total":773094113280,"available":772857790464,"internally_reserved":0,"allocated":7143424,"data_stored":3979974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":229113856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14219,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-68,"num_write_kb":-36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001582"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":94,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:55.775584+0000","last_change":"2023-07-14T11:53:09.145201+0000","last_active":"2023-07-14T11:56:55.775584+0000","last_peered":"2023-07-14T11:56:55.775584+0000","last_clean":"2023-07-14T11:56:55.775584+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:56:55.775584+0000","last_undegraded":"2023-07-14T11:56:55.775584+0000","last_fullsized":"2023-07-14T11:56:55.775584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:07:32.171071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:55.779236+0000","last_change":"2023-07-14T11:53:09.146150+0000","last_active":"2023-07-14T11:56:55.779236+0000","last_peered":"2023-07-14T11:56:55.779236+0000","last_clean":"2023-07-14T11:56:55.779236+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:56:55.779236+0000","last_undegraded":"2023-07-14T11:56:55.779236+0000","last_fullsized":"2023-07-14T11:56:55.779236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:04:42.879586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":80,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:56.043674+0000","last_change":"2023-07-14T11:53:09.435876+0000","last_active":"2023-07-14T11:56:56.043674+0000","last_peered":"2023-07-14T11:56:56.043674+0000","last_clean":"2023-07-14T11:56:56.043674+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:56:56.043674+0000","last_undegraded":"2023-07-14T11:56:56.043674+0000","last_fullsized":"2023-07-14T11:56:56.043674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:20:52.617276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:55.778954+0000","last_change":"2023-07-14T11:53:09.434255+0000","last_active":"2023-07-14T11:56:55.778954+0000","last_peered":"2023-07-14T11:56:55.778954+0000","last_clean":"2023-07-14T11:56:55.778954+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:56:55.778954+0000","last_undegraded":"2023-07-14T11:56:55.778954+0000","last_fullsized":"2023-07-14T11:56:55.778954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:26:32.085932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":92,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:56.323019+0000","last_change":"2023-07-14T11:53:09.147040+0000","last_active":"2023-07-14T11:56:56.323019+0000","last_peered":"2023-07-14T11:56:56.323019+0000","last_clean":"2023-07-14T11:56:56.323019+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:56:56.323019+0000","last_undegraded":"2023-07-14T11:56:56.323019+0000","last_fullsized":"2023-07-14T11:56:56.323019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:10.775563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:55.769790+0000","last_change":"2023-07-14T11:53:09.619294+0000","last_active":"2023-07-14T11:56:55.769790+0000","last_peered":"2023-07-14T11:56:55.769790+0000","last_clean":"2023-07-14T11:56:55.769790+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:56:55.769790+0000","last_undegraded":"2023-07-14T11:56:55.769790+0000","last_fullsized":"2023-07-14T11:56:55.769790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:13:01.915093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":80,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:56.044051+0000","last_change":"2023-07-14T11:53:09.146156+0000","last_active":"2023-07-14T11:56:56.044051+0000","last_peered":"2023-07-14T11:56:56.044051+0000","last_clean":"2023-07-14T11:56:56.044051+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:56:56.044051+0000","last_undegraded":"2023-07-14T11:56:56.044051+0000","last_fullsized":"2023-07-14T11:56:56.044051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:49:56.714534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":92,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:56.323076+0000","last_change":"2023-07-14T11:53:09.146556+0000","last_active":"2023-07-14T11:56:56.323076+0000","last_peered":"2023-07-14T11:56:56.323076+0000","last_clean":"2023-07-14T11:56:56.323076+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:56:56.323076+0000","last_undegraded":"2023-07-14T11:56:56.323076+0000","last_fullsized":"2023-07-14T11:56:56.323076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:27:14.150354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":236,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-14T11:56:56.044116+0000","last_change":"2023-07-14T11:53:03.159818+0000","last_active":"2023-07-14T11:56:56.044116+0000","last_peered":"2023-07-14T11:56:56.044116+0000","last_clean":"2023-07-14T11:56:56.044116+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:56:56.044116+0000","last_undegraded":"2023-07-14T11:56:56.044116+0000","last_fullsized":"2023-07-14T11:56:56.044116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_clean_scrub_stamp":"2023-07-14T11:53:02.054348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:45:44.242761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28264,"kb_used_data":1312,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343576,"statfs":{"total":96636764160,"available":96607821824,"internally_reserved":0,"allocated":1343488,"data_stored":936918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.093,"15min":1.111},"min":{"1min":0.754,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.474,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.014},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.103,"15min":1.127},"min":{"1min":0.65100000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2430000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.0429999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1499999999999999,"15min":1.171},"min":{"1min":0.68300000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.298,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.073},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.22,"15min":1.262},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.4159999999999999,"5min":2.234,"15min":2.234},"last":1.232}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.234,"15min":1.288},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.536,"5min":2.04,"15min":2.04},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.25,"15min":1.2829999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8089999999999999,"5min":2.492,"15min":2.492},"last":0.90800000000000003}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.282,"15min":1.3360000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.665,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.2430000000000001,"15min":1.274},"min":{"1min":0.81799999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5940000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.95499999999999996}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.321,"15min":1.385},"min":{"1min":0.84399999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5660000000000001,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":1.149},{"interface":"front","average":{"1min":1.286,"5min":1.3400000000000001,"15min":1.3799999999999999},"min":{"1min":0.92500000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.76,"5min":2.399,"15min":2.399},"last":0.873}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.284,"15min":1.3200000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6459999999999999,"5min":2.387,"15min":2.387},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.456,"15min":1.538},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.464,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":1.2969999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.5309999999999999,"15min":1.609},"min":{"1min":1.0940000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5289999999999999,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.4550000000000001,"15min":1.5229999999999999},"min":{"1min":0.95199999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.53,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.196}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":737280,"data_stored":345808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0700000000000001,"15min":1.0620000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.397,"5min":1.954,"15min":1.954},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.1160000000000001,"15min":1.1000000000000001},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.90600000000000003}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1890000000000001,"15min":1.2090000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7110000000000001,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.214,"15min":1.23},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7769999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.171,"15min":1.1659999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4690000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2709999999999999,"15min":1.2729999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6819999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.159}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.3140000000000001,"15min":1.3440000000000001},"min":{"1min":0.94099999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.6579999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.258,"5min":1.1899999999999999,"15min":1.1799999999999999},"min":{"1min":0.91700000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.482,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.2609999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.236,"15min":1.2529999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.625,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.228},{"interface":"front","average":{"1min":1.194,"5min":1.3129999999999999,"15min":1.3400000000000001},"min":{"1min":0.78000000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.512,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.99399999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.238,"15min":1.2270000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4039999999999999,"15min":1.425},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.9139999999999999,"5min":1.984,"15min":1.984},"last":1.306}]},{"osd":7,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.4330000000000001,"15min":1.4650000000000001},"min":{"1min":0.99199999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.6519999999999999,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.121},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.365,"15min":1.379},"min":{"1min":0.92900000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.728,"5min":1.879,"15min":1.879},"last":1.3919999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":737280,"data_stored":345796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.1359999999999999,"15min":1.1819999999999999},"min":{"1min":0.68100000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4790000000000001,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.129,"15min":1.175},"min":{"1min":0.70299999999999996,"5min":0.622,"15min":0.622},"max":{"1min":1.474,"5min":1.581,"15min":1.581},"last":1.1519999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.167,"15min":1.1990000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.524,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.175,"15min":1.1890000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.498,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.92800000000000005}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.198,"15min":1.226},"min":{"1min":0.86699999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4550000000000001,"5min":1.839,"15min":1.839},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.282,"15min":1.347},"min":{"1min":0.73799999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4990000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.2829999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.272,"15min":1.3220000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.5409999999999999,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.3440000000000001,"15min":1.3919999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.5349999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.2110000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2090000000000001,"15min":1.2490000000000001},"min":{"1min":0.75900000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5900000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.3540000000000001,"15min":1.3740000000000001},"min":{"1min":1.024,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.5109999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.337}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3959999999999999,"15min":1.4179999999999999},"min":{"1min":1.1459999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.544,"5min":1.99,"15min":1.99},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.375,"15min":1.4159999999999999},"min":{"1min":0.86399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6779999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.1240000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4039999999999999,"15min":1.421},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.6220000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.387,"5min":1.5569999999999999,"15min":1.6160000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.5880000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.446}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":757760,"data_stored":361499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.149,"15min":1.171},"min":{"1min":0.67200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.5069999999999999,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.133,"15min":1.1599999999999999},"min":{"1min":0.71099999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.99399999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.159,"15min":1.1819999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.508,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1879999999999999,"15min":1.2090000000000001},"min":{"1min":0.75900000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.516,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.246}]},{"osd":2,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1359999999999999,"15min":1.143},"min":{"1min":0.83899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.546,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3400000000000001,"15min":1.3620000000000001},"min":{"1min":0.96599999999999997,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.581,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":1.514}]},{"osd":3,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2789999999999999,"15min":1.321},"min":{"1min":0.82199999999999995,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.681,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.194},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2270000000000001,"15min":1.2450000000000001},"min":{"1min":0.88100000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.4770000000000001,"5min":1.823,"15min":1.823},"last":1.4099999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2989999999999999,"15min":1.3420000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6299999999999999,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.89500000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.47,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.3819999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.264,"15min":1.2669999999999999},"min":{"1min":0.97799999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.522,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.349},{"interface":"front","average":{"1min":1.351,"5min":1.476,"15min":1.532},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6759999999999999,"5min":3.5289999999999999,"15min":3.5289999999999999},"last":1.5760000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.466,"15min":1.5049999999999999},"min":{"1min":1.0589999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.6899999999999999,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.47,"15min":1.5209999999999999},"min":{"1min":1.0640000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":1.605,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":1.4790000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":733184,"data_stored":345774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.84099999999999997,"15min":0.871},"min":{"1min":0.47699999999999998,"5min":0.247,"15min":0.247},"max":{"1min":1.234,"5min":1.325,"15min":1.325},"last":0.437},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.875,"15min":0.90800000000000003},"min":{"1min":0.60599999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2490000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.53900000000000003}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92300000000000004,"15min":0.96299999999999997},"min":{"1min":0.52600000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6950000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.0600000000000001,"15min":1.111},"min":{"1min":0.54300000000000004,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.319,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.97399999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.097,"15min":1.141},"min":{"1min":0.63500000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4259999999999999,"5min":1.627,"15min":1.627},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.079,"5min":1.0669999999999999,"15min":1.091},"min":{"1min":0.73399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.81299999999999994}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2410000000000001,"15min":1.2849999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.53,"5min":2.335,"15min":2.335},"last":1.022},{"interface":"front","average":{"1min":1.246,"5min":1.2470000000000001,"15min":1.2869999999999999},"min":{"1min":0.77700000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.2669999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.323,"15min":1.359},"min":{"1min":0.94999999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.357},{"interface":"front","average":{"1min":1.212,"5min":1.3580000000000001,"15min":1.421},"min":{"1min":0.86899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.532,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.0589999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.329,"15min":1.387},"min":{"1min":0.85899999999999999,"5min":0.625,"15min":0.625},"max":{"1min":1.7010000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.4550000000000001,"15min":1.5009999999999999},"min":{"1min":1.083,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.0790000000000002,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.4419999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.419,"15min":1.4810000000000001},"min":{"1min":0.90500000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.712,"5min":3.395,"15min":3.395},"last":1.21},{"interface":"front","average":{"1min":1.306,"5min":1.367,"15min":1.431},"min":{"1min":1.0680000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.619,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.95199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":733184,"data_stored":345774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.874,"15min":0.90600000000000003},"min":{"1min":0.53800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.22,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.876,"15min":0.90200000000000002},"min":{"1min":0.48799999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1799999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.53}]},{"osd":1,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.88100000000000001,"15min":0.89900000000000002},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.093,"5min":1.365,"15min":1.365},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0269999999999999,"15min":1.069},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.425,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.6319999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0649999999999999,"15min":1.0960000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5089999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.018,"5min":1.026,"15min":1.052},"min":{"1min":0.65300000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.5900000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.90200000000000002}]},{"osd":4,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2709999999999999,"15min":1.2989999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.718,"5min":2.097,"15min":2.097},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.2769999999999999,"15min":1.3149999999999999},"min":{"1min":0.67800000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.861,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.9359999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.2889999999999999,"15min":1.319},"min":{"1min":0.95699999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.4710000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3180000000000001,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.645,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.98499999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.333,"15min":1.383},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7649999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.8280000000000001},{"interface":"front","average":{"1min":1.361,"5min":1.3919999999999999,"15min":1.425},"min":{"1min":1.1040000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.726,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.008}]},{"osd":7,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.397,"15min":1.444},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.677,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.8839999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4219999999999999,"15min":1.466},"min":{"1min":1.01,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6899999999999999,"5min":2.536,"15min":2.536},"last":0.94299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":774144,"data_stored":362263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.80100000000000005,"15min":0.83299999999999996},"min":{"1min":0.48899999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.26,"5min":1.349,"15min":1.349},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83299999999999996,"15min":0.83399999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84799999999999998,"15min":0.85799999999999998},"min":{"1min":0.499,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.93500000000000005,"15min":0.93999999999999995},"min":{"1min":0.55900000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.97499999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.99199999999999999,"15min":1.002},"min":{"1min":0.78500000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.399,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0169999999999999,"15min":1.0229999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.8169999999999999,"5min":1.857,"15min":1.857},"last":1.1579999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.2729999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6279999999999999,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.1830000000000001,"15min":1.2170000000000001},"min":{"1min":0.754,"5min":0.745,"15min":0.745},"max":{"1min":1.6279999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.88400000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.23,"15min":1.256},"min":{"1min":0.876,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7190000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3149999999999999,"15min":1.365},"min":{"1min":0.93100000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.778,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.107}]},{"osd":6,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3560000000000001,"15min":1.387},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.9119999999999999,"5min":2.633,"15min":2.633},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.335,"15min":1.359},"min":{"1min":1.091,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.873,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.135}]},{"osd":7,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3580000000000001,"15min":1.3979999999999999},"min":{"1min":1.004,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.992,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3300000000000001,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.754,"15min":0.754},"max":{"1min":2.056,"5min":2.375,"15min":2.375},"last":1.2070000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1327104,"data_stored":936142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.879,"15min":0.89300000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3100000000000001,"5min":1.581,"15min":1.581},"last":0.83699999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0049999999999999,"15min":1},"min":{"1min":0.64800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.5880000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90800000000000003,"15min":0.95099999999999996},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.1639999999999999,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.2470000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.93600000000000005,"15min":0.94599999999999995},"min":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.228,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.099,"15min":1.1319999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3799999999999999,"5min":1.704,"15min":1.704},"last":1.1910000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2430000000000001,"15min":1.278},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5149999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3420000000000001,"15min":1.3720000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.837,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.3460000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.228,"15min":1.284},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9870000000000001,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.161,"5min":1.276,"15min":1.3200000000000001},"min":{"1min":0.79800000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.7749999999999999,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.3149999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.0459999999999998,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.319,"15min":1.357},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.9299999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.1619999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.335,"15min":1.3819999999999999},"min":{"1min":0.83299999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.865,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.379,"15min":1.417},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.6870000000000001,"5min":2.109,"15min":2.109},"last":1.1279999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:57:07.732 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-14T11:57:07.733 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:57:07.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.916+0000 7f056316b700 1 Processor -- start 2023-07-14T11:57:07.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.916+0000 7f056316b700 1 -- start start 2023-07-14T11:57:07.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.920+0000 7f056316b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f055c0b0070 0x7f055c060320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.920+0000 7f056316b700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055c158730 con 0x7f055c071070 2023-07-14T11:57:07.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.920+0000 7f056316b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055c0c9140 con 0x7f055c0b4310 2023-07-14T11:57:07.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.920+0000 7f056316b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055c0afe90 con 0x7f055c0b0070 2023-07-14T11:57:07.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.920+0000 7f056296a700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f055c071070 0x7f055c163100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53696/0 (socket says 172.21.15.104:53696) 2023-07-14T11:57:07.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.920+0000 7f056296a700 1 -- 172.21.15.104:0/3186741979 learned_addr learned my addr 172.21.15.104:0/3186741979 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.920+0000 7f0562169700 1 --2- 172.21.15.104:0/3186741979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f055c0b0070 0x7f055c060320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f0561167700 1 -- 172.21.15.104:0/3186741979 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320804709 0 0) 0x7f0554001490 con 0x7f055c071070 2023-07-14T11:57:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f0561167700 1 -- 172.21.15.104:0/3186741979 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05480057e0 con 0x7f055c071070 2023-07-14T11:57:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f0561167700 1 -- 172.21.15.104:0/3186741979 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576743153 0 0) 0x7f055c0c9140 con 0x7f055c0b4310 2023-07-14T11:57:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f0562169700 1 -- 172.21.15.104:0/3186741979 >> v1:172.21.15.121:6789/0 conn(0x7f055c0b4310 legacy=0x7f055c070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f0562169700 1 -- 172.21.15.104:0/3186741979 >> v1:172.21.15.104:6789/0 conn(0x7f055c071070 legacy=0x7f055c163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f0562169700 1 -- 172.21.15.104:0/3186741979 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c0c9420 con 0x7f055c0b0070 2023-07-14T11:57:07.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f0562169700 1 --2- 172.21.15.104:0/3186741979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f055c0b0070 0x7f055c060320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0558009880 tx=0x7f0558009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d7fc8830a8bb0d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:07.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f0561167700 1 -- 172.21.15.104:0/3186741979 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558011830 con 0x7f055c0b0070 2023-07-14T11:57:07.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f0561167700 1 -- 172.21.15.104:0/3186741979 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0558011990 con 0x7f055c0b0070 2023-07-14T11:57:07.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f0561167700 1 -- 172.21.15.104:0/3186741979 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558011d20 con 0x7f055c0b0070 2023-07-14T11:57:07.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f056316b700 1 -- 172.21.15.104:0/3186741979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f055c0b0070 msgr2=0x7f055c060320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.911 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f056316b700 1 --2- 172.21.15.104:0/3186741979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f055c0b0070 0x7f055c060320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0558009880 tx=0x7f0558009b90 comp rx=0 tx=0).stop 2023-07-14T11:57:07.911 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f056316b700 1 -- 172.21.15.104:0/3186741979 shutdown_connections 2023-07-14T11:57:07.911 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f056316b700 1 --2- 172.21.15.104:0/3186741979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f055c0b0070 0x7f055c060320 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f056316b700 1 -- 172.21.15.104:0/3186741979 >> 172.21.15.104:0/3186741979 conn(0x7f055c0b04e0 msgr2=0x7f055c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:07.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f056316b700 1 -- 172.21.15.104:0/3186741979 shutdown_connections 2023-07-14T11:57:07.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.924+0000 7f056316b700 1 -- 172.21.15.104:0/3186741979 wait complete. 2023-07-14T11:57:07.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.928+0000 7f056316b700 1 Processor -- start 2023-07-14T11:57:07.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.928+0000 7f056316b700 1 -- start start 2023-07-14T11:57:07.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.928+0000 7f056316b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f055c071070 0x7f055c147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.928+0000 7f056316b700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f055c0b0070 0x7f055c06e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.928+0000 7f056316b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f055c0b4310 0x7f055c06eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.928+0000 7f056316b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c1dd5e0 con 0x7f055c071070 2023-07-14T11:57:07.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.928+0000 7f056316b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c1e1b20 con 0x7f055c0b4310 2023-07-14T11:57:07.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.928+0000 7f056316b700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055c1e1e00 con 0x7f055c0b0070 2023-07-14T11:57:07.916 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.928+0000 7f0562169700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f055c071070 0x7f055c147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f0562169700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f055c071070 0x7f055c147d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46552/0 (socket says 172.21.15.104:46552) 2023-07-14T11:57:07.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f0562169700 1 -- 172.21.15.104:0/1562227672 learned_addr learned my addr 172.21.15.104:0/1562227672 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:07.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f056296a700 1 --2- 172.21.15.104:0/1562227672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f055c0b4310 0x7f055c06eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f0561968700 1 --2- 172.21.15.104:0/1562227672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f055c0b0070 0x7f055c06e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f0562169700 1 -- 172.21.15.104:0/1562227672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f055c0b0070 msgr2=0x7f055c06e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f0562169700 1 --2- 172.21.15.104:0/1562227672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f055c0b0070 0x7f055c06e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f0562169700 1 -- 172.21.15.104:0/1562227672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f055c0b4310 msgr2=0x7f055c06eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:07.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f0562169700 1 --2- 172.21.15.104:0/1562227672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f055c0b4310 0x7f055c06eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:07.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f0562169700 1 -- 172.21.15.104:0/1562227672 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05580094b0 con 0x7f055c071070 2023-07-14T11:57:07.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f056296a700 1 --2- 172.21.15.104:0/1562227672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f055c0b4310 0x7f055c06eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:57:07.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f0562169700 1 --2- 172.21.15.104:0/1562227672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f055c071070 0x7f055c147d10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0558009bd0 tx=0x7f0558004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:07.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f0552ffd700 1 -- 172.21.15.104:0/1562227672 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558019a80 con 0x7f055c071070 2023-07-14T11:57:07.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f0552ffd700 1 -- 172.21.15.104:0/1562227672 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055801e070 con 0x7f055c071070 2023-07-14T11:57:07.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f056316b700 1 -- 172.21.15.104:0/1562227672 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c1e20e0 con 0x7f055c071070 2023-07-14T11:57:07.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f0552ffd700 1 -- 172.21.15.104:0/1562227672 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558019e40 con 0x7f055c071070 2023-07-14T11:57:07.920 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f056316b700 1 -- 172.21.15.104:0/1562227672 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c148250 con 0x7f055c071070 2023-07-14T11:57:07.920 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.932+0000 7f056316b700 1 -- 172.21.15.104:0/1562227672 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055c04e500 con 0x7f055c071070 2023-07-14T11:57:07.920 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.936+0000 7f0552ffd700 1 -- 172.21.15.104:0/1562227672 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f055802d450 con 0x7f055c071070 2023-07-14T11:57:07.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.936+0000 7f0552ffd700 1 --2- 172.21.15.104:0/1562227672 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f05400662f0 0x7f05400687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:07.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.936+0000 7f0552ffd700 1 -- 172.21.15.104:0/1562227672 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f0558084cb0 con 0x7f055c071070 2023-07-14T11:57:07.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.936+0000 7f0561968700 1 --2- 172.21.15.104:0/1562227672 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f05400662f0 0x7f05400687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:07.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.940+0000 7f0561968700 1 --2- 172.21.15.104:0/1562227672 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f05400662f0 0x7f05400687a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f055c06fb50 tx=0x7f054c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:07.928 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:07.944+0000 7f0552ffd700 1 -- 172.21.15.104:0/1562227672 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f05580523b0 con 0x7f055c071070 2023-07-14T11:57:08.094 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.108+0000 7f056316b700 1 -- 172.21.15.104:0/1562227672 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f055c0c9700 con 0x7f05400662f0 2023-07-14T11:57:08.100 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.116+0000 7f0552ffd700 1 -- 172.21.15.104:0/1562227672 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70690 (secure 0 0 0) 0x7f054c0013d0 con 0x7f05400662f0 2023-07-14T11:57:08.101 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:57:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.120+0000 7f056316b700 1 -- 172.21.15.104:0/1562227672 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f05400662f0 msgr2=0x7f05400687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.120+0000 7f056316b700 1 --2- 172.21.15.104:0/1562227672 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f05400662f0 0x7f05400687a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f055c06fb50 tx=0x7f054c009040 comp rx=0 tx=0).stop 2023-07-14T11:57:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.120+0000 7f056316b700 1 -- 172.21.15.104:0/1562227672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f055c071070 msgr2=0x7f055c147d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.120+0000 7f056316b700 1 --2- 172.21.15.104:0/1562227672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f055c071070 0x7f055c147d10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0558009bd0 tx=0x7f0558004770 comp rx=0 tx=0).stop 2023-07-14T11:57:08.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.120+0000 7f056316b700 1 -- 172.21.15.104:0/1562227672 shutdown_connections 2023-07-14T11:57:08.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.120+0000 7f056316b700 1 --2- 172.21.15.104:0/1562227672 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f05400662f0 0x7f05400687a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:08.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.120+0000 7f056316b700 1 --2- 172.21.15.104:0/1562227672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f055c0b4310 0x7f055c06eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:08.106 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.120+0000 7f056316b700 1 --2- 172.21.15.104:0/1562227672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f055c0b0070 0x7f055c06e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:08.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.120+0000 7f056316b700 1 --2- 172.21.15.104:0/1562227672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f055c071070 0x7f055c147d10 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:08.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.120+0000 7f056316b700 1 -- 172.21.15.104:0/1562227672 >> 172.21.15.104:0/1562227672 conn(0x7f055c0b04e0 msgr2=0x7f055c0b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:08.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.120+0000 7f056316b700 1 -- 172.21.15.104:0/1562227672 shutdown_connections 2023-07-14T11:57:08.107 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.120+0000 7f056316b700 1 -- 172.21.15.104:0/1562227672 wait complete. 2023-07-14T11:57:08.108 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:57:08.128 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2023-07-14T11:57:07.979318+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235144,"kb_used_data":7240,"kb_used_omap":0,"kb_used_meta":227840,"kb_avail":754739576,"statfs":{"total":773094113280,"available":772853325824,"internally_reserved":0,"allocated":7413760,"data_stored":4109128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":233308160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16523,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-72,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.677059"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":100,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.864337+0000","last_change":"2023-07-14T11:53:09.145201+0000","last_active":"2023-07-14T11:57:04.864337+0000","last_peered":"2023-07-14T11:57:04.864337+0000","last_clean":"2023-07-14T11:57:04.864337+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:57:04.864337+0000","last_undegraded":"2023-07-14T11:57:04.864337+0000","last_fullsized":"2023-07-14T11:57:04.864337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:18:49.597233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":88,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.980029+0000","last_change":"2023-07-14T11:53:09.146150+0000","last_active":"2023-07-14T11:57:04.980029+0000","last_peered":"2023-07-14T11:57:04.980029+0000","last_clean":"2023-07-14T11:57:04.980029+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:57:04.980029+0000","last_undegraded":"2023-07-14T11:57:04.980029+0000","last_fullsized":"2023-07-14T11:57:04.980029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:35:34.359651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":86,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.975835+0000","last_change":"2023-07-14T11:53:09.435876+0000","last_active":"2023-07-14T11:57:04.975835+0000","last_peered":"2023-07-14T11:57:04.975835+0000","last_clean":"2023-07-14T11:57:04.975835+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:57:04.975835+0000","last_undegraded":"2023-07-14T11:57:04.975835+0000","last_fullsized":"2023-07-14T11:57:04.975835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:19:05.382083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":88,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.980620+0000","last_change":"2023-07-14T11:53:09.434255+0000","last_active":"2023-07-14T11:57:04.980620+0000","last_peered":"2023-07-14T11:57:04.980620+0000","last_clean":"2023-07-14T11:57:04.980620+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:57:04.980620+0000","last_undegraded":"2023-07-14T11:57:04.980620+0000","last_fullsized":"2023-07-14T11:57:04.980620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:09:49.234144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":98,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.908750+0000","last_change":"2023-07-14T11:53:09.147040+0000","last_active":"2023-07-14T11:57:04.908750+0000","last_peered":"2023-07-14T11:57:04.908750+0000","last_clean":"2023-07-14T11:57:04.908750+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:57:04.908750+0000","last_undegraded":"2023-07-14T11:57:04.908750+0000","last_fullsized":"2023-07-14T11:57:04.908750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:35:55.791012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":100,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.982691+0000","last_change":"2023-07-14T11:53:09.619294+0000","last_active":"2023-07-14T11:57:04.982691+0000","last_peered":"2023-07-14T11:57:04.982691+0000","last_clean":"2023-07-14T11:57:04.982691+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:57:04.982691+0000","last_undegraded":"2023-07-14T11:57:04.982691+0000","last_fullsized":"2023-07-14T11:57:04.982691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:29:22.000512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":86,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.976637+0000","last_change":"2023-07-14T11:53:09.146156+0000","last_active":"2023-07-14T11:57:04.976637+0000","last_peered":"2023-07-14T11:57:04.976637+0000","last_clean":"2023-07-14T11:57:04.976637+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:57:04.976637+0000","last_undegraded":"2023-07-14T11:57:04.976637+0000","last_fullsized":"2023-07-14T11:57:04.976637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:21:36.155230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":98,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.910974+0000","last_change":"2023-07-14T11:53:09.146556+0000","last_active":"2023-07-14T11:57:04.910974+0000","last_peered":"2023-07-14T11:57:04.910974+0000","last_clean":"2023-07-14T11:57:04.910974+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:57:04.910974+0000","last_undegraded":"2023-07-14T11:57:04.910974+0000","last_fullsized":"2023-07-14T11:57:04.910974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:06:38.512393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":243,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.976949+0000","last_change":"2023-07-14T11:53:03.159818+0000","last_active":"2023-07-14T11:57:04.976949+0000","last_peered":"2023-07-14T11:57:04.976949+0000","last_clean":"2023-07-14T11:57:04.976949+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:57:04.976949+0000","last_undegraded":"2023-07-14T11:57:04.976949+0000","last_fullsized":"2023-07-14T11:57:04.976949+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_clean_scrub_stamp":"2023-07-14T11:53:02.054348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:00:32.142331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1372160,"data_stored":956417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.093,"15min":1.111},"min":{"1min":0.754,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.474,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.103,"15min":1.127},"min":{"1min":0.65100000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2430000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.0669999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1499999999999999,"15min":1.171},"min":{"1min":0.68300000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.298,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.012},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.22,"15min":1.262},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.4159999999999999,"5min":2.234,"15min":2.234},"last":0.98399999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.234,"15min":1.288},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.536,"5min":2.04,"15min":2.04},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.25,"15min":1.2829999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8089999999999999,"5min":2.492,"15min":2.492},"last":1.2809999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.282,"15min":1.3360000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.665,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.2430000000000001,"15min":1.274},"min":{"1min":0.81799999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5940000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.222}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.321,"15min":1.385},"min":{"1min":0.84399999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5660000000000001,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.286,"5min":1.3400000000000001,"15min":1.3799999999999999},"min":{"1min":0.92500000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.76,"5min":2.399,"15min":2.399},"last":1.244}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.284,"15min":1.3200000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6459999999999999,"5min":2.387,"15min":2.387},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.456,"15min":1.538},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.464,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":1.129}]},{"osd":6,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.5309999999999999,"15min":1.609},"min":{"1min":1.0940000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5289999999999999,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.4550000000000001,"15min":1.5229999999999999},"min":{"1min":0.95199999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.53,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.331}]}]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0700000000000001,"15min":1.0620000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.397,"5min":1.954,"15min":1.954},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.165,"5min":1.1160000000000001,"15min":1.1000000000000001},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.3560000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1890000000000001,"15min":1.2090000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7110000000000001,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.121},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.214,"15min":1.23},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7769999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.752}]},{"osd":2,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.171,"15min":1.1659999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4690000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.163},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2709999999999999,"15min":1.2729999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6819999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.3109999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.3140000000000001,"15min":1.3440000000000001},"min":{"1min":0.94099999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.6579999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.218},{"interface":"front","average":{"1min":1.258,"5min":1.1899999999999999,"15min":1.1799999999999999},"min":{"1min":0.91700000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.482,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.2649999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.236,"15min":1.2529999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.625,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.3129999999999999,"15min":1.3400000000000001},"min":{"1min":0.78000000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.512,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.4510000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.238,"15min":1.2270000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.488},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4039999999999999,"15min":1.425},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.9139999999999999,"5min":1.984,"15min":1.984},"last":1.5289999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.4330000000000001,"15min":1.4650000000000001},"min":{"1min":0.99199999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.6519999999999999,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.365,"15min":1.379},"min":{"1min":0.92900000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.728,"5min":1.879,"15min":1.879},"last":1.677}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.1359999999999999,"15min":1.1819999999999999},"min":{"1min":0.68100000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4790000000000001,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.129,"15min":1.175},"min":{"1min":0.70299999999999996,"5min":0.622,"15min":0.622},"max":{"1min":1.474,"5min":1.581,"15min":1.581},"last":1.222}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.167,"15min":1.1990000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.524,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.175,"15min":1.1890000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.498,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.84599999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.198,"15min":1.226},"min":{"1min":0.86699999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4550000000000001,"5min":1.839,"15min":1.839},"last":1.355},{"interface":"front","average":{"1min":1.113,"5min":1.282,"15min":1.347},"min":{"1min":0.73799999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4990000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.92000000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.272,"15min":1.3220000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.5409999999999999,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.3440000000000001,"15min":1.3919999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.5349999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.105}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2090000000000001,"15min":1.2490000000000001},"min":{"1min":0.75900000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5900000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.284},{"interface":"front","average":{"1min":1.321,"5min":1.3540000000000001,"15min":1.3740000000000001},"min":{"1min":1.024,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.5109999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.4710000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3959999999999999,"15min":1.4179999999999999},"min":{"1min":1.1459999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.544,"5min":1.99,"15min":1.99},"last":1.208},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.375,"15min":1.4159999999999999},"min":{"1min":0.86399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6779999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.4470000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4039999999999999,"15min":1.421},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.6220000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.581},{"interface":"front","average":{"1min":1.387,"5min":1.5569999999999999,"15min":1.6160000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.5880000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.1539999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.149,"15min":1.171},"min":{"1min":0.67200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.5069999999999999,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.133,"15min":1.1599999999999999},"min":{"1min":0.71099999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.161}]},{"osd":1,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.159,"15min":1.1819999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.508,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1879999999999999,"15min":1.2090000000000001},"min":{"1min":0.75900000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.516,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.27}]},{"osd":2,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1359999999999999,"15min":1.143},"min":{"1min":0.83899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.546,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3400000000000001,"15min":1.3620000000000001},"min":{"1min":0.96599999999999997,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.581,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":1.5349999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2789999999999999,"15min":1.321},"min":{"1min":0.82199999999999995,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.681,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2270000000000001,"15min":1.2450000000000001},"min":{"1min":0.88100000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.4770000000000001,"5min":1.823,"15min":1.823},"last":1.0029999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2989999999999999,"15min":1.3420000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6299999999999999,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.89500000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.47,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.97999999999999998}]},{"osd":6,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.264,"15min":1.2669999999999999},"min":{"1min":0.97799999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.522,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.476,"15min":1.532},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6759999999999999,"5min":3.5289999999999999,"15min":3.5289999999999999},"last":1.3340000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.466,"15min":1.5049999999999999},"min":{"1min":1.0589999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.6899999999999999,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.47,"15min":1.5209999999999999},"min":{"1min":1.0640000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":1.605,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":1.365}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.84099999999999997,"15min":0.871},"min":{"1min":0.47699999999999998,"5min":0.247,"15min":0.247},"max":{"1min":1.234,"5min":1.325,"15min":1.325},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.875,"15min":0.90800000000000003},"min":{"1min":0.60599999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2490000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.90800000000000003}]},{"osd":1,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92300000000000004,"15min":0.96299999999999997},"min":{"1min":0.52600000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6950000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.0600000000000001,"15min":1.111},"min":{"1min":0.54300000000000004,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.319,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":1.3129999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.097,"15min":1.141},"min":{"1min":0.63500000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4259999999999999,"5min":1.627,"15min":1.627},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.079,"5min":1.0669999999999999,"15min":1.091},"min":{"1min":0.73399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.84199999999999997}]},{"osd":4,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2410000000000001,"15min":1.2849999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.53,"5min":2.335,"15min":2.335},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.2470000000000001,"15min":1.2869999999999999},"min":{"1min":0.77700000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.2190000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.323,"15min":1.359},"min":{"1min":0.94999999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.28},{"interface":"front","average":{"1min":1.212,"5min":1.3580000000000001,"15min":1.421},"min":{"1min":0.86899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.532,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.3560000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.329,"15min":1.387},"min":{"1min":0.85899999999999999,"5min":0.625,"15min":0.625},"max":{"1min":1.7010000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.125},{"interface":"front","average":{"1min":1.425,"5min":1.4550000000000001,"15min":1.5009999999999999},"min":{"1min":1.083,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.0790000000000002,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.3300000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:56:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.419,"15min":1.4810000000000001},"min":{"1min":0.90500000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.712,"5min":3.395,"15min":3.395},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.367,"15min":1.431},"min":{"1min":1.0680000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.619,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.0389999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.874,"15min":0.90600000000000003},"min":{"1min":0.53800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.22,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.876,"15min":0.90200000000000002},"min":{"1min":0.48799999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1799999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.93200000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.88100000000000001,"15min":0.89900000000000002},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.093,"5min":1.365,"15min":1.365},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0269999999999999,"15min":1.069},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.425,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.04}]},{"osd":3,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0649999999999999,"15min":1.0960000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5089999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.002},{"interface":"front","average":{"1min":1.018,"5min":1.026,"15min":1.052},"min":{"1min":0.65300000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.5900000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.63}]},{"osd":4,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2709999999999999,"15min":1.2989999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.718,"5min":2.097,"15min":2.097},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.242,"5min":1.2769999999999999,"15min":1.3149999999999999},"min":{"1min":0.67800000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.861,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.1839999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.2889999999999999,"15min":1.319},"min":{"1min":0.95699999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.4710000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.087},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3180000000000001,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.645,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.429}]},{"osd":6,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.333,"15min":1.383},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7649999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.393},{"interface":"front","average":{"1min":1.361,"5min":1.3919999999999999,"15min":1.425},"min":{"1min":1.1040000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.726,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.27}]},{"osd":7,"last update":"Fri Jul 14 11:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.397,"15min":1.444},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.677,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.304},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4219999999999999,"15min":1.466},"min":{"1min":1.01,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6899999999999999,"5min":2.536,"15min":2.536},"last":1.2110000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.80100000000000005,"15min":0.83299999999999996},"min":{"1min":0.48899999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.26,"5min":1.349,"15min":1.349},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83299999999999996,"15min":0.83399999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84799999999999998,"15min":0.85799999999999998},"min":{"1min":0.499,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.93500000000000005,"15min":0.93999999999999995},"min":{"1min":0.55900000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":1.0469999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.99199999999999999,"15min":1.002},"min":{"1min":0.78500000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.399,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0169999999999999,"15min":1.0229999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.8169999999999999,"5min":1.857,"15min":1.857},"last":0.88400000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.2729999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6279999999999999,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.1830000000000001,"15min":1.2170000000000001},"min":{"1min":0.754,"5min":0.745,"15min":0.745},"max":{"1min":1.6279999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.347}]},{"osd":5,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.23,"15min":1.256},"min":{"1min":0.876,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7190000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3149999999999999,"15min":1.365},"min":{"1min":0.93100000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.778,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.2829999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3560000000000001,"15min":1.387},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.9119999999999999,"5min":2.633,"15min":2.633},"last":1.542},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.335,"15min":1.359},"min":{"1min":1.091,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.873,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.3049999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3580000000000001,"15min":1.3979999999999999},"min":{"1min":1.004,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.992,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3300000000000001,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.754,"15min":0.754},"max":{"1min":2.056,"5min":2.375,"15min":2.375},"last":1.3839999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1372160,"data_stored":956417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.879,"15min":0.89300000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3100000000000001,"5min":1.581,"15min":1.581},"last":0.91300000000000003}]},{"osd":2,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0049999999999999,"15min":1},"min":{"1min":0.64800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.5880000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.129},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90800000000000003,"15min":0.95099999999999996},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.1639999999999999,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.0269999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.93600000000000005,"15min":0.94599999999999995},"min":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.228,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.002,"5min":1.099,"15min":1.1319999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3799999999999999,"5min":1.704,"15min":1.704},"last":1.079}]},{"osd":4,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2430000000000001,"15min":1.278},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5149999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3420000000000001,"15min":1.3720000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.837,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.2509999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.228,"15min":1.284},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9870000000000001,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.276,"15min":1.3200000000000001},"min":{"1min":0.79800000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.7749999999999999,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.149}]},{"osd":6,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.0459999999999998,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.47},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.319,"15min":1.357},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.9299999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.2330000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.335,"15min":1.3819999999999999},"min":{"1min":0.83299999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.865,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.331},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.379,"15min":1.417},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.6870000000000001,"5min":2.109,"15min":2.109},"last":1.4079999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:57:08.131 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:57:08.220 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:57:08.228 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:57:08.299 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.316+0000 7fbf1db4a700 1 Processor -- start 2023-07-14T11:57:08.300 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.316+0000 7fbf1db4a700 1 -- start start 2023-07-14T11:57:08.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.316+0000 7fbf1db4a700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbf181599d0 0x7fbf18071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:08.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.316+0000 7fbf1db4a700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbf180caa20 0x7fbf180cae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:08.301 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.316+0000 7fbf1db4a700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf180cde10 con 0x7fbf1815ccf0 2023-07-14T11:57:08.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.316+0000 7fbf1db4a700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf180cb380 con 0x7fbf180caa20 2023-07-14T11:57:08.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.316+0000 7fbf1db4a700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf180cd520 con 0x7fbf181599d0 2023-07-14T11:57:08.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.316+0000 7fbf1d349700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbf180caa20 0x7fbf180cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:08.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.316+0000 7fbf1d349700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fbf180caa20 0x7fbf180cae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46558/0 (socket says 172.21.15.104:46558) 2023-07-14T11:57:08.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.316+0000 7fbf1d349700 1 -- 172.21.15.104:0/2778059003 learned_addr learned my addr 172.21.15.104:0/2778059003 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:08.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.316+0000 7fbf1cb48700 1 --2- 172.21.15.104:0/2778059003 >> v2:172.21.15.121:3300/0 conn(0x7fbf181599d0 0x7fbf18071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:08.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.316+0000 7fbf1d349700 1 -- 172.21.15.104:0/2778059003 >> v2:172.21.15.121:3300/0 conn(0x7fbf181599d0 msgr2=0x7fbf18071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:08.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.316+0000 7fbf1d349700 1 --2- 172.21.15.104:0/2778059003 >> v2:172.21.15.121:3300/0 conn(0x7fbf181599d0 0x7fbf18071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:08.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.316+0000 7fbf1d349700 1 -- 172.21.15.104:0/2778059003 >> v1:172.21.15.121:6789/0 conn(0x7fbf1815ccf0 legacy=0x7fbf180c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:08.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.316+0000 7fbf1d349700 1 -- 172.21.15.104:0/2778059003 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1815d190 con 0x7fbf180caa20 2023-07-14T11:57:08.304 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.320+0000 7fbf1d349700 1 --2- 172.21.15.104:0/2778059003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf180caa20 0x7fbf180cae40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbf180cf0d0 tx=0x7fbf1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e83eafbf36193db server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:08.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.320+0000 7fbf177fe700 1 -- 172.21.15.104:0/2778059003 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1000e070 con 0x7fbf180caa20 2023-07-14T11:57:08.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.320+0000 7fbf177fe700 1 -- 172.21.15.104:0/2778059003 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1000b910 con 0x7fbf180caa20 2023-07-14T11:57:08.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.320+0000 7fbf177fe700 1 -- 172.21.15.104:0/2778059003 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1000bcc0 con 0x7fbf180caa20 2023-07-14T11:57:08.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.320+0000 7fbf1db4a700 1 -- 172.21.15.104:0/2778059003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf180caa20 msgr2=0x7fbf180cae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:08.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.320+0000 7fbf1db4a700 1 --2- 172.21.15.104:0/2778059003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf180caa20 0x7fbf180cae40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbf180cf0d0 tx=0x7fbf1000b330 comp rx=0 tx=0).stop 2023-07-14T11:57:08.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.320+0000 7fbf1db4a700 1 -- 172.21.15.104:0/2778059003 shutdown_connections 2023-07-14T11:57:08.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.320+0000 7fbf1db4a700 1 --2- 172.21.15.104:0/2778059003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf180caa20 0x7fbf180cae40 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:08.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.320+0000 7fbf1db4a700 1 --2- 172.21.15.104:0/2778059003 >> v2:172.21.15.121:3300/0 conn(0x7fbf181599d0 0x7fbf18071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:08.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.320+0000 7fbf1db4a700 1 -- 172.21.15.104:0/2778059003 >> 172.21.15.104:0/2778059003 conn(0x7fbf1806e960 msgr2=0x7fbf1806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:08.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.320+0000 7fbf1db4a700 1 -- 172.21.15.104:0/2778059003 shutdown_connections 2023-07-14T11:57:08.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.320+0000 7fbf1db4a700 1 -- 172.21.15.104:0/2778059003 wait complete. 2023-07-14T11:57:08.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1db4a700 1 Processor -- start 2023-07-14T11:57:08.308 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1db4a700 1 -- start start 2023-07-14T11:57:08.309 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1db4a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf180caa20 0x7fbf18153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:08.309 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1db4a700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf181599d0 0x7fbf180c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:08.309 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1db4a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf1815ccf0 0x7fbf18159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:08.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1db4a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf18153a90 con 0x7fbf180caa20 2023-07-14T11:57:08.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1db4a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180b1c30 con 0x7fbf1815ccf0 2023-07-14T11:57:08.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1db4a700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180b1020 con 0x7fbf181599d0 2023-07-14T11:57:08.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1cb48700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf180caa20 0x7fbf18153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:08.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1cb48700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf180caa20 0x7fbf18153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46562/0 (socket says 172.21.15.104:46562) 2023-07-14T11:57:08.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1cb48700 1 -- 172.21.15.104:0/1713720755 learned_addr learned my addr 172.21.15.104:0/1713720755 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:08.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf17fff700 1 --2- 172.21.15.104:0/1713720755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf181599d0 0x7fbf180c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:08.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1d349700 1 --2- 172.21.15.104:0/1713720755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf1815ccf0 0x7fbf18159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:08.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1cb48700 1 -- 172.21.15.104:0/1713720755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf181599d0 msgr2=0x7fbf180c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:08.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1cb48700 1 --2- 172.21.15.104:0/1713720755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf181599d0 0x7fbf180c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:08.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1cb48700 1 -- 172.21.15.104:0/1713720755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf1815ccf0 msgr2=0x7fbf18159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:08.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1cb48700 1 --2- 172.21.15.104:0/1713720755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf1815ccf0 0x7fbf18159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:08.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1cb48700 1 -- 172.21.15.104:0/1713720755 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10009b30 con 0x7fbf180caa20 2023-07-14T11:57:08.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1cb48700 1 --2- 172.21.15.104:0/1713720755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf180caa20 0x7fbf18153550 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00e9d0 tx=0x7fbf0c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:08.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf157fa700 1 -- 172.21.15.104:0/1713720755 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c009720 con 0x7fbf180caa20 2023-07-14T11:57:08.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf157fa700 1 -- 172.21.15.104:0/1713720755 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0c00ee70 con 0x7fbf180caa20 2023-07-14T11:57:08.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.324+0000 7fbf1db4a700 1 -- 172.21.15.104:0/1713720755 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf180b1300 con 0x7fbf180caa20 2023-07-14T11:57:08.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.328+0000 7fbf157fa700 1 -- 172.21.15.104:0/1713720755 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c00ca50 con 0x7fbf180caa20 2023-07-14T11:57:08.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.328+0000 7fbf1db4a700 1 -- 172.21.15.104:0/1713720755 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf180b18c0 con 0x7fbf180caa20 2023-07-14T11:57:08.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.328+0000 7fbf1db4a700 1 -- 172.21.15.104:0/1713720755 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1804e500 con 0x7fbf180caa20 2023-07-14T11:57:08.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.328+0000 7fbf157fa700 1 -- 172.21.15.104:0/1713720755 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fbf0c00cbb0 con 0x7fbf180caa20 2023-07-14T11:57:08.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.332+0000 7fbf157fa700 1 --2- 172.21.15.104:0/1713720755 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbefc0662f0 0x7fbefc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:08.316 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.332+0000 7fbf157fa700 1 -- 172.21.15.104:0/1713720755 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fbf0c083e20 con 0x7fbf180caa20 2023-07-14T11:57:08.316 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.332+0000 7fbf17fff700 1 --2- 172.21.15.104:0/1713720755 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbefc0662f0 0x7fbefc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:08.317 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.332+0000 7fbf17fff700 1 --2- 172.21.15.104:0/1713720755 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbefc0662f0 0x7fbefc0687a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbf181606a0 tx=0x7fbf04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:08.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.340+0000 7fbf157fa700 1 -- 172.21.15.104:0/1713720755 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fbf0c054f80 con 0x7fbf180caa20 2023-07-14T11:57:08.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.508+0000 7fbf1db4a700 1 -- 172.21.15.104:0/1713720755 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf18070de0 con 0x7fbf180caa20 2023-07-14T11:57:08.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.512+0000 7fbf157fa700 1 -- 172.21.15.104:0/1713720755 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+15012 (secure 0 0 0) 0x7fbf0c017090 con 0x7fbf180caa20 2023-07-14T11:57:08.497 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:57:08.497 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":64,"fsid":"c5576993-99e1-43b1-a253-659a3cee5212","created":"2023-07-14T11:52:47.742995+0000","modified":"2023-07-14T11:57:04.843600+0000","last_up_change":"2023-07-14T11:53:00.012117+0000","last_in_change":"2023-07-14T11:52:50.810860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:53:01.976900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-14T11:53:07.267421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e83f7df-2e45-460d-a746-d48694cee08b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6803","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6807","nonce":3734215753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6811","nonce":3734215753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6815","nonce":3734215753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3734215753},{"type":"v1","addr":"172.21.15.104:6813","nonce":3734215753}]},"public_addr":"172.21.15.104:6807/3734215753","cluster_addr":"172.21.15.104:6811/3734215753","heartbeat_back_addr":"172.21.15.104:6815/3734215753","heartbeat_front_addr":"172.21.15.104:6813/3734215753","state":["exists","up"]},{"osd":1,"uuid":"c4210db3-7801-45f7-89d2-811b2327e932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6825","nonce":2317846209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6827","nonce":2317846209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6831","nonce":2317846209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2317846209},{"type":"v1","addr":"172.21.15.104:6829","nonce":2317846209}]},"public_addr":"172.21.15.104:6825/2317846209","cluster_addr":"172.21.15.104:6827/2317846209","heartbeat_back_addr":"172.21.15.104:6831/2317846209","heartbeat_front_addr":"172.21.15.104:6829/2317846209","state":["exists","up"]},{"osd":2,"uuid":"85424c50-7fbb-43e9-8de1-e9acdc98e422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6801","nonce":3827055184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6804","nonce":3827055184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6809","nonce":3827055184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":3827055184},{"type":"v1","addr":"172.21.15.104:6806","nonce":3827055184}]},"public_addr":"172.21.15.104:6801/3827055184","cluster_addr":"172.21.15.104:6804/3827055184","heartbeat_back_addr":"172.21.15.104:6809/3827055184","heartbeat_front_addr":"172.21.15.104:6806/3827055184","state":["exists","up"]},{"osd":3,"uuid":"75db666c-73d2-4934-8417-4ce190953af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6817","nonce":831948081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6819","nonce":831948081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6823","nonce":831948081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":831948081},{"type":"v1","addr":"172.21.15.104:6821","nonce":831948081}]},"public_addr":"172.21.15.104:6817/831948081","cluster_addr":"172.21.15.104:6819/831948081","heartbeat_back_addr":"172.21.15.104:6823/831948081","heartbeat_front_addr":"172.21.15.104:6821/831948081","state":["exists","up"]},{"osd":4,"uuid":"d2ea3990-a14c-4c2e-9655-3928c83b1b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6825","nonce":2394615815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6827","nonce":2394615815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6831","nonce":2394615815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2394615815},{"type":"v1","addr":"172.21.15.121:6829","nonce":2394615815}]},"public_addr":"172.21.15.121:6825/2394615815","cluster_addr":"172.21.15.121:6827/2394615815","heartbeat_back_addr":"172.21.15.121:6831/2394615815","heartbeat_front_addr":"172.21.15.121:6829/2394615815","state":["exists","up"]},{"osd":5,"uuid":"52f5a094-1bdd-4e2d-a6f0-6f5ba37eb57c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6801","nonce":1185793020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6803","nonce":1185793020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6807","nonce":1185793020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1185793020},{"type":"v1","addr":"172.21.15.121:6805","nonce":1185793020}]},"public_addr":"172.21.15.121:6801/1185793020","cluster_addr":"172.21.15.121:6803/1185793020","heartbeat_back_addr":"172.21.15.121:6807/1185793020","heartbeat_front_addr":"172.21.15.121:6805/1185793020","state":["exists","up"]},{"osd":6,"uuid":"8546ebcd-0a98-424c-b845-13ce61b9cea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6817","nonce":754390529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6819","nonce":754390529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6824","nonce":754390529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":754390529},{"type":"v1","addr":"172.21.15.121:6821","nonce":754390529}]},"public_addr":"172.21.15.121:6817/754390529","cluster_addr":"172.21.15.121:6819/754390529","heartbeat_back_addr":"172.21.15.121:6824/754390529","heartbeat_front_addr":"172.21.15.121:6821/754390529","state":["exists","up"]},{"osd":7,"uuid":"c406079a-1e13-45eb-850c-f2eb69790571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6809","nonce":2610729951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6811","nonce":2610729951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6815","nonce":2610729951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2610729951},{"type":"v1","addr":"172.21.15.121:6813","nonce":2610729951}]},"public_addr":"172.21.15.121:6809/2610729951","cluster_addr":"172.21.15.121:6811/2610729951","heartbeat_back_addr":"172.21.15.121:6815/2610729951","heartbeat_front_addr":"172.21.15.121:6813/2610729951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.507508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.609558+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.523500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.608344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.813551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.559693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.466391+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:52:56.805030+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.104:6836/2705739882":"2023-07-15T11:57:01.293544+0000","172.21.15.104:6834/2705739882":"2023-07-15T11:57:01.293544+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:57:08.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.516+0000 7fbf1db4a700 1 -- 172.21.15.104:0/1713720755 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbefc0662f0 msgr2=0x7fbefc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:08.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.516+0000 7fbf1db4a700 1 --2- 172.21.15.104:0/1713720755 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbefc0662f0 0x7fbefc0687a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbf181606a0 tx=0x7fbf04009040 comp rx=0 tx=0).stop 2023-07-14T11:57:08.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.516+0000 7fbf1db4a700 1 -- 172.21.15.104:0/1713720755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf180caa20 msgr2=0x7fbf18153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:08.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.516+0000 7fbf1db4a700 1 --2- 172.21.15.104:0/1713720755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf180caa20 0x7fbf18153550 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00e9d0 tx=0x7fbf0c00ece0 comp rx=0 tx=0).stop 2023-07-14T11:57:08.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.516+0000 7fbf1db4a700 1 -- 172.21.15.104:0/1713720755 shutdown_connections 2023-07-14T11:57:08.502 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.516+0000 7fbf1db4a700 1 --2- 172.21.15.104:0/1713720755 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fbefc0662f0 0x7fbefc0687a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:08.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.516+0000 7fbf1db4a700 1 --2- 172.21.15.104:0/1713720755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf1815ccf0 0x7fbf18159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:08.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.516+0000 7fbf1db4a700 1 --2- 172.21.15.104:0/1713720755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf181599d0 0x7fbf180c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:08.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.516+0000 7fbf1db4a700 1 --2- 172.21.15.104:0/1713720755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf180caa20 0x7fbf18153550 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:08.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.516+0000 7fbf1db4a700 1 -- 172.21.15.104:0/1713720755 >> 172.21.15.104:0/1713720755 conn(0x7fbf1806e960 msgr2=0x7fbf1806d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:08.503 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.516+0000 7fbf1db4a700 1 -- 172.21.15.104:0/1713720755 shutdown_connections 2023-07-14T11:57:08.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:08.516+0000 7fbf1db4a700 1 -- 172.21.15.104:0/1713720755 wait complete. 2023-07-14T11:57:09.525 INFO:tasks.ceph:Scrubbing osd.0 2023-07-14T11:57:09.525 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-07-14T11:57:09.715 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.728+0000 7f8d23fff700 1 Processor -- start 2023-07-14T11:57:09.715 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.732+0000 7f8d23fff700 1 -- start start 2023-07-14T11:57:09.720 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.732+0000 7f8d23fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d24158aa0 0x7f8d2415d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:09.720 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.732+0000 7f8d23fff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8d240b1350 0x7f8d240cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:09.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.732+0000 7f8d23fff700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d240ccac0 con 0x7f8d2406cea0 2023-07-14T11:57:09.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.732+0000 7f8d23fff700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2415d600 con 0x7f8d240b1350 2023-07-14T11:57:09.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.732+0000 7f8d23fff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2406d3a0 con 0x7f8d24158aa0 2023-07-14T11:57:09.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.732+0000 7f8d237fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8d240b1350 0x7f8d240cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:09.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.732+0000 7f8d22ffd700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d24158aa0 0x7f8d2415d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:09.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.732+0000 7f8d237fe700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8d240b1350 0x7f8d240cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46576/0 (socket says 172.21.15.104:46576) 2023-07-14T11:57:09.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.732+0000 7f8d237fe700 1 -- 172.21.15.104:0/3445144589 learned_addr learned my addr 172.21.15.104:0/3445144589 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:09.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.732+0000 7f8d227fc700 1 --1- 172.21.15.104:0/3445144589 >> v1:172.21.15.104:6789/0 conn(0x7f8d2406cea0 0x7f8d240b0c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53704/0 (socket says 172.21.15.104:53704) 2023-07-14T11:57:09.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d237fe700 1 -- 172.21.15.104:0/3445144589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d24158aa0 msgr2=0x7f8d2415d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:09.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d237fe700 1 --2- 172.21.15.104:0/3445144589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d24158aa0 0x7f8d2415d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:09.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d237fe700 1 -- 172.21.15.104:0/3445144589 >> v1:172.21.15.104:6789/0 conn(0x7f8d2406cea0 legacy=0x7f8d240b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:09.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d237fe700 1 -- 172.21.15.104:0/3445144589 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2406d680 con 0x7f8d240b1350 2023-07-14T11:57:09.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d237fe700 1 --2- 172.21.15.104:0/3445144589 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d240b1350 0x7f8d240cd170 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00d150 tx=0x7f8d1c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d3aebd099f0c07 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:09.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d21ffb700 1 -- 172.21.15.104:0/3445144589 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c010070 con 0x7f8d240b1350 2023-07-14T11:57:09.723 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d21ffb700 1 -- 172.21.15.104:0/3445144589 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c00dd10 con 0x7f8d240b1350 2023-07-14T11:57:09.724 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d21ffb700 1 -- 172.21.15.104:0/3445144589 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c00b310 con 0x7f8d240b1350 2023-07-14T11:57:09.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d23fff700 1 -- 172.21.15.104:0/3445144589 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d240b1350 msgr2=0x7f8d240cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:09.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d23fff700 1 --2- 172.21.15.104:0/3445144589 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d240b1350 0x7f8d240cd170 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00d150 tx=0x7f8d1c00d460 comp rx=0 tx=0).stop 2023-07-14T11:57:09.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d23fff700 1 -- 172.21.15.104:0/3445144589 shutdown_connections 2023-07-14T11:57:09.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d23fff700 1 --2- 172.21.15.104:0/3445144589 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d240b1350 0x7f8d240cd170 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:09.725 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d23fff700 1 --2- 172.21.15.104:0/3445144589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d24158aa0 0x7f8d2415d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:09.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d23fff700 1 -- 172.21.15.104:0/3445144589 >> 172.21.15.104:0/3445144589 conn(0x7f8d240cbc40 msgr2=0x7f8d240cc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:09.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d23fff700 1 -- 172.21.15.104:0/3445144589 shutdown_connections 2023-07-14T11:57:09.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.736+0000 7f8d23fff700 1 -- 172.21.15.104:0/3445144589 wait complete. 2023-07-14T11:57:09.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.740+0000 7f8d23fff700 1 Processor -- start 2023-07-14T11:57:09.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.740+0000 7f8d23fff700 1 -- start start 2023-07-14T11:57:09.727 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.740+0000 7f8d23fff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d2406cea0 0x7f8d240c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:09.728 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.740+0000 7f8d23fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d240b1350 0x7f8d240c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:09.728 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.740+0000 7f8d23fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d24158aa0 0x7f8d240b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:09.728 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.740+0000 7f8d23fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d240c7330 con 0x7f8d24158aa0 2023-07-14T11:57:09.728 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.740+0000 7f8d23fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d240c24d0 con 0x7f8d240b1350 2023-07-14T11:57:09.729 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.740+0000 7f8d23fff700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d240c6290 con 0x7f8d2406cea0 2023-07-14T11:57:09.729 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.740+0000 7f8d237fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d24158aa0 0x7f8d240b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:09.729 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.740+0000 7f8d237fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d24158aa0 0x7f8d240b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46582/0 (socket says 172.21.15.104:46582) 2023-07-14T11:57:09.729 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.740+0000 7f8d237fe700 1 -- 172.21.15.104:0/3368782170 learned_addr learned my addr 172.21.15.104:0/3368782170 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:09.729 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.744+0000 7f8d22ffd700 1 --2- 172.21.15.104:0/3368782170 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d2406cea0 0x7f8d240c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:09.730 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.744+0000 7f8d227fc700 1 --2- 172.21.15.104:0/3368782170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d240b1350 0x7f8d240c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:09.730 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.744+0000 7f8d237fe700 1 -- 172.21.15.104:0/3368782170 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d2406cea0 msgr2=0x7f8d240c1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:09.730 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.744+0000 7f8d237fe700 1 --2- 172.21.15.104:0/3368782170 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d2406cea0 0x7f8d240c1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:09.730 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.744+0000 7f8d237fe700 1 -- 172.21.15.104:0/3368782170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d240b1350 msgr2=0x7f8d240c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:09.730 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.744+0000 7f8d237fe700 1 --2- 172.21.15.104:0/3368782170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d240b1350 0x7f8d240c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:09.730 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.744+0000 7f8d237fe700 1 -- 172.21.15.104:0/3368782170 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c00bd40 con 0x7f8d24158aa0 2023-07-14T11:57:09.731 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.744+0000 7f8d22ffd700 1 --2- 172.21.15.104:0/3368782170 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d2406cea0 0x7f8d240c1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:57:09.731 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.744+0000 7f8d237fe700 1 --2- 172.21.15.104:0/3368782170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d24158aa0 0x7f8d240b0320 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c014000 tx=0x7f8d1c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:09.732 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.744+0000 7f8d0bfff700 1 -- 172.21.15.104:0/3368782170 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c00bbc0 con 0x7f8d24158aa0 2023-07-14T11:57:09.732 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.744+0000 7f8d0bfff700 1 -- 172.21.15.104:0/3368782170 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c00b4c0 con 0x7f8d24158aa0 2023-07-14T11:57:09.732 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.744+0000 7f8d23fff700 1 -- 172.21.15.104:0/3368782170 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d240c6570 con 0x7f8d24158aa0 2023-07-14T11:57:09.732 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.744+0000 7f8d0bfff700 1 -- 172.21.15.104:0/3368782170 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c004040 con 0x7f8d24158aa0 2023-07-14T11:57:09.732 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.744+0000 7f8d23fff700 1 -- 172.21.15.104:0/3368782170 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d240c6b30 con 0x7f8d24158aa0 2023-07-14T11:57:09.733 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.744+0000 7f8d23fff700 1 -- 172.21.15.104:0/3368782170 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8d2406dc40 con 0x7f8d24158aa0 2023-07-14T11:57:09.733 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.748+0000 7f8d0bfff700 1 -- 172.21.15.104:0/3368782170 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8d1c0041a0 con 0x7f8d24158aa0 2023-07-14T11:57:09.734 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.748+0000 7f8d0bfff700 1 --2- 172.21.15.104:0/3368782170 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8d100662f0 0x7f8d100687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:09.734 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.748+0000 7f8d0bfff700 1 -- 172.21.15.104:0/3368782170 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f8d1c0870c0 con 0x7f8d24158aa0 2023-07-14T11:57:09.734 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.748+0000 7f8d22ffd700 1 --2- 172.21.15.104:0/3368782170 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8d100662f0 0x7f8d100687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:09.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.748+0000 7f8d0bfff700 1 --2- 172.21.15.104:0/3368782170 >> [v2:172.21.15.104:6803/3734215753,v1:172.21.15.104:6807/3734215753] conn(0x7f8d1006cc00 0x7f8d1006f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:09.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.748+0000 7f8d0bfff700 1 -- 172.21.15.104:0/3368782170 --> [v2:172.21.15.104:6803/3734215753,v1:172.21.15.104:6807/3734215753] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d1006f700 con 0x7f8d1006cc00 2023-07-14T11:57:09.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.748+0000 7f8d0bfff700 1 -- 172.21.15.104:0/3368782170 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d1c0896a0 con 0x7f8d24158aa0 2023-07-14T11:57:09.735 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.748+0000 7f8d227fc700 1 --2- 172.21.15.104:0/3368782170 >> [v2:172.21.15.104:6803/3734215753,v1:172.21.15.104:6807/3734215753] conn(0x7f8d1006cc00 0x7f8d1006f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:09.736 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.748+0000 7f8d22ffd700 1 --2- 172.21.15.104:0/3368782170 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8d100662f0 0x7f8d100687a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009c40 tx=0x7f8d14008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:09.736 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.752+0000 7f8d227fc700 1 --2- 172.21.15.104:0/3368782170 >> [v2:172.21.15.104:6803/3734215753,v1:172.21.15.104:6807/3734215753] conn(0x7f8d1006cc00 0x7f8d1006f030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:09.736 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.752+0000 7f8d0bfff700 1 -- 172.21.15.104:0/3368782170 <== osd.0 v2:172.21.15.104:6803/3734215753 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8d1006f700 con 0x7f8d1006cc00 2023-07-14T11:57:09.752 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.768+0000 7f8d23fff700 1 -- 172.21.15.104:0/3368782170 --> [v2:172.21.15.104:6803/3734215753,v1:172.21.15.104:6807/3734215753] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8d2406d960 con 0x7f8d1006cc00 2023-07-14T11:57:09.753 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.768+0000 7f8d0bfff700 1 -- 172.21.15.104:0/3368782170 <== osd.0 v2:172.21.15.104:6803/3734215753 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8d18004890 con 0x7f8d1006cc00 2023-07-14T11:57:09.754 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:57:09.754 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-14T11:57:09.754 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:57:09.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.768+0000 7f8d23fff700 1 -- 172.21.15.104:0/3368782170 >> [v2:172.21.15.104:6803/3734215753,v1:172.21.15.104:6807/3734215753] conn(0x7f8d1006cc00 msgr2=0x7f8d1006f030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:09.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.768+0000 7f8d23fff700 1 --2- 172.21.15.104:0/3368782170 >> [v2:172.21.15.104:6803/3734215753,v1:172.21.15.104:6807/3734215753] conn(0x7f8d1006cc00 0x7f8d1006f030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:09.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.768+0000 7f8d23fff700 1 -- 172.21.15.104:0/3368782170 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8d100662f0 msgr2=0x7f8d100687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:09.755 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.768+0000 7f8d23fff700 1 --2- 172.21.15.104:0/3368782170 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f8d100662f0 0x7f8d100687a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009c40 tx=0x7f8d14008040 comp rx=0 tx=0).stop 2023-07-14T11:57:09.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.768+0000 7f8d23fff700 1 -- 172.21.15.104:0/3368782170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d24158aa0 msgr2=0x7f8d240b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:09.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.768+0000 7f8d23fff700 1 --2- 172.21.15.104:0/3368782170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d24158aa0 0x7f8d240b0320 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c014000 tx=0x7f8d1c004af0 comp rx=0 tx=0).stop 2023-07-14T11:57:09.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.768+0000 7f8d237fe700 1 -- 172.21.15.104:0/3368782170 reap_dead start 2023-07-14T11:57:09.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.768+0000 7f8d23fff700 1 -- 172.21.15.104:0/3368782170 shutdown_connections 2023-07-14T11:57:09.756 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.768+0000 7f8d23fff700 1 -- 172.21.15.104:0/3368782170 >> 172.21.15.104:0/3368782170 conn(0x7f8d240cbc40 msgr2=0x7f8d240c9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:09.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.768+0000 7f8d23fff700 1 -- 172.21.15.104:0/3368782170 shutdown_connections 2023-07-14T11:57:09.757 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.768+0000 7f8d23fff700 1 -- 172.21.15.104:0/3368782170 wait complete. 2023-07-14T11:57:09.775 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-07-14T11:57:09.942 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.956+0000 7fa382ceb700 1 Processor -- start 2023-07-14T11:57:09.943 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.956+0000 7fa382ceb700 1 -- start start 2023-07-14T11:57:09.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa382ceb700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa37c159f20 0x7fa37c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:09.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa382ceb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa37c0caf70 0x7fa37c15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:09.944 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa382ceb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa37c0b0ef0 con 0x7fa37c0c9d90 2023-07-14T11:57:09.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa382ceb700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37c0ce9c0 con 0x7fa37c159f20 2023-07-14T11:57:09.945 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa382ceb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37c0cee50 con 0x7fa37c0caf70 2023-07-14T11:57:09.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa3814e8700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa37c159f20 0x7fa37c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:09.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa3814e8700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa37c159f20 0x7fa37c0caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46588/0 (socket says 172.21.15.104:46588) 2023-07-14T11:57:09.946 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa3814e8700 1 -- 172.21.15.104:0/4057568893 learned_addr learned my addr 172.21.15.104:0/4057568893 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:09.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa3824ea700 1 --2- 172.21.15.104:0/4057568893 >> v2:172.21.15.121:3300/0 conn(0x7fa37c0caf70 0x7fa37c15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:09.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa3814e8700 1 -- 172.21.15.104:0/4057568893 >> v2:172.21.15.121:3300/0 conn(0x7fa37c0caf70 msgr2=0x7fa37c15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:09.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa3814e8700 1 --2- 172.21.15.104:0/4057568893 >> v2:172.21.15.121:3300/0 conn(0x7fa37c0caf70 0x7fa37c15d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:09.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa3814e8700 1 -- 172.21.15.104:0/4057568893 >> v1:172.21.15.121:6789/0 conn(0x7fa37c0c9d90 legacy=0x7fa37c0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:09.947 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa3814e8700 1 -- 172.21.15.104:0/4057568893 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c0cb4f0 con 0x7fa37c159f20 2023-07-14T11:57:09.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa3814e8700 1 --2- 172.21.15.104:0/4057568893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c159f20 0x7fa37c0caa30 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0ca950 tx=0x7fa36c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc8e90b57549dfa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:09.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa380ce7700 1 -- 172.21.15.104:0/4057568893 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c004230 con 0x7fa37c159f20 2023-07-14T11:57:09.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa380ce7700 1 -- 172.21.15.104:0/4057568893 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c004390 con 0x7fa37c159f20 2023-07-14T11:57:09.948 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa380ce7700 1 -- 172.21.15.104:0/4057568893 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c0036c0 con 0x7fa37c159f20 2023-07-14T11:57:09.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa382ceb700 1 -- 172.21.15.104:0/4057568893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c159f20 msgr2=0x7fa37c0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:09.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.960+0000 7fa382ceb700 1 --2- 172.21.15.104:0/4057568893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c159f20 0x7fa37c0caa30 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0ca950 tx=0x7fa36c009bf0 comp rx=0 tx=0).stop 2023-07-14T11:57:09.949 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.964+0000 7fa382ceb700 1 -- 172.21.15.104:0/4057568893 shutdown_connections 2023-07-14T11:57:09.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.964+0000 7fa382ceb700 1 --2- 172.21.15.104:0/4057568893 >> v2:172.21.15.121:3300/0 conn(0x7fa37c0caf70 0x7fa37c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:09.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.964+0000 7fa382ceb700 1 --2- 172.21.15.104:0/4057568893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c159f20 0x7fa37c0caa30 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:09.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.964+0000 7fa382ceb700 1 -- 172.21.15.104:0/4057568893 >> 172.21.15.104:0/4057568893 conn(0x7fa37c06e960 msgr2=0x7fa37c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:09.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.964+0000 7fa382ceb700 1 -- 172.21.15.104:0/4057568893 shutdown_connections 2023-07-14T11:57:09.950 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.964+0000 7fa382ceb700 1 -- 172.21.15.104:0/4057568893 wait complete. 2023-07-14T11:57:09.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.964+0000 7fa382ceb700 1 Processor -- start 2023-07-14T11:57:09.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.964+0000 7fa382ceb700 1 -- start start 2023-07-14T11:57:09.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa382ceb700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa37c0c9d90 0x7fa37c14f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:09.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa382ceb700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c0caf70 0x7fa37c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:09.952 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa382ceb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa37c159f20 0x7fa37c0b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:09.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa382ceb700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c0c1e30 con 0x7fa37c0caf70 2023-07-14T11:57:09.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa382ceb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c14fb30 con 0x7fa37c159f20 2023-07-14T11:57:09.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa382ceb700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c150d40 con 0x7fa37c0c9d90 2023-07-14T11:57:09.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa3814e8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c0caf70 0x7fa37c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:09.953 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa3814e8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c0caf70 0x7fa37c0cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46598/0 (socket says 172.21.15.104:46598) 2023-07-14T11:57:09.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa3814e8700 1 -- 172.21.15.104:0/3509254893 learned_addr learned my addr 172.21.15.104:0/3509254893 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:09.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa381ce9700 1 --2- 172.21.15.104:0/3509254893 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa37c0c9d90 0x7fa37c14f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:09.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa3824ea700 1 --2- 172.21.15.104:0/3509254893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa37c159f20 0x7fa37c0b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:09.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa3814e8700 1 -- 172.21.15.104:0/3509254893 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa37c0c9d90 msgr2=0x7fa37c14f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:09.955 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa3814e8700 1 --2- 172.21.15.104:0/3509254893 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa37c0c9d90 0x7fa37c14f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:09.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa3814e8700 1 -- 172.21.15.104:0/3509254893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa37c159f20 msgr2=0x7fa37c0b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:09.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa3814e8700 1 --2- 172.21.15.104:0/3509254893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa37c159f20 0x7fa37c0b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:09.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa3814e8700 1 -- 172.21.15.104:0/3509254893 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c009570 con 0x7fa37c0caf70 2023-07-14T11:57:09.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa381ce9700 1 --2- 172.21.15.104:0/3509254893 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa37c0c9d90 0x7fa37c14f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:57:09.956 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa3814e8700 1 --2- 172.21.15.104:0/3509254893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c0caf70 0x7fa37c0cd050 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa36c004000 tx=0x7fa36c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:09.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa3727fc700 1 -- 172.21.15.104:0/3509254893 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c003d00 con 0x7fa37c0caf70 2023-07-14T11:57:09.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa3727fc700 1 -- 172.21.15.104:0/3509254893 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c01f070 con 0x7fa37c0caf70 2023-07-14T11:57:09.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa3727fc700 1 -- 172.21.15.104:0/3509254893 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c00b8a0 con 0x7fa37c0caf70 2023-07-14T11:57:09.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa382ceb700 1 -- 172.21.15.104:0/3509254893 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37c151020 con 0x7fa37c0caf70 2023-07-14T11:57:09.957 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.968+0000 7fa382ceb700 1 -- 172.21.15.104:0/3509254893 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37c1515e0 con 0x7fa37c0caf70 2023-07-14T11:57:09.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.972+0000 7fa382ceb700 1 -- 172.21.15.104:0/3509254893 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37c04e500 con 0x7fa37c0caf70 2023-07-14T11:57:09.958 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.972+0000 7fa3727fc700 1 -- 172.21.15.104:0/3509254893 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa36c021030 con 0x7fa37c0caf70 2023-07-14T11:57:09.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.972+0000 7fa3727fc700 1 --2- 172.21.15.104:0/3509254893 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3600662f0 0x7fa3600687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:09.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.972+0000 7fa3727fc700 1 -- 172.21.15.104:0/3509254893 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa36c085040 con 0x7fa37c0caf70 2023-07-14T11:57:09.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.972+0000 7fa381ce9700 1 --2- 172.21.15.104:0/3509254893 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3600662f0 0x7fa3600687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:09.959 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.972+0000 7fa381ce9700 1 --2- 172.21.15.104:0/3509254893 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3600662f0 0x7fa3600687a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa37800b820 tx=0x7fa37800d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:09.964 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:09.980+0000 7fa3727fc700 1 -- 172.21.15.104:0/3509254893 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fa36c052690 con 0x7fa37c0caf70 2023-07-14T11:57:10.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.152+0000 7fa382ceb700 1 -- 172.21.15.104:0/3509254893 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fa37c0ce370 con 0x7fa3600662f0 2023-07-14T11:57:10.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.152+0000 7fa3727fc700 1 -- 172.21.15.104:0/3509254893 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa37800a560 con 0x7fa3600662f0 2023-07-14T11:57:10.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.156+0000 7fa382ceb700 1 -- 172.21.15.104:0/3509254893 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3600662f0 msgr2=0x7fa3600687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.156+0000 7fa382ceb700 1 --2- 172.21.15.104:0/3509254893 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3600662f0 0x7fa3600687a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa37800b820 tx=0x7fa37800d040 comp rx=0 tx=0).stop 2023-07-14T11:57:10.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.156+0000 7fa382ceb700 1 -- 172.21.15.104:0/3509254893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c0caf70 msgr2=0x7fa37c0cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.156+0000 7fa382ceb700 1 --2- 172.21.15.104:0/3509254893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c0caf70 0x7fa37c0cd050 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa36c004000 tx=0x7fa36c004770 comp rx=0 tx=0).stop 2023-07-14T11:57:10.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.156+0000 7fa382ceb700 1 -- 172.21.15.104:0/3509254893 shutdown_connections 2023-07-14T11:57:10.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.156+0000 7fa382ceb700 1 --2- 172.21.15.104:0/3509254893 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa3600662f0 0x7fa3600687a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.156+0000 7fa382ceb700 1 --2- 172.21.15.104:0/3509254893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c0caf70 0x7fa37c0cd050 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.156+0000 7fa382ceb700 1 --2- 172.21.15.104:0/3509254893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa37c159f20 0x7fa37c0b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.156+0000 7fa382ceb700 1 --2- 172.21.15.104:0/3509254893 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa37c0c9d90 0x7fa37c14f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.156+0000 7fa382ceb700 1 -- 172.21.15.104:0/3509254893 >> 172.21.15.104:0/3509254893 conn(0x7fa37c06e960 msgr2=0x7fa37c06d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:10.143 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.156+0000 7fa382ceb700 1 -- 172.21.15.104:0/3509254893 shutdown_connections 2023-07-14T11:57:10.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.156+0000 7fa382ceb700 1 -- 172.21.15.104:0/3509254893 wait complete. 2023-07-14T11:57:10.144 INFO:teuthology.orchestra.run.smithi104.stderr:instructed osd(s) 0 to deep-scrub 2023-07-14T11:57:10.163 INFO:tasks.ceph:Scrubbing osd.1 2023-07-14T11:57:10.163 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-07-14T11:57:10.340 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.356+0000 7f755759e700 1 Processor -- start 2023-07-14T11:57:10.341 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.356+0000 7f755759e700 1 -- start start 2023-07-14T11:57:10.344 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f755759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7558160000 0x7f755806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.345 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f755759e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f75581603d0 0x7f75580c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f755759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75580cd7f0 0x7f75580ca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f755759e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7558070e40 con 0x7f75581603d0 2023-07-14T11:57:10.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f755759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75580cdfb0 con 0x7f7558160000 2023-07-14T11:57:10.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f755759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75580ca8c0 con 0x7f75580cd7f0 2023-07-14T11:57:10.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f7555d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f75581603d0 0x7f75580c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f7555d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f75581603d0 0x7f75580c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44468/0 (socket says 172.21.15.104:44468) 2023-07-14T11:57:10.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f7555d9b700 1 -- 172.21.15.104:0/1915002069 learned_addr learned my addr 172.21.15.104:0/1915002069 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:10.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f755659c700 1 --2- 172.21.15.104:0/1915002069 >> v2:172.21.15.121:3300/0 conn(0x7f7558160000 0x7f755806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f7556d9d700 1 --2- 172.21.15.104:0/1915002069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75580cd7f0 0x7f75580ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f7555d9b700 1 -- 172.21.15.104:0/1915002069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75580cd7f0 msgr2=0x7f75580ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:10.350 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f7555d9b700 1 --2- 172.21.15.104:0/1915002069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75580cd7f0 0x7f75580ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f7555d9b700 1 -- 172.21.15.104:0/1915002069 >> v2:172.21.15.121:3300/0 conn(0x7f7558160000 msgr2=0x7f755806dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:10.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f7555d9b700 1 --2- 172.21.15.104:0/1915002069 >> v2:172.21.15.121:3300/0 conn(0x7f7558160000 0x7f755806dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f7555d9b700 1 -- 172.21.15.104:0/1915002069 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75580cdcd0 con 0x7f75581603d0 2023-07-14T11:57:10.351 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f7555d9b700 1 --2- 172.21.15.104:0/1915002069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75581603d0 0x7f75580c9e40 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f755806b3b0 tx=0x7f754c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac36ca37d17de86 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:10.352 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f755559a700 1 -- 172.21.15.104:0/1915002069 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c011830 con 0x7f75581603d0 2023-07-14T11:57:10.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f755559a700 1 -- 172.21.15.104:0/1915002069 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c011990 con 0x7f75581603d0 2023-07-14T11:57:10.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.360+0000 7f755559a700 1 -- 172.21.15.104:0/1915002069 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c011d20 con 0x7f75581603d0 2023-07-14T11:57:10.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.364+0000 7f755759e700 1 -- 172.21.15.104:0/1915002069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75581603d0 msgr2=0x7f75580c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.353 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.364+0000 7f755759e700 1 --2- 172.21.15.104:0/1915002069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75581603d0 0x7f75580c9e40 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f755806b3b0 tx=0x7f754c009bf0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.364+0000 7f755759e700 1 -- 172.21.15.104:0/1915002069 shutdown_connections 2023-07-14T11:57:10.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.364+0000 7f755759e700 1 --2- 172.21.15.104:0/1915002069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75580cd7f0 0x7f75580ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.354 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.364+0000 7f755759e700 1 --2- 172.21.15.104:0/1915002069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75581603d0 0x7f75580c9e40 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.364+0000 7f755759e700 1 --2- 172.21.15.104:0/1915002069 >> v2:172.21.15.121:3300/0 conn(0x7f7558160000 0x7f755806dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.364+0000 7f755759e700 1 -- 172.21.15.104:0/1915002069 >> 172.21.15.104:0/1915002069 conn(0x7f75580afc60 msgr2=0x7f75580b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:10.355 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.364+0000 7f755759e700 1 -- 172.21.15.104:0/1915002069 shutdown_connections 2023-07-14T11:57:10.356 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.364+0000 7f755759e700 1 -- 172.21.15.104:0/1915002069 wait complete. 2023-07-14T11:57:10.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.364+0000 7f755759e700 1 Processor -- start 2023-07-14T11:57:10.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755759e700 1 -- start start 2023-07-14T11:57:10.359 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755759e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75580cd7f0 0x7f755815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7558160000 0x7f75581621e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75581603d0 0x7f75580b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755759e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7558162a30 con 0x7f75580cd7f0 2023-07-14T11:57:10.360 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7558162d10 con 0x7f75581603d0 2023-07-14T11:57:10.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7558162ff0 con 0x7f7558160000 2023-07-14T11:57:10.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75580cd7f0 0x7f755815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75580cd7f0 0x7f755815a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44472/0 (socket says 172.21.15.104:44472) 2023-07-14T11:57:10.361 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755659c700 1 -- 172.21.15.104:0/36121576 learned_addr learned my addr 172.21.15.104:0/36121576 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:10.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f7555d9b700 1 --2- 172.21.15.104:0/36121576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7558160000 0x7f75581621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f7556d9d700 1 --2- 172.21.15.104:0/36121576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75581603d0 0x7f75580b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.362 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755659c700 1 -- 172.21.15.104:0/36121576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7558160000 msgr2=0x7f75581621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755659c700 1 --2- 172.21.15.104:0/36121576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7558160000 0x7f75581621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.363 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755659c700 1 -- 172.21.15.104:0/36121576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75581603d0 msgr2=0x7f75580b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755659c700 1 --2- 172.21.15.104:0/36121576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75581603d0 0x7f75580b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.364 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755659c700 1 -- 172.21.15.104:0/36121576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c009570 con 0x7f75580cd7f0 2023-07-14T11:57:10.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755659c700 1 --2- 172.21.15.104:0/36121576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75580cd7f0 0x7f755815a9d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f754800e9d0 tx=0x7f754800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:10.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f7546ffd700 1 -- 172.21.15.104:0/36121576 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548009720 con 0x7f75580cd7f0 2023-07-14T11:57:10.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f7546ffd700 1 -- 172.21.15.104:0/36121576 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754800c850 con 0x7f75580cd7f0 2023-07-14T11:57:10.365 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755759e700 1 -- 172.21.15.104:0/36121576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75581632d0 con 0x7f75580cd7f0 2023-07-14T11:57:10.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f7546ffd700 1 -- 172.21.15.104:0/36121576 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754800cbc0 con 0x7f75580cd7f0 2023-07-14T11:57:10.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.368+0000 7f755759e700 1 -- 172.21.15.104:0/36121576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7558162720 con 0x7f75580cd7f0 2023-07-14T11:57:10.366 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.372+0000 7f755759e700 1 -- 172.21.15.104:0/36121576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7558070a60 con 0x7f75580cd7f0 2023-07-14T11:57:10.367 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.372+0000 7f7546ffd700 1 -- 172.21.15.104:0/36121576 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f754800cd20 con 0x7f75580cd7f0 2023-07-14T11:57:10.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.376+0000 7f7546ffd700 1 --2- 172.21.15.104:0/36121576 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f75380662f0 0x7f75380687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.376+0000 7f7546ffd700 1 -- 172.21.15.104:0/36121576 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f7548083c70 con 0x7f75580cd7f0 2023-07-14T11:57:10.370 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.376+0000 7f7555d9b700 1 --2- 172.21.15.104:0/36121576 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f75380662f0 0x7f75380687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.376+0000 7f7546ffd700 1 --2- 172.21.15.104:0/36121576 >> [v2:172.21.15.104:6824/2317846209,v1:172.21.15.104:6825/2317846209] conn(0x7f753806cc00 0x7f753806f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.371 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.376+0000 7f7546ffd700 1 -- 172.21.15.104:0/36121576 --> [v2:172.21.15.104:6824/2317846209,v1:172.21.15.104:6825/2317846209] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f753806f700 con 0x7f753806cc00 2023-07-14T11:57:10.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.376+0000 7f7546ffd700 1 -- 172.21.15.104:0/36121576 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75480882a0 con 0x7f75580cd7f0 2023-07-14T11:57:10.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.376+0000 7f7556d9d700 1 --2- 172.21.15.104:0/36121576 >> [v2:172.21.15.104:6824/2317846209,v1:172.21.15.104:6825/2317846209] conn(0x7f753806cc00 0x7f753806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.376+0000 7f7555d9b700 1 --2- 172.21.15.104:0/36121576 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f75380662f0 0x7f75380687a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f754c000c00 tx=0x7f754c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:10.372 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.376+0000 7f7556d9d700 1 --2- 172.21.15.104:0/36121576 >> [v2:172.21.15.104:6824/2317846209,v1:172.21.15.104:6825/2317846209] conn(0x7f753806cc00 0x7f753806f030 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:10.373 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.376+0000 7f7546ffd700 1 -- 172.21.15.104:0/36121576 <== osd.1 v2:172.21.15.104:6824/2317846209 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f753806f700 con 0x7f753806cc00 2023-07-14T11:57:10.380 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.396+0000 7f755759e700 1 -- 172.21.15.104:0/36121576 --> [v2:172.21.15.104:6824/2317846209,v1:172.21.15.104:6825/2317846209] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7558070780 con 0x7f753806cc00 2023-07-14T11:57:10.381 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.396+0000 7f7546ffd700 1 -- 172.21.15.104:0/36121576 <== osd.1 v2:172.21.15.104:6824/2317846209 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7550009a50 con 0x7f753806cc00 2023-07-14T11:57:10.382 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:57:10.382 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-14T11:57:10.382 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:57:10.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.396+0000 7f755759e700 1 -- 172.21.15.104:0/36121576 >> [v2:172.21.15.104:6824/2317846209,v1:172.21.15.104:6825/2317846209] conn(0x7f753806cc00 msgr2=0x7f753806f030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.384 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.396+0000 7f755759e700 1 --2- 172.21.15.104:0/36121576 >> [v2:172.21.15.104:6824/2317846209,v1:172.21.15.104:6825/2317846209] conn(0x7f753806cc00 0x7f753806f030 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.396+0000 7f755759e700 1 -- 172.21.15.104:0/36121576 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f75380662f0 msgr2=0x7f75380687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.396+0000 7f755759e700 1 --2- 172.21.15.104:0/36121576 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f75380662f0 0x7f75380687a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f754c000c00 tx=0x7f754c008030 comp rx=0 tx=0).stop 2023-07-14T11:57:10.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.396+0000 7f755759e700 1 -- 172.21.15.104:0/36121576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75580cd7f0 msgr2=0x7f755815a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.385 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.396+0000 7f755759e700 1 --2- 172.21.15.104:0/36121576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75580cd7f0 0x7f755815a9d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f754800e9d0 tx=0x7f754800ed90 comp rx=0 tx=0).stop 2023-07-14T11:57:10.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.396+0000 7f7556d9d700 1 -- 172.21.15.104:0/36121576 reap_dead start 2023-07-14T11:57:10.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.396+0000 7f755759e700 1 -- 172.21.15.104:0/36121576 shutdown_connections 2023-07-14T11:57:10.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.396+0000 7f755759e700 1 -- 172.21.15.104:0/36121576 >> 172.21.15.104:0/36121576 conn(0x7f75580afc60 msgr2=0x7f755806c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:10.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.396+0000 7f755759e700 1 -- 172.21.15.104:0/36121576 shutdown_connections 2023-07-14T11:57:10.386 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.396+0000 7f755759e700 1 -- 172.21.15.104:0/36121576 wait complete. 2023-07-14T11:57:10.403 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-07-14T11:57:10.577 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.592+0000 7fa70659e700 1 Processor -- start 2023-07-14T11:57:10.578 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.592+0000 7fa70659e700 1 -- start start 2023-07-14T11:57:10.580 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.596+0000 7fa70659e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7001599d0 0x7fa700071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.596+0000 7fa70659e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa7000caa20 0x7fa7000cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.596+0000 7fa70659e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7000cde10 con 0x7fa70015ccf0 2023-07-14T11:57:10.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.596+0000 7fa70659e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7000cb3c0 con 0x7fa7000caa20 2023-07-14T11:57:10.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.596+0000 7fa70659e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7000cd520 con 0x7fa7001599d0 2023-07-14T11:57:10.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.596+0000 7fa704d9b700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fa70015ccf0 0x7fa7000c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43538/0 (socket says 172.21.15.104:43538) 2023-07-14T11:57:10.585 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.596+0000 7fa704d9b700 1 -- 172.21.15.104:0/2628742979 learned_addr learned my addr 172.21.15.104:0/2628742979 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:10.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.596+0000 7fa705d9d700 1 --2- 172.21.15.104:0/2628742979 >> v2:172.21.15.104:3300/0 conn(0x7fa7000caa20 0x7fa7000cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.596+0000 7fa70559c700 1 --2- 172.21.15.104:0/2628742979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7001599d0 0x7fa700071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.586 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.596+0000 7fa6fffff700 1 -- 172.21.15.104:0/2628742979 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995718645 0 0) 0x7fa7000cde10 con 0x7fa70015ccf0 2023-07-14T11:57:10.587 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.596+0000 7fa6fffff700 1 -- 172.21.15.104:0/2628742979 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e80035d0 con 0x7fa70015ccf0 2023-07-14T11:57:10.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.596+0000 7fa705d9d700 1 -- 172.21.15.104:0/2628742979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7001599d0 msgr2=0x7fa700071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:10.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.596+0000 7fa705d9d700 1 --2- 172.21.15.104:0/2628742979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7001599d0 0x7fa700071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.596+0000 7fa705d9d700 1 -- 172.21.15.104:0/2628742979 >> v1:172.21.15.104:6789/0 conn(0x7fa70015ccf0 legacy=0x7fa7000c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.596+0000 7fa705d9d700 1 -- 172.21.15.104:0/2628742979 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7000715e0 con 0x7fa7000caa20 2023-07-14T11:57:10.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.600+0000 7fa705d9d700 1 --2- 172.21.15.104:0/2628742979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7000caa20 0x7fa7000cae80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009850 tx=0x7fa6f8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c430e451bd71a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:10.592 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.600+0000 7fa6fffff700 1 -- 172.21.15.104:0/2628742979 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8011830 con 0x7fa7000caa20 2023-07-14T11:57:10.592 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.600+0000 7fa6fffff700 1 -- 172.21.15.104:0/2628742979 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f8011990 con 0x7fa7000caa20 2023-07-14T11:57:10.592 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.600+0000 7fa6fffff700 1 -- 172.21.15.104:0/2628742979 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8011d00 con 0x7fa7000caa20 2023-07-14T11:57:10.592 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.600+0000 7fa70659e700 1 -- 172.21.15.104:0/2628742979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7000caa20 msgr2=0x7fa7000cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.600+0000 7fa70659e700 1 --2- 172.21.15.104:0/2628742979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7000caa20 0x7fa7000cae80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009850 tx=0x7fa6f8009b60 comp rx=0 tx=0).stop 2023-07-14T11:57:10.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.600+0000 7fa70659e700 1 -- 172.21.15.104:0/2628742979 shutdown_connections 2023-07-14T11:57:10.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.600+0000 7fa70659e700 1 --2- 172.21.15.104:0/2628742979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7000caa20 0x7fa7000cae80 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.600+0000 7fa70659e700 1 --2- 172.21.15.104:0/2628742979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7001599d0 0x7fa700071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.593 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.600+0000 7fa70659e700 1 -- 172.21.15.104:0/2628742979 >> 172.21.15.104:0/2628742979 conn(0x7fa70006e960 msgr2=0x7fa70006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:10.594 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.600+0000 7fa70659e700 1 -- 172.21.15.104:0/2628742979 shutdown_connections 2023-07-14T11:57:10.594 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.600+0000 7fa70659e700 1 -- 172.21.15.104:0/2628742979 wait complete. 2023-07-14T11:57:10.594 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70659e700 1 Processor -- start 2023-07-14T11:57:10.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70659e700 1 -- start start 2023-07-14T11:57:10.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70659e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7000caa20 0x7fa70015de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70659e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7001599d0 0x7fa7000acf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70659e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa70015ccf0 0x7fa70015f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.595 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70659e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7000b2aa0 con 0x7fa7000caa20 2023-07-14T11:57:10.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70659e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70015e380 con 0x7fa7001599d0 2023-07-14T11:57:10.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70659e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7000b1e90 con 0x7fa70015ccf0 2023-07-14T11:57:10.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70559c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7000caa20 0x7fa70015de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.596 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70559c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7000caa20 0x7fa70015de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44486/0 (socket says 172.21.15.104:44486) 2023-07-14T11:57:10.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70559c700 1 -- 172.21.15.104:0/2847540158 learned_addr learned my addr 172.21.15.104:0/2847540158 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:10.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa705d9d700 1 --2- 172.21.15.104:0/2847540158 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa70015ccf0 0x7fa70015f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.597 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa704d9b700 1 --2- 172.21.15.104:0/2847540158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7001599d0 0x7fa7000acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70559c700 1 -- 172.21.15.104:0/2847540158 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa70015ccf0 msgr2=0x7fa70015f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70559c700 1 --2- 172.21.15.104:0/2847540158 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa70015ccf0 0x7fa70015f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70559c700 1 -- 172.21.15.104:0/2847540158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7001599d0 msgr2=0x7fa7000acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.598 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70559c700 1 --2- 172.21.15.104:0/2847540158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7001599d0 0x7fa7000acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70559c700 1 -- 172.21.15.104:0/2847540158 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f80094b0 con 0x7fa7000caa20 2023-07-14T11:57:10.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa704d9b700 1 --2- 172.21.15.104:0/2847540158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7001599d0 0x7fa7000acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:57:10.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70559c700 1 --2- 172.21.15.104:0/2847540158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7000caa20 0x7fa70015de40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000efc0 tx=0x7fa6f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:10.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa6fdffb700 1 -- 172.21.15.104:0/2847540158 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0009b80 con 0x7fa7000caa20 2023-07-14T11:57:10.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa6fdffb700 1 -- 172.21.15.104:0/2847540158 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f000f040 con 0x7fa7000caa20 2023-07-14T11:57:10.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa70659e700 1 -- 172.21.15.104:0/2847540158 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7000b2170 con 0x7fa7000caa20 2023-07-14T11:57:10.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.604+0000 7fa6fdffb700 1 -- 172.21.15.104:0/2847540158 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0014480 con 0x7fa7000caa20 2023-07-14T11:57:10.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.608+0000 7fa70659e700 1 -- 172.21.15.104:0/2847540158 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7000b2730 con 0x7fa7000caa20 2023-07-14T11:57:10.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.608+0000 7fa70659e700 1 -- 172.21.15.104:0/2847540158 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70004e500 con 0x7fa7000caa20 2023-07-14T11:57:10.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.608+0000 7fa6fdffb700 1 -- 172.21.15.104:0/2847540158 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa6f00145e0 con 0x7fa7000caa20 2023-07-14T11:57:10.603 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.612+0000 7fa6fdffb700 1 --2- 172.21.15.104:0/2847540158 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa6ec0662f0 0x7fa6ec0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.612+0000 7fa6fdffb700 1 -- 172.21.15.104:0/2847540158 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa6f00847d0 con 0x7fa7000caa20 2023-07-14T11:57:10.604 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.612+0000 7fa704d9b700 1 --2- 172.21.15.104:0/2847540158 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa6ec0662f0 0x7fa6ec0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.612+0000 7fa704d9b700 1 --2- 172.21.15.104:0/2847540158 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa6ec0662f0 0x7fa6ec0687a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa7001f4b20 tx=0x7fa6f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:10.605 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.616+0000 7fa6fdffb700 1 -- 172.21.15.104:0/2847540158 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fa6f0055820 con 0x7fa7000caa20 2023-07-14T11:57:10.771 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.788+0000 7fa70659e700 1 -- 172.21.15.104:0/2847540158 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7000cdaa0 con 0x7fa6ec0662f0 2023-07-14T11:57:10.772 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.788+0000 7fa6fdffb700 1 -- 172.21.15.104:0/2847540158 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa6f4001cb0 con 0x7fa6ec0662f0 2023-07-14T11:57:10.775 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.788+0000 7fa70659e700 1 -- 172.21.15.104:0/2847540158 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa6ec0662f0 msgr2=0x7fa6ec0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.788+0000 7fa70659e700 1 --2- 172.21.15.104:0/2847540158 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa6ec0662f0 0x7fa6ec0687a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa7001f4b20 tx=0x7fa6f4009040 comp rx=0 tx=0).stop 2023-07-14T11:57:10.776 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.788+0000 7fa70659e700 1 -- 172.21.15.104:0/2847540158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7000caa20 msgr2=0x7fa70015de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.788+0000 7fa70659e700 1 --2- 172.21.15.104:0/2847540158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7000caa20 0x7fa70015de40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000efc0 tx=0x7fa6f000c3a0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.777 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.788+0000 7fa70659e700 1 -- 172.21.15.104:0/2847540158 shutdown_connections 2023-07-14T11:57:10.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.788+0000 7fa70659e700 1 --2- 172.21.15.104:0/2847540158 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa6ec0662f0 0x7fa6ec0687a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.788+0000 7fa70659e700 1 --2- 172.21.15.104:0/2847540158 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa70015ccf0 0x7fa70015f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.788+0000 7fa70659e700 1 --2- 172.21.15.104:0/2847540158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7001599d0 0x7fa7000acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.778 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.788+0000 7fa70659e700 1 --2- 172.21.15.104:0/2847540158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7000caa20 0x7fa70015de40 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.788+0000 7fa70659e700 1 -- 172.21.15.104:0/2847540158 >> 172.21.15.104:0/2847540158 conn(0x7fa70006e960 msgr2=0x7fa7000cc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:10.779 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.792+0000 7fa70659e700 1 -- 172.21.15.104:0/2847540158 shutdown_connections 2023-07-14T11:57:10.780 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.792+0000 7fa70659e700 1 -- 172.21.15.104:0/2847540158 wait complete. 2023-07-14T11:57:10.780 INFO:teuthology.orchestra.run.smithi104.stderr:instructed osd(s) 1 to deep-scrub 2023-07-14T11:57:10.794 INFO:tasks.ceph:Scrubbing osd.2 2023-07-14T11:57:10.795 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-07-14T11:57:10.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.984+0000 7fa9f7caa700 1 Processor -- start 2023-07-14T11:57:10.969 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.984+0000 7fa9f7caa700 1 -- start start 2023-07-14T11:57:10.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.988+0000 7fa9f7caa700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa9f00ce300 0x7fa9f006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.988+0000 7fa9f7caa700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9f00b40f0 0x7fa9f0070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.988+0000 7fa9f7caa700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f0071310 con 0x7fa9f00d02d0 2023-07-14T11:57:10.973 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.988+0000 7fa9f7caa700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f0071bb0 con 0x7fa9f00ce300 2023-07-14T11:57:10.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.988+0000 7fa9f7caa700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00715f0 con 0x7fa9f00b40f0 2023-07-14T11:57:10.974 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.988+0000 7fa9f5245700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa9f00ce300 0x7fa9f006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.988+0000 7fa9f5245700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa9f00ce300 0x7fa9f006fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44498/0 (socket says 172.21.15.104:44498) 2023-07-14T11:57:10.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.988+0000 7fa9f5245700 1 -- 172.21.15.104:0/2914743315 learned_addr learned my addr 172.21.15.104:0/2914743315 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:10.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.988+0000 7fa9f6247700 1 --2- 172.21.15.104:0/2914743315 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9f00b40f0 0x7fa9f0070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.975 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.988+0000 7fa9f5245700 1 -- 172.21.15.104:0/2914743315 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9f00b40f0 msgr2=0x7fa9f0070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:10.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.988+0000 7fa9f5245700 1 --2- 172.21.15.104:0/2914743315 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9f00b40f0 0x7fa9f0070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.988+0000 7fa9f5245700 1 -- 172.21.15.104:0/2914743315 >> v1:172.21.15.121:6789/0 conn(0x7fa9f00d02d0 legacy=0x7fa9f00d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.988+0000 7fa9f5245700 1 -- 172.21.15.104:0/2914743315 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f00718d0 con 0x7fa9f00ce300 2023-07-14T11:57:10.976 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.988+0000 7fa9f5245700 1 --2- 172.21.15.104:0/2914743315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00ce300 0x7fa9f006fd20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009910 tx=0x7fa9e0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142a7c4811e11b26 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:10.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.992+0000 7fa9f4a44700 1 -- 172.21.15.104:0/2914743315 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0011830 con 0x7fa9f00ce300 2023-07-14T11:57:10.977 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.992+0000 7fa9f4a44700 1 -- 172.21.15.104:0/2914743315 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e0011990 con 0x7fa9f00ce300 2023-07-14T11:57:10.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.992+0000 7fa9f4a44700 1 -- 172.21.15.104:0/2914743315 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0011d40 con 0x7fa9f00ce300 2023-07-14T11:57:10.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.992+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2914743315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00ce300 msgr2=0x7fa9f006fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.992+0000 7fa9f7caa700 1 --2- 172.21.15.104:0/2914743315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00ce300 0x7fa9f006fd20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009910 tx=0x7fa9e0009c20 comp rx=0 tx=0).stop 2023-07-14T11:57:10.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.992+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2914743315 shutdown_connections 2023-07-14T11:57:10.978 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.992+0000 7fa9f7caa700 1 --2- 172.21.15.104:0/2914743315 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9f00b40f0 0x7fa9f0070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.979 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.992+0000 7fa9f7caa700 1 --2- 172.21.15.104:0/2914743315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00ce300 0x7fa9f006fd20 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.979 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.992+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2914743315 >> 172.21.15.104:0/2914743315 conn(0x7fa9f009b9e0 msgr2=0x7fa9f00d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:10.979 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.992+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2914743315 shutdown_connections 2023-07-14T11:57:10.979 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.992+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2914743315 wait complete. 2023-07-14T11:57:10.980 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.992+0000 7fa9f7caa700 1 Processor -- start 2023-07-14T11:57:10.980 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.992+0000 7fa9f7caa700 1 -- start start 2023-07-14T11:57:10.980 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f7caa700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00b40f0 0x7fa9f009ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f7caa700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9f00ce300 0x7fa9f015cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f7caa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9f00d02d0 0x7fa9f00ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f7caa700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f01f1ec0 con 0x7fa9f00b40f0 2023-07-14T11:57:10.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f7caa700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f01f1900 con 0x7fa9f00d02d0 2023-07-14T11:57:10.981 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f7caa700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f01f1be0 con 0x7fa9f00ce300 2023-07-14T11:57:10.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f5a46700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00b40f0 0x7fa9f009ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f5a46700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00b40f0 0x7fa9f009ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44500/0 (socket says 172.21.15.104:44500) 2023-07-14T11:57:10.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f5a46700 1 -- 172.21.15.104:0/2389638829 learned_addr learned my addr 172.21.15.104:0/2389638829 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:10.984 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f5245700 1 --2- 172.21.15.104:0/2389638829 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9f00ce300 0x7fa9f015cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.984 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f6247700 1 --2- 172.21.15.104:0/2389638829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9f00d02d0 0x7fa9f00ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.984 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f5a46700 1 -- 172.21.15.104:0/2389638829 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9f00ce300 msgr2=0x7fa9f015cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.984 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f5a46700 1 --2- 172.21.15.104:0/2389638829 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9f00ce300 0x7fa9f015cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.984 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f5a46700 1 -- 172.21.15.104:0/2389638829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9f00d02d0 msgr2=0x7fa9f00ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:10.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f5a46700 1 --2- 172.21.15.104:0/2389638829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9f00d02d0 0x7fa9f00ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:10.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f5a46700 1 -- 172.21.15.104:0/2389638829 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e0009570 con 0x7fa9f00b40f0 2023-07-14T11:57:10.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f6247700 1 --2- 172.21.15.104:0/2389638829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9f00d02d0 0x7fa9f00ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:57:10.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f5a46700 1 --2- 172.21.15.104:0/2389638829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00b40f0 0x7fa9f009ffd0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00c500 tx=0x7fa9ec00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:10.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9e67fc700 1 -- 172.21.15.104:0/2389638829 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0052d0 con 0x7fa9f00b40f0 2023-07-14T11:57:10.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9e67fc700 1 -- 172.21.15.104:0/2389638829 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec005430 con 0x7fa9f00b40f0 2023-07-14T11:57:10.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9e67fc700 1 -- 172.21.15.104:0/2389638829 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0055a0 con 0x7fa9f00b40f0 2023-07-14T11:57:10.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2389638829 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f00c9830 con 0x7fa9f00b40f0 2023-07-14T11:57:10.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:10.996+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2389638829 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f01f2040 con 0x7fa9f00b40f0 2023-07-14T11:57:10.987 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.000+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2389638829 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa9f00c96b0 con 0x7fa9f00b40f0 2023-07-14T11:57:10.987 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.000+0000 7fa9e67fc700 1 -- 172.21.15.104:0/2389638829 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa9ec0038e0 con 0x7fa9f00b40f0 2023-07-14T11:57:10.988 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.000+0000 7fa9e67fc700 1 --2- 172.21.15.104:0/2389638829 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa9d40662f0 0x7fa9d40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.988 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.000+0000 7fa9e67fc700 1 -- 172.21.15.104:0/2389638829 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa9ec082ef0 con 0x7fa9f00b40f0 2023-07-14T11:57:10.988 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.000+0000 7fa9e67fc700 1 --2- 172.21.15.104:0/2389638829 >> [v2:172.21.15.104:6800/3827055184,v1:172.21.15.104:6801/3827055184] conn(0x7fa9d406cc00 0x7fa9d406f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:10.988 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.000+0000 7fa9e67fc700 1 -- 172.21.15.104:0/2389638829 --> [v2:172.21.15.104:6800/3827055184,v1:172.21.15.104:6801/3827055184] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9d406f700 con 0x7fa9d406cc00 2023-07-14T11:57:10.988 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.000+0000 7fa9e67fc700 1 -- 172.21.15.104:0/2389638829 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9ec085590 con 0x7fa9f00b40f0 2023-07-14T11:57:10.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.004+0000 7fa9f5245700 1 --2- 172.21.15.104:0/2389638829 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa9d40662f0 0x7fa9d40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.004+0000 7fa9f6247700 1 --2- 172.21.15.104:0/2389638829 >> [v2:172.21.15.104:6800/3827055184,v1:172.21.15.104:6801/3827055184] conn(0x7fa9d406cc00 0x7fa9d406f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:10.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.004+0000 7fa9f5245700 1 --2- 172.21.15.104:0/2389638829 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa9d40662f0 0x7fa9d40687a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009860 tx=0x7fa9e0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:10.990 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.004+0000 7fa9f6247700 1 --2- 172.21.15.104:0/2389638829 >> [v2:172.21.15.104:6800/3827055184,v1:172.21.15.104:6801/3827055184] conn(0x7fa9d406cc00 0x7fa9d406f030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:10.990 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.004+0000 7fa9e67fc700 1 -- 172.21.15.104:0/2389638829 <== osd.2 v2:172.21.15.104:6800/3827055184 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa9d406f700 con 0x7fa9d406cc00 2023-07-14T11:57:11.007 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.020+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2389638829 --> [v2:172.21.15.104:6800/3827055184,v1:172.21.15.104:6801/3827055184] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa9f008bdb0 con 0x7fa9d406cc00 2023-07-14T11:57:11.008 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.020+0000 7fa9e67fc700 1 -- 172.21.15.104:0/2389638829 <== osd.2 v2:172.21.15.104:6800/3827055184 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa9e800f040 con 0x7fa9d406cc00 2023-07-14T11:57:11.008 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:57:11.009 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-14T11:57:11.009 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:57:11.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.024+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2389638829 >> [v2:172.21.15.104:6800/3827055184,v1:172.21.15.104:6801/3827055184] conn(0x7fa9d406cc00 msgr2=0x7fa9d406f030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.024+0000 7fa9f7caa700 1 --2- 172.21.15.104:0/2389638829 >> [v2:172.21.15.104:6800/3827055184,v1:172.21.15.104:6801/3827055184] conn(0x7fa9d406cc00 0x7fa9d406f030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.010 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.024+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2389638829 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa9d40662f0 msgr2=0x7fa9d40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.024+0000 7fa9f7caa700 1 --2- 172.21.15.104:0/2389638829 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fa9d40662f0 0x7fa9d40687a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009860 tx=0x7fa9e0008030 comp rx=0 tx=0).stop 2023-07-14T11:57:11.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.024+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2389638829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00b40f0 msgr2=0x7fa9f009ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.024+0000 7fa9f7caa700 1 --2- 172.21.15.104:0/2389638829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9f00b40f0 0x7fa9f009ffd0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00c500 tx=0x7fa9ec00c810 comp rx=0 tx=0).stop 2023-07-14T11:57:11.011 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.024+0000 7fa9f6247700 1 -- 172.21.15.104:0/2389638829 reap_dead start 2023-07-14T11:57:11.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.024+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2389638829 shutdown_connections 2023-07-14T11:57:11.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.024+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2389638829 >> 172.21.15.104:0/2389638829 conn(0x7fa9f009b9e0 msgr2=0x7fa9f00cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:11.012 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.024+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2389638829 shutdown_connections 2023-07-14T11:57:11.013 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.024+0000 7fa9f7caa700 1 -- 172.21.15.104:0/2389638829 wait complete. 2023-07-14T11:57:11.026 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-07-14T11:57:11.204 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.220+0000 7f60b899e700 1 Processor -- start 2023-07-14T11:57:11.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.220+0000 7f60b899e700 1 -- start start 2023-07-14T11:57:11.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.220+0000 7f60b899e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f60b40afd30 0x7f60b40b0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.220+0000 7f60b899e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60b40c8900 con 0x7f60b4160640 2023-07-14T11:57:11.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.220+0000 7f60b899e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60b40b06b0 con 0x7f60b40cddb0 2023-07-14T11:57:11.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.220+0000 7f60b899e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b40c8fc0 con 0x7f60b40afd30 2023-07-14T11:57:11.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.220+0000 7f60b2d9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f60b40afd30 0x7f60b40b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.220+0000 7f60b259c700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f60b4160640 0x7f60b4164000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43546/0 (socket says 172.21.15.104:43546) 2023-07-14T11:57:11.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.220+0000 7f60b259c700 1 -- 172.21.15.104:0/1742439468 learned_addr learned my addr 172.21.15.104:0/1742439468 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:11.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.220+0000 7f60b2d9d700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f60b40afd30 0x7f60b40b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44510/0 (socket says 172.21.15.104:44510) 2023-07-14T11:57:11.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.220+0000 7f60b2d9d700 1 -- 172.21.15.104:0/1742439468 >> v1:172.21.15.121:6789/0 conn(0x7f60b40cddb0 legacy=0x7f60b4070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.209 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.220+0000 7f60b2d9d700 1 -- 172.21.15.104:0/1742439468 >> v1:172.21.15.104:6789/0 conn(0x7f60b4160640 legacy=0x7f60b4164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.220+0000 7f60b2d9d700 1 -- 172.21.15.104:0/1742439468 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b40caae0 con 0x7f60b40afd30 2023-07-14T11:57:11.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.220+0000 7f60b2d9d700 1 --2- 172.21.15.104:0/1742439468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60b40afd30 0x7f60b40b0170 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009850 tx=0x7f60ac009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399deb12f6f16ae4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:11.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.224+0000 7f60b1d9b700 1 -- 172.21.15.104:0/1742439468 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac011830 con 0x7f60b40afd30 2023-07-14T11:57:11.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.224+0000 7f60b1d9b700 1 -- 172.21.15.104:0/1742439468 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ac011990 con 0x7f60b40afd30 2023-07-14T11:57:11.210 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.224+0000 7f60b1d9b700 1 -- 172.21.15.104:0/1742439468 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac011d20 con 0x7f60b40afd30 2023-07-14T11:57:11.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.224+0000 7f60b899e700 1 -- 172.21.15.104:0/1742439468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60b40afd30 msgr2=0x7f60b40b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.224+0000 7f60b899e700 1 --2- 172.21.15.104:0/1742439468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60b40afd30 0x7f60b40b0170 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009850 tx=0x7f60ac009b60 comp rx=0 tx=0).stop 2023-07-14T11:57:11.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.224+0000 7f60b899e700 1 -- 172.21.15.104:0/1742439468 shutdown_connections 2023-07-14T11:57:11.211 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.224+0000 7f60b899e700 1 --2- 172.21.15.104:0/1742439468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60b40afd30 0x7f60b40b0170 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.224+0000 7f60b899e700 1 -- 172.21.15.104:0/1742439468 >> 172.21.15.104:0/1742439468 conn(0x7f60b40cad30 msgr2=0x7f60b40cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:11.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.224+0000 7f60b899e700 1 -- 172.21.15.104:0/1742439468 shutdown_connections 2023-07-14T11:57:11.212 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.224+0000 7f60b899e700 1 -- 172.21.15.104:0/1742439468 wait complete. 2023-07-14T11:57:11.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b899e700 1 Processor -- start 2023-07-14T11:57:11.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b899e700 1 -- start start 2023-07-14T11:57:11.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b899e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60b40afd30 0x7f60b40b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b899e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60b40cddb0 0x7f60b40b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b899e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60b4160640 0x7f60b40b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b899e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b4070540 con 0x7f60b4160640 2023-07-14T11:57:11.215 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b899e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b40c63d0 con 0x7f60b40cddb0 2023-07-14T11:57:11.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b899e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b40b3f10 con 0x7f60b40afd30 2023-07-14T11:57:11.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b2d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60b4160640 0x7f60b40b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b2d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60b4160640 0x7f60b40b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44526/0 (socket says 172.21.15.104:44526) 2023-07-14T11:57:11.216 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b2d9d700 1 -- 172.21.15.104:0/241101534 learned_addr learned my addr 172.21.15.104:0/241101534 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:11.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b259c700 1 --2- 172.21.15.104:0/241101534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60b40afd30 0x7f60b40b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60a9d9b700 1 --2- 172.21.15.104:0/241101534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60b40cddb0 0x7f60b40b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b2d9d700 1 -- 172.21.15.104:0/241101534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60b40afd30 msgr2=0x7f60b40b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b2d9d700 1 --2- 172.21.15.104:0/241101534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60b40afd30 0x7f60b40b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b2d9d700 1 -- 172.21.15.104:0/241101534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60b40cddb0 msgr2=0x7f60b40b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b2d9d700 1 --2- 172.21.15.104:0/241101534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60b40cddb0 0x7f60b40b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b2d9d700 1 -- 172.21.15.104:0/241101534 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac0094b0 con 0x7f60b4160640 2023-07-14T11:57:11.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b259c700 1 --2- 172.21.15.104:0/241101534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60b40afd30 0x7f60b40b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:57:11.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.228+0000 7f60b2d9d700 1 --2- 172.21.15.104:0/241101534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60b4160640 0x7f60b40b2030 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009fd0 tx=0x7f60ac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:11.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.232+0000 7f60ab7fe700 1 -- 172.21.15.104:0/241101534 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac004560 con 0x7f60b4160640 2023-07-14T11:57:11.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.232+0000 7f60ab7fe700 1 -- 172.21.15.104:0/241101534 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ac01f070 con 0x7f60b4160640 2023-07-14T11:57:11.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.232+0000 7f60b899e700 1 -- 172.21.15.104:0/241101534 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b40c6670 con 0x7f60b4160640 2023-07-14T11:57:11.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.232+0000 7f60ab7fe700 1 -- 172.21.15.104:0/241101534 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac003900 con 0x7f60b4160640 2023-07-14T11:57:11.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.232+0000 7f60b899e700 1 -- 172.21.15.104:0/241101534 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b40c6c30 con 0x7f60b4160640 2023-07-14T11:57:11.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.232+0000 7f60b899e700 1 -- 172.21.15.104:0/241101534 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b40ca7c0 con 0x7f60b4160640 2023-07-14T11:57:11.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.232+0000 7f60ab7fe700 1 -- 172.21.15.104:0/241101534 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f60ac021030 con 0x7f60b4160640 2023-07-14T11:57:11.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.236+0000 7f60ab7fe700 1 --2- 172.21.15.104:0/241101534 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f60940662f0 0x7f60940687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.236+0000 7f60ab7fe700 1 -- 172.21.15.104:0/241101534 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f60ac0851e0 con 0x7f60b4160640 2023-07-14T11:57:11.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.236+0000 7f60b259c700 1 --2- 172.21.15.104:0/241101534 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f60940662f0 0x7f60940687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.236+0000 7f60b259c700 1 --2- 172.21.15.104:0/241101534 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f60940662f0 0x7f60940687a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f60a4005cb0 tx=0x7f60a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:11.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.240+0000 7f60ab7fe700 1 -- 172.21.15.104:0/241101534 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f60ac052860 con 0x7f60b4160640 2023-07-14T11:57:11.401 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.416+0000 7f60b899e700 1 -- 172.21.15.104:0/241101534 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f60b40ca520 con 0x7f60940662f0 2023-07-14T11:57:11.402 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.416+0000 7f60ab7fe700 1 -- 172.21.15.104:0/241101534 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f60a400d700 con 0x7f60940662f0 2023-07-14T11:57:11.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.420+0000 7f60b899e700 1 -- 172.21.15.104:0/241101534 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f60940662f0 msgr2=0x7f60940687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.420+0000 7f60b899e700 1 --2- 172.21.15.104:0/241101534 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f60940662f0 0x7f60940687a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f60a4005cb0 tx=0x7f60a4009040 comp rx=0 tx=0).stop 2023-07-14T11:57:11.405 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.420+0000 7f60b899e700 1 -- 172.21.15.104:0/241101534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60b4160640 msgr2=0x7f60b40b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.420+0000 7f60b899e700 1 --2- 172.21.15.104:0/241101534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60b4160640 0x7f60b40b2030 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009fd0 tx=0x7f60ac004770 comp rx=0 tx=0).stop 2023-07-14T11:57:11.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.420+0000 7f60b899e700 1 -- 172.21.15.104:0/241101534 shutdown_connections 2023-07-14T11:57:11.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.420+0000 7f60b899e700 1 --2- 172.21.15.104:0/241101534 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f60940662f0 0x7f60940687a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.406 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.420+0000 7f60b899e700 1 --2- 172.21.15.104:0/241101534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60b4160640 0x7f60b40b2030 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.420+0000 7f60b899e700 1 --2- 172.21.15.104:0/241101534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60b40cddb0 0x7f60b40b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.407 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.420+0000 7f60b899e700 1 --2- 172.21.15.104:0/241101534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60b40afd30 0x7f60b40b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.420+0000 7f60b899e700 1 -- 172.21.15.104:0/241101534 >> 172.21.15.104:0/241101534 conn(0x7f60b40cad30 msgr2=0x7f60b40af580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:11.408 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.420+0000 7f60b899e700 1 -- 172.21.15.104:0/241101534 shutdown_connections 2023-07-14T11:57:11.409 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.420+0000 7f60b899e700 1 -- 172.21.15.104:0/241101534 wait complete. 2023-07-14T11:57:11.409 INFO:teuthology.orchestra.run.smithi104.stderr:instructed osd(s) 2 to deep-scrub 2023-07-14T11:57:11.425 INFO:tasks.ceph:Scrubbing osd.3 2023-07-14T11:57:11.426 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-07-14T11:57:11.609 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.624+0000 7f5995264700 1 Processor -- start 2023-07-14T11:57:11.610 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.624+0000 7f5995264700 1 -- start start 2023-07-14T11:57:11.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.624+0000 7f5995264700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f5990159c60 0x7f599006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f5995264700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f59900b0770 0x7f5990070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.611 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f5995264700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59900cab00 0x7f59900caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f5995264700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59900cb4c0 con 0x7f5990159c60 2023-07-14T11:57:11.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f5995264700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f599006da00 con 0x7f59900b0770 2023-07-14T11:57:11.612 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f5995264700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599006dce0 con 0x7f59900cab00 2023-07-14T11:57:11.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f598ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f5990159c60 0x7f599006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.613 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f598ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f5990159c60 0x7f599006d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44528/0 (socket says 172.21.15.104:44528) 2023-07-14T11:57:11.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f598ffff700 1 -- 172.21.15.104:0/3644271685 learned_addr learned my addr 172.21.15.104:0/3644271685 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:11.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f598ffff700 1 -- 172.21.15.104:0/3644271685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59900cab00 msgr2=0x7f59900caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:11.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f5994a63700 1 --2- 172.21.15.104:0/3644271685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59900cab00 0x7f59900caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.614 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f598ffff700 1 --2- 172.21.15.104:0/3644271685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59900cab00 0x7f59900caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f598f7fe700 1 --2- 172.21.15.104:0/3644271685 >> v2:172.21.15.121:3300/0 conn(0x7f59900b0770 0x7f5990070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f598ffff700 1 -- 172.21.15.104:0/3644271685 >> v2:172.21.15.121:3300/0 conn(0x7f59900b0770 msgr2=0x7f5990070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:11.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f598ffff700 1 --2- 172.21.15.104:0/3644271685 >> v2:172.21.15.121:3300/0 conn(0x7f59900b0770 0x7f5990070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f598ffff700 1 -- 172.21.15.104:0/3644271685 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599006dfc0 con 0x7f5990159c60 2023-07-14T11:57:11.615 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f598f7fe700 1 --2- 172.21.15.104:0/3644271685 >> v2:172.21.15.121:3300/0 conn(0x7f59900b0770 0x7f5990070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:57:11.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f598ffff700 1 --2- 172.21.15.104:0/3644271685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5990159c60 0x7f599006d3b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f599015d7d0 tx=0x7f5980009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4ba41976d6dc99 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:11.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f598effd700 1 -- 172.21.15.104:0/3644271685 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598000b8a0 con 0x7f5990159c60 2023-07-14T11:57:11.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f598effd700 1 -- 172.21.15.104:0/3644271685 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598000ba00 con 0x7f5990159c60 2023-07-14T11:57:11.616 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f598effd700 1 -- 172.21.15.104:0/3644271685 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598000bd90 con 0x7f5990159c60 2023-07-14T11:57:11.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f5995264700 1 -- 172.21.15.104:0/3644271685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5990159c60 msgr2=0x7f599006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f5995264700 1 --2- 172.21.15.104:0/3644271685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5990159c60 0x7f599006d3b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f599015d7d0 tx=0x7f5980009bf0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f5995264700 1 -- 172.21.15.104:0/3644271685 shutdown_connections 2023-07-14T11:57:11.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f5995264700 1 --2- 172.21.15.104:0/3644271685 >> v2:172.21.15.121:3300/0 conn(0x7f59900b0770 0x7f5990070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f5995264700 1 --2- 172.21.15.104:0/3644271685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59900cab00 0x7f59900caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f5995264700 1 --2- 172.21.15.104:0/3644271685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5990159c60 0x7f599006d3b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f5995264700 1 -- 172.21.15.104:0/3644271685 >> 172.21.15.104:0/3644271685 conn(0x7f59900afc60 msgr2=0x7f59900b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:11.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f5995264700 1 -- 172.21.15.104:0/3644271685 shutdown_connections 2023-07-14T11:57:11.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.628+0000 7f5995264700 1 -- 172.21.15.104:0/3644271685 wait complete. 2023-07-14T11:57:11.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.632+0000 7f5995264700 1 Processor -- start 2023-07-14T11:57:11.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.632+0000 7f5995264700 1 -- start start 2023-07-14T11:57:11.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.632+0000 7f5995264700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59900b0770 0x7f59900b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.632+0000 7f5995264700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59900cab00 0x7f59900c1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.632+0000 7f5995264700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5990159c60 0x7f5990154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.632+0000 7f5995264700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59900c1dd0 con 0x7f59900cab00 2023-07-14T11:57:11.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.632+0000 7f5995264700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59900c2000 con 0x7f5990159c60 2023-07-14T11:57:11.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.632+0000 7f5995264700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59900c22e0 con 0x7f59900b0770 2023-07-14T11:57:11.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f598f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59900cab00 0x7f59900c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f598f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59900cab00 0x7f59900c1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44540/0 (socket says 172.21.15.104:44540) 2023-07-14T11:57:11.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f598f7fe700 1 -- 172.21.15.104:0/3052485631 learned_addr learned my addr 172.21.15.104:0/3052485631 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:11.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f598ffff700 1 --2- 172.21.15.104:0/3052485631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59900b0770 0x7f59900b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f5994a63700 1 --2- 172.21.15.104:0/3052485631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5990159c60 0x7f5990154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f598f7fe700 1 -- 172.21.15.104:0/3052485631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59900b0770 msgr2=0x7f59900b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f598f7fe700 1 --2- 172.21.15.104:0/3052485631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59900b0770 0x7f59900b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f598f7fe700 1 -- 172.21.15.104:0/3052485631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5990159c60 msgr2=0x7f5990154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f598f7fe700 1 --2- 172.21.15.104:0/3052485631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5990159c60 0x7f5990154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f598f7fe700 1 -- 172.21.15.104:0/3052485631 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5980009570 con 0x7f59900cab00 2023-07-14T11:57:11.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f598f7fe700 1 --2- 172.21.15.104:0/3052485631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59900cab00 0x7f59900c1890 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f598400c7f0 tx=0x7f598400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:11.626 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f598cff9700 1 -- 172.21.15.104:0/3052485631 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984011800 con 0x7f59900cab00 2023-07-14T11:57:11.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f598cff9700 1 -- 172.21.15.104:0/3052485631 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5984011960 con 0x7f59900cab00 2023-07-14T11:57:11.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f598cff9700 1 -- 172.21.15.104:0/3052485631 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984011d10 con 0x7f59900cab00 2023-07-14T11:57:11.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f5995264700 1 -- 172.21.15.104:0/3052485631 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59900c25c0 con 0x7f59900cab00 2023-07-14T11:57:11.627 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.636+0000 7f5995264700 1 -- 172.21.15.104:0/3052485631 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59900c2b80 con 0x7f59900cab00 2023-07-14T11:57:11.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.640+0000 7f5995264700 1 -- 172.21.15.104:0/3052485631 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f59900708c0 con 0x7f59900cab00 2023-07-14T11:57:11.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.640+0000 7f598cff9700 1 -- 172.21.15.104:0/3052485631 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5984018360 con 0x7f59900cab00 2023-07-14T11:57:11.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.640+0000 7f598cff9700 1 --2- 172.21.15.104:0/3052485631 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f597c0662f0 0x7f597c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.640+0000 7f598cff9700 1 -- 172.21.15.104:0/3052485631 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f5984083b70 con 0x7f59900cab00 2023-07-14T11:57:11.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.640+0000 7f598cff9700 1 --2- 172.21.15.104:0/3052485631 >> [v2:172.21.15.104:6816/831948081,v1:172.21.15.104:6817/831948081] conn(0x7f597c06cc00 0x7f597c06f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.640+0000 7f598cff9700 1 -- 172.21.15.104:0/3052485631 --> [v2:172.21.15.104:6816/831948081,v1:172.21.15.104:6817/831948081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f597c06f700 con 0x7f597c06cc00 2023-07-14T11:57:11.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.640+0000 7f598cff9700 1 -- 172.21.15.104:0/3052485631 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59840882a0 con 0x7f59900cab00 2023-07-14T11:57:11.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.640+0000 7f598ffff700 1 --2- 172.21.15.104:0/3052485631 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f597c0662f0 0x7f597c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.640+0000 7f5994a63700 1 --2- 172.21.15.104:0/3052485631 >> [v2:172.21.15.104:6816/831948081,v1:172.21.15.104:6817/831948081] conn(0x7f597c06cc00 0x7f597c06f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.640+0000 7f598ffff700 1 --2- 172.21.15.104:0/3052485631 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f597c0662f0 0x7f597c0687a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5980000c00 tx=0x7f5980008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:11.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.640+0000 7f5994a63700 1 --2- 172.21.15.104:0/3052485631 >> [v2:172.21.15.104:6816/831948081,v1:172.21.15.104:6817/831948081] conn(0x7f597c06cc00 0x7f597c06f030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:11.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.644+0000 7f598cff9700 1 -- 172.21.15.104:0/3052485631 <== osd.3 v2:172.21.15.104:6816/831948081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f597c06f700 con 0x7f597c06cc00 2023-07-14T11:57:11.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.660+0000 7f5995264700 1 -- 172.21.15.104:0/3052485631 --> [v2:172.21.15.104:6816/831948081,v1:172.21.15.104:6817/831948081] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f59900705e0 con 0x7f597c06cc00 2023-07-14T11:57:11.647 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.664+0000 7f598cff9700 1 -- 172.21.15.104:0/3052485631 <== osd.3 v2:172.21.15.104:6816/831948081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5988007a70 con 0x7f597c06cc00 2023-07-14T11:57:11.648 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:57:11.648 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-14T11:57:11.648 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:57:11.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.664+0000 7f5995264700 1 -- 172.21.15.104:0/3052485631 >> [v2:172.21.15.104:6816/831948081,v1:172.21.15.104:6817/831948081] conn(0x7f597c06cc00 msgr2=0x7f597c06f030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.664+0000 7f5995264700 1 --2- 172.21.15.104:0/3052485631 >> [v2:172.21.15.104:6816/831948081,v1:172.21.15.104:6817/831948081] conn(0x7f597c06cc00 0x7f597c06f030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.664+0000 7f5995264700 1 -- 172.21.15.104:0/3052485631 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f597c0662f0 msgr2=0x7f597c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.664+0000 7f5995264700 1 --2- 172.21.15.104:0/3052485631 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f597c0662f0 0x7f597c0687a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5980000c00 tx=0x7f5980008030 comp rx=0 tx=0).stop 2023-07-14T11:57:11.650 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.664+0000 7f5995264700 1 -- 172.21.15.104:0/3052485631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59900cab00 msgr2=0x7f59900c1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.664+0000 7f5995264700 1 --2- 172.21.15.104:0/3052485631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59900cab00 0x7f59900c1890 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f598400c7f0 tx=0x7f598400cb00 comp rx=0 tx=0).stop 2023-07-14T11:57:11.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.664+0000 7f5994a63700 1 -- 172.21.15.104:0/3052485631 reap_dead start 2023-07-14T11:57:11.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.664+0000 7f5995264700 1 -- 172.21.15.104:0/3052485631 shutdown_connections 2023-07-14T11:57:11.651 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.664+0000 7f5995264700 1 -- 172.21.15.104:0/3052485631 >> 172.21.15.104:0/3052485631 conn(0x7f59900afc60 msgr2=0x7f59900ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:11.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.664+0000 7f5995264700 1 -- 172.21.15.104:0/3052485631 shutdown_connections 2023-07-14T11:57:11.652 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.664+0000 7f5995264700 1 -- 172.21.15.104:0/3052485631 wait complete. 2023-07-14T11:57:11.670 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-07-14T11:57:11.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.856+0000 7f316d313700 1 Processor -- start 2023-07-14T11:57:11.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.856+0000 7f316d313700 1 -- start start 2023-07-14T11:57:11.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f316d313700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f31680cb130 0x7f316815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f316d313700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316815d230 con 0x7f31681599d0 2023-07-14T11:57:11.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f316d313700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31680cde10 con 0x7f3168071000 2023-07-14T11:57:11.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f316d313700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31680cd520 con 0x7f31680cb130 2023-07-14T11:57:11.848 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f3166ffd700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f31681599d0 0x7f31680c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43552/0 (socket says 172.21.15.104:43552) 2023-07-14T11:57:11.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f3166ffd700 1 -- 172.21.15.104:0/1308039894 learned_addr learned my addr 172.21.15.104:0/1308039894 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:11.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f31677fe700 1 --2- 172.21.15.104:0/1308039894 >> v2:172.21.15.121:3300/0 conn(0x7f31680cb130 0x7f316815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.849 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f31667fc700 1 -- 172.21.15.104:0/1308039894 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391331653 0 0) 0x7f316815d230 con 0x7f31681599d0 2023-07-14T11:57:11.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f31667fc700 1 -- 172.21.15.104:0/1308039894 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31540035d0 con 0x7f31681599d0 2023-07-14T11:57:11.850 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f31667fc700 1 -- 172.21.15.104:0/1308039894 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260017441 0 0) 0x7f31680cde10 con 0x7f3168071000 2023-07-14T11:57:11.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f31677fe700 1 -- 172.21.15.104:0/1308039894 >> v1:172.21.15.121:6789/0 conn(0x7f3168071000 legacy=0x7f31680caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f31677fe700 1 -- 172.21.15.104:0/1308039894 >> v1:172.21.15.104:6789/0 conn(0x7f31681599d0 legacy=0x7f31680c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.851 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f31677fe700 1 -- 172.21.15.104:0/1308039894 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31680715e0 con 0x7f31680cb130 2023-07-14T11:57:11.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f31677fe700 1 --2- 172.21.15.104:0/1308039894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31680cb130 0x7f316815ccf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3160009850 tx=0x7f3160009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=156b5d7c21e19e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:11.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f31667fc700 1 -- 172.21.15.104:0/1308039894 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160011830 con 0x7f31680cb130 2023-07-14T11:57:11.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f31667fc700 1 -- 172.21.15.104:0/1308039894 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3160011990 con 0x7f31680cb130 2023-07-14T11:57:11.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.860+0000 7f31667fc700 1 -- 172.21.15.104:0/1308039894 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160011d40 con 0x7f31680cb130 2023-07-14T11:57:11.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.864+0000 7f316d313700 1 -- 172.21.15.104:0/1308039894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31680cb130 msgr2=0x7f316815ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.864+0000 7f316d313700 1 --2- 172.21.15.104:0/1308039894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31680cb130 0x7f316815ccf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3160009850 tx=0x7f3160009b60 comp rx=0 tx=0).stop 2023-07-14T11:57:11.853 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.864+0000 7f316d313700 1 -- 172.21.15.104:0/1308039894 shutdown_connections 2023-07-14T11:57:11.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.864+0000 7f316d313700 1 --2- 172.21.15.104:0/1308039894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31680cb130 0x7f316815ccf0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.864+0000 7f316d313700 1 -- 172.21.15.104:0/1308039894 >> 172.21.15.104:0/1308039894 conn(0x7f316806e960 msgr2=0x7f316806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:11.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.864+0000 7f316d313700 1 -- 172.21.15.104:0/1308039894 shutdown_connections 2023-07-14T11:57:11.854 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.864+0000 7f316d313700 1 -- 172.21.15.104:0/1308039894 wait complete. 2023-07-14T11:57:11.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.864+0000 7f316d313700 1 Processor -- start 2023-07-14T11:57:11.856 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.864+0000 7f316d313700 1 -- start start 2023-07-14T11:57:11.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f316d313700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3168071000 0x7f31680afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f316d313700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31680cb130 0x7f31680b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.857 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f316d313700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31681599d0 0x7f316806be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f316d313700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31680b2b20 con 0x7f31681599d0 2023-07-14T11:57:11.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f316d313700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31681f57b0 con 0x7f3168071000 2023-07-14T11:57:11.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f316d313700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31681f4c30 con 0x7f31680cb130 2023-07-14T11:57:11.858 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f31677fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31681599d0 0x7f316806be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f3166ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3168071000 0x7f31680afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f31677fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31681599d0 0x7f316806be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44544/0 (socket says 172.21.15.104:44544) 2023-07-14T11:57:11.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f31677fe700 1 -- 172.21.15.104:0/3576275968 learned_addr learned my addr 172.21.15.104:0/3576275968 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:11.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f3166ffd700 1 --2- 172.21.15.104:0/3576275968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3168071000 0x7f31680afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:58092/0 (socket says 172.21.15.104:58092) 2023-07-14T11:57:11.859 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f315e7fc700 1 --2- 172.21.15.104:0/3576275968 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31680cb130 0x7f31680b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f31677fe700 1 -- 172.21.15.104:0/3576275968 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31680cb130 msgr2=0x7f31680b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f31677fe700 1 --2- 172.21.15.104:0/3576275968 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31680cb130 0x7f31680b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f31677fe700 1 -- 172.21.15.104:0/3576275968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3168071000 msgr2=0x7f31680afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:11.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f31677fe700 1 --2- 172.21.15.104:0/3576275968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3168071000 0x7f31680afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:11.860 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f31677fe700 1 -- 172.21.15.104:0/3576275968 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31600094b0 con 0x7f31681599d0 2023-07-14T11:57:11.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f31677fe700 1 --2- 172.21.15.104:0/3576275968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31681599d0 0x7f316806be80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3160009eb0 tx=0x7f3160004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:11.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f315ffff700 1 -- 172.21.15.104:0/3576275968 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316001a950 con 0x7f31681599d0 2023-07-14T11:57:11.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f315ffff700 1 -- 172.21.15.104:0/3576275968 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316001f070 con 0x7f31681599d0 2023-07-14T11:57:11.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f316d313700 1 -- 172.21.15.104:0/3576275968 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31681f4f10 con 0x7f31681599d0 2023-07-14T11:57:11.861 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f315ffff700 1 -- 172.21.15.104:0/3576275968 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316001ad10 con 0x7f31681599d0 2023-07-14T11:57:11.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.868+0000 7f316d313700 1 -- 172.21.15.104:0/3576275968 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31681f54d0 con 0x7f31681599d0 2023-07-14T11:57:11.862 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.872+0000 7f316d313700 1 -- 172.21.15.104:0/3576275968 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31680b2c90 con 0x7f31681599d0 2023-07-14T11:57:11.863 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.872+0000 7f315ffff700 1 -- 172.21.15.104:0/3576275968 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f316001b430 con 0x7f31681599d0 2023-07-14T11:57:11.863 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.872+0000 7f315ffff700 1 --2- 172.21.15.104:0/3576275968 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31480662f0 0x7f31480687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:11.864 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.872+0000 7f315ffff700 1 -- 172.21.15.104:0/3576275968 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f3160084ad0 con 0x7f31681599d0 2023-07-14T11:57:11.864 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.876+0000 7f3166ffd700 1 --2- 172.21.15.104:0/3576275968 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31480662f0 0x7f31480687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:11.864 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.876+0000 7f3166ffd700 1 --2- 172.21.15.104:0/3576275968 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31480662f0 0x7f31480687a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3158007fd0 tx=0x7f3158009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:11.865 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:11.880+0000 7f315ffff700 1 -- 172.21.15.104:0/3576275968 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f31680b2c90 con 0x7f31681599d0 2023-07-14T11:57:12.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.052+0000 7f316d313700 1 -- 172.21.15.104:0/3576275968 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f31680cdaa0 con 0x7f31480662f0 2023-07-14T11:57:12.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.052+0000 7f315ffff700 1 -- 172.21.15.104:0/3576275968 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3158001d70 con 0x7f31480662f0 2023-07-14T11:57:12.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.052+0000 7f316d313700 1 -- 172.21.15.104:0/3576275968 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31480662f0 msgr2=0x7f31480687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.039 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.052+0000 7f316d313700 1 --2- 172.21.15.104:0/3576275968 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31480662f0 0x7f31480687a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3158007fd0 tx=0x7f3158009040 comp rx=0 tx=0).stop 2023-07-14T11:57:12.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.052+0000 7f316d313700 1 -- 172.21.15.104:0/3576275968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31681599d0 msgr2=0x7f316806be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.052+0000 7f316d313700 1 --2- 172.21.15.104:0/3576275968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31681599d0 0x7f316806be80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3160009eb0 tx=0x7f3160004770 comp rx=0 tx=0).stop 2023-07-14T11:57:12.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.056+0000 7f316d313700 1 -- 172.21.15.104:0/3576275968 shutdown_connections 2023-07-14T11:57:12.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.056+0000 7f316d313700 1 --2- 172.21.15.104:0/3576275968 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f31480662f0 0x7f31480687a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.040 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.056+0000 7f316d313700 1 --2- 172.21.15.104:0/3576275968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31681599d0 0x7f316806be80 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.056+0000 7f316d313700 1 --2- 172.21.15.104:0/3576275968 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31680cb130 0x7f31680b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.056+0000 7f316d313700 1 --2- 172.21.15.104:0/3576275968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3168071000 0x7f31680afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.041 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.056+0000 7f316d313700 1 -- 172.21.15.104:0/3576275968 >> 172.21.15.104:0/3576275968 conn(0x7f316806e960 msgr2=0x7f31680ca650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:12.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.056+0000 7f316d313700 1 -- 172.21.15.104:0/3576275968 shutdown_connections 2023-07-14T11:57:12.042 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.056+0000 7f316d313700 1 -- 172.21.15.104:0/3576275968 wait complete. 2023-07-14T11:57:12.042 INFO:teuthology.orchestra.run.smithi104.stderr:instructed osd(s) 3 to deep-scrub 2023-07-14T11:57:12.062 INFO:tasks.ceph:Scrubbing osd.4 2023-07-14T11:57:12.063 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-07-14T11:57:12.245 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.260+0000 7f13f0a10700 1 Processor -- start 2023-07-14T11:57:12.246 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.260+0000 7f13f0a10700 1 -- start start 2023-07-14T11:57:12.249 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.264+0000 7f13f0a10700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f13ec0cd260 0x7f13ec0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.249 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.264+0000 7f13f0a10700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec0d1a60 con 0x7f13ec0d0610 2023-07-14T11:57:12.250 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.264+0000 7f13f0a10700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec0d1ef0 con 0x7f13ec0b0ee0 2023-07-14T11:57:12.250 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.264+0000 7f13f0a10700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ec0d0e50 con 0x7f13ec0cd260 2023-07-14T11:57:12.250 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.264+0000 7f13ea59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f13ec0cd260 0x7f13ec0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.251 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.264+0000 7f13ea59c700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f13ec0cd260 0x7f13ec0d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44546/0 (socket says 172.21.15.104:44546) 2023-07-14T11:57:12.251 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.264+0000 7f13ea59c700 1 -- 172.21.15.104:0/729837710 learned_addr learned my addr 172.21.15.104:0/729837710 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:12.251 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.264+0000 7f13eb59e700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f13ec0d0610 0x7f13ec0af0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43566/0 (socket says 172.21.15.104:43566) 2023-07-14T11:57:12.252 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.264+0000 7f13ea59c700 1 -- 172.21.15.104:0/729837710 >> v1:172.21.15.121:6789/0 conn(0x7f13ec0b0ee0 legacy=0x7f13ec0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.252 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.264+0000 7f13e9d9b700 1 -- 172.21.15.104:0/729837710 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353017615 0 0) 0x7f13ec0d1a60 con 0x7f13ec0d0610 2023-07-14T11:57:12.253 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.264+0000 7f13ea59c700 1 -- 172.21.15.104:0/729837710 >> v1:172.21.15.104:6789/0 conn(0x7f13ec0d0610 legacy=0x7f13ec0af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.253 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.264+0000 7f13ea59c700 1 -- 172.21.15.104:0/729837710 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec0d1130 con 0x7f13ec0cd260 2023-07-14T11:57:12.253 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.268+0000 7f13ea59c700 1 --2- 172.21.15.104:0/729837710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13ec0cd260 0x7f13ec0d00d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009910 tx=0x7f13e0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94faf3977b3354a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:12.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.268+0000 7f13e9d9b700 1 -- 172.21.15.104:0/729837710 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e000ba60 con 0x7f13ec0cd260 2023-07-14T11:57:12.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.268+0000 7f13e9d9b700 1 -- 172.21.15.104:0/729837710 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e000bbc0 con 0x7f13ec0cd260 2023-07-14T11:57:12.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.268+0000 7f13e9d9b700 1 -- 172.21.15.104:0/729837710 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0009e60 con 0x7f13ec0cd260 2023-07-14T11:57:12.254 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.268+0000 7f13f0a10700 1 -- 172.21.15.104:0/729837710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13ec0cd260 msgr2=0x7f13ec0d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.255 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.268+0000 7f13f0a10700 1 --2- 172.21.15.104:0/729837710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13ec0cd260 0x7f13ec0d00d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009910 tx=0x7f13e0009c20 comp rx=0 tx=0).stop 2023-07-14T11:57:12.255 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.268+0000 7f13f0a10700 1 -- 172.21.15.104:0/729837710 shutdown_connections 2023-07-14T11:57:12.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.268+0000 7f13f0a10700 1 --2- 172.21.15.104:0/729837710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13ec0cd260 0x7f13ec0d00d0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.256 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.268+0000 7f13f0a10700 1 -- 172.21.15.104:0/729837710 >> 172.21.15.104:0/729837710 conn(0x7f13ec060c40 msgr2=0x7f13ec09b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:12.257 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.268+0000 7f13f0a10700 1 -- 172.21.15.104:0/729837710 shutdown_connections 2023-07-14T11:57:12.257 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.268+0000 7f13f0a10700 1 -- 172.21.15.104:0/729837710 wait complete. 2023-07-14T11:57:12.257 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13f0a10700 1 Processor -- start 2023-07-14T11:57:12.258 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13f0a10700 1 -- start start 2023-07-14T11:57:12.260 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13f0a10700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13ec0b0ee0 0x7f13ec0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.260 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13f0a10700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13ec0cd260 0x7f13ec155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.260 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13f0a10700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13ec0d0610 0x7f13ec0b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.261 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13f0a10700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0b1af0 con 0x7f13ec0cd260 2023-07-14T11:57:12.261 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13f0a10700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0b3da0 con 0x7f13ec0d0610 2023-07-14T11:57:12.261 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13f0a10700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0b2040 con 0x7f13ec0b0ee0 2023-07-14T11:57:12.261 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13ea59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13ec0cd260 0x7f13ec155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.261 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13ea59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13ec0cd260 0x7f13ec155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44562/0 (socket says 172.21.15.104:44562) 2023-07-14T11:57:12.262 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13ea59c700 1 -- 172.21.15.104:0/3132281004 learned_addr learned my addr 172.21.15.104:0/3132281004 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:12.262 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13eb59e700 1 --2- 172.21.15.104:0/3132281004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13ec0d0610 0x7f13ec0b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.262 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13ead9d700 1 --2- 172.21.15.104:0/3132281004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13ec0b0ee0 0x7f13ec0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.263 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13ea59c700 1 -- 172.21.15.104:0/3132281004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13ec0b0ee0 msgr2=0x7f13ec0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.263 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13ea59c700 1 --2- 172.21.15.104:0/3132281004 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13ec0b0ee0 0x7f13ec0b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.263 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13ea59c700 1 -- 172.21.15.104:0/3132281004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13ec0d0610 msgr2=0x7f13ec0b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.263 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13ea59c700 1 --2- 172.21.15.104:0/3132281004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13ec0d0610 0x7f13ec0b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.263 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13ea59c700 1 -- 172.21.15.104:0/3132281004 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e0009570 con 0x7f13ec0cd260 2023-07-14T11:57:12.264 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13eb59e700 1 --2- 172.21.15.104:0/3132281004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13ec0d0610 0x7f13ec0b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:57:12.264 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13ea59c700 1 --2- 172.21.15.104:0/3132281004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13ec0cd260 0x7f13ec155d60 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f13e0000c00 tx=0x7f13e0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:12.264 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13d37fe700 1 -- 172.21.15.104:0/3132281004 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0011a70 con 0x7f13ec0cd260 2023-07-14T11:57:12.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13d37fe700 1 -- 172.21.15.104:0/3132281004 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e001e070 con 0x7f13ec0cd260 2023-07-14T11:57:12.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13d37fe700 1 -- 172.21.15.104:0/3132281004 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0011e10 con 0x7f13ec0cd260 2023-07-14T11:57:12.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.272+0000 7f13f0a10700 1 -- 172.21.15.104:0/3132281004 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec157230 con 0x7f13ec0cd260 2023-07-14T11:57:12.265 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.276+0000 7f13f0a10700 1 -- 172.21.15.104:0/3132281004 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ec0b37e0 con 0x7f13ec0cd260 2023-07-14T11:57:12.266 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.276+0000 7f13f0a10700 1 -- 172.21.15.104:0/3132281004 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f13ec0d16f0 con 0x7f13ec0cd260 2023-07-14T11:57:12.267 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.276+0000 7f13d37fe700 1 -- 172.21.15.104:0/3132281004 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f13e002d450 con 0x7f13ec0cd260 2023-07-14T11:57:12.268 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.280+0000 7f13d37fe700 1 --2- 172.21.15.104:0/3132281004 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f13d80662f0 0x7f13d80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.268 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.280+0000 7f13d37fe700 1 -- 172.21.15.104:0/3132281004 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f13e0084d10 con 0x7f13ec0cd260 2023-07-14T11:57:12.268 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.280+0000 7f13d37fe700 1 --2- 172.21.15.104:0/3132281004 >> [v2:172.21.15.121:6822/2394615815,v1:172.21.15.121:6825/2394615815] conn(0x7f13d806cc00 0x7f13d806f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.268 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.280+0000 7f13d37fe700 1 -- 172.21.15.104:0/3132281004 --> [v2:172.21.15.121:6822/2394615815,v1:172.21.15.121:6825/2394615815] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13d806f700 con 0x7f13d806cc00 2023-07-14T11:57:12.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.280+0000 7f13d37fe700 1 -- 172.21.15.104:0/3132281004 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13e0088020 con 0x7f13ec0cd260 2023-07-14T11:57:12.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.280+0000 7f13ead9d700 1 --2- 172.21.15.104:0/3132281004 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f13d80662f0 0x7f13d80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.280+0000 7f13eb59e700 1 --2- 172.21.15.104:0/3132281004 >> [v2:172.21.15.121:6822/2394615815,v1:172.21.15.121:6825/2394615815] conn(0x7f13d806cc00 0x7f13d806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.269 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.280+0000 7f13ead9d700 1 --2- 172.21.15.104:0/3132281004 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f13d80662f0 0x7f13d80687a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f13dc000f60 tx=0x7f13dc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:12.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.280+0000 7f13eb59e700 1 --2- 172.21.15.104:0/3132281004 >> [v2:172.21.15.121:6822/2394615815,v1:172.21.15.121:6825/2394615815] conn(0x7f13d806cc00 0x7f13d806f030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:12.270 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.284+0000 7f13d37fe700 1 -- 172.21.15.104:0/3132281004 <== osd.4 v2:172.21.15.121:6822/2394615815 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f13d806f700 con 0x7f13d806cc00 2023-07-14T11:57:12.284 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.300+0000 7f13f0a10700 1 -- 172.21.15.104:0/3132281004 --> [v2:172.21.15.121:6822/2394615815,v1:172.21.15.121:6825/2394615815] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f13ec0d1410 con 0x7f13d806cc00 2023-07-14T11:57:12.285 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.300+0000 7f13d37fe700 1 -- 172.21.15.104:0/3132281004 <== osd.4 v2:172.21.15.121:6822/2394615815 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f13e4007230 con 0x7f13d806cc00 2023-07-14T11:57:12.286 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:57:12.286 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-14T11:57:12.286 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:57:12.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.300+0000 7f13f0a10700 1 -- 172.21.15.104:0/3132281004 >> [v2:172.21.15.121:6822/2394615815,v1:172.21.15.121:6825/2394615815] conn(0x7f13d806cc00 msgr2=0x7f13d806f030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.300+0000 7f13f0a10700 1 --2- 172.21.15.104:0/3132281004 >> [v2:172.21.15.121:6822/2394615815,v1:172.21.15.121:6825/2394615815] conn(0x7f13d806cc00 0x7f13d806f030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.287 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.304+0000 7f13f0a10700 1 -- 172.21.15.104:0/3132281004 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f13d80662f0 msgr2=0x7f13d80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.304+0000 7f13f0a10700 1 --2- 172.21.15.104:0/3132281004 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f13d80662f0 0x7f13d80687a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f13dc000f60 tx=0x7f13dc009040 comp rx=0 tx=0).stop 2023-07-14T11:57:12.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.304+0000 7f13f0a10700 1 -- 172.21.15.104:0/3132281004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13ec0cd260 msgr2=0x7f13ec155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.304+0000 7f13f0a10700 1 --2- 172.21.15.104:0/3132281004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13ec0cd260 0x7f13ec155d60 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f13e0000c00 tx=0x7f13e0004770 comp rx=0 tx=0).stop 2023-07-14T11:57:12.288 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.304+0000 7f13eb59e700 1 -- 172.21.15.104:0/3132281004 reap_dead start 2023-07-14T11:57:12.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.304+0000 7f13f0a10700 1 -- 172.21.15.104:0/3132281004 shutdown_connections 2023-07-14T11:57:12.289 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.304+0000 7f13f0a10700 1 -- 172.21.15.104:0/3132281004 >> 172.21.15.104:0/3132281004 conn(0x7f13ec060c40 msgr2=0x7f13ec15abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:12.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.304+0000 7f13f0a10700 1 -- 172.21.15.104:0/3132281004 shutdown_connections 2023-07-14T11:57:12.290 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.304+0000 7f13f0a10700 1 -- 172.21.15.104:0/3132281004 wait complete. 2023-07-14T11:57:12.311 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-07-14T11:57:12.481 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.496+0000 7fd6c759e700 1 Processor -- start 2023-07-14T11:57:12.482 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.496+0000 7fd6c759e700 1 -- start start 2023-07-14T11:57:12.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c759e700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fd6c80cd240 0x7fd6c80ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c759e700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c80af0a0 con 0x7fd6c80ceff0 2023-07-14T11:57:12.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c759e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c80cf720 con 0x7fd6c815bff0 2023-07-14T11:57:12.484 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c759e700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c80af2f0 con 0x7fd6c80cd240 2023-07-14T11:57:12.485 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c5d9b700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fd6c80cd240 0x7fd6c80ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c659c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd6c815bff0 0x7fd6c80ccb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:43440/0 (socket says 172.21.15.104:43440) 2023-07-14T11:57:12.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c659c700 1 -- 172.21.15.104:0/3514970939 learned_addr learned my addr 172.21.15.104:0/3514970939 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:12.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c559a700 1 -- 172.21.15.104:0/3514970939 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569510519 0 0) 0x7fd6c80af0a0 con 0x7fd6c80ceff0 2023-07-14T11:57:12.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c559a700 1 -- 172.21.15.104:0/3514970939 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6b00035d0 con 0x7fd6c80ceff0 2023-07-14T11:57:12.486 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c5d9b700 1 -- 172.21.15.104:0/3514970939 >> v1:172.21.15.121:6789/0 conn(0x7fd6c815bff0 legacy=0x7fd6c80ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c5d9b700 1 -- 172.21.15.104:0/3514970939 >> v1:172.21.15.104:6789/0 conn(0x7fd6c80ceff0 legacy=0x7fd6c80afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c5d9b700 1 -- 172.21.15.104:0/3514970939 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80af5d0 con 0x7fd6c80cd240 2023-07-14T11:57:12.487 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c5d9b700 1 --2- 172.21.15.104:0/3514970939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c80cd240 0x7fd6c80ceab0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009850 tx=0x7fd6bc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce01f7708e39e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:12.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c559a700 1 -- 172.21.15.104:0/3514970939 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc011830 con 0x7fd6c80cd240 2023-07-14T11:57:12.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c559a700 1 -- 172.21.15.104:0/3514970939 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc011990 con 0x7fd6c80cd240 2023-07-14T11:57:12.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.500+0000 7fd6c559a700 1 -- 172.21.15.104:0/3514970939 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc011d40 con 0x7fd6c80cd240 2023-07-14T11:57:12.488 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.504+0000 7fd6c759e700 1 -- 172.21.15.104:0/3514970939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c80cd240 msgr2=0x7fd6c80ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.504+0000 7fd6c759e700 1 --2- 172.21.15.104:0/3514970939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c80cd240 0x7fd6c80ceab0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009850 tx=0x7fd6bc009b60 comp rx=0 tx=0).stop 2023-07-14T11:57:12.489 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.504+0000 7fd6c759e700 1 -- 172.21.15.104:0/3514970939 shutdown_connections 2023-07-14T11:57:12.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.504+0000 7fd6c759e700 1 --2- 172.21.15.104:0/3514970939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c80cd240 0x7fd6c80ceab0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.504+0000 7fd6c759e700 1 -- 172.21.15.104:0/3514970939 >> 172.21.15.104:0/3514970939 conn(0x7fd6c8060c40 msgr2=0x7fd6c809b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:12.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.504+0000 7fd6c759e700 1 -- 172.21.15.104:0/3514970939 shutdown_connections 2023-07-14T11:57:12.490 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.504+0000 7fd6c759e700 1 -- 172.21.15.104:0/3514970939 wait complete. 2023-07-14T11:57:12.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.504+0000 7fd6c759e700 1 Processor -- start 2023-07-14T11:57:12.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.504+0000 7fd6c759e700 1 -- start start 2023-07-14T11:57:12.492 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6c80cd240 0x7fd6c814e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c759e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c80ceff0 0x7fd6c815e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c759e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6c815bff0 0x7fd6c815eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c759e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80b6870 con 0x7fd6c80ceff0 2023-07-14T11:57:12.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c814e800 con 0x7fd6c80cd240 2023-07-14T11:57:12.493 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c759e700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80b5fd0 con 0x7fd6c815bff0 2023-07-14T11:57:12.494 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c5d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c80ceff0 0x7fd6c815e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c5d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c80ceff0 0x7fd6c815e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44592/0 (socket says 172.21.15.104:44592) 2023-07-14T11:57:12.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c5d9b700 1 -- 172.21.15.104:0/3065352531 learned_addr learned my addr 172.21.15.104:0/3065352531 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:12.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c5d9b700 1 -- 172.21.15.104:0/3065352531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6c815bff0 msgr2=0x7fd6c815eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.495 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c5d9b700 1 --2- 172.21.15.104:0/3065352531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6c815bff0 0x7fd6c815eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c659c700 1 --2- 172.21.15.104:0/3065352531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6c80cd240 0x7fd6c814e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c5d9b700 1 -- 172.21.15.104:0/3065352531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6c80cd240 msgr2=0x7fd6c814e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c5d9b700 1 --2- 172.21.15.104:0/3065352531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6c80cd240 0x7fd6c814e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c5d9b700 1 -- 172.21.15.104:0/3065352531 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6bc0094b0 con 0x7fd6c80ceff0 2023-07-14T11:57:12.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c659c700 1 --2- 172.21.15.104:0/3065352531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6c80cd240 0x7fd6c814e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:57:12.496 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c5d9b700 1 --2- 172.21.15.104:0/3065352531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c80ceff0 0x7fd6c815e960 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc014040 tx=0x7fd6bc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:12.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6b6ffd700 1 -- 172.21.15.104:0/3065352531 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc019ab0 con 0x7fd6c80ceff0 2023-07-14T11:57:12.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6b6ffd700 1 -- 172.21.15.104:0/3065352531 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc01e070 con 0x7fd6c80ceff0 2023-07-14T11:57:12.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c759e700 1 -- 172.21.15.104:0/3065352531 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c80b62b0 con 0x7fd6c80ceff0 2023-07-14T11:57:12.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6b6ffd700 1 -- 172.21.15.104:0/3065352531 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc019e50 con 0x7fd6c80ceff0 2023-07-14T11:57:12.497 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.508+0000 7fd6c759e700 1 -- 172.21.15.104:0/3065352531 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c815c560 con 0x7fd6c80ceff0 2023-07-14T11:57:12.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.512+0000 7fd6c759e700 1 -- 172.21.15.104:0/3065352531 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c815c3c0 con 0x7fd6c80ceff0 2023-07-14T11:57:12.498 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.512+0000 7fd6b6ffd700 1 -- 172.21.15.104:0/3065352531 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd6bc02d450 con 0x7fd6c80ceff0 2023-07-14T11:57:12.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.512+0000 7fd6b6ffd700 1 --2- 172.21.15.104:0/3065352531 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd6a80662f0 0x7fd6a80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.512+0000 7fd6b6ffd700 1 -- 172.21.15.104:0/3065352531 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fd6bc084c70 con 0x7fd6c80ceff0 2023-07-14T11:57:12.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.512+0000 7fd6c659c700 1 --2- 172.21.15.104:0/3065352531 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd6a80662f0 0x7fd6a80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.499 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.516+0000 7fd6c659c700 1 --2- 172.21.15.104:0/3065352531 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd6a80662f0 0x7fd6a80687a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8005930 tx=0x7fd6b8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:12.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.520+0000 7fd6b6ffd700 1 -- 172.21.15.104:0/3065352531 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fd6c815c3c0 con 0x7fd6c80ceff0 2023-07-14T11:57:12.677 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.692+0000 7fd6c759e700 1 -- 172.21.15.104:0/3065352531 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6c80d1fa0 con 0x7fd6a80662f0 2023-07-14T11:57:12.678 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.692+0000 7fd6b6ffd700 1 -- 172.21.15.104:0/3065352531 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd6b8001c70 con 0x7fd6a80662f0 2023-07-14T11:57:12.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.696+0000 7fd6c759e700 1 -- 172.21.15.104:0/3065352531 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd6a80662f0 msgr2=0x7fd6a80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.696+0000 7fd6c759e700 1 --2- 172.21.15.104:0/3065352531 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd6a80662f0 0x7fd6a80687a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8005930 tx=0x7fd6b8009040 comp rx=0 tx=0).stop 2023-07-14T11:57:12.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.696+0000 7fd6c759e700 1 -- 172.21.15.104:0/3065352531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c80ceff0 msgr2=0x7fd6c815e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.682 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.696+0000 7fd6c759e700 1 --2- 172.21.15.104:0/3065352531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c80ceff0 0x7fd6c815e960 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc014040 tx=0x7fd6bc004770 comp rx=0 tx=0).stop 2023-07-14T11:57:12.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.696+0000 7fd6c759e700 1 -- 172.21.15.104:0/3065352531 shutdown_connections 2023-07-14T11:57:12.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.696+0000 7fd6c759e700 1 --2- 172.21.15.104:0/3065352531 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fd6a80662f0 0x7fd6a80687a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.696+0000 7fd6c759e700 1 --2- 172.21.15.104:0/3065352531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c80ceff0 0x7fd6c815e960 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.683 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.696+0000 7fd6c759e700 1 --2- 172.21.15.104:0/3065352531 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6c815bff0 0x7fd6c815eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.696+0000 7fd6c759e700 1 --2- 172.21.15.104:0/3065352531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6c80cd240 0x7fd6c814e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.696+0000 7fd6c759e700 1 -- 172.21.15.104:0/3065352531 >> 172.21.15.104:0/3065352531 conn(0x7fd6c8060c40 msgr2=0x7fd6c8070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:12.684 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.696+0000 7fd6c759e700 1 -- 172.21.15.104:0/3065352531 shutdown_connections 2023-07-14T11:57:12.685 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.696+0000 7fd6c759e700 1 -- 172.21.15.104:0/3065352531 wait complete. 2023-07-14T11:57:12.685 INFO:teuthology.orchestra.run.smithi104.stderr:instructed osd(s) 4 to deep-scrub 2023-07-14T11:57:12.705 INFO:tasks.ceph:Scrubbing osd.5 2023-07-14T11:57:12.705 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-07-14T11:57:12.889 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.904+0000 7f36174f2700 1 Processor -- start 2023-07-14T11:57:12.890 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.904+0000 7f36174f2700 1 -- start start 2023-07-14T11:57:12.891 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36174f2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f36100cd260 0x7f36100d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36174f2700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36100d1a60 con 0x7f36100b0ee0 2023-07-14T11:57:12.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36174f2700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36100d1ef0 con 0x7f36100d0610 2023-07-14T11:57:12.892 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36174f2700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36100d0e50 con 0x7f36100cd260 2023-07-14T11:57:12.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36164f0700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f36100b0ee0 0x7f36100ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43580/0 (socket says 172.21.15.104:43580) 2023-07-14T11:57:12.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36164f0700 1 -- 172.21.15.104:0/1356243825 learned_addr learned my addr 172.21.15.104:0/1356243825 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:12.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f3615cef700 1 --2- 172.21.15.104:0/1356243825 >> v2:172.21.15.121:3300/0 conn(0x7f36100cd260 0x7f36100d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.894 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744321044 0 0) 0x7f36100d1a60 con 0x7f36100b0ee0 2023-07-14T11:57:12.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35fc0035d0 con 0x7f36100b0ee0 2023-07-14T11:57:12.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597944063 0 0) 0x7f36100d1ef0 con 0x7f36100d0610 2023-07-14T11:57:12.895 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36100d1a60 con 0x7f36100d0610 2023-07-14T11:57:12.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1300045250 0 0) 0x7f35fc0035d0 con 0x7f36100b0ee0 2023-07-14T11:57:12.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 >> v2:172.21.15.121:3300/0 conn(0x7f36100cd260 msgr2=0x7f36100d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:12.896 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 --2- 172.21.15.104:0/1356243825 >> v2:172.21.15.121:3300/0 conn(0x7f36100cd260 0x7f36100d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 >> v1:172.21.15.121:6789/0 conn(0x7f36100d0610 legacy=0x7f36100af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36100d1130 con 0x7f36100b0ee0 2023-07-14T11:57:12.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3246083461 0 0) 0x7f360c001670 con 0x7f36100b0ee0 2023-07-14T11:57:12.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 >> v1:172.21.15.104:6789/0 conn(0x7f36100b0ee0 legacy=0x7f36100ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.897 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 --2- 172.21.15.104:0/1356243825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35fc003bb0 0x7f35fc003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 --2- 172.21.15.104:0/1356243825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35fc004870 0x7f35fc004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 --2- 172.21.15.104:0/1356243825 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35fc005220 0x7f35fc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36100d0e50 con 0x7f35fc003bb0 2023-07-14T11:57:12.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc00c080 con 0x7f35fc004870 2023-07-14T11:57:12.898 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc00c240 con 0x7f35fc005220 2023-07-14T11:57:12.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36164f0700 1 --2- 172.21.15.104:0/1356243825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35fc003bb0 0x7f35fc003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.901 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f3615cef700 1 --2- 172.21.15.104:0/1356243825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35fc004870 0x7f35fc004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f3616cf1700 1 --2- 172.21.15.104:0/1356243825 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35fc005220 0x7f35fc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.902 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.908+0000 7f36164f0700 1 -- 172.21.15.104:0/1356243825 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35fc005220 msgr2=0x7f35fc00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:12.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f36164f0700 1 --2- 172.21.15.104:0/1356243825 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35fc005220 0x7f35fc00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.903 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f36164f0700 1 -- 172.21.15.104:0/1356243825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35fc004870 msgr2=0x7f35fc004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:12.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f36164f0700 1 --2- 172.21.15.104:0/1356243825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35fc004870 0x7f35fc004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f36164f0700 1 -- 172.21.15.104:0/1356243825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc0035d0 con 0x7f35fc003bb0 2023-07-14T11:57:12.904 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f3616cf1700 1 -- 172.21.15.104:0/1356243825 reap_dead start 2023-07-14T11:57:12.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f36164f0700 1 --2- 172.21.15.104:0/1356243825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35fc003bb0 0x7f35fc003fa0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f360c007fd0 tx=0x7f35fc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69a7b8a35e20bf9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:12.905 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c00cd80 con 0x7f35fc003bb0 2023-07-14T11:57:12.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c001af0 con 0x7f35fc003bb0 2023-07-14T11:57:12.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f36154ee700 1 -- 172.21.15.104:0/1356243825 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c005bc0 con 0x7f35fc003bb0 2023-07-14T11:57:12.906 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f36174f2700 1 -- 172.21.15.104:0/1356243825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35fc003bb0 msgr2=0x7f35fc003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f36174f2700 1 --2- 172.21.15.104:0/1356243825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35fc003bb0 0x7f35fc003fa0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f360c007fd0 tx=0x7f35fc0058a0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.907 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f36174f2700 1 -- 172.21.15.104:0/1356243825 shutdown_connections 2023-07-14T11:57:12.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f36174f2700 1 --2- 172.21.15.104:0/1356243825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35fc003bb0 0x7f35fc003fa0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f36174f2700 1 -- 172.21.15.104:0/1356243825 >> 172.21.15.104:0/1356243825 conn(0x7f3610060c40 msgr2=0x7f361009b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:12.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f36174f2700 1 -- 172.21.15.104:0/1356243825 shutdown_connections 2023-07-14T11:57:12.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.912+0000 7f36174f2700 1 -- 172.21.15.104:0/1356243825 wait complete. 2023-07-14T11:57:12.909 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.916+0000 7f36174f2700 1 Processor -- start 2023-07-14T11:57:12.910 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.916+0000 7f36174f2700 1 -- start start 2023-07-14T11:57:12.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.916+0000 7f36174f2700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35fc003bb0 0x7f361015e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.916+0000 7f36174f2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3610155850 0x7f361015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.916+0000 7f36174f2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3610155350 0x7f36100b6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.916+0000 7f36174f2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361015f270 con 0x7f3610155850 2023-07-14T11:57:12.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.916+0000 7f36174f2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361015cfe0 con 0x7f3610155350 2023-07-14T11:57:12.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.916+0000 7f36174f2700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36100b7860 con 0x7f35fc003bb0 2023-07-14T11:57:12.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.916+0000 7f3615cef700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3610155850 0x7f361015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.916 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.916+0000 7f3615cef700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3610155850 0x7f361015ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44616/0 (socket says 172.21.15.104:44616) 2023-07-14T11:57:12.916 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.916+0000 7f3615cef700 1 -- 172.21.15.104:0/1797506114 learned_addr learned my addr 172.21.15.104:0/1797506114 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:12.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.916+0000 7f36164f0700 1 --2- 172.21.15.104:0/1797506114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35fc003bb0 0x7f361015e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.916+0000 7f3615cef700 1 -- 172.21.15.104:0/1797506114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35fc003bb0 msgr2=0x7f361015e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.917 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.916+0000 7f3616cf1700 1 --2- 172.21.15.104:0/1797506114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3610155350 0x7f36100b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.916+0000 7f3615cef700 1 --2- 172.21.15.104:0/1797506114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35fc003bb0 0x7f361015e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.920+0000 7f3615cef700 1 -- 172.21.15.104:0/1797506114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3610155350 msgr2=0x7f36100b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.918 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.920+0000 7f3615cef700 1 --2- 172.21.15.104:0/1797506114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3610155350 0x7f36100b6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.920+0000 7f3615cef700 1 -- 172.21.15.104:0/1797506114 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36100b7b40 con 0x7f3610155850 2023-07-14T11:57:12.919 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.920+0000 7f36164f0700 1 --2- 172.21.15.104:0/1797506114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35fc003bb0 0x7f361015e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:57:12.920 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.920+0000 7f3615cef700 1 --2- 172.21.15.104:0/1797506114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3610155850 0x7f361015ed30 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3600011900 tx=0x7f3600011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:12.920 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.920+0000 7f3606ffd700 1 -- 172.21.15.104:0/1797506114 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600004240 con 0x7f3610155850 2023-07-14T11:57:12.920 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.920+0000 7f3606ffd700 1 -- 172.21.15.104:0/1797506114 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36000043a0 con 0x7f3610155850 2023-07-14T11:57:12.921 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.920+0000 7f36174f2700 1 -- 172.21.15.104:0/1797506114 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36100b7e20 con 0x7f3610155850 2023-07-14T11:57:12.921 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.920+0000 7f3606ffd700 1 -- 172.21.15.104:0/1797506114 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360000f730 con 0x7f3610155850 2023-07-14T11:57:12.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.920+0000 7f36174f2700 1 -- 172.21.15.104:0/1797506114 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361015cc70 con 0x7f3610155850 2023-07-14T11:57:12.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.920+0000 7f36174f2700 1 -- 172.21.15.104:0/1797506114 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f36100b1720 con 0x7f3610155850 2023-07-14T11:57:12.922 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.924+0000 7f3606ffd700 1 -- 172.21.15.104:0/1797506114 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f360000f890 con 0x7f3610155850 2023-07-14T11:57:12.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.924+0000 7f3606ffd700 1 --2- 172.21.15.104:0/1797506114 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f35f40662f0 0x7f35f40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.923 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.924+0000 7f3606ffd700 1 -- 172.21.15.104:0/1797506114 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f36000845e0 con 0x7f3610155850 2023-07-14T11:57:12.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.924+0000 7f3606ffd700 1 --2- 172.21.15.104:0/1797506114 >> [v2:172.21.15.121:6800/1185793020,v1:172.21.15.121:6801/1185793020] conn(0x7f35f406cc00 0x7f35f406f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:12.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.924+0000 7f3606ffd700 1 -- 172.21.15.104:0/1797506114 --> [v2:172.21.15.121:6800/1185793020,v1:172.21.15.121:6801/1185793020] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35f406f700 con 0x7f35f406cc00 2023-07-14T11:57:12.924 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.924+0000 7f3606ffd700 1 -- 172.21.15.104:0/1797506114 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3600084950 con 0x7f3610155850 2023-07-14T11:57:12.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.924+0000 7f36164f0700 1 --2- 172.21.15.104:0/1797506114 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f35f40662f0 0x7f35f40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.925 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.924+0000 7f3616cf1700 1 --2- 172.21.15.104:0/1797506114 >> [v2:172.21.15.121:6800/1185793020,v1:172.21.15.121:6801/1185793020] conn(0x7f35f406cc00 0x7f35f406f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:12.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.924+0000 7f36164f0700 1 --2- 172.21.15.104:0/1797506114 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f35f40662f0 0x7f35f40687a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f360c015040 tx=0x7f360c00a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:12.926 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.928+0000 7f3616cf1700 1 --2- 172.21.15.104:0/1797506114 >> [v2:172.21.15.121:6800/1185793020,v1:172.21.15.121:6801/1185793020] conn(0x7f35f406cc00 0x7f35f406f030 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:12.927 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.928+0000 7f3606ffd700 1 -- 172.21.15.104:0/1797506114 <== osd.5 v2:172.21.15.121:6800/1185793020 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f35f406f700 con 0x7f35f406cc00 2023-07-14T11:57:12.929 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.944+0000 7f36174f2700 1 -- 172.21.15.104:0/1797506114 --> [v2:172.21.15.121:6800/1185793020,v1:172.21.15.121:6801/1185793020] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f36100d16f0 con 0x7f35f406cc00 2023-07-14T11:57:12.930 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.944+0000 7f3606ffd700 1 -- 172.21.15.104:0/1797506114 <== osd.5 v2:172.21.15.121:6800/1185793020 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f360800d040 con 0x7f35f406cc00 2023-07-14T11:57:12.931 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:57:12.931 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-14T11:57:12.931 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:57:12.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.948+0000 7f36174f2700 1 -- 172.21.15.104:0/1797506114 >> [v2:172.21.15.121:6800/1185793020,v1:172.21.15.121:6801/1185793020] conn(0x7f35f406cc00 msgr2=0x7f35f406f030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.932 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.948+0000 7f36174f2700 1 --2- 172.21.15.104:0/1797506114 >> [v2:172.21.15.121:6800/1185793020,v1:172.21.15.121:6801/1185793020] conn(0x7f35f406cc00 0x7f35f406f030 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.948+0000 7f36174f2700 1 -- 172.21.15.104:0/1797506114 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f35f40662f0 msgr2=0x7f35f40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.948+0000 7f36174f2700 1 --2- 172.21.15.104:0/1797506114 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f35f40662f0 0x7f35f40687a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f360c015040 tx=0x7f360c00a000 comp rx=0 tx=0).stop 2023-07-14T11:57:12.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.948+0000 7f36174f2700 1 -- 172.21.15.104:0/1797506114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3610155850 msgr2=0x7f361015ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:12.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.948+0000 7f36174f2700 1 --2- 172.21.15.104:0/1797506114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3610155850 0x7f361015ed30 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3600011900 tx=0x7f3600011cc0 comp rx=0 tx=0).stop 2023-07-14T11:57:12.933 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.948+0000 7f3616cf1700 1 -- 172.21.15.104:0/1797506114 reap_dead start 2023-07-14T11:57:12.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.948+0000 7f36174f2700 1 -- 172.21.15.104:0/1797506114 shutdown_connections 2023-07-14T11:57:12.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.948+0000 7f36174f2700 1 -- 172.21.15.104:0/1797506114 >> 172.21.15.104:0/1797506114 conn(0x7f3610060c40 msgr2=0x7f36100b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:12.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.948+0000 7f36174f2700 1 -- 172.21.15.104:0/1797506114 shutdown_connections 2023-07-14T11:57:12.934 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:12.948+0000 7f36174f2700 1 -- 172.21.15.104:0/1797506114 wait complete. 2023-07-14T11:57:12.956 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-07-14T11:57:13.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.144+0000 7ff22a4e7700 1 Processor -- start 2023-07-14T11:57:13.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.148+0000 7ff22a4e7700 1 -- start start 2023-07-14T11:57:13.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.148+0000 7ff22a4e7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22415ffa0 0x7ff22406d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.148+0000 7ff22a4e7700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff2240cb180 0x7ff2240ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.148+0000 7ff22a4e7700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2240ca8a0 con 0x7ff22406dc50 2023-07-14T11:57:13.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.148+0000 7ff22a4e7700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2240cbcd0 con 0x7ff2240cb180 2023-07-14T11:57:13.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.148+0000 7ff22a4e7700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2240cbf70 con 0x7ff22415ffa0 2023-07-14T11:57:13.135 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.148+0000 7ff228ce4700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ff22406dc50 0x7ff2240caa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43592/0 (socket says 172.21.15.104:43592) 2023-07-14T11:57:13.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.148+0000 7ff228ce4700 1 -- 172.21.15.104:0/2234205896 learned_addr learned my addr 172.21.15.104:0/2234205896 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:13.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.148+0000 7ff229ce6700 1 --2- 172.21.15.104:0/2234205896 >> v2:172.21.15.104:3300/0 conn(0x7ff2240cb180 0x7ff2240ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff2294e5700 1 --2- 172.21.15.104:0/2234205896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22415ffa0 0x7ff22406d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff21bfff700 1 -- 172.21.15.104:0/2234205896 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1676770358 0 0) 0x7ff214001110 con 0x7ff22406dc50 2023-07-14T11:57:13.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff21bfff700 1 -- 172.21.15.104:0/2234205896 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2100035d0 con 0x7ff22406dc50 2023-07-14T11:57:13.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff21bfff700 1 -- 172.21.15.104:0/2234205896 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4062884072 0 0) 0x7ff2100035d0 con 0x7ff22406dc50 2023-07-14T11:57:13.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff229ce6700 1 -- 172.21.15.104:0/2234205896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22415ffa0 msgr2=0x7ff22406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:13.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff229ce6700 1 --2- 172.21.15.104:0/2234205896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22415ffa0 0x7ff22406d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff229ce6700 1 -- 172.21.15.104:0/2234205896 >> v1:172.21.15.104:6789/0 conn(0x7ff22406dc50 legacy=0x7ff2240caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.138 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff229ce6700 1 -- 172.21.15.104:0/2234205896 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2240cc250 con 0x7ff2240cb180 2023-07-14T11:57:13.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff229ce6700 1 --2- 172.21.15.104:0/2234205896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2240cb180 0x7ff2240ce340 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009d90 tx=0x7ff21c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ce0d25004c6fe2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:13.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff21bfff700 1 -- 172.21.15.104:0/2234205896 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c00e070 con 0x7ff2240cb180 2023-07-14T11:57:13.139 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff21bfff700 1 -- 172.21.15.104:0/2234205896 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c003a40 con 0x7ff2240cb180 2023-07-14T11:57:13.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff21bfff700 1 -- 172.21.15.104:0/2234205896 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214001820 con 0x7ff2240cb180 2023-07-14T11:57:13.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff22a4e7700 1 -- 172.21.15.104:0/2234205896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2240cb180 msgr2=0x7ff2240ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff22a4e7700 1 --2- 172.21.15.104:0/2234205896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2240cb180 0x7ff2240ce340 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009d90 tx=0x7ff21c00b330 comp rx=0 tx=0).stop 2023-07-14T11:57:13.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff22a4e7700 1 -- 172.21.15.104:0/2234205896 shutdown_connections 2023-07-14T11:57:13.140 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff22a4e7700 1 --2- 172.21.15.104:0/2234205896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2240cb180 0x7ff2240ce340 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff22a4e7700 1 --2- 172.21.15.104:0/2234205896 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22415ffa0 0x7ff22406d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff22a4e7700 1 -- 172.21.15.104:0/2234205896 >> 172.21.15.104:0/2234205896 conn(0x7ff22406e960 msgr2=0x7ff22406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:13.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff22a4e7700 1 -- 172.21.15.104:0/2234205896 shutdown_connections 2023-07-14T11:57:13.141 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.152+0000 7ff22a4e7700 1 -- 172.21.15.104:0/2234205896 wait complete. 2023-07-14T11:57:13.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.156+0000 7ff22a4e7700 1 Processor -- start 2023-07-14T11:57:13.142 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.156+0000 7ff22a4e7700 1 -- start start 2023-07-14T11:57:13.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff22a4e7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22406dc50 0x7ff2240c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff22a4e7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2240cb180 0x7ff2240bccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff22a4e7700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22415ffa0 0x7ff22414ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.144 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff22a4e7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22414c650 con 0x7ff2240cb180 2023-07-14T11:57:13.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff22a4e7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22414b9d0 con 0x7ff22406dc50 2023-07-14T11:57:13.145 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff22a4e7700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22414bcb0 con 0x7ff22415ffa0 2023-07-14T11:57:13.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff228ce4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2240cb180 0x7ff2240bccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff228ce4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2240cb180 0x7ff2240bccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44638/0 (socket says 172.21.15.104:44638) 2023-07-14T11:57:13.146 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff228ce4700 1 -- 172.21.15.104:0/3501057486 learned_addr learned my addr 172.21.15.104:0/3501057486 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:13.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff2294e5700 1 --2- 172.21.15.104:0/3501057486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22406dc50 0x7ff2240c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff229ce6700 1 --2- 172.21.15.104:0/3501057486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22415ffa0 0x7ff22414ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff228ce4700 1 -- 172.21.15.104:0/3501057486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22415ffa0 msgr2=0x7ff22414ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff228ce4700 1 --2- 172.21.15.104:0/3501057486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22415ffa0 0x7ff22414ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.147 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff228ce4700 1 -- 172.21.15.104:0/3501057486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22406dc50 msgr2=0x7ff2240c07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff228ce4700 1 --2- 172.21.15.104:0/3501057486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22406dc50 0x7ff2240c07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff228ce4700 1 -- 172.21.15.104:0/3501057486 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c009a70 con 0x7ff2240cb180 2023-07-14T11:57:13.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff228ce4700 1 --2- 172.21.15.104:0/3501057486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2240cb180 0x7ff2240bccd0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff21400c700 tx=0x7ff21400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:13.148 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff219ffb700 1 -- 172.21.15.104:0/3501057486 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214007560 con 0x7ff2240cb180 2023-07-14T11:57:13.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff219ffb700 1 -- 172.21.15.104:0/3501057486 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2140076e0 con 0x7ff2240cb180 2023-07-14T11:57:13.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff22a4e7700 1 -- 172.21.15.104:0/3501057486 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22414bf90 con 0x7ff2240cb180 2023-07-14T11:57:13.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff219ffb700 1 -- 172.21.15.104:0/3501057486 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214004ad0 con 0x7ff2240cb180 2023-07-14T11:57:13.149 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.160+0000 7ff22a4e7700 1 -- 172.21.15.104:0/3501057486 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2240c0d00 con 0x7ff2240cb180 2023-07-14T11:57:13.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.164+0000 7ff22a4e7700 1 -- 172.21.15.104:0/3501057486 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff224071620 con 0x7ff2240cb180 2023-07-14T11:57:13.150 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.164+0000 7ff219ffb700 1 -- 172.21.15.104:0/3501057486 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff214004c30 con 0x7ff2240cb180 2023-07-14T11:57:13.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.168+0000 7ff219ffb700 1 --2- 172.21.15.104:0/3501057486 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2080662f0 0x7ff2080687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.151 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.168+0000 7ff219ffb700 1 -- 172.21.15.104:0/3501057486 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7ff2140836d0 con 0x7ff2240cb180 2023-07-14T11:57:13.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.168+0000 7ff2294e5700 1 --2- 172.21.15.104:0/3501057486 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2080662f0 0x7ff2080687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.152 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.168+0000 7ff2294e5700 1 --2- 172.21.15.104:0/3501057486 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2080662f0 0x7ff2080687a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff220009b80 tx=0x7ff220008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:13.159 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.172+0000 7ff219ffb700 1 -- 172.21.15.104:0/3501057486 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7ff214050d20 con 0x7ff2240cb180 2023-07-14T11:57:13.331 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.348+0000 7ff22a4e7700 1 -- 172.21.15.104:0/3501057486 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2240c0f20 con 0x7ff2080662f0 2023-07-14T11:57:13.333 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.348+0000 7ff219ffb700 1 -- 172.21.15.104:0/3501057486 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff2240c0f20 con 0x7ff2080662f0 2023-07-14T11:57:13.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.348+0000 7ff22a4e7700 1 -- 172.21.15.104:0/3501057486 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2080662f0 msgr2=0x7ff2080687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.348+0000 7ff22a4e7700 1 --2- 172.21.15.104:0/3501057486 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2080662f0 0x7ff2080687a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff220009b80 tx=0x7ff220008040 comp rx=0 tx=0).stop 2023-07-14T11:57:13.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.348+0000 7ff22a4e7700 1 -- 172.21.15.104:0/3501057486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2240cb180 msgr2=0x7ff2240bccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.348+0000 7ff22a4e7700 1 --2- 172.21.15.104:0/3501057486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2240cb180 0x7ff2240bccd0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff21400c700 tx=0x7ff21400cac0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.348+0000 7ff22a4e7700 1 -- 172.21.15.104:0/3501057486 shutdown_connections 2023-07-14T11:57:13.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.348+0000 7ff22a4e7700 1 --2- 172.21.15.104:0/3501057486 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7ff2080662f0 0x7ff2080687a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.348+0000 7ff22a4e7700 1 --2- 172.21.15.104:0/3501057486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2240cb180 0x7ff2240bccd0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.348+0000 7ff22a4e7700 1 --2- 172.21.15.104:0/3501057486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff22415ffa0 0x7ff22414ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.348+0000 7ff22a4e7700 1 --2- 172.21.15.104:0/3501057486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff22406dc50 0x7ff2240c07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.336 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.348+0000 7ff22a4e7700 1 -- 172.21.15.104:0/3501057486 >> 172.21.15.104:0/3501057486 conn(0x7ff22406e960 msgr2=0x7ff2240b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:13.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.348+0000 7ff22a4e7700 1 -- 172.21.15.104:0/3501057486 shutdown_connections 2023-07-14T11:57:13.337 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.348+0000 7ff22a4e7700 1 -- 172.21.15.104:0/3501057486 wait complete. 2023-07-14T11:57:13.337 INFO:teuthology.orchestra.run.smithi104.stderr:instructed osd(s) 5 to deep-scrub 2023-07-14T11:57:13.354 INFO:tasks.ceph:Scrubbing osd.6 2023-07-14T11:57:13.355 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-07-14T11:57:13.541 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.556+0000 7f28fd693700 1 Processor -- start 2023-07-14T11:57:13.542 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.556+0000 7f28fd693700 1 -- start start 2023-07-14T11:57:13.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.560+0000 7f28fd693700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f28f80b0ee0 0x7f28f80ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.545 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.560+0000 7f28fd693700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f28f80d07e0 0x7f28f80af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.560+0000 7f28fd693700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f80cd4e0 con 0x7f28f80cd090 2023-07-14T11:57:13.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.560+0000 7f28fd693700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f80d0c60 con 0x7f28f80d07e0 2023-07-14T11:57:13.546 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.560+0000 7f28fd693700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f80d1a60 con 0x7f28f80b0ee0 2023-07-14T11:57:13.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.560+0000 7f28fce92700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f28f80d07e0 0x7f28f80af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.547 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.560+0000 7f28f77fe700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f28f80cd090 0x7f28f80d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43594/0 (socket says 172.21.15.104:43594) 2023-07-14T11:57:13.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.560+0000 7f28f77fe700 1 -- 172.21.15.104:0/1833978665 learned_addr learned my addr 172.21.15.104:0/1833978665 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:13.548 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.560+0000 7f28f7fff700 1 --2- 172.21.15.104:0/1833978665 >> v2:172.21.15.121:3300/0 conn(0x7f28f80b0ee0 0x7f28f80ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.560+0000 7f28fce92700 1 -- 172.21.15.104:0/1833978665 >> v2:172.21.15.121:3300/0 conn(0x7f28f80b0ee0 msgr2=0x7f28f80ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:13.549 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.560+0000 7f28fce92700 1 --2- 172.21.15.104:0/1833978665 >> v2:172.21.15.121:3300/0 conn(0x7f28f80b0ee0 0x7f28f80ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.560+0000 7f28fce92700 1 -- 172.21.15.104:0/1833978665 >> v1:172.21.15.104:6789/0 conn(0x7f28f80cd090 legacy=0x7f28f80d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.550 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.560+0000 7f28fce92700 1 -- 172.21.15.104:0/1833978665 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f80d1ef0 con 0x7f28f80d07e0 2023-07-14T11:57:13.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.560+0000 7f28f6ffd700 1 -- 172.21.15.104:0/1833978665 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617180773 0 0) 0x7f28ec001110 con 0x7f28f80cd090 2023-07-14T11:57:13.551 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.564+0000 7f28fce92700 1 --2- 172.21.15.104:0/1833978665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28f80d07e0 0x7f28f80af0c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f28f808b380 tx=0x7f28f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c95a600973540bc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:13.556 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.564+0000 7f28f6ffd700 1 -- 172.21.15.104:0/1833978665 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f000e070 con 0x7f28f80d07e0 2023-07-14T11:57:13.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.564+0000 7f28f6ffd700 1 -- 172.21.15.104:0/1833978665 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f000bb20 con 0x7f28f80d07e0 2023-07-14T11:57:13.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.564+0000 7f28f6ffd700 1 -- 172.21.15.104:0/1833978665 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0012430 con 0x7f28f80d07e0 2023-07-14T11:57:13.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.564+0000 7f28fd693700 1 -- 172.21.15.104:0/1833978665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28f80d07e0 msgr2=0x7f28f80af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.564+0000 7f28fd693700 1 --2- 172.21.15.104:0/1833978665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28f80d07e0 0x7f28f80af0c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f28f808b380 tx=0x7f28f000b330 comp rx=0 tx=0).stop 2023-07-14T11:57:13.557 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.564+0000 7f28fd693700 1 -- 172.21.15.104:0/1833978665 shutdown_connections 2023-07-14T11:57:13.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.564+0000 7f28fd693700 1 --2- 172.21.15.104:0/1833978665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28f80d07e0 0x7f28f80af0c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.564+0000 7f28fd693700 1 --2- 172.21.15.104:0/1833978665 >> v2:172.21.15.121:3300/0 conn(0x7f28f80b0ee0 0x7f28f80ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.564+0000 7f28fd693700 1 -- 172.21.15.104:0/1833978665 >> 172.21.15.104:0/1833978665 conn(0x7f28f8060c40 msgr2=0x7f28f809b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:13.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.564+0000 7f28fd693700 1 -- 172.21.15.104:0/1833978665 shutdown_connections 2023-07-14T11:57:13.558 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.564+0000 7f28fd693700 1 -- 172.21.15.104:0/1833978665 wait complete. 2023-07-14T11:57:13.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fd693700 1 Processor -- start 2023-07-14T11:57:13.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fd693700 1 -- start start 2023-07-14T11:57:13.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fd693700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28f80b0ee0 0x7f28f8159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.559 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fd693700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28f80cd090 0x7f28f809c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fd693700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28f80d07e0 0x7f28f809cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fd693700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f809f5e0 con 0x7f28f80d07e0 2023-07-14T11:57:13.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fd693700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f809f8c0 con 0x7f28f80cd090 2023-07-14T11:57:13.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fd693700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f809fba0 con 0x7f28f80b0ee0 2023-07-14T11:57:13.560 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fce92700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28f80d07e0 0x7f28f809cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fce92700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28f80d07e0 0x7f28f809cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44652/0 (socket says 172.21.15.104:44652) 2023-07-14T11:57:13.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fce92700 1 -- 172.21.15.104:0/2352648869 learned_addr learned my addr 172.21.15.104:0/2352648869 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:13.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28f77fe700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28f80cd090 0x7f28f809c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fce92700 1 -- 172.21.15.104:0/2352648869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28f80b0ee0 msgr2=0x7f28f8159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.561 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28f7fff700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28f80b0ee0 0x7f28f8159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fce92700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28f80b0ee0 0x7f28f8159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fce92700 1 -- 172.21.15.104:0/2352648869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28f80cd090 msgr2=0x7f28f809c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fce92700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28f80cd090 0x7f28f809c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28fce92700 1 -- 172.21.15.104:0/2352648869 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f0009a70 con 0x7f28f80d07e0 2023-07-14T11:57:13.562 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28f77fe700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28f80cd090 0x7f28f809c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:57:13.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.568+0000 7f28f7fff700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28f80b0ee0 0x7f28f8159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:57:13.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.572+0000 7f28fce92700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28f80d07e0 0x7f28f809cb90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f28f808adc0 tx=0x7f28f0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:13.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.572+0000 7f28f4ff9700 1 -- 172.21.15.104:0/2352648869 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0012430 con 0x7f28f80d07e0 2023-07-14T11:57:13.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.572+0000 7f28f4ff9700 1 -- 172.21.15.104:0/2352648869 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f0020070 con 0x7f28f80d07e0 2023-07-14T11:57:13.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.572+0000 7f28f4ff9700 1 -- 172.21.15.104:0/2352648869 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0004280 con 0x7f28f80d07e0 2023-07-14T11:57:13.563 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.572+0000 7f28fd693700 1 -- 172.21.15.104:0/2352648869 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f80caad0 con 0x7f28f80d07e0 2023-07-14T11:57:13.564 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.572+0000 7f28fd693700 1 -- 172.21.15.104:0/2352648869 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f809e860 con 0x7f28f80d07e0 2023-07-14T11:57:13.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.572+0000 7f28fd693700 1 -- 172.21.15.104:0/2352648869 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f28f80d1130 con 0x7f28f80d07e0 2023-07-14T11:57:13.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.576+0000 7f28f4ff9700 1 -- 172.21.15.104:0/2352648869 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f28f00043e0 con 0x7f28f80d07e0 2023-07-14T11:57:13.565 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.576+0000 7f28f4ff9700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f28e4066320 0x7f28e40687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.576+0000 7f28f4ff9700 1 -- 172.21.15.104:0/2352648869 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f28f0022070 con 0x7f28f80d07e0 2023-07-14T11:57:13.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.576+0000 7f28f4ff9700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.121:6816/754390529,v1:172.21.15.121:6817/754390529] conn(0x7f28e406cc30 0x7f28e406f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.576+0000 7f28f4ff9700 1 -- 172.21.15.104:0/2352648869 --> [v2:172.21.15.121:6816/754390529,v1:172.21.15.121:6817/754390529] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28e406f730 con 0x7f28e406cc30 2023-07-14T11:57:13.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.576+0000 7f28f4ff9700 1 -- 172.21.15.104:0/2352648869 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28f00860b0 con 0x7f28f80d07e0 2023-07-14T11:57:13.566 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.576+0000 7f28f7fff700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f28e4066320 0x7f28e40687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.576+0000 7f28f77fe700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.121:6816/754390529,v1:172.21.15.121:6817/754390529] conn(0x7f28e406cc30 0x7f28e406f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.576+0000 7f28f7fff700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f28e4066320 0x7f28e40687d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009ad0 tx=0x7f28e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:13.567 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.576+0000 7f28f77fe700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.121:6816/754390529,v1:172.21.15.121:6817/754390529] conn(0x7f28e406cc30 0x7f28e406f060 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:13.568 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.580+0000 7f28f4ff9700 1 -- 172.21.15.104:0/2352648869 <== osd.6 v2:172.21.15.121:6816/754390529 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f28e406f730 con 0x7f28e406cc30 2023-07-14T11:57:13.583 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.600+0000 7f28fd693700 1 -- 172.21.15.104:0/2352648869 --> [v2:172.21.15.121:6816/754390529,v1:172.21.15.121:6817/754390529] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f28f80d0e50 con 0x7f28e406cc30 2023-07-14T11:57:13.584 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.600+0000 7f28f4ff9700 1 -- 172.21.15.104:0/2352648869 <== osd.6 v2:172.21.15.121:6816/754390529 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f28ec001920 con 0x7f28e406cc30 2023-07-14T11:57:13.585 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:57:13.586 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-14T11:57:13.586 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:57:13.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.600+0000 7f28fd693700 1 -- 172.21.15.104:0/2352648869 >> [v2:172.21.15.121:6816/754390529,v1:172.21.15.121:6817/754390529] conn(0x7f28e406cc30 msgr2=0x7f28e406f060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.588 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.600+0000 7f28fd693700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.121:6816/754390529,v1:172.21.15.121:6817/754390529] conn(0x7f28e406cc30 0x7f28e406f060 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.600+0000 7f28fd693700 1 -- 172.21.15.104:0/2352648869 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f28e4066320 msgr2=0x7f28e40687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.600+0000 7f28fd693700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f28e4066320 0x7f28e40687d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009ad0 tx=0x7f28e8008040 comp rx=0 tx=0).stop 2023-07-14T11:57:13.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.600+0000 7f28fd693700 1 -- 172.21.15.104:0/2352648869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28f80d07e0 msgr2=0x7f28f809cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.600+0000 7f28fd693700 1 --2- 172.21.15.104:0/2352648869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28f80d07e0 0x7f28f809cb90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f28f808adc0 tx=0x7f28f0004af0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.589 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.600+0000 7f28fce92700 1 -- 172.21.15.104:0/2352648869 reap_dead start 2023-07-14T11:57:13.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.600+0000 7f28fd693700 1 -- 172.21.15.104:0/2352648869 shutdown_connections 2023-07-14T11:57:13.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.600+0000 7f28fd693700 1 -- 172.21.15.104:0/2352648869 >> 172.21.15.104:0/2352648869 conn(0x7f28f8060c40 msgr2=0x7f28f80b59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:13.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.600+0000 7f28fd693700 1 -- 172.21.15.104:0/2352648869 shutdown_connections 2023-07-14T11:57:13.590 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.600+0000 7f28fd693700 1 -- 172.21.15.104:0/2352648869 wait complete. 2023-07-14T11:57:13.607 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-07-14T11:57:13.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.804+0000 7fc0f4919700 1 Processor -- start 2023-07-14T11:57:13.789 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.804+0000 7fc0f4919700 1 -- start start 2023-07-14T11:57:13.790 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.804+0000 7fc0f4919700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc0f015ffa0 0x7fc0f006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.804+0000 7fc0f4919700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0f00cb180 0x7fc0f00ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.804+0000 7fc0f4919700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f00ca8a0 con 0x7fc0f006dc50 2023-07-14T11:57:13.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.804+0000 7fc0f4919700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f00cbcd0 con 0x7fc0f015ffa0 2023-07-14T11:57:13.791 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.804+0000 7fc0f4919700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f00cbf70 con 0x7fc0f00cb180 2023-07-14T11:57:13.792 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0eed9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc0f015ffa0 0x7fc0f006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0ef59e700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0f00cb180 0x7fc0f00ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0eed9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc0f015ffa0 0x7fc0f006d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:58158/0 (socket says 172.21.15.104:58158) 2023-07-14T11:57:13.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0eed9d700 1 -- 172.21.15.104:0/3482245844 learned_addr learned my addr 172.21.15.104:0/3482245844 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:13.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0eed9d700 1 -- 172.21.15.104:0/3482245844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0f00cb180 msgr2=0x7fc0f00ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:13.793 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0eed9d700 1 --2- 172.21.15.104:0/3482245844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0f00cb180 0x7fc0f00ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0eed9d700 1 -- 172.21.15.104:0/3482245844 >> v1:172.21.15.121:6789/0 conn(0x7fc0f006dc50 legacy=0x7fc0f00caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0eed9d700 1 -- 172.21.15.104:0/3482245844 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f00cc250 con 0x7fc0f015ffa0 2023-07-14T11:57:13.794 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0ee59c700 1 -- 172.21.15.104:0/3482245844 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106461974 0 0) 0x7fc0d8001490 con 0x7fc0f006dc50 2023-07-14T11:57:13.796 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0eed9d700 1 --2- 172.21.15.104:0/3482245844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0f015ffa0 0x7fc0f006d710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009910 tx=0x7fc0e0009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1ae28c85b38d4d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:13.797 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0ee59c700 1 -- 172.21.15.104:0/3482245844 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0011830 con 0x7fc0f015ffa0 2023-07-14T11:57:13.798 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0ee59c700 1 -- 172.21.15.104:0/3482245844 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e0011990 con 0x7fc0f015ffa0 2023-07-14T11:57:13.799 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0ee59c700 1 -- 172.21.15.104:0/3482245844 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0011d40 con 0x7fc0f015ffa0 2023-07-14T11:57:13.799 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0f4919700 1 -- 172.21.15.104:0/3482245844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0f015ffa0 msgr2=0x7fc0f006d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0f4919700 1 --2- 172.21.15.104:0/3482245844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0f015ffa0 0x7fc0f006d710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009910 tx=0x7fc0e0009c20 comp rx=0 tx=0).stop 2023-07-14T11:57:13.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0f4919700 1 -- 172.21.15.104:0/3482245844 shutdown_connections 2023-07-14T11:57:13.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0f4919700 1 --2- 172.21.15.104:0/3482245844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0f00cb180 0x7fc0f00ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0f4919700 1 --2- 172.21.15.104:0/3482245844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0f015ffa0 0x7fc0f006d710 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.800 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0f4919700 1 -- 172.21.15.104:0/3482245844 >> 172.21.15.104:0/3482245844 conn(0x7fc0f006e960 msgr2=0x7fc0f006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:13.801 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0f4919700 1 -- 172.21.15.104:0/3482245844 shutdown_connections 2023-07-14T11:57:13.801 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.808+0000 7fc0f4919700 1 -- 172.21.15.104:0/3482245844 wait complete. 2023-07-14T11:57:13.802 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.812+0000 7fc0f4919700 1 Processor -- start 2023-07-14T11:57:13.803 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.812+0000 7fc0f4919700 1 -- start start 2023-07-14T11:57:13.803 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0f4919700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0f006dc50 0x7fc0f015e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.803 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0f4919700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0f00cb180 0x7fc0f00c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.803 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0f4919700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0f015ffa0 0x7fc0f0161bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0f4919700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f00c19b0 con 0x7fc0f015ffa0 2023-07-14T11:57:13.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0f4919700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f00c2530 con 0x7fc0f006dc50 2023-07-14T11:57:13.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0f4919700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f015ec20 con 0x7fc0f00cb180 2023-07-14T11:57:13.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0ef59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0f015ffa0 0x7fc0f0161bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.804 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0ef59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0f015ffa0 0x7fc0f0161bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44658/0 (socket says 172.21.15.104:44658) 2023-07-14T11:57:13.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0eed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0f006dc50 0x7fc0f015e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0e659c700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0f00cb180 0x7fc0f00c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0ef59e700 1 -- 172.21.15.104:0/2972769088 learned_addr learned my addr 172.21.15.104:0/2972769088 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:13.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0ef59e700 1 -- 172.21.15.104:0/2972769088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0f00cb180 msgr2=0x7fc0f00c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.805 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0ef59e700 1 --2- 172.21.15.104:0/2972769088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0f00cb180 0x7fc0f00c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.806 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0ef59e700 1 -- 172.21.15.104:0/2972769088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0f006dc50 msgr2=0x7fc0f015e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.806 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0ef59e700 1 --2- 172.21.15.104:0/2972769088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0f006dc50 0x7fc0f015e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.806 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0ef59e700 1 -- 172.21.15.104:0/2972769088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e0009570 con 0x7fc0f015ffa0 2023-07-14T11:57:13.806 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0ef59e700 1 --2- 172.21.15.104:0/2972769088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0f015ffa0 0x7fc0f0161bc0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009fd0 tx=0x7fc0e800ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:13.806 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0e7fff700 1 -- 172.21.15.104:0/2972769088 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e80096e0 con 0x7fc0f015ffa0 2023-07-14T11:57:13.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0e7fff700 1 -- 172.21.15.104:0/2972769088 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e8013680 con 0x7fc0f015ffa0 2023-07-14T11:57:13.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0f4919700 1 -- 172.21.15.104:0/2972769088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f00c1c00 con 0x7fc0f015ffa0 2023-07-14T11:57:13.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0e7fff700 1 -- 172.21.15.104:0/2972769088 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8013a10 con 0x7fc0f015ffa0 2023-07-14T11:57:13.807 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.816+0000 7fc0f4919700 1 -- 172.21.15.104:0/2972769088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f00c21c0 con 0x7fc0f015ffa0 2023-07-14T11:57:13.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.820+0000 7fc0f4919700 1 -- 172.21.15.104:0/2972769088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f00c26a0 con 0x7fc0f015ffa0 2023-07-14T11:57:13.808 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.820+0000 7fc0e7fff700 1 -- 172.21.15.104:0/2972769088 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc0e8013b70 con 0x7fc0f015ffa0 2023-07-14T11:57:13.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.820+0000 7fc0e7fff700 1 --2- 172.21.15.104:0/2972769088 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc0d0066320 0x7fc0d00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:13.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.820+0000 7fc0e7fff700 1 -- 172.21.15.104:0/2972769088 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fc0e8083ab0 con 0x7fc0f015ffa0 2023-07-14T11:57:13.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.820+0000 7fc0eed9d700 1 --2- 172.21.15.104:0/2972769088 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc0d0066320 0x7fc0d00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:13.809 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.824+0000 7fc0eed9d700 1 --2- 172.21.15.104:0/2972769088 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc0d0066320 0x7fc0d00687d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0000c00 tx=0x7fc0e0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:13.813 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:13.828+0000 7fc0e7fff700 1 -- 172.21.15.104:0/2972769088 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fc0e8054b10 con 0x7fc0f015ffa0 2023-07-14T11:57:13.984 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.000+0000 7fc0f4919700 1 -- 172.21.15.104:0/2972769088 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0f00cc810 con 0x7fc0d0066320 2023-07-14T11:57:13.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.000+0000 7fc0e7fff700 1 -- 172.21.15.104:0/2972769088 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc0e0011c00 con 0x7fc0d0066320 2023-07-14T11:57:13.987 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.004+0000 7fc0f4919700 1 -- 172.21.15.104:0/2972769088 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc0d0066320 msgr2=0x7fc0d00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.987 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.004+0000 7fc0f4919700 1 --2- 172.21.15.104:0/2972769088 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc0d0066320 0x7fc0d00687d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0000c00 tx=0x7fc0e0008030 comp rx=0 tx=0).stop 2023-07-14T11:57:13.987 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.004+0000 7fc0f4919700 1 -- 172.21.15.104:0/2972769088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0f015ffa0 msgr2=0x7fc0f0161bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:13.988 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.004+0000 7fc0f4919700 1 --2- 172.21.15.104:0/2972769088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0f015ffa0 0x7fc0f0161bc0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009fd0 tx=0x7fc0e800ebe0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.988 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.004+0000 7fc0f4919700 1 -- 172.21.15.104:0/2972769088 shutdown_connections 2023-07-14T11:57:13.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.004+0000 7fc0f4919700 1 --2- 172.21.15.104:0/2972769088 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc0d0066320 0x7fc0d00687d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.004+0000 7fc0f4919700 1 --2- 172.21.15.104:0/2972769088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0f015ffa0 0x7fc0f0161bc0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.004+0000 7fc0f4919700 1 --2- 172.21.15.104:0/2972769088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0f00cb180 0x7fc0f00c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.004+0000 7fc0f4919700 1 --2- 172.21.15.104:0/2972769088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0f006dc50 0x7fc0f015e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:13.990 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.004+0000 7fc0f4919700 1 -- 172.21.15.104:0/2972769088 >> 172.21.15.104:0/2972769088 conn(0x7fc0f006e960 msgr2=0x7fc0f006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:13.990 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.004+0000 7fc0f4919700 1 -- 172.21.15.104:0/2972769088 shutdown_connections 2023-07-14T11:57:13.990 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.004+0000 7fc0f4919700 1 -- 172.21.15.104:0/2972769088 wait complete. 2023-07-14T11:57:13.990 INFO:teuthology.orchestra.run.smithi104.stderr:instructed osd(s) 6 to deep-scrub 2023-07-14T11:57:14.008 INFO:tasks.ceph:Scrubbing osd.7 2023-07-14T11:57:14.009 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-07-14T11:57:14.187 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.200+0000 7fc9a55dc700 1 Processor -- start 2023-07-14T11:57:14.188 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.200+0000 7fc9a55dc700 1 -- start start 2023-07-14T11:57:14.189 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc9a55dc700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc9a01597a0 0x7fc9a00c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc9a55dc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9a00705e0 0x7fc9a00cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc9a55dc700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a0071560 con 0x7fc9a0159b70 2023-07-14T11:57:14.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc9a55dc700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a0070a30 con 0x7fc9a01597a0 2023-07-14T11:57:14.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc9a55dc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a0070cc0 con 0x7fc9a00705e0 2023-07-14T11:57:14.190 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc99effd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc9a01597a0 0x7fc9a00c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc99effd700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc9a01597a0 0x7fc9a00c87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44660/0 (socket says 172.21.15.104:44660) 2023-07-14T11:57:14.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc99effd700 1 -- 172.21.15.104:0/4081212606 learned_addr learned my addr 172.21.15.104:0/4081212606 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:14.191 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc99f7fe700 1 --2- 172.21.15.104:0/4081212606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9a00705e0 0x7fc9a00cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.192 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc99effd700 1 -- 172.21.15.104:0/4081212606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9a00705e0 msgr2=0x7fc9a00cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:14.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc99effd700 1 --2- 172.21.15.104:0/4081212606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9a00705e0 0x7fc9a00cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc99effd700 1 -- 172.21.15.104:0/4081212606 >> v1:172.21.15.121:6789/0 conn(0x7fc9a0159b70 legacy=0x7fc9a00cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:14.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc99effd700 1 -- 172.21.15.104:0/4081212606 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a0070fa0 con 0x7fc9a01597a0 2023-07-14T11:57:14.193 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc99effd700 1 --2- 172.21.15.104:0/4081212606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a01597a0 0x7fc9a00c87a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc990009910 tx=0x7fc990009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b53ce3e48f5f8c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:14.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc99dffb700 1 -- 172.21.15.104:0/4081212606 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99000ba60 con 0x7fc9a01597a0 2023-07-14T11:57:14.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.204+0000 7fc99dffb700 1 -- 172.21.15.104:0/4081212606 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99000bbc0 con 0x7fc9a01597a0 2023-07-14T11:57:14.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.208+0000 7fc99dffb700 1 -- 172.21.15.104:0/4081212606 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc990009e60 con 0x7fc9a01597a0 2023-07-14T11:57:14.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.208+0000 7fc9a55dc700 1 -- 172.21.15.104:0/4081212606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a01597a0 msgr2=0x7fc9a00c87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.194 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.208+0000 7fc9a55dc700 1 --2- 172.21.15.104:0/4081212606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a01597a0 0x7fc9a00c87a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc990009910 tx=0x7fc990009c20 comp rx=0 tx=0).stop 2023-07-14T11:57:14.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.208+0000 7fc9a55dc700 1 -- 172.21.15.104:0/4081212606 shutdown_connections 2023-07-14T11:57:14.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.208+0000 7fc9a55dc700 1 --2- 172.21.15.104:0/4081212606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9a00705e0 0x7fc9a00cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.208+0000 7fc9a55dc700 1 --2- 172.21.15.104:0/4081212606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a01597a0 0x7fc9a00c87a0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.195 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.208+0000 7fc9a55dc700 1 -- 172.21.15.104:0/4081212606 >> 172.21.15.104:0/4081212606 conn(0x7fc9a00afc60 msgr2=0x7fc9a00b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:14.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.208+0000 7fc9a55dc700 1 -- 172.21.15.104:0/4081212606 shutdown_connections 2023-07-14T11:57:14.196 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.208+0000 7fc9a55dc700 1 -- 172.21.15.104:0/4081212606 wait complete. 2023-07-14T11:57:14.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.208+0000 7fc9a55dc700 1 Processor -- start 2023-07-14T11:57:14.197 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc9a55dc700 1 -- start start 2023-07-14T11:57:14.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc9a55dc700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9a00705e0 0x7fc9a00ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc9a55dc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a01597a0 0x7fc9a00ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc9a55dc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9a0159b70 0x7fc9a015d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc9a55dc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a015f290 con 0x7fc9a01597a0 2023-07-14T11:57:14.198 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc9a55dc700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a00af2a0 con 0x7fc9a0159b70 2023-07-14T11:57:14.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc9a55dc700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a00af580 con 0x7fc9a00705e0 2023-07-14T11:57:14.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc99e7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a01597a0 0x7fc9a00ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc99e7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a01597a0 0x7fc9a00ace80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44670/0 (socket says 172.21.15.104:44670) 2023-07-14T11:57:14.199 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc99e7fc700 1 -- 172.21.15.104:0/76874196 learned_addr learned my addr 172.21.15.104:0/76874196 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:14.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc99effd700 1 --2- 172.21.15.104:0/76874196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9a00705e0 0x7fc9a00ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.200 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc99f7fe700 1 --2- 172.21.15.104:0/76874196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9a0159b70 0x7fc9a015d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc99e7fc700 1 -- 172.21.15.104:0/76874196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9a00705e0 msgr2=0x7fc9a00ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc99e7fc700 1 --2- 172.21.15.104:0/76874196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9a00705e0 0x7fc9a00ac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.201 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc99e7fc700 1 -- 172.21.15.104:0/76874196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9a0159b70 msgr2=0x7fc9a015d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc99e7fc700 1 --2- 172.21.15.104:0/76874196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9a0159b70 0x7fc9a015d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc99e7fc700 1 -- 172.21.15.104:0/76874196 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc990009570 con 0x7fc9a01597a0 2023-07-14T11:57:14.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc99f7fe700 1 --2- 172.21.15.104:0/76874196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9a0159b70 0x7fc9a015d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:57:14.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc99e7fc700 1 --2- 172.21.15.104:0/76874196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a01597a0 0x7fc9a00ace80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc99400c500 tx=0x7fc99400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:14.202 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc9877fe700 1 -- 172.21.15.104:0/76874196 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9940052d0 con 0x7fc9a01597a0 2023-07-14T11:57:14.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc9877fe700 1 -- 172.21.15.104:0/76874196 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc994005430 con 0x7fc9a01597a0 2023-07-14T11:57:14.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc9a55dc700 1 -- 172.21.15.104:0/76874196 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a00af860 con 0x7fc9a01597a0 2023-07-14T11:57:14.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc9877fe700 1 -- 172.21.15.104:0/76874196 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9940055a0 con 0x7fc9a01597a0 2023-07-14T11:57:14.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.212+0000 7fc9a55dc700 1 -- 172.21.15.104:0/76874196 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a00b1ae0 con 0x7fc9a01597a0 2023-07-14T11:57:14.203 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.216+0000 7fc9a55dc700 1 -- 172.21.15.104:0/76874196 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc9a00af410 con 0x7fc9a01597a0 2023-07-14T11:57:14.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.216+0000 7fc9877fe700 1 -- 172.21.15.104:0/76874196 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc9940038e0 con 0x7fc9a01597a0 2023-07-14T11:57:14.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.220+0000 7fc9877fe700 1 --2- 172.21.15.104:0/76874196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc98c0662f0 0x7fc98c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.220+0000 7fc9877fe700 1 -- 172.21.15.104:0/76874196 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fc994082fb0 con 0x7fc9a01597a0 2023-07-14T11:57:14.205 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.220+0000 7fc99effd700 1 --2- 172.21.15.104:0/76874196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc98c0662f0 0x7fc98c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.220+0000 7fc9877fe700 1 --2- 172.21.15.104:0/76874196 >> [v2:172.21.15.121:6808/2610729951,v1:172.21.15.121:6809/2610729951] conn(0x7fc98c06cc00 0x7fc98c06f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.220+0000 7fc9877fe700 1 -- 172.21.15.104:0/76874196 --> [v2:172.21.15.121:6808/2610729951,v1:172.21.15.121:6809/2610729951] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc98c06f700 con 0x7fc98c06cc00 2023-07-14T11:57:14.206 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.220+0000 7fc9877fe700 1 -- 172.21.15.104:0/76874196 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc994085590 con 0x7fc9a01597a0 2023-07-14T11:57:14.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.220+0000 7fc99f7fe700 1 --2- 172.21.15.104:0/76874196 >> [v2:172.21.15.121:6808/2610729951,v1:172.21.15.121:6809/2610729951] conn(0x7fc98c06cc00 0x7fc98c06f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.220+0000 7fc99effd700 1 --2- 172.21.15.104:0/76874196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc98c0662f0 0x7fc98c0687a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc990000c00 tx=0x7fc990008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:14.207 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.220+0000 7fc99f7fe700 1 --2- 172.21.15.104:0/76874196 >> [v2:172.21.15.121:6808/2610729951,v1:172.21.15.121:6809/2610729951] conn(0x7fc98c06cc00 0x7fc98c06f030 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:14.208 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.224+0000 7fc9877fe700 1 -- 172.21.15.104:0/76874196 <== osd.7 v2:172.21.15.121:6808/2610729951 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc98c06f700 con 0x7fc98c06cc00 2023-07-14T11:57:14.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.240+0000 7fc9a55dc700 1 -- 172.21.15.104:0/76874196 --> [v2:172.21.15.121:6808/2610729951,v1:172.21.15.121:6809/2610729951] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc9a00ca880 con 0x7fc98c06cc00 2023-07-14T11:57:14.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.240+0000 7fc9877fe700 1 -- 172.21.15.104:0/76874196 <== osd.7 v2:172.21.15.121:6808/2610729951 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc998009b90 con 0x7fc98c06cc00 2023-07-14T11:57:14.226 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-14T11:57:14.226 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-14T11:57:14.227 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-14T11:57:14.227 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.240+0000 7fc9a55dc700 1 -- 172.21.15.104:0/76874196 >> [v2:172.21.15.121:6808/2610729951,v1:172.21.15.121:6809/2610729951] conn(0x7fc98c06cc00 msgr2=0x7fc98c06f030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.240+0000 7fc9a55dc700 1 --2- 172.21.15.104:0/76874196 >> [v2:172.21.15.121:6808/2610729951,v1:172.21.15.121:6809/2610729951] conn(0x7fc98c06cc00 0x7fc98c06f030 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.240+0000 7fc9a55dc700 1 -- 172.21.15.104:0/76874196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc98c0662f0 msgr2=0x7fc98c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.240+0000 7fc9a55dc700 1 --2- 172.21.15.104:0/76874196 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fc98c0662f0 0x7fc98c0687a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc990000c00 tx=0x7fc990008030 comp rx=0 tx=0).stop 2023-07-14T11:57:14.228 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.240+0000 7fc9a55dc700 1 -- 172.21.15.104:0/76874196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a01597a0 msgr2=0x7fc9a00ace80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.240+0000 7fc9a55dc700 1 --2- 172.21.15.104:0/76874196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a01597a0 0x7fc9a00ace80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc99400c500 tx=0x7fc99400c810 comp rx=0 tx=0).stop 2023-07-14T11:57:14.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.240+0000 7fc99f7fe700 1 -- 172.21.15.104:0/76874196 reap_dead start 2023-07-14T11:57:14.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.240+0000 7fc9a55dc700 1 -- 172.21.15.104:0/76874196 shutdown_connections 2023-07-14T11:57:14.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.240+0000 7fc9a55dc700 1 -- 172.21.15.104:0/76874196 >> 172.21.15.104:0/76874196 conn(0x7fc9a00afc60 msgr2=0x7fc9a015a910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:14.229 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.240+0000 7fc9a55dc700 1 -- 172.21.15.104:0/76874196 shutdown_connections 2023-07-14T11:57:14.230 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.240+0000 7fc9a55dc700 1 -- 172.21.15.104:0/76874196 wait complete. 2023-07-14T11:57:14.251 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-07-14T11:57:14.429 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.444+0000 7fdde68c6700 1 Processor -- start 2023-07-14T11:57:14.430 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.444+0000 7fdde68c6700 1 -- start start 2023-07-14T11:57:14.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdde68c6700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdde015c070 0x7fdde00ad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdde68c6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdde00b0440 0x7fdde00703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.432 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdde68c6700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde00cd530 con 0x7fdde00ad8b0 2023-07-14T11:57:14.433 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdde68c6700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde015c440 con 0x7fdde015c070 2023-07-14T11:57:14.434 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdde68c6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde00cc1b0 con 0x7fdde00b0440 2023-07-14T11:57:14.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fddd7fff700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fdde00ad8b0 0x7fdde00afd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43606/0 (socket says 172.21.15.104:43606) 2023-07-14T11:57:14.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fddd7fff700 1 -- 172.21.15.104:0/1526169204 learned_addr learned my addr 172.21.15.104:0/1526169204 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:14.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdddffff700 1 --2- 172.21.15.104:0/1526169204 >> v2:172.21.15.104:3300/0 conn(0x7fdde015c070 0x7fdde00ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdde4e63700 1 --2- 172.21.15.104:0/1526169204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdde00b0440 0x7fdde00703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdddf7fe700 1 -- 172.21.15.104:0/1526169204 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873964710 0 0) 0x7fdde00cd530 con 0x7fdde00ad8b0 2023-07-14T11:57:14.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdddf7fe700 1 -- 172.21.15.104:0/1526169204 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddcc0035d0 con 0x7fdde00ad8b0 2023-07-14T11:57:14.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdddffff700 1 -- 172.21.15.104:0/1526169204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdde00b0440 msgr2=0x7fdde00703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:14.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdddffff700 1 --2- 172.21.15.104:0/1526169204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdde00b0440 0x7fdde00703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdddffff700 1 -- 172.21.15.104:0/1526169204 >> v1:172.21.15.104:6789/0 conn(0x7fdde00ad8b0 legacy=0x7fdde00afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdddffff700 1 -- 172.21.15.104:0/1526169204 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde00cc490 con 0x7fdde015c070 2023-07-14T11:57:14.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdddffff700 1 --2- 172.21.15.104:0/1526169204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde015c070 0x7fdde00ad370 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009850 tx=0x7fddd0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821ba5ef41e99516 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:14.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdddf7fe700 1 -- 172.21.15.104:0/1526169204 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0011830 con 0x7fdde015c070 2023-07-14T11:57:14.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdddf7fe700 1 -- 172.21.15.104:0/1526169204 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd0011990 con 0x7fdde015c070 2023-07-14T11:57:14.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.448+0000 7fdddf7fe700 1 -- 172.21.15.104:0/1526169204 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0011d40 con 0x7fdde015c070 2023-07-14T11:57:14.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.452+0000 7fdde68c6700 1 -- 172.21.15.104:0/1526169204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde015c070 msgr2=0x7fdde00ad370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.452+0000 7fdde68c6700 1 --2- 172.21.15.104:0/1526169204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde015c070 0x7fdde00ad370 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009850 tx=0x7fddd0009b60 comp rx=0 tx=0).stop 2023-07-14T11:57:14.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.452+0000 7fdde68c6700 1 -- 172.21.15.104:0/1526169204 shutdown_connections 2023-07-14T11:57:14.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.452+0000 7fdde68c6700 1 --2- 172.21.15.104:0/1526169204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdde00b0440 0x7fdde00703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.452+0000 7fdde68c6700 1 --2- 172.21.15.104:0/1526169204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde015c070 0x7fdde00ad370 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.452+0000 7fdde68c6700 1 -- 172.21.15.104:0/1526169204 >> 172.21.15.104:0/1526169204 conn(0x7fdde00cad30 msgr2=0x7fdde00cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:14.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.452+0000 7fdde68c6700 1 -- 172.21.15.104:0/1526169204 shutdown_connections 2023-07-14T11:57:14.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.452+0000 7fdde68c6700 1 -- 172.21.15.104:0/1526169204 wait complete. 2023-07-14T11:57:14.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.452+0000 7fdde68c6700 1 Processor -- start 2023-07-14T11:57:14.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.452+0000 7fdde68c6700 1 -- start start 2023-07-14T11:57:14.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde68c6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdde00ad8b0 0x7fdde0149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde68c6700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdde00b0440 0x7fdde0145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde68c6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde015c070 0x7fdde01f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde68c6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde0146320 con 0x7fdde015c070 2023-07-14T11:57:14.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde68c6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde01f2470 con 0x7fdde00ad8b0 2023-07-14T11:57:14.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde68c6700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde01f5760 con 0x7fdde00b0440 2023-07-14T11:57:14.443 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde4e63700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde015c070 0x7fdde01f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde4e63700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde015c070 0x7fdde01f1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44686/0 (socket says 172.21.15.104:44686) 2023-07-14T11:57:14.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde4e63700 1 -- 172.21.15.104:0/3390716616 learned_addr learned my addr 172.21.15.104:0/3390716616 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:14.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fddd7fff700 1 --2- 172.21.15.104:0/3390716616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdde00b0440 0x7fdde0145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.444 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdddffff700 1 --2- 172.21.15.104:0/3390716616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdde00ad8b0 0x7fdde0149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde4e63700 1 -- 172.21.15.104:0/3390716616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdde00b0440 msgr2=0x7fdde0145de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde4e63700 1 --2- 172.21.15.104:0/3390716616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdde00b0440 0x7fdde0145de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde4e63700 1 -- 172.21.15.104:0/3390716616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdde00ad8b0 msgr2=0x7fdde0149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde4e63700 1 --2- 172.21.15.104:0/3390716616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdde00ad8b0 0x7fdde0149190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.445 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde4e63700 1 -- 172.21.15.104:0/3390716616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd00094b0 con 0x7fdde015c070 2023-07-14T11:57:14.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde4e63700 1 --2- 172.21.15.104:0/3390716616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde015c070 0x7fdde01f1f30 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fddd800ef90 tx=0x7fddd800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:14.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdddd7fa700 1 -- 172.21.15.104:0/3390716616 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8009ca0 con 0x7fdde015c070 2023-07-14T11:57:14.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdddd7fa700 1 -- 172.21.15.104:0/3390716616 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd800f040 con 0x7fdde015c070 2023-07-14T11:57:14.446 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde68c6700 1 -- 172.21.15.104:0/3390716616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde0158850 con 0x7fdde015c070 2023-07-14T11:57:14.447 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdddd7fa700 1 -- 172.21.15.104:0/3390716616 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8014480 con 0x7fdde015c070 2023-07-14T11:57:14.447 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.456+0000 7fdde68c6700 1 -- 172.21.15.104:0/3390716616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde01486b0 con 0x7fdde015c070 2023-07-14T11:57:14.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.460+0000 7fdde68c6700 1 -- 172.21.15.104:0/3390716616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde004dcb0 con 0x7fdde015c070 2023-07-14T11:57:14.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.460+0000 7fdddd7fa700 1 -- 172.21.15.104:0/3390716616 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fddd80145e0 con 0x7fdde015c070 2023-07-14T11:57:14.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.460+0000 7fdddd7fa700 1 --2- 172.21.15.104:0/3390716616 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fddc0066320 0x7fddc00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.448 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.460+0000 7fdddd7fa700 1 -- 172.21.15.104:0/3390716616 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fddd8084410 con 0x7fdde015c070 2023-07-14T11:57:14.449 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.460+0000 7fdddffff700 1 --2- 172.21.15.104:0/3390716616 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fddc0066320 0x7fddc00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.449 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.464+0000 7fdddffff700 1 --2- 172.21.15.104:0/3390716616 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fddc0066320 0x7fddc00687d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fddd00090b0 tx=0x7fddd0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:14.453 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.468+0000 7fdddd7fa700 1 -- 172.21.15.104:0/3390716616 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fddd8055570 con 0x7fdde015c070 2023-07-14T11:57:14.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.640+0000 7fdde68c6700 1 -- 172.21.15.104:0/3390716616 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fdde00cca50 con 0x7fddc0066320 2023-07-14T11:57:14.625 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.640+0000 7fdddd7fa700 1 -- 172.21.15.104:0/3390716616 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fddd0006c80 con 0x7fddc0066320 2023-07-14T11:57:14.628 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.644+0000 7fdde68c6700 1 -- 172.21.15.104:0/3390716616 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fddc0066320 msgr2=0x7fddc00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.644+0000 7fdde68c6700 1 --2- 172.21.15.104:0/3390716616 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fddc0066320 0x7fddc00687d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fddd00090b0 tx=0x7fddd0008030 comp rx=0 tx=0).stop 2023-07-14T11:57:14.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.644+0000 7fdde68c6700 1 -- 172.21.15.104:0/3390716616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde015c070 msgr2=0x7fdde01f1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.644+0000 7fdde68c6700 1 --2- 172.21.15.104:0/3390716616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde015c070 0x7fdde01f1f30 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fddd800ef90 tx=0x7fddd800c3a0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.644+0000 7fdde68c6700 1 -- 172.21.15.104:0/3390716616 shutdown_connections 2023-07-14T11:57:14.629 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.644+0000 7fdde68c6700 1 --2- 172.21.15.104:0/3390716616 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fddc0066320 0x7fddc00687d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.644+0000 7fdde68c6700 1 --2- 172.21.15.104:0/3390716616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdde015c070 0x7fdde01f1f30 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.644+0000 7fdde68c6700 1 --2- 172.21.15.104:0/3390716616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdde00b0440 0x7fdde0145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.644+0000 7fdde68c6700 1 --2- 172.21.15.104:0/3390716616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdde00ad8b0 0x7fdde0149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.644+0000 7fdde68c6700 1 -- 172.21.15.104:0/3390716616 >> 172.21.15.104:0/3390716616 conn(0x7fdde00cad30 msgr2=0x7fdde00c9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:14.630 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.644+0000 7fdde68c6700 1 -- 172.21.15.104:0/3390716616 shutdown_connections 2023-07-14T11:57:14.631 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.644+0000 7fdde68c6700 1 -- 172.21.15.104:0/3390716616 wait complete. 2023-07-14T11:57:14.631 INFO:teuthology.orchestra.run.smithi104.stderr:instructed osd(s) 7 to deep-scrub 2023-07-14T11:57:14.649 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:57:14.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.840+0000 7f655d600700 1 Processor -- start 2023-07-14T11:57:14.827 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f655d600700 1 -- start start 2023-07-14T11:57:14.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f655d600700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6558071000 0x7f655815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f655d600700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f65580caa20 0x7f65580cae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f655d600700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655815d230 con 0x7f65581599d0 2023-07-14T11:57:14.829 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f655d600700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65580cde10 con 0x7f6558071000 2023-07-14T11:57:14.830 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f655d600700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65580cb380 con 0x7f65580caa20 2023-07-14T11:57:14.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f654ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6558071000 0x7f655815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f654ffff700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6558071000 0x7f655815ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44700/0 (socket says 172.21.15.104:44700) 2023-07-14T11:57:14.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f654ffff700 1 -- 172.21.15.104:0/1169786327 learned_addr learned my addr 172.21.15.104:0/1169786327 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:14.831 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f65577fe700 1 --2- 172.21.15.104:0/1169786327 >> v2:172.21.15.121:3300/0 conn(0x7f65580caa20 0x7f65580cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f654ffff700 1 -- 172.21.15.104:0/1169786327 >> v2:172.21.15.121:3300/0 conn(0x7f65580caa20 msgr2=0x7f65580cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:14.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f654ffff700 1 --2- 172.21.15.104:0/1169786327 >> v2:172.21.15.121:3300/0 conn(0x7f65580caa20 0x7f65580cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f654ffff700 1 -- 172.21.15.104:0/1169786327 >> v1:172.21.15.121:6789/0 conn(0x7f65581599d0 legacy=0x7f65580c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f654ffff700 1 -- 172.21.15.104:0/1169786327 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65580cd520 con 0x7f6558071000 2023-07-14T11:57:14.832 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f654ffff700 1 --2- 172.21.15.104:0/1169786327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6558071000 0x7f655815ccf0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f65580ca100 tx=0x7f6540009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4acfb87d6b74e01d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:14.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f65567fc700 1 -- 172.21.15.104:0/1169786327 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654000b8a0 con 0x7f6558071000 2023-07-14T11:57:14.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f65567fc700 1 -- 172.21.15.104:0/1169786327 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654000ba00 con 0x7f6558071000 2023-07-14T11:57:14.833 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.844+0000 7f65567fc700 1 -- 172.21.15.104:0/1169786327 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654000bd90 con 0x7f6558071000 2023-07-14T11:57:14.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.848+0000 7f655d600700 1 -- 172.21.15.104:0/1169786327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6558071000 msgr2=0x7f655815ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.848+0000 7f655d600700 1 --2- 172.21.15.104:0/1169786327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6558071000 0x7f655815ccf0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f65580ca100 tx=0x7f6540009bf0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.848+0000 7f655d600700 1 -- 172.21.15.104:0/1169786327 shutdown_connections 2023-07-14T11:57:14.834 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.848+0000 7f655d600700 1 --2- 172.21.15.104:0/1169786327 >> v2:172.21.15.121:3300/0 conn(0x7f65580caa20 0x7f65580cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.848+0000 7f655d600700 1 --2- 172.21.15.104:0/1169786327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6558071000 0x7f655815ccf0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.848+0000 7f655d600700 1 -- 172.21.15.104:0/1169786327 >> 172.21.15.104:0/1169786327 conn(0x7f655806e960 msgr2=0x7f655806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:14.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.848+0000 7f655d600700 1 -- 172.21.15.104:0/1169786327 shutdown_connections 2023-07-14T11:57:14.835 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.848+0000 7f655d600700 1 -- 172.21.15.104:0/1169786327 wait complete. 2023-07-14T11:57:14.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f655d600700 1 Processor -- start 2023-07-14T11:57:14.836 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f655d600700 1 -- start start 2023-07-14T11:57:14.837 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f655d600700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6558071000 0x7f65580b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f655d600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65580caa20 0x7f655815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f655d600700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65581599d0 0x7f6558143040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f655d600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6558142d80 con 0x7f65580caa20 2023-07-14T11:57:14.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f655d600700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655815cd40 con 0x7f6558071000 2023-07-14T11:57:14.838 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f655d600700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65580adf20 con 0x7f65581599d0 2023-07-14T11:57:14.839 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f654ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65580caa20 0x7f655815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f654ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65580caa20 0x7f655815c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44706/0 (socket says 172.21.15.104:44706) 2023-07-14T11:57:14.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f654ffff700 1 -- 172.21.15.104:0/1624730670 learned_addr learned my addr 172.21.15.104:0/1624730670 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:14.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f6556ffd700 1 --2- 172.21.15.104:0/1624730670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6558071000 0x7f65580b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.840 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f65577fe700 1 --2- 172.21.15.104:0/1624730670 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65581599d0 0x7f6558143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f654ffff700 1 -- 172.21.15.104:0/1624730670 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65581599d0 msgr2=0x7f6558143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f654ffff700 1 --2- 172.21.15.104:0/1624730670 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65581599d0 0x7f6558143040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f654ffff700 1 -- 172.21.15.104:0/1624730670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6558071000 msgr2=0x7f65580b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:14.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f654ffff700 1 --2- 172.21.15.104:0/1624730670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6558071000 0x7f65580b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:14.841 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f654ffff700 1 -- 172.21.15.104:0/1624730670 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6540009570 con 0x7f65580caa20 2023-07-14T11:57:14.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.852+0000 7f6556ffd700 1 --2- 172.21.15.104:0/1624730670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6558071000 0x7f65580b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:57:14.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.856+0000 7f654ffff700 1 --2- 172.21.15.104:0/1624730670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65580caa20 0x7f655815c800 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6540000c00 tx=0x7f6540004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:14.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.856+0000 7f654f7fe700 1 -- 172.21.15.104:0/1624730670 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540003ad0 con 0x7f65580caa20 2023-07-14T11:57:14.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.856+0000 7f654f7fe700 1 -- 172.21.15.104:0/1624730670 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654001e070 con 0x7f65580caa20 2023-07-14T11:57:14.842 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.856+0000 7f654f7fe700 1 -- 172.21.15.104:0/1624730670 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540003e90 con 0x7f65580caa20 2023-07-14T11:57:14.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.856+0000 7f655d600700 1 -- 172.21.15.104:0/1624730670 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65580b29f0 con 0x7f65580caa20 2023-07-14T11:57:14.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.856+0000 7f655d600700 1 -- 172.21.15.104:0/1624730670 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65580b2fb0 con 0x7f65580caa20 2023-07-14T11:57:14.843 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.856+0000 7f655d600700 1 -- 172.21.15.104:0/1624730670 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655804e500 con 0x7f65580caa20 2023-07-14T11:57:14.844 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.860+0000 7f654f7fe700 1 -- 172.21.15.104:0/1624730670 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6540011420 con 0x7f65580caa20 2023-07-14T11:57:14.845 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.860+0000 7f654f7fe700 1 --2- 172.21.15.104:0/1624730670 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f65380662f0 0x7f65380687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:14.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.860+0000 7f654f7fe700 1 -- 172.21.15.104:0/1624730670 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f6540020070 con 0x7f65580caa20 2023-07-14T11:57:14.846 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.860+0000 7f6556ffd700 1 --2- 172.21.15.104:0/1624730670 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f65380662f0 0x7f65380687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:14.847 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.864+0000 7f6556ffd700 1 --2- 172.21.15.104:0/1624730670 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f65380662f0 0x7f65380687a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f65480017a0 tx=0x7f6548009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:14.852 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:14.868+0000 7f654f7fe700 1 -- 172.21.15.104:0/1624730670 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f6540052a10 con 0x7f65580caa20 2023-07-14T11:57:15.023 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:15.040+0000 7f655d600700 1 -- 172.21.15.104:0/1624730670 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f65580715e0 con 0x7f65380662f0 2023-07-14T11:57:15.030 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:15.044+0000 7f654f7fe700 1 -- 172.21.15.104:0/1624730670 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70439 (secure 0 0 0) 0x7f6548001c70 con 0x7f65380662f0 2023-07-14T11:57:15.032 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:57:15.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:15.048+0000 7f655d600700 1 -- 172.21.15.104:0/1624730670 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f65380662f0 msgr2=0x7f65380687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:15.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:15.048+0000 7f655d600700 1 --2- 172.21.15.104:0/1624730670 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f65380662f0 0x7f65380687a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f65480017a0 tx=0x7f6548009040 comp rx=0 tx=0).stop 2023-07-14T11:57:15.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:15.048+0000 7f655d600700 1 -- 172.21.15.104:0/1624730670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65580caa20 msgr2=0x7f655815c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:15.034 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:15.048+0000 7f655d600700 1 --2- 172.21.15.104:0/1624730670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65580caa20 0x7f655815c800 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6540000c00 tx=0x7f6540004770 comp rx=0 tx=0).stop 2023-07-14T11:57:15.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:15.048+0000 7f655d600700 1 -- 172.21.15.104:0/1624730670 shutdown_connections 2023-07-14T11:57:15.035 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:15.048+0000 7f655d600700 1 --2- 172.21.15.104:0/1624730670 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f65380662f0 0x7f65380687a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:15.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:15.048+0000 7f655d600700 1 --2- 172.21.15.104:0/1624730670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65580caa20 0x7f655815c800 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:15.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:15.048+0000 7f655d600700 1 --2- 172.21.15.104:0/1624730670 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65581599d0 0x7f6558143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:15.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:15.048+0000 7f655d600700 1 --2- 172.21.15.104:0/1624730670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6558071000 0x7f65580b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:15.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:15.048+0000 7f655d600700 1 -- 172.21.15.104:0/1624730670 >> 172.21.15.104:0/1624730670 conn(0x7f655806e960 msgr2=0x7f65580cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:15.036 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:15.048+0000 7f655d600700 1 -- 172.21.15.104:0/1624730670 shutdown_connections 2023-07-14T11:57:15.037 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:15.048+0000 7f655d600700 1 -- 172.21.15.104:0/1624730670 wait complete. 2023-07-14T11:57:15.037 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:57:15.057 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2023-07-14T11:57:13.980889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235144,"kb_used_data":7240,"kb_used_omap":0,"kb_used_meta":227840,"kb_avail":754739576,"statfs":{"total":773094113280,"available":772853325824,"internally_reserved":0,"allocated":7413760,"data_stored":4109128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":233308160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002407"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":100,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.864337+0000","last_change":"2023-07-14T11:53:09.145201+0000","last_active":"2023-07-14T11:57:04.864337+0000","last_peered":"2023-07-14T11:57:04.864337+0000","last_clean":"2023-07-14T11:57:04.864337+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:57:04.864337+0000","last_undegraded":"2023-07-14T11:57:04.864337+0000","last_fullsized":"2023-07-14T11:57:04.864337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:18:49.597233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":89,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.980077+0000","last_change":"2023-07-14T11:53:09.146150+0000","last_active":"2023-07-14T11:57:04.980077+0000","last_peered":"2023-07-14T11:57:04.980077+0000","last_clean":"2023-07-14T11:57:04.980077+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:57:04.980077+0000","last_undegraded":"2023-07-14T11:57:04.980077+0000","last_fullsized":"2023-07-14T11:57:04.980077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":86,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.975835+0000","last_change":"2023-07-14T11:53:09.435876+0000","last_active":"2023-07-14T11:57:04.975835+0000","last_peered":"2023-07-14T11:57:04.975835+0000","last_clean":"2023-07-14T11:57:04.975835+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:57:04.975835+0000","last_undegraded":"2023-07-14T11:57:04.975835+0000","last_fullsized":"2023-07-14T11:57:04.975835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:19:05.382083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":89,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.980659+0000","last_change":"2023-07-14T11:53:09.434255+0000","last_active":"2023-07-14T11:57:04.980659+0000","last_peered":"2023-07-14T11:57:04.980659+0000","last_clean":"2023-07-14T11:57:04.980659+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:57:04.980659+0000","last_undegraded":"2023-07-14T11:57:04.980659+0000","last_fullsized":"2023-07-14T11:57:04.980659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":98,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.908750+0000","last_change":"2023-07-14T11:53:09.147040+0000","last_active":"2023-07-14T11:57:04.908750+0000","last_peered":"2023-07-14T11:57:04.908750+0000","last_clean":"2023-07-14T11:57:04.908750+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:57:04.908750+0000","last_undegraded":"2023-07-14T11:57:04.908750+0000","last_fullsized":"2023-07-14T11:57:04.908750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:35:55.791012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":100,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.982691+0000","last_change":"2023-07-14T11:53:09.619294+0000","last_active":"2023-07-14T11:57:04.982691+0000","last_peered":"2023-07-14T11:57:04.982691+0000","last_clean":"2023-07-14T11:57:04.982691+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:57:04.982691+0000","last_undegraded":"2023-07-14T11:57:04.982691+0000","last_fullsized":"2023-07-14T11:57:04.982691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:29:22.000512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":87,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.976706+0000","last_change":"2023-07-14T11:53:09.146156+0000","last_active":"2023-07-14T11:57:04.976706+0000","last_peered":"2023-07-14T11:57:04.976706+0000","last_clean":"2023-07-14T11:57:04.976706+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:57:04.976706+0000","last_undegraded":"2023-07-14T11:57:04.976706+0000","last_fullsized":"2023-07-14T11:57:04.976706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:21:36.155230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":98,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.910974+0000","last_change":"2023-07-14T11:53:09.146556+0000","last_active":"2023-07-14T11:57:04.910974+0000","last_peered":"2023-07-14T11:57:04.910974+0000","last_clean":"2023-07-14T11:57:04.910974+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:57:04.910974+0000","last_undegraded":"2023-07-14T11:57:04.910974+0000","last_fullsized":"2023-07-14T11:57:04.910974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:08.116076+0000","last_clean_scrub_stamp":"2023-07-14T11:53:08.116076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:06:38.512393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":243,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:04.976949+0000","last_change":"2023-07-14T11:53:03.159818+0000","last_active":"2023-07-14T11:57:04.976949+0000","last_peered":"2023-07-14T11:57:04.976949+0000","last_clean":"2023-07-14T11:57:04.976949+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:57:04.976949+0000","last_undegraded":"2023-07-14T11:57:04.976949+0000","last_fullsized":"2023-07-14T11:57:04.976949+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:53:02.054348+0000","last_clean_scrub_stamp":"2023-07-14T11:53:02.054348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:00:32.142331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1372160,"data_stored":956417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.093,"15min":1.111},"min":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5509999999999999,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.069},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0920000000000001,"15min":1.123},"min":{"1min":0.50700000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.98499999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.1200000000000001,"15min":1.161},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.601,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.1799999999999999,"15min":1.2490000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.3420000000000001,"5min":2.234,"15min":2.234},"last":1.302}]},{"osd":2,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.173,"15min":1.268},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.407,"5min":2.04,"15min":2.04},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.208,"15min":1.2689999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.4279999999999999,"5min":2.492,"15min":2.492},"last":0.83799999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2589999999999999,"15min":1.3280000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.732,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.163,"5min":1.218,"15min":1.266},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.482,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.0189999999999999}]},{"osd":4,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.26,"15min":1.3640000000000001},"min":{"1min":0.56499999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.54,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":1.254},{"interface":"front","average":{"1min":1.266,"5min":1.3129999999999999,"15min":1.371},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9099999999999999,"5min":2.399,"15min":2.399},"last":1.3240000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.262,"15min":1.3120000000000001},"min":{"1min":0.83799999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5980000000000001,"5min":2.387,"15min":2.387},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.397,"15min":1.518},"min":{"1min":0.84899999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.595,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":1.127}]},{"osd":6,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.4730000000000001,"15min":1.589},"min":{"1min":0.751,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7689999999999999,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.351},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.417,"15min":1.51},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.8280000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.2689999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0609999999999999,"15min":1.0589999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.694,"5min":1.954,"15min":1.954},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.117,"15min":1.1000000000000001},"min":{"1min":0.61099999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.577,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.65900000000000003}]},{"osd":1,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.1599999999999999,"15min":1.1990000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.3779999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.304},{"interface":"front","average":{"1min":1.135,"5min":1.1930000000000001,"15min":1.2230000000000001},"min":{"1min":0.71799999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7330000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.099}]},{"osd":2,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.1970000000000001,"15min":1.175},"min":{"1min":0.752,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.2589999999999999,"15min":1.2689999999999999},"min":{"1min":0.86099999999999999,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.526,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.98099999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2969999999999999,"15min":1.3380000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.8109999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1970000000000001,"15min":1.1830000000000001},"min":{"1min":0.85699999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.147}]},{"osd":4,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.228,"15min":1.25},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.181},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3109999999999999,"15min":1.339},"min":{"1min":0.92100000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.8680000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.3540000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.248,"15min":1.23},"min":{"1min":0.85099999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6240000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.3759999999999999,"15min":1.4159999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.631,"5min":1.984,"15min":1.984},"last":1.3959999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.4179999999999999,"15min":1.46},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.9119999999999999,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.452},{"interface":"front","average":{"1min":1.446,"5min":1.377,"15min":1.383},"min":{"1min":1.01,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.4299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.081,"15min":1.163},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.371,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.038},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0980000000000001,"15min":1.165},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.452,"5min":1.581,"15min":1.581},"last":1.107}]},{"osd":1,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.1299999999999999,"15min":1.1859999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.365,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.349},{"interface":"front","average":{"1min":1.119,"5min":1.1599999999999999,"15min":1.1839999999999999},"min":{"1min":0.78500000000000003,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5269999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.1599999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1719999999999999,"15min":1.2170000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3680000000000001,"5min":1.839,"15min":1.839},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.2210000000000001,"15min":1.327},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6759999999999999,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.2070000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2470000000000001,"15min":1.3140000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.5349999999999999,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.403},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.2889999999999999,"15min":1.3740000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6439999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.292}]},{"osd":4,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1970000000000001,"15min":1.2450000000000001},"min":{"1min":0.82999999999999996,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6319999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.327,"15min":1.365},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.571,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.571}]},{"osd":6,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.359,"15min":1.4059999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5580000000000001,"5min":1.99,"15min":1.99},"last":1.502},{"interface":"front","average":{"1min":1.286,"5min":1.345,"15min":1.4059999999999999},"min":{"1min":0.89900000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6899999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.462}]},{"osd":7,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.395,"15min":1.4179999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.6180000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.498,"15min":1.5960000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7370000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.5509999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.1220000000000001,"15min":1.1619999999999999},"min":{"1min":0.67200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4450000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1200000000000001,"15min":1.1559999999999999},"min":{"1min":0.76500000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.2729999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.131,"15min":1.173},"min":{"1min":0.77800000000000002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4650000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.167,"15min":1.202},"min":{"1min":0.69399999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6779999999999999,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.83099999999999996}]},{"osd":2,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.129,"15min":1.141},"min":{"1min":0.749,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.4830000000000001,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.306,"15min":1.351},"min":{"1min":0.78600000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5349999999999999,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":1.05}]},{"osd":3,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2529999999999999,"15min":1.3129999999999999},"min":{"1min":0.84599999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.516,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.089},{"interface":"front","average":{"1min":1.206,"5min":1.2170000000000001,"15min":1.242},"min":{"1min":0.82999999999999996,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.4370000000000001,"5min":1.823,"15min":1.823},"last":1.367}]},{"osd":5,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.2569999999999999,"15min":1.3280000000000001},"min":{"1min":0.72399999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.54,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.2749999999999999,"15min":1.3260000000000001},"min":{"1min":0.84999999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.583,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.4119999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.262,"15min":1.266},"min":{"1min":0.86899999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.639,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.4350000000000001,"15min":1.5189999999999999},"min":{"1min":0.90400000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.863,"5min":3.5289999999999999,"15min":3.5289999999999999},"last":1.3109999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.431,"15min":1.4930000000000001},"min":{"1min":1.002,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.996,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.4330000000000001,"15min":1.5089999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":1.716,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":1.464}]}]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.84499999999999997,"15min":0.872},"min":{"1min":0.437,"5min":0.247,"15min":0.247},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86199999999999999,"15min":0.90400000000000003},"min":{"1min":0.53900000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2450000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.68600000000000005}]},{"osd":1,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.90500000000000003,"15min":0.95699999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1779999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.03,"15min":1.101},"min":{"1min":0.621,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.3129999999999999,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":1.248}]},{"osd":2,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0800000000000001,"15min":1.135},"min":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.496,"5min":1.627,"15min":1.627},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.91000000000000003,"5min":1.0289999999999999,"15min":1.0780000000000001},"min":{"1min":0.51000000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.234,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.2050000000000001,"15min":1.2729999999999999},"min":{"1min":0.89500000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.514,"5min":2.335,"15min":2.335},"last":1.038},{"interface":"front","average":{"1min":1.222,"5min":1.23,"15min":1.282},"min":{"1min":0.97999999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.7809999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.0149999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3049999999999999,"15min":1.353},"min":{"1min":0.94099999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.974,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.296,"5min":1.3260000000000001,"15min":1.4099999999999999},"min":{"1min":1.0429999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.54,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.3169999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2989999999999999,"15min":1.377},"min":{"1min":0.96899999999999997,"5min":0.625,"15min":0.625},"max":{"1min":1.7130000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4299999999999999,"15min":1.492},"min":{"1min":1.069,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.8939999999999999,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.069}]},{"osd":7,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.385,"15min":1.4690000000000001},"min":{"1min":0.95099999999999996,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5800000000000001,"5min":3.395,"15min":3.395},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.323,"15min":1.417},"min":{"1min":0.95099999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.837,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.96599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.85199999999999998,"15min":0.89900000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2130000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.87,"15min":0.90000000000000002},"min":{"1min":0.52700000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.58599999999999997}]},{"osd":1,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.879,"15min":0.89800000000000002},"min":{"1min":0.58199999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.498},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.01,"15min":1.0629999999999999},"min":{"1min":0.70099999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6319999999999999,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":0.90100000000000002}]},{"osd":3,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0449999999999999,"15min":1.089},"min":{"1min":0.57199999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.4610000000000001,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.004,"15min":1.044},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3360000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.038}]},{"osd":4,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2430000000000001,"15min":1.2889999999999999},"min":{"1min":0.91000000000000003,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.526,"5min":2.097,"15min":2.097},"last":1.161},{"interface":"front","average":{"1min":1.218,"5min":1.2529999999999999,"15min":1.3069999999999999},"min":{"1min":0.86499999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.9359999999999999,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.135}]},{"osd":5,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.26,"15min":1.3100000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.5640000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2989999999999999,"15min":1.349},"min":{"1min":0.96999999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.4550000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.2210000000000001}]},{"osd":6,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.3009999999999999,"15min":1.3720000000000001},"min":{"1min":0.79200000000000004,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8280000000000001,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.365,"15min":1.4159999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.659,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.0069999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3640000000000001,"15min":1.4330000000000001},"min":{"1min":0.88100000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.9079999999999999,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.393,"15min":1.4570000000000001},"min":{"1min":0.94299999999999995,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.859,"5min":2.536,"15min":2.536},"last":1.1080000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.80200000000000005,"15min":0.83299999999999996},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.23,"5min":1.349,"15min":1.349},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.52600000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2470000000000001,"5min":1.476,"15min":1.476},"last":1.0329999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.85299999999999998,"15min":0.85999999999999999},"min":{"1min":0.60699999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.2969999999999999,"5min":1.331,"15min":1.331},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.94999999999999996,"15min":0.94499999999999995},"min":{"1min":0.73799999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.359,"5min":1.665,"15min":1.665},"last":1.1359999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.79600000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4379999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.016,"15min":1.0229999999999999},"min":{"1min":0.58399999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.4890000000000001,"5min":1.857,"15min":1.857},"last":0.874}]},{"osd":4,"last update":"Fri Jul 14 11:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.22,"15min":1.266},"min":{"1min":0.96099999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.591,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1719999999999999,"15min":1.2130000000000001},"min":{"1min":0.88400000000000001,"5min":0.745,"15min":0.745},"max":{"1min":1.944,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.159}]},{"osd":5,"last update":"Fri Jul 14 11:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.216,"15min":1.252},"min":{"1min":0.80300000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.024,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2789999999999999,"15min":1.353},"min":{"1min":1.0229999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.7050000000000001,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.1899999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3320000000000001,"15min":1.379},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9850000000000001,"5min":2.633,"15min":2.633},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3240000000000001,"15min":1.355},"min":{"1min":1.097,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1440000000000001,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.401}]},{"osd":7,"last update":"Fri Jul 14 11:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.337,"15min":1.391},"min":{"1min":0.98499999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.8919999999999999,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.333},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3200000000000001,"15min":1.3480000000000001},"min":{"1min":1.0049999999999999,"5min":0.754,"15min":0.754},"max":{"1min":2.0489999999999999,"5min":2.375,"15min":2.375},"last":1.212}]}]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1372160,"data_stored":956417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.85799999999999998,"15min":0.88600000000000001},"min":{"1min":0.58999999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.371,"5min":1.599,"15min":1.599},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.88100000000000001,"15min":0.90500000000000003},"min":{"1min":0.62,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.2709999999999999,"5min":1.581,"15min":1.581},"last":0.62}]},{"osd":2,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.98599999999999999,"15min":0.99399999999999999},"min":{"1min":0.60399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.129,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.88500000000000001,"15min":0.94299999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.335,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.998}]},{"osd":3,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.93899999999999995,"15min":0.94699999999999995},"min":{"1min":0.60099999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.262,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.077,"15min":1.125},"min":{"1min":0.75700000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4239999999999999,"5min":1.704,"15min":1.704},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.228,"15min":1.2729999999999999},"min":{"1min":0.96499999999999997,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.597,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.3},{"interface":"front","average":{"1min":1.218,"5min":1.3080000000000001,"15min":1.361},"min":{"1min":0.96799999999999997,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.542,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.038}]},{"osd":5,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2,"15min":1.2749999999999999},"min":{"1min":0.88,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.51,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":0.88},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2589999999999999,"15min":1.3140000000000001},"min":{"1min":1.0369999999999999,"5min":0.754,"15min":0.754},"max":{"1min":1.595,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.204}]},{"osd":6,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.4039999999999999,"15min":1.45},"min":{"1min":0.98699999999999999,"5min":0.877,"15min":0.877},"max":{"1min":1.673,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.3,"15min":1.351},"min":{"1min":1.0009999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6240000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.339}]},{"osd":7,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.3080000000000001,"15min":1.373},"min":{"1min":0.98899999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.603,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.3660000000000001,"15min":1.413},"min":{"1min":1.0740000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.706,"5min":2.109,"15min":2.109},"last":1.3959999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:57:15.120 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-07-14T11:53:08.116076+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=53, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=57, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=0) 2023-07-14T11:57:15.121 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-07-14T11:53:08.116076+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=53, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=57, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=0) 2023-07-14T11:57:15.122 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-07-14T11:53:08.116076+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=53, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=57, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=0) 2023-07-14T11:57:15.122 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-07-14T11:53:08.116076+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=53, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=57, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=0) 2023-07-14T11:57:15.123 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-07-14T11:53:08.116076+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=53, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=57, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=0) 2023-07-14T11:57:15.123 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-07-14T11:53:08.116076+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=53, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=57, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=0) 2023-07-14T11:57:15.124 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-07-14T11:53:08.116076+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=53, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=57, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=0) 2023-07-14T11:57:15.124 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-07-14T11:53:08.116076+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=53, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=57, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=0) 2023-07-14T11:57:15.125 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-07-14T11:53:02.054348+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=53, tm_sec=2, tm_wday=4, tm_yday=195, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=14, tm_hour=11, tm_min=57, tm_sec=8, tm_wday=4, tm_yday=195, tm_isdst=0) 2023-07-14T11:57:15.125 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-07-14T11:57:35.126 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T11:57:35.302 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.315+0000 7f9e01dbe700 1 Processor -- start 2023-07-14T11:57:35.303 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.315+0000 7f9e01dbe700 1 -- start start 2023-07-14T11:57:35.305 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.319+0000 7f9e01dbe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9dfc162b10 0x7f9dfc164ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:35.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.319+0000 7f9e01dbe700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc0b0ef0 con 0x7f9dfc0c9d90 2023-07-14T11:57:35.306 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.319+0000 7f9e01dbe700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc0ce9c0 con 0x7f9dfc159f20 2023-07-14T11:57:35.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.319+0000 7f9e01dbe700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc0cee50 con 0x7f9dfc162b10 2023-07-14T11:57:35.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.319+0000 7f9e00dbc700 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f9dfc0c9d90 0x7f9dfc0ca160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:40238/0 (socket says 172.21.15.104:40238) 2023-07-14T11:57:35.307 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.319+0000 7f9e00dbc700 1 -- 172.21.15.104:0/4201352041 learned_addr learned my addr 172.21.15.104:0/4201352041 (peer_addr_for_me v1:172.21.15.104:0/0) 2023-07-14T11:57:35.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.319+0000 7f9e015bd700 1 --2- 172.21.15.104:0/4201352041 >> v2:172.21.15.121:3300/0 conn(0x7f9dfc162b10 0x7f9dfc164ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:35.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.319+0000 7f9dfb7fe700 1 -- 172.21.15.104:0/4201352041 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3545016198 0 0) 0x7f9dfc0b0ef0 con 0x7f9dfc0c9d90 2023-07-14T11:57:35.310 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.319+0000 7f9dfb7fe700 1 -- 172.21.15.104:0/4201352041 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dec0035d0 con 0x7f9dfc0c9d90 2023-07-14T11:57:35.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.319+0000 7f9e015bd700 1 -- 172.21.15.104:0/4201352041 >> v1:172.21.15.121:6789/0 conn(0x7f9dfc159f20 legacy=0x7f9dfc0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:35.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.319+0000 7f9e015bd700 1 -- 172.21.15.104:0/4201352041 >> v1:172.21.15.104:6789/0 conn(0x7f9dfc0c9d90 legacy=0x7f9dfc0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:35.311 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.319+0000 7f9e015bd700 1 -- 172.21.15.104:0/4201352041 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc0cb4f0 con 0x7f9dfc162b10 2023-07-14T11:57:35.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.323+0000 7f9e015bd700 1 --2- 172.21.15.104:0/4201352041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dfc162b10 0x7f9dfc164ef0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009d90 tx=0x7f9df400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4020c3904d8c8000 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:35.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.323+0000 7f9dfb7fe700 1 -- 172.21.15.104:0/4201352041 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df400e070 con 0x7f9dfc162b10 2023-07-14T11:57:35.312 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.323+0000 7f9dfb7fe700 1 -- 172.21.15.104:0/4201352041 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df400bb20 con 0x7f9dfc162b10 2023-07-14T11:57:35.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.323+0000 7f9dfb7fe700 1 -- 172.21.15.104:0/4201352041 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4012430 con 0x7f9dfc162b10 2023-07-14T11:57:35.313 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.323+0000 7f9e01dbe700 1 -- 172.21.15.104:0/4201352041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dfc162b10 msgr2=0x7f9dfc164ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:35.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.323+0000 7f9e01dbe700 1 --2- 172.21.15.104:0/4201352041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dfc162b10 0x7f9dfc164ef0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009d90 tx=0x7f9df400b330 comp rx=0 tx=0).stop 2023-07-14T11:57:35.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.323+0000 7f9e01dbe700 1 -- 172.21.15.104:0/4201352041 shutdown_connections 2023-07-14T11:57:35.314 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.323+0000 7f9e01dbe700 1 --2- 172.21.15.104:0/4201352041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dfc162b10 0x7f9dfc164ef0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.323+0000 7f9e01dbe700 1 -- 172.21.15.104:0/4201352041 >> 172.21.15.104:0/4201352041 conn(0x7f9dfc06e960 msgr2=0x7f9dfc06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:35.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.323+0000 7f9e01dbe700 1 -- 172.21.15.104:0/4201352041 shutdown_connections 2023-07-14T11:57:35.315 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.323+0000 7f9e01dbe700 1 -- 172.21.15.104:0/4201352041 wait complete. 2023-07-14T11:57:35.319 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.327+0000 7f9e01dbe700 1 Processor -- start 2023-07-14T11:57:35.319 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.327+0000 7f9e01dbe700 1 -- start start 2023-07-14T11:57:35.319 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e01dbe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dfc0c9d90 0x7f9dfc0c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:35.320 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e01dbe700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dfc159f20 0x7f9dfc15ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:35.320 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e01dbe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dfc162b10 0x7f9dfc15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:35.321 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e01dbe700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0c98c0 con 0x7f9dfc162b10 2023-07-14T11:57:35.321 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e01dbe700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc15a9f0 con 0x7f9dfc0c9d90 2023-07-14T11:57:35.321 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e01dbe700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc15acb0 con 0x7f9dfc159f20 2023-07-14T11:57:35.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e015bd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dfc162b10 0x7f9dfc15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:35.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e00dbc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dfc0c9d90 0x7f9dfc0c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:35.322 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e015bd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dfc162b10 0x7f9dfc15bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52100/0 (socket says 172.21.15.104:52100) 2023-07-14T11:57:35.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e015bd700 1 -- 172.21.15.104:0/1180520892 learned_addr learned my addr 172.21.15.104:0/1180520892 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:35.323 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9dfbfff700 1 --2- 172.21.15.104:0/1180520892 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dfc159f20 0x7f9dfc15ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:35.324 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e015bd700 1 -- 172.21.15.104:0/1180520892 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dfc159f20 msgr2=0x7f9dfc15ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:35.324 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e015bd700 1 --2- 172.21.15.104:0/1180520892 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dfc159f20 0x7f9dfc15ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.325 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e015bd700 1 -- 172.21.15.104:0/1180520892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dfc0c9d90 msgr2=0x7f9dfc0c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:35.325 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e015bd700 1 --2- 172.21.15.104:0/1180520892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dfc0c9d90 0x7f9dfc0c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e015bd700 1 -- 172.21.15.104:0/1180520892 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df4009a70 con 0x7f9dfc162b10 2023-07-14T11:57:35.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e015bd700 1 --2- 172.21.15.104:0/1180520892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dfc162b10 0x7f9dfc15bfb0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc06a6f0 tx=0x7f9df4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:35.326 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9df97fa700 1 -- 172.21.15.104:0/1180520892 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4012430 con 0x7f9dfc162b10 2023-07-14T11:57:35.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9df97fa700 1 -- 172.21.15.104:0/1180520892 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df4020070 con 0x7f9dfc162b10 2023-07-14T11:57:35.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e01dbe700 1 -- 172.21.15.104:0/1180520892 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc15af90 con 0x7f9dfc162b10 2023-07-14T11:57:35.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9df97fa700 1 -- 172.21.15.104:0/1180520892 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4004280 con 0x7f9dfc162b10 2023-07-14T11:57:35.327 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.331+0000 7f9e01dbe700 1 -- 172.21.15.104:0/1180520892 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc15b550 con 0x7f9dfc162b10 2023-07-14T11:57:35.328 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.335+0000 7f9e01dbe700 1 -- 172.21.15.104:0/1180520892 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dfc04e500 con 0x7f9dfc162b10 2023-07-14T11:57:35.329 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.335+0000 7f9df97fa700 1 -- 172.21.15.104:0/1180520892 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9df40043e0 con 0x7f9dfc162b10 2023-07-14T11:57:35.329 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.339+0000 7f9df97fa700 1 --2- 172.21.15.104:0/1180520892 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9de00662f0 0x7f9de00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:35.329 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.339+0000 7f9df97fa700 1 -- 172.21.15.104:0/1180520892 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f9df4022070 con 0x7f9dfc162b10 2023-07-14T11:57:35.329 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.339+0000 7f9e00dbc700 1 --2- 172.21.15.104:0/1180520892 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9de00662f0 0x7f9de00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:35.329 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.339+0000 7f9e00dbc700 1 --2- 172.21.15.104:0/1180520892 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9de00662f0 0x7f9de00687a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9df0007fd0 tx=0x7f9df0007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:35.332 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.347+0000 7f9df97fa700 1 -- 172.21.15.104:0/1180520892 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f9df4052e10 con 0x7f9dfc162b10 2023-07-14T11:57:35.501 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.515+0000 7f9e01dbe700 1 -- 172.21.15.104:0/1180520892 --> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9dfc0ce370 con 0x7f9de00662f0 2023-07-14T11:57:35.504 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.519+0000 7f9df97fa700 1 -- 172.21.15.104:0/1180520892 <== mgr.4108 v2:172.21.15.104:6832/608103909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70765 (secure 0 0 0) 0x7f9df0001390 con 0x7f9de00662f0 2023-07-14T11:57:35.506 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:57:35.507 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.523+0000 7f9e01dbe700 1 -- 172.21.15.104:0/1180520892 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9de00662f0 msgr2=0x7f9de00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:35.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.523+0000 7f9e01dbe700 1 --2- 172.21.15.104:0/1180520892 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9de00662f0 0x7f9de00687a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9df0007fd0 tx=0x7f9df0007de0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.523+0000 7f9e01dbe700 1 -- 172.21.15.104:0/1180520892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dfc162b10 msgr2=0x7f9dfc15bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:35.508 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.523+0000 7f9e01dbe700 1 --2- 172.21.15.104:0/1180520892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dfc162b10 0x7f9dfc15bfb0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc06a6f0 tx=0x7f9df4004af0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.509 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.523+0000 7f9e01dbe700 1 -- 172.21.15.104:0/1180520892 shutdown_connections 2023-07-14T11:57:35.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.523+0000 7f9e01dbe700 1 --2- 172.21.15.104:0/1180520892 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f9de00662f0 0x7f9de00687a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.523+0000 7f9e01dbe700 1 --2- 172.21.15.104:0/1180520892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dfc162b10 0x7f9dfc15bfb0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.523+0000 7f9e01dbe700 1 --2- 172.21.15.104:0/1180520892 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dfc159f20 0x7f9dfc15ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.510 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.523+0000 7f9e01dbe700 1 --2- 172.21.15.104:0/1180520892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dfc0c9d90 0x7f9dfc0c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.523+0000 7f9e01dbe700 1 -- 172.21.15.104:0/1180520892 >> 172.21.15.104:0/1180520892 conn(0x7f9dfc06e960 msgr2=0x7f9dfc15cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:35.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.523+0000 7f9e01dbe700 1 -- 172.21.15.104:0/1180520892 shutdown_connections 2023-07-14T11:57:35.511 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.523+0000 7f9e01dbe700 1 -- 172.21.15.104:0/1180520892 wait complete. 2023-07-14T11:57:35.511 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2023-07-14T11:57:35.529 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2023-07-14T11:57:33.987222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235144,"kb_used_data":7240,"kb_used_omap":0,"kb_used_meta":227840,"kb_avail":754739576,"statfs":{"total":773094113280,"available":772853325824,"internally_reserved":0,"allocated":7413760,"data_stored":4109128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":233308160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003903"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:20.624187+0000","last_change":"2023-07-14T11:57:20.624187+0000","last_active":"2023-07-14T11:57:20.624187+0000","last_peered":"2023-07-14T11:57:20.624187+0000","last_clean":"2023-07-14T11:57:20.624187+0000","last_became_active":"2023-07-14T11:53:09.144759+0000","last_became_peered":"2023-07-14T11:53:09.144759+0000","last_unstale":"2023-07-14T11:57:20.624187+0000","last_undegraded":"2023-07-14T11:57:20.624187+0000","last_fullsized":"2023-07-14T11:57:20.624187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:57:20.623898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:57:20.623898+0000","last_clean_scrub_stamp":"2023-07-14T11:57:20.623898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:40:08.905897+0000","scrub_duration":0.0044200259999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":98,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:12.758627+0000","last_change":"2023-07-14T11:57:12.758627+0000","last_active":"2023-07-14T11:57:12.758627+0000","last_peered":"2023-07-14T11:57:12.758627+0000","last_clean":"2023-07-14T11:57:12.758627+0000","last_became_active":"2023-07-14T11:53:09.145722+0000","last_became_peered":"2023-07-14T11:53:09.145722+0000","last_unstale":"2023-07-14T11:57:12.758627+0000","last_undegraded":"2023-07-14T11:57:12.758627+0000","last_fullsized":"2023-07-14T11:57:12.758627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:57:12.758430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:57:12.758430+0000","last_clean_scrub_stamp":"2023-07-14T11:57:12.758430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:50:59.556307+0000","scrub_duration":0.0041316770000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":96,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:30.107245+0000","last_change":"2023-07-14T11:57:30.107327+0000","last_active":"2023-07-14T11:57:30.107245+0000","last_peered":"2023-07-14T11:57:30.107245+0000","last_clean":"2023-07-14T11:57:30.107245+0000","last_became_active":"2023-07-14T11:53:09.435174+0000","last_became_peered":"2023-07-14T11:53:09.435174+0000","last_unstale":"2023-07-14T11:57:30.107245+0000","last_undegraded":"2023-07-14T11:57:30.107245+0000","last_fullsized":"2023-07-14T11:57:30.107245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:57:30.107095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:57:30.107095+0000","last_clean_scrub_stamp":"2023-07-14T11:57:30.107095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:58:12.836185+0000","scrub_duration":0.0052056730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":98,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:13.789599+0000","last_change":"2023-07-14T11:57:13.789599+0000","last_active":"2023-07-14T11:57:13.789599+0000","last_peered":"2023-07-14T11:57:13.789599+0000","last_clean":"2023-07-14T11:57:13.789599+0000","last_became_active":"2023-07-14T11:53:09.433554+0000","last_became_peered":"2023-07-14T11:53:09.433554+0000","last_unstale":"2023-07-14T11:57:13.789599+0000","last_undegraded":"2023-07-14T11:57:13.789599+0000","last_fullsized":"2023-07-14T11:57:13.789599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:57:13.789279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:57:13.789279+0000","last_clean_scrub_stamp":"2023-07-14T11:57:13.789279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:02:55.906479+0000","scrub_duration":0.0039354790000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:19.653728+0000","last_change":"2023-07-14T11:57:19.653728+0000","last_active":"2023-07-14T11:57:19.653728+0000","last_peered":"2023-07-14T11:57:19.653728+0000","last_clean":"2023-07-14T11:57:19.653728+0000","last_became_active":"2023-07-14T11:53:09.146656+0000","last_became_peered":"2023-07-14T11:53:09.146656+0000","last_unstale":"2023-07-14T11:57:19.653728+0000","last_undegraded":"2023-07-14T11:57:19.653728+0000","last_fullsized":"2023-07-14T11:57:19.653728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:57:19.653344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:57:19.653344+0000","last_clean_scrub_stamp":"2023-07-14T11:57:19.653344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:58:02.382434+0000","scrub_duration":0.0051825789999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":110,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:12.170752+0000","last_change":"2023-07-14T11:57:12.170752+0000","last_active":"2023-07-14T11:57:12.170752+0000","last_peered":"2023-07-14T11:57:12.170752+0000","last_clean":"2023-07-14T11:57:12.170752+0000","last_became_active":"2023-07-14T11:53:09.618732+0000","last_became_peered":"2023-07-14T11:53:09.618732+0000","last_unstale":"2023-07-14T11:57:12.170752+0000","last_undegraded":"2023-07-14T11:57:12.170752+0000","last_fullsized":"2023-07-14T11:57:12.170752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:57:12.170518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:57:12.170518+0000","last_clean_scrub_stamp":"2023-07-14T11:57:12.170518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:17:47.301451+0000","scrub_duration":0.0040157250000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":96,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:17.982524+0000","last_change":"2023-07-14T11:57:17.982689+0000","last_active":"2023-07-14T11:57:17.982524+0000","last_peered":"2023-07-14T11:57:17.982524+0000","last_clean":"2023-07-14T11:57:17.982524+0000","last_became_active":"2023-07-14T11:53:09.145828+0000","last_became_peered":"2023-07-14T11:53:09.145828+0000","last_unstale":"2023-07-14T11:57:17.982524+0000","last_undegraded":"2023-07-14T11:57:17.982524+0000","last_fullsized":"2023-07-14T11:57:17.982524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:57:17.982315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:57:17.982315+0000","last_clean_scrub_stamp":"2023-07-14T11:57:17.982315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:42:34.440695+0000","scrub_duration":0.0058326380000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:14.567569+0000","last_change":"2023-07-14T11:57:14.567569+0000","last_active":"2023-07-14T11:57:14.567569+0000","last_peered":"2023-07-14T11:57:14.567569+0000","last_clean":"2023-07-14T11:57:14.567569+0000","last_became_active":"2023-07-14T11:53:09.146211+0000","last_became_peered":"2023-07-14T11:53:09.146211+0000","last_unstale":"2023-07-14T11:57:14.567569+0000","last_undegraded":"2023-07-14T11:57:14.567569+0000","last_fullsized":"2023-07-14T11:57:14.567569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:57:14.567274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:57:14.567274+0000","last_clean_scrub_stamp":"2023-07-14T11:57:14.567274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:09:39.884845+0000","scrub_duration":0.0045236230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"64'119","reported_seq":254,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-07-14T11:57:16.011895+0000","last_change":"2023-07-14T11:57:16.011969+0000","last_active":"2023-07-14T11:57:16.011895+0000","last_peered":"2023-07-14T11:57:16.011895+0000","last_clean":"2023-07-14T11:57:16.011895+0000","last_became_active":"2023-07-14T11:53:03.158274+0000","last_became_peered":"2023-07-14T11:53:03.158274+0000","last_unstale":"2023-07-14T11:57:16.011895+0000","last_undegraded":"2023-07-14T11:57:16.011895+0000","last_fullsized":"2023-07-14T11:57:16.011895+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'119","last_scrub_stamp":"2023-07-14T11:57:16.011785+0000","last_deep_scrub":"64'119","last_deep_scrub_stamp":"2023-07-14T11:57:16.011785+0000","last_clean_scrub_stamp":"2023-07-14T11:57:16.011785+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:01:49.087500+0000","scrub_duration":0.011975513,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1372160,"data_stored":956417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.093,"15min":1.111},"min":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5509999999999999,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0920000000000001,"15min":1.123},"min":{"1min":0.50700000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.84699999999999998}]},{"osd":1,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.1200000000000001,"15min":1.161},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.601,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.141},{"interface":"front","average":{"1min":1.087,"5min":1.1799999999999999,"15min":1.2490000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.3420000000000001,"5min":2.234,"15min":2.234},"last":1.179}]},{"osd":2,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.173,"15min":1.268},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.407,"5min":2.04,"15min":2.04},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.208,"15min":1.2689999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.4279999999999999,"5min":2.492,"15min":2.492},"last":1.0389999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2589999999999999,"15min":1.3280000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.732,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.016},{"interface":"front","average":{"1min":1.163,"5min":1.218,"15min":1.266},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.482,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.2250000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.26,"15min":1.3640000000000001},"min":{"1min":0.56499999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.54,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.266,"5min":1.3129999999999999,"15min":1.371},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9099999999999999,"5min":2.399,"15min":2.399},"last":0.67600000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.262,"15min":1.3120000000000001},"min":{"1min":0.83799999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5980000000000001,"5min":2.387,"15min":2.387},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.397,"15min":1.518},"min":{"1min":0.84899999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.595,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":1.2649999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.4730000000000001,"15min":1.589},"min":{"1min":0.751,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7689999999999999,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.417,"15min":1.51},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.8280000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":0.63900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0609999999999999,"15min":1.0589999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.694,"5min":1.954,"15min":1.954},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.117,"15min":1.1000000000000001},"min":{"1min":0.61099999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.577,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.286}]},{"osd":1,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.1599999999999999,"15min":1.1990000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.3779999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.474},{"interface":"front","average":{"1min":1.135,"5min":1.1930000000000001,"15min":1.2230000000000001},"min":{"1min":0.71799999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.7330000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.1659999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.1970000000000001,"15min":1.175},"min":{"1min":0.752,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.216,"5min":1.2589999999999999,"15min":1.2689999999999999},"min":{"1min":0.86099999999999999,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.526,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.3140000000000001}]},{"osd":3,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2969999999999999,"15min":1.3380000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.8109999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1970000000000001,"15min":1.1830000000000001},"min":{"1min":0.85699999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.4330000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.228,"15min":1.25},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.099},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3109999999999999,"15min":1.339},"min":{"1min":0.92100000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.8680000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.93899999999999995}]},{"osd":5,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.248,"15min":1.23},"min":{"1min":0.85099999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6240000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.3759999999999999,"15min":1.4159999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.631,"5min":1.984,"15min":1.984},"last":1.2110000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.4179999999999999,"15min":1.46},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.9119999999999999,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.036},{"interface":"front","average":{"1min":1.446,"5min":1.377,"15min":1.383},"min":{"1min":1.01,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.5069999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.081,"15min":1.163},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.371,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0980000000000001,"15min":1.165},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.452,"5min":1.581,"15min":1.581},"last":0.98999999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.1299999999999999,"15min":1.1859999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.365,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.071},{"interface":"front","average":{"1min":1.119,"5min":1.1599999999999999,"15min":1.1839999999999999},"min":{"1min":0.78500000000000003,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5269999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.0429999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1719999999999999,"15min":1.2170000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3680000000000001,"5min":1.839,"15min":1.839},"last":1.407},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.2210000000000001,"15min":1.327},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6759999999999999,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.3089999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2470000000000001,"15min":1.3140000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.5349999999999999,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.2889999999999999,"15min":1.3740000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6439999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.264}]},{"osd":4,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1970000000000001,"15min":1.2450000000000001},"min":{"1min":0.82999999999999996,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6319999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.248,"5min":1.327,"15min":1.365},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.571,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.153}]},{"osd":6,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.359,"15min":1.4059999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5580000000000001,"5min":1.99,"15min":1.99},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.345,"15min":1.4059999999999999},"min":{"1min":0.89900000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6899999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.119}]},{"osd":7,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.395,"15min":1.4179999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.6180000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.234},{"interface":"front","average":{"1min":1.351,"5min":1.498,"15min":1.5960000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7370000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.359}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.1220000000000001,"15min":1.1619999999999999},"min":{"1min":0.67200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4450000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":1.194},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1200000000000001,"15min":1.1559999999999999},"min":{"1min":0.76500000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.038}]},{"osd":1,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.131,"15min":1.173},"min":{"1min":0.77800000000000002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4650000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.167,"15min":1.202},"min":{"1min":0.69399999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6779999999999999,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.0780000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.129,"15min":1.141},"min":{"1min":0.749,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.4830000000000001,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.111},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.306,"15min":1.351},"min":{"1min":0.78600000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5349999999999999,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":1.4239999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2529999999999999,"15min":1.3129999999999999},"min":{"1min":0.84599999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.516,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.2170000000000001,"15min":1.242},"min":{"1min":0.82999999999999996,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.4370000000000001,"5min":1.823,"15min":1.823},"last":0.998}]},{"osd":5,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.2569999999999999,"15min":1.3280000000000001},"min":{"1min":0.72399999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.54,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.296},{"interface":"front","average":{"1min":1.242,"5min":1.2749999999999999,"15min":1.3260000000000001},"min":{"1min":0.84999999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.583,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.3779999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.262,"15min":1.266},"min":{"1min":0.86899999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.639,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.333},{"interface":"front","average":{"1min":1.355,"5min":1.4350000000000001,"15min":1.5189999999999999},"min":{"1min":0.90400000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.863,"5min":3.5289999999999999,"15min":3.5289999999999999},"last":1.5529999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.431,"15min":1.4930000000000001},"min":{"1min":1.002,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.996,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.4330000000000001,"15min":1.5089999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":1.716,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":1.512}]}]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.84499999999999997,"15min":0.872},"min":{"1min":0.437,"5min":0.247,"15min":0.247},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86199999999999999,"15min":0.90400000000000003},"min":{"1min":0.53900000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2450000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.012}]},{"osd":1,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.90500000000000003,"15min":0.95699999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1779999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.03,"15min":1.101},"min":{"1min":0.621,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.3129999999999999,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":1.3129999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0800000000000001,"15min":1.135},"min":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.496,"5min":1.627,"15min":1.627},"last":1.204},{"interface":"front","average":{"1min":0.91000000000000003,"5min":1.0289999999999999,"15min":1.0780000000000001},"min":{"1min":0.51000000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.234,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.075}]},{"osd":4,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.2050000000000001,"15min":1.2729999999999999},"min":{"1min":0.89500000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.514,"5min":2.335,"15min":2.335},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.23,"15min":1.282},"min":{"1min":0.97999999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.7809999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.1579999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3049999999999999,"15min":1.353},"min":{"1min":0.94099999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.974,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.29},{"interface":"front","average":{"1min":1.296,"5min":1.3260000000000001,"15min":1.4099999999999999},"min":{"1min":1.0429999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.54,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.407}]},{"osd":6,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2989999999999999,"15min":1.377},"min":{"1min":0.96899999999999997,"5min":0.625,"15min":0.625},"max":{"1min":1.7130000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4299999999999999,"15min":1.492},"min":{"1min":1.069,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.8939999999999999,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.345}]},{"osd":7,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.385,"15min":1.4690000000000001},"min":{"1min":0.95099999999999996,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5800000000000001,"5min":3.395,"15min":3.395},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.323,"15min":1.417},"min":{"1min":0.95099999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.837,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.2589999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.85199999999999998,"15min":0.89900000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2130000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.87,"15min":0.90000000000000002},"min":{"1min":0.52700000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":1.2969999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.879,"15min":0.89800000000000002},"min":{"1min":0.58199999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.01,"15min":1.0629999999999999},"min":{"1min":0.70099999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6319999999999999,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.427}]},{"osd":3,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0449999999999999,"15min":1.089},"min":{"1min":0.57199999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.4610000000000001,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.111},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.004,"15min":1.044},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3360000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.97799999999999998}]},{"osd":4,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2430000000000001,"15min":1.2889999999999999},"min":{"1min":0.91000000000000003,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.526,"5min":2.097,"15min":2.097},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.2529999999999999,"15min":1.3069999999999999},"min":{"1min":0.86499999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.9359999999999999,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.609}]},{"osd":5,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.26,"15min":1.3100000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.5640000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.347},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2989999999999999,"15min":1.349},"min":{"1min":0.96999999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.4550000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.3859999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.3009999999999999,"15min":1.3720000000000001},"min":{"1min":0.79200000000000004,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8280000000000001,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.365,"15min":1.4159999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.659,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":1.4690000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3640000000000001,"15min":1.4330000000000001},"min":{"1min":0.88100000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.9079999999999999,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.393,"15min":1.4570000000000001},"min":{"1min":0.94299999999999995,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.859,"5min":2.536,"15min":2.536},"last":1.5680000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":778240,"data_stored":366049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.80200000000000005,"15min":0.83299999999999996},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.23,"5min":1.349,"15min":1.349},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.52600000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2470000000000001,"5min":1.476,"15min":1.476},"last":0.72899999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.85299999999999998,"15min":0.85999999999999999},"min":{"1min":0.60699999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.2969999999999999,"5min":1.331,"15min":1.331},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.94999999999999996,"15min":0.94499999999999995},"min":{"1min":0.73799999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.359,"5min":1.665,"15min":1.665},"last":1.0209999999999999}]},{"osd":3,"last update":"Fri Jul 14 11:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.79600000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4379999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.016,"15min":1.0229999999999999},"min":{"1min":0.58399999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.4890000000000001,"5min":1.857,"15min":1.857},"last":1.1020000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.22,"15min":1.266},"min":{"1min":0.96099999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.591,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.177},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1719999999999999,"15min":1.2130000000000001},"min":{"1min":0.88400000000000001,"5min":0.745,"15min":0.745},"max":{"1min":1.944,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.1399999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.216,"15min":1.252},"min":{"1min":0.80300000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.024,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.073},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2789999999999999,"15min":1.353},"min":{"1min":1.0229999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.7050000000000001,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.3799999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3320000000000001,"15min":1.379},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9850000000000001,"5min":2.633,"15min":2.633},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3240000000000001,"15min":1.355},"min":{"1min":1.097,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1440000000000001,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.337,"15min":1.391},"min":{"1min":0.98499999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.8919999999999999,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.302},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3200000000000001,"15min":1.3480000000000001},"min":{"1min":1.0049999999999999,"5min":0.754,"15min":0.754},"max":{"1min":2.0489999999999999,"5min":2.375,"15min":2.375},"last":1.246}]}]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1372160,"data_stored":956417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.85799999999999998,"15min":0.88600000000000001},"min":{"1min":0.58999999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.371,"5min":1.599,"15min":1.599},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.88100000000000001,"15min":0.90500000000000003},"min":{"1min":0.62,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.2709999999999999,"5min":1.581,"15min":1.581},"last":0.86799999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.98599999999999999,"15min":0.99399999999999999},"min":{"1min":0.60399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.129,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.048},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.88500000000000001,"15min":0.94299999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.335,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.92500000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.93899999999999995,"15min":0.94699999999999995},"min":{"1min":0.60099999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.262,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.077,"15min":1.125},"min":{"1min":0.75700000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4239999999999999,"5min":1.704,"15min":1.704},"last":1.3320000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.228,"15min":1.2729999999999999},"min":{"1min":0.96499999999999997,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.597,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.3080000000000001,"15min":1.361},"min":{"1min":0.96799999999999997,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.542,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.4059999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2,"15min":1.2749999999999999},"min":{"1min":0.88,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.51,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2589999999999999,"15min":1.3140000000000001},"min":{"1min":1.0369999999999999,"5min":0.754,"15min":0.754},"max":{"1min":1.595,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.3}]},{"osd":6,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.4039999999999999,"15min":1.45},"min":{"1min":0.98699999999999999,"5min":0.877,"15min":0.877},"max":{"1min":1.673,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.3,"15min":1.351},"min":{"1min":1.0009999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6240000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.2190000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.3080000000000001,"15min":1.373},"min":{"1min":0.98899999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.603,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.478},{"interface":"front","average":{"1min":1.371,"5min":1.3660000000000001,"15min":1.413},"min":{"1min":1.0740000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.706,"5min":2.109,"15min":2.109},"last":1.4259999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T11:57:35.535 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-07-14T11:57:35.698 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.711+0000 7fdca582f700 1 Processor -- start 2023-07-14T11:57:35.699 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.711+0000 7fdca582f700 1 -- start start 2023-07-14T11:57:35.701 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.715+0000 7fdca582f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdca01603d0 0x7fdca006dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:35.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.715+0000 7fdca582f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdca00cd7f0 0x7fdca00c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:35.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.715+0000 7fdca582f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca00cdfb0 con 0x7fdca0160000 2023-07-14T11:57:35.702 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.715+0000 7fdca582f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca0070e40 con 0x7fdca00cd7f0 2023-07-14T11:57:35.703 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.715+0000 7fdca582f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca00cdcd0 con 0x7fdca01603d0 2023-07-14T11:57:35.706 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.715+0000 7fdca502e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdca00cd7f0 0x7fdca00c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:35.706 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.715+0000 7fdc9ffff700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdca01603d0 0x7fdca006dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:35.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.715+0000 7fdca502e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdca00cd7f0 0x7fdca00c0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.104:45370/0 (socket says 172.21.15.104:45370) 2023-07-14T11:57:35.707 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.715+0000 7fdca502e700 1 -- 172.21.15.104:0/2003135642 learned_addr learned my addr 172.21.15.104:0/2003135642 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:35.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.715+0000 7fdca482d700 1 --1- 172.21.15.104:0/2003135642 >> v1:172.21.15.121:6789/0 conn(0x7fdca0160000 0x7fdca00c9e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.104:52384/0 (socket says 172.21.15.104:52384) 2023-07-14T11:57:35.708 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.715+0000 7fdc9ffff700 1 -- 172.21.15.104:0/2003135642 >> v2:172.21.15.121:3300/0 conn(0x7fdca00cd7f0 msgr2=0x7fdca00c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:35.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.715+0000 7fdc9ffff700 1 --2- 172.21.15.104:0/2003135642 >> v2:172.21.15.121:3300/0 conn(0x7fdca00cd7f0 0x7fdca00c0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.719+0000 7fdc9ffff700 1 -- 172.21.15.104:0/2003135642 >> v1:172.21.15.121:6789/0 conn(0x7fdca0160000 legacy=0x7fdca00c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:35.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.719+0000 7fdc9ffff700 1 -- 172.21.15.104:0/2003135642 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca0070780 con 0x7fdca01603d0 2023-07-14T11:57:35.709 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.715+0000 7fdc9f7fe700 1 -- 172.21.15.104:0/2003135642 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936842372 0 0) 0x7fdc94001110 con 0x7fdca0160000 2023-07-14T11:57:35.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.719+0000 7fdca502e700 1 --2- 172.21.15.104:0/2003135642 >> v2:172.21.15.121:3300/0 conn(0x7fdca00cd7f0 0x7fdca00c0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:57:35.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.719+0000 7fdc9ffff700 1 --2- 172.21.15.104:0/2003135642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdca01603d0 0x7fdca006dd10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009850 tx=0x7fdc8c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f60705b5b5f7971c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:35.710 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.719+0000 7fdc9f7fe700 1 -- 172.21.15.104:0/2003135642 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c011830 con 0x7fdca01603d0 2023-07-14T11:57:35.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.719+0000 7fdc9f7fe700 1 -- 172.21.15.104:0/2003135642 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c011990 con 0x7fdca01603d0 2023-07-14T11:57:35.711 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.719+0000 7fdc9f7fe700 1 -- 172.21.15.104:0/2003135642 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c011d40 con 0x7fdca01603d0 2023-07-14T11:57:35.712 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.719+0000 7fdca582f700 1 -- 172.21.15.104:0/2003135642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdca01603d0 msgr2=0x7fdca006dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:35.712 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.719+0000 7fdca582f700 1 --2- 172.21.15.104:0/2003135642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdca01603d0 0x7fdca006dd10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009850 tx=0x7fdc8c009b60 comp rx=0 tx=0).stop 2023-07-14T11:57:35.712 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.719+0000 7fdca582f700 1 -- 172.21.15.104:0/2003135642 shutdown_connections 2023-07-14T11:57:35.712 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.719+0000 7fdca582f700 1 --2- 172.21.15.104:0/2003135642 >> v2:172.21.15.121:3300/0 conn(0x7fdca00cd7f0 0x7fdca00c0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.713 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.719+0000 7fdca582f700 1 --2- 172.21.15.104:0/2003135642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdca01603d0 0x7fdca006dd10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.713 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.719+0000 7fdca582f700 1 -- 172.21.15.104:0/2003135642 >> 172.21.15.104:0/2003135642 conn(0x7fdca00afc60 msgr2=0x7fdca00b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:35.713 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.719+0000 7fdca582f700 1 -- 172.21.15.104:0/2003135642 shutdown_connections 2023-07-14T11:57:35.713 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.719+0000 7fdca582f700 1 -- 172.21.15.104:0/2003135642 wait complete. 2023-07-14T11:57:35.715 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.723+0000 7fdca582f700 1 Processor -- start 2023-07-14T11:57:35.715 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.723+0000 7fdca582f700 1 -- start start 2023-07-14T11:57:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca582f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca00cd7f0 0x7fdca0153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca582f700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdca0160000 0x7fdca00b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca582f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdca01603d0 0x7fdca015c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca582f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca00b2470 con 0x7fdca00cd7f0 2023-07-14T11:57:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca582f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca00b1860 con 0x7fdca01603d0 2023-07-14T11:57:35.717 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca582f700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca00b1b40 con 0x7fdca0160000 2023-07-14T11:57:35.717 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca482d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca00cd7f0 0x7fdca0153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:35.717 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca482d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca00cd7f0 0x7fdca0153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52116/0 (socket says 172.21.15.104:52116) 2023-07-14T11:57:35.717 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca482d700 1 -- 172.21.15.104:0/1899508296 learned_addr learned my addr 172.21.15.104:0/1899508296 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:35.717 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca502e700 1 --2- 172.21.15.104:0/1899508296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdca01603d0 0x7fdca015c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:35.718 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdc9ffff700 1 --2- 172.21.15.104:0/1899508296 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdca0160000 0x7fdca00b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:35.718 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca482d700 1 -- 172.21.15.104:0/1899508296 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdca0160000 msgr2=0x7fdca00b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:35.718 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca482d700 1 --2- 172.21.15.104:0/1899508296 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdca0160000 0x7fdca00b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.718 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca482d700 1 -- 172.21.15.104:0/1899508296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdca01603d0 msgr2=0x7fdca015c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:35.718 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca482d700 1 --2- 172.21.15.104:0/1899508296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdca01603d0 0x7fdca015c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.719 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca482d700 1 -- 172.21.15.104:0/1899508296 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98009b30 con 0x7fdca00cd7f0 2023-07-14T11:57:35.719 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca482d700 1 --2- 172.21.15.104:0/1899508296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca00cd7f0 0x7fdca0153d60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400c750 tx=0x7fdc9400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:35.719 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdc9d7fa700 1 -- 172.21.15.104:0/1899508296 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94004ca0 con 0x7fdca00cd7f0 2023-07-14T11:57:35.719 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdc9d7fa700 1 -- 172.21.15.104:0/1899508296 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc94004e00 con 0x7fdca00cd7f0 2023-07-14T11:57:35.719 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca582f700 1 -- 172.21.15.104:0/1899508296 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8c0094b0 con 0x7fdca00cd7f0 2023-07-14T11:57:35.719 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdc9d7fa700 1 -- 172.21.15.104:0/1899508296 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94007310 con 0x7fdca00cd7f0 2023-07-14T11:57:35.720 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.727+0000 7fdca582f700 1 -- 172.21.15.104:0/1899508296 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca00b2100 con 0x7fdca00cd7f0 2023-07-14T11:57:35.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.731+0000 7fdca582f700 1 -- 172.21.15.104:0/1899508296 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca0071240 con 0x7fdca00cd7f0 2023-07-14T11:57:35.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.731+0000 7fdc9d7fa700 1 -- 172.21.15.104:0/1899508296 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fdc94007470 con 0x7fdca00cd7f0 2023-07-14T11:57:35.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.731+0000 7fdc9d7fa700 1 --2- 172.21.15.104:0/1899508296 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdc840662f0 0x7fdc840687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:35.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.735+0000 7fdc9d7fa700 1 -- 172.21.15.104:0/1899508296 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fdc94083600 con 0x7fdca00cd7f0 2023-07-14T11:57:35.721 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.735+0000 7fdc9ffff700 1 --2- 172.21.15.104:0/1899508296 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdc840662f0 0x7fdc840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:35.722 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.735+0000 7fdc9ffff700 1 --2- 172.21.15.104:0/1899508296 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdc840662f0 0x7fdc840687a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0090b0 tx=0x7fdc8c01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:35.726 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.739+0000 7fdc9d7fa700 1 -- 172.21.15.104:0/1899508296 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7fdca0071240 con 0x7fdca00cd7f0 2023-07-14T11:57:35.900 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.915+0000 7fdca582f700 1 -- 172.21.15.104:0/1899508296 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fdca0071240 con 0x7fdca00cd7f0 2023-07-14T11:57:35.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.923+0000 7fdc9d7fa700 1 -- 172.21.15.104:0/1899508296 <== mon.0 v2:172.21.15.104:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdc94050c40 con 0x7fdca00cd7f0 2023-07-14T11:57:35.908 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.923+0000 7fdc9d7fa700 1 -- 172.21.15.104:0/1899508296 <== mon.0 v2:172.21.15.104:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7fdca0071240 con 0x7fdca00cd7f0 2023-07-14T11:57:35.912 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.927+0000 7fdca582f700 1 -- 172.21.15.104:0/1899508296 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdc840662f0 msgr2=0x7fdc840687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:35.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.927+0000 7fdca582f700 1 --2- 172.21.15.104:0/1899508296 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdc840662f0 0x7fdc840687a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0090b0 tx=0x7fdc8c01c040 comp rx=0 tx=0).stop 2023-07-14T11:57:35.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.927+0000 7fdca582f700 1 -- 172.21.15.104:0/1899508296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca00cd7f0 msgr2=0x7fdca0153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:35.913 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.927+0000 7fdca582f700 1 --2- 172.21.15.104:0/1899508296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca00cd7f0 0x7fdca0153d60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400c750 tx=0x7fdc9400ca60 comp rx=0 tx=0).stop 2023-07-14T11:57:35.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.927+0000 7fdca582f700 1 -- 172.21.15.104:0/1899508296 shutdown_connections 2023-07-14T11:57:35.914 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.927+0000 7fdca582f700 1 --2- 172.21.15.104:0/1899508296 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7fdc840662f0 0x7fdc840687a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.927+0000 7fdca582f700 1 --2- 172.21.15.104:0/1899508296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdca01603d0 0x7fdca015c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.927+0000 7fdca582f700 1 --2- 172.21.15.104:0/1899508296 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdca0160000 0x7fdca00b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.927+0000 7fdca582f700 1 --2- 172.21.15.104:0/1899508296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca00cd7f0 0x7fdca0153d60 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:35.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.927+0000 7fdca582f700 1 -- 172.21.15.104:0/1899508296 >> 172.21.15.104:0/1899508296 conn(0x7fdca00afc60 msgr2=0x7fdca00cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:35.915 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.927+0000 7fdca582f700 1 -- 172.21.15.104:0/1899508296 shutdown_connections 2023-07-14T11:57:35.916 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:35.927+0000 7fdca582f700 1 -- 172.21.15.104:0/1899508296 wait complete. 2023-07-14T11:57:35.934 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-14T11:57:35.935 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:57:36.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.123+0000 7f3053add700 1 Processor -- start 2023-07-14T11:57:36.109 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.123+0000 7f3053add700 1 -- start start 2023-07-14T11:57:36.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.123+0000 7f3053add700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f304c160640 0x7f304c164000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:36.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.123+0000 7f3053add700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f304c164540 0x7f304c070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:36.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.123+0000 7f3053add700 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304c0c8900 con 0x7f304c0cddb0 2023-07-14T11:57:36.111 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.123+0000 7f3053add700 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304c0c8fc0 con 0x7f304c160640 2023-07-14T11:57:36.112 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3053add700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304c070540 con 0x7f304c164540 2023-07-14T11:57:36.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3051879700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f304c160640 0x7f304c164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:36.114 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3051879700 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f304c160640 0x7f304c164000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52124/0 (socket says 172.21.15.104:52124) 2023-07-14T11:57:36.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3051879700 1 -- 172.21.15.104:0/4032827726 learned_addr learned my addr 172.21.15.104:0/4032827726 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:36.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3051078700 1 --2- 172.21.15.104:0/4032827726 >> v2:172.21.15.121:3300/0 conn(0x7f304c164540 0x7f304c070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:36.115 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3051879700 1 -- 172.21.15.104:0/4032827726 >> v2:172.21.15.121:3300/0 conn(0x7f304c164540 msgr2=0x7f304c070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:57:36.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3051879700 1 --2- 172.21.15.104:0/4032827726 >> v2:172.21.15.121:3300/0 conn(0x7f304c164540 0x7f304c070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:36.116 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3051879700 1 -- 172.21.15.104:0/4032827726 >> v1:172.21.15.104:6789/0 conn(0x7f304c0cddb0 legacy=0x7f304c0afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:36.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3051879700 1 -- 172.21.15.104:0/4032827726 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c0caae0 con 0x7f304c160640 2023-07-14T11:57:36.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3051879700 1 --2- 172.21.15.104:0/4032827726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304c160640 0x7f304c164000 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f304c06bc70 tx=0x7f3048009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b63f99b73a313f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:36.117 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3050877700 1 -- 172.21.15.104:0/4032827726 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048011830 con 0x7f304c160640 2023-07-14T11:57:36.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3050877700 1 -- 172.21.15.104:0/4032827726 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3048011990 con 0x7f304c160640 2023-07-14T11:57:36.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3050877700 1 -- 172.21.15.104:0/4032827726 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048011d60 con 0x7f304c160640 2023-07-14T11:57:36.118 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3053add700 1 -- 172.21.15.104:0/4032827726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304c160640 msgr2=0x7f304c164000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:36.119 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3053add700 1 --2- 172.21.15.104:0/4032827726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304c160640 0x7f304c164000 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f304c06bc70 tx=0x7f3048009c20 comp rx=0 tx=0).stop 2023-07-14T11:57:36.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3053add700 1 -- 172.21.15.104:0/4032827726 shutdown_connections 2023-07-14T11:57:36.123 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3053add700 1 --2- 172.21.15.104:0/4032827726 >> v2:172.21.15.121:3300/0 conn(0x7f304c164540 0x7f304c070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:36.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3053add700 1 --2- 172.21.15.104:0/4032827726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304c160640 0x7f304c164000 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:36.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3053add700 1 -- 172.21.15.104:0/4032827726 >> 172.21.15.104:0/4032827726 conn(0x7f304c0cad30 msgr2=0x7f304c0cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:36.124 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3053add700 1 -- 172.21.15.104:0/4032827726 shutdown_connections 2023-07-14T11:57:36.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.127+0000 7f3053add700 1 -- 172.21.15.104:0/4032827726 wait complete. 2023-07-14T11:57:36.125 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.131+0000 7f3053add700 1 Processor -- start 2023-07-14T11:57:36.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.131+0000 7f3053add700 1 -- start start 2023-07-14T11:57:36.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.131+0000 7f3053add700 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f304c0cddb0 0x7f304c1f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:36.126 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.131+0000 7f3053add700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f304c160640 0x7f304c161c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:36.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.131+0000 7f3053add700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304c164540 0x7f304c0c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:36.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.131+0000 7f3053add700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1586b0 con 0x7f304c164540 2023-07-14T11:57:36.127 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.131+0000 7f3053add700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c160b90 con 0x7f304c160640 2023-07-14T11:57:36.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.131+0000 7f3053add700 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c160e70 con 0x7f304c0cddb0 2023-07-14T11:57:36.128 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f305207a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304c164540 0x7f304c0c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:36.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f305207a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304c164540 0x7f304c0c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52134/0 (socket says 172.21.15.104:52134) 2023-07-14T11:57:36.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f305207a700 1 -- 172.21.15.104:0/4138659151 learned_addr learned my addr 172.21.15.104:0/4138659151 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-14T11:57:36.129 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f3051078700 1 --2- 172.21.15.104:0/4138659151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f304c160640 0x7f304c161c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:36.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f3051879700 1 --2- 172.21.15.104:0/4138659151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f304c0cddb0 0x7f304c1f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:36.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f305207a700 1 -- 172.21.15.104:0/4138659151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f304c0cddb0 msgr2=0x7f304c1f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:36.130 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f305207a700 1 --2- 172.21.15.104:0/4138659151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f304c0cddb0 0x7f304c1f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:36.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f305207a700 1 -- 172.21.15.104:0/4138659151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f304c160640 msgr2=0x7f304c161c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:36.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f305207a700 1 --2- 172.21.15.104:0/4138659151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f304c160640 0x7f304c161c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:36.131 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f305207a700 1 -- 172.21.15.104:0/4138659151 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3048009570 con 0x7f304c164540 2023-07-14T11:57:36.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f305207a700 1 --2- 172.21.15.104:0/4138659151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304c164540 0x7f304c0c5d60 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f304400cc50 tx=0x7f304400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:36.132 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f30427fc700 1 -- 172.21.15.104:0/4138659151 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044003c20 con 0x7f304c164540 2023-07-14T11:57:36.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f30427fc700 1 -- 172.21.15.104:0/4138659151 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3044003d80 con 0x7f304c164540 2023-07-14T11:57:36.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f3053add700 1 -- 172.21.15.104:0/4138659151 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c161150 con 0x7f304c164540 2023-07-14T11:57:36.133 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f30427fc700 1 -- 172.21.15.104:0/4138659151 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304400aad0 con 0x7f304c164540 2023-07-14T11:57:36.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f3053add700 1 -- 172.21.15.104:0/4138659151 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c161710 con 0x7f304c164540 2023-07-14T11:57:36.134 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.135+0000 7f3053add700 1 -- 172.21.15.104:0/4138659151 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304c04dcb0 con 0x7f304c164540 2023-07-14T11:57:36.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.139+0000 7f30427fc700 1 -- 172.21.15.104:0/4138659151 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f304400ac30 con 0x7f304c164540 2023-07-14T11:57:36.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.139+0000 7f30427fc700 1 --2- 172.21.15.104:0/4138659151 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3030066320 0x7f30300687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:36.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.139+0000 7f30427fc700 1 -- 172.21.15.104:0/4138659151 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f3044083b10 con 0x7f304c164540 2023-07-14T11:57:36.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.139+0000 7f3051879700 1 --2- 172.21.15.104:0/4138659151 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3030066320 0x7f30300687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:36.136 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.143+0000 7f3051879700 1 --2- 172.21.15.104:0/4138659151 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3030066320 0x7f30300687d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3048000c00 tx=0x7f3048008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:36.137 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.147+0000 7f30427fc700 1 -- 172.21.15.104:0/4138659151 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169738 (secure 0 0 0) 0x7f3044051180 con 0x7f304c164540 2023-07-14T11:57:36.342 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.355+0000 7f3053add700 1 -- 172.21.15.104:0/4138659151 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f304c0cb9f0 con 0x7f304c164540 2023-07-14T11:57:36.343 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.359+0000 7f30427fc700 1 -- 172.21.15.104:0/4138659151 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f30440547a0 con 0x7f304c164540 2023-07-14T11:57:36.344 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-14T11:57:36.344 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/4186544624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":4186544624},{"type":"v1","addr":"172.21.15.104:6837","nonce":4186544624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4234575863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4234575863},{"type":"v1","addr":"172.21.15.121:6835","nonce":4234575863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6220,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4127455794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4127455794},{"type":"v1","addr":"172.21.15.121:6833","nonce":4127455794}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6795,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/1586732128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1586732128},{"type":"v1","addr":"172.21.15.104:6836","nonce":1586732128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[]} 2023-07-14T11:57:36.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.359+0000 7f3053add700 1 -- 172.21.15.104:0/4138659151 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3030066320 msgr2=0x7f30300687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:36.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.359+0000 7f3053add700 1 --2- 172.21.15.104:0/4138659151 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3030066320 0x7f30300687d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3048000c00 tx=0x7f3048008030 comp rx=0 tx=0).stop 2023-07-14T11:57:36.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.359+0000 7f3053add700 1 -- 172.21.15.104:0/4138659151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304c164540 msgr2=0x7f304c0c5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:36.346 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.359+0000 7f3053add700 1 --2- 172.21.15.104:0/4138659151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304c164540 0x7f304c0c5d60 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f304400cc50 tx=0x7f304400cf60 comp rx=0 tx=0).stop 2023-07-14T11:57:36.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.359+0000 7f3053add700 1 -- 172.21.15.104:0/4138659151 shutdown_connections 2023-07-14T11:57:36.347 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.359+0000 7f3053add700 1 --2- 172.21.15.104:0/4138659151 >> [v2:172.21.15.104:6832/608103909,v1:172.21.15.104:6833/608103909] conn(0x7f3030066320 0x7f30300687d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:36.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.359+0000 7f3053add700 1 --2- 172.21.15.104:0/4138659151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304c164540 0x7f304c0c5d60 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:36.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.359+0000 7f3053add700 1 --2- 172.21.15.104:0/4138659151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f304c160640 0x7f304c161c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:36.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.359+0000 7f3053add700 1 --2- 172.21.15.104:0/4138659151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f304c0cddb0 0x7f304c1f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:36.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.359+0000 7f3053add700 1 -- 172.21.15.104:0/4138659151 >> 172.21.15.104:0/4138659151 conn(0x7f304c0cad30 msgr2=0x7f304c0b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:36.348 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.359+0000 7f3053add700 1 -- 172.21.15.104:0/4138659151 shutdown_connections 2023-07-14T11:57:36.349 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-14T11:57:36.359+0000 7f3053add700 1 -- 172.21.15.104:0/4138659151 wait complete. 2023-07-14T11:57:36.349 INFO:teuthology.orchestra.run.smithi104.stderr:dumped fsmap epoch 29 2023-07-14T11:57:36.368 DEBUG:tasks.cephfs.filesystem:already dead... 2023-07-14T11:57:36.368 INFO:teuthology.misc:Shutting down mds daemons... 2023-07-14T11:57:36.369 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-14T11:57:36.369 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:36.392 INFO:tasks.ceph.mds.a:Stopped 2023-07-14T11:57:36.392 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-14T11:57:36.393 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:36.459 INFO:tasks.ceph.mds.c:Stopped 2023-07-14T11:57:36.460 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-14T11:57:36.461 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:36.482 INFO:tasks.ceph.mds.b:Stopped 2023-07-14T11:57:36.482 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-14T11:57:36.482 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:36.543 INFO:tasks.ceph.mds.d:Stopped 2023-07-14T11:57:36.544 INFO:teuthology.misc:Shutting down osd daemons... 2023-07-14T11:57:36.544 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-07-14T11:57:36.544 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:36.639 INFO:tasks.ceph.osd.0:Stopped 2023-07-14T11:57:36.639 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-07-14T11:57:36.640 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:36.721 INFO:tasks.ceph.osd.1:Stopped 2023-07-14T11:57:36.722 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-07-14T11:57:36.722 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:36.816 INFO:tasks.ceph.osd.2:Stopped 2023-07-14T11:57:36.817 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-07-14T11:57:36.817 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:36.889 INFO:tasks.ceph.osd.3:Stopped 2023-07-14T11:57:36.890 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-07-14T11:57:36.890 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:36.985 INFO:tasks.ceph.osd.4:Stopped 2023-07-14T11:57:36.986 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-07-14T11:57:36.986 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:37.074 INFO:tasks.ceph.osd.5:Stopped 2023-07-14T11:57:37.075 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-07-14T11:57:37.075 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:38.298 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:57:38.305 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:57:39.171 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-07-14T11:57:43.177 INFO:tasks.ceph.osd.6:Stopped 2023-07-14T11:57:43.178 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-07-14T11:57:43.178 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:44.676 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-07-14T11:57:44.677 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-07-14T11:57:49.280 INFO:tasks.ceph.osd.7:Stopped 2023-07-14T11:57:49.280 INFO:teuthology.misc:Shutting down mgr daemons... 2023-07-14T11:57:49.281 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-07-14T11:57:49.281 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:49.344 INFO:tasks.ceph.mgr.y:Stopped 2023-07-14T11:57:49.345 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-07-14T11:57:49.345 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:49.406 INFO:tasks.ceph.mgr.x:Stopped 2023-07-14T11:57:49.406 INFO:teuthology.misc:Shutting down mon daemons... 2023-07-14T11:57:49.407 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-07-14T11:57:49.407 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:49.442 INFO:tasks.ceph.mon.a:Stopped 2023-07-14T11:57:49.443 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-07-14T11:57:49.443 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:49.474 INFO:tasks.ceph.mon.b:Stopped 2023-07-14T11:57:49.475 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-07-14T11:57:49.475 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T11:57:49.545 INFO:tasks.ceph.mon.c:Stopped 2023-07-14T11:57:49.545 INFO:tasks.ceph:Checking cluster log for badness... 2023-07-14T11:57:49.546 DEBUG:teuthology.orchestra.run.smithi104:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-07-14T11:57:49.589 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:57:49.590 DEBUG:teuthology.orchestra.run.smithi104:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-07-14T11:57:49.678 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-07-14T11:57:50.156 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:57:50.157 DEBUG:teuthology.orchestra.run.smithi104:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-07-14T11:57:50.648 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:57:50.648 DEBUG:teuthology.orchestra.run.smithi104:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-07-14T11:57:51.095 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:57:51.096 DEBUG:teuthology.orchestra.run.smithi104:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-07-14T11:57:51.492 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:57:51.493 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-07-14T11:57:51.897 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:57:51.898 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-07-14T11:57:52.308 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:57:52.309 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-07-14T11:57:52.760 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:57:52.761 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-07-14T11:57:53.233 INFO:tasks.ceph:Archiving mon data... 2023-07-14T11:57:53.235 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435/data/mon.a.tgz 2023-07-14T11:57:53.236 DEBUG:teuthology.orchestra.run.smithi104:> mktemp 2023-07-14T11:57:53.257 INFO:teuthology.orchestra.run.smithi104.stdout:/tmp/tmp.ex1pvNjt01 2023-07-14T11:57:53.257 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ex1pvNjt01 2023-07-14T11:57:53.439 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0666 /tmp/tmp.ex1pvNjt01 2023-07-14T11:57:53.540 DEBUG:teuthology.orchestra.remote:smithi104:/tmp/tmp.ex1pvNjt01 is 390KB 2023-07-14T11:57:53.572 DEBUG:teuthology.orchestra.run.smithi104:> rm -fr /tmp/tmp.ex1pvNjt01 2023-07-14T11:57:53.588 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435/data/mon.b.tgz 2023-07-14T11:57:53.588 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2023-07-14T11:57:53.605 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.QGZePYNuzT 2023-07-14T11:57:53.605 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.QGZePYNuzT 2023-07-14T11:57:53.806 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.QGZePYNuzT 2023-07-14T11:57:53.899 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.QGZePYNuzT is 507KB 2023-07-14T11:57:53.978 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.QGZePYNuzT 2023-07-14T11:57:53.997 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435/data/mon.c.tgz 2023-07-14T11:57:53.997 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2023-07-14T11:57:54.059 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.FLezE2NE3n 2023-07-14T11:57:54.060 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.FLezE2NE3n 2023-07-14T11:57:54.260 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.FLezE2NE3n 2023-07-14T11:57:54.351 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.FLezE2NE3n is 507KB 2023-07-14T11:57:54.426 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.FLezE2NE3n 2023-07-14T11:57:54.446 INFO:tasks.ceph:Cleaning ceph cluster... 2023-07-14T11:57:54.446 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-07-14T11:57:54.454 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-07-14T11:57:54.521 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-07-14T11:57:54.522 DEBUG:teuthology.orchestra.run.smithi104:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-14T11:57:54.533 DEBUG:teuthology.orchestra.run.smithi121:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-14T11:57:54.581 INFO:teuthology.orchestra.run.smithi104.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-07-14T11:57:54.618 INFO:teuthology.orchestra.run.smithi121.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-07-14T11:57:54.631 INFO:tasks.ceph:Archiving crash dumps... 2023-07-14T11:57:54.633 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435/remote/smithi104/crash 2023-07-14T11:57:54.634 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-14T11:57:54.671 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435/remote/smithi121/crash 2023-07-14T11:57:54.672 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-14T11:57:54.706 INFO:tasks.ceph:Shutting down logrotate 2023-07-14T11:57:54.707 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:57:54.721 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-14T11:57:54.827 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-07-14T11:57:54.859 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-07-14T11:57:54.893 INFO:tasks.ceph:Compressing logs... 2023-07-14T11:57:54.894 DEBUG:teuthology.orchestra.run.smithi104:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T11:57:54.909 DEBUG:teuthology.orchestra.run.smithi121:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T11:57:54.948 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51708.log 2023-07-14T11:57:54.950 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51807.log 2023-07-14T11:57:54.952 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46839.log 2023-07-14T11:57:54.953 INFO:teuthology.orchestra.run.smithi104.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51708.log.gz 2023-07-14T11:57:54.955 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41906.log 2023-07-14T11:57:54.956 INFO:teuthology.orchestra.run.smithi104.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51807.log.gz 2023-07-14T11:57:54.957 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47192.log 2023-07-14T11:57:54.958 INFO:teuthology.orchestra.run.smithi104.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46839.log.gz 2023-07-14T11:57:54.959 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41825.log 2023-07-14T11:57:54.960 INFO:teuthology.orchestra.run.smithi104.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41906.log.gz 2023-07-14T11:57:54.961 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47192.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47192.log.gz 2023-07-14T11:57:54.962 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44829.log 2023-07-14T11:57:54.963 INFO:teuthology.orchestra.run.smithi104.stderr:gzip /var/log/ceph/ceph-client.admin.41825.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50258.log 2023-07-14T11:57:54.964 INFO:teuthology.orchestra.run.smithi104.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41825.log.gz 2023-07-14T11:57:54.965 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48590.log 2023-07-14T11:57:54.965 INFO:teuthology.orchestra.run.smithi104.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44829.log.gz 2023-07-14T11:57:54.966 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50258.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50258.log.gz 2023-07-14T11:57:54.966 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43924.log 2023-07-14T11:57:54.967 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48590.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48590.log.gz 2023-07-14T11:57:54.968 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49254.log 2023-07-14T11:57:54.969 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43924.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43924.log.gz 2023-07-14T11:57:54.969 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41798.log 2023-07-14T11:57:54.971 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50071.log 2023-07-14T11:57:54.972 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49254.log: /var/log/ceph/ceph-client.admin.41798.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49254.log.gz 2023-07-14T11:57:54.972 INFO:teuthology.orchestra.run.smithi104.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41798.log.gz 2023-07-14T11:57:54.973 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42329.log 2023-07-14T11:57:54.974 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42770.log 2023-07-14T11:57:54.976 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.0.50071.log: /var/log/ceph/ceph-client.admin.42329.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42329.log.gz 2023-07-14T11:57:54.976 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48068.log 2023-07-14T11:57:54.977 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42770.log: 91.9% -- replaced with /var/log/ceph/ceph-client.0.50071.log.gz 2023-07-14T11:57:54.978 INFO:teuthology.orchestra.run.smithi104.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42770.log.gz 2023-07-14T11:57:54.978 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46054.log 2023-07-14T11:57:54.979 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48068.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48068.log.gz 2023-07-14T11:57:54.980 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43654.log 2023-07-14T11:57:54.981 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46054.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46054.log.gz 2023-07-14T11:57:54.982 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-07-14T11:57:54.983 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46389.log 2023-07-14T11:57:54.983 INFO:teuthology.orchestra.run.smithi104.stderr: 88.6%/var/log/ceph/ceph-mon.a.log: -- replaced with /var/log/ceph/ceph-client.admin.43654.log.gz 2023-07-14T11:57:54.984 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47572.log 2023-07-14T11:57:54.985 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46154.log 2023-07-14T11:57:54.986 INFO:teuthology.orchestra.run.smithi104.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46389.log.gz 2023-07-14T11:57:54.986 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35104.log 2023-07-14T11:57:54.987 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47572.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47572.log.gz 2023-07-14T11:57:54.987 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46027.log 2023-07-14T11:57:54.988 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-07-14T11:57:54.989 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46154.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46154.log.gz 2023-07-14T11:57:54.989 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42662.log 2023-07-14T11:57:54.990 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.35104.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35104.log.gz 2023-07-14T11:57:54.990 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40306.log 2023-07-14T11:57:54.991 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34915.log 2023-07-14T11:57:54.991 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46027.log: /var/log/ceph/ceph-client.admin.42662.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46027.log.gz 2023-07-14T11:57:54.992 INFO:teuthology.orchestra.run.smithi104.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42662.log.gz 2023-07-14T11:57:54.992 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41223.log 2023-07-14T11:57:54.993 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-07-14T11:57:54.994 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52348.log 2023-07-14T11:57:54.995 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41223.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40306.log.gz 2023-07-14T11:57:54.995 INFO:teuthology.orchestra.run.smithi104.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41223.log.gz 2023-07-14T11:57:54.995 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.34915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34968.log 2023-07-14T11:57:54.996 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34915.log.gz 2023-07-14T11:57:54.997 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45919.log 2023-07-14T11:57:54.997 INFO:teuthology.orchestra.run.smithi121.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-07-14T11:57:54.997 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-07-14T11:57:54.998 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.52348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51168.log 2023-07-14T11:57:54.998 INFO:teuthology.orchestra.run.smithi104.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52348.log.gz 2023-07-14T11:57:54.999 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.34968.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-07-14T11:57:54.999 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45919.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45919.log.gz 2023-07-14T11:57:54.999 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44775.log 2023-07-14T11:57:55.000 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34968.log.gz 2023-07-14T11:57:55.001 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35023.log 2023-07-14T11:57:55.001 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49910.log 2023-07-14T11:57:55.001 INFO:teuthology.orchestra.run.smithi104.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51168.log.gz 2023-07-14T11:57:55.002 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44775.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44775.log.gz 2023-07-14T11:57:55.002 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49058.log 2023-07-14T11:57:55.003 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-07-14T11:57:55.003 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52268.log 2023-07-14T11:57:55.004 INFO:teuthology.orchestra.run.smithi104.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49910.log.gz 2023-07-14T11:57:55.005 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.35023.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35023.log.gz 2023-07-14T11:57:55.005 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-07-14T11:57:55.005 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49058.log: gzip -5 --verbose -- 87.9%/var/log/ceph/ceph-client.admin.51473.log 2023-07-14T11:57:55.006 INFO:teuthology.orchestra.run.smithi104.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49058.log.gz 2023-07-14T11:57:55.007 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34995.log 2023-07-14T11:57:55.007 INFO:teuthology.orchestra.run.smithi121.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-07-14T11:57:55.007 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.52268.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52268.log.gz 2023-07-14T11:57:55.007 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50448.log 2023-07-14T11:57:55.008 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-07-14T11:57:55.009 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48536.log 2023-07-14T11:57:55.009 INFO:teuthology.orchestra.run.smithi104.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51473.log.gz 2023-07-14T11:57:55.010 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50448.log: gzip -5 --verbose -- 88.9%/var/log/ceph/ceph-client.admin.46971.log -- replaced with /var/log/ceph/ceph-client.admin.50448.log.gz 2023-07-14T11:57:55.010 INFO:teuthology.orchestra.run.smithi104.stderr: 2023-07-14T11:57:55.010 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.34995.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34995.log.gz 2023-07-14T11:57:55.011 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35076.log 2023-07-14T11:57:55.011 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.6.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-07-14T11:57:55.012 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39527.log 2023-07-14T11:57:55.013 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.35076.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-07-14T11:57:55.013 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35076.log.gz 2023-07-14T11:57:55.014 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48536.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48536.log.gz 2023-07-14T11:57:55.014 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51917.log 2023-07-14T11:57:55.014 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46971.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46971.log.gz 2023-07-14T11:57:55.015 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43040.log 2023-07-14T11:57:55.016 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51917.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51917.log.gz 2023-07-14T11:57:55.017 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49380.log 2023-07-14T11:57:55.018 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.3.39527.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-07-14T11:57:55.019 INFO:teuthology.orchestra.run.smithi121.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.3.39527.log.gz 2023-07-14T11:57:55.019 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35051.log 2023-07-14T11:57:55.020 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-07-14T11:57:55.020 INFO:teuthology.orchestra.run.smithi121.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-07-14T11:57:55.021 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.35051.log: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2023-07-14T11:57:55.021 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.35051.log.gz 2023-07-14T11:57:55.022 INFO:teuthology.orchestra.run.smithi121.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.34941.log 2023-07-14T11:57:55.025 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-07-14T11:57:55.025 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.34941.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39418.log 2023-07-14T11:57:55.026 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34941.log.gz 2023-07-14T11:57:55.027 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43040.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.43040.log.gz 2023-07-14T11:57:55.028 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40964.log 2023-07-14T11:57:55.028 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44113.log 2023-07-14T11:57:55.028 INFO:teuthology.orchestra.run.smithi104.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49380.log.gz 2023-07-14T11:57:55.028 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47878.log 2023-07-14T11:57:55.029 INFO:teuthology.orchestra.run.smithi104.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40964.log.gz 2023-07-14T11:57:55.029 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42328.log 2023-07-14T11:57:55.030 INFO:teuthology.orchestra.run.smithi104.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44113.log.gz 2023-07-14T11:57:55.030 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47878.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47878.log.gz 2023-07-14T11:57:55.030 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49085.log 2023-07-14T11:57:55.031 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42328.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42328.log.gz 2023-07-14T11:57:55.031 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-07-14T11:57:55.031 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49085.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49085.log.gz 2023-07-14T11:57:55.031 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48127.log 2023-07-14T11:57:55.032 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-client.1.39418.log: 91.0% -- replaced with /var/log/ceph/ceph-client.1.39418.log.gz 2023-07-14T11:57:55.033 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45458.log 2023-07-14T11:57:55.033 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48127.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48127.log.gz 2023-07-14T11:57:55.033 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49166.log 2023-07-14T11:57:55.034 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43359.log 2023-07-14T11:57:55.034 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49166.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45458.log.gz 2023-07-14T11:57:55.034 INFO:teuthology.orchestra.run.smithi104.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49166.log.gz 2023-07-14T11:57:55.035 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50589.log 2023-07-14T11:57:55.035 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.2.43359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41015.log 2023-07-14T11:57:55.036 INFO:teuthology.orchestra.run.smithi104.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.2.43359.log.gz 2023-07-14T11:57:55.037 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43708.log 2023-07-14T11:57:55.038 INFO:teuthology.orchestra.run.smithi104.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50589.log.gz 2023-07-14T11:57:55.039 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50861.log 2023-07-14T11:57:55.040 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41015.log: /var/log/ceph/ceph-client.admin.43708.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43708.log.gz 89.8% 2023-07-14T11:57:55.041 INFO:teuthology.orchestra.run.smithi104.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41015.log.gz 2023-07-14T11:57:55.041 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45541.log 2023-07-14T11:57:55.043 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40802.log 2023-07-14T11:57:55.043 INFO:teuthology.orchestra.run.smithi104.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50861.log.gz 2023-07-14T11:57:55.044 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45377.log 2023-07-14T11:57:55.044 INFO:teuthology.orchestra.run.smithi104.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45541.log.gz 2023-07-14T11:57:55.046 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40802.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40802.log.gz 2023-07-14T11:57:55.046 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41277.log 2023-07-14T11:57:55.046 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45377.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45377.log.gz 2023-07-14T11:57:55.047 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45811.log 2023-07-14T11:57:55.048 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45676.log 2023-07-14T11:57:55.049 INFO:teuthology.orchestra.run.smithi104.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41277.log.gz 2023-07-14T11:57:55.050 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43681.log 2023-07-14T11:57:55.051 INFO:teuthology.orchestra.run.smithi104.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45811.log.gz 2023-07-14T11:57:55.051 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45676.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45676.log.gz 2023-07-14T11:57:55.052 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-07-14T11:57:55.053 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43681.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43681.log.gz 2023-07-14T11:57:55.053 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42112.log 2023-07-14T11:57:55.054 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49435.log 2023-07-14T11:57:55.055 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42112.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42112.log.gz 2023-07-14T11:57:55.055 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48509.log 2023-07-14T11:57:55.056 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44445.log 2023-07-14T11:57:55.057 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49435.log: /var/log/ceph/ceph-client.admin.48509.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49435.log.gz 2023-07-14T11:57:55.057 INFO:teuthology.orchestra.run.smithi104.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48509.log.gz 2023-07-14T11:57:55.058 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41358.log 2023-07-14T11:57:55.058 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47410.log 2023-07-14T11:57:55.059 INFO:teuthology.orchestra.run.smithi104.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44445.log.gz 2023-07-14T11:57:55.060 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41358.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41358.log.gz 2023-07-14T11:57:55.060 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51114.log 2023-07-14T11:57:55.061 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50394.log 2023-07-14T11:57:55.062 INFO:teuthology.orchestra.run.smithi104.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47410.log.gz 2023-07-14T11:57:55.062 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51114.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51114.log.gz 2023-07-14T11:57:55.063 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42554.log 2023-07-14T11:57:55.064 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44666.log 2023-07-14T11:57:55.065 INFO:teuthology.orchestra.run.smithi104.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50394.log.gz 2023-07-14T11:57:55.066 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42554.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42554.log.gz 2023-07-14T11:57:55.066 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52133.log 2023-07-14T11:57:55.067 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48014.log 2023-07-14T11:57:55.068 INFO:teuthology.orchestra.run.smithi104.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44666.log.gz 2023-07-14T11:57:55.069 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.52133.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52133.log.gz 2023-07-14T11:57:55.069 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50947.log 2023-07-14T11:57:55.071 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40682.log 2023-07-14T11:57:55.071 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48014.log: /var/log/ceph/ceph-client.admin.50947.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48014.log.gz 2023-07-14T11:57:55.072 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose 88.2%-- -- replaced with /var/log/ceph/ceph-client.admin.50947.log.gz/var/log/ceph/ceph-client.admin.40368.log 2023-07-14T11:57:55.072 INFO:teuthology.orchestra.run.smithi104.stderr: 2023-07-14T11:57:55.073 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40937.log 2023-07-14T11:57:55.074 INFO:teuthology.orchestra.run.smithi104.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40682.log.gz 2023-07-14T11:57:55.074 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40368.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40368.log.gz 2023-07-14T11:57:55.075 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42115.log 2023-07-14T11:57:55.075 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49353.log 2023-07-14T11:57:55.076 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42115.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.40937.log.gz 2023-07-14T11:57:55.076 INFO:teuthology.orchestra.run.smithi104.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42115.log.gz 2023-07-14T11:57:55.077 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43978.log 2023-07-14T11:57:55.077 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50421.log 2023-07-14T11:57:55.078 INFO:teuthology.orchestra.run.smithi104.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49353.log.gz 2023-07-14T11:57:55.078 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43978.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43978.log.gz 2023-07-14T11:57:55.079 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48482.log 2023-07-14T11:57:55.080 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45649.log 2023-07-14T11:57:55.080 INFO:teuthology.orchestra.run.smithi104.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-07-14T11:57:55.081 INFO:teuthology.orchestra.run.smithi104.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50421.log.gz 2023-07-14T11:57:55.081 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48482.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48482.log.gz 2023-07-14T11:57:55.082 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47662.log 2023-07-14T11:57:55.082 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45350.log 2023-07-14T11:57:55.083 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.0.47662.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45649.log.gz 2023-07-14T11:57:55.083 INFO:teuthology.orchestra.run.smithi104.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-07-14T11:57:55.084 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50780.log 2023-07-14T11:57:55.085 INFO:teuthology.orchestra.run.smithi104.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.47662.log.gz 2023-07-14T11:57:55.085 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42959.log 2023-07-14T11:57:55.085 INFO:teuthology.orchestra.run.smithi104.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45350.log.gz 2023-07-14T11:57:55.086 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50780.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50780.log.gz 2023-07-14T11:57:55.087 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45784.log 2023-07-14T11:57:55.087 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42959.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42959.log.gz 2023-07-14T11:57:55.088 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45703.log 2023-07-14T11:57:55.089 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40212.log 2023-07-14T11:57:55.090 INFO:teuthology.orchestra.run.smithi104.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45784.log.gz 2023-07-14T11:57:55.090 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45703.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45703.log.gz 2023-07-14T11:57:55.091 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41901.log 2023-07-14T11:57:55.092 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51284.log 2023-07-14T11:57:55.092 INFO:teuthology.orchestra.run.smithi104.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40212.log.gz 2023-07-14T11:57:55.093 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52323.log 2023-07-14T11:57:55.093 INFO:teuthology.orchestra.run.smithi104.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41901.log.gz 2023-07-14T11:57:55.095 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51284.log: gzip -5 88.7%--verbose -- replaced with /var/log/ceph/ceph-client.admin.51284.log.gz-- 2023-07-14T11:57:55.095 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48877.log 2023-07-14T11:57:55.096 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.52323.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52323.log.gz 2023-07-14T11:57:55.097 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52052.log 2023-07-14T11:57:55.097 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49491.log 2023-07-14T11:57:55.099 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48877.log: /var/log/ceph/ceph-client.admin.52052.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52052.log.gz 2023-07-14T11:57:55.099 INFO:teuthology.orchestra.run.smithi104.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48877.log.gz 2023-07-14T11:57:55.100 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49802.log 2023-07-14T11:57:55.100 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49491.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49491.log.gz 2023-07-14T11:57:55.101 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45183.log 2023-07-14T11:57:55.102 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49802.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49802.log.gz 2023-07-14T11:57:55.102 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51365.log 2023-07-14T11:57:55.103 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51257.log 2023-07-14T11:57:55.104 INFO:teuthology.orchestra.run.smithi104.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45183.log.gz 2023-07-14T11:57:55.104 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51365.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51365.log.gz 2023-07-14T11:57:55.105 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49937.log 2023-07-14T11:57:55.106 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50617.log 2023-07-14T11:57:55.107 INFO:teuthology.orchestra.run.smithi104.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51257.log.gz 2023-07-14T11:57:55.108 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49937.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49937.log.gz 2023-07-14T11:57:55.108 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51554.log 2023-07-14T11:57:55.109 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50617.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50617.log.gz 2023-07-14T11:57:55.109 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-07-14T11:57:55.110 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50121.log 2023-07-14T11:57:55.111 INFO:teuthology.orchestra.run.smithi104.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51554.log.gz 2023-07-14T11:57:55.112 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47851.log 2023-07-14T11:57:55.114 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50121.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50121.log.gz 2023-07-14T11:57:55.114 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47770.log 2023-07-14T11:57:55.116 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44174.log 2023-07-14T11:57:55.117 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47851.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47851.log.gz 2023-07-14T11:57:55.118 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47770.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47770.log.gz 2023-07-14T11:57:55.118 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41467.log 2023-07-14T11:57:55.119 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50536.log 2023-07-14T11:57:55.120 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44174.log: /var/log/ceph/ceph-client.admin.41467.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44174.log.gz 2023-07-14T11:57:55.121 INFO:teuthology.orchestra.run.smithi104.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41467.log.gz 2023-07-14T11:57:55.121 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42851.log 2023-07-14T11:57:55.122 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.0.50536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48769.log 2023-07-14T11:57:55.122 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42851.log: 90.3% -- replaced with /var/log/ceph/ceph-client.0.50536.log.gz 2023-07-14T11:57:55.123 INFO:teuthology.orchestra.run.smithi104.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42851.log.gz 2023-07-14T11:57:55.123 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44693.log 2023-07-14T11:57:55.124 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42608.log 2023-07-14T11:57:55.124 INFO:teuthology.orchestra.run.smithi104.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48769.log.gz 2023-07-14T11:57:55.125 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44693.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44693.log.gz 2023-07-14T11:57:55.125 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43544.log 2023-07-14T11:57:55.126 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45047.log 2023-07-14T11:57:55.127 INFO:teuthology.orchestra.run.smithi104.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42608.log.gz 2023-07-14T11:57:55.127 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48185.log 2023-07-14T11:57:55.128 INFO:teuthology.orchestra.run.smithi104.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43544.log.gz 2023-07-14T11:57:55.129 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50231.log 2023-07-14T11:57:55.129 INFO:teuthology.orchestra.run.smithi104.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45047.log.gz 2023-07-14T11:57:55.130 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.0.48185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44282.log 2023-07-14T11:57:55.131 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50231.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.48185.log.gz 2023-07-14T11:57:55.131 INFO:teuthology.orchestra.run.smithi104.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50231.log.gz 2023-07-14T11:57:55.132 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51446.log 2023-07-14T11:57:55.132 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47518.log 2023-07-14T11:57:55.133 INFO:teuthology.orchestra.run.smithi104.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44282.log.gz 2023-07-14T11:57:55.133 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51446.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51446.log.gz 2023-07-14T11:57:55.134 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51998.log 2023-07-14T11:57:55.135 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47075.log 2023-07-14T11:57:55.136 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47518.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47518.log.gz 2023-07-14T11:57:55.136 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51998.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51998.log.gz 2023-07-14T11:57:55.137 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46362.log 2023-07-14T11:57:55.137 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47075.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47075.log.gzgzip 2023-07-14T11:57:55.138 INFO:teuthology.orchestra.run.smithi104.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.47606.log 2023-07-14T11:57:55.138 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46362.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46362.log.gz 2023-07-14T11:57:55.139 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46729.log 2023-07-14T11:57:55.140 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49193.log 2023-07-14T11:57:55.140 INFO:teuthology.orchestra.run.smithi104.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47606.log.gz 2023-07-14T11:57:55.141 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46729.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46729.log.gz 2023-07-14T11:57:55.141 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49721.log 2023-07-14T11:57:55.142 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49856.log 2023-07-14T11:57:55.143 INFO:teuthology.orchestra.run.smithi104.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49193.log.gz 2023-07-14T11:57:55.143 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49721.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49721.log.gz 2023-07-14T11:57:55.144 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46785.log 2023-07-14T11:57:55.145 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51581.log 2023-07-14T11:57:55.145 INFO:teuthology.orchestra.run.smithi104.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49856.log.gz 2023-07-14T11:57:55.146 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46785.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46785.log.gz 2023-07-14T11:57:55.147 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44720.log 2023-07-14T11:57:55.148 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44255.log 2023-07-14T11:57:55.148 INFO:teuthology.orchestra.run.smithi104.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51581.log.gz 2023-07-14T11:57:55.149 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34860.log 2023-07-14T11:57:55.150 INFO:teuthology.orchestra.run.smithi104.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44720.log.gz 2023-07-14T11:57:55.150 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44255.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44255.log.gz 2023-07-14T11:57:55.150 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42326.log 2023-07-14T11:57:55.152 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.34860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40883.log 2023-07-14T11:57:55.152 INFO:teuthology.orchestra.run.smithi104.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34860.log.gz 2023-07-14T11:57:55.152 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42326.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42326.log.gz 2023-07-14T11:57:55.153 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50752.log 2023-07-14T11:57:55.154 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43951.log 2023-07-14T11:57:55.154 INFO:teuthology.orchestra.run.smithi104.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40883.log.gz 2023-07-14T11:57:55.155 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50752.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50752.log.gz 2023-07-14T11:57:55.156 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50340.log 2023-07-14T11:57:55.157 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42120.log 2023-07-14T11:57:55.157 INFO:teuthology.orchestra.run.smithi104.stderr: 87.6%/var/log/ceph/ceph-client.admin.50340.log: -- replaced with /var/log/ceph/ceph-client.admin.43951.log.gz 2023-07-14T11:57:55.157 INFO:teuthology.orchestra.run.smithi104.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50340.log.gz 2023-07-14T11:57:55.158 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45210.log 2023-07-14T11:57:55.159 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42120.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42120.log.gz 2023-07-14T11:57:55.160 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50644.log 2023-07-14T11:57:55.161 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45210.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45210.log.gzgzip 2023-07-14T11:57:55.161 INFO:teuthology.orchestra.run.smithi104.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.43627.log 2023-07-14T11:57:55.163 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50644.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50644.log.gz 2023-07-14T11:57:55.163 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50698.log 2023-07-14T11:57:55.164 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47437.log 2023-07-14T11:57:55.164 INFO:teuthology.orchestra.run.smithi104.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43627.log.gz 2023-07-14T11:57:55.165 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50698.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50698.log.gz 2023-07-14T11:57:55.165 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43223.log 2023-07-14T11:57:55.166 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41744.log 2023-07-14T11:57:55.167 INFO:teuthology.orchestra.run.smithi104.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47437.log.gz 2023-07-14T11:57:55.167 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.0.43223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48623.log 2023-07-14T11:57:55.168 INFO:teuthology.orchestra.run.smithi104.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.0.43223.log.gz 2023-07-14T11:57:55.169 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46417.log 2023-07-14T11:57:55.169 INFO:teuthology.orchestra.run.smithi104.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41744.log.gz 2023-07-14T11:57:55.170 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48623.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48623.log.gz 2023-07-14T11:57:55.170 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45485.log 2023-07-14T11:57:55.171 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46417.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46417.log.gz 2023-07-14T11:57:55.171 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45757.log 2023-07-14T11:57:55.172 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45485.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45485.log.gz 2023-07-14T11:57:55.173 INFO:teuthology.orchestra.run.smithi104.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-07-14T11:57:55.173 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35096.log 2023-07-14T11:57:55.174 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48796.log 2023-07-14T11:57:55.174 INFO:teuthology.orchestra.run.smithi104.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45757.log.gz 2023-07-14T11:57:55.175 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.35096.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35096.log.gz 2023-07-14T11:57:55.175 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40280.log 2023-07-14T11:57:55.176 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46581.log 2023-07-14T11:57:55.176 INFO:teuthology.orchestra.run.smithi104.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48796.log.gz 2023-07-14T11:57:55.177 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40280.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40280.log.gz 2023-07-14T11:57:55.177 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-07-14T11:57:55.179 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45074.log 2023-07-14T11:57:55.179 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph.audit.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46581.log.gz 2023-07-14T11:57:55.180 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42581.log 2023-07-14T11:57:55.181 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45074.log: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-07-14T11:57:55.182 INFO:teuthology.orchestra.run.smithi104.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45074.log.gz 2023-07-14T11:57:55.182 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44993.log 2023-07-14T11:57:55.183 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42123.log 2023-07-14T11:57:55.183 INFO:teuthology.orchestra.run.smithi104.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42581.log.gz 2023-07-14T11:57:55.184 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44993.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44993.log.gz 2023-07-14T11:57:55.184 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46000.log 2023-07-14T11:57:55.185 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41331.log 2023-07-14T11:57:55.186 INFO:teuthology.orchestra.run.smithi104.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42123.log.gz 2023-07-14T11:57:55.186 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46000.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46000.log.gz 2023-07-14T11:57:55.187 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50204.log 2023-07-14T11:57:55.188 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40653.log 2023-07-14T11:57:55.188 INFO:teuthology.orchestra.run.smithi104.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41331.log.gz 2023-07-14T11:57:55.189 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51890.log 2023-07-14T11:57:55.189 INFO:teuthology.orchestra.run.smithi104.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50204.log.gz 2023-07-14T11:57:55.190 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40653.log: gzip 88.4%-5 -- replaced with /var/log/ceph/ceph-client.admin.40653.log.gz--verbose 2023-07-14T11:57:55.190 INFO:teuthology.orchestra.run.smithi104.stderr:-- /var/log/ceph/ceph-client.admin.44585.log 2023-07-14T11:57:55.191 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51890.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51890.log.gz 2023-07-14T11:57:55.191 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45512.log 2023-07-14T11:57:55.192 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40489.log 2023-07-14T11:57:55.193 INFO:teuthology.orchestra.run.smithi104.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44585.log.gz 2023-07-14T11:57:55.194 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45512.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45512.log.gz 2023-07-14T11:57:55.194 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48373.log 2023-07-14T11:57:55.195 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46187.log 2023-07-14T11:57:55.195 INFO:teuthology.orchestra.run.smithi104.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40489.log.gz 2023-07-14T11:57:55.196 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48373.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48373.log.gz 2023-07-14T11:57:55.196 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44472.log 2023-07-14T11:57:55.197 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43490.log 2023-07-14T11:57:55.197 INFO:teuthology.orchestra.run.smithi104.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46187.log.gz 2023-07-14T11:57:55.198 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44472.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44472.log.gz 2023-07-14T11:57:55.198 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50149.log 2023-07-14T11:57:55.199 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46307.log 2023-07-14T11:57:55.200 INFO:teuthology.orchestra.run.smithi104.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43490.log.gz 2023-07-14T11:57:55.201 INFO:teuthology.orchestra.run.smithi121.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-07-14T11:57:55.201 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50149.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50149.log.gz 2023-07-14T11:57:55.201 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45865.log 2023-07-14T11:57:55.202 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42117.log 2023-07-14T11:57:55.203 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46307.log: /var/log/ceph/ceph-client.admin.45865.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45865.log.gz 2023-07-14T11:57:55.204 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41900.log 2023-07-14T11:57:55.204 INFO:teuthology.orchestra.run.smithi104.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46307.log.gz 2023-07-14T11:57:55.205 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47715.log 2023-07-14T11:57:55.206 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41900.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42117.log.gz 2023-07-14T11:57:55.206 INFO:teuthology.orchestra.run.smithi104.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41900.log.gz 2023-07-14T11:57:55.207 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51141.log 2023-07-14T11:57:55.208 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41891.log 2023-07-14T11:57:55.208 INFO:teuthology.orchestra.run.smithi104.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47715.log.gz 2023-07-14T11:57:55.209 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51141.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51141.log.gz 2023-07-14T11:57:55.209 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43735.log 2023-07-14T11:57:55.210 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42338.log 2023-07-14T11:57:55.211 INFO:teuthology.orchestra.run.smithi104.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41891.log.gz 2023-07-14T11:57:55.212 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43735.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43735.log.gz 2023-07-14T11:57:55.212 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46280.log 2023-07-14T11:57:55.213 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45264.log 2023-07-14T11:57:55.213 INFO:teuthology.orchestra.run.smithi104.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42338.log.gz 2023-07-14T11:57:55.214 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46280.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46280.log.gz 2023-07-14T11:57:55.214 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42114.log 2023-07-14T11:57:55.216 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45264.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45264.log.gz 2023-07-14T11:57:55.216 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50015.log 2023-07-14T11:57:55.217 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42114.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42114.log.gz 2023-07-14T11:57:55.217 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41907.log 2023-07-14T11:57:55.218 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51500.log 2023-07-14T11:57:55.218 INFO:teuthology.orchestra.run.smithi104.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50015.log.gz 2023-07-14T11:57:55.219 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41907.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41907.log.gz 2023-07-14T11:57:55.220 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48041.log 2023-07-14T11:57:55.221 INFO:teuthology.orchestra.run.smithi121.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-07-14T11:57:55.222 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43843.log 2023-07-14T11:57:55.222 INFO:teuthology.orchestra.run.smithi104.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51500.log.gz 2023-07-14T11:57:55.223 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48041.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48041.log.gz 2023-07-14T11:57:55.223 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43789.log 2023-07-14T11:57:55.224 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43843.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43843.log.gz 2023-07-14T11:57:55.224 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51003.log 2023-07-14T11:57:55.225 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43789.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43789.log.gz 2023-07-14T11:57:55.226 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42331.log 2023-07-14T11:57:55.227 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.0.51003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48985.log 2023-07-14T11:57:55.228 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42331.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42331.log.gz 2023-07-14T11:57:55.229 INFO:teuthology.orchestra.run.smithi104.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.0.51003.log.gz 2023-07-14T11:57:55.229 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41413.log 2023-07-14T11:57:55.230 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42716.log 2023-07-14T11:57:55.231 INFO:teuthology.orchestra.run.smithi104.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48985.log.gz 2023-07-14T11:57:55.231 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41413.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41413.log.gz 2023-07-14T11:57:55.232 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47106.log 2023-07-14T11:57:55.233 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42716.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42716.log.gz 2023-07-14T11:57:55.233 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47133.log 2023-07-14T11:57:55.234 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47106.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47106.log.gz 2023-07-14T11:57:55.235 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44639.log 2023-07-14T11:57:55.236 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43102.log 2023-07-14T11:57:55.237 INFO:teuthology.orchestra.run.smithi104.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47133.log.gz 2023-07-14T11:57:55.237 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44639.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44639.log.gz 2023-07-14T11:57:55.238 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44883.log 2023-07-14T11:57:55.240 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46100.log 2023-07-14T11:57:55.240 INFO:teuthology.orchestra.run.smithi104.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43102.log.gz 2023-07-14T11:57:55.241 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44883.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44883.log.gz 2023-07-14T11:57:55.242 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44059.log 2023-07-14T11:57:55.244 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43600.log 2023-07-14T11:57:55.244 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46100.log: /var/log/ceph/ceph-client.admin.44059.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46100.log.gz 2023-07-14T11:57:55.245 INFO:teuthology.orchestra.run.smithi104.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44059.log.gz 2023-07-14T11:57:55.245 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49220.log 2023-07-14T11:57:55.247 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43600.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43600.log.gz 2023-07-14T11:57:55.248 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43463.log 2023-07-14T11:57:55.249 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49220.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49220.log.gz 2023-07-14T11:57:55.249 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48904.log 2023-07-14T11:57:55.250 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43463.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43463.log.gz 2023-07-14T11:57:55.251 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43403.log 2023-07-14T11:57:55.252 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48904.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48904.log.gz 2023-07-14T11:57:55.253 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44032.log 2023-07-14T11:57:55.254 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43403.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43403.log.gz 2023-07-14T11:57:55.254 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48850.log 2023-07-14T11:57:55.255 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44032.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44032.log.gz 2023-07-14T11:57:55.255 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44418.log 2023-07-14T11:57:55.256 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41139.log 2023-07-14T11:57:55.257 INFO:teuthology.orchestra.run.smithi104.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48850.log.gz 2023-07-14T11:57:55.258 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44418.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44418.log.gz 2023-07-14T11:57:55.259 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50915.log 2023-07-14T11:57:55.259 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41663.log 2023-07-14T11:57:55.261 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50915.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50915.log.gz 2023-07-14T11:57:55.262 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41893.log 2023-07-14T11:57:55.262 INFO:teuthology.orchestra.run.smithi104.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.41139.log.gz 2023-07-14T11:57:55.262 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42635.log 2023-07-14T11:57:55.262 INFO:teuthology.orchestra.run.smithi104.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41663.log.gz 2023-07-14T11:57:55.263 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41893.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41893.log.gz 2023-07-14T11:57:55.263 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40624.log 2023-07-14T11:57:55.264 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42635.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42635.log.gz 2023-07-14T11:57:55.264 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44499.log 2023-07-14T11:57:55.265 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40624.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40624.log.gz 2023-07-14T11:57:55.265 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47933.log 2023-07-14T11:57:55.266 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44499.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44499.log.gz 2023-07-14T11:57:55.266 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51311.log 2023-07-14T11:57:55.267 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51971.log 2023-07-14T11:57:55.268 INFO:teuthology.orchestra.run.smithi104.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47933.log.gz 2023-07-14T11:57:55.269 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51311.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51311.log.gz 2023-07-14T11:57:55.269 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46699.log 2023-07-14T11:57:55.270 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51971.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51971.log.gz 2023-07-14T11:57:55.270 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40746.log 2023-07-14T11:57:55.271 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46699.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46699.log.gz 2023-07-14T11:57:55.272 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51654.log 2023-07-14T11:57:55.273 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40746.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40746.log.gz 2023-07-14T11:57:55.273 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45404.log 2023-07-14T11:57:55.274 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51654.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51654.log.gz 2023-07-14T11:57:55.274 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40856.log 2023-07-14T11:57:55.275 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45404.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45404.log.gz 2023-07-14T11:57:55.276 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41196.log 2023-07-14T11:57:55.277 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40856.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40856.log.gz 2023-07-14T11:57:55.277 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47328.log 2023-07-14T11:57:55.278 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41196.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41196.log.gz 2023-07-14T11:57:55.278 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45622.log 2023-07-14T11:57:55.279 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47328.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-07-14T11:57:55.280 INFO:teuthology.orchestra.run.smithi104.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47328.log.gz 2023-07-14T11:57:55.281 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46866.log 2023-07-14T11:57:55.281 INFO:teuthology.orchestra.run.smithi104.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45622.log.gz 2023-07-14T11:57:55.282 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45730.log 2023-07-14T11:57:55.283 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44748.log 2023-07-14T11:57:55.284 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46866.log: /var/log/ceph/ceph-client.admin.45730.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46866.log.gz 2023-07-14T11:57:55.285 INFO:teuthology.orchestra.run.smithi104.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45730.log.gz 2023-07-14T11:57:55.285 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43897.log 2023-07-14T11:57:55.286 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44748.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44748.log.gz 2023-07-14T11:57:55.287 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45595.log 2023-07-14T11:57:55.288 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43897.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43897.log.gz 2023-07-14T11:57:55.289 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43870.log 2023-07-14T11:57:55.290 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51338.log 2023-07-14T11:57:55.290 INFO:teuthology.orchestra.run.smithi104.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45595.log.gz 2023-07-14T11:57:55.291 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43870.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43870.log.gz 2023-07-14T11:57:55.292 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43762.log 2023-07-14T11:57:55.293 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51338.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51338.log.gz 2023-07-14T11:57:55.294 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42986.log 2023-07-14T11:57:55.294 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44228.log 2023-07-14T11:57:55.295 INFO:teuthology.orchestra.run.smithi104.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43762.log.gz 2023-07-14T11:57:55.295 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42986.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42986.log.gz 2023-07-14T11:57:55.296 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46525.log 2023-07-14T11:57:55.297 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44228.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44228.log.gz 2023-07-14T11:57:55.298 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44201.log 2023-07-14T11:57:55.298 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42797.log 2023-07-14T11:57:55.299 INFO:teuthology.orchestra.run.smithi104.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46525.log.gz 2023-07-14T11:57:55.300 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45431.log 2023-07-14T11:57:55.300 INFO:teuthology.orchestra.run.smithi104.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44201.log.gz 2023-07-14T11:57:55.301 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42797.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42797.log.gz 2023-07-14T11:57:55.301 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52187.log 2023-07-14T11:57:55.302 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49667.log 2023-07-14T11:57:55.303 INFO:teuthology.orchestra.run.smithi104.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45431.log.gz 2023-07-14T11:57:55.303 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.52187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41690.log 2023-07-14T11:57:55.304 INFO:teuthology.orchestra.run.smithi104.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52187.log.gz 2023-07-14T11:57:55.305 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49667.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49667.log.gz 2023-07-14T11:57:55.305 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44336.log 2023-07-14T11:57:55.306 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45323.log 2023-07-14T11:57:55.306 INFO:teuthology.orchestra.run.smithi104.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41690.log.gz 2023-07-14T11:57:55.307 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44336.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44336.log.gz 2023-07-14T11:57:55.308 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50367.log 2023-07-14T11:57:55.309 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45323.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45323.log.gz 2023-07-14T11:57:55.309 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51780.log 2023-07-14T11:57:55.311 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45291.log 2023-07-14T11:57:55.311 INFO:teuthology.orchestra.run.smithi104.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50367.log.gz 2023-07-14T11:57:55.312 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51780.log: gzip -5 --verbose -- 89.3%/var/log/ceph/ceph-client.admin.47301.log 2023-07-14T11:57:55.312 INFO:teuthology.orchestra.run.smithi104.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51780.log.gz 2023-07-14T11:57:55.314 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50725.log 2023-07-14T11:57:55.314 INFO:teuthology.orchestra.run.smithi104.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45291.log.gz 2023-07-14T11:57:55.315 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47301.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47301.log.gz 2023-07-14T11:57:55.316 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41548.log 2023-07-14T11:57:55.317 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50725.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50725.log.gz 2023-07-14T11:57:55.317 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47824.log 2023-07-14T11:57:55.318 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41548.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41548.log.gz 2023-07-14T11:57:55.318 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46610.log 2023-07-14T11:57:55.320 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47824.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47824.log.gz 2023-07-14T11:57:55.320 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41304.log 2023-07-14T11:57:55.321 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46610.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46610.log.gz 2023-07-14T11:57:55.321 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52214.log 2023-07-14T11:57:55.323 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41304.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41304.log.gz 2023-07-14T11:57:55.323 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41905.log 2023-07-14T11:57:55.324 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.52214.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52214.log.gz 2023-07-14T11:57:55.324 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42689.log 2023-07-14T11:57:55.324 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41905.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41905.log.gz 2023-07-14T11:57:55.324 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40543.log 2023-07-14T11:57:55.325 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42689.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42689.log.gz 2023-07-14T11:57:55.325 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48715.log 2023-07-14T11:57:55.326 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40543.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40543.log.gz 2023-07-14T11:57:55.326 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49775.log 2023-07-14T11:57:55.326 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48715.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48715.log.gz 2023-07-14T11:57:55.327 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49407.log 2023-07-14T11:57:55.327 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49775.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49775.log.gz 2023-07-14T11:57:55.328 INFO:teuthology.orchestra.run.smithi104.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.48823.log 2023-07-14T11:57:55.328 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49407.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49407.log.gz 2023-07-14T11:57:55.329 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46471.log 2023-07-14T11:57:55.330 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48823.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48823.log.gz 2023-07-14T11:57:55.330 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47960.log 2023-07-14T11:57:55.331 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42321.log 2023-07-14T11:57:55.331 INFO:teuthology.orchestra.run.smithi104.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46471.log.gz 2023-07-14T11:57:55.332 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47960.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47960.log.gz 2023-07-14T11:57:55.332 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51054.log 2023-07-14T11:57:55.333 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42321.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42321.log.gz 2023-07-14T11:57:55.333 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44910.log 2023-07-14T11:57:55.334 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51054.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51054.log.gz 2023-07-14T11:57:55.335 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49112.log 2023-07-14T11:57:55.335 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44910.log: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-07-14T11:57:55.336 INFO:teuthology.orchestra.run.smithi104.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44910.log.gz 2023-07-14T11:57:55.336 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44086.log 2023-07-14T11:57:55.337 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49112.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49112.log.gz 2023-07-14T11:57:55.337 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40717.log 2023-07-14T11:57:55.338 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42905.log 2023-07-14T11:57:55.338 INFO:teuthology.orchestra.run.smithi104.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44086.log.gz 2023-07-14T11:57:55.339 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44939.log 2023-07-14T11:57:55.339 INFO:teuthology.orchestra.run.smithi104.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40717.log.gz 2023-07-14T11:57:55.340 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42905.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42905.log.gz 2023-07-14T11:57:55.340 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45237.log 2023-07-14T11:57:55.341 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40775.log 2023-07-14T11:57:55.342 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45237.log: /var/log/ceph/ceph-client.admin.44939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51419.log 2023-07-14T11:57:55.343 INFO:teuthology.orchestra.run.smithi104.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45237.log.gz 2023-07-14T11:57:55.343 INFO:teuthology.orchestra.run.smithi104.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44939.log.gz 2023-07-14T11:57:55.344 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44005.log 2023-07-14T11:57:55.344 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51419.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40775.log.gz 2023-07-14T11:57:55.345 INFO:teuthology.orchestra.run.smithi104.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51419.log.gz 2023-07-14T11:57:55.345 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44802.log 2023-07-14T11:57:55.347 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46253.log 2023-07-14T11:57:55.348 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44005.log: /var/log/ceph/ceph-client.admin.44802.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44005.log.gz 2023-07-14T11:57:55.348 INFO:teuthology.orchestra.run.smithi104.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44802.log.gz 2023-07-14T11:57:55.349 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45020.log 2023-07-14T11:57:55.349 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47247.log 2023-07-14T11:57:55.350 INFO:teuthology.orchestra.run.smithi104.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46253.log.gz 2023-07-14T11:57:55.350 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45020.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45020.log.gz 2023-07-14T11:57:55.351 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48742.log 2023-07-14T11:57:55.352 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48319.log 2023-07-14T11:57:55.352 INFO:teuthology.orchestra.run.smithi104.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47247.log.gz 2023-07-14T11:57:55.353 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48742.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48742.log.gz 2023-07-14T11:57:55.353 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44526.log 2023-07-14T11:57:55.354 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48319.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48319.log.gz 2023-07-14T11:57:55.355 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47987.log 2023-07-14T11:57:55.356 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44526.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44526.log.gz 2023-07-14T11:57:55.356 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46920.log 2023-07-14T11:57:55.357 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47987.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47987.log.gz 2023-07-14T11:57:55.357 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52025.log 2023-07-14T11:57:55.358 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48265.log 2023-07-14T11:57:55.359 INFO:teuthology.orchestra.run.smithi104.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46920.log.gz 2023-07-14T11:57:55.359 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.52025.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52025.log.gz 2023-07-14T11:57:55.360 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45568.log 2023-07-14T11:57:55.361 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48265.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48265.log.gz 2023-07-14T11:57:55.362 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41609.log 2023-07-14T11:57:55.363 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42311.log 2023-07-14T11:57:55.364 INFO:teuthology.orchestra.run.smithi104.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45568.log.gz 2023-07-14T11:57:55.364 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41609.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41609.log.gz 2023-07-14T11:57:55.365 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45101.log 2023-07-14T11:57:55.367 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41250.log 2023-07-14T11:57:55.367 INFO:teuthology.orchestra.run.smithi104.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42311.log.gz 2023-07-14T11:57:55.368 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45101.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45101.log.gz 2023-07-14T11:57:55.368 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49829.log 2023-07-14T11:57:55.369 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46552.log 2023-07-14T11:57:55.370 INFO:teuthology.orchestra.run.smithi104.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41250.log.gz 2023-07-14T11:57:55.371 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49829.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49829.log.gz 2023-07-14T11:57:55.371 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43816.log 2023-07-14T11:57:55.372 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46552.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46552.log.gz 2023-07-14T11:57:55.373 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49611.log 2023-07-14T11:57:55.373 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41636.log 2023-07-14T11:57:55.374 INFO:teuthology.orchestra.run.smithi104.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43816.log.gz 2023-07-14T11:57:55.375 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51195.log 2023-07-14T11:57:55.375 INFO:teuthology.orchestra.run.smithi104.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49611.log.gz 2023-07-14T11:57:55.376 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41636.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41636.log.gz 2023-07-14T11:57:55.377 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48346.log 2023-07-14T11:57:55.378 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43517.log 2023-07-14T11:57:55.380 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51195.log: /var/log/ceph/ceph-client.admin.48346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44612.log 87.7% 2023-07-14T11:57:55.380 INFO:teuthology.orchestra.run.smithi104.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48346.log.gz 2023-07-14T11:57:55.380 INFO:teuthology.orchestra.run.smithi104.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51195.log.gz 2023-07-14T11:57:55.381 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45156.log 2023-07-14T11:57:55.381 INFO:teuthology.orchestra.run.smithi104.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43517.log.gz 2023-07-14T11:57:55.382 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44612.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44612.log.gz 2023-07-14T11:57:55.383 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47028.log 2023-07-14T11:57:55.384 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41717.log 2023-07-14T11:57:55.385 INFO:teuthology.orchestra.run.smithi104.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45156.log.gz 2023-07-14T11:57:55.386 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.0.47028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42527.log 2023-07-14T11:57:55.387 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41717.log: 91.8% -- replaced with /var/log/ceph/ceph-client.0.47028.log.gz 2023-07-14T11:57:55.388 INFO:teuthology.orchestra.run.smithi104.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41717.log.gz 2023-07-14T11:57:55.388 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49549.log 2023-07-14T11:57:55.389 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42527.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42527.log.gz 2023-07-14T11:57:55.389 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48661.log 2023-07-14T11:57:55.390 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49549.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49549.log.gz 2023-07-14T11:57:55.390 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43267.log 2023-07-14T11:57:55.391 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48400.log 2023-07-14T11:57:55.392 INFO:teuthology.orchestra.run.smithi104.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48661.log.gz 2023-07-14T11:57:55.392 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43267.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43267.log.gz 2023-07-14T11:57:55.393 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50834.log 2023-07-14T11:57:55.394 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41582.log 2023-07-14T11:57:55.394 INFO:teuthology.orchestra.run.smithi104.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48400.log.gz 2023-07-14T11:57:55.395 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50834.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50834.log.gz 2023-07-14T11:57:55.395 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45892.log 2023-07-14T11:57:55.396 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47545.log 2023-07-14T11:57:55.396 INFO:teuthology.orchestra.run.smithi104.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41582.log.gz 2023-07-14T11:57:55.397 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45892.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45892.log.gz 2023-07-14T11:57:55.398 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41897.log 2023-07-14T11:57:55.398 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47545.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47545.log.gz 2023-07-14T11:57:55.399 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47742.log 2023-07-14T11:57:55.399 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41897.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41897.log.gz 2023-07-14T11:57:55.400 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51087.log 2023-07-14T11:57:55.401 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47742.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47742.log.gz 2023-07-14T11:57:55.401 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41494.log 2023-07-14T11:57:55.402 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51087.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51087.log.gz 2023-07-14T11:57:55.402 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42743.log 2023-07-14T11:57:55.403 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41494.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41494.log.gz 2023-07-14T11:57:55.403 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47491.log 2023-07-14T11:57:55.404 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42743.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42743.log.gz 2023-07-14T11:57:55.404 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47906.log 2023-07-14T11:57:55.405 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47491.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47491.log.gz 2023-07-14T11:57:55.406 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41440.log 2023-07-14T11:57:55.406 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47906.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47906.log.gz 2023-07-14T11:57:55.407 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45946.log 2023-07-14T11:57:55.407 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41440.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41440.log.gz 2023-07-14T11:57:55.408 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44364.log 2023-07-14T11:57:55.409 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45946.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45946.log.gz 2023-07-14T11:57:55.409 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46498.log 2023-07-14T11:57:55.410 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44364.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44364.log.gz 2023-07-14T11:57:55.410 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47382.log 2023-07-14T11:57:55.411 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46498.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46498.log.gz 2023-07-14T11:57:55.411 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49883.log 2023-07-14T11:57:55.412 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47382.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47382.log.gz 2023-07-14T11:57:55.413 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45129.log 2023-07-14T11:57:55.414 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49883.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49883.log.gz 2023-07-14T11:57:55.414 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44309.log 2023-07-14T11:57:55.415 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45129.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45129.log.gz 2023-07-14T11:57:55.415 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46758.log 2023-07-14T11:57:55.416 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44309.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44309.log.gz 2023-07-14T11:57:55.416 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49584.log 2023-07-14T11:57:55.417 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46758.log: gzip -5 --verbose 88.3%-- -- replaced with /var/log/ceph/ceph-client.admin.46758.log.gz/var/log/ceph/ceph-client.admin.42122.log 2023-07-14T11:57:55.417 INFO:teuthology.orchestra.run.smithi104.stderr: 2023-07-14T11:57:55.418 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49584.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49584.log.gz 2023-07-14T11:57:55.418 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48688.log 2023-07-14T11:57:55.419 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49012.log 2023-07-14T11:57:55.420 INFO:teuthology.orchestra.run.smithi104.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42122.log.gz 2023-07-14T11:57:55.420 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48688.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48688.log.gz 2023-07-14T11:57:55.421 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50807.log 2023-07-14T11:57:55.421 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49012.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49012.log.gz 2023-07-14T11:57:55.422 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40570.log 2023-07-14T11:57:55.422 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50807.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50807.log.gz 2023-07-14T11:57:55.423 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45838.log 2023-07-14T11:57:55.424 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40570.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40570.log.gz 2023-07-14T11:57:55.424 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47274.log 2023-07-14T11:57:55.425 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45838.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45838.log.gz 2023-07-14T11:57:55.425 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46127.log 2023-07-14T11:57:55.426 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47274.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47274.log.gz 2023-07-14T11:57:55.426 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-07-14T11:57:55.427 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46127.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46127.log.gz 2023-07-14T11:57:55.427 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51392.log 2023-07-14T11:57:55.428 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41521.log 2023-07-14T11:57:55.429 INFO:teuthology.orchestra.run.smithi104.stderr: 85.9% -- replaced with /var/log/ceph/ceph.log.gz 2023-07-14T11:57:55.429 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51392.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51392.log.gz 2023-07-14T11:57:55.430 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51627.log 2023-07-14T11:57:55.431 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48292.log 2023-07-14T11:57:55.432 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51627.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41521.log.gz 2023-07-14T11:57:55.432 INFO:teuthology.orchestra.run.smithi104.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51627.log.gz 2023-07-14T11:57:55.433 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51751.log 2023-07-14T11:57:55.434 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48292.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48292.log.gz 2023-07-14T11:57:55.434 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49694.log 2023-07-14T11:57:55.435 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51751.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51751.log.gz 2023-07-14T11:57:55.436 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49638.log 2023-07-14T11:57:55.437 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49694.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49694.log.gz 2023-07-14T11:57:55.437 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52295.log 2023-07-14T11:57:55.438 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49638.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49638.log.gz 2023-07-14T11:57:55.438 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47464.log 2023-07-14T11:57:55.439 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.52295.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52295.log.gz 2023-07-14T11:57:55.440 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47160.log 2023-07-14T11:57:55.440 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47464.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47464.log.gz 2023-07-14T11:57:55.441 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-07-14T11:57:55.442 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47160.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47160.log.gz 2023-07-14T11:57:55.442 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40829.log 2023-07-14T11:57:55.443 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51222.log 2023-07-14T11:57:55.444 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40829.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40829.log.gz 2023-07-14T11:57:55.445 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44856.log 2023-07-14T11:57:55.446 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50671.log 2023-07-14T11:57:55.446 INFO:teuthology.orchestra.run.smithi104.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51222.log.gz 2023-07-14T11:57:55.447 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44856.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44856.log.gz 2023-07-14T11:57:55.448 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41386.log 2023-07-14T11:57:55.449 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47220.log 2023-07-14T11:57:55.450 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41386.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50671.log.gz 2023-07-14T11:57:55.451 INFO:teuthology.orchestra.run.smithi104.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41386.log.gz 2023-07-14T11:57:55.451 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49520.log 2023-07-14T11:57:55.452 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47220.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47220.log.gz 2023-07-14T11:57:55.453 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50313.log 2023-07-14T11:57:55.454 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50888.log 2023-07-14T11:57:55.455 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49520.log: /var/log/ceph/ceph-client.admin.50313.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50313.log.gz 2023-07-14T11:57:55.456 INFO:teuthology.orchestra.run.smithi104.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49520.log.gz 2023-07-14T11:57:55.456 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46672.log 2023-07-14T11:57:55.457 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50888.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50888.log.gz 2023-07-14T11:57:55.458 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50177.log 2023-07-14T11:57:55.459 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46672.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46672.log.gz 2023-07-14T11:57:55.459 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44558.log 2023-07-14T11:57:55.460 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52106.log 2023-07-14T11:57:55.460 INFO:teuthology.orchestra.run.smithi104.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50177.log.gz 2023-07-14T11:57:55.461 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44558.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44558.log.gz 2023-07-14T11:57:55.461 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41771.log 2023-07-14T11:57:55.462 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.52106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40461.log 2023-07-14T11:57:55.463 INFO:teuthology.orchestra.run.smithi104.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52106.log.gz 2023-07-14T11:57:55.463 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41771.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41771.log.gz 2023-07-14T11:57:55.464 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46444.log 2023-07-14T11:57:55.465 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40434.log 2023-07-14T11:57:55.465 INFO:teuthology.orchestra.run.smithi104.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40461.log.gz 2023-07-14T11:57:55.466 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46444.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46444.log.gz 2023-07-14T11:57:55.466 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46335.log 2023-07-14T11:57:55.467 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49964.log 2023-07-14T11:57:55.468 INFO:teuthology.orchestra.run.smithi104.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40434.log.gz 2023-07-14T11:57:55.469 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49748.log 88.0% 2023-07-14T11:57:55.469 INFO:teuthology.orchestra.run.smithi104.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46335.log.gz 2023-07-14T11:57:55.470 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49462.log 2023-07-14T11:57:55.470 INFO:teuthology.orchestra.run.smithi104.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49964.log.gz 2023-07-14T11:57:55.471 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49748.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49748.log.gz 2023-07-14T11:57:55.471 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42878.log 2023-07-14T11:57:55.472 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49462.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30051.log 2023-07-14T11:57:55.473 INFO:teuthology.orchestra.run.smithi104.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49462.log.gz 2023-07-14T11:57:55.473 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42878.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42878.log.gz 2023-07-14T11:57:55.474 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40910.log 2023-07-14T11:57:55.475 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph.tmp-client.admin.30051.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30051.log.gz 2023-07-14T11:57:55.475 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46893.log 2023-07-14T11:57:55.476 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40910.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40910.log.gz 2023-07-14T11:57:55.476 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47797.log 2023-07-14T11:57:55.477 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46893.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-07-14T11:57:55.477 INFO:teuthology.orchestra.run.smithi104.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46893.log.gz 2023-07-14T11:57:55.478 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52160.log 2023-07-14T11:57:55.478 INFO:teuthology.orchestra.run.smithi104.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47797.log.gz 2023-07-14T11:57:55.479 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40335.log 2023-07-14T11:57:55.480 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.52160.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52160.log.gz 2023-07-14T11:57:55.481 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51944.log 2023-07-14T11:57:55.482 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45973.log 2023-07-14T11:57:55.483 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40335.log: /var/log/ceph/ceph-client.admin.51944.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40335.log.gz 2023-07-14T11:57:55.483 INFO:teuthology.orchestra.run.smithi104.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51944.log.gz 2023-07-14T11:57:55.484 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52375.log 2023-07-14T11:57:55.485 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34987.log 2023-07-14T11:57:55.485 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.45973.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45973.log.gz 2023-07-14T11:57:55.486 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.52375.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52375.log.gz 2023-07-14T11:57:55.486 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50285.log 2023-07-14T11:57:55.487 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.34987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44966.log 2023-07-14T11:57:55.488 INFO:teuthology.orchestra.run.smithi104.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34987.log.gz 2023-07-14T11:57:55.489 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50285.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50285.log.gz 2023-07-14T11:57:55.489 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43013.log 2023-07-14T11:57:55.490 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44966.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44966.log.gz 2023-07-14T11:57:55.491 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51527.log 2023-07-14T11:57:55.492 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49287.log 2023-07-14T11:57:55.492 INFO:teuthology.orchestra.run.smithi104.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43013.log.gz 2023-07-14T11:57:55.494 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51527.log: gzip -5 --verbose -- 88.1%/var/log/ceph/ceph-client.admin.44140.log 2023-07-14T11:57:55.494 INFO:teuthology.orchestra.run.smithi104.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51527.log.gz 2023-07-14T11:57:55.495 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49287.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49287.log.gz 2023-07-14T11:57:55.495 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48238.log 2023-07-14T11:57:55.496 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42824.log 2023-07-14T11:57:55.497 INFO:teuthology.orchestra.run.smithi104.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44140.log.gz 2023-07-14T11:57:55.498 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48238.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48238.log.gz 2023-07-14T11:57:55.499 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44391.log 2023-07-14T11:57:55.499 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42824.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42824.log.gz 2023-07-14T11:57:55.500 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46645.log 2023-07-14T11:57:55.501 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.44391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48958.log 2023-07-14T11:57:55.501 INFO:teuthology.orchestra.run.smithi104.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44391.log.gz 2023-07-14T11:57:55.503 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46645.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46645.log.gz 2023-07-14T11:57:55.503 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51834.log 2023-07-14T11:57:55.504 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48958.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48958.log.gz 2023-07-14T11:57:55.505 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52241.log 2023-07-14T11:57:55.506 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41852.log 2023-07-14T11:57:55.506 INFO:teuthology.orchestra.run.smithi104.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51834.log.gz 2023-07-14T11:57:55.507 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.52241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47355.log 2023-07-14T11:57:55.508 INFO:teuthology.orchestra.run.smithi104.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52241.log.gz 2023-07-14T11:57:55.509 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.41852.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41852.log.gz 2023-07-14T11:57:55.509 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46812.log 2023-07-14T11:57:55.510 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.47355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43573.log 2023-07-14T11:57:55.511 INFO:teuthology.orchestra.run.smithi104.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47355.log.gz 2023-07-14T11:57:55.512 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.46812.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46812.log.gz 2023-07-14T11:57:55.512 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48931.log 2023-07-14T11:57:55.514 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42932.log 2023-07-14T11:57:55.514 INFO:teuthology.orchestra.run.smithi104.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43573.log.gz 2023-07-14T11:57:55.516 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43430.log 2023-07-14T11:57:55.516 INFO:teuthology.orchestra.run.smithi104.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48931.log.gz 2023-07-14T11:57:55.518 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48428.log 2023-07-14T11:57:55.518 INFO:teuthology.orchestra.run.smithi104.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42932.log.gz 2023-07-14T11:57:55.519 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.43430.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43430.log.gz 2023-07-14T11:57:55.520 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42109.log 2023-07-14T11:57:55.520 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42339.log 2023-07-14T11:57:55.521 INFO:teuthology.orchestra.run.smithi104.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48428.log.gz 2023-07-14T11:57:55.522 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42109.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42109.log.gz 2023-07-14T11:57:55.523 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40597.log 2023-07-14T11:57:55.524 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.42339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52079.log 2023-07-14T11:57:55.524 INFO:teuthology.orchestra.run.smithi104.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42339.log.gz 2023-07-14T11:57:55.525 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40597.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40597.log.gz 2023-07-14T11:57:55.525 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49139.log 2023-07-14T11:57:55.526 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.52079.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-07-14T11:57:55.527 INFO:teuthology.orchestra.run.smithi104.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52079.log.gz 2023-07-14T11:57:55.528 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.49139.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49139.log.gz 2023-07-14T11:57:55.528 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48455.log 2023-07-14T11:57:55.530 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40516.log 2023-07-14T11:57:55.531 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48100.log 2023-07-14T11:57:55.531 INFO:teuthology.orchestra.run.smithi104.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48455.log.gz 2023-07-14T11:57:55.533 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.40516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50480.log 2023-07-14T11:57:55.533 INFO:teuthology.orchestra.run.smithi104.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40516.log.gz 2023-07-14T11:57:55.534 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48100.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48100.log.gz 2023-07-14T11:57:55.534 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48563.log 2023-07-14T11:57:55.536 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.50480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51681.log 2023-07-14T11:57:55.536 INFO:teuthology.orchestra.run.smithi104.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50480.log.gz 2023-07-14T11:57:55.537 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.48563.log: gzip 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48563.log.gz 2023-07-14T11:57:55.538 INFO:teuthology.orchestra.run.smithi104.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.51863.log 2023-07-14T11:57:55.539 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51681.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51681.log.gz 2023-07-14T11:57:55.541 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.51863.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51863.log.gz 2023-07-14T11:57:55.783 INFO:teuthology.orchestra.run.smithi121.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-07-14T11:57:55.783 INFO:teuthology.orchestra.run.smithi121.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-07-14T11:57:55.864 INFO:teuthology.orchestra.run.smithi121.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-07-14T11:57:55.940 INFO:teuthology.orchestra.run.smithi121.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-07-14T11:57:55.942 INFO:teuthology.orchestra.run.smithi121.stderr: 2023-07-14T11:57:55.943 INFO:teuthology.orchestra.run.smithi121.stderr:real 0m0.988s 2023-07-14T11:57:55.943 INFO:teuthology.orchestra.run.smithi121.stderr:user 0m3.460s 2023-07-14T11:57:55.943 INFO:teuthology.orchestra.run.smithi121.stderr:sys 0m0.410s 2023-07-14T11:57:56.085 INFO:teuthology.orchestra.run.smithi104.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-07-14T11:57:56.249 INFO:teuthology.orchestra.run.smithi104.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-07-14T11:57:56.257 INFO:teuthology.orchestra.run.smithi104.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-07-14T11:57:56.282 INFO:teuthology.orchestra.run.smithi104.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-07-14T11:57:56.287 INFO:teuthology.orchestra.run.smithi104.stderr: 2023-07-14T11:57:56.287 INFO:teuthology.orchestra.run.smithi104.stderr:real 0m1.368s 2023-07-14T11:57:56.287 INFO:teuthology.orchestra.run.smithi104.stderr:user 0m3.825s 2023-07-14T11:57:56.288 INFO:teuthology.orchestra.run.smithi104.stderr:sys 0m1.159s 2023-07-14T11:57:56.288 INFO:tasks.ceph:Archiving logs... 2023-07-14T11:57:56.288 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435/remote/smithi104/log 2023-07-14T11:57:56.289 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T11:57:57.071 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435/remote/smithi121/log 2023-07-14T11:57:57.072 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T11:57:57.498 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-14T11:57:57.507 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-14T11:57:57.508 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T11:57:57.516 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T11:57:57.614 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-14T11:57:57.624 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-14T11:57:57.625 DEBUG:teuthology.orchestra.run.smithi104:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T11:57:57.632 DEBUG:teuthology.orchestra.run.smithi121:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T11:57:57.665 INFO:teuthology.orchestra.run.smithi104.stdout: remote refid st t when poll reach delay offset jitter 2023-07-14T11:57:57.666 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================== 2023-07-14T11:57:57.666 INFO:teuthology.orchestra.run.smithi104.stdout:*hv01.front.sepi 44.4.53.2 2 u 2 64 377 0.121 2.836 1.949 2023-07-14T11:57:57.666 INFO:teuthology.orchestra.run.smithi104.stdout:+hv02.front.sepi 63.231.80.2 3 u 64 64 377 0.117 3.581 2.019 2023-07-14T11:57:57.666 INFO:teuthology.orchestra.run.smithi104.stdout:+hv03.front.sepi 74.6.168.72 3 u 60 64 377 0.105 -3.664 2.003 2023-07-14T11:57:57.666 INFO:teuthology.orchestra.run.smithi104.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-07-14T11:57:57.673 INFO:teuthology.orchestra.run.smithi121.stdout: remote refid st t when poll reach delay offset jitter 2023-07-14T11:57:57.673 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================== 2023-07-14T11:57:57.674 INFO:teuthology.orchestra.run.smithi121.stdout:*hv01.front.sepi 44.4.53.2 2 u 60 64 377 0.104 1.360 2.199 2023-07-14T11:57:57.674 INFO:teuthology.orchestra.run.smithi121.stdout:+hv02.front.sepi 63.231.80.2 3 u 66 64 377 0.113 2.427 2.217 2023-07-14T11:57:57.674 INFO:teuthology.orchestra.run.smithi121.stdout:+hv03.front.sepi 74.6.168.72 3 u 61 64 377 0.117 -0.882 1.668 2023-07-14T11:57:57.674 INFO:teuthology.orchestra.run.smithi121.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-07-14T11:57:57.675 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-14T11:57:57.685 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-14T11:57:57.685 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-14T11:57:57.734 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-14T11:57:57.802 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-14T11:57:57.830 INFO:teuthology.task.internal:Duration was 1072.139634 seconds 2023-07-14T11:57:57.830 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-14T11:57:57.864 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-14T11:57:57.864 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T11:57:57.871 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T11:57:58.011 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-14T11:57:58.012 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi104.front.sepia.ceph.com 2023-07-14T11:57:58.012 DEBUG:teuthology.orchestra.run.smithi104:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T11:57:58.047 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi121.front.sepia.ceph.com 2023-07-14T11:57:58.047 DEBUG:teuthology.orchestra.run.smithi121:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T11:57:58.083 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-14T11:57:58.084 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T11:57:58.101 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T11:57:58.190 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-14T11:57:58.190 DEBUG:teuthology.orchestra.run.smithi104:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T11:57:58.197 DEBUG:teuthology.orchestra.run.smithi121:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T11:57:58.302 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-14T11:57:58.314 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-14T11:57:58.314 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T11:57:58.344 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T11:57:58.377 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-14T11:57:58.399 DEBUG:teuthology.orchestra.run.smithi104:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T11:57:58.403 DEBUG:teuthology.orchestra.run.smithi121:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T11:57:58.430 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = core 2023-07-14T11:57:58.461 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = core 2023-07-14T11:57:58.498 DEBUG:teuthology.orchestra.run.smithi104:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T11:57:58.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:57:58.523 DEBUG:teuthology.orchestra.run.smithi121:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T11:57:58.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:57:58.559 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-14T11:57:58.571 INFO:teuthology.task.internal:Transferring archived files... 2023-07-14T11:57:58.572 DEBUG:teuthology.misc:Transferring archived files from smithi104:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435/remote/smithi104 2023-07-14T11:57:58.572 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T11:57:58.615 DEBUG:teuthology.misc:Transferring archived files from smithi121:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435/remote/smithi121 2023-07-14T11:57:58.616 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T11:57:58.658 INFO:teuthology.task.internal:Removing archive directory... 2023-07-14T11:57:58.659 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T11:57:58.668 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T11:57:58.725 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-14T11:57:58.736 INFO:teuthology.task.internal:Not uploading archives. 2023-07-14T11:57:58.737 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-14T11:57:58.773 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-14T11:57:58.773 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T11:57:58.780 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T11:57:58.795 INFO:teuthology.orchestra.run.smithi104.stdout: 1048618 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 11:57 /home/ubuntu/cephtest 2023-07-14T11:57:58.803 INFO:teuthology.orchestra.run.smithi121.stdout: 1048617 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 11:57 /home/ubuntu/cephtest 2023-07-14T11:57:58.809 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-14T11:57:58.823 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-14T11:57:58.924 INFO:teuthology.nuke:Checking targets against current locks 2023-07-14T11:57:58.952 DEBUG:teuthology.nuke:shortname: smithi104 2023-07-14T11:57:58.953 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T11:57:58.978 DEBUG:teuthology.nuke:shortname: smithi121 2023-07-14T11:57:58.979 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T11:57:58.981 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 11:18:43.961934', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0='} 2023-07-14T11:57:59.002 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337435', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 11:18:43.964802', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0='} 2023-07-14T11:57:59.035 INFO:teuthology.orchestra.console:Power off smithi104 2023-07-14T11:57:59.036 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T11:57:59.063 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T11:57:59.083 INFO:teuthology.orchestra.console:Power off smithi121 2023-07-14T11:57:59.084 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T11:57:59.109 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T11:58:03.097 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T11:58:03.135 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T11:58:07.212 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T11:58:07.250 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T11:58:07.463 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2023-07-14T11:58:11.325 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T11:58:11.537 INFO:teuthology.orchestra.console:Power off for smithi104 completed 2023-07-14T11:58:11.643 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} duration: 1072.1396341323853 failure_reason: 'Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)' flavor: default mon.a-kernel-sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 mon.b-kernel-sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=5dc8b464a56c4996ab01c933dc6cde10 status: fail success: false 2023-07-14T11:58:11.644 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T11:58:11.727 INFO:teuthology.run:FAIL